End_Of_File but not really

End_Of_File but not really

[Adam Beneschan]

This is based on some of the things I've been saying on Maciej's thread
about the Get_Line problem.  I ran this test using GNAT.  Note that my
input file is a disk file, to avoid the additional issues that arise
with interactive files.

with Ada.Text_IO;
use Ada.Text_IO;
procedure IOTest is
    Line : String(1..100);
    Last : Integer;
    F    : File_Type;
    EOF  : Boolean;
begin
    Open(F, In_File, "f1");
    loop
        begin
            EOF := End_Of_File (F);
            Get_Line (F, Line, Last);
            if EOF then
                Put_Line
("End_Of_File returned TRUE but Get_Line did not raise an exception");
                exit;
            end if;
        exception
            when End_Error =>
                Put_Line ("End_Error raised");
                exit;
        end;
    end loop;
end IOTest;

If I try this on Linux, on a file whose bytes are abc<LF><LF>, then the
message "End_Of_File returned TRUE but Get_Line did not raise an
exception".  This strikes me as bizarre---if End_Of_File returns True,
then a subsequent read operation should raise an End_Error, but that
isn't what's happening.  One of these must be true:

(1) The RM does not allow this behavior, and GNAT is broken.

(2) This behavior is what the RM requires.  To me, this means the RM is
broken---it just doesn't make sense to me that End_Of_File would return
True if there is more information in the file to get with Get_Line
(even if the information is "the presence of a blank line").  But
perhaps it's my own understanding that is broken; perhaps my
understanding of what End_Of_File is supposed to do is wrong, even
though I think it's what a reasonable person would expect a function
called "End_Of_File" to do.

(3) The behavior is implementation-defined according to the RM (because
the nature of terminators is implementation-defined), and it's possible
to have an implementation that never displays this message and an
implementation that is capable of displaying message both conforming to
the RM.  In this case, though, I'd say (3a) the RM is a little bit
broken, because even though the representation of terminators is
implementation-defined, it would seem that the definitions of
End_Of_File and End_Error ought to conform to each other, so that the
above message could never appear, and (3b) GNAT is broken, because even
though the RM allows it to implement Text_IO in a way that causes the
above message to appear, it shouldn't because it doesn't make sense.

So which is it?  Thoughts?  Comments?

                                      -- Adam

Re: End_Of_File but not really

[Jeffrey R. Carter]

Adam Beneschan wrote:
> 
> If I try this on Linux, on a file whose bytes are abc<LF><LF>, then the
> message "End_Of_File returned TRUE but Get_Line did not raise an
> exception".  This strikes me as bizarre---if End_Of_File returns True,
> then a subsequent read operation should raise an End_Error, but that
> isn't what's happening.  One of these must be true:

This is the behavior I described in an earlier post.

 From a Text_IO point of view, this is a degenerate file, since it 
doesn't end in <EOL><EOP><EOF>. But consider if you wrote this file with 
Text_IO:

File : Ada.Text_IO.File_Type;
...
Ada.Text_IO.Create
    (File => File, Mode => Ada.Text_IO.Out_File, Name => "f1");
Ada.Text_IO.Put_Line (File => File, Item => "abc");
Ada.Text_IO.Put_Line (File => File, Item => "");
Ada.Text_IO.Close (File => File);

Then you would get what Text_IO considers a correct text file, containing

abc<EOL><EOL><EOP><EOF>

(however those are encoded for your OS/compiler). After the first 
Get_Line, the remainder of the file contains <EOL><EOP><EOF>--the 
definition of the condition when End_Of_File returns True. Yet clearly 
there is an empty line still to be read in that file, since it was 
created by writing "abc" followed by writing an empty line. So the 
situation you describe is true even for a file written by Text_IO: 
End_Of_File returns True and Get_Line reads an empty string.

Another way of looking at it: End_Of_File looks ahead and sees 
<EOL><EOP><EOF>, so it returns True. Get_Line reads <EOL>, so it sets 
Last to Item'First - 1 and returns.

I'm not saying this is sensible, just that it's what the RM specifies. 
Text_IO would be better without pages, and without column and line (and 
page) counting.

-- 
Jeff Carter
"Every sperm is sacred."
Monty Python's the Meaning of Life
55

Re: End_Of_File but not really

[Adam Beneschan]

Jeffrey R. Carter wrote:
> Adam Beneschan wrote:
> >
> > If I try this on Linux, on a file whose bytes are abc<LF><LF>, then the
> > message "End_Of_File returned TRUE but Get_Line did not raise an
> > exception".  This strikes me as bizarre---if End_Of_File returns True,
> > then a subsequent read operation should raise an End_Error, but that
> > isn't what's happening.

> So the
> situation you describe is true even for a file written by Text_IO:
> End_Of_File returns True and Get_Line reads an empty string.

Yes, and for what it's worth, I just tested it, using Text_IO to create
and then reopen the file, and I'm still getting that message.

                 -- Adam

Re: End_Of_File but not really

[Randy Brukardt]

"Adam Beneschan" <adam@irvine.com> wrote in message
news:1165514423.562024.70510@j72g2000cwa.googlegroups.com...
...
> (2) This behavior is what the RM requires.  To me, this means the RM is
> broken---it just doesn't make sense to me that End_Of_File would return
> True if there is more information in the file to get with Get_Line
> (even if the information is "the presence of a blank line").  But
> perhaps it's my own understanding that is broken; perhaps my
> understanding of what End_Of_File is supposed to do is wrong, even
> though I think it's what a reasonable person would expect a function
> called "End_Of_File" to do.

This is exactly the state. It's the way these things were defined in Ada 83,
and required by the ACATS since the very begining. I'd be very surprised if
you found any Ada compiler that doesn't have this behavior (didn't you try
it on your compiler to cross-check)?

As I noted before, there is absoletely no chance that this behavior (or any
behavior of Text_IO) would be changed, because there's no possible way for
such a change to the compatible. If the program "knows" about the "lost"
blank line, it might fail badly if the definition was to be changed. So it's
23 years too late to fix Text_IO.

The moral is simply to never use Text_IO.End_of_File, but rather handle
End_Error instead. If you try to use End_of_File, you have the potential of
"losing" the last line of the file, you still can have End_Error raised if
you call Get or Get_Line when there is no <LF> character at the end of the
file, your program will run slower, and your program won't be able to read
interactively from the keyboard. It's not worth it, no matter what you think
about using exceptions for non-errors.

                              Randy.

Re: End_Of_File but not really

[Adam Beneschan]

Randy Brukardt wrote:
> "Adam Beneschan" <adam@irvine.com> wrote in message
> news:1165514423.562024.70510@j72g2000cwa.googlegroups.com...
> ...
> > (2) This behavior is what the RM requires.  To me, this means the RM is
> > broken---it just doesn't make sense to me that End_Of_File would return
> > True if there is more information in the file to get with Get_Line
> > (even if the information is "the presence of a blank line").  But
> > perhaps it's my own understanding that is broken; perhaps my
> > understanding of what End_Of_File is supposed to do is wrong, even
> > though I think it's what a reasonable person would expect a function
> > called "End_Of_File" to do.
>
> This is exactly the state. It's the way these things were defined in Ada 83,
> and required by the ACATS since the very begining. I'd be very surprised if
> you found any Ada compiler that doesn't have this behavior (didn't you try
> it on your compiler to cross-check)?

> As I noted before, there is absoletely no chance that this behavior (or any
> behavior of Text_IO) would be changed, because there's no possible way for
> such a change to the compatible. If the program "knows" about the "lost"
> blank line, it might fail badly if the definition was to be changed. So it's
> 23 years too late to fix Text_IO.

OK, this is the sort of clarification I was looking for.  I was under
the impression that, since the definitions of the terminators were left
up to the implementation, it would be possible for an implementation to
define them in a way so that things would work "correctly" (i.e. in a
way that would make sense to me).  But I guess that's not possible, or
in any case implementations aren't expected to do this.  Thanks.


> The moral is simply to never use Text_IO.End_of_File, but rather handle
> End_Error instead. If you try to use End_of_File, you have the potential of
> "losing" the last line of the file, you still can have End_Error raised if
> you call Get or Get_Line when there is no <LF> character at the end of the
> file, your program will run slower, and your program won't be able to read
> interactively from the keyboard. It's not worth it, no matter what you think
> about using exceptions for non-errors.

It might have made sense to provide a version of Get_Line with an
"End_Of_File" OUT Boolean parameter; this version would behave exactly
like the other Get_Line except that it would set this parameter to True
instead of raising End_Error.  That would have satisfied people who
don't like exceptions, while avoiding the hokey semantics of the
End_Of_File function (and not requiring Text_IO to do any lookahead).
Of course, anyone can easily write their own version of Get_Line that
works like that.  I agree that, based on the discussions in this
thread, the Text_IO.End_Of_File function should just be avoided.

                                 -- Adam

Re: End_Of_File but not really

[Randy Brukardt]

"Adam Beneschan" <adam@irvine.com> wrote in message
news:1165597373.375204.16280@l12g2000cwl.googlegroups.com...
> Randy Brukardt wrote:
> > "Adam Beneschan" <adam@irvine.com> wrote in message
> > news:1165514423.562024.70510@j72g2000cwa.googlegroups.com...
> > ...
> > > (2) This behavior is what the RM requires.  To me, this means the RM
is
> > > broken---it just doesn't make sense to me that End_Of_File would
return
> > > True if there is more information in the file to get with Get_Line
> > > (even if the information is "the presence of a blank line").  But
> > > perhaps it's my own understanding that is broken; perhaps my
> > > understanding of what End_Of_File is supposed to do is wrong, even
> > > though I think it's what a reasonable person would expect a function
> > > called "End_Of_File" to do.
> >
> > This is exactly the state. It's the way these things were defined in Ada
83,
> > and required by the ACATS since the very begining. I'd be very surprised
if
> > you found any Ada compiler that doesn't have this behavior (didn't you
try
> > it on your compiler to cross-check)?
>
> > As I noted before, there is absoletely no chance that this behavior (or
any
> > behavior of Text_IO) would be changed, because there's no possible way
for
> > such a change to the compatible. If the program "knows" about the "lost"
> > blank line, it might fail badly if the definition was to be changed. So
it's
> > 23 years too late to fix Text_IO.
>
> OK, this is the sort of clarification I was looking for.  I was under
> the impression that, since the definitions of the terminators were left
> up to the implementation, it would be possible for an implementation to
> define them in a way so that things would work "correctly" (i.e. in a
> way that would make sense to me).  But I guess that's not possible, or
> in any case implementations aren't expected to do this.  Thanks.

No, it's not possible. That's because of the definition of writing files.

Recall that closing a file adds (logical) line and page terminators before
the file terminator if they are not present. Let's show these terminators as
<EOL><EOP><EOF> (the real representation doesn't matter).

Now consider the program:
    Create (File1, ....);
    Close (File1);

File1 will contain just <EOL><EOP><EOF> after this program.

Now consider the similar program:
   Create (File2, ...);
   New_Line (File2);
   Close (File2);

File 2 will *also* contain just <EOL><EOP><EOF>.

But clearly, when you re-read these two files, you'd expect different
behavior. The first should return End_of_File = True immediately and raise
End_Error if you call Get_Line, and the second should return End_of_File =
False and allow a single call to Get_Line. But these files have the exact
same contents! There is no possible way for them to have different behavior,
even though they're clearly different from a user perspective.

It appears that Ada 83 chose End_of_File = True and a single call to
Get_Line to work in order that the most important feature of each of these
files works as expected. (That is, if the last line of a file that you write
is a blank line, you can read a blank line; and that a Get(char) following
End_of_File = False will always work.) But the result of that is that
neither file will read quite as expected! IMHO, it would have been better to
define End_of_File such that a subsequent Get_Line would always raise
End_Error (it does mean that for Get(char)). But such hindsight is 20-20.

Since these two files are identical to the language, and certainly the ACATS
could check that they're considered identical (it makes similar checks, I
don't know if this exact one is made), no amount of implementor tricks can
make things work sensibly. In our implementation, the two files would indeed
have different contents (the first would be empty and the second would have
an explict <CR><LF> [or just <LF>, depending on the target]}. But that
doesn't help, because we have to consider them logically the same, since the
language does -- indeed, it makes it more complex than simply writing out
all of the markers. (Depending on all of the markers to be present would
make Text_IO useless on files written by something other than Ada, which
would be unusuable even if it is technically correct.)

                                      Randy.

Re: End_Of_File but not really

[Craig Carey]

On 8 Dec 2006 09:02:53 -0800, "Adam Beneschan" wrote:
>Randy Brukardt wrote:
>> "Adam Beneschan" <adam@irvine.com> wrote in message
>> ...
>> > (2) This behavior is what the RM requires.  To me, this means the RM is
>> > broken---it just doesn't make sense to me that End_Of_File would return
>> > True if there is more information in the file to get with Get_Line
>> > (even if the information is "the presence of a blank line").  But
>> > perhaps it's my own understanding that is broken; perhaps my
>> > understanding of what End_Of_File is supposed to do is wrong, even
>> > though I think it's what a reasonable person would expect a function
>> > called "End_Of_File" to do.
>>
>> This is exactly the state. It's the way these things were defined in Ada 83,
>> and required by the ACATS since the very begining. I'd be very surprised if

It is a GNAT bug. My argument:
(1) I confirmed GNAT's behaviour: the Enter key has to be pressed twice
 as noted (somewhat). I used GNAT 3.15p in Windows 2003.
(2) The behaviour seems unacceptable.
(3) Debugging I added showed that 3.15p's End_Of_Line seems to regard
  perceive  that 2 line feed chars at the end of the file are missing.
(4) The reference manual says that End_Of_Line () pattern matches
 (0..1 EOLs (CR?+LF)) + (0..1 EOPs) + (0..1 EOFs).

Maybe EOF is ASCII 26 or the end of the file. In any case, the RM
 allows GNAT to lose 0 to 1 ASCII 10 chars, and GNAT loses two.

I have written a replacement for Ada.Text_IO. Internal dossiers
and above all, some dates and dossiers and surnames will be
released by Mr Warren Tucker, and his black magicians faking
vampirism: All you military officers pop on the pyjamas at 1:30
or 2:30 am. I none ofthe captured e-mails I suspected the only
New Zealand government spy agency ot be moronic, selfish,
truly somewhat illiterate (presumably MI6 could fly out some
staff of the Ombudsmen's Office with MI6 paying), but apparently
the 30 years of steel cage cruelties to a pack of ANZUS militaries
is something that the NZSIS.govt.nz was obtaining a very large
amount of warnings aboutl. So far the NZ government does not
send men out to shoot dogs. In fact there is no type of
terrorism that can get a man out to a front-door. The NZ secret
intelligence service did seem to send a man out to politely
lie at me on 13 October 2006. He knocked about 4 times, and
quit. Some USA agency counterattacked after being lied at, I
guess the CIA. However Australia (training for years), Korea
(caused a mildy adverse effect: Warren sending in 100-300 lying
phone calls, all in Chinese), and if stressed, then
England (maybe they could hire the butler: Warren Tucker had
a party for 1 Canadian and 2 Australian sy chiefs, reported in
the Tue 19 Herald of December 2006. They got a letter from my
DNS hoster, Mr Glenn Reid, in October precisely predicting that
the militaries of most nations are blocked by copyright, ie.
using my software improperly. As best as I can tell, every
New Zealand ISP shows a moral weakness that is astonishing.

Perhaps the best was the Quicksilver.net.nz, the www hoster
the men at the SIS felt I should stay with forever, so as
a side effect of gathering handwritten letters (infrquently
I reply to US+UK overbilling protcols) predicting a ton of
misfortunes for merely ANZUS militaries (and not overlooking
France's, not that I wish to emphasize principles), I got let
off without a debt demand. In that particular instance, I
noted in passing that I would make a complaint to the Auditor
General about the black magic agency: every conduct that it
actually does seems never involve leaving the office.

All anti-terrorism surveillance in NZ instantly stops if a
ICONZ.co.nz or a front for a foreign spy agency receives a
Privacy Act request. Australia is returning presumably giving
some consideration to make inquiries to the Korean spies on
Tuesday, via the Australia of restricted interests.

In New Zealand there is a word for Google disasters spreading
across Europe and Russia and Spain etc.: to quote Hon Helen
Clark: just a "no brainer".An immediate purpose is present
the ANZUS militaries as a pack of duds, to quote my own words
in a handwritten letter to ICONZ.

Also I have a replacement for Ada, winning the war against
indentation. None thought it important.

A quick look at GNAT source shows it much harmed by indenting
after "declare ... begin" lines. 

I am notreally planning on making it available to the public
except by the posting of floppy disks. The unofficial government
therapy of ruthless 56 k blockades, endless deleting of e-mails
with Germany, France, and India, apparently with absolutely
no evidence implicating the CIA, seems a little minor compared
with the doctrinal of shifting local navy chiefs into their
pyjamas at 1:30. Maybe an OIA request to the minister could
find out: a way test if documents can still be couriered out
to parliament buidings in WNG...

It is a trite matter for me and ANZUS to interrogate and
dispute for 3-18 years. The US and UK spies knew that or
else Australia might be able to produce a better replacement
for the nzsis if approved by the Privacy Commissioner and
what lingers in the way of a PM.


>> you found any Ada compiler that doesn't have this behavior (didn't you try
>> it on your compiler to cross-check)?
>

I have no e-mail address.

rushed (from a cafe, using a US news server)
Craig Carey research@ijs.co.nz (dead address and no files)
PO Box 19463 Avondale Auckland.

Re: End_Of_File but not really

[Craig Carey <research@ijs.co.nz>]

Craig Carey wrote:
> On 8 Dec 2006 09:02:53 -0800, "Adam Beneschan" wrote:
> >Randy Brukardt wrote:
> >> "Adam Beneschan" <adam@irvine.com> wrote in message
...
> I have no e-mail address.

I have a Russian address now (almost instant signup):
tangenttangle@pochta.ru
I posted the last into forteinc.com's service. It was delayed
and so the NZ govt had pre-arranged that what I post be
sometimes blocked. A problem I have is that I investigate the
banking sector. The SIS did not want to harm my phone after
the last message. Regarding vampirism, it is normal for NZ
blacklisters to run a surveillance primarily for a purpose of
discovering new ways of sabotaging Privacy Commissioner
investigations (or 'derailing').

The current director took his post after I stopped using the
Internet from my house (last use on 8 August 2006):
constitutional conventions prohibit a finding here.

http://www.nzsis.govt.nz/who/index.html

The Service's Chief Executive, the Director of Security, is
 appointed by the Governor-General on a limited term
 contract. The current Director is Warren Tucker, formerly
 head of the Government Security Communications Bureau.
 He took up his position on 1 November 2006.

Craig Carey
PO BOX 19463 Avondale
!

Re: End_Of_File but not really

[Steve]

"Randy Brukardt" <randy@rrsoftware.com> wrote in message 
news:oJednfF3KqVVLOXYnZ2dnUVZ_q-dnZ2d@megapath.net...
[snip]
>
> As I noted before, there is absoletely no chance that this behavior (or 
> any
> behavior of Text_IO) would be changed, because there's no possible way for
> such a change to the compatible. If the program "knows" about the "lost"
> blank line, it might fail badly if the definition was to be changed. So 
> it's
> 23 years too late to fix Text_IO.
>

Actually... couldn't the "Form" option that is passed to Open include an 
option that changes the end of file behavior?  That wouldn't break old code, 
but would make new code easier.

Steve
(The Duck)


> The moral is simply to never use Text_IO.End_of_File, but rather handle
> End_Error instead. If you try to use End_of_File, you have the potential 
> of
> "losing" the last line of the file, you still can have End_Error raised if
> you call Get or Get_Line when there is no <LF> character at the end of the
> file, your program will run slower, and your program won't be able to read
> interactively from the keyboard. It's not worth it, no matter what you 
> think
> about using exceptions for non-errors.
>
>                              Randy.
>
> 

Re: End_Of_File but not really

[Randy Brukardt]

"Steve" <nospam_steved94@comcast.net> wrote in message
news:4-adnQeuHcj3DubYnZ2dnUVZ_oOonZ2d@comcast.com...
>
> "Randy Brukardt" <randy@rrsoftware.com> wrote in message
> news:oJednfF3KqVVLOXYnZ2dnUVZ_q-dnZ2d@megapath.net...
> [snip]
> >
> > As I noted before, there is absoletely no chance that this behavior (or
> > any
> > behavior of Text_IO) would be changed, because there's no possible way
for
> > such a change to the compatible. If the program "knows" about the "lost"
> > blank line, it might fail badly if the definition was to be changed. So
it's
> > 23 years too late to fix Text_IO.
>
> Actually... couldn't the "Form" option that is passed to Open include an
> option that changes the end of file behavior?  That wouldn't break old
code,
> but would make new code easier.

I suppose, but it wouldn't help much, because there is no "Form" for
Standard_Input (it's already open). So the behavior of that can't be
changed -- but that's where the worst problem is. So it's not clear that a
new "Form" would be very useful (remember that the default behavior would
still have to be the "bad" behavior).

It's also clear that the bad behavior is inevitable with the current model
of terminators. So we'd need a new underlying model, which sounds like a
mess. Besides, getting programmers comfortable with using exceptions is a
good thing: I/O can fail in many ways other than reaching the end, it's
hardly sensible to write any I/O without some handlers. So I'd be more
inclined to make the End_of_File function Obsolescent - it's much like the
Constrained attribute and specific Suppress (which are already Obsolescent):
it seems like they should work, but they don't.

Net result: it's probably not worth the headache.

                        Randy.