GNAT's Text_IO & empty files

GNAT's Text_IO & empty files

[Gautier de Montmollin]

When you create a text file and close it without writing
anything in it, you get an empty line. This is due to an
explicit statement in procedure Terminate_Line from Ada.Text_IO.

This causes an annoying compatibility problem a in DOS or Windows
context, since you get a 1-line, 2-bytes text instead of a 0-line,
0-byte one.

My question is: 
Is it a (a) text file formatting question or (b) a file terminator
question  ? The (b) point of view would be that Text_IO tries
to have a line feed at the end of each text file, although
the only explicit (and facultative) file terminator seems to be the
EOF character. Mine is (a) and I feel it's both harmless and 
profitable to comment out the lines following this remark in procedure
Terminate_Line:

    --  For files other than standard error and standard output, we
    --  make sure that an empty file has a single line feed, so that
    --  it is properly formatted. We avoid this for the standard files
    --  because it is too much of a nuisance to have these odd line
    --  feeds when nothing has been written to the file.

         elsif (File /= Standard_Err and then File /= Standard_Out)
           and then (File.Line = 1 and then File.Page = 1)
         then
            New_Line (File);
         end if;

Comments ?... Reactions ?...

-- 
Gautier

Re: GNAT's Text_IO & empty files

[Pascal MALAISE]

Gautier de Montmollin wrote:
> 
> When you create a text file and close it without writing
> anything in it, you get an empty line. This is due to an
> explicit statement in procedure Terminate_Line from Ada.Text_IO.
> 
> This causes an annoying compatibility problem a in DOS or Windows
> context, since you get a 1-line, 2-bytes text instead of a 0-line,
> 0-byte one.

What I see is a 1-line 0-bytes file (with gnat 3.10p on linux)
> ls -al toto
-rw-r--r--   1 malaise  users           1 Jul  6 20:40 toto
> od -x toto
0000000 000a
0000001

On which platform did you make your test on? Dos?
Which bytes do you get? CR LF? or LF ^Z?

Some Unix tools don't like a file ending in the middle of an "active"
line, (it is better to add an empty line at the end of text files).
This is what Gnat does. If you put en "T" in the file, you'll get
54 0a, and, if you put_line "T", you get 54 0a as well :-)
There is no concept of file terminator (EOF) on Unix.

From the original empty file (0a) on unix, unix2dos generates
0d 0a 1a (a CR LF sequence, which is the dos format of a new line, plus
the famous ^Z, which is a Dos convention for the end of file). I think
that it is what ftp in ASCII more would generate from a Unix machine to
a Dos one (LF becomes CR LF and ^Z is appended).
What would be gnat reaction reading such a file??

Creating such empty file on Dos (with an old Meridain, I don't have gnat
on dos)
generates an empty file, and with a put("T") generates a 1-byte file
with 54.


My conclusion is that it is a file formatting question (including file
terminator).
- Unix has a "kind" of standard and Gnat on Linux complies to it.
- Dos has another standard (CR/LF and CtrlZ) but many tools I tried on
Dos don't comply to it (even an ada compiler). Perhaps this convention
is becomming obsolete.
- If you remove this new_line, check that text_io is still able to read
such a file and that it can read a file with some text but no line
terminator as well.
- From the LRM "...the package Text_IO, which provides facilities for
input and output in human-readable form". Readable with what? :-) Some
editing tools of the same platform? Then there is a need for a
"standard" for ascii text on the platform.

-- 
Pascal MALAISE		| E-mail:
22 Avenue de CHOISY	|  (priv) malaise@magic.fr
75013 PARIS		|  (prof) malaise@fr.airsysatm.thomson-csf.com
FRANCE

Re: GNAT's Text_IO & empty files

[Gautier]

> > When you create a text file and close it without writing
> > anything in it, you get an empty line. This is due to an
> > explicit statement in procedure Terminate_Line from Ada.Text_IO.
> >
> > This causes an annoying compatibility problem a in DOS or Windows
> > context, since you get a 1-line, 2-bytes text instead of a 0-line,
> > 0-byte one.
> 
> What I see is a 1-line 0-bytes file (with gnat 3.10p on linux)
> > ls -al toto
> -rw-r--r--   1 malaise  users           1 Jul  6 20:40 toto
> > od -x toto
> 0000000 000a
> 0000001
> 
> On which platform did you make your test on? Dos?

DOS or Windows

> Which bytes do you get? CR LF? or LF ^Z?

CR LF; should be nothing

(...)

> Creating such empty file on Dos (with an old Meridain, I don't have gnat
> on dos)
> generates an empty file, and with a put("T") generates a 1-byte file
> with 54.

That's the correct behaviour. Same with Alsys Ada, Janus Ada, Turbo Pascal.

> My conclusion is that it is a file formatting question (including file
> terminator).

It's mine too.

> - Unix has a "kind" of standard and Gnat on Linux complies to it.

GNAT on DOS and (one says) on NT too. That's the problem.

> - Dos has another standard (CR/LF and CtrlZ) but many tools I tried on
> Dos don't comply to it (even an ada compiler). Perhaps this convention
> is becomming obsolete.

CtrlZ is obsolete and is not issued by GNAT, so no problem.
On DOS & Windows, CR/LF is facultative (but welcome) at the end of the
last line; for an empty file there must be no CR/LF at all since the
presence of it means that there is 1 (empty) line. Any word processor
will show that.

> - If you remove this new_line, check that text_io is still able to read
> such a file and that it can read a file with some text but no line
> terminator as well.

It works.

> - From the LRM "...the package Text_IO, which provides facilities for
> input and output in human-readable form". Readable with what? :-) Some
> editing tools of the same platform? Then there is a need for a
> "standard" for ascii text on the platform.

Applause! 

-- 
Gautier

Re: GNAT's Text_IO & empty files

[Kevin Radke]

How should an ada compiler act when it finds an "ill-formed"
text file?  As an example, on the Win32 platform, it is VERY
easy to create a text file (with a brain-dead editor like notepad)
that does not terminate the final line of the file with
CR/LF before the EOF is reached.

What should a compiler do when an user tries to do a
text_io.get_line on this final line of the file?  One
compiler that we use raises "end_error".  This has
forced us to change our code to read in a file character
by character... :(  Not very efficient...

Thanks!
Kevin

Re: GNAT's Text_IO & empty files

[Robert Dewar]

G D Montmollin asks

<<My question is:
Is it a (a) text file formatting question or (b) a file terminator
question  ? The (b) point of view would be that Text_IO tries
to have a line feed at the end of each text file, although
the only explicit (and facultative) file terminator seems to be the
EOF character. Mine is (a) and I feel it's both harmless and
profitable to comment out the lines following this remark in procedure
Terminate_Line:

    --  For files other than standard error and standard output, we
    --  make sure that an empty file has a single line feed, so that
    --  it is properly formatted. We avoid this for the standard files
    --  because it is too much of a nuisance to have these odd line
    --  feeds when nothing has been written to the file.

         elsif (File /= Standard_Err and then File /= Standard_Out)
           and then (File.Line = 1 and then File.Page = 1)
         then
            New_Line (File);
         end if;

Comments ?... Reactions ?...
>>

Most certainly if you comment these lines out, your resulting runtime will
not be compliant with annex A of the reference manual. These lines were not
put in for amusement!

Re: GNAT's Text_IO & empty files

[Markus Kuhn]

Robert Dewar wrote:
> Most certainly if you comment these lines out, your resulting runtime will
> not be compliant with annex A of the reference manual. These lines were not
> put in for amusement!

In addition, it might be worth noting that it is common practive
under Unix that all text files end with a LF character. Editors
such as vi do not allow you to create a file that does not end with
LF. In this sense, the Ada convention goes nicely with the Unix
convention.

[The Unix line terminator convention is the only useful one anyway.
The MSDOS text file convention with its silly CR and Ctrl-Z
characters should be sent to the bit graveyard as soon as possible.
What for do we need a line feed character that does not include
the carriage return functionality?]

Markus

-- 
Markus G. Kuhn, Security Group, Computer Lab, Cambridge University, UK
email: mkuhn at acm.org,  home page: <http://www.cl.cam.ac.uk/~mgk25/>

Re: GNAT's Text_IO & empty files

[Gautier]

> In addition, it might be worth noting that it is common practive
> under Unix that all text files end with a LF character. Editors
> such as vi do not allow you to create a file that does not end with
> LF. In this sense, the Ada convention goes nicely with the Unix
> convention.

Very good - as long as you work in the Unix world...

-- 
Gautier

Re: GNAT's Text_IO & empty files

[Samuel Tardieu]

>>>>> "Markus" == Markus Kuhn <Markus.Kuhn@cl.cam.ac.uk> writes:

Markus> In addition, it might be worth noting that it is common
Markus> practive under Unix that all text files end with a LF
Markus> character. Editors such as vi do not allow you to create a
Markus> file that does not end with LF.

Editors such as Emacs define this as a user-option, defaulting to
"don't add any extra LF", so I'm not sure we can call this "common practice".

  Sam
-- 
Samuel Tardieu -- sam@ada.eu.org

Re: GNAT's Text_IO & empty files

[Aaro Koskinen]

Samuel Tardieu <sam@ada.eu.org> writes:
> >>>>> "Markus" == Markus Kuhn <Markus.Kuhn@cl.cam.ac.uk> writes:
>
> Markus> In addition, it might be worth noting that it is common
> Markus> practive under Unix that all text files end with a LF
> Markus> character. Editors such as vi do not allow you to create a
> Markus> file that does not end with LF.
> 
> Editors such as Emacs define this as a user-option, defaulting to
> "don't add any extra LF", so I'm not sure we can call this "common practice".

Check the POSIX.2 definition of a text file. It says that lines
terminate with LF, and a text file contains at least one line.
-- 
Aaro Koskinen, aaro@iki.fi, http://www.iki.fi/aaro

Re: GNAT's Text_IO & empty files

[Gautier.DeMontmollin]

aaro@iki.fi (Aaro Koskinen) writes:

> Check the POSIX.2 definition of a text file. It says that lines
> terminate with LF, and a text file contains at least one line.

More and more interesting...

Is it so explicit in the Ada 95 definition of a text file ?

If no, there is a debate of whether we need compatibity between
different GNAT ports _or_ between a GNAT port and the programs
surrounding it.

The two notions are, in this case, partly incompatible...

To satisfy the 2nd, I suggest something like:

type t_text_file_style is (unix,dos);
text_file_style: t_text_file_style: constant boolean:= unix; 

The output will conform to text_file_style - not more than

         elsif text_file_style = unix
           and then (File /= Standard_Err and then File /= Standard_Out)
           and then (File.Line = 1 and then File.Page = 1)
         then
            New_Line (File);
         end if;

in Terminate_Line.
                                 
The input will be as tolerant as possible to satisfy the 1st
compatibility notion.

N.B.: I make abstraction of the fact that Unix'CR = DOS'CR/LF; this
is translated transparently now via networks or utilities or by lower level IO.

-- 
Gautier

Re: GNAT's Text_IO & empty files

[Samuel Tardieu]

Markus> In addition, it might be worth noting that it is common
Markus> practive under Unix that all text files end with a LF
Markus> character. Editors such as vi do not allow you to create a
Markus> file that does not end with LF.

Sam> Editors such as Emacs define this as a user-option, defaulting to
Sam> "don't add any extra LF", so I'm not sure we can call this
Sam> "common practice".

Aaro> Check the POSIX.2 definition of a text file. It says that lines
Aaro> terminate with LF, and a text file contains at least one line.

We were not talking about standards here, but about common practice :)

  Sam
-- 
Samuel Tardieu -- sam@ada.eu.org

Re: GNAT's Text_IO & empty files

[dennison]

In article <35A16917.18DE1396@cl.cam.ac.uk>,
  Markus Kuhn <Markus.Kuhn@cl.cam.ac.uk> wrote:

> What for do we need a line feed character that does not include
> the carriage return functionality?]

No, Don't get rid of separate CR and LF! The line feed w/o CR saves me tons of
time when I print out source code on my teletype machine.  :-)

Damn! My ENIAC just blew another tube. Gotta go...

T.E.D.

-----== Posted via Deja News, The Leader in Internet Discussion ==-----
http://www.dejanews.com/rg_mkgrp.xp   Create Your Own Free Member Forum

Re: GNAT's Text_IO & empty files

[Markus Kuhn]

Robert Dewar wrote:
>     --  For files other than standard error and standard output, we
>     --  make sure that an empty file has a single line feed, so that
>     --  it is properly formatted. We avoid this for the standard files
>     --  because it is too much of a nuisance to have these odd line
>     --  feeds when nothing has been written to the file.
> 
>          elsif (File /= Standard_Err and then File /= Standard_Out)
>            and then (File.Line = 1 and then File.Page = 1)
>          then
>             New_Line (File);
>          end if;
> 
> 
> Most certainly if you comment these lines out, your resulting runtime will
> not be compliant with annex A of the reference manual. These lines were not
> put in for amusement!

Having just read RM A.10 �7, I think the Ada standard is badly
formulated here. It is not clear how a file or a page with zero
lines or a file with zero pages has to be encoded. The standard writes
that a file is a sequence of pages and a page is a sequence of lines,
but it does not handle the obvious border case of zero-length
sequences adequatly. It would have been better to write just that
every line terminates with a line terminator, and every page
terminates with a page terminator, and then everything would have
been clear and consistent. With a bit good will, the Ada RM language
can even be interpreted that way.

Back to my Unix convention remark: Although vi cannot produce a
file in which the last line does not end with LF, it can very
well produce a file that contains zero lines and is therefore
zero bytes long.

Markus

-- 
Markus G. Kuhn, Security Group, Computer Lab, Cambridge University, UK
email: mkuhn at acm.org,  home page: <http://www.cl.cam.ac.uk/~mgk25/>

Re: GNAT's Text_IO & empty files

[Robert Dewar]

Markus says

<<Having just read RM A.10 M-'7, I think the Ada standard is badly
  formulated here. It is not clear how a file or a page with zero
  lines or a file with zero pages has to be encoded. The standard writes
  that a file is a sequence of pages and a page is a sequence of lines,
  but it does not handle the obvious border case of zero-length
  sequences adequatly. It would have been better to write just that
  every line terminates with a line terminator, and every page
  terminates with a page terminator, and then everything would have
  been clear and consistent. With a bit good will, the Ada RM language
  can even be interpreted that way.>>


I disagree. The RM deliberately stays out of the business of describing
how a file is encoded, and that is as it should be, since the appropriate
encoding is most certainly system dependent. It just describes a set of
high level features, and you can implement them anyway you want. People
often assume for instance that the codes in a Text_IO file will match
those of Standard.Character, but nowhere in the RM is there such a
requirement, and a compiler that always used Unicode for Text_IO files
is perfectly well conforming!

Re: GNAT's Text_IO & empty files

[Michael F Brenner]

   > ... just stating that every line ends with an end-of-line ...

Just stating that every (line/page) end with an end-of-(line/page) 
does not quite answer the question of whether a file with zero lines
and zero pages is permitted. 

The manual should explicitly state that the user may create lines with no
characters other than the end-of-line sequence, pages with no lines,
and files with no pages, to make it perfectly clear what is required in
order to faithfully copy files.

Re: GNAT's Text_IO & empty files

[Gautier]

<<
 Most certainly if you comment these lines out, your resulting runtime will
 not be compliant with annex A of the reference manual. >>

Maybe you mean that the line feed is a line and file terminator .  It's true
on Unix, it false on DOS or Windows, where it is a line separator.
It's like the ';' which follows intructions in Ada but separate them
in Pascal.

<< These lines were not put in for amusement! >>

Oh, you surprise me! Are you serious?

-- 
Gautier

--------
Homepage: http://www.unine.ch/math/Personnel/Assistants/Gautier/Montmollin.html
Software: http://www.unine.ch/math/Personnel/Assistants/Gautier/Gaut_FTP.htm

Re: GNAT's Text_IO & empty files

[Robert Dewar]

Gautier asks

<<Oh, you surprise me! Are you serious?
>>

Certainly, try running the COMPLETE set of Text_IO ACVC tests to see
what I mean!

Re: GNAT's Text_IO & empty files

[Robert I. Eachus]

In article <35A200D9.E5AAC021@maths.unine.ch> Gautier <gautier.demontmollin@maths.unine.ch> writes:

 > Maybe you mean that the line feed is a line and file terminator .  It's true
 > on Unix, it false on DOS or Windows, where it is a line separator.
 > It's like the ';' which follows intructions in Ada but separate them
 > in Pascal.

 > << These lines were not put in for amusement! >>

 > Oh, you surprise me! Are you serious?

   It is possible to have a mapping in Text_IO such that an empty file
contains no bytes.  However, to do that correctly requires relying on
state in the (Ada) file descriptor, since the Ada notion of a text
file ends with an line terminator, followed by a page terminator,
followed by the end of file.  Try it.  Open an empty file for reading,
and call Page.  It will return 1.  Now call End_of_File.  It will
return true.  Now call Skip_Line (or Skip_Page). Page is now 2, but
End_of_File is still true.  Call Skip_Line again and get End_Error.
--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...