Atomic file creation

Atomic file creation

[vlc]

Hi *,

I am looking for a way to create a file without overwriting it. A
simple approach would be:

1  if not Ada.Directories.Exists ("file") then
2     Ada.Text_IO.Create ("file");
3  end if;

But this is not completely safe as it is not atomic. A higher-priority
task e.g. could intercept my task between lines 1 and 2 and create the
same file. In this case the file would be overwritten.

Is there something like C's O_EXCL flag for "open" in Ada?

Thanks a lot in advance.

Re: Atomic file creation

[vlc]

I also tried something like this:

 1  Ada.Text_IO.Create (Handle, Out_File);
 2  declare
 3     Temp_File : String := Ada.Text_IO.Name (Handle);
 4  begin
 5     Ada.Text_IO.Close (Handle);
 6     Ada.Text_IO.Rename (Temp_File, "file");
 7     Ada.Text_IO.Put_Line ("Success");
 8  exception
 9     when Ada.IO_Exceptions.Use_Error =>
10        Ada.Text_IO.Put_Line ("Failure");
11     when others =>
12        raise;
13  end;

But Ada.Text_IO.Rename always raises Ada.IO_Exceptions.Use_Error, even
if the file does not exist. But if I replace line 1 with

 1  Ada.Text_IO.Create (Handle, Out_File, "my_temp_file");

it works. It seems that Ada.Text_IO.Rename cannot rename temporary
files for whatever reason.

Using Ada.Directories.Copy_File instead of Ada.Text_IO.Rename does not
help as Ada.Directories.Copy_File overwrites files without raising an
exception. Bad luck ...

Re: Atomic file creation

[Georg Bauhaus]

vlc schrieb:

> But this is not completely safe as it is not atomic. A higher-priority
> task e.g. could intercept my task between lines 1 and 2 and create the
> same file. In this case the file would be overwritten.

A multitasking program can use a protected object
for this.  Have all tasks perform I/O if and only if
it is their turn as decided by the queuing control of
the protected object.

> Is there something like C's O_EXCL flag for "open" in Ada?

Strictly speaking, O_EXCL is not C, but it is provided by Unix's
open(2) call. fopen(3), which is C, does not offer exclusive access
IIRC.

The Form string parameter of your Ada implementation might
provide sharing control.  The docs will then explain how a
Form value affects files opened by many tasks; possibly even
explain whether the form strings have an effect on using the
external file in other programs.

Re: Atomic file creation

[vlc]

On Jan 4, 3:25 pm, Georg Bauhaus <rm.dash-bauh...@futureapps.de>
wrote:
> A multitasking program can use a protected object
> for this.  Have all tasks perform I/O if and only if
> it is their turn as decided by the queuing control of
> the protected object.

The problem is that this file can also be created by other processes,
neither written by me nor in Ada. I doubt that I can convince an
application that I only have as a binary to respect something like
Ada's protected objects ...

> Strictly speaking, O_EXCL is not C, but it is provided by Unix's
> open(2) call. fopen(3), which is C, does not offer exclusive access
> IIRC.

That's right. I slowly mix up C with Unix.

> The Form string parameter of your Ada implementation might
> provide sharing control.  The docs will then explain how a
> Form value affects files opened by many tasks; possibly even
> explain whether the form strings have an effect on using the
> external file in other programs.

Gnat only supports the SHARED and WCEM form strings, and if I
understand the documentation correctly, they have nothing to do with
exclusive open.

Thanks a lot!

Re: Atomic file creation

[Jacob Sparre Andersen]

vlc <just.another.spam.account@googlemail.com> writes:

> I am looking for a way to create a file without overwriting it. A
> simple approach would be:
>
> 1  if not Ada.Directories.Exists ("file") then
> 2     Ada.Text_IO.Create ("file");
> 3  end if;
>
> But this is not completely safe as it is not atomic.

POSIX.IO.Open_Or_Create (Name        => [...],
                         Mode        => [...],
                         Permissions => [...],
                         Options     => POSIX.IO.Exclusive);

Greetings,

Jacob
-- 
Insanity: doing the same thing over and over again and expecting
different results. (Albert Einstein)

Re: Atomic file creation

[vlc]

On Jan 4, 3:49 pm, Jacob Sparre Andersen <spa...@nbi.dk> wrote:
> POSIX.IO.Open_Or_Create (Name        => [...],
>                          Mode        => [...],
>                          Permissions => [...],
>                          Options     => POSIX.IO.Exclusive);
>
> Greetings,
>
> Jacob

Looks good, but for some reason this package is not installed on my
box. I will look for it on the Internet.

Thanks a lot!

Re: Atomic file creation

[Jean-Pierre Rosen]

vlc a �crit :
> [...] It seems that Ada.Text_IO.Rename cannot rename temporary
> files for whatever reason.
> 
Maybe because temporary files have no names ;-)

Kidding appart, in the old time when people still knew how to write
operating systems, the notion of temporary file was supported by the OS,
and those files had really no name (as opposed to a junk name in /var/tmp).
-- 
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: Atomic file creation

[vlc]

On Jan 4, 4:52 pm, Jean-Pierre Rosen <ro...@adalog.fr> wrote:
> Maybe because temporary files have no names ;-)
>
> Kidding appart, in the old time when people still knew how to write
> operating systems, the notion of temporary file was supported by the OS,
> and those files had really no name (as opposed to a junk name in /var/tmp).

Well, they have names, are stored in in /tmp/ and still exist after
the call of Ada.Text_IO.Close in line 5. I actually checked this :-)

Re: Atomic file creation

[Jean-Pierre Rosen]

vlc a �crit :
> On Jan 4, 4:52 pm, Jean-Pierre Rosen <ro...@adalog.fr> wrote:
>> Maybe because temporary files have no names ;-)
>>
>> Kidding appart, in the old time when people still knew how to write
>> operating systems, the notion of temporary file was supported by the OS,
>> and those files had really no name (as opposed to a junk name in /var/tmp).
> 
> Well, they have names, are stored in in /tmp/ and still exist after
> the call of Ada.Text_IO.Close in line 5. I actually checked this :-)

I presume you are using some elementary OS like Unix, that fakes
temporary files short of having true ones... But the implementation is
free to behave as if they were true temporary files. This protects your
portability (to VMS for example, sigh of nostalgia...)
-- 
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: Atomic file creation

[Jacob Sparre Andersen]

vlc <just.another.spam.account@googlemail.com> writes:

> Looks good, but for some reason this package is not installed on my
> box. I will look for it on the Internet.

If you use Debian, the POSIX packages are in "libflorist-dev" (stable)
and "libflorist2009-dev" (testing).

Greetings,

Jacob
-- 
"Two silk worms had a race. They ended up in a tie."

Re: Atomic file creation

[Ludovic Brenta]

Jean-Pierre Rosen wrote:
> vlc a écrit :
>
> > On Jan 4, 4:52 pm, Jean-Pierre Rosen <ro...@adalog.fr> wrote:
> >> Maybe because temporary files have no names ;-)
>
> >> Kidding appart, in the old time when people still knew how to write
> >> operating systems, the notion of temporary file was supported by the OS,
> >> and those files had really no name (as opposed to a junk name in /var/tmp).
>
> > Well, they have names, are stored in in /tmp/ and still exist after
> > the call of Ada.Text_IO.Close in line 5. I actually checked this :-)
>
> I presume you are using some elementary OS like Unix, that fakes
> temporary files short of having true ones... But the implementation is
> free to behave as if they were true temporary files. This protects your
> portability (to VMS for example, sigh of nostalgia...)

http://www.circa.ufl.edu/handouts/vms/tempfile.html

suggests that (1) not all users of VMS know about nameless temporary
files and (2) VMS does allow one to create named temporary files.

Also I am wondering how VMS allows programs to communicate by means of
nameless temporary files (e.g. program A writes a large amount of data
in a new file and tells program B to process it and then delete the
file).  Are the file names replaced with some kind of handle?

PS. VMS may be "non-elementary" but it is proprietary, ergo they have
something to hide, ergo I don't want it on *my* computer :)

--
Ludovic Brenta.

Re: Atomic file creation

[Dmitry A. Kazakov]

On Mon, 4 Jan 2010 14:38:00 -0800 (PST), Ludovic Brenta wrote:

> Jean-Pierre Rosen wrote:
>> vlc a �crit :
>>
>>> On Jan 4, 4:52 pm, Jean-Pierre Rosen <ro...@adalog.fr> wrote:
>>>> Maybe because temporary files have no names ;-)
>>
>>>> Kidding appart, in the old time when people still knew how to write
>>>> operating systems, the notion of temporary file was supported by the OS,
>>>> and those files had really no name (as opposed to a junk name in /var/tmp).
>>
>>> Well, they have names, are stored in in /tmp/ and still exist after
>>> the call of Ada.Text_IO.Close in line 5. I actually checked this :-)
>>
>> I presume you are using some elementary OS like Unix, that fakes
>> temporary files short of having true ones... But the implementation is
>> free to behave as if they were true temporary files. This protects your
>> portability (to VMS for example, sigh of nostalgia...)
> 
> http://www.circa.ufl.edu/handouts/vms/tempfile.html
> 
> suggests that (1) not all users of VMS know about nameless temporary
> files

No don't think so. This feature existed already in RSX-11. I remember
frequently used it.

> and (2) VMS does allow one to create named temporary files.
> 
> Also I am wondering how VMS allows programs to communicate by means of
> nameless temporary files (e.g. program A writes a large amount of data
> in a new file and tells program B to process it and then delete the
> file).  Are the file names replaced with some kind of handle?

Unnamed temporary files were used as a storage. It was especially important
under RSX-11, which was 16-bit. For example the compiler could store its
between-the-passes stuff in such files. You didn't care to delete the file,
because it was done by the OS, even if the program crashed. Memory mapped
files were supposed to supersede temporal files. But UNIX slashed further
evolution of the operating systems design...

> PS. VMS may be "non-elementary" but it is proprietary, ergo they have
> something to hide, ergo I don't want it on *my* computer :)

Huh, I don't well remember VMS before it became OpenVMS, but RSX-11 was
100% open source, even though that term didn't existed then. All kernel and
drivers sources (in MACRO-11) were available. BTW, I still remember how
much impressed me the quality and style of the kernel code.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Atomic file creation

[Stephen Leake]

vlc <just.another.spam.account@googlemail.com> writes:

> I also tried something like this:
>
>  1  Ada.Text_IO.Create (Handle, Out_File);
>  2  declare
>  3     Temp_File : String := Ada.Text_IO.Name (Handle);
>  4  begin
>  5     Ada.Text_IO.Close (Handle);
>  6     Ada.Text_IO.Rename (Temp_File, "file");
>  7     Ada.Text_IO.Put_Line ("Success");
>  8  exception
>  9     when Ada.IO_Exceptions.Use_Error =>
> 10        Ada.Text_IO.Put_Line ("Failure");
> 11     when others =>
> 12        raise;
> 13  end;
>
> But Ada.Text_IO.Rename always raises Ada.IO_Exceptions.Use_Error, even
> if the file does not exist. 

Ada.Text_IO.Rename is not in the LRM. Ada.Directories.Rename is.

Some OS's don't allow renaming files across devices; if you are on
Unix, and Temp_File is /tmp/.., while "file" is /usr/..., and /tmp is
mounted to disk1, while /usr is mounted to disk2, that might explain
the use error.

In fact, the ALRM for Ada.Directories.Rename says:

67.a/2
          Implementation Note: This operation is expected to work
          within a a single directory, and implementers are encouraged
          to support it across directories on a single device. Copying
          files from one device to another is discouraged (that's what
          Copy_File is for). However, there is no requirement to detect
          file copying by the target system. If the target system has
          an API that gives that for "free", it can be used. For
          Windows®, for instance, MoveFile can be used to implement
          Rename.

> But if I replace line 1 with
>
>  1  Ada.Text_IO.Create (Handle, Out_File, "my_temp_file");
>
> it works. It seems that Ada.Text_IO.Rename cannot rename temporary
> files for whatever reason.
>
> Using Ada.Directories.Copy_File instead of Ada.Text_IO.Rename does not
> help as Ada.Directories.Copy_File overwrites files without raising an
> exception. Bad luck ...

It does seem you need to use an OS specific Form parameter to
Copy_File.

I think it's a bug that Rename raises Use_Error if Target_Name exists,
but Copy_File does not, since Copy_File is supposed to be used to
rename a file across devices.

-- 
-- Stephe

Re: Atomic file creation

[Jacob Sparre Andersen]

Jean-Pierre Rosen wrote:

> Maybe because temporary files have no names ;-)

Exactly.

On POSIX systems it seems like they have to have a name.  At least at
the time when you create them.

It would be so easy to let POSIX.IO.Open_Or_Create create an unnamed
inode, when a program passes an empty string as a path name.  As it is
now, you have to remember to call POSIX.Files.Unlink just after[1]
POSIX.IO.Open_Or_Create, when you want a temporary file.

Greetings,

Jacob

[1] No need to wait until you are finished using the file.
-- 
"... while the C compiler will happily generate code for
 almost anything produced by leaning on the keyboard."

Re: Atomic file creation

[vlc]

On Jan 4, 6:14 pm, Jacob Sparre Andersen <spa...@nbi.dk> wrote:
> If you use Debian, the POSIX packages are in "libflorist-dev" (stable)
> and "libflorist2009-dev" (testing).
>
> Greetings,
>
> Jacob
> --
> "Two silk worms had a race. They ended up in a tie."

After a short fight with the linker I got this library running, which
also solves my problem.

Thanks a lot again to all of you!

Re: Atomic file creation

[vlc]

On Jan 5, 5:16 am, Stephen Leake <stephen_le...@stephe-leake.org>
wrote:
> Ada.Text_IO.Rename is not in the LRM. Ada.Directories.Rename is.
>
> Some OS's don't allow renaming files across devices; if you are on
> Unix, and Temp_File is /tmp/.., while "file" is /usr/..., and /tmp is
> mounted to disk1, while /usr is mounted to disk2, that might explain
> the use error.

Source and target are on the same device, but in different partitions.
Maybe that's an issue (?).

> It does seem you need to use an OS specific Form parameter to
> Copy_File.
>
> I think it's a bug that Rename raises Use_Error if Target_Name exists,
> but Copy_File does not, since Copy_File is supposed to be used to
> rename a file across devices.

Maybe it's a bug, in any case it's confusing ...

Thanks!

Re: Atomic file creation

[Brad Moore]

vlc wrote:
> On Jan 4, 3:49 pm, Jacob Sparre Andersen <spa...@nbi.dk> wrote:
>> POSIX.IO.Open_Or_Create (Name        => [...],
>>                          Mode        => [...],
>>                          Permissions => [...],
>>                          Options     => POSIX.IO.Exclusive);
>>
>> Greetings,
>>
>> Jacob
> 
> Looks good, but for some reason this package is not installed on my
> box. I will look for it on the Internet.
> 
> Thanks a lot!

If you are using GNAT, another alternative is to use the package 
GNAT.Lock_Files

Cheers,

Brad

Re: Atomic file creation

[Stephen Leake]

vlc <just.another.spam.account@googlemail.com> writes:

> On Jan 5, 5:16 am, Stephen Leake <stephen_le...@stephe-leake.org>
> wrote:
>> Ada.Text_IO.Rename is not in the LRM. Ada.Directories.Rename is.
>>
>> Some OS's don't allow renaming files across devices; if you are on
>> Unix, and Temp_File is /tmp/.., while "file" is /usr/..., and /tmp is
>> mounted to disk1, while /usr is mounted to disk2, that might explain
>> the use error.
>
> Source and target are on the same device, but in different partitions.
> Maybe that's an issue (?).

yes, but it might depend on your OS, and on what filesystem you are
using. In Unix, the real issue is what "filesystem" the two paths
point to; they have to be the same. Different partitions are normally
different filesystems.

-- 
-- Stephe