Direct_Io for Filesystem question

Direct_Io for Filesystem question

[mhamel_98]

Hello all,

I have a "filesystem" that I've been using for some time now and with
some free time on my hand I thought I would play around with
concurrency.  I use the term filesystem very loosely, its simply an
instantiation of Direct_Io with a variant record, the first record in
a file is a header and the rest is data and it has worked well
enough.

My approach to concurrency was instead of giving the
direct_io.file_type to a requesting process, I would create my own
file_type which really only contains a token and location value.  The
token would direct the filesystem package to the actual file and the
location value which record to read/write.  When an 'open' is called,
it checks to see if the file is already open and if so returns an
existing token, and if not, opens it and creates a token to perhaps be
shared with others later.  All reads and writes are done using the
Direct_Io calls requiring a record index.  What the internal index of
the file_type is I don't care (maybe I should?)  This also works well
in a single user case.

In the case of two readers, the second reader reads a little bit then
tosses an  End_Error exception, the first reader completes normally.
Again, I'm wondering if the internal index value is relevant.  I would
have thought that explicitly telling direct_io which record to read it
would not care where it last read.

In the case of simultaneous reading and writing (only one writer
allowed), I designed the code so that when a reader opens the file, it
is told that the end of the file is where the writer is currently
located, so the reader will stop before overruning the writing
process' location, and this implementation is fine, no real time
requirements (that might be a neat addition later).  When I run a test
of one reader and one writer, on the surface everything seems ok, no
errors thrown.  When I go over the data later I get "discriminant
check failed" for a few records. Its as if the few moments a reader is
active, the writer tosses in garbage for the records written during
that time span.

I guess the big question is, am I ok to use Direct_Io for this sort of
operation?  It seems to me that the case of multiple readers is really
not much more than one reader doing very random reads?  If anyone has
specific information on the symptoms I'm seeing, I'd really appreciate
it!

Thanks for any advice or opinions!

Re: Direct_Io for Filesystem question

[Jeffrey R. Carter]

mhamel_98@yahoo.com wrote:
> 
> I guess the big question is, am I ok to use Direct_Io for this sort of
> operation?  It seems to me that the case of multiple readers is really
> not much more than one reader doing very random reads?  If anyone has
> specific information on the symptoms I'm seeing, I'd really appreciate
> it!

You must serialize all accesses to the same file object.

-- 
Jeff Carter
"When danger reared its ugly head, he bravely
turned his tail and fled."
Monty Python and the Holy Grail
60

Re: Direct_Io for Filesystem question

[anon]

A quick answer is yes.  You use a user define package from the generic 
"Direct_IO" package to control a large precreated file to simulate a 
file_system.

And each user would be assigned by opening a file, an index within that 
simulated file (disk), basically simulating a sector (record) location on 
that disk. This index could be store in the File-Control-Block.

And example of a sequentail read for your file system could be 
Note: in the FCB record the file Index is store in a variable called 
Location.

  -- FS_FCB => File_System_File_Control_Block.

  procedure read ( fcb : in out FS_FCB ; Data : out Data_Record ) is

    begin
       FS_IO.Set_Index ( File_System_File, fcb.Location  ) ;
       Read ( File_System_File, Data ) ; -- read will advance index pointer
       fcb.Location := Index ( File_System_File ) ;
   end if ;


In <817032b7-4e8a-4589-9e44-00e7aef1fb06@v53g2000hsa.googlegroups.com>, mhamel_98@yahoo.com writes:
>Hello all,
>
>I have a "filesystem" that I've been using for some time now and with
>some free time on my hand I thought I would play around with
>concurrency.  I use the term filesystem very loosely, its simply an
>instantiation of Direct_Io with a variant record, the first record in
>a file is a header and the rest is data and it has worked well
>enough.
>
>My approach to concurrency was instead of giving the
>direct_io.file_type to a requesting process, I would create my own
>file_type which really only contains a token and location value.  The
>token would direct the filesystem package to the actual file and the
>location value which record to read/write.  When an 'open' is called,
>it checks to see if the file is already open and if so returns an
>existing token, and if not, opens it and creates a token to perhaps be
>shared with others later.  All reads and writes are done using the
>Direct_Io calls requiring a record index.  What the internal index of
>the file_type is I don't care (maybe I should?)  This also works well
>in a single user case.
>
>In the case of two readers, the second reader reads a little bit then
>tosses an  End_Error exception, the first reader completes normally.
>Again, I'm wondering if the internal index value is relevant.  I would
>have thought that explicitly telling direct_io which record to read it
>would not care where it last read.
>
>In the case of simultaneous reading and writing (only one writer
>allowed), I designed the code so that when a reader opens the file, it
>is told that the end of the file is where the writer is currently
>located, so the reader will stop before overruning the writing
>process' location, and this implementation is fine, no real time
>requirements (that might be a neat addition later).  When I run a test
>of one reader and one writer, on the surface everything seems ok, no
>errors thrown.  When I go over the data later I get "discriminant
>check failed" for a few records. Its as if the few moments a reader is
>active, the writer tosses in garbage for the records written during
>that time span.
>
>I guess the big question is, am I ok to use Direct_Io for this sort of
>operation?  It seems to me that the case of multiple readers is really
>not much more than one reader doing very random reads?  If anyone has
>specific information on the symptoms I'm seeing, I'd really appreciate
>it!
>
>Thanks for any advice or opinions!

Re: Direct_Io for Filesystem question

[mhamel_98]

On Oct 23, 2:54 pm, "Jeffrey R. Carter"
<spam.jrcarter....@spam.acm.org> wrote:

> You must serialize all accesses to the same file object.

I'm not *certain* I understand what you are saying, but my
interpretation of it led me to wrapping the direct_io.file_type and
its functionality in a protected type and bingo, all is well.  I'm
curious if you meant something altogether different.

I guess "serializing access" should have been more obvious, and it did
occur to me (utilizing a protected type), I just felt since I wasn't
stomping on someone else's state I could bypass protecting the object,
guess not.

Thank you for the help!

Re: Direct_Io for Filesystem question

[Jeffrey R. Carter]

mhamel_98@yahoo.com wrote:
> 
> I'm not *certain* I understand what you are saying, but my
> interpretation of it led me to wrapping the direct_io.file_type and
> its functionality in a protected type and bingo, all is well.  I'm
> curious if you meant something altogether different.

Yes, that's what I meant. Technically a protected object is wrong, because I/O 
operations are "potentially blocking".

-- 
Jeff Carter
"When danger reared its ugly head, he bravely
turned his tail and fled."
Monty Python and the Holy Grail
60