Stream IO - Correct usage of Set_Index

Stream IO - Correct usage of Set_Index

[Rofi]

Platform : Solaris, Linux

Compiler : GNAT 3.10p

Already Looked in : 
	Ada Reference Manual;
	Programming in Ada95 by John Barnes; 
	www.dejanews.com, searching for streams in comp.lang.ada.


Hi

	I am trying to use Stream_IO to create (and use) a file that contains
different data types.  Creating, writing to and reading from the file in a
sequential manner is not a problem.
	My problem is using Set_Index to move to a particular data structure
that is situated part way through the file.  How do I find out the size of
a data structures so that I can use Set_Index correctly.

E.G.

type1 and type2 are of difference sizes.


Start of File

	Var1 of type1
	Var2 of type1
	Var3 of type1
	Var4 of type1
	Var5 of type2
	Var6 of type2     <-- how would I access this variable directly
	.....

End of File

	I have tried using the attribute Size which appears to return the
size of the datastructure in bits.  On this compiler I can divide this by 8
and I get the correct values but this method seem to me to be very Adalike.
Am I missing something monumentally obvious or ...... ?

Cheers

Rofi

Re: Stream IO - Correct usage of Set_Index

[dennison]

In article <7bemc8$snn$1@whatsit.aston.ac.uk>,
  gaukrogi@aston.ac.uk (Rofi) wrote:

> 	I am trying to use Stream_IO to create (and use) a file that contains
> different data types.  Creating, writing to and reading from the file in a
> sequential manner is not a problem.
> 	My problem is using Set_Index to move to a particular data structure
> that is situated part way through the file.  How do I find out the size of
> a data structures so that I can use Set_Index correctly.

> 	I have tried using the attribute Size which appears to return the
> size of the datastructure in bits.  On this compiler I can divide this by 8
> and I get the correct values but this method seem to me to be very Adalike.
> Am I missing something monumentally obvious or ...... ?

First off, the safe way to find the byte size of an object from 'size is to
add 7 and then divide by 8. Otherwise you are likely to get a byte size of 0
for Boolean.

Now for the larger question, I don't think there's a way to know beforehand
how many storage elements an object will take up in a stream, unless you
wrote the 'Write routines for the type and the Write routine for the stream
yourself. For all you know, it may be variable. This is quite likely if
'Output is used on a String. But either 'Write or Write could have used some
kind of compression algorithm as well. You just don't know.

Additionally, Ada.Streams.Stream_IO.Index returns a count of stream elements,
not bytes. While there's a good chance they are the same, you don't even know
that for sure. You have to do a bit of math first. Here's how I do it inside
one of my stream pacakges:

   Bytes_Per_Element : constant := (Ada.Streams.Stream_Element'Size + 7) / 8;

  Elements_Per_Header : constant Ada.Streams.Stream_Element_Offset := 
Ada.Streams.Stream_Element_Offset(  ((Instance'Size + 7) / 8) +
(Bytes_Per_Element - 1) / Bytes_Per_Element);

"Instance" is a type of mine I want to convert to stream elements.

So you may now ask, what are Ada.Streams.Index and Ada.Streams.Set_Index good
for, if you can't use them to skip unnessecary entries in a stream? Well,
what I'm currently using them for is to keep track of previous locations in a
stream that I may want to go back to.

To search for a certian entry (eg: an entry with a timestamp past a certian
time), I will read in all entries one by one, noting their index before I do
so. When an entry with the proper value is found, I call
Ada.Streams.Set_Index with the index I saved before I read the item, and
continue from there.

So what if you *do* need to know how big every entry in a stream is? Simple;
create your own stream that keeps track of that information. (more info on
doing that is available upon request).

T.E.D.

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Stream IO - Correct usage of Set_Index

[Rofi]

In article <7bh1me$v4t$1@nnrp1.dejanews.com>,
	dennison@telepath.com writes:
> In article <7bemc8$snn$1@whatsit.aston.ac.uk>,
>   gaukrogi@aston.ac.uk (Rofi) wrote:
> 
>> 	My problem is using Set_Index to move to a particular data structure
>> that is situated part way through the file.  How do I find out the size of
>> a data structures so that I can use Set_Index correctly.
> 
>> 	I have tried using the attribute Size which appears to return the
>> size of the datastructure in bits.  On this compiler I can divide this by 8
>> and I get the correct values but this method seem to me to be very Adalike.
>> Am I missing something monumentally obvious or ...... ?
> 
> First off, the safe way to find the byte size of an object from 'size is to
> add 7 and then divide by 8. Otherwise you are likely to get a byte size of 0
> for Boolean.

Fair point (should have thought of that one).

> 
> Now for the larger question, I don't think there's a way to know beforehand
> how many storage elements an object will take up in a stream, unless you
> wrote the 'Write routines for the type and the Write routine for the stream
> yourself. For all you know, it may be variable. This is quite likely if
> 'Output is used on a String. But either 'Write or Write could have used some
> kind of compression algorithm as well. You just don't know.

(Sinking feeling in stomarch)

Hadn't thought of that.  Of course that makes what I'm trying to do very
....erm interesting.  I hadn't planned on writing my own Write routines and
the strings should be fixed/bounded ones so .....

> 
> So you may now ask, what are Ada.Streams.Index and Ada.Streams.Set_Index good

<grin>

> for, if you can't use them to skip unnessecary entries in a stream? Well,
> what I'm currently using them for is to keep track of previous locations in a
> stream that I may want to go back to.

Like that idea.  Errrmm (thinking), I'm wondering whether it would be 
possible to actually store positions within a file in the file for use next
time I access the file.  As I know what positions I'm likely to want to go 
to then this could be a nice solution.  Will have to try that one out. 

> 
> To search for a certian entry (eg: an entry with a timestamp past a certian
> time), I will read in all entries one by one, noting their index before I do
> so. When an entry with the proper value is found, I call
> Ada.Streams.Set_Index with the index I saved before I read the item, and
> continue from there.
> 
> So what if you *do* need to know how big every entry in a stream is? Simple;
> create your own stream that keeps track of that information. (more info on
> doing that is available upon request).

Request: more info on doing that :-) (please)
I don't know whether it will the be the best solution for my current problem
but if you have more info easily to hand it would be greatly appreciated.

> 
> T.E.D.

Cheers

Rofi

Re: Stream IO - Correct usage of Set_Index

[dennison]

In article <7bmka1$mso$1@whatsit.aston.ac.uk>,
  gaukrogi@aston.ac.uk (Rofi) wrote:
> In article <7bh1me$v4t$1@nnrp1.dejanews.com>,
> 	dennison@telepath.com writes:
> > what I'm currently using them for is to keep track of previous locations in
a
> > stream that I may want to go back to.
>
> Like that idea.  Errrmm (thinking), I'm wondering whether it would be
> possible to actually store positions within a file in the file for use next
> time I access the file.  As I know what positions I'm likely to want to go
> to then this could be a nice solution.  Will have to try that one out.

That's one possibility. You could also make a "initial pass" through the
stream, taking note of important indexes, before your real reads begin.

> > So what if you *do* need to know how big every entry in a stream is? Simple;
> > create your own stream that keeps track of that information. (more info on
> > doing that is available upon request).
>
> Request: more info on doing that :-) (please)
> I don't know whether it will the be the best solution for my current problem
> but if you have more info easily to hand it would be greatly appreciated.

It is a bit more painful than one would think at first blush. For one thing
you have to define what you consider an "entry" in the stream. On the stream
side you will probably see one Write call for every scalar subobject
contained in an object passed into 'Write. That means every character in a
string will cause one stream Write call! If you tack a length onto the front
of everything passed into the stream's Write, you may end up with more
overhead than data. So you have to have some way to group all the data from
one data structure that you care about on the 'Write side.

The way I solved this problem (after copious advice from c.l.a., thank you
everyone) is to add two calls to the stream interface that clients are to put
around every 'Write and 'Output. The first call places an integer marker in
the stream, and the second replaces the previous marker with the delta in
stream elements from that marker to the end of the stream. There also need to
be calls around 'Read and 'Input so your stream can discard those markers
before they get passed back to the user as data.

Now if I want to implement a Skip_Entry feature for this stream, it could
easily be done by just moving the stream's read pointer forward by the number
of elements stored in the marker it currently points to.

Now odds are that your implementation of file streams doesn't do anything too
goofy (like automatic LZW compression of the data). I haven't tried this, but
you may be able to implement something like this on the 'Write/'Read side, if
you have control of all the code doing the 'Writeing. But that is assuming
Stream_IO.Index and Stream_IO.Set_Index have some meaning during output
operations, which I have not checked into. I said, this method would not be
portable. A new version of your same compiler might even break it, if it were
to change the file stream implementation for some reason.

T.E.D.

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Stream IO - Correct usage of Set_Index

[bourguet]

In article <7bemc8$snn$1@whatsit.aston.ac.uk>,
  gaukrogi@aston.ac.uk (Rofi) wrote:
> Platform : Solaris, Linux
>
> Compiler : GNAT 3.10p
>
> Already Looked in :
> 	Ada Reference Manual;

The answer is there :-)

> 	I have tried using the attribute Size which appears to return the
> size of the datastructure in bits.  On this compiler I can divide this by 8
> and I get the correct values but this method seem to me to be very Adalike.
> Am I missing something monumentally obvious or ...... ?

A stream is a stream of Stream_Element, so the size you want to divide by
is Stream_Element'Size.

But I'd usually design my program to have only to move to result of the
"give me the current position" function (I've not my RM here, so I can't
check).

-- Jean-Marc

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re: Stream IO - Correct usage of Set_Index

[Rofi]

In article <7bgej7$erj$1@nnrp1.dejanews.com>,
	bourguet@my-dejanews.com writes:
> In article <7bemc8$snn$1@whatsit.aston.ac.uk>,
>   gaukrogi@aston.ac.uk (Rofi) wrote:
>>
>> Already Looked in :
>> 	Ada Reference Manual;
> 
> The answer is there :-)

With the correct directing in hindsight of course it is (d'oh). 

> 
>> 	I have tried using the attribute Size which appears to return the
>> size of the datastructure in bits.  On this compiler I can divide this by 8
>> and I get the correct values but this method seem to me to be very Adalike.
>> Am I missing something monumentally obvious or ...... ?
> 
> A stream is a stream of Stream_Element, so the size you want to divide by
> is Stream_Element'Size.
>

Yep, works like a dream.
 
> But I'd usually design my program to have only to move to result of the
> "give me the current position" function (I've not my RM here, so I can't
> check).

Yes, after considering the other reply to my question I think that would
be best. 

> 
> -- Jean-Marc
>

Thanks alot, you saved a lot of hair pulling etc :-).  ( Sorry for my late
reply but I wanted to try out your answer (and then of course got side-
tracked by something else !) )

Cheers

Rofi