Re: S-expression I/O in Ada

[Natacha Kerensikova <lithiumcat@gmail.com>]

On Aug 10, 12:36 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Tue, 10 Aug 2010 01:56:22 -0700 (PDT), Natacha Kerensikova wrote:
> > On Aug 9, 12:56 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> >> I thought you wanted to represent *objects* ... as S-sequences?
>
> > It depends what you call object. Here again, my vocabulary might has
> > been tainted by C Standard. Take for example a record, I would call
> > each component an object, as well as the whole record itself.
>
> That's OK. Using this definition S-sequence in the memory is an object.

Just to be sure, what is it exactly you call S-sequence? For the rest
of this post I will assume it synonym of S-expression atom, I hope my
answers won't be too misguided.

> Which was the question: what was wrong with the first object so that you
> wanted another instead?

The first object is the internal memory representation designed for
actual efficient use. For example, an integer will probably be
represented by its binary value with machine-defined endianness and
machine-defined size.

The other object is a "serialized" representation, in the sense that
it's designed for communication and storage, for example the same
integer, in a context where it will be sent over a network, can be for
example represented as an ASCII-encoded decimal number, or in binary
but with a predefined size and endianness. This is really the same
considerations as when storing or sending an object directly, except
that is has to reside in memory for a short time. There is no more
conversions or representations than when S-expression-lessly storing
or sending objects; the only difference is the memory buffering to
allow S-expression-specific information to be inserted around the
stream.

> My first take was S-sequence used as an object presentation outside the
> memory, because you used the word "format". Now it looks as a solution
> before the problem. You seem going to convert objects to S-sequences *in*
> the memory and then dump the result them into files. Is it so?

Yes, it is.

> What was the problem then?

The problem is to organize different objects inside a single file. S-
expression standardize the organization and relations between objects,
while something else has to be done beyond S-expression to turn
objects into representations suitable to live in a file.

Or the same thing with, instead of a file, an IPC socket or a network
socket or whatever, I just don't know how to call it generically
without resorting to a word derived from "serialize", but I hope you
get the point anyway.

> Because it cannot work without a conversion between
> S-sequence in the memory (object) and S-sequence in the file
> (representation).

The S-expression standard describe what conversions are allowed and in
what they consist. I cannot follow the standard without such a
conversion anyway, so either I do it or I drop the idea of S-
expressions, but then I don't have anything to portably store or
transmit  objects, so back to square one.

> Why do you need S-sequence in the memory, while dumping
> objects directly into files as S-sequences (if you insist on having them)
> is simpler, cleaner, thinner, faster.

Because I need to examine the S-sequence before writing it to disk, in
order to have enough information to write S-expression metadata. At
the very lest, I need to know the total size of the atom before
allowing its first byte to be send into the file.

> >> Note that Stream_Element is machine-depended as well.
>
> > I'm sadly aware of that. I need an octet-sequence to follow the S-
> > expression standard, and there is here an implementation trade-off:
> > assuming objects already know how to serialize themselves into a
> > Stream_Element_Array, I can either code a converter from
> > Stream_Element_Array to octet-sequence, or reinvent the wheel and code
> > a converter for each type directly into an octet-sequence. For some
> > strange reason I prefer by far the first possibility.
>
> That depends on your goal. Streams are machine-dependent. Streams of octets
> are not. If you want to exchange objects in the form of S-sequences across
> the network you have to drop standard stream implementations of the objects
> and replace them with your own, based on the stream octets.

This looks like a very strong point in favor of using an array-of-
octets to represent S-expression atoms.

> In this case
> you will not use Stream_Element_Array directly. You will read and write
> octets, by Octet'Read and Octet'Write. Provided that octet streams work,
> which is about 99.9%, I guess. When they are not capable to handle octets
> properly, you will have to implement I/O manually. If you wrap Octet'Read
> into a function, you will be able to exchange the I/O layer without
> affecting the upper ones. If we look at all this mechanics we will see the
> old good OSI model.

That sounds like a very nice way of doing it. So in the most common
case, there will still be a stream, provided by the platform-specific
socket facilities, which will accept an array-of-octets, and said
array would have to be created from objects by custom code, right?
(just want to be sure I understood correctly).

> In other post Jeffrey Carter described this as low-level. Why not to tell
> the object: store yourself and all relations you need, I just don't care
> which and how?

That's indeed a higher-level question. That's how it will happen at
some point in my code; however at some other point I will still have
to actually implement said object storage, and that's when I will
really care about which and how. I'm aware from the very beginning
that a S-expression library is low-level and is only used by mid-level
objects before reaching the application.

I've discussed with a friend of mine who has a better knowledge than
me about what actually is a parser and a lexer and things like that.
It happens that what my S-expression code is intended to be is
actually a partial parser, requiring some more specific stuff on top
on it to actually be called a parser. Just like S-expression is a
partial format in that it describes how to serialize relations between
atoms without describing how objects are serialized into atoms.

At some point in my projects I will have to write various
configuration file parsers, template parsers, and maybe a lot of other
parsers. They can all be treated as independent parsers, and
implemented with independent code, maybe derived from YACC or
something. I didn't choose that path, I chose rather to use what I
called a S-expression library, which is sort of a common core to all
these parsers, so I only have left to write the specific part of each
situations, which is matching the keywords and typing/deserializing
the objects.

> > But there are still situations where S-expression have to be stored in
> > memory.
>
> There is no such cases!

Right, I guess just like goto-use, they can always be avoided, but I'm
still not convinced it's always the best.

> > For examples the templates, where S-expressions represent a
> > kind of limited programming language that is re-interpreted for each
> > template extension.
>
> I am not sure what do you mean here, but in general template is not the
> object, its instance is.

Just to be sure we use the same words, I'm talking here about HTML
templates. The basic definition of a S-expression is a list of nodes,
each of which can be a list or an atom; for a template I copy directly
atom contents into the output, and lists are interpreted as functions,
with an "instance" (or whatever is called the thing containing data to
populate the template) handling the actual "execution" of the
function. Now the interest I find in the recursive definition of S-
expression is that it's easy to pass template-fragments as arguments
of these functions. For this it's much easier to let the S-expression
describing the template reside into memory. S-expressions are of
course not required here, but provide a nice and simple unified
format.

> You do not need S-expressions here either. You can
> store/restore templates as S-sequences. A template in the memory would be
> an object with some operations like Instantiate_With_Parameters etc. The
> result of instantiation will be again an object and no S-sequence.

Well how would solve the problem described above without S-
expressions? (That's a real question, if something simpler and/or more
efficient than my way of doing it exists, I'm genuinely interested.)

> > the latter being
> > roughly the low-level (as in "close to the hardware", at least close
> > enough not to rule out programming for embedded platforms and system
> > programming) and the performance.
>
> (BTW, Ada is closer to the hardware than C is. You can even describe
> interrupt handlers in Ada. Try it in C.)

Fantastic \o/

> > Now I can't explain why your posts often make me feel Ada is
> > completely out of my tastes in programming languages,
>
> In the way of programming you mean? I wanted to convey that a
> "C-programmer" will have to change some habits when switching to Ada.

I unsure about you mean here by "C-programmer". As I said in other
posts, I don't really code the same way as other C coders I've met. I
still don't know whether it's a good thing or not. I'm willing to
change some of my habits to switch to Ada, while I won't give up some
other even if mean giving up Ada. The main criterion being that coding
must remain fun, because it makes no sense to continue doing a leisure
activity that isn't fun anymore.

> But a stubborn hardcore "C-programmer" might become very frustrated
> very soon. A competent C developer will only enjoy Ada.

I somehow hope I'm more of the latter than the former.

BTW, that post of yours is one that encourages me rather than restrain
me.


Thanks for your interesting replies,
Natacha