S-expression I/O in Ada

S-expression I/O in Ada

[Natacha Kerensikova]

Hi,

I'm trying to learn Ada, coming from a C background. The first thing I
planned to code is a S-expression parser, because it's quite easy (at
least in C, less than 1000 lines including comments and memory
handling (dynamic arrays reinvented)) and very useful considering
almost all my existing programs use S-expressions as a serialization
format.

To describe briefly S-expressions, I consider them to be the simplest
existing data organization beyond raw sequences of bits. They are
basically lists of elements, each element being either a list or an
atom, and atoms being raw sequences of bits.

While I'm still not deep enough into Ada to know how to represent the
lists, I guess there won't be major issues, I think I can handle it
myself (though pointers and hints would still be welcome).

My question here is about how to represent the atoms in Ada. In C it
was merely a void pointer and a size, but it seems more difficult in
Ada because of the strong typing. Because it's up to the application
to make sense (i.e. type) out of the raw sequences of bits in atoms,
the S-expression library has to handle them as a sort of untyped
memory chunk. Do you know of a way to handle that?

Please correct me if I'm wrong, but my guess would be that the S-
expression library would provide a Sexp_Atom type, which refers to the
untyped memory chunks, and the application would have procedures to
convert back and forth between Sexp_Atom and whatever types it
internally uses (i.e. serialization and deserialization procedures).
However the library would provide these procedures for the most common
types (e.g. strings and numeric types).

Though it looks like a fine Ada API (at least to my eyes), I have
absolutely no idea about how to implement the library. How to define
the application-opaque Sexp_Atom type? How to read Sexp_Atom objects
from a file? How to build them from Ada types? How to write them back
to disk?

Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 1 Aug 2010 05:17:45 -0700 (PDT), Natacha Kerensikova wrote:

> To describe briefly S-expressions, I consider them to be the simplest
> existing data organization beyond raw sequences of bits. They are
> basically lists of elements, each element being either a list or an
> atom, and atoms being raw sequences of bits.

So, it is a tree?

> While I'm still not deep enough into Ada to know how to represent the
> lists, I guess there won't be major issues, I think I can handle it
> myself (though pointers and hints would still be welcome).
> 
> My question here is about how to represent the atoms in Ada. In C it
> was merely a void pointer and a size, but it seems more difficult in
> Ada because of the strong typing. Because it's up to the application
> to make sense (i.e. type) out of the raw sequences of bits in atoms,

How can it make sense if the type is unknown? If the type is known, why not
to state it?

> the S-expression library has to handle them as a sort of untyped
> memory chunk. Do you know of a way to handle that?

There are many way to do it.

1. Static polymorphism, generics in Ada. The type of the leaves is the
formal parameter of the package.

2. Dynamic polymorphism.

2.a. The type of a leaf is class wide, each leaf is derived from some
abstract base type. This requires referential approach, i.e. pointers.

2.b. The type of a leaf is a variant type. This is more limiting, but can
be by-value.

> Please correct me if I'm wrong, but my guess would be that the S-
> expression library would provide a Sexp_Atom type, which refers to the
> untyped memory chunks, and the application would have procedures to
> convert back and forth between Sexp_Atom and whatever types it
> internally uses (i.e. serialization and deserialization procedures).

I would not do that.

> However the library would provide these procedures for the most common
> types (e.g. strings and numeric types).
> 
> Though it looks like a fine Ada API (at least to my eyes), I have
> absolutely no idea about how to implement the library. How to define
> the application-opaque Sexp_Atom type?

It is not clear why do you need such a type. What are the properties of,
and what is it for, given the application need to convert it anyway?

As for raw memory addresses, it is possible to reinterpret them to any
desired type in Ada, as you would do it in C, provided you know what are
you doing. For this you can use so-called address-to-access conversion (see
RM 13.7.2) or placement attribute 'Address (see 13.3(11)).

> How to read Sexp_Atom objects from a file?

See stream I/O attributes of types (RM 13.13.2). Otherwise, standard design
patterns apply too. In any case you have to know the object's type in order
to write/read it.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 1, 2:53 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Sun, 1 Aug 2010 05:17:45 -0700 (PDT), Natacha Kerensikova wrote:
> > To describe briefly S-expressions, I consider them to be the simplest
> > existing data organization beyond raw sequences of bits. They are
> > basically lists of elements, each element being either a list or an
> > atom, and atoms being raw sequences of bits.
>
> So, it is a tree?

Yes, it can be thought of as a binary tree whose leaves are labeled
(the
atom being the label) and with different semantics for the left and
right
children. I'm unsure this represent correctly empty lists, but
otherwise
that's it.

> > My question here is about how to represent the atoms in Ada. In C it
> > was merely a void pointer and a size, but it seems more difficult in
> > Ada because of the strong typing. Because it's up to the application
> > to make sense (i.e. type) out of the raw sequences of bits in atoms,
>
> How can it make sense if the type is unknown? If the type is known, why not
> to state it?

Actually the type is deduced from the context, e.g.
(tcp-connect (host foo.example) (port 80))
We've got here five atoms and three lists, and the application reading
as a
configuration file would know the atom after "host" is a string to be
resolved and the atom after port is a number serialized in a decimal
representation. However this serialization of a number is an
application
choice (which I usually choose to make S-expressions text files) but
it could
have been serialized as a network-ordered 2-byte integer.

This means that as far as the S-expression library is concerned, the
byte
sequence read from the file is not typed. Its typing actually has to
be
delayed until the application has enough context to interpret it.
Hence
the need of a typeless chunk of data.

> > Though it looks like a fine Ada API (at least to my eyes), I have
> > absolutely no idea about how to implement the library. How to define
> > the application-opaque Sexp_Atom type?
>
> It is not clear why do you need such a type. What are the properties of,
> and what is it for, given the application need to convert it anyway?

Well, I imagined making such a new type to distinguish 'raw data
coming
from a S-expression to be interpreted' from other raw data types.
I thought this was the strong type safety to prevent (de)serialization
procedure from trying to interpret just any chunk of memory.

> As for raw memory addresses, it is possible to reinterpret them to any
> desired type in Ada, as you would do it in C, provided you know what are
> you doing. For this you can use so-called address-to-access conversion (see
> RM 13.7.2) or placement attribute 'Address (see 13.3(11)).

Ok, thanks for this references, I guess I'll find there how many
guarantees
there are on such raw memory inspection/interpretation. I've already
encountered
a few situations in C where the code looks very dangerous but where I
know
perfectly well what I'm doing (I also sort-of expected it would be
easier to
convince other people I really know what's going on and what I'm doing
in such
cases when the code is in Ada rather than in C).

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 1 Aug 2010 10:35:17 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 1, 2:53�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:

>> How can it make sense if the type is unknown? If the type is known, why not
>> to state it?
> 
> Actually the type is deduced from the context, e.g.
> (tcp-connect (host foo.example) (port 80))

Hmm, why do you consider brackets as separate elements? I mean, more
natural would be "procedural":

   tcp-connect (host (foo.example), port (80))

or

   tcp-connect (foo.example, 80)

> This means that as far as the S-expression library is concerned, the byte
> sequence read from the file is not typed. Its typing actually has to be
> delayed until the application has enough context to interpret it. Hence
> the need of a typeless chunk of data.

The above is mere syntax, it is not clear why internal/external
representation must be as you described. (Actually, the structures as above
are widely used in compiler construction e.g. syntax tree, Reverse Polish
notation etc.)

There is no such thing as untyped data. The information about the type must
be somewhere. In your example it could be:

   type Connect is new Abstract_Node with record
       Host : IP_Address;
       Port : Port_Type;
   end record;

So I would derive leaves from some abstract base type and link them into an
acyclic graph. It is relatively simple to do using either standard Ada
containers or other libraries containing trees and/or graphs.

>>> Though it looks like a fine Ada API (at least to my eyes), I have
>>> absolutely no idea about how to implement the library. How to define
>>> the application-opaque Sexp_Atom type?
>>
>> It is not clear why do you need such a type. What are the properties of,
>> and what is it for, given the application need to convert it anyway?
> 
> Well, I imagined making such a new type to distinguish 'raw data coming
> from a S-expression to be interpreted' from other raw data types.

But you know the type if you are going to interpret raw data. It must be
somewhere, hard-coded or dynamically stored/restored.

BTW, Ada supports I/O of types (i.e type tags) implicitly in the form of
already mentioned stream attributes, but also explicitly by external type
tag representations (RM 3.9(7/2)) and generic dispatching constructors (RM
3.9(18/1)).

> I thought this was the strong type safety to prevent (de)serialization
> procedure from trying to interpret just any chunk of memory.

No, actually it eases serialization because you can define
Serialize/Unserialize operations on the type.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 1, 8:49 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Sun, 1 Aug 2010 10:35:17 -0700 (PDT), Natacha Kerensikova wrote:
> > On Aug 1, 2:53 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> > wrote:
> >> How can it make sense if the type is unknown? If the type is known, why not
> >> to state it?
>
> > Actually the type is deduced from the context, e.g.
> > (tcp-connect (host foo.example) (port 80))
>
> Hmm, why do you consider brackets as separate elements?

Because that's the definition of S-expressions :-)
I'm referring to this almost-RFC: http://people.csail.mit.edu/rivest/Sexp.txt

> I mean, more natural would be "procedural":
>
>    tcp-connect (host (foo.example), port (80))
>
> or
>
>    tcp-connect (foo.example, 80)

Those are also perfectly valid ways of serializing the same
information, but they are simply not S-expressions.

I use S-expression as a sort of universal text-based container, just
like most people use XML these days, except S-expressions can easily
embed binary data and they are greatly simpler (as I said, ~1000 lines
of commented 80-column C code with a few reinvented wheels embedded).

> > This means that as far as the S-expression library is concerned, the byte
> > sequence read from the file is not typed. Its typing actually has to be
> > delayed until the application has enough context to interpret it. Hence
> > the need of a typeless chunk of data.
>
> The above is mere syntax, it is not clear why internal/external
> representation must be as you described. (Actually, the structures as above
> are widely used in compiler construction e.g. syntax tree, Reverse Polish
> notation etc.)
>
> There is no such thing as untyped data. The information about the type must
> be somewhere. In your example it could be:
>
>    type Connect is new Abstract_Node with record
>        Host : IP_Address;
>        Port : Port_Type;
>    end record;

It is indeed somewhere, but beyond the reach of a S-expression
library: its belongs to the application using the library. The record
you show here can't be part of a generic S-expression handling
library.

In my example, as far as the S-expression library is concerned, it
looks like:
"(<some_stuff_1> (<some_stuff_2> <some_stuff_3>) (<some_stuff_4>
<some_stuff_5>))"

The library is not supposed to care about what those some_stuff_ are.
Actually, the library is suppose to get binary data from the
application along with the tree structure described above, store it
into a sequence of bytes (on a disk or over a network or whatever),
and to retrieve from the byte sequence the original tree structure
along with the binary data provided.

But now that I think about it, I'm wondering whether I'm stuck in my C
way of thinking and trying to apply it to Ada. Am I missing an Ada way
of storing structured data in a text-based way?

> > I thought this was the strong type safety to prevent (de)serialization
> > procedure from trying to interpret just any chunk of memory.
>
> No, actually it eases serialization because you can define
> Serialize/Unserialize operations on the type.

I honestly don't understand what you mean here.

Am I misusing the word "serialization" to describe the process of
converting an object from an internal representation (e.g. an integer)
to a byte sequence (e.g. the ASCII string of its decimal
representation, or the byte sequence in memory, both of them being two
acceptable but different serializations, corresponding to different
trade-offs (portability and readability vs space and time
efficiency))?


Thanks for making me think,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 1 Aug 2010 13:06:10 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 1, 8:49�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> On Sun, 1 Aug 2010 10:35:17 -0700 (PDT), Natacha Kerensikova wrote:
>>> On Aug 1, 2:53�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
>>> wrote:
>>>> How can it make sense if the type is unknown? If the type is known, why not
>>>> to state it?
>>
>>> Actually the type is deduced from the context, e.g.
>>> (tcp-connect (host foo.example) (port 80))
>>
>> Hmm, why do you consider brackets as separate elements?
> 
> Because that's the definition of S-expressions :-)

OK, why S-expressions, then? (:-))

> I'm referring to this almost-RFC: http://people.csail.mit.edu/rivest/Sexp.txt

I see, yet another poor data format.

> I use S-expression as a sort of universal text-based container, just
> like most people use XML these days,

These wonder me too. I see no need in text-based containers for binary
data. Binary data aren't supposed to be read by people, they read texts.
And conversely the machine shall not read texts, it has no idea of good
literature...

>>> This means that as far as the S-expression library is concerned, the byte
>>> sequence read from the file is not typed. Its typing actually has to be
>>> delayed until the application has enough context to interpret it. Hence
>>> the need of a typeless chunk of data.
>>
>> The above is mere syntax, it is not clear why internal/external
>> representation must be as you described. (Actually, the structures as above
>> are widely used in compiler construction e.g. syntax tree, Reverse Polish
>> notation etc.)
>>
>> There is no such thing as untyped data. The information about the type must
>> be somewhere. In your example it could be:
>>
>> � �type Connect is new Abstract_Node with record
>> � � � �Host : IP_Address;
>> � � � �Port : Port_Type;
>> � �end record;
> 
> It is indeed somewhere, but beyond the reach of a S-expression
> library: its belongs to the application using the library.

Yes

> The record
> you show here can't be part of a generic S-expression handling
> library.

It need not to be. Abstract_Node should declare Serialize and Unserialize
operations, which Connect does implement. The library makes dispatching
calls. Of course you can declare something like:

 � type List_Of_Anything is new Abstract_Node with
       -- Container of class-wide objects

and implement its Serialize and Unserialize through walking the items of an
calling its Serialize and Unserialize. BTW, this is how attributes 'Input
and 'Output work with arrays.

> In my example, as far as the S-expression library is concerned, it
> looks like:
> "(<some_stuff_1> (<some_stuff_2> <some_stuff_3>) (<some_stuff_4>
> <some_stuff_5>))"

That is no matter. Serialize/Unserialize will use this format if they have
to.

> The library is not supposed to care about what those some_stuff_ are.
> Actually, the library is suppose to get binary data from the
> application along with the tree structure described above, store it
> into a sequence of bytes (on a disk or over a network or whatever),
> and to retrieve from the byte sequence the original tree structure
> along with the binary data provided.

Serialize can yield a binary chunk, but if you have to write it anyway, why
not to write text? You insisted on having text, why do mess with binary
stuff?

> But now that I think about it, I'm wondering whether I'm stuck in my C
> way of thinking and trying to apply it to Ada. Am I missing an Ada way
> of storing structured data in a text-based way?

I think yes. Though it is not Ada-specific, rather commonly used OOP design
patterns.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 8/1/10 11:13 PM, Dmitry A. Kazakov wrote:

>> I use S-expression as a sort of universal text-based container, just
>> like most people use XML these days,
>
> These wonder me too. I see no need in text-based containers for binary
> data. Binary data aren't supposed to be read by people, they read texts.
> And conversely the machine shall not read texts, it has no idea of good
> literature...

The whole idea of machine readable Ada source text is silly, right?

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Mon, 02 Aug 2010 09:17:15 +0200, Georg Bauhaus wrote:

> On 8/1/10 11:13 PM, Dmitry A. Kazakov wrote:
> 
>>> I use S-expression as a sort of universal text-based container, just
>>> like most people use XML these days,
>>
>> These wonder me too. I see no need in text-based containers for binary
>> data. Binary data aren't supposed to be read by people, they read texts.
>> And conversely the machine shall not read texts, it has no idea of good
>> literature...
> 
> The whole idea of machine readable Ada source text is silly, right?

Right.

1. Ada sources are read by the compiler.

2. Sources need not to be text file based. Compare it with the word
processor, an application the whole purpose of which is to produce texts,
that aren't text-based... (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 1, 11:13 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Sun, 1 Aug 2010 13:06:10 -0700 (PDT), Natacha Kerensikova wrote:
> > On Aug 1, 8:49 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> > wrote:
> >> Hmm, why do you consider brackets as separate elements?
> > Because that's the definition of S-expressions :-)
> OK, why S-expressions, then? (:-))

First, because this is the existing format in most of my programs, so
for interoperability I can choose only between using this format or
converting from and to it (or rewrite everything). In both cases there
is a strong code-reuse argument in favor of writing a S-expression
library instead of writing a bunch of ad-hoc similar code chunks.

Second, because I like this format and I find it good (see below).

> > I'm referring to this almost-RFC:http://people.csail.mit.edu/rivest/Sexp.txt
> I see, yet another poor data format.

Could you please explain why it is so poor?

I consider it good because it is flexible, expressive and simple. I
already mentioned quite a few times why it looks simple: my parser
written from scratch in ~1000 lines. I looks expressive, because most
data structures I can think of, and all data structures I have
actually used, can be easily represented: an array can be written down
as a list, a hash table as a list of two-element lists (key then
value), and so on. And I see flexibility coming from the fact that any
sequence of bytes can be encoded in an atom.

What am I missing?

> > I use S-expression as a sort of universal text-based container, just
> > like most people use XML these days,
>
> These wonder me too. I see no need in text-based containers for binary
> data. Binary data aren't supposed to be read by people, they read texts.
> And conversely the machine shall not read texts, it has no idea of good
> literature...

Actually this an interesting remark, it made me realize I'm mixing two
very different use of a data format (though I still think S-
expressions are adequate for both of them):

I think text-based format is very useful when the file has to be dealt
with by both humans and programs. The typical example would be
configuration files: read and written by humans, and used by the
program. And that's where I believe XML is really poor, because it's
too heavy for human use. I occasionally feel the need of embedding
binary data in some configuration files, e.g. cryptographic keys or
binary initialization sequences to send as-is over whatever
communication channel. In these occasion I do use the text-based
binary encoding allowed by S-expressions, base-64 or hexadecimal, so
that the configuration file is still a text file. The huge advantage
of text files here is that there is already a lot of tools to deal
with it, while using a binary format would require writing specific
tools for humans to deal with it, with is IMO a waste of time compared
to the text-based approach.

The other application is actual serialization, i.e. converting
internal types into a byte sequence in order to be stored on disk or
transmitted over a network or whatever. In this situation, humans
don't need to interact with the data (except for debugging purposes,
but it's an argument so light it's only justified when everything else
is otherwise equal).


In my previous posts I have talked a lot about serialization, while my
actual use of S-expression is more often the first one. And
historically I first used S-expressions for configuration files,
because of their expressiveness over all other text format I know,
while still being extremely simple. I then used this format for
serialization mostly for code reuse sake: I did have a code for S-
expressions, so it was very cheap to use it for serialization
purposes. Compared to using another serialization format, it leads to
less code being more used, hence less opportunities to write bugs and
more opportunities to find and fix bugs. So it seems like a very
rational choice.

> > The library is not supposed to care about what those some_stuff_ are.
> > Actually, the library is suppose to get binary data from the
> > application along with the tree structure described above, store it
> > into a sequence of bytes (on a disk or over a network or whatever),
> > and to retrieve from the byte sequence the original tree structure
> > along with the binary data provided.
>
> Serialize can yield a binary chunk, but if you have to write it anyway, why
> not to write text? You insisted on having text, why do mess with binary
> stuff?

I hope the above already answered this: I mostly use S-expressions for
text purposes, yet occasionally I feel the need of embedding binary
data. Of course there are a lot of way to testify binary data, like
hexadecimal or base-64, but considering S-expressions already handle
textification, I don't see the point of having the application deal
with it too.

> > But now that I think about it, I'm wondering whether I'm stuck in my C
> > way of thinking and trying to apply it to Ada. Am I missing an Ada way
> > of storing structured data in a text-based way?
>
> I think yes. Though it is not Ada-specific, rather commonly used OOP design
> patterns.

I heard people claiming that the first language shapes the mind of
coders (and they continue saying a whole generation of programmers has
been mind-crippled by BASIC). My first language happened to be 386
assembly, that might explain things. Anyway, I genuinely tried OOP
with C++ (which I dropped because it's way too complex for me (and I'm
tempted to say way too complex for the average coder, it should be
reserved to the few geniuses actually able to fully master it)), but I
never felt the need of anything beyond what can be done with a C
struct containing function pointers.

Now back to the topic, thanks to your post and some others in this
thread (for which I'm also thankful), I came to realize my mistake is
maybe wanting to parse S-expressions and atom contents separately. The
problem is, I just can't manage to imagine how to go in a single step
from the byte sequence containing a S-expression describing multiple
objects to the internal memory representation and vice-versa.


Thanks for your help and your patience,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sat, 7 Aug 2010 00:23:01 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 1, 11:13�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> On Sun, 1 Aug 2010 13:06:10 -0700 (PDT), Natacha Kerensikova wrote:
>>> On Aug 1, 8:49�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
>>> wrote:
>>>> Hmm, why do you consider brackets as separate elements?
>>> Because that's the definition of S-expressions :-)
>> OK, why S-expressions, then? (:-))
> 
> First, because this is the existing format in most of my programs, so
> for interoperability I can choose only between using this format or
> converting from and to it (or rewrite everything). In both cases there
> is a strong code-reuse argument in favor of writing a S-expression
> library instead of writing a bunch of ad-hoc similar code chunks.

Legacy stuff also. That is a valid argument.

>>> I'm referring to this almost-RFC:http://people.csail.mit.edu/rivest/Sexp.txt
>> I see, yet another poor data format.
> 
> Could you please explain why it is so poor?
> 
> I consider it good because it is flexible, expressive and simple. I
> already mentioned quite a few times why it looks simple: my parser
> written from scratch in ~1000 lines. I looks expressive, because most
> data structures I can think of, and all data structures I have
> actually used, can be easily represented: an array can be written down
> as a list, a hash table as a list of two-element lists (key then
> value), and so on. And I see flexibility coming from the fact that any
> sequence of bytes can be encoded in an atom.
> 
> What am I missing?

The requirements.

One cannot judge a format without knowing what is the purpose of. Most of
the formats like S-expressions are purposeless, in the sense that there is
no *rational* purpose behind them. As you wrote above, it is either legacy
(we have to overcome some limitations of some other poorly designed
components of the system) or personal preferences (some people like angle
brackets others do curly ones).
 
>>> I use S-expression as a sort of universal text-based container, just
>>> like most people use XML these days,
>>
>> These wonder me too. I see no need in text-based containers for binary
>> data. Binary data aren't supposed to be read by people, they read texts.
>> And conversely the machine shall not read texts, it has no idea of good
>> literature...
> 
> Actually this an interesting remark, it made me realize I'm mixing two
> very different use of a data format (though I still think S-
> expressions are adequate for both of them):
> 
> I think text-based format is very useful when the file has to be dealt
> with by both humans and programs. The typical example would be
> configuration files: read and written by humans, and used by the
> program.

There should be no configuration files at all. The idea that a
configuration can be edited using a text editor is corrupt.

> And that's where I believe XML is really poor, because it's
> too heavy for human use. I occasionally feel the need of embedding
> binary data in some configuration files, e.g. cryptographic keys or
> binary initialization sequences to send as-is over whatever
> communication channel. In these occasion I do use the text-based
> binary encoding allowed by S-expressions, base-64 or hexadecimal, so
> that the configuration file is still a text file. The huge advantage
> of text files here is that there is already a lot of tools to deal
> with it, while using a binary format would require writing specific
> tools for humans to deal with it, with is IMO a waste of time compared
> to the text-based approach.

All these tools are here exclusively to handle poor formats of these files.
They add absolutely nothing to the actual purpose of configuration, namely
to handle the *semantics* of the given configuration parameter. None
answers simple questions like: How do I make the 3-rd button on the left
4cm large? Less than none verify the parameter values.

The king is naked.

> The other application is actual serialization,

That should not be a text.

>>> But now that I think about it, I'm wondering whether I'm stuck in my C
>>> way of thinking and trying to apply it to Ada. Am I missing an Ada way
>>> of storing structured data in a text-based way?
>>
>> I think yes. Though it is not Ada-specific, rather commonly used OOP design
>> patterns.
> 
> I heard people claiming that the first language shapes the mind of
> coders (and they continue saying a whole generation of programmers has
> been mind-crippled by BASIC). My first language happened to be 386
> assembly, that might explain things.

I see where mixing abstraction layers comes from...

> Anyway, I genuinely tried OOP
> with C++ (which I dropped because it's way too complex for me (and I'm
> tempted to say way too complex for the average coder, it should be
> reserved to the few geniuses actually able to fully master it)), but I
> never felt the need of anything beyond what can be done with a C
> struct containing function pointers.

Everything is Turing-complete you know... (:-))

> The
> problem is, I just can't manage to imagine how to go in a single step
> from the byte sequence containing a S-expression describing multiple
> objects to the internal memory representation and vice-versa.

You need not, that is the power of OOP you dislike so much. Consider each
object knows how to construct itself from a stream of octets. It is trivial
to simple objects like number. E.g. you read until the octets are '0'..'9'
and generate the result interpreting it as a decimal representation. Or you
take four octets and treat them as big-endian binary representation etc.
For a container type, you call the constructors for each container member
in order. If the container is unbounded, e.g. has variable length, you read
its bounds first or you use some terminator in the stream to mark the
container end. For containers of dynamically typed elements you must learn
the component type before you construct it.

In the theory this is called the recursive descent parser, the simplest
thing ever.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 7, 10:39 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> One cannot judge a format without knowing what is the purpose of. Most of
> the formats like S-expressions are purposeless, in the sense that there is
> no *rational* purpose behind them. As you wrote above, it is either legacy
> (we have to overcome some limitations of some other poorly designed
> components of the system) or personal preferences (some people like angle
> brackets others do curly ones).

Why can't there be general-purpose format, just like there are general-
purpose programming languages?

Can we at least agree on the fact that a sequence of bytes is a
general-purpose format, widely used for storing and transmitting data?
(this question is just a matter of vocabulary)

Now byte sequences are a very crude format, because it doesn't have
any semantics besides what the application specifically put into it.

So let's add as few semantics as possible, to keep as much generality
as possible. We end up with a bunch of byte sequences, whose semantics
are still left to the application, linked together by some kind of
semantic link. When the chosen links are "brother of" and "sublist of"
you get exactly S-expressions. The almost-RFC I linked is only this
definition along with a standardization of how to serialize the links
and the bytes sequences.

This is undoubtedly still a crude format. You might argue it's useless
to add so little semantics on top of byte sequences, and that
serialization should be engineered only when you what you are about to
serialize, i.e. make a much larger leap between byte sequences and
meaningful objects. I might even agree on a philosophical point of
view.

However from a purely practical point of view, and using the fact that
in my background languages (C and 386 asm) bytes sequences and strings
are so similar, these crude semantics are all I need (or at least, all
I've ever needed so far). Now if we agree that simplicity is a
desirable quality (because it leads to less bugs, more productivity,
etc), I still fail to see the issues of such a format.

When I mentioned earlier flexibility as a strong point for S-
expressions, I meant that just like byte sequences, they can
accommodate whatever purpose you might want to put on top of them.

Now regarding personal preferences about braces, I have to admit I'm
always shocked at the way so many people dismiss S-expressions on
first sight because of the LISP-looking parentheses. I'm very glad I
can have a higher-level conversation about them.

> > The other application is actual serialization,
>
> That should not be a text.

I tend to agree with that as a generality, however I believe some
particular cases might benefit from a text-based serializations, in
order to harness the power of existing text-based tools.

> >>> But now that I think about it, I'm wondering whether I'm stuck in my C
> >>> way of thinking and trying to apply it to Ada. Am I missing an Ada way
> >>> of storing structured data in a text-based way?
>
> >> I think yes. Though it is not Ada-specific, rather commonly used OOP design
> >> patterns.
>
> > I heard people claiming that the first language shapes the mind of
> > coders (and they continue saying a whole generation of programmers has
> > been mind-crippled by BASIC). My first language happened to be 386
> > assembly, that might explain things.
>
> I see where mixing abstraction layers comes from...

Could you please point me where I am mixing what? I genuinely want to
learn, but I just don't understand what you're referring to.

> > Anyway, I genuinely tried OOP
> > with C++ (which I dropped because it's way too complex for me (and I'm
> > tempted to say way too complex for the average coder, it should be
> > reserved to the few geniuses actually able to fully master it)), but I
> > never felt the need of anything beyond what can be done with a C
> > struct containing function pointers.
>
> Everything is Turing-complete you know... (:-))

I should know, I was accessing from assembly some (DirectX) C++
objects' vtable array before I knew anything about OOP.

My point is, most of my (currently non-OOP) code can be expressed as
well in an OOP style. When I defined a C structure along with a bunch
of functions that perform operations on it, I'm conceptually defining
a class and its methods, only expressed in a non-OOP language. I
sometimes put function pointers in the structure, to have a form of
dynamic dispatch or virtual methods. I occasionally even leave the
structure declared but undefined publicly, to hide internals (do call
that encapsulation?), along with functions that could well be called
accessors and mutators. In my opinion that doesn't count as OOP
because it doesn't use OOP-specific features like inheritance.

> > The
> > problem is, I just can't manage to imagine how to go in a single step
> > from the byte sequence containing a S-expression describing multiple
> > objects to the internal memory representation and vice-versa.
>
> You need not, that is the power of OOP you dislike so much.

I don't dislike at all. I just seldom think that way. I've met people
who saw objects everywhere, while I tend to see bunch of bits
everywhere. As a part of demoscene said when I learned programming,
"100% asm, a way of life".

> Consider each
> object knows how to construct itself from a stream of octets. It is trivial
> to simple objects like number. E.g. you read until the octets are '0'..'9'
> and generate the result interpreting it as a decimal representation. Or you
> take four octets and treat them as big-endian binary representation etc.
> For a container type, you call the constructors for each container member
> in order. If the container is unbounded, e.g. has variable length, you read
> its bounds first or you use some terminator in the stream to mark the
> container end. For containers of dynamically typed elements you must learn
> the component type before you construct it.

That's exactly how I use S-expressions, except that instead of
starting from a stream of octets, I start from an array of octets
(whose length is known), but if I understood correctly that doesn't
change your point. And the reason why I started this thread is only to
know how to buffer into memory the arrays of octets, because I need
(in my usual use of S-expressions) to resolve the links between atoms
before I can know the type of atoms. So I need a way to delay the
typing, and in the meantime handle data as a generic byte sequence
whose only known information is its size and its place in the S-
expression tree. What exactly is so bad with that approach?


I hope I don't bother you too much with my noobity and my will to
understand,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sat, 7 Aug 2010 05:56:50 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 7, 10:39�am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> One cannot judge a format without knowing what is the purpose of. Most of
>> the formats like S-expressions are purposeless, in the sense that there is
>> no *rational* purpose behind them. As you wrote above, it is either legacy
>> (we have to overcome some limitations of some other poorly designed
>> components of the system) or personal preferences (some people like angle
>> brackets others do curly ones).
> 
> Why can't there be general-purpose format, just like there are general-
> purpose programming languages?

An interesting question. I would say no, there cannot be such formats. Any
presentation format is of course a language. The only difference to the
true programming languages is in complexity, any maybe in a tendency to
being declarative rather than imperative. There are border cases like
Postscript, which IMO illustrate the point, more general purpose it has to
be, less "format" it would become.

> Can we at least agree on the fact that a sequence of bytes is a
> general-purpose format, widely used for storing and transmitting data?
> (this question is just a matter of vocabulary)

I don't think so. Namely it don't think that "general" is a synonym to
"completeness." It is rather about the abstraction level under the
condition of completeness.

> So let's add as few semantics as possible, to keep as much generality
> as possible. We end up with a bunch of byte sequences, whose semantics
> are still left to the application, linked together by some kind of
> semantic link. When the chosen links are "brother of" and "sublist of"
> you get exactly S-expressions.

Yes, the language of S-expressions is about hierarchical structures of
elements lacking any semantics.

I see no purpose such descriptions. But this is a very old and bearded
issue. The same question arise when it is discussed why RDBMS are so
boring. For the same reason: a naked structure, be it relational,
hierarchical, whichever, is useless without the semantics. The semantics
when dealt with, is capable to catch such simple relationships as "sibling"
with no efforts. DB people believe that one could bridge the gap and
somehow come to the semantics from the structure's side. Translated into
your S-expressions, it is by putting a proper pattern of opening and
closing brackets one could describe everything...

> However from a purely practical point of view, and using the fact that
> in my background languages (C and 386 asm) bytes sequences and strings
> are so similar, these crude semantics are all I need (or at least, all
> I've ever needed so far).

Lower you descend down the abstraction levels less differences you see.
Everything is a bunch of transistors...

> Now if we agree that simplicity is a
> desirable quality (because it leads to less bugs, more productivity,
> etc), I still fail to see the issues of such a format.

Programs in 386 Assembler are sufficiently more complex than programs in
Ada. Simplicity of nature by no means implies simplicity of use.
 
> Now regarding personal preferences about braces, I have to admit I'm
> always shocked at the way so many people dismiss S-expressions on
> first sight because of the LISP-looking parentheses.

Do you mean LISP does not deserve its fame? (:-))

>>>>> But now that I think about it, I'm wondering whether I'm stuck in my C
>>>>> way of thinking and trying to apply it to Ada. Am I missing an Ada way
>>>>> of storing structured data in a text-based way?
>>
>>>> I think yes. Though it is not Ada-specific, rather commonly used OOP design
>>>> patterns.
>>
>>> I heard people claiming that the first language shapes the mind of
>>> coders (and they continue saying a whole generation of programmers has
>>> been mind-crippled by BASIC). My first language happened to be 386
>>> assembly, that might explain things.
>>
>> I see where mixing abstraction layers comes from...
> 
> Could you please point me where I am mixing what?

Encoding, representation, states, behavior, values, objects, everything is
a sequence of bytes, so?

> My point is, most of my (currently non-OOP) code can be expressed as
> well in an OOP style. When I defined a C structure along with a bunch
> of functions that perform operations on it, I'm conceptually defining
> a class and its methods, only expressed in a non-OOP language. I
> sometimes put function pointers in the structure, to have a form of
> dynamic dispatch or virtual methods. I occasionally even leave the
> structure declared but undefined publicly, to hide internals (do call
> that encapsulation?), along with functions that could well be called
> accessors and mutators. In my opinion that doesn't count as OOP
> because it doesn't use OOP-specific features like inheritance.

I disagree because in my view this is all what OO is about. OO is not about
the tools (OOPL), it is about the way of programming.

>>> The
>>> problem is, I just can't manage to imagine how to go in a single step
>>> from the byte sequence containing a S-expression describing multiple
>>> objects to the internal memory representation and vice-versa.
>>
>> You need not, that is the power of OOP you dislike so much.
> 
> I don't dislike at all. I just seldom think that way. I've met people
> who saw objects everywhere, while I tend to see bunch of bits
> everywhere.

Yes, I know them too. I don't believe that everything is object. But I do
believe in abstract type systems, that every object in a well-designed
program must have a type and that type shall describe the behavior as
precise as possible.

> And the reason why I started this thread is only to
> know how to buffer into memory the arrays of octets, because I need
> (in my usual use of S-expressions) to resolve the links between atoms
> before I can know the type of atoms. So I need a way to delay the
> typing, and in the meantime handle data as a generic byte sequence
> whose only known information is its size and its place in the S-
> expression tree. What exactly is so bad with that approach?

Nothing wrong when at the implementation level. However I don't see why
links need to be resolved first. In comparable cases - I do much messy
protocol/communication stuff - I usually first restore objects and then
resolve links.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 7, 4:23 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Sat, 7 Aug 2010 05:56:50 -0700 (PDT), Natacha Kerensikova wrote:
> > Can we at least agree on the fact that a sequence of bytes is a
> > general-purpose format, widely used for storing and transmitting data?
> > (this question is just a matter of vocabulary)
>
> I don't think so. Namely it don't think that "general" is a synonym to
> "completeness." It is rather about the abstraction level under the
> condition of completeness.

Well, then I'm afraid I can discuss anymore, because I fail to
understand your definition here.

I was using "general-purpose" as the opposite of "specific-purpose".
If we make the parallel with compression schemes, FLAC sure is as
complete as bzip2, yet the first one has a specific purpose
(compressing sounds) while the other is general-purpose. So back to
data format, I made the distinction in the amount preliminary
assumptions about data to be contained. In that sense the raw byte-
sequence is the most general format in that there is no assumption
about the contained data (except that its number of bits is a multiple
of the number of bits per byte).

> > So let's add as few semantics as possible, to keep as much generality
> > as possible. We end up with a bunch of byte sequences, whose semantics
> > are still left to the application, linked together by some kind of
> > semantic link. When the chosen links are "brother of" and "sublist of"
> > you get exactly S-expressions.
>
> Yes, the language of S-expressions is about hierarchical structures of
> elements lacking any semantics.
>
> I see no purpose such descriptions.

Indeed, I don't see any either, and that's the point: there is room to
add your application-specific purpose on top of this format.

> > However from a purely practical point of view, and using the fact that
> > in my background languages (C and 386 asm) bytes sequences and strings
> > are so similar, these crude semantics are all I need (or at least, all
> > I've ever needed so far).
>
> Lower you descend down the abstraction levels less differences you see.
> Everything is a bunch of transistors...

In the Get procedure from your last post, you don't seem to make that
much difference between a binary byte and a Character. I would seem
Ada Strings are also very similar to byte sequences/arrays.

> > Now if we agree that simplicity is a
> > desirable quality (because it leads to less bugs, more productivity,
> > etc), I still fail to see the issues of such a format.
>
> Programs in 386 Assembler are sufficiently more complex than programs in
> Ada. Simplicity of nature by no means implies simplicity of use.

Guess why I haven't written a single line in assembly during the last
8 years ;-)

> > Now regarding personal preferences about braces, I have to admit I'm
> > always shocked at the way so many people dismiss S-expressions on
> > first sight because of the LISP-looking parentheses.
>
> Do you mean LISP does not deserve its fame? (:-))

I honestly don't know enough about both LISP and its fame to mean
anything like that. I just meant that judging format only from its
relatively heavy use of parenthesis is about as silly as judging
skills of a person only from the amount of melanin in their skin.

> > My point is, most of my (currently non-OOP) code can be expressed as
> > well in an OOP style. When I defined a C structure along with a bunch
> > of functions that perform operations on it, I'm conceptually defining
> > a class and its methods, only expressed in a non-OOP language. I
> > sometimes put function pointers in the structure, to have a form of
> > dynamic dispatch or virtual methods. I occasionally even leave the
> > structure declared but undefined publicly, to hide internals (do call
> > that encapsulation?), along with functions that could well be called
> > accessors and mutators. In my opinion that doesn't count as OOP
> > because it doesn't use OOP-specific features like inheritance.
>
> I disagree because in my view this is all what OO is about. OO is not about
> the tools (OOPL), it is about the way of programming.

Then I guess you could say I'm twisting C into OO programming, though
I readily violate OOP principles when it significantly improves code
readability or simplicity (which I guess happens much more often in C
than in Ada).

> > And the reason why I started this thread is only to
> > know how to buffer into memory the arrays of octets, because I need
> > (in my usual use of S-expressions) to resolve the links between atoms
> > before I can know the type of atoms. So I need a way to delay the
> > typing, and in the meantime handle data as a generic byte sequence
> > whose only known information is its size and its place in the S-
> > expression tree. What exactly is so bad with that approach?
>
> Nothing wrong when at the implementation level. However I don't see why
> links need to be resolved first. In comparable cases - I do much messy
> protocol/communication stuff - I usually first restore objects and then
> resolve links.

That's because some atom types are only known after having examined
other atoms. I you remember my example (tcp-connect (host foo.example)
(port 80)), here is how would it be interpreted: from the context or
initial state, we expect a list beginning with a atom which is a
string describing what to with whatever is after. "tcp-connect" is
therefore interpreted as a string, from the string value we know the
following is a list of settings, each of them being a list whose first
element is a atom which is a string describing the particular setting.
"host" is therefore a string, as its value tells us the following
atoms are also strings, describing host names to connect to, in
decreasing priority order. There "foo.example" is a string to be
resolve into a network address. "port" is also a string, and from its
value we know it's followed by atom being the decimal representation
of a port number, which in Ada would probably be a type on its own
(probably Integer mod 2**16 or something like that).

Of course, all those "we know" is actually knowledge derived from the
configuration file specification.

In this particular example, atoms are treated in the order in which
they appear in the byte stream, so there is already enough context to
know the type of an atom before reading it. This is not always the
case, for example it might be necessary to build an associative array
from a list of list before being able to know the type of non-head
atoms, or the S-expression might have to be kept uninterpreted (and
thus untyped) before some other run-time actions are performed (this
is quite common in the template system, where the template and the
data can change independently, and both changes induce a S-expression
re-interpretation).

Is it clearer now?


Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 8 Aug 2010 05:23:37 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 7, 4:23�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> On Sat, 7 Aug 2010 05:56:50 -0700 (PDT), Natacha Kerensikova wrote:
>>> Can we at least agree on the fact that a sequence of bytes is a
>>> general-purpose format, widely used for storing and transmitting data?
>>> (this question is just a matter of vocabulary)
>>
>> I don't think so. Namely it don't think that "general" is a synonym to
>> "completeness." It is rather about the abstraction level under the
>> condition of completeness.
> 
> Well, then I'm afraid I can discuss anymore, because I fail to
> understand your definition here.
> 
> I was using "general-purpose" as the opposite of "specific-purpose".
> If we make the parallel with compression schemes, FLAC sure is as
> complete as bzip2, yet the first one has a specific purpose
> (compressing sounds) while the other is general-purpose. So back to
> data format, I made the distinction in the amount preliminary
> assumptions about data to be contained. In that sense the raw byte-
> sequence is the most general format in that there is no assumption
> about the contained data (except that its number of bits is a multiple
> of the number of bits per byte).

And how are you going to make any assumptions at the level of raw bytes?
For a sequence of bytes to become sound you need to move many abstraction
layers - and OSI layers - up.

>>> However from a purely practical point of view, and using the fact that
>>> in my background languages (C and 386 asm) bytes sequences and strings
>>> are so similar, these crude semantics are all I need (or at least, all
>>> I've ever needed so far).
>>
>> Lower you descend down the abstraction levels less differences you see.
>> Everything is a bunch of transistors...
> 
> In the Get procedure from your last post, you don't seem to make that
> much difference between a binary byte and a Character.

No I do. But you have defined it as a text file. A streamed text file is a
sequence of Character items.

> I would seem
> Ada Strings are also very similar to byte sequences/arrays.

I remember a machine where char was 32-bit long.

Byte, octet, character are three different things (and code point is a
fourth).

> I just meant that judging format only from its
> relatively heavy use of parenthesis is about as silly as judging
> skills of a person only from the amount of melanin in their skin.

The amount of melanin is unrelated to the virtues we count in human beings.
An excessive need in indistinguishable brackets would definitely reduce
readability.

>>> And the reason why I started this thread is only to
>>> know how to buffer into memory the arrays of octets, because I need
>>> (in my usual use of S-expressions) to resolve the links between atoms
>>> before I can know the type of atoms. So I need a way to delay the
>>> typing, and in the meantime handle data as a generic byte sequence
>>> whose only known information is its size and its place in the S-
>>> expression tree. What exactly is so bad with that approach?
>>
>> Nothing wrong when at the implementation level. However I don't see why
>> links need to be resolved first. In comparable cases - I do much messy
>> protocol/communication stuff - I usually first restore objects and then
>> resolve links.
> 
> That's because some atom types are only known after having examined
> other atoms. I you remember my example (tcp-connect (host foo.example)
> (port 80)), here is how would it be interpreted: from the context or
> initial state, we expect a list beginning with a atom which is a
> string describing what to with whatever is after. "tcp-connect" is
> therefore interpreted as a string, from the string value we know the
> following is a list of settings,

Once you matched "tcp-connect", you know all the types of the following
components.

> This is not always the
> case, for example it might be necessary to build an associative array
> from a list of list before being able to know the type of non-head
> atoms,

What for? Even if such cases might be invented, I see no reason to do that.
It is difficult to parse, it is difficult to read. So why to mess with?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 8, 3:01 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> And how are you going to make any assumptions at the level of raw bytes?

I'm not, hence the byte-sequence being general-purpose using my
definition. (However one could split hair by saying that at the level
of raw bytes you're making assumptions about the number and endianness
of bits in each byte.)

S-expressions on the other hand are slightly less general-purpose in
that they contain the assumption that data is organized on the leaves
of a binary tree.

The more specialized the format, the more assumptions on the contained
data. Right?

> > In the Get procedure from your last post, you don't seem to make that
> > much difference between a binary byte and a Character.
>
> No I do. But you have defined it as a text file. A streamed text file is a
> sequence of Character items.

Actually, I didn't. I only defined it as a bunch of byte sequences
organized in a certain way.

The fact I usually choose a text representation of S-expressions is
purely a personal choice (motivated by the power of existing text
utilities like grep and sed), but I've never written a S-expression
library assuming it will deal with texts. The canonical form of a S-
expression, where atoms are embedded verbatim, is as binary as one can
get.

> > I would seem
> > Ada Strings are also very similar to byte sequences/arrays.
>
> I remember a machine where char was 32-bit long.

I've often wanted to get access to one of those PDP with 9-bit bytes,
just to further check my C programs.

> Byte, octet, character are three different things (and code point is a
> fourth).

I know very well these differences, except octet vs character,
especially considering Ada's definition of a Character. Or is it only
that Character is an enumeration while octet is a modular integer?

This leads to a question I had in mind since quite early in the
thread, should I really use an array of Storage_Element, while S-
expression standard considers only sequences of octets?

> > I just meant that judging format only from its
> > relatively heavy use of parenthesis is about as silly as judging
> > skills of a person only from the amount of melanin in their skin.
>
> The amount of melanin is unrelated to the virtues we count in human beings.
> An excessive need in indistinguishable brackets would definitely reduce
> readability.

Of course, this the same issue as curly brackets in C. My opinion
being that those brackets are not meant to be read by humans, only by
the compiler. Indentation is supposed to provide the same information
to humans while being ignored by the compiler. I apply the same rule
to S-expressions. Don't you think one should at least have a serious
look at a file before freaking out and calling it unreadable?

> > That's because some atom types are only known after having examined
> > other atoms. I you remember my example (tcp-connect (host foo.example)
> > (port 80)), here is how would it be interpreted: from the context or
> > initial state, we expect a list beginning with a atom which is a
> > string describing what to with whatever is after. "tcp-connect" is
> > therefore interpreted as a string, from the string value we know the
> > following is a list of settings,
>
> Once you matched "tcp-connect", you know all the types of the following
> components.

Unfortunately, you know "80" is a 16-bit integer only after having
matched "port".

> > This is not always the
> > case, for example it might be necessary to build an associative array
> > from a list of list before being able to know the type of non-head
> > atoms,
>
> What for? Even if such cases might be invented, I see no reason to do that.
> It is difficult to parse, it is difficult to read. So why to mess with?

For example, you might have a sub-S-expression describing a seldom
used object that is expensive to build, wouldn't you want to be sure
you actually need it before building it?


Thanks for the discussion,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 8 Aug 2010 06:49:09 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 8, 3:01�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:

> The more specialized the format, the more assumptions on the contained
> data. Right?

Yes if the specialization addresses the encoded entities. No if it does the
medium.

>>> In the Get procedure from your last post, you don't seem to make that
>>> much difference between a binary byte and a Character.
>>
>> No I do. But you have defined it as a text file. A streamed text file is a
>> sequence of Character items.
> 
> Actually, I didn't. I only defined it as a bunch of byte sequences
> organized in a certain way.

I see. This confuses things even more. Why should I represent anything as a
byte sequence? It already is, and in 90% cases I just don't care how the
compiler does that. Why to convert byte sequences into other sequences and
then into a text file. It just does not make sense to me. Any conversion
must be accompanied by moving the thing from one medium to another.
Otherwise it is wasting.

>> Byte, octet, character are three different things (and code point is a
>> fourth).
> 
> I know very well these differences, except octet vs character,
> especially considering Ada's definition of a Character. Or is it only
> that Character is an enumeration while octet is a modular integer?

The difference is that Character represents code points and octet does
atomic arrays of 8 bits.

> This leads to a question I had in mind since quite early in the
> thread, should I really use an array of Storage_Element, while S-
> expression standard considers only sequences of octets?

That depends on what are you going to do. Storage_Element is a
machine-dependent addressable memory unit. Octet is a machine independent
presentation layer unit, a thing of 256 independent states. Yes
incidentally Character has 256 code points.

> Don't you think one should at least have a serious
> look at a file before freaking out and calling it unreadable?

There are well-known things which do not require reconsidering. Curly or
round brackets aren't bad because of they form. They are because of
excessive overloading: the closing brackets of a loop, aggregate, block etc
are indistinguishable in C. Further in C you have brackets where there none
needed and don't have them where they should be. This do apply to LISP and
S-expressions.

>>> That's because some atom types are only known after having examined
>>> other atoms. I you remember my example (tcp-connect (host foo.example)
>>> (port 80)), here is how would it be interpreted: from the context or
>>> initial state, we expect a list beginning with a atom which is a
>>> string describing what to with whatever is after. "tcp-connect" is
>>> therefore interpreted as a string, from the string value we know the
>>> following is a list of settings,
>>
>> Once you matched "tcp-connect", you know all the types of the following
>> components.
> 
> Unfortunately, you know "80" is a 16-bit integer only after having
> matched "port".

Nope, we certainly know that each TCP connection needs a port. There is
nothing to resolve since the notation is not reverse. Parse it top down, it
is simple, it is safe, it allows excellent diagnostics, it works.

>>> This is not always the
>>> case, for example it might be necessary to build an associative array
>>> from a list of list before being able to know the type of non-head
>>> atoms,
>>
>> What for? Even if such cases might be invented, I see no reason to do that.
>> It is difficult to parse, it is difficult to read. So why to mess with?
> 
> For example, you might have a sub-S-expression describing a seldom
> used object that is expensive to build, wouldn't you want to be sure
> you actually need it before building it?

See above, if you parse top down, you know if you need that object before
begin. Then having a bracketed structure, it is trivial to skip the
object's description without construction. Just count brackets.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 8, 5:15 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Sun, 8 Aug 2010 06:49:09 -0700 (PDT), Natacha Kerensikova wrote:
> > On Aug 8, 3:01 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> >> No I do. But you have defined it as a text file. A streamed text file is a
> >> sequence of Character items.
>
> > Actually, I didn't. I only defined it as a bunch of byte sequences
> > organized in a certain way.
>
> I see. This confuses things even more. Why should I represent anything as a
> byte sequence? It already is, and in 90% cases I just don't care how the
> compiler does that. Why to convert byte sequences into other sequences and
> then into a text file. It just does not make sense to me. Any conversion
> must be accompanied by moving the thing from one medium to another.
> Otherwise it is wasting.

Representing something as a byte sequence is serialization (at least,
according to my (perhaps wrong) definition of serialization). Actually
there is no byte sequences converted into other sequences converted
into a text file. The only conversion is from in-memory representation
(which happens to be also a byte sequence, but maybe not contiguous or
context-dependent or whatever, that's besides the point) into a
serialized byte sequence.

S-expressions are not a format on top or below that, it's a format
*besides* that, at the same level. Objects are serialized into byte
sequences forming S-expression atoms, and relations between objects/
atoms are serialized by the S-expression format. This is how one get
the canonical representation of a S-expression.

Now depending on the situation one might want additional constrains on
the representation, for example human-readability or being text-based,
and the S-expression standard defines non-canonical representations
for such situations.

> > I know very well these differences, except octet vs character,
> > especially considering Ada's definition of a Character. Or is it only
> > that Character is an enumeration while octet is a modular integer?
>
> The difference is that Character represents code points and octet does
> atomic arrays of 8 bits.

Considering Ada's Character also spans over 8 bits (256-element
enumeration), both are equivalent, right? The only difference is the
intent and the meaning of values, right? (unlike byte vs octet, where
quantitative differences might exist on some platforms).

> > This leads to a question I had in mind since quite early in the
> > thread, should I really use an array of Storage_Element, while S-
> > expression standard considers only sequences of octets?
>
> That depends on what are you going to do. Storage_Element is a
> machine-dependent addressable memory unit. Octet is a machine independent
> presentation layer unit, a thing of 256 independent states. Yes
> incidentally Character has 256 code points.

Actually I've started to wonder whether Stream_Element might even more
appropriated: considering a S-expression atom is the serialization of
an object, and I guess objects which know how to serialize themselves
do so using the Stream subsystem, so maybe I could more easily
leverage existing serialization codes if I use Stream_Element_Array
for atoms. But then I don't know whether it's possible to have object
hand over a Stream_Element_Array representing themselves, and I don't
know either how to deal with cases where Stream_Element is not an
octet.

> >> Once you matched "tcp-connect", you know all the types of the following
> >> components.
>
> > Unfortunately, you know "80" is a 16-bit integer only after having
> > matched "port".
>
> Nope, we certainly know that each TCP connection needs a port. There is
> nothing to resolve since the notation is not reverse. Parse it top down, it
> is simple, it is safe, it allows excellent diagnostics, it works.

Consider:
(tcp-connect (host foo.example) (port 80))
and:
(tcp-connect (port 80) (host foo.example))

Both of these are semantically equivalent, but know which of the tail
atom is a 16-bit integer and which is the string, you have to first
match "port" and "host" head atoms.

Or am I misunderstanding your point?

> >>> This is not always the
> >>> case, for example it might be necessary to build an associative array
> >>> from a list of list before being able to know the type of non-head
> >>> atoms,
>
> >> What for? Even if such cases might be invented, I see no reason to do that.
> >> It is difficult to parse, it is difficult to read. So why to mess with?
>
> > For example, you might have a sub-S-expression describing a seldom
> > used object that is expensive to build, wouldn't you want to be sure
> > you actually need it before building it?
>
> See above, if you parse top down, you know if you need that object before
> begin. Then having a bracketed structure, it is trivial to skip the
> object's description without construction. Just count brackets.

Well in that example I was considering something outside from the S-
expression selects which object to use. For example a database
containing thousands of templates or whatever, and user selection
picking only one of them.


Thanks for your patience with me,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Mon, 9 Aug 2010 02:55:03 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 8, 5:15�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> On Sun, 8 Aug 2010 06:49:09 -0700 (PDT), Natacha Kerensikova wrote:
>>> On Aug 8, 3:01�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
>>>> No I do. But you have defined it as a text file. A streamed text file is a
>>>> sequence of Character items.
>>
>>> Actually, I didn't. I only defined it as a bunch of byte sequences
>>> organized in a certain way.
>>
>> I see. This confuses things even more. Why should I represent anything as a
>> byte sequence? It already is, and in 90% cases I just don't care how the
>> compiler does that. Why to convert byte sequences into other sequences and
>> then into a text file. It just does not make sense to me. Any conversion
>> must be accompanied by moving the thing from one medium to another.
>> Otherwise it is wasting.
> 
> Representing something as a byte sequence is serialization (at least,
> according to my (perhaps wrong) definition of serialization).

"Byte" here is a RAM unit or a disk file item? I meant the former. All
objects are already sequences of bytes in the RAM.

> S-expressions are not a format on top or below that, it's a format
> *besides* that, at the same level. Objects are serialized into byte
> sequences forming S-expression atoms, and relations between objects/
> atoms are serialized by the S-expression format. This is how one get
> the canonical representation of a S-expression.

I thought you wanted to represent *objects* ... as S-sequences?

Where these representations are supposed to live? In the RAM? Why are you
then talking about text files, configurations and humans reading something?
I cannot remember last time I read memory dump...

> Now depending on the situation one might want additional constrains on
> the representation, for example human-readability or being text-based,
> and the S-expression standard defines non-canonical representations
> for such situations.
> 
>>> I know very well these differences, except octet vs character,
>>> especially considering Ada's definition of a Character. Or is it only
>>> that Character is an enumeration while octet is a modular integer?
>>
>> The difference is that Character represents code points and octet does
>> atomic arrays of 8 bits.
> 
> Considering Ada's Character also spans over 8 bits (256-element
> enumeration), both are equivalent, right?

Equivalent defiled as? In terms of types they are not, because the types
are different. In terms of the relation "=" they are not either, because
"=" is not defined on the tuple Character x Unsigned_8 (or whatever).

> The only difference is the
> intent and the meaning of values, right?

Huh, there is *nothing* beyond the meaning (semantics).

>>> This leads to a question I had in mind since quite early in the
>>> thread, should I really use an array of Storage_Element, while S-
>>> expression standard considers only sequences of octets?
>>
>> That depends on what are you going to do. Storage_Element is a
>> machine-dependent addressable memory unit. Octet is a machine independent
>> presentation layer unit, a thing of 256 independent states. Yes
>> incidentally Character has 256 code points.
> 
> Actually I've started to wonder whether Stream_Element might even more
> appropriated: considering a S-expression atom is the serialization of
> an object, and I guess objects which know how to serialize themselves
> do so using the Stream subsystem, so maybe I could more easily
> leverage existing serialization codes if I use Stream_Element_Array
> for atoms.

Note that Stream_Element is machine-depended as well.

> But then I don't know whether it's possible to have object
> hand over a Stream_Element_Array representing themselves,

This does not make sense to me, it is mixing abstractions:
Stream_Element_Array is a representation of an object in a stream.
Storage_Array might be a representation of in the memory. These are two
different objects. You cannot consider them same, even if they shared
physically same memory (but they do not). The whole purpose of
serialization to a raw stream is conversion of a Storage_Array to
Stream_Element_Array. Deserialization is a backward conversion.

> and I don't
> know either how to deal with cases where Stream_Element is not an
> octet.

By not using Stream_Element_Array, obviously. You should use the encoding
you want to. That is all.

If the encoding is for a text file you have to read Characters, you don't
care about how they land into a Stream_Element_Array, it is not your
business, it is an implementation detail of the text stream. If the
encoding is about octets, you have to read them. You have to chose.

>>>> Once you matched "tcp-connect", you know all the types of the following
>>>> components.
>>
>>> Unfortunately, you know "80" is a 16-bit integer only after having
>>> matched "port".
>>
>> Nope, we certainly know that each TCP connection needs a port. There is
>> nothing to resolve since the notation is not reverse. Parse it top down, it
>> is simple, it is safe, it allows excellent diagnostics, it works.
> 
> Consider:
> (tcp-connect (host foo.example) (port 80))
> and:
> (tcp-connect (port 80) (host foo.example))
>
> Both of these are semantically equivalent, but know which of the tail
> atom is a 16-bit integer and which is the string, you have to first
> match "port" and "host" head atoms.

Sure
 
> Or am I misunderstanding your point?

The point is that you never meet 80 before knowing that this is a "port",
you never meet "port" before knowing it is of "tcp-connect". You always
know all types in advance. It is strictly top-down.
 
> Thanks for your patience with me,

You are welcome. I think from the responses of the people here you see that
the difference between Ada and C is much deeper than begin/end instead of
curly brackets. Ada does not let you through without a clear concept of
what are you going to do. Surely with some proficiency one could write
classical C programs in Ada, messing everything up. You could even create
buffer overflow in Ada. But it is difficult for a beginner...

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 9, 12:56 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Mon, 9 Aug 2010 02:55:03 -0700 (PDT), Natacha Kerensikova wrote:
> > On Aug 8, 5:15 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> > wrote:
> >> I see. This confuses things even more. Why should I represent anything as a
> >> byte sequence? It already is, and in 90% cases I just don't care how the
> >> compiler does that. Why to convert byte sequences into other sequences and
> >> then into a text file. It just does not make sense to me. Any conversion
> >> must be accompanied by moving the thing from one medium to another.
> >> Otherwise it is wasting.
>
> > Representing something as a byte sequence is serialization (at least,
> > according to my (perhaps wrong) definition of serialization).
>
> "Byte" here is a RAM unit or a disk file item? I meant the former. All
> objects are already sequences of bytes in the RAM.

I have to admit I tend to confuse these two kind of bytes. Considering
my C-bitching background, I call "byte" C's char, i.e. the smallest
memory unit addressable independently (from the language point of
view, I'm well aware modern i386 have memory bus much wider than 8
bits, and transfer whole chunks at one time, but the CPU is following
the "as-if" rule so it looks like 8-bits are addressed independently,
hence 8-bit chars on these platforms). My confusion is that I often
take for granted that disk files can be addressed the same way, while
I perfectly know it's not always the case (e.g. DS has a 16-bit GBA
bus and 8 bit operations have to load the whole 16 bit chunk, just
like operating on 4-bit nibbles on i386).

It must be because I've never thought of disk I/O as unbuffered, and
the buffer does follow RAM rules of addressing.

> > S-expressions are not a format on top or below that, it's a format
> > *besides* that, at the same level. Objects are serialized into byte
> > sequences forming S-expression atoms, and relations between objects/
> > atoms are serialized by the S-expression format. This is how one get
> > the canonical representation of a S-expression.
>
> I thought you wanted to represent *objects* ... as S-sequences?

It depends what you call object. Here again, my vocabulary might has
been tainted by C Standard. Take for example a record, I would call
each component an object, as well as the whole record itself. I guess
I would call "object" an area of memory meaning something for the
compiler (e.g. a byte in some multibyte internal representation of
anything is not an object, while the byte sequence in RAM used by the
compiler to know what an access variable accesses is an object).

A S-expression atom must be build from something, so from this
definition it comes that whatever is turned into a S-expression atom
is an object (Well one could make an atom from multiple object but I
don't think it's a good idea). So there are objects meant to end up in
a single S-expression atom.

There are indeed objects that I want to represent as more elaborate S-
expressions. For example I wouldn't store a record into a single atom,
I'd rather define a S-expression specification to represent it and
store only simple components into atoms. But that's purely a personal
choice, motivated by the preference for human-readable disk files.

> Where these representations are supposed to live? In the RAM? Why are you
> then talking about text files, configurations and humans reading something?
> I cannot remember last time I read memory dump...

Ultimately these representations are supposed to be stored on disk or
transferred over a network or things like that, hence the need to be
built from serialized representations of objects.

However there is processing I might want to perform that require them
to be buffered in memory. So they temporary live in RAM until they are
stored or sent.

Text file, configurations and human reading somethings are details of
some applications of S-expressions, that might or might not be
relevant in a discussion about the deep nature of what a S-expression
is. But they have to be taken into consideration when a S-expression
library is written, lest the library turn out unsuitable for these
applications.

> >> The difference is that Character represents code points and octet does
> >> atomic arrays of 8 bits.
>
> > Considering Ada's Character also spans over 8 bits (256-element
> > enumeration), both are equivalent, right?
>
> Equivalent defiled as? In terms of types they are not, because the types
> are different. In terms of the relation "=" they are not either, because
> "=" is not defined on the tuple Character x Unsigned_8 (or whatever).

Sorry, "equivalent" in the mathematical that there is a bijection
between the set of Characters and the set of Octets, which allows to
use any of them to represent the other. Agreed, this a very week
equivalence, it just means there are exactly as many octet values as
Character values.

On the other hand, Storage_Element and Character are not bijection-
equivalent because there is no guarantee they will always have the
same number of values, even though they often do.

> > Actually I've started to wonder whether Stream_Element might even more
> > appropriated: considering a S-expression atom is the serialization of
> > an object, and I guess objects which know how to serialize themselves
> > do so using the Stream subsystem, so maybe I could more easily
> > leverage existing serialization codes if I use Stream_Element_Array
> > for atoms.
>
> Note that Stream_Element is machine-depended as well.

I'm sadly aware of that. I need an octet-sequence to follow the S-
expression standard, and there is here an implementation trade-off:
assuming objects already know how to serialize themselves into a
Stream_Element_Array, I can either code a converter from
Stream_Element_Array to octet-sequence, or reinvent the wheel and code
a converter for each type directly into an octet-sequence. For some
strange reason I prefer by far the first possibility.

> > But then I don't know whether it's possible to have object
> > hand over a Stream_Element_Array representing themselves,
>
> This does not make sense to me, it is mixing abstractions:
> Stream_Element_Array is a representation of an object in a stream.
> Storage_Array might be a representation of in the memory. These are two
> different objects. You cannot consider them same, even if they shared
> physically same memory (but they do not). The whole purpose of
> serialization to a raw stream is conversion of a Storage_Array to
> Stream_Element_Array. Deserialization is a backward conversion.

I don't consider them the same, otherwise I wouldn't be pondering
about which one to use.

If it helps, you can think of S-expressions as a standardized way of
serializing some relations between objects. However the objects still
have to be serialized, and that's outside of the scope of S-
expressions. From what I understood, the existing serializations of
objects use Stream_Element_Array as a low-level type. So the natural
choice for serializing the relations seems to be taking the
Stream_Element_Array from each object, and hand over to the lower-
level I/O a unified Stream_Element_Array.

Does it make sense or am I completely missing something?

> The point is that you never meet 80 before knowing that this is a "port",
> you never meet "port" before knowing it is of "tcp-connect". You always
> know all types in advance. It is strictly top-down.

Right, in that simple example it the case. It is even quite often the
case, hence my thinking about a Sexp_Stream in another post, which
would allow S-expression I/O without having more than a single node at
the same time in memory.

But there are still situations where S-expression have to be stored in
memory. For examples the templates, where S-expressions represent a
kind of limited programming language that is re-interpreted for each
template extension. Either store the S-expression in memory or re-read
the file form disk (which doesn't work efficiently when there is more
than one template in a file).

> > Thanks for your patience with me,
>
> You are welcome. I think from the responses of the people here you see that
> the difference between Ada and C is much deeper than begin/end instead of
> curly brackets.

Of course, if there was no other difference I would have stayed in C
(I still prefer brackets rather than words).

> Ada does not let you through without a clear concept of
> what are you going to do. Surely with some proficiency one could write
> classical C programs in Ada, messing everything up. You could even create
> buffer overflow in Ada. But it is difficult for a beginner...

I'm not here to Cifiy Ada. C is undoubtedly the best language to do C-
styled stuff.

I felt a mismatch between the nature of C and my way of programming. I
got interested in Ada (rather than in any language in the plethora of
existing languages out there) because it seemed to have what I lacked
in C while still having what I like in C. The former being roughly the
emphasis on robustness, readability and quality; the latter being
roughly the low-level (as in "close to the hardware", at least close
enough not to rule out programming for embedded platforms and system
programming) and the performance. The previous phrase lacking a lot of
irrational elements that I don't manage to word and that can be
summarized as "personal taste".

Now I can't explain why your posts often make me feel Ada is
completely out of my tastes in programming languages, while other
people's posts often make me feel Ada is probably an improvement over
C in term of personal preferences. However I have the feeling I got
much more personal improvement from your posts (improvement still
meaningful even if I eventually drop Ada to come back to C), and I'm
grateful for that.


Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 10.08.10 10:56, Natacha Kerensikova wrote:

>> Note that Stream_Element is machine-depended as well.
> 
> I'm sadly aware of that. I need an octet-sequence to follow the S-
> expression standard, and there is here an implementation trade-off:
> assuming objects already know how to serialize themselves into a
> Stream_Element_Array, I can either code a converter from
> Stream_Element_Array to octet-sequence, or reinvent the wheel and code
> a converter for each type directly into an octet-sequence. For some
> strange reason I prefer by far the first possibility.

In any case, these small articles on data representation might
be worth reading:

http://www2.adacore.com/2008/03/03/gem-27/
http://www2.adacore.com/2008/03/17/gem-28/

Georg

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Tue, 10 Aug 2010 01:56:22 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 9, 12:56�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> On Mon, 9 Aug 2010 02:55:03 -0700 (PDT), Natacha Kerensikova wrote:
>>> On Aug 8, 5:15�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
>>> wrote:
>>> S-expressions are not a format on top or below that, it's a format
>>> *besides* that, at the same level. Objects are serialized into byte
>>> sequences forming S-expression atoms, and relations between objects/
>>> atoms are serialized by the S-expression format. This is how one get
>>> the canonical representation of a S-expression.
>>
>> I thought you wanted to represent *objects* ... as S-sequences?
> 
> It depends what you call object. Here again, my vocabulary might has
> been tainted by C Standard. Take for example a record, I would call
> each component an object, as well as the whole record itself.

That's OK. Using this definition S-sequence in the memory is an object.
Which was the question: what was wrong with the first object so that you
wanted another instead?

My first take was S-sequence used as an object presentation outside the
memory, because you used the word "format". Now it looks as a solution
before the problem. You seem going to convert objects to S-sequences *in*
the memory and then dump the result them into files. Is it so? What was the
problem then? Because it cannot work without a conversion between
S-sequence in the memory (object) and S-sequence in the file
(representation). Why do you need S-sequence in the memory, while dumping
objects directly into files as S-sequences (if you insist on having them)
is simpler, cleaner, thinner, faster.

>>>> The difference is that Character represents code points and octet does
>>>> atomic arrays of 8 bits.
>>
>>> Considering Ada's Character also spans over 8 bits (256-element
>>> enumeration), both are equivalent, right?
>>
>> Equivalent defiled as? In terms of types they are not, because the types
>> are different. In terms of the relation "=" they are not either, because
>> "=" is not defined on the tuple Character x Unsigned_8 (or whatever).
> 
> Sorry, "equivalent" in the mathematical that there is a bijection
> between the set of Characters and the set of Octets, which allows to
> use any of them to represent the other. Agreed, this a very week
> equivalence, it just means there are exactly as many octet values as
> Character values.

No it is OK. Character can be used to represent octets. Ada provides means
for that, e.g.:

   type Octet is private; -- Interface
private
   type Octet is new Character; -- Implementation

> On the other hand, Storage_Element and Character are not bijection-
> equivalent because there is no guarantee they will always have the
> same number of values, even though they often do.

Yes.

>>> Actually I've started to wonder whether Stream_Element might even more
>>> appropriated: considering a S-expression atom is the serialization of
>>> an object, and I guess objects which know how to serialize themselves
>>> do so using the Stream subsystem, so maybe I could more easily
>>> leverage existing serialization codes if I use Stream_Element_Array
>>> for atoms.
>>
>> Note that Stream_Element is machine-depended as well.
> 
> I'm sadly aware of that. I need an octet-sequence to follow the S-
> expression standard, and there is here an implementation trade-off:
> assuming objects already know how to serialize themselves into a
> Stream_Element_Array, I can either code a converter from
> Stream_Element_Array to octet-sequence, or reinvent the wheel and code
> a converter for each type directly into an octet-sequence. For some
> strange reason I prefer by far the first possibility.

That depends on your goal. Streams are machine-dependent. Streams of octets
are not. If you want to exchange objects in the form of S-sequences across
the network you have to drop standard stream implementations of the objects
and replace them with your own, based on the stream octets. In this case
you will not use Stream_Element_Array directly. You will read and write
octets, by Octet'Read and Octet'Write. Provided that octet streams work,
which is about 99.9%, I guess. When they are not capable to handle octets
properly, you will have to implement I/O manually. If you wrap Octet'Read
into a function, you will be able to exchange the I/O layer without
affecting the upper ones. If we look at all this mechanics we will see the
old good OSI model.

> If it helps, you can think of S-expressions as a standardized way of
> serializing some relations between objects. However the objects still
> have to be serialized, and that's outside of the scope of S-
> expressions. From what I understood, the existing serializations of
> objects use Stream_Element_Array as a low-level type. So the natural
> choice for serializing the relations seems to be taking the
> Stream_Element_Array from each object, and hand over to the lower-
> level I/O a unified Stream_Element_Array.
> 
> Does it make sense or am I completely missing something?

In other post Jeffrey Carter described this as low-level. Why not to tell
the object: store yourself and all relations you need, I just don't care
which and how?

In fact I did something like that, persistent objects with dependencies
between them and collection of unreferenced objects. But I did it as
Jeffrey suggested. There is Put (Store, Object) the rest is hidden.

BUT, S-expression do not support references, they are strictly by-value, so
you don't need that stuff anyway.

>> The point is that you never meet 80 before knowing that this is a "port",
>> you never meet "port" before knowing it is of "tcp-connect". You always
>> know all types in advance. It is strictly top-down.
> 
> Right, in that simple example it the case. It is even quite often the
> case, hence my thinking about a Sexp_Stream in another post, which
> would allow S-expression I/O without having more than a single node at
> the same time in memory.
> 
> But there are still situations where S-expression have to be stored in
> memory.

There is no such cases!

> For examples the templates, where S-expressions represent a
> kind of limited programming language that is re-interpreted for each
> template extension.

I am not sure what do you mean here, but in general template is not the
object, its instance is. You do not need S-expressions here either. You can
store/restore templates as S-sequences. A template in the memory would be
an object with some operations like Instantiate_With_Parameters etc. The
result of instantiation will be again an object and no S-sequence.

BTW, for an interpreter I would certainly prefer the Reverse Polish
Notation to a tree. (I think this too boils down to a solution before the
problem.)

> the latter being
> roughly the low-level (as in "close to the hardware", at least close
> enough not to rule out programming for embedded platforms and system
> programming) and the performance.

(BTW, Ada is closer to the hardware than C is. You can even describe
interrupt handlers in Ada. Try it in C.)

> Now I can't explain why your posts often make me feel Ada is
> completely out of my tastes in programming languages,

In the way of programming you mean? I wanted to convey that a
"C-programmer" will have to change some habits when switching to Ada. Ada
enforces certain attitude to programming. Some people would say to software
engineering. It is not obvious because you can do in Ada anything you can
in C. But a stubborn hardcore "C-programmer" might become very frustrated
very soon. A competent C developer will only enjoy Ada.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 10, 12:36 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Tue, 10 Aug 2010 01:56:22 -0700 (PDT), Natacha Kerensikova wrote:
> > On Aug 9, 12:56 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> >> I thought you wanted to represent *objects* ... as S-sequences?
>
> > It depends what you call object. Here again, my vocabulary might has
> > been tainted by C Standard. Take for example a record, I would call
> > each component an object, as well as the whole record itself.
>
> That's OK. Using this definition S-sequence in the memory is an object.

Just to be sure, what is it exactly you call S-sequence? For the rest
of this post I will assume it synonym of S-expression atom, I hope my
answers won't be too misguided.

> Which was the question: what was wrong with the first object so that you
> wanted another instead?

The first object is the internal memory representation designed for
actual efficient use. For example, an integer will probably be
represented by its binary value with machine-defined endianness and
machine-defined size.

The other object is a "serialized" representation, in the sense that
it's designed for communication and storage, for example the same
integer, in a context where it will be sent over a network, can be for
example represented as an ASCII-encoded decimal number, or in binary
but with a predefined size and endianness. This is really the same
considerations as when storing or sending an object directly, except
that is has to reside in memory for a short time. There is no more
conversions or representations than when S-expression-lessly storing
or sending objects; the only difference is the memory buffering to
allow S-expression-specific information to be inserted around the
stream.

> My first take was S-sequence used as an object presentation outside the
> memory, because you used the word "format". Now it looks as a solution
> before the problem. You seem going to convert objects to S-sequences *in*
> the memory and then dump the result them into files. Is it so?

Yes, it is.

> What was the problem then?

The problem is to organize different objects inside a single file. S-
expression standardize the organization and relations between objects,
while something else has to be done beyond S-expression to turn
objects into representations suitable to live in a file.

Or the same thing with, instead of a file, an IPC socket or a network
socket or whatever, I just don't know how to call it generically
without resorting to a word derived from "serialize", but I hope you
get the point anyway.

> Because it cannot work without a conversion between
> S-sequence in the memory (object) and S-sequence in the file
> (representation).

The S-expression standard describe what conversions are allowed and in
what they consist. I cannot follow the standard without such a
conversion anyway, so either I do it or I drop the idea of S-
expressions, but then I don't have anything to portably store or
transmit  objects, so back to square one.

> Why do you need S-sequence in the memory, while dumping
> objects directly into files as S-sequences (if you insist on having them)
> is simpler, cleaner, thinner, faster.

Because I need to examine the S-sequence before writing it to disk, in
order to have enough information to write S-expression metadata. At
the very lest, I need to know the total size of the atom before
allowing its first byte to be send into the file.

> >> Note that Stream_Element is machine-depended as well.
>
> > I'm sadly aware of that. I need an octet-sequence to follow the S-
> > expression standard, and there is here an implementation trade-off:
> > assuming objects already know how to serialize themselves into a
> > Stream_Element_Array, I can either code a converter from
> > Stream_Element_Array to octet-sequence, or reinvent the wheel and code
> > a converter for each type directly into an octet-sequence. For some
> > strange reason I prefer by far the first possibility.
>
> That depends on your goal. Streams are machine-dependent. Streams of octets
> are not. If you want to exchange objects in the form of S-sequences across
> the network you have to drop standard stream implementations of the objects
> and replace them with your own, based on the stream octets.

This looks like a very strong point in favor of using an array-of-
octets to represent S-expression atoms.

> In this case
> you will not use Stream_Element_Array directly. You will read and write
> octets, by Octet'Read and Octet'Write. Provided that octet streams work,
> which is about 99.9%, I guess. When they are not capable to handle octets
> properly, you will have to implement I/O manually. If you wrap Octet'Read
> into a function, you will be able to exchange the I/O layer without
> affecting the upper ones. If we look at all this mechanics we will see the
> old good OSI model.

That sounds like a very nice way of doing it. So in the most common
case, there will still be a stream, provided by the platform-specific
socket facilities, which will accept an array-of-octets, and said
array would have to be created from objects by custom code, right?
(just want to be sure I understood correctly).

> In other post Jeffrey Carter described this as low-level. Why not to tell
> the object: store yourself and all relations you need, I just don't care
> which and how?

That's indeed a higher-level question. That's how it will happen at
some point in my code; however at some other point I will still have
to actually implement said object storage, and that's when I will
really care about which and how. I'm aware from the very beginning
that a S-expression library is low-level and is only used by mid-level
objects before reaching the application.

I've discussed with a friend of mine who has a better knowledge than
me about what actually is a parser and a lexer and things like that.
It happens that what my S-expression code is intended to be is
actually a partial parser, requiring some more specific stuff on top
on it to actually be called a parser. Just like S-expression is a
partial format in that it describes how to serialize relations between
atoms without describing how objects are serialized into atoms.

At some point in my projects I will have to write various
configuration file parsers, template parsers, and maybe a lot of other
parsers. They can all be treated as independent parsers, and
implemented with independent code, maybe derived from YACC or
something. I didn't choose that path, I chose rather to use what I
called a S-expression library, which is sort of a common core to all
these parsers, so I only have left to write the specific part of each
situations, which is matching the keywords and typing/deserializing
the objects.

> > But there are still situations where S-expression have to be stored in
> > memory.
>
> There is no such cases!

Right, I guess just like goto-use, they can always be avoided, but I'm
still not convinced it's always the best.

> > For examples the templates, where S-expressions represent a
> > kind of limited programming language that is re-interpreted for each
> > template extension.
>
> I am not sure what do you mean here, but in general template is not the
> object, its instance is.

Just to be sure we use the same words, I'm talking here about HTML
templates. The basic definition of a S-expression is a list of nodes,
each of which can be a list or an atom; for a template I copy directly
atom contents into the output, and lists are interpreted as functions,
with an "instance" (or whatever is called the thing containing data to
populate the template) handling the actual "execution" of the
function. Now the interest I find in the recursive definition of S-
expression is that it's easy to pass template-fragments as arguments
of these functions. For this it's much easier to let the S-expression
describing the template reside into memory. S-expressions are of
course not required here, but provide a nice and simple unified
format.

> You do not need S-expressions here either. You can
> store/restore templates as S-sequences. A template in the memory would be
> an object with some operations like Instantiate_With_Parameters etc. The
> result of instantiation will be again an object and no S-sequence.

Well how would solve the problem described above without S-
expressions? (That's a real question, if something simpler and/or more
efficient than my way of doing it exists, I'm genuinely interested.)

> > the latter being
> > roughly the low-level (as in "close to the hardware", at least close
> > enough not to rule out programming for embedded platforms and system
> > programming) and the performance.
>
> (BTW, Ada is closer to the hardware than C is. You can even describe
> interrupt handlers in Ada. Try it in C.)

Fantastic \o/

> > Now I can't explain why your posts often make me feel Ada is
> > completely out of my tastes in programming languages,
>
> In the way of programming you mean? I wanted to convey that a
> "C-programmer" will have to change some habits when switching to Ada.

I unsure about you mean here by "C-programmer". As I said in other
posts, I don't really code the same way as other C coders I've met. I
still don't know whether it's a good thing or not. I'm willing to
change some of my habits to switch to Ada, while I won't give up some
other even if mean giving up Ada. The main criterion being that coding
must remain fun, because it makes no sense to continue doing a leisure
activity that isn't fun anymore.

> But a stubborn hardcore "C-programmer" might become very frustrated
> very soon. A competent C developer will only enjoy Ada.

I somehow hope I'm more of the latter than the former.

BTW, that post of yours is one that encourages me rather than restrain
me.


Thanks for your interesting replies,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Tue, 10 Aug 2010 05:06:29 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 10, 12:36�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>>
>> Which was the question: what was wrong with the first object so that you
>> wanted another instead?
> 
> The first object is the internal memory representation designed for
> actual efficient use. For example, an integer will probably be
> represented by its binary value with machine-defined endianness and
> machine-defined size.
> 
> The other object is a "serialized" representation, in the sense that
> it's designed for communication and storage, for example the same
> integer, in a context where it will be sent over a network, can be for
> example represented as an ASCII-encoded decimal number, or in binary
> but with a predefined size and endianness.

Why don't you send it at once?

> This is really the same
> considerations as when storing or sending an object directly, except
> that is has to reside in memory for a short time. There is no more
> conversions or representations than when S-expression-lessly storing
> or sending objects; the only difference is the memory buffering to
> allow S-expression-specific information to be inserted around the
> stream.

This is impossible in general case, so the question why. As an example
consider a stateful communication protocol (existing in real life) which is
reacts only on changes. When you send your integer nothing happens because
the device reacts only when the bit pattern changes. So if you wanted to
really send it to another side you have to change something in the
representation of integer, e.g. to toggle some extra bit.

>> What was the problem then?
> 
> The problem is to organize different objects inside a single file. S-
> expression standardize the organization and relations between objects,
> while something else has to be done beyond S-expression to turn
> objects into representations suitable to live in a file.
> 
> Or the same thing with, instead of a file, an IPC socket or a network
> socket or whatever, I just don't know how to call it generically
> without resorting to a word derived from "serialize", but I hope you
> get the point anyway.

Yes, I don't see how S-expression might help there. They do not add value,
because the work of serialization or equivalent to serialization is already
done while construction of the expression object.

>> Because it cannot work without a conversion between
>> S-sequence in the memory (object) and S-sequence in the file
>> (representation).
> 
> The S-expression standard describe what conversions are allowed and in
> what they consist. I cannot follow the standard without such a
> conversion anyway, so either I do it or I drop the idea of S-
> expressions, but then I don't have anything to portably store or
> transmit  objects, so back to square one.

There are two questions to discuss:

1. The external storage format: S-expressions vs. other
2. Construction of an intermediate S-expression object in the memory

You justified #1 by an argument to legacy. You cannot [re-]use that
argument for #2. (OK, since Ludovic had already done it, you could (:-))

>> Why do you need S-sequence in the memory, while dumping
>> objects directly into files as S-sequences (if you insist on having them)
>> is simpler, cleaner, thinner, faster.
> 
> Because I need to examine the S-sequence before writing it to disk, in
> order to have enough information to write S-expression metadata. At
> the very lest, I need to know the total size of the atom before
> allowing its first byte to be send into the file.

That does not look like a stream! But this is again about abstraction
layers. Why do you care?

> That sounds like a very nice way of doing it. So in the most common
> case, there will still be a stream, provided by the platform-specific
> socket facilities, which will accept an array-of-octets, and said
> array would have to be created from objects by custom code, right?

Yes, if TCP sockets is what you use. There is a hell of other protocols
even on the Ethernet, some of which are not stream-oriented.

>> In other post Jeffrey Carter described this as low-level. Why not to tell
>> the object: store yourself and all relations you need, I just don't care
>> which and how?
> 
> That's indeed a higher-level question. That's how it will happen at
> some point in my code; however at some other point I will still have
> to actually implement said object storage, and that's when I will
> really care about which and how. I'm aware from the very beginning
> that a S-expression library is low-level and is only used by mid-level
> objects before reaching the application.

This is what caused the questions. Because if the problem is serialization,
then S-expression does not look good.

>> You do not need S-expressions here either. You can
>> store/restore templates as S-sequences. A template in the memory would be
>> an object with some operations like Instantiate_With_Parameters etc. The
>> result of instantiation will be again an object and no S-sequence.
> 
> Well how would solve the problem described above without S-
> expressions? (That's a real question, if something simpler and/or more
> efficient than my way of doing it exists, I'm genuinely interested.)

The PPN, a simple stack machine. Push arguments onto the stack, pop to
execute an operation. Push the results back. Repeat.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Simon Wright]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> Yes, if TCP sockets is what you use. There is a hell of other
> protocols even on the Ethernet, some of which are not stream-oriented.

We leverage stream i/o via UDP (which on GNAT is seriously broken,
because it tries to do an OS sendto() for each elementary component!
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9535) by using 'Write into a
memory stream[1] on one side and then sendto() the constructed
Stream_Element_Array; and on the other side recvfrom() to get the
datagram contents, use this as input to a memory stream, then 'Read. (Of
course we could use 'Output and 'Input, but the data types involved are
all definite).

[1]
http://booch95.svn.sourceforge.net/viewvc/booch95/trunk/src/bc-support-memory_streams.ads?revision=1420&view=markup
http://booch95.svn.sourceforge.net/viewvc/booch95/trunk/src/bc-support-memory_streams.adb?revision=1428&view=markup

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Tue, 10 Aug 2010 22:22:02 +0100, Simon Wright wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
>> Yes, if TCP sockets is what you use. There is a hell of other
>> protocols even on the Ethernet, some of which are not stream-oriented.
> 
> We leverage stream i/o via UDP (which on GNAT is seriously broken,
> because it tries to do an OS sendto() for each elementary component!

Wow! Even in the latest GNAT Pro/GPL?

I am asking because AdaCore fixed this very issue for String'Write, which
did Character'Write per element. Was it String-only fix?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Simon Wright]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> On Tue, 10 Aug 2010 22:22:02 +0100, Simon Wright wrote:
>
>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
>> 
>>> Yes, if TCP sockets is what you use. There is a hell of other
>>> protocols even on the Ethernet, some of which are not stream-oriented.
>> 
>> We leverage stream i/o via UDP (which on GNAT is seriously broken,
>> because it tries to do an OS sendto() for each elementary component!
>
> Wow! Even in the latest GNAT Pro/GPL?
>
> I am asking because AdaCore fixed this very issue for String'Write, which
> did Character'Write per element. Was it String-only fix?

There was an efficiency problem, where GNAT did as you say. I raised a
ticket but don't recall what happened about it (we are stuck with an
archaic GNAT, don't ask).

But I think you will still find that a String is still written as a
C_Sendto of an Integer (Item'First), another Integer (Item'Last) and
then (if this change has been made!) the String itself.

Checked in 6.2.2.

The best cure IMHO is to refuse to return a Stream for a Datagram_Socket.

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Wed, 11 Aug 2010 18:32:39 +0100, Simon Wright wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
>> On Tue, 10 Aug 2010 22:22:02 +0100, Simon Wright wrote:
>>
>>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
>>> 
>>>> Yes, if TCP sockets is what you use. There is a hell of other
>>>> protocols even on the Ethernet, some of which are not stream-oriented.
>>> 
>>> We leverage stream i/o via UDP (which on GNAT is seriously broken,
>>> because it tries to do an OS sendto() for each elementary component!
>>
>> Wow! Even in the latest GNAT Pro/GPL?
>>
>> I am asking because AdaCore fixed this very issue for String'Write, which
>> did Character'Write per element. Was it String-only fix?
> 
> There was an efficiency problem, where GNAT did as you say. I raised a
> ticket but don't recall what happened about it (we are stuck with an
> archaic GNAT, don't ask).
> 
> But I think you will still find that a String is still written as a
> C_Sendto of an Integer (Item'First), another Integer (Item'Last) and
> then (if this change has been made!) the String itself.
> 
> Checked in 6.2.2.
> 
> The best cure IMHO is to refuse to return a Stream for a Datagram_Socket.

We never used the stream interface with TCP sockets either. Socket stream
is certainly a nice-to-have feature, but, I don't know, we didn't even
considered to use it. Somehow it was out of question from the start. 

Otherwise, we would have noticed it, since we are doing a lot of heavy-duty
TCP and UDP stuff over GNAT.Sockets. In the beginning we a bit suspicious
and considered a possibility to use another socket library, but
GNAT.Sockets worked so well that we dropped the idea.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 10, 5:46 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Tue, 10 Aug 2010 05:06:29 -0700 (PDT), Natacha Kerensikova wrote:
> > The first object is the internal memory representation designed for
> > actual efficient use. For example, an integer will probably be
> > represented by its binary value with machine-defined endianness and
> > machine-defined size.
>
> > The other object is a "serialized" representation, in the sense that
> > it's designed for communication and storage, for example the same
> > integer, in a context where it will be sent over a network, can be for
> > example represented as an ASCII-encoded decimal number, or in binary
> > but with a predefined size and endianness.
>
> Why don't you send it at once?

As I said, I can't just insert the raw object in the stream, I need at
least to know its size. I might need further inspection of the
serialized representation in case I want a "smart" chose of atom
encoding, but I'm afraid it wasn't a good idea to mention that point
because it doesn't fit into the simplified concept of S-expression I
have been discussing with you for quite some posts.

However my proposed Sexp_Stream does send it as soon as it gets the
whole representation, and this idea comes from our discussion.

> > This is really the same
> > considerations as when storing or sending an object directly, except
> > that is has to reside in memory for a short time. There is no more
> > conversions or representations than when S-expression-lessly storing
> > or sending objects; the only difference is the memory buffering to
> > allow S-expression-specific information to be inserted around the
> > stream.
>
> This is impossible in general case, so the question why. As an example
> consider a stateful communication protocol (existing in real life) which is
> reacts only on changes. When you send your integer nothing happens because
> the device reacts only when the bit pattern changes. So if you wanted to
> really send it to another side you have to change something in the
> representation of integer, e.g. to toggle some extra bit.

Well obviously S-expressions aren't designed to be transmitted over
such a protocol. The basic assumption behind S-expression that we can
transmit/store/whatever octet sequences and receive/retrieve/whatever
them intact. When the assumption doesn't hold, either something must
be done to make it true (e.g. add another layer) or S-expressions must
be abandoned.

For example, S-expressions small enough to fit in one packet can be
easily transferred over UDP. S-expression parsers (or at least mine)
handle well fragmented data (even when unevenly fragmented) but fail
when data is missing or mis-ordered, which prevent large S-expressions
to be simply spread over as many packets as needed. However one might
solve this issue by adding a sequence number inside the UDP payload,
along with a mechanism to re-send lost packet; however that would be
(at least partially) re-inventing TCP.

> >> What was the problem then?
>
> > The problem is to organize different objects inside a single file. S-
> > expression standardize the organization and relations between objects,
> > while something else has to be done beyond S-expression to turn
> > objects into representations suitable to live in a file.
>
> > [...]
>
> Yes, I don't see how S-expression might help there. They do not add value,
> because the work of serialization or equivalent to serialization is already
> done while construction of the expression object.

There are two things that are to be serialized: objects into atoms,
and relations between objects into S-expression-specific stuff. The S-
expression object is an unserialized in-memory representation of
relations between serialized representations of objects. The writing
of an S-expression into a stream is the final serialization-of-
relations stage.

> There are two questions to discuss:
>
> 1. The external storage format: S-expressions vs. other
> 2. Construction of an intermediate S-expression object in the memory
>
> You justified #1 by an argument to legacy. You cannot [re-]use that
> argument for #2. (OK, since Ludovic had already done it, you could (:-))

I don't re-use that argument. And actually if you followed the
description of my Sexp_Stream, I don't need a S-expression object in
memory, I only need serialized representation of atoms. The rest can
be directly send into a stream.

And while I occasionally feel the need of an in-memory S-expression
object, so far it has never been for writing or sending, it was always
for specific sub-S-expression that are read or received. I believe
this need happens only when I have a variable of type S-expression,
which I consider to be as good a type as String or Natural. It is then
a data-structure choice, which happens at a higher level than
encoding, serialization, I/O or most of what we have discussed so far.

> >> Why do you need S-sequence in the memory, while dumping
> >> objects directly into files as S-sequences (if you insist on having them)
> >> is simpler, cleaner, thinner, faster.
>
> > Because I need to examine the S-sequence before writing it to disk, in
> > order to have enough information to write S-expression metadata. At
> > the very lest, I need to know the total size of the atom before
> > allowing its first byte to be send into the file.
>
> That does not look like a stream! But this is again about abstraction
> layers. Why do you care?

The "verbatim encoding" of an atom, which is the only one allowed in
canonical representation of a S-expression, is defined as follow: a
size, represented as the ASCII encoding of the decimal representation
of the number of octets in the atom, without leading zero (therefore
of variable length); followed by the ASCII character ':'; followed by
the octet sequence of the atom.

You can't write an atom using such a format when you don't know in
advance the number of octets in the atom.

The idea behind S-expressions could be seen as the serialization of a
list of serialized objects. When serializing such a list one much be
able to distinguish between the different objects; to the best of my
knowledge this can only be done either by keeping track of object
sizes, or by using separators. To prevent the restriction of possible
atom contents, the first solution has been chosen.

> > That sounds like a very nice way of doing it. So in the most common
> > case, there will still be a stream, provided by the platform-specific
> > socket facilities, which will accept an array-of-octets, and said
> > array would have to be created from objects by custom code, right?
>
> Yes, if TCP sockets is what you use. There is a hell of other protocols
> even on the Ethernet, some of which are not stream-oriented.

But you were talking about Octet'Read and Octet'Write. Aren't these
Ada Stream based stuff?

> >> In other post Jeffrey Carter described this as low-level. Why not to tell
> >> the object: store yourself and all relations you need, I just don't care
> >> which and how?
>
> > That's indeed a higher-level question. That's how it will happen at
> > some point in my code; however at some other point I will still have
> > to actually implement said object storage, and that's when I will
> > really care about which and how. I'm aware from the very beginning
> > that a S-expression library is low-level and is only used by mid-level
> > objects before reaching the application.
>
> This is what caused the questions. Because if the problem is serialization,
> then S-expression does not look good.

Why? Because it's a partial serialization? Because it serializes stuff
you deem as useless? Because it's a way of serializing stuff you would
have serialized in another way? I still don't understand what is so
bad with S-expressions. While I understand gut-rejection of anything-
with-parentheses (including lisp and S-expressions), you seem way
above that.

> >> You do not need S-expressions here either. You can
> >> store/restore templates as S-sequences. A template in the memory would be
> >> an object with some operations like Instantiate_With_Parameters etc. The
> >> result of instantiation will be again an object and no S-sequence.
>
> > Well how would solve the problem described above without S-
> > expressions? (That's a real question, if something simpler and/or more
> > efficient than my way of doing it exists, I'm genuinely interested.)
>
> The PPN, a simple stack machine. Push arguments onto the stack, pop to
> execute an operation. Push the results back. Repeat.

Does that allows to push an operation and its arguments, to have it
executed by another operation? S-expressions do it naturally, and I
find it very useful in conditional or loop constructs.



Thanks for the discussion,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Wed, 11 Aug 2010 02:43:58 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 10, 5:46�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:

>> Why don't you send it at once?
> 
> As I said, I can't just insert the raw object in the stream,

You can encode it.

> I need at least to know its size.

No, you don't and you cannot, because the object's size has nothing to do
to the object representation in the encoding used for the stream. Consider
two systems using common vocabulary. Instead of sending objects you do the
ids asking the partner to construct an object for this id.

> For example, S-expressions small enough to fit in one packet can be
> easily transferred over UDP. S-expression parsers (or at least mine)
> handle well fragmented data (even when unevenly fragmented) but fail
> when data is missing or mis-ordered, which prevent large S-expressions
> to be simply spread over as many packets as needed. However one might
> solve this issue by adding a sequence number inside the UDP payload,
> along with a mechanism to re-send lost packet; however that would be
> (at least partially) re-inventing TCP.

You can recode/pack S-expressions into something else. That does not answer
the question why. And from you responses it becomes less and less clear
where. What is the intended S-expression implementation according to the
OSI classification? We tried almost all levels, it looks pretty much
everything and nothing, a Ding-an-sich.
 
>>>> What was the problem then?
>>
>>> The problem is to organize different objects inside a single file. S-
>>> expression standardize the organization and relations between objects,
>>> while something else has to be done beyond S-expression to turn
>>> objects into representations suitable to live in a file.
>>
>>> [...]
>>
>> Yes, I don't see how S-expression might help there. They do not add value,
>> because the work of serialization or equivalent to serialization is already
>> done while construction of the expression object.
> 
> There are two things that are to be serialized: objects into atoms,
> and relations between objects into S-expression-specific stuff. The S-
> expression object is an unserialized in-memory representation of
> relations between serialized representations of objects. The writing
> of an S-expression into a stream is the final serialization-of-
> relations stage.

It is #2? (see below)

>> There are two questions to discuss:
>>
>> 1. The external storage format: S-expressions vs. other
>> 2. Construction of an intermediate S-expression object in the memory
>>
>> You justified #1 by an argument to legacy. You cannot [re-]use that
>> argument for #2. (OK, since Ludovic had already done it, you could (:-))
> 
> I don't re-use that argument. And actually if you followed the
> description of my Sexp_Stream, I don't need a S-expression object in
> memory, I only need serialized representation of atoms. The rest can
> be directly send into a stream.

It is not #2, only #1?

>>>> Why do you need S-sequence in the memory, while dumping
>>>> objects directly into files as S-sequences (if you insist on having them)
>>>> is simpler, cleaner, thinner, faster.
>>
>>> Because I need to examine the S-sequence before writing it to disk, in
>>> order to have enough information to write S-expression metadata. At
>>> the very lest, I need to know the total size of the atom before
>>> allowing its first byte to be send into the file.
>>
>> That does not look like a stream! But this is again about abstraction
>> layers. Why do you care?
> 
> The "verbatim encoding" of an atom, which is the only one allowed in
> canonical representation of a S-expression, is defined as follow: a
> size, represented as the ASCII encoding of the decimal representation
> of the number of octets in the atom, without leading zero (therefore
> of variable length); followed by the ASCII character ':'; followed by
> the octet sequence of the atom.

And TCP requires checksums, port numbers, destination address etc. Do you
care of them writing a socket?

>>> That sounds like a very nice way of doing it. So in the most common
>>> case, there will still be a stream, provided by the platform-specific
>>> socket facilities, which will accept an array-of-octets, and said
>>> array would have to be created from objects by custom code, right?
>>
>> Yes, if TCP sockets is what you use. There is a hell of other protocols
>> even on the Ethernet, some of which are not stream-oriented.
> 
> But you were talking about Octet'Read and Octet'Write. Aren't these
> Ada Stream based stuff?

It is a stream interface. You can hang this interface on something that is
not a stream, e.g. a text file. (We are hobbling in circles because of
mixed abstractions.)

>> This is what caused the questions. Because if the problem is serialization,
>> then S-expression does not look good.
> 
> Why? Because it's a partial serialization?

Yes

> Because it serializes stuff you deem as useless?

Yes, it does a structure of nested brackets. It does not serialize objects,
you need a lot of meat on the carcass. So if the problem is serialization
(of objects), then S-expression yet is not a solution.

>> The PPN, a simple stack machine. Push arguments onto the stack, pop to
>> execute an operation. Push the results back. Repeat.
> 
> Does that allows to push an operation and its arguments, to have it
> executed by another operation?

Operation is executed, no matter by what. PPN is executable. Whatever you
want to be executed has to be expressed in PPN. The PPN itself can be
pushed onto the stack if you wanted (=a von Neumann machine). I don't know
why would you need that stuff (~trampoline), however.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 11, 12:37 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Wed, 11 Aug 2010 02:43:58 -0700 (PDT), Natacha Kerensikova wrote:
> > On Aug 10, 5:46 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> > wrote:
> >> Why don't you send it at once?
>
> > As I said, I can't just insert the raw object in the stream,
>
> You can encode it.

Indeed, but to encode anything I need to access it, i.e. having it
live in the RAM. That's exactly what my in-memory serialized are for.

> > I need at least to know its size.
>
> No, you don't and you cannot, because the object's size has nothing to do
> to the object representation in the encoding used for the stream. Consider
> two systems using common vocabulary. Instead of sending objects you do the
> ids asking the partner to construct an object for this id.

I meant the size of the serialized representation. I can't store or
transmit something when I don't know the size of what is to be stored
or to be transmitted. If you want to transfer id instead of objects,
fine, then the id is the transmitted object and I have to know its
size. Using id is just moving the problem without solving it.

>  What is the intended S-expression implementation according to the
> OSI classification?

The primary intended use of this implementation is disk file I/O,
which doesn't fit well into the OSI classification.

When I used its C counterpart over a network, it has always been over
stream sockets (UNIX stream sockets or TCP sockets), so it's clearly
above the transport layer. I've never been able to figure out what is
what on the level 5 and above, but that's where my implementation
would be.

Now real-life S-expressions are used as a part of SPKI and IMAP. I
think IMAP is on level 7, but I'm not that tells us where IMAP S-
expressions are; and I haven't found enough information about SPKI to
know where it is, but from its purpose I guess it's probably also on
level 5 or above.

I don't know how this information helps you though.

> >>>> What was the problem then?
>
> >>> The problem is to organize different objects inside a single file. S-
> >>> expression standardize the organization and relations between objects,
> >>> while something else has to be done beyond S-expression to turn
> >>> objects into representations suitable to live in a file.
>
> >>> [...]
>
> >> Yes, I don't see how S-expression might help there. They do not add value,
> >> because the work of serialization or equivalent to serialization is already
> >> done while construction of the expression object.
>
> > There are two things that are to be serialized: objects into atoms,
> > and relations between objects into S-expression-specific stuff. The S-
> > expression object is an unserialized in-memory representation of
> > relations between serialized representations of objects. The writing
> > of an S-expression into a stream is the final serialization-of-
> > relations stage.
>
> It is #2? (see below)

When we are talking about a S-expression object, considering I
understand the word "object" as something that lives in RAM, then yes
we are in #2.

But this isn't required: my proposed Sexp_Stream never actually
construct a S-expression object, it serializes S-expression stuff
directly into the underlying stream, around the provided
serializations that are atoms.

> >> There are two questions to discuss:
>
> >> 1. The external storage format: S-expressions vs. other
> >> 2. Construction of an intermediate S-expression object in the memory
>
> >> You justified #1 by an argument to legacy. You cannot [re-]use that
> >> argument for #2. (OK, since Ludovic had already done it, you could (:-))
>
> > I don't re-use that argument. And actually if you followed the
> > description of my Sexp_Stream, I don't need a S-expression object in
> > memory, I only need serialized representation of atoms. The rest can
> > be directly send into a stream.
>
> It is not #2, only #1?

#1 using S-expression storage format is a choice driven by existing
data being in that format and personal taste.

#2 constructing a S-expression object in memory, is something that may
or may not be needed, depending on the application. Parsing a
configuration like my tcp-connect example does not, hence my
proposition of Sexp_Stream which functions without S-expression
objects in memory. My S-expression templates still seem (to me) to
require a S-expression object in memory (to be handed around to the
correct handler after dispatch).

When I proposed my Sexp_Stream, I postponed the design of a S-
expression object library, because it would be built upon the
Sexp_Stream which isn't fixed yet. And depending on Ludovic Brenta's
progress, I might never design my S-expression object and re-use his.

> >>>> Why do you need S-sequence in the memory, while dumping
> >>>> objects directly into files as S-sequences (if you insist on having them)
> >>>> is simpler, cleaner, thinner, faster.
>
> >>> Because I need to examine the S-sequence before writing it to disk, in
> >>> order to have enough information to write S-expression metadata. At
> >>> the very lest, I need to know the total size of the atom before
> >>> allowing its first byte to be send into the file.
>
> >> That does not look like a stream! But this is again about abstraction
> >> layers. Why do you care?
>
> > The "verbatim encoding" of an atom, which is the only one allowed in
> > canonical representation of a S-expression, is defined as follow: a
> > size, represented as the ASCII encoding of the decimal representation
> > of the number of octets in the atom, without leading zero (therefore
> > of variable length); followed by the ASCII character ':'; followed by
> > the octet sequence of the atom.
>
> And TCP requires checksums, port numbers, destination address etc. Do you
> care of them writing a socket?

I would if I were implementing a TCP library based on something of
lower OSI-level.

Now I'm writing a S-expression library based on TCP sockets (or disk
streams), so I don't care about TCP details. However I do care about S-
expression details, among which is the "verbatim encoding".

> >>> That sounds like a very nice way of doing it. So in the most common
> >>> case, there will still be a stream, provided by the platform-specific
> >>> socket facilities, which will accept an array-of-octets, and said
> >>> array would have to be created from objects by custom code, right?
>
> >> Yes, if TCP sockets is what you use. There is a hell of other protocols
> >> even on the Ethernet, some of which are not stream-oriented.
>
> > But you were talking about Octet'Read and Octet'Write. Aren't these
> > Ada Stream based stuff?
>
> It is a stream interface. You can hang this interface on something that is
> not a stream, e.g. a text file. (We are hobbling in circles because of
> mixed abstractions.)

And what do you deduce from this?

In the fourth level of quote above, I'm basically asking whether I
correctly understood your idea by reformulating it and asking for
confirmation ("…, right?").

Third level of quote, you seem to agree with a restriction.

Second level of quote, I try to understand the restriction.

And now I still don't know whether the idea in the fourth quote level
is right or wrong, and if it's wrong what it needs to be set right.
That's indeed a sad circle, and I don't even have a clue of what mixed
abstractions have to do with the circle. That's almost the point where
I stop trying to understand and just give up everything, Ada looks way
above my brain's capabilities.

> >> This is what caused the questions. Because if the problem is serialization,
> >> then S-expression does not look good.
>
> > Why? Because it's a partial serialization?
>
> Yes
>
> > Because it serializes stuff you deem as useless?
>
> Yes, it does a structure of nested brackets. It does not serialize objects,
> you need a lot of meat on the carcass. So if the problem is serialization
> (of objects), then S-expression yet is not a solution.

Indeed, because S-expressions assume the serialization of objects
(into atoms) is already solved, and address the problem of serializing
a higher level of information about these objects.

> Operation is executed, no matter by what. PPN is executable. Whatever you
> want to be executed has to be expressed in PPN. The PPN itself can be
> pushed onto the stack if you wanted (=a von Neumann machine). I don't know
> why would you need that stuff (~trampoline), however.

I still haven't found what PPN is, but from your description it looks
similar to my S-expression with semantics provided by an interpreter.

What I need can be basically examplified as the following S-expression
being sent to a list-of-object objects:
("<h1>Contents of " (title) " list</h1>"
    (for-each-item "<h2>" (title) "</h2><p>" (text) "</p>"))

In the first list line, atoms (double-quoted) are taken as-is into the
output HTML stream. (title) is a call to the list's "title" function,
which is substituted by that particular list's title. Then the "for-
each-item" function is called, which sends its arguments to each of
item of the list. Here again, following atoms are copied as-is, and
this time (title) is interpreted by item objects and substituted by
the item's title, and then the item's text substitutes (text).

That doesn't sound like an alien use of operations being passed as an
argument to another operation.



Regards,
Natacha

Re: S-expression I/O in Ada

[Robert A Duff]

Natacha Kerensikova <lithiumcat@gmail.com> writes:

> That's almost the point where
> I stop trying to understand and just give up everything, Ada looks way
> above my brain's capabilities.

Most of this discussion is about the design of a program that
might or might not use S-expressions.  It's got little or nothing
to do with Ada!

If your brain can understand the complexities of C (when do
arrays "decay" into pointers?  how do you typedef an array
of pointers to functions returning pointers to functions
returning void?  what's a "sequence point"?), then I assure
you it can deal with Ada.

- Bob

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 11, 2:58 pm, Robert A Duff <bobd...@shell01.TheWorld.com>
wrote:
> If your brain can understand the complexities of C (when do
> arrays "decay" into pointers?

Always, except when they're an argument of sizeof, unary &, or a
string literal used to initialize an array (C99 §6.3.2.1.3). Though I
have to admit from the top of my head I only remember sizeof and that
there are three of them, after think a bit I remembered unary &, and I
had to look up the Standard to find out the thrid one (even though I
use fluently this feature when I needed it).

> how do you typedef an array
> of pointers to functions returning pointers to functions
> returning void?

typedef void (*voidfn)(arguments unspecified above);
typedef voidfn (*returning_voidfn)(other arguments unspecified above);
typedef returning_voidfn wanted_type[SIZE_UNSPECIFIED];

Now of course this is probably less impressive than the much-less-
readable version doing everything in one statement:

typedef void (*(*wanted_type[SIZE])(void))(void);

> what's a "sequence point"?

Basically, points in the program flow between which whatever should
happen happens in an undefined order. They happen roughly at commas
(expression separator), semicolons (statement separators) and lazy
logical operators.

> ), then I assure you it can deal with Ada.

But what's the point of knowing all this when one can't design a three-
digit LOC program properly?
It's just like knowing the dictionary contents by heart without
knowing anything about grammar.



Thanks for your time,
Natacha

Re: S-expression I/O in Ada

[Randy Brukardt]

"Natacha Kerensikova" <lithiumcat@gmail.com> wrote in message 
news:d1c9014b-c78c-4cb5-ae34-506f8e5702c7@i13g2000yqd.googlegroups.com...
...
>> ), then I assure you it can deal with Ada.
>
>But what's the point of knowing all this when one can't design a three-
>digit LOC program properly?
>It's just like knowing the dictionary contents by heart without
>knowing anything about grammar.

I think Bob's point is that most of this discussion is about software 
engineering and program decomposition. That's orthogonal to the 
implementation language: your programs would be better in any language, be 
it C, Ada, Fortran, Pascal, or anything else when you (or anyone) take those 
concepts into account.

But there is no *requirement* to take those concepts into account in order 
to write Ada code. There is no requirement that you have to separate the 
creation and writing of atoms. Your code would be better in *any* language 
if you did, but it isn't always obvious how to accomplish that, and 
depending on the situation, it might not be worth the mental effort to do 
it.

In any case, you are spending way too much effort trying to figure out how 
to fit your work into what Dmitry is suggesting. Bob and I have both tried 
to point out that there are lots of other ways to structure programs over 
what Dmitry is suggesting. Don't let one person's opinions push you away 
from Ada!! There are a lot of strong opinions here, and not all of them are 
the same. There is no one right way to use Ada!

                      Randy Brukardt, Editor, Ada 2005 and Ada 2012 
Standards.

Re: S-expression I/O in Ada

[Robert A Duff]

"Randy Brukardt" <randy@rrsoftware.com> writes:

> I think Bob's point is that most of this discussion is about software 
> engineering and program decomposition.

Thank you, Randy.  Yes, exactly.

I don't think Natacha Kerensikova should be frightened away from Ada
just because of this discussion.  I don't know if S-expressions with
untyped octet-sequence atoms are appropriate for some application
or other, but if they are, Natacha's design seems reasonable to me.
And that opinion is not specific to Ada.

- Bob

Re: S-expression I/O in Ada

[Natacha Porté]

On Aug 12, 1:39 am, "Randy Brukardt" <ra...@rrsoftware.com> wrote:
> But there is no *requirement* to take those concepts into account in order
> to write Ada code. There is no requirement that you have to separate the
> creation and writing of atoms.

That's exactly to address these point that I (tried to) imagined
Sexp_Stream; that was a while back and I haven't felt any progress
between before and after this. It's very depressing to feel there is
something here without being able to grasp it.

> Your code would be better in *any* language
> if you did, but it isn't always obvious how to accomplish that, and
> depending on the situation, it might not be worth the mental effort to do
> it.

I have to admit I was quite surprised to find my old way of using S-
expressions to be so extremely more difficult in Ada compared to C
(whaddoyoumean I can't just feed to strcmp() raw data I've just read
from a file!?). But on other hand it might be a good thing to be
shaken into realizing the deep implication from the difference in
typing strength.

> In any case, you are spending way too much effort trying to figure out how
> to fit your work into what Dmitry is suggesting.

That's because I feel there is something there that might help a lot
in my personal improvement. Even I end up not using his point of view,
for any rational or non-rational reason, I'm sure there is something
to get from understanding it.

>  Bob and I have both tried
> to point out that there are lots of other ways to structure programs over
> what Dmitry is suggesting.

Thanks a lot to both of your for that.

> Don't let one person's opinions push you away from Ada!!

Indeed, I have to admit I overreacted. It looks like I'm still
vulnerable to unrelated personal problems overflow, I'll have to find
a way to patch that.



Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 8/12/10 10:53 AM, Natacha Port� wrote:

> I have to admit I was quite surprised to find my old way of using S-
> expressions to be so extremely more difficult in Ada compared to C
> (whaddoyoumean I can't just feed to strcmp() raw data I've just read
> from a file!?).

You can "=" just those bytes: you'll just have to drop type
checking by using an instance of Unchecked_Conversion of
the-bytes-you-read to String.

> But on other hand it might be a good thing to be
> shaken into realizing the deep implication from the difference in
> typing strength.

The 12K SLOC Python programs on my screen surely would
be a little easier to rewrite if Python had explicit typing.
assert isinstance(this, that) seems a helpful workaround,
but tedious in comparison to strong typing during translation...

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 12, 11:22 am, Georg Bauhaus <rm-
host.bauh...@maps.futureapps.de> wrote:
> On 8/12/10 10:53 AM, Natacha Porté wrote:
>
> > I have to admit I was quite surprised to find my old way of using S-
> > expressions to be so extremely more difficult in Ada compared to C
> > (whaddoyoumean I can't just feed to strcmp() raw data I've just read
> > from a file!?).
>
> You can "=" just those bytes: you'll just have to drop type
> checking by using an instance of Unchecked_Conversion of
> the-bytes-you-read to String.

Isn't that more-than-a-bit un-Ada-ish?

But then again there might be simple
command_string = String(array_of_read_octets);
if command_string = "my-command" them

which would solve the problem.

And if said string converter/constructor doesn't exist, I guess the
same thing can be done with an ad-hoc String_From_Atom function. But
it would be nice if there was a way to make that operation a no-op in
machine code (at least on platforms where array-of-octets and String
share the same representation, which should be quite often considering
the definition of Character).

> > But on other hand it might be a good thing to be
> > shaken into realizing the deep implication from the difference in
> > typing strength.
>
> The 12K SLOC Python programs on my screen surely would
> be a little easier to rewrite if Python had explicit typing.
> assert isinstance(this, that) seems a helpful workaround,
> but tedious in comparison to strong typing during translation...

Explicit typing is what made me drop python. I remember very well the
last straw, when I spend a long time chasing around a bug which was
cased by a 1 given as a function argument, which was then passed to
another function and then another where it divided by 2. However the
context was completely floating point, so the 0 obtained was quite a
surprise compared to the expected 0.5. And even today I still think 1
and 2 are completely valid floating-point constants.


Natacha

Re: S-expression I/O in Ada

[Randy Brukardt]

"Natacha Kerensikova" <lithiumcat@gmail.com> wrote in message 
news:9db37b80-acbb-4c9f-a646-34f108f52985@v15g2000yqe.googlegroups.com...
...
>Now I can't explain why your posts [referring to Dmitry here - RLB]
>often make me feel Ada is
>completely out of my tastes in programming languages, while other
>people's posts often make me feel Ada is probably an improvement over
>C in term of personal preferences. However I have the feeling I got
>much more personal improvement from your posts (improvement still
>meaningful even if I eventually drop Ada to come back to C), and I'm
>grateful for that.

I suspect this is because Dmitry's worldview is ummm, unusual, for the lack 
of a better word. I suspect some of what he is describing is more confusing 
than helpful to a newcomer. As someone else pointed out, there are about as 
many worldviews as there are programmers. This newsgroup has a very diverse 
set of worldviews, and you will find that there rarely is agreement about 
anything much more complicated than the proper way to write "Ada". So it is 
important that you find a style that works for you, and don't read too much 
into anyone else's style.

OTOH, you'll also get suggestions on improving your style that most of us 
would agree with. *Everyone* can improve their style and learn something 
from others...

                   Randy.

Re: S-expression I/O in Ada

[Simon Wright]

Natacha Kerensikova <lithiumcat@gmail.com> writes:

> and I don't know either how to deal with cases where Stream_Element is
> not an octet.

By ignoring them, I think!

Re: S-expression I/O in Ada

[Robert A Duff]

Simon Wright <simon@pushface.org> writes:

> Natacha Kerensikova <lithiumcat@gmail.com> writes:
>
>> and I don't know either how to deal with cases where Stream_Element is
>> not an octet.
>
> By ignoring them, I think!

Are there any such cases in the real world?  There are machines
in the embedded world where Storage_Element is not 8 bits.

But any machine that's hooked up to any sort of network
is going to be talking Octets, I think.  Does anybody know of
counterexamples?

Perhaps Stream_Element should have been defined as exactly
8 bits.  Perhaps it should have been called "Octet".

- Bob

Re: S-expression I/O in Ada

[Randy Brukardt]

"Robert A Duff" <bobduff@shell01.TheWorld.com> wrote in message 
news:wccbp9bspzy.fsf@shell01.TheWorld.com...
> Simon Wright <simon@pushface.org> writes:
>
>> Natacha Kerensikova <lithiumcat@gmail.com> writes:
>>
>>> and I don't know either how to deal with cases where Stream_Element is
>>> not an octet.
>>
>> By ignoring them, I think!
>
> Are there any such cases in the real world?  There are machines
> in the embedded world where Storage_Element is not 8 bits.

Not sure if it still exists in the real world, but the compiler we did for 
the Unisys mainframes used Stream_Element'Size = 9. (The Unisys mainframes 
are 36-bit machines.) Stream_Element'Size = 8 would have made the code for 
handling arrays awful.

Similarly, Character'Size = 9 on that machine. While the upper bit was not 
used in the representation, this is surely not an Octet.

> But any machine that's hooked up to any sort of network
> is going to be talking Octets, I think.  Does anybody know of
> counterexamples?

The network interfaces on the Unisys machines required binary or text mode 
settings: in text mode, the extra bit was just dropped; in binary mode all 
of the bits were sent (not sure precisely how that was done, it was in the 
OS).

 > Perhaps Stream_Element should have been defined as exactly
> 8 bits.  Perhaps it should have been called "Octet".

That would have made a compiler for the Unisys machines impossible; it would 
have made streaming impossible. There is no sane way to put 36-bit values 
into Octets - the only way that would have worked would have been to use 
16-bits for every 9-bit byte.

Whether this is a significant consideration today (in 2010) is debatable, 
but it surely was a real consideration back in 1993-4. So Ada 95 could not 
have made this choice.

                                        Randy.

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/09/2010 05:51 PM, Randy Brukardt wrote:
>
> Not sure if it still exists in the real world, but the compiler we did for
> the Unisys mainframes used Stream_Element'Size = 9. (The Unisys mainframes
> are 36-bit machines.) Stream_Element'Size = 8 would have made the code for
> handling arrays awful.

What was Storage_Unit? 36?

-- 
Jeff Carter
"Run away! Run away!"
Monty Python and the Holy Grail
58

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Randy Brukardt]

"Jeffrey Carter" <spam.jrcarter.not@spam.not.acm.org> wrote in message 
news:i3q8av$2o94$1@adenine.netfront.net...
> On 08/09/2010 05:51 PM, Randy Brukardt wrote:
>>
>> Not sure if it still exists in the real world, but the compiler we did 
>> for
>> the Unisys mainframes used Stream_Element'Size = 9. (The Unisys 
>> mainframes
>> are 36-bit machines.) Stream_Element'Size = 8 would have made the code 
>> for
>> handling arrays awful.
>
> What was Storage_Unit? 36?

It was originally 36-bits (and really ought to have been 36), but that 
caused our front end to make strings with 36-bit characters. So ultimately, 
we changed it to 9-bits, and put strong alignment requirements on 18-bit and 
36-bit integers. That fit better with the existing compiler implementation. 
(The change took more than 6-months of work, however, and really delayed the 
completion of the compiler, probably a contributing factor to none of the 
intended customers ever buying it.)

                            Randy.

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/10/2010 02:36 PM, Randy Brukardt wrote:
>
> It was originally 36-bits (and really ought to have been 36), but that
> caused our front end to make strings with 36-bit characters. So ultimately,
> we changed it to 9-bits, and put strong alignment requirements on 18-bit and
> 36-bit integers. That fit better with the existing compiler implementation.
> (The change took more than 6-months of work, however, and really delayed the
> completion of the compiler, probably a contributing factor to none of the
> intended customers ever buying it.)

36-bit characters. Ouch. Reminds me of the advice in /Software Tools/ to use the 
R format for characters in FORTRAN via Ratfor; on the CDC machine I was using, 
that would have put each character in its own 60-bit word, so I did otherwise. 
(CDC used 6-bit characters.)

-- 
Jeff Carter
"Strange women lying in ponds distributing swords
is no basis for a system of government."
Monty Python & the Holy Grail
66

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Robert A Duff]

"Randy Brukardt" <randy@rrsoftware.com> writes:

> Not sure if it still exists in the real world, but the compiler we did for 
> the Unisys mainframes used Stream_Element'Size = 9.

Interesting.  Can these machines communicate with non-Unisys machines
over a regular TCP/IP network?  E.g. send an e-mail using standard
protocols, that can be read on a x86?

I assume Storage_Element'Size = 9, too.  Correct?

Next question: Is (was) there any Ada implementation where
Stream_Element'Size /= Storage_Element'Size?

>...(The Unisys mainframes 
> are 36-bit machines.) Stream_Element'Size = 8 would have made the code for 
> handling arrays awful.
>
> Similarly, Character'Size = 9 on that machine.

That sounds like a non-conformance, at least if the SP Annex
is supported.  Maybe you mean X'Size = 9, where X is of
type Character?  You'd certainly want 'Component_Size = 9
for array-of-Character.

> That would have made a compiler for the Unisys machines impossible; it would 
> have made streaming impossible. There is no sane way to put 36-bit values 
> into Octets - the only way that would have worked would have been to use 
> 16-bits for every 9-bit byte.
>
> Whether this is a significant consideration today (in 2010) is debatable, 
> but it surely was a real consideration back in 1993-4. So Ada 95 could not 
> have made this choice.

I think it would not be a good idea to make Ada unimplementable
on "odd-ball" machines.

- Bob

Re: S-expression I/O in Ada

[Randy Brukardt]

"Robert A Duff" <bobduff@shell01.TheWorld.com> wrote in message 
news:wccy6ceaaxz.fsf@shell01.TheWorld.com...
> "Randy Brukardt" <randy@rrsoftware.com> writes:
>
>> Not sure if it still exists in the real world, but the compiler we did 
>> for
>> the Unisys mainframes used Stream_Element'Size = 9.
>
> Interesting.  Can these machines communicate with non-Unisys machines
> over a regular TCP/IP network?  E.g. send an e-mail using standard
> protocols, that can be read on a x86?
>
> I assume Storage_Element'Size = 9, too.  Correct?

I think so, although we had tried for a while to use 36 for such things (it 
was a word-addressable machine). But since we were using the C code 
generator, it emulated a 9-bit byte addressable machine for the obvious 
reasons, and eventually we changed to that model, using strong alignments to 
make the code more efficient.

> Next question: Is (was) there any Ada implementation where
> Stream_Element'Size /= Storage_Element'Size?
>
>>...(The Unisys mainframes
>> are 36-bit machines.) Stream_Element'Size = 8 would have made the code 
>> for
>> handling arrays awful.
>>
>> Similarly, Character'Size = 9 on that machine.
>
> That sounds like a non-conformance, at least if the SP Annex
> is supported.  Maybe you mean X'Size = 9, where X is of
> type Character?  You'd certainly want 'Component_Size = 9
> for array-of-Character.

You're probably right. I never implemented Ada 95 'Size because it is 
meaningless. Janus/Ada 'Size is essentially what GNAT calls 'Object_Size, 
and it is used for all allocation/representation decisions in the absence of 
representation clauses. (Indeed, what I intend to do "someday" is covert the 
existing Size code to be reported as Object_Size, following the model of the 
rejected AI on that attribute, and then implement 'Size as mandated by the 
Standard, for no purpose other than matching the legality rules of the 
Standard. (Object_Size would continue to be the significant factor in object 
layout.)

                  Randy.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 9, 5:40 pm, Simon Wright <si...@pushface.org> wrote:
> Natacha Kerensikova <lithium...@gmail.com> writes:
> > and I don't know either how to deal with cases where Stream_Element is
> > not an octet.
>
> By ignoring them, I think!

That's something I'm quite uncomfortable with. However I see the point
of the other related posts here. So my question is, if I assume
Stream_Element is an octet, is there a way to make compilation fail
with an explicit error message when the assumption doesn't hold? C's
preprocessor allows this through a #if on CHAR_BITS and a #error to
stop compilation with a custom message. Any Ada equivalent?


Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Robert A Duff]

Natacha Kerensikova <lithiumcat@gmail.com> writes:

> That's something I'm quite uncomfortable with. However I see the point
> of the other related posts here. So my question is, if I assume
> Stream_Element is an octet, is there a way to make compilation fail
> with an explicit error message when the assumption doesn't hold? C's
> preprocessor allows this through a #if on CHAR_BITS and a #error to
> stop compilation with a custom message. Any Ada equivalent?

Put:

    pragma Assert(Stream_Element'Modulus = 2**8);
    pragma Assert(Stream_Element'Size = 8);

in any library package that depends on this fact.

This will fail at run time if it's wrong.  But it will fail on
EVERY execution of the program, so it's pretty-much just as
good as a compile-time error.  Also, if you're lucky,
the compiler will warn you at compile time (and you can
use warnings-as-errors mode if you like).

You might also want to look up the GNAT-specific Compile_Time_Error
and Compile_Time_Warning pragmas.

- Bob

Re: S-expression I/O in Ada

[Duke Normandin]

On 2010-08-08, Natacha Kerensikova <lithiumcat@gmail.com> wrote:

>
> Is it clearer now?

GO! Natacha! GO! -- Natacha ROCKS! -- GO! Natacha! GO!

;D

I've _now_ decided to go out and learn _all_ about S-Expressions. ;)
-- 
Duke

Re: S-expression I/O in Ada

[Robert A Duff]

Natacha Kerensikova <lithiumcat@gmail.com> writes:

> (probably Integer mod 2**16 or something like that).

There's no such thing as "Integer mod 2**16".  "mod ..." is
how you declare an unsigned integer type (called "modular type" in
Ada), and "range ..." is how you declare a signed integer type.

Integer is a predefined signed integer type.  It's range is
not portable, so if you care about that, it should be avoided.

It's a good idea to declare your own integer types -- different
types for different purposes.  For example, if you have an array
of indexes into another array, using different index types for
the two arrays will often make the code readable -- it will be
obvious that (say) X is intended for indexing into the
outer array.

Modular types are evil, and should usually be avoided.
You might want modular types when interfacing with C code,
but don't use them just because you happen to have
no negative numbers.  For example, I would usually
(not always!) prefer this:

    type Octet is range 0..2**8-1; -- signed

over this:

    type Octet is mod 2**8; -- unsigned

The index type of an unconstrained array should (almost) never
be modular.

- Bob

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 08 Aug 2010 11:34:23 -0400, Robert A Duff wrote:

> Modular types are evil, and should usually be avoided.

They aren't, modular arithmetic is. I wished to be able to define a modular
types without +,-,*,/, and, or, xor, and have a more decent notation for
S'Succ and S'Pred.

> The index type of an unconstrained array should (almost) never
> be modular.

That is again not their fault. If a subtype of a modular type were modular,
e,g,

   type T is mod 256;
   subtype S is T range 2..10;

   X : S := 10;
begin   
   X := X + 1;  -- The result is 2!

then they could be used as indices.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Robert A Duff]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> On Sun, 08 Aug 2010 11:34:23 -0400, Robert A Duff wrote:
>
>> Modular types are evil, and should usually be avoided.
>
> They aren't, modular arithmetic is.

Hmm.

Implicit modular arithmetic is evil.  I have no problem with
expressions like "X mod N".

>...I wished to be able to define a modular
> types without +,-,*,/, and, or, xor, and have a more decent notation for
> S'Succ and S'Pred.
>
>> The index type of an unconstrained array should (almost) never
>> be modular.
>
> That is again not their fault. If a subtype of a modular type were modular,
> e,g,
>
>    type T is mod 256;
>    subtype S is T range 2..10;
>
>    X : S := 10;
> begin   
>    X := X + 1;  -- The result is 2!

2 factorial?  ;-)

For scalars, the primitive operations mostly don't know the
subtype bounds -- they are operations of the type.  So this
wouldn't fit into Ada well.

> ...then they could be used as indices.

The problem I was alluding to is that if you have
an unconstrained array type, sooner or later you might
have an empty one.  So you want bounds 0..-1, but -1
wraps around.

Same thing with a "for" loop that goes from T'First up to
something.

If 2..10 wrapped around, then you'd want range 2..1, which
has the same problem.

Did I mention that modular types are my least favorite
feature of Ada?  ;-)

- Bob

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 08 Aug 2010 16:03:22 -0400, Robert A Duff wrote:

> For scalars, the primitive operations mostly don't know the
> subtype bounds -- they are operations of the type.  So this
> wouldn't fit into Ada well.

But array subtypes silently modify their operations or, if you want,
parametrize them using the dope. A modular subtype would have a dope as
well.

>> ...then they could be used as indices.
> 
> The problem I was alluding to is that if you have
> an unconstrained array type, sooner or later you might
> have an empty one.  So you want bounds 0..-1, but -1
> wraps around.

Empty range does not have bounds. What are bounds of

   type Empty is mod 0; -- Illegal in Ada, but must be legal

> Same thing with a "for" loop that goes from T'First up to
> something.

If ranges (contiguous sets of indices) were proper types you would be able
to construct them in many different ways. E.g.

   function ":" (From : Index; Length : Universal_Integer)
      return Index'Range

   L := 0
   ...
   for I in T'First :L loop

instead of what you can find in every Ada program:

   for I in T'First..T'First + L - 1 loop

or for enumerations:

   for I in T'First..T'Val (T'Pos (T'First) + L - 1) loop

which does not work anyway...

> If 2..10 wrapped around, then you'd want range 2..1, which
> has the same problem.

2..1 actually is 2,3,4,5,6,7,8,9,1

Modular numbers are not ordered.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Robert A Duff]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> On Sun, 08 Aug 2010 16:03:22 -0400, Robert A Duff wrote:
>
>> For scalars, the primitive operations mostly don't know the
>> subtype bounds -- they are operations of the type.  So this
>> wouldn't fit into Ada well.
>
> But array subtypes silently modify their operations or, if you want,
> parametrize them using the dope.

Yes, and discriminated subtypes are similar to arrays in that way.
The dividing line is "scalar" vs. "composite".  (I'm not sure that's
a good idea, but that's Ada.) You are suggesting to make modular
types (a kind of scalar) work more like composites, which doesn't
fit well, even if it's a good idea in the abstract.

I actually have no opinion whether it's a good idea, because I
don't like modular types in the first place.  ;-)

I didn't mention "access" above, which are "elementary" but not
"scalar".  Constrained access subtypes are a nightmare!

>... A modular subtype would have a dope as
> well.

They already do, but the arithmetic ops don't consult that dope.

>>> ...then they could be used as indices.
>> 
>> The problem I was alluding to is that if you have
>> an unconstrained array type, sooner or later you might
>> have an empty one.  So you want bounds 0..-1, but -1
>> wraps around.
>
> Empty range does not have bounds.

Except that in Ada, they do.  An empty String is (1..0).
It could also be 10..9 or even -100..-10_000_000, but
that's not a good idea.

>...What are bounds of
>
>    type Empty is mod 0; -- Illegal in Ada, but must be legal
>
>> Same thing with a "for" loop that goes from T'First up to
>> something.
>
> If ranges (contiguous sets of indices) were proper types you would be able
> to construct them in many different ways. E.g.
>
>    function ":" (From : Index; Length : Universal_Integer)
>       return Index'Range
>
>    L := 0
>    ...
>    for I in T'First :L loop
>
> instead of what you can find in every Ada program:
>
>    for I in T'First..T'First + L - 1 loop
>
> or for enumerations:
>
>    for I in T'First..T'Val (T'Pos (T'First) + L - 1) loop
>
> which does not work anyway...

Good points.

>> If 2..10 wrapped around, then you'd want range 2..1, which
>> has the same problem.
>
> 2..1 actually is 2,3,4,5,6,7,8,9,1
>
> Modular numbers are not ordered.

But they are -- they have "<", and "for" and so on.
Perhaps they _should_ be unordered, but I won't agree or disagree,
since I think in an ideal world they should be banished.

By the way, one defense of modular types I've heard is that
they are used in mathematics.  True.  But mathematicians do
not use _implicit_ mod.  They say things like "X = Y (mod N)",
which is pronounced "X is congruent to Y (modulo N)".
Congruent, not equal.

- Bob

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 08 Aug 2010 17:08:50 -0400, Robert A Duff wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
> I actually have no opinion whether it's a good idea, because I
> don't like modular types in the first place.  ;-)

The implementation or the idea? Would you agree that objects with some
properties of modular integers have place in Ada programs which do not
interface C?

> I didn't mention "access" above, which are "elementary" but not
> "scalar".  Constrained access subtypes are a nightmare!

Yes, because the language should decide whether the constraint does
influence the behavior (all operations potentially), or is a kludge that
prohibits some values when assigned or passed. I understand the motivation
why Ada 83 chosen the second, but it does not make it right. Assignment is
an operation as any other.

>> Empty range does not have bounds.
> 
> Except that in Ada, they do.  An empty String is (1..0).
> It could also be 10..9 or even -100..-10_000_000, but
> that's not a good idea.

Yes, because it is wrong. Doing something wrong always will hit back.
 
>> Modular numbers are not ordered.
> 
> But they are -- they have "<", and "for" and so on.

"<" is wrong when non-transitive. I wished Ada clarified difference between
enumeration and total order.

> Perhaps they _should_ be unordered, but I won't agree or disagree,
> since I think in an ideal world they should be banished.

I think they could be fixed.

> By the way, one defense of modular types I've heard is that
> they are used in mathematics.  True.

>  But mathematicians do
> not use _implicit_ mod.  They say things like "X = Y (mod N)",
> which is pronounced "X is congruent to Y (modulo N)".
> Congruent, not equal.

The mathematical notation (mod N) is untyped. It applies to any natural
numbers and what is worse you have to add it at each point of the program
you use the type.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Robert A Duff]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> The implementation or the idea? Would you agree that objects with some
> properties of modular integers have place in Ada programs which do not
> interface C?

No.  I do not like implicit "mod".  But that's the essence of
modular types.

If I ran the circus, then this:

    M : constant := 2**64;
    type My_Unsigned is range 0..M-1;
    pragma Assert(My_Unsigned'Size = 64);

    X : My_Unsigned := ...;
    Y : My_Unsigned := (X + 1) mod M;

would be legal, and would not raise any exceptions, even when
X = M-1.  (And I'd want "(X + 1) mod M" to be implemented
as a single "add" instruction on a typical 64-bit machine.)
The above is illegal in every Ada compiler, because My_Unsigned
is a _signed_ integer type, and nobody supports that range.

Part of the reason modular types were invented is because
signed integers don't work in cases like the above.

>> Perhaps they _should_ be unordered, but I won't agree or disagree,
>> since I think in an ideal world they should be banished.
>
> I think they could be fixed.

How?  And having fixed them, when would you use them?
That is, when would you prefer them over signed integers?

>> By the way, one defense of modular types I've heard is that
>> they are used in mathematics.  True.
>
>>  But mathematicians do
>> not use _implicit_ mod.  They say things like "X = Y (mod N)",
>> which is pronounced "X is congruent to Y (modulo N)".
>> Congruent, not equal.
>
> The mathematical notation (mod N) is untyped. It applies to any natural
> numbers and what is worse you have to add it at each point of the program
> you use the type.

Writing "mod" whenever I want an expression that takes the modulus is
a GOOD thing.  "+" should always do an add, and nothing else.
If want to negate the result of "+", I should write "-".
If want to take the modulus of the result of "+", I should
write "mod".

Look at how unsigned types are used in C.  size_t is a good
example.  It's used to count up the sizes of things.
If I have 1000 objects of size 10_000_000, the total
size is 1000*10_000_000 = 10_000_000_000.  If that
calculation wraps around on a 32-bit machine, the
answer is just plain wrong.  I'd rather get Constraint_Error.

If I interface to C's size_t, and do similar calculations on
the Ada side, wraparound is equally wrong.

- Bob

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Mon, 09 Aug 2010 09:48:02 -0400, Robert A Duff wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
>> The implementation or the idea? Would you agree that objects with some
>> properties of modular integers have place in Ada programs which do not
>> interface C?
> 
> No.  I do not like implicit "mod".  But that's the essence of
> modular types.
> 
> If I ran the circus, then this:
> 
>     M : constant := 2**64;
>     type My_Unsigned is range 0..M-1;
>     pragma Assert(My_Unsigned'Size = 64);
> 
>     X : My_Unsigned := ...;
>     Y : My_Unsigned := (X + 1) mod M;
> 
> would be legal, and would not raise any exceptions, even when
> X = M-1.  (And I'd want "(X + 1) mod M" to be implemented
> as a single "add" instruction on a typical 64-bit machine.)

What does "+" above return? Is "mod M" a required part of the notation or
not?

>>> Perhaps they _should_ be unordered, but I won't agree or disagree,
>>> since I think in an ideal world they should be banished.
>>
>> I think they could be fixed.
> 
> How?  And having fixed them, when would you use them?
> That is, when would you prefer them over signed integers?

Ring buffer indexing, flags (by-value sets actually), cryptography,
interfacing, communication.

I think that all these usages require different types of modular types with
different sets of operations. So I would prefer a language extension which
would allow me construction of such types rather than built-in ones, which
satisfy no one. Just one example from a huge list, why "in" is not an
operation?

   if 0 /= (Mode and Alarm) then  -- Isn't it awful?

why am I not allowed to have:

   if Alarm in Mode then

>>> By the way, one defense of modular types I've heard is that
>>> they are used in mathematics.  True.
>>
>>>  But mathematicians do
>>> not use _implicit_ mod.  They say things like "X = Y (mod N)",
>>> which is pronounced "X is congruent to Y (modulo N)".
>>> Congruent, not equal.
>>
>> The mathematical notation (mod N) is untyped. It applies to any natural
>> numbers and what is worse you have to add it at each point of the program
>> you use the type.
> 
> Writing "mod" whenever I want an expression that takes the modulus is
> a GOOD thing.  "+" should always do an add, and nothing else.

Huh, modular "+" *does* add in the ring. Your "+" does not!

> If want to negate the result of "+", I should write "-".

Negative inverse in a ring is not one of integers. They are different
types. It is good and helpful that Ada promotes this difference.

> If want to take the modulus of the result of "+", I should
> write "mod".

That would be a type conversion. Bad thing. But I see no problem with that.
Give me universal integers or equivalent "2 x width" type and you will get
what you want in return:

   function "+" (Left, Right : Modular) return Universal_Integer;
       -- overloads standard +, if that exists

--  function "mod" (Left, Right : Universal_Integer) return Modular;
--      just to remember, it already exists

That's it.

> Look at how unsigned types are used in C.  size_t is a good
> example.  It's used to count up the sizes of things.
> If I have 1000 objects of size 10_000_000, the total
> size is 1000*10_000_000 = 10_000_000_000.  If that
> calculation wraps around on a 32-bit machine, the
> answer is just plain wrong.  I'd rather get Constraint_Error.
> 
> If I interface to C's size_t, and do similar calculations on
> the Ada side, wraparound is equally wrong.

I agree. But it rather means that C is wrong defining size_t modular. On
Ada side it must be Natural_32. Why Ada does not support that? Why is it
impossible to declare 32-bit range 0..2**32-1 without wrapping (=with
overflow check)?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 09.08.10 16:38, Dmitry A. Kazakov wrote:

> I think that all these usages require different types of modular types with
> different sets of operations. So I would prefer a language extension which
> would allow me construction of such types rather than built-in ones, which
> satisfy no one. Just one example from a huge list, why "in" is not an
> operation?
> 
>    if 0 /= (Mode and Alarm) then  -- Isn't it awful?
> 
> why am I not allowed to have:
> 
>    if Alarm in Mode then

Is anything wrong with packed arrays of Booleans for status thingies?


>> Writing "mod" whenever I want an expression that takes the modulus is
>> a GOOD thing.  "+" should always do an add, and nothing else.
> 
> Huh, modular "+" *does* add in the ring. Your "+" does not!

How many programmers, among those without a degree in math or
in physics, know what a ring is?


Hence, no "+" can do the "right thing", since "+" is overloaded.
Not just in Ada, but also in popular understanding.
A definition of the "+" operator, whose meaning(s) is (are)
stipulated to be well known, seems impossible:
everyone considers its meaning(s) as clearly established as
the proper ingredients of lentil soup. Because the frames of
reference used in the definition are an obvious pick, aren't they!?

I don't remember when I have last read a final report on either
the statistical meaning of "+" or on lentil soup ingredients research
(other than our bi-weekly CVE :-)
But I do remember finding collections of good recipes.
Each recipe there is named with a unique identifier, and it lists
operands (ingredients) and operations (preparation, cooking).
The essential bit: use words or phrases that say what you
mean. (By implication, express what you want others to understand.)
Obviously, "+" does not express a recipe clearly, or not without
lengthy exegesis...

Imagine a programming language that is just like Ada with
SPARK restrictions on naming. Then, throw out overloading of
predefined names, too, of "+", for example.

No more misunderstandings, then.

But will this language be usable with beginners?

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Mon, 09 Aug 2010 17:14:36 +0200, Georg Bauhaus wrote:

> On 09.08.10 16:38, Dmitry A. Kazakov wrote:
> 
>> I think that all these usages require different types of modular types with
>> different sets of operations. So I would prefer a language extension which
>> would allow me construction of such types rather than built-in ones, which
>> satisfy no one. Just one example from a huge list, why "in" is not an
>> operation?
>> 
>>    if 0 /= (Mode and Alarm) then  -- Isn't it awful?
>> 
>> why am I not allowed to have:
>> 
>>    if Alarm in Mode then
> 
> Is anything wrong with packed arrays of Booleans for status thingies?

1. The membership/subset tests look even worse
2. No ranges
3. No loops
4. No case statement
5. Array aggregates as literals aren't not fun
6. No discriminants of
7. No array index of

Note that array must be a view of a modular type with the modulus 2**N, but
the type itself must remain scalar.

>>> Writing "mod" whenever I want an expression that takes the modulus is
>>> a GOOD thing.  "+" should always do an add, and nothing else.
>> 
>> Huh, modular "+" *does* add in the ring. Your "+" does not!
> 
> How many programmers, among those without a degree in math or
> in physics, know what a ring is?

Why should they use the type then?

> Hence, no "+" can do the "right thing", since "+" is overloaded.

Most of modular types should have no + at all. There should be a readable
notation for X+1 instead of infernal T'Succ(X)

> A definition of the "+" operator, whose meaning(s) is (are)
> stipulated to be well known, seems impossible:

http://en.wikipedia.org/wiki/Abelian_group ?

> Imagine a programming language that is just like Ada with
> SPARK restrictions on naming. Then, throw out overloading of
> predefined names, too, of "+", for example.

I don't think this were possible, but basically it is what inheritance is
all about. The concept of checking would be the LSP-conformity.
 
> No more misunderstandings, then.
> 
> But will this language be usable with beginners?

Come on, children start counting before knowing anything about groups. Most
of them die before they do. The language is a tool it is not a library.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 09.08.10 18:11, Dmitry A. Kazakov wrote:
> On Mon, 09 Aug 2010 17:14:36 +0200, Georg Bauhaus wrote:
> 
>> On 09.08.10 16:38, Dmitry A. Kazakov wrote:
>>
>>> I think that all these usages require different types of modular types with
>>> different sets of operations. So I would prefer a language extension which
>>> would allow me construction of such types rather than built-in ones, which
>>> satisfy no one. Just one example from a huge list, why "in" is not an
>>> operation?
>>>
>>>    if 0 /= (Mode and Alarm) then  -- Isn't it awful?
>>>
>>> why am I not allowed to have:
>>>
>>>    if Alarm in Mode then
>>
>> Is anything wrong with packed arrays of Booleans for status thingies?
> 
> 1. The membership/subset tests look even worse

OK. But I guess we could write inlined functions?

> 2. No ranges
> 3. No loops
> 4. No case statement

You have the index subtype at the point where you have
the array type.  Why not ranges, loops, and case statements?

> 5. Array aggregates as literals aren't not fun

Fun?

> 6. No discriminants of
> 7. No array index of

These refer to arrays in general, not Boolean arrays as sets and,
I guess, the item numbered 7 follows a general rule:
Where the array type is visible, its index type is
visible.  (Or else, you have been too lazy and used
an anonymous index type; still, you have 'first etc.)
Therefore, the index type can be named without
indirection, that is, via some array attribute function.
(Or else, index values can be made from 'first etc.)
Array access will be type checked either way.

> Note that array must be a view of a modular type

An array of Booleans for status fields must be a view of a
modular type?  I don't understand that.

> with the modulus 2**N, but
> the type itself must remain scalar.

Uh, which type must remain scalar?


>> A definition of the "+" operator, whose meaning(s) is (are)
>> stipulated to be well known, seems impossible:
> 
> http://en.wikipedia.org/wiki/Abelian_group ?

That's normative ontology, not empirically justified language
design ;-)
I remember having seen introductory texts covering algebraic
structures that use a symbol other than "+" to denote the additive
operation. I think the authors have a good reason.
One reason might be that they know their audience.
This is an example of empirically justified use of operator symbols.

When good average programmers write programs, are they
structuring their data by conscious use of mathematical
structures like groups?  I don't think so.

It is these programmers that need to know the meaning
of "+".  This seems vital.  Example: C's "definition" of the data
type int and its "+" operation demonstrates what happens
if you are optimistic about mastery of C's "+" basics.

I think Bob's example involving "not" of a mod 3 variable
shows a related issue. (And will be included in a test of advanced
Ada knowledge? ;-P)  Because *even* *though* there is a precise
mathematical definition (LRM) of what "not" should return,
referring to some overly complex set of rules explaining
the operation is hardly convincing?  Consistency of the language
(with itself) is only an excuse, then.


Georg

Re: S-expression I/O in Ada

[Robert A Duff]

Georg Bauhaus <rm.dash-bauhaus@futureapps.de> writes:

>> 7. No array index of
>
> These refer to arrays in general, not Boolean arrays as sets and,
> I guess, the item numbered 7 follows a general rule:
> Where the array type is visible, its index type is
> visible.  (Or else, you have been too lazy and used
> an anonymous index type; still, you have 'first etc.)
> Therefore, the index type can be named without
> indirection, that is, via some array attribute function.
> (Or else, index values can be made from 'first etc.)
> Array access will be type checked either way.

I think Dmitry was complaining in 7 that you can't do this:

    type Bit_Map is array(...) of Boolean;
    type T is array (Bit_Map) of Something; -- Illegal index type.

whereas if Bit_Map is modular then you CAN do that.

You can, but only if the number of bits is small.

- Bob

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 09.08.10 19:05, Robert A Duff wrote:

>>> 7. No array index of

> I think Dmitry was complaining in 7 that you can't do this:
> 
>     type Bit_Map is array(...) of Boolean;
>     type T is array (Bit_Map) of Something; -- Illegal index type.
> 
> whereas if Bit_Map is modular then you CAN do that.

Always takes a while until my slow head grasps Dmitry's
non-Ada semantics.

case {T0 x T1 x T3} is
 when ... => ...

Static dispatch tables on composite types' values
switching to non-subprogram sequences of instructions
wrapped in a case statement that's likely to have
impressive combinatorial properties...

Is there no better way than this?

Experimenting further, what will be a useful type of
case_statement_alternative's discrete choice list, then?
One that can be explored programmatically?


Georg

Re: S-expression I/O in Ada

[Robert A Duff]

Georg Bauhaus <rm.dash-bauhaus@futureapps.de> writes:

> Always takes a while until my slow head grasps Dmitry's
> non-Ada semantics.

;-)

Dmitry has many "interesting" ideas about what Ada ought to be like,
some (many?) of which I agree with.  But it's important to keep
straight the difference between "Ada as it is" and "Ada as Dmitry
(or Bob, or ...) would like it to be".  ;-)

> case {T0 x T1 x T3} is
>  when ... => ...
>
> Static dispatch tables on composite types' values
> switching to non-subprogram sequences of instructions
> wrapped in a case statement that's likely to have
> impressive combinatorial properties...

I can think of several reasonable ways to implement such a
feature.  It would, of course, require a rather massive
change to what "static expression" means.

You can already do "case" on a 64-bit integer,
so the compiler already has to deal with cases where
a simple jump table is infeasible.

> Is there no better way than this?
>
> Experimenting further, what will be a useful type of
> case_statement_alternative's discrete choice list, then?
> One that can be explored programmatically?

Any non-limited type, I guess.

- Bob

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 8/9/10 9:18 PM, Robert A Duff wrote:

>> case {T0 x T1 x T3} is
>>   when ... =>  ...

>> Experimenting further, what will be a useful type of
>> case_statement_alternative's discrete choice list, then?
>> One that can be explored programmatically?
>
> Any non-limited type, I guess.

I was fancying effects that programmers might want to inject into
the switching machinery of the case statement?  In the sense of
what I think they are calling aspect oriented programming.

   type C is  ...  record ... end record;
   type D is new C and Aspects with null record;  -- for case
     -- or use aspect notation for D?

   x : D := (C with null record);

   case x is
    when D'(comp_1, ..., comp_n) => ...


Generalizing, could we have aspects of types permitting the
specification of effects to be triggered, for example when
a primitive op is called?  (Much like SNOBOL-4's TRACE feature
works.)


Georg

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Mon, 09 Aug 2010 18:46:27 +0200, Georg Bauhaus wrote:

> When good average programmers write programs, are they
> structuring their data by conscious use of mathematical
> structures like groups?  I don't think so.

They do it unconsciously. It is no matter if we do it willingly or not,
there is only one right way.

> It is these programmers that need to know the meaning
> of "+".  This seems vital.  Example: C's "definition" of the data
> type int and its "+" operation demonstrates what happens
> if you are optimistic about mastery of C's "+" basics.

I don't see your point. It is broken in C because it contradicts to the
mathematical meaning of. So what do you propose? To keep it broken? Because
people better understand broken stuff? They don't.

> I think Bob's example involving "not" of a mod 3 variable
> shows a related issue. (And will be included in a test of advanced
> Ada knowledge? ;-P)  Because *even* *though* there is a precise
> mathematical definition (LRM) of what "not" should return,
> referring to some overly complex set of rules explaining
> the operation is hardly convincing?  Consistency of the language
> (with itself) is only an excuse, then.

Nope, "not" is just broken for mod 3. It is a lattice operation, it has
nothing to do with rings.

The problem with modular types is that one tried to build a submarine which
could fly. It sails like an aircraft and flies like a ship.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Georg Bauhaus]

Dmitry A. Kazakov wrote:
> there is only one right way.

(There is one right way per person. Some agree on the right way.
Even those who believe(!) in the one right way don't know(!)
for sure ... ;-)

> ["+"] is broken in C because it contradicts to the
> mathematical meaning of.

The mathematical meaning of "+" is ambiguous or context dependent.
It depends on the definitions of "+" etc.

Programmers usually pick one meaning of "+", which then turns out
to be the traditional mathematical preconception. There is no way
to fix preconceptions other than through shock, pain, and learning.
The thing to be unlearned has a name: "+".


 > So what do you propose? To keep it broken? Because
> people better understand broken stuff? They don't.

Use operator symbols that clearly separate:

- traditional mathematical addition
- computer addition

If you don't keep the concepts apart visually, programmers see "+",
see something seemingly familiar, and will think that "+" means "+".
But "+" doesn't mean "+",  and this causes all kinds of problems.
Expensive problems.

Even more so when "+" is considered magic and is thus admired.
C does use "+" in all sorts of places, overloading "+" for
arithmetic and pointers. You can't do anything substantial in
C without "+", explicitly or implicitly.


int f() {}
int (*g)() = f + 4;
int main()
{
   return g();
}

I'm sure someone familiar with C will find it easy to explain
why this is a harmless program in most implementations of C...


I believe APL's intent shows a way to rectify the imprecise
operator names issue.



Georg

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Tue, 10 Aug 2010 00:21:46 +0200, Georg Bauhaus wrote:

> Dmitry A. Kazakov wrote:
>> there is only one right way.
> 
> (There is one right way per person. Some agree on the right way.
> Even those who believe(!) in the one right way don't know(!)
> for sure ... ;-)

It is the purpose of science to find the right way. No, it wondered so
many, why is there only one mathematics, a product of pure imagination? But
it is out of comp.lang.ada scope.

>> ["+"] is broken in C because it contradicts to the
>> mathematical meaning of.
> 
> The mathematical meaning of "+" is ambiguous or context dependent.

It is exact in each context. C's authors didn't mean matrix addition when
they designed +.

> It depends on the definitions of "+" etc.

Meaning depends on definitions? There must be something wrong in this
sentence...

>  > So what do you propose? To keep it broken? Because
>> people better understand broken stuff? They don't.
> 
> Use operator symbols that clearly separate:
> 
> - traditional mathematical addition
> - computer addition

I don't know what computer addition is. Computer does not count. Computer
*models* counting. Function "+" is a model of what?
 
> int f() {}
> int (*g)() = f + 4;
> int main()
> {
>    return g();
> }

I have no problem with this. They don't model integers here. They do
software faults. As such + perfectly serves the purpose.  (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Robert A Duff]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> 1. The membership/subset tests look even worse

We're talking about bit maps now, right?

Membership is "if Flags(X)...".  Subset is
"if Is_Subset(These_Flags, Those_Flags)...".
The body of Is_Subset says, "(X and Y) = X".

Not so horrible.

> 2. No ranges
> 3. No loops

I don't know what you want ranges and loops for.

If you really want these, you can Unchecked_Convert to
integer (signed range 0..2**something).  But only if
something is small.

> 4. No case statement

Yeah, that's a shame.  I'd like to have case statements
on bit maps, but also on other composite types.

> 5. Array aggregates as literals aren't not fun

(This | That => True, The_Other => False)

I agree that's ugly, but it beats 2#101#.

> 6. No discriminants of

I'm not sure why you'd want discriminants of bit maps,
but I agree that discriminants should be more general.

> 7. No array index of

Unchecked_Conversion again.

> Note that array must be a view of a modular type with the modulus 2**N, but
> the type itself must remain scalar.

All Ada compilers support:

    type Index is range 0..100;
    type Bit_Map is array (Index) of Boolean;".

And this:

    type Bit_Map is array (Character) of Boolean;".

But no Ada compilers support this:

    type Bit_Map is mod 2**100;

> Most of modular types should have no + at all. There should be a readable
> notation for X+1 instead of infernal T'Succ(X)

What notation would you like?

How about:

    function Succ (...) return ... renames T'Succ;

?

- Bob

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Mon, 09 Aug 2010 12:47:45 -0400, Robert A Duff wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
>> 1. The membership/subset tests look even worse
> 
> We're talking about bit maps now, right?

Yes

> Membership is "if Flags(X)...".  Subset is
> "if Is_Subset(These_Flags, Those_Flags)...".
> The body of Is_Subset says, "(X and Y) = X".
> 
> Not so horrible.

But a lot of manual work. The type should just do what you expect it to do.
 
>> 2. No ranges
>> 3. No loops
> 
> I don't know what you want ranges and loops for.

A subset of flags. E.g. Critical_Errors, Warnings, Any_Error.

> If you really want these, you can Unchecked_Convert to
> integer (signed range 0..2**something).  But only if
> something is small.
> 
>> 4. No case statement
> 
> Yeah, that's a shame.  I'd like to have case statements
> on bit maps, but also on other composite types.

Yes, it is a type property that the domain is a set of identifiable values,
in fact, some mapping to an enumeration.

The language should provide means to implement this interface with any
type. E.g. why I cannot specify this interface for String to be able to
write:

   case Get_Token is
       when "type" => ...
       when "package" => ...
       when others => raise Syntax_Error;
   end case;

>> 6. No discriminants of
> 
> I'm not sure why you'd want discriminants of bit maps,
> but I agree that discriminants should be more general.

type Message (Urgency : Alarm_Type) is ...
 
>> Note that array must be a view of a modular type with the modulus 2**N, but
>> the type itself must remain scalar.
> 
> All Ada compilers support:
> 
>     type Index is range 0..100;
>     type Bit_Map is array (Index) of Boolean;".
> 
> And this:
> 
>     type Bit_Map is array (Character) of Boolean;".
> 
> But no Ada compilers support this:
> 
>     type Bit_Map is mod 2**100;

Why not? It is not that difficult to implement mod 2**100 or range
-2**1_000_000..2**1_000_000+1. I think the standard should require it on
any hardware.

>> Most of modular types should have no + at all. There should be a readable
>> notation for X+1 instead of infernal T'Succ(X)
> 
> What notation would you like?
> 
> How about:
> 
>     function Succ (...) return ... renames T'Succ;

It should be something more like an operator. The problem is that generics
should understand it without passing around as an extra parameter.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Robert A Duff]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

>>> 2. No ranges
>>> 3. No loops
>> 
>> I don't know what you want ranges and loops for.
>
> A subset of flags. E.g. Critical_Errors, Warnings, Any_Error.

I don't understand this part.  If subrange of a modular type
used as a bit map is not a coherent subset of the flags.

> Yes, it is a type property that the domain is a set of identifiable values,
> in fact, some mapping to an enumeration.
>
> The language should provide means to implement this interface with any
> type. E.g. why I cannot specify this interface for String to be able to
> write:
>
>    case Get_Token is
>        when "type" => ...
>        when "package" => ...
>        when others => raise Syntax_Error;
>    end case;

It would be nice.

>> But no Ada compilers support this:
>> 
>>     type Bit_Map is mod 2**100;
>
> Why not? It is not that difficult to implement mod 2**100 or range
> -2**1_000_000..2**1_000_000+1. I think the standard should require it on
> any hardware.

I agree.  But you and I are in a small minority here!

The frustrating thing to me is that every Ada compiler must support
arbitrary-range integer arithmetic at compile time, but can't make
that available in a portable way at run time.

- Bob

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/09/2010 02:34 PM, Robert A Duff wrote:
>
> The frustrating thing to me is that every Ada compiler must support
> arbitrary-range integer arithmetic at compile time, but can't make
> that available in a portable way at run time.

Yes. And arbitrary-range rational arithmetic. Every compiler that supports the 
container library has a hash table and some sort of searchable structure 
(balanced tree, skip list) implementation. None of these are portably available 
to developers. This is a mistake, IMO.

-- 
Jeff Carter
"Run away! Run away!"
Monty Python and the Holy Grail
58

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Mon, 09 Aug 2010 17:34:34 -0400, Robert A Duff wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
>>>> 2. No ranges
>>>> 3. No loops
>>> 
>>> I don't know what you want ranges and loops for.
>>
>> A subset of flags. E.g. Critical_Errors, Warnings, Any_Error.
> 
> I don't understand this part.  If subrange of a modular type
> used as a bit map is not a coherent subset of the flags.

Yes. There are two issues here:

1. The modular type used for flags has ranges with the meaning a subset,
like Character subsets: 'A'..'Z'. The modular type used for ring arithmetic
should have ranges with the meaning of a smaller ring. They are two
absolutely different things. They cannot be united in just one type.

2. There should be elements of the set. Individual flags must have a
separate type. A range should be constructed using the element's type and
not sets. When sets are used, they must be singletons, e.g. 1, 2, 4... The
range 3..7 is just a mess because 3 and 7 aren't singletons.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[_FrnchFrgg_]

Le 09/08/2010 21:59, Dmitry A. Kazakov a �crit :

>> Yeah, that's a shame.  I'd like to have case statements
>> on bit maps, but also on other composite types.
>
> Yes, it is a type property that the domain is a set of identifiable values,
> in fact, some mapping to an enumeration.
>
> The language should provide means to implement this interface with any
> type. E.g. why I cannot specify this interface for String to be able to
> write:
>
>     case Get_Token is
>         when "type" =>  ...
>         when "package" =>  ...
>         when others =>  raise Syntax_Error;
>     end case;

I think that you want pattern matching 
(http://en.wikipedia.org/wiki/Standard_ML#Algebraic_datatypes_and_pattern_matching)

With the OCaml syntax, you can do

match expression with
| (Foo bar, [Int 4, u]) -> <some code>
| _ -> <catchall code>

and the engine will attempt unification (
http://en.wikipedia.org/wiki/Unification_%28computing%29) between 
expression and (Foo bar, [Int 4, u]), and if it succeeds, bar and u will 
be variables (immutable in caml) of the right value and right type so as 
to "fill in the blanks". (_ is a special-purpose variable whose content 
is discarded, because in the catch-all code we don't need a new name for 
expression -- here _ will match the whole expression of course).

It is a pretty powerfull thing, and one of the reasons I keep using 
functionnal languages (I don't know if Lisp does it, though). Structured 
pattern matching begins to kick ass when you use big typed trees, like 
for instance ASTs.

I don't know enough ada to even imagine how you would implement such a 
thing in Ada, if possible, or whether it already exists (but I gather no 
from what you say).

(Sorry to intrude in an Ada newsgroup, I was just passing by)

_FrnchFrgg_

Re: S-expression I/O in Ada

[Georg Bauhaus]

_FrnchFrgg_ wrote:

> I don't know enough ada to even imagine how you would implement such a 
> thing in Ada, if possible, or whether it already exists (but I gather no 
> from what you say).

For the basic types like discrete types, value matching
is fully supported and re1quired.

   type Color is (Red, Green, Blue);
   C : Color;

   case C is
     when Red => Add_Green;
     when Green => Add_Blue;
   end case;


      7.   case C is
           |
         >>> missing case value: "BLUE"


Some of the structure matching can, I think, be handled
by choosing meaningful type hierarchies (Tree, Leaf and
the like) and use O-O overridings (for Tree and Leaf,
respectively).


Georg

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Mon, 09 Aug 2010 23:54:00 +0200, _FrnchFrgg_ wrote:

> Le 09/08/2010 21:59, Dmitry A. Kazakov a �crit :
> 
>>> Yeah, that's a shame.  I'd like to have case statements
>>> on bit maps, but also on other composite types.
>>
>> Yes, it is a type property that the domain is a set of identifiable values,
>> in fact, some mapping to an enumeration.
>>
>> The language should provide means to implement this interface with any
>> type. E.g. why I cannot specify this interface for String to be able to
>> write:
>>
>>     case Get_Token is
>>         when "type" =>  ...
>>         when "package" =>  ...
>>         when others =>  raise Syntax_Error;
>>     end case;
> 
> I think that you want pattern matching 
> (http://en.wikipedia.org/wiki/Standard_ML#Algebraic_datatypes_and_pattern_matching)

No, I don't believe in type inference, in fact I strongly disbelieve in it. 

What I want is a manifested strong type system properly done. The above
does not require matching all types involved were known in advance. It
requires some enumeration of strings (like hash function), which the
compiler could use in order to build the case statement.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[_FrnchFrgg_]

Le 10/08/2010 09:16, Dmitry A. Kazakov a �crit :
> On Mon, 09 Aug 2010 23:54:00 +0200, _FrnchFrgg_ wrote:
>> I think that you want pattern matching
>> (http://en.wikipedia.org/wiki/Standard_ML#Algebraic_datatypes_and_pattern_matching)
>
> No, I don't believe in type inference, in fact I strongly disbelieve in it.

Unification and pattern matching is independent of type inference. Sure, 
most of the time you find both in the same languages, but IIRC in the 
course of my master, I have encountered languages with one but not the 
other.

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Tue, 10 Aug 2010 13:06:58 +0200, _FrnchFrgg_ wrote:

> Le 10/08/2010 09:16, Dmitry A. Kazakov a �crit :
>> On Mon, 09 Aug 2010 23:54:00 +0200, _FrnchFrgg_ wrote:
>>> I think that you want pattern matching
>>> (http://en.wikipedia.org/wiki/Standard_ML#Algebraic_datatypes_and_pattern_matching)
>>
>> No, I don't believe in type inference, in fact I strongly disbelieve in it.
> 
> Unification and pattern matching is independent of type inference.

Did you mean the standard meaning of pattern matching instead of Standard
ML's Volap�k?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[_FrnchFrgg_]

Le 10/08/2010 13:19, Dmitry A. Kazakov a �crit :
> On Tue, 10 Aug 2010 13:06:58 +0200, _FrnchFrgg_ wrote:

>> Unification and pattern matching is independent of type inference.
>
> Did you mean the standard meaning of pattern matching instead of Standard
> ML's Volap�k?

I meant pattern matching as a ML construct, which is in fact structural 
unification. It can be done even without type inference, and only needs 
some kind of polymorphism; essentially you have an object and an 
expression made of (possibly nested) constructors, with leaves being 
either constants or variables, and the unification engine answers

a) if the object could be obtained by the sequence of constructors, or not
b) if yes, the content the variables would have had so that the sequence 
of constructors would produce the object.

For convenience, you often have a list of expressions, and the engine 
executes the code of the first which fits the object.

Of course, if you don't want a type orgy (aka dynamic types), you'd 
probably want all the constructor expressions create subtypes of a base 
one, and the object also be a subtype of this base one. When the 
language also does type inference, then this "greatest common 
denominator" is calculated at compile-time (and the language typically 
is able to tell you have forgotten to cover all possible constructor 
sequences for the type).

But nothing prevents a language to require explicit types and enforce 
them just by checking with no inference at all. And in fact, IIRC I was 
shown a language having unification without type inference (not PROLOG, 
because its unification is for predicates and doesn't count). I do not 
recall the name of the language (it was probably not much used). Anyway 
OCaml could arguably be considered as an exemple(1).

Note that "pattern matching" as described here is well tailored to 
immutable objects; I think there are ways to accomodate mutability, but 
I do not remember clearly. Since OCaml has mutability in objects if 
wanted, and pattern matching on objects (not only on algebraic types), I 
suppose they solved the problem in some way.

(1) Even if you don't like type inference (and I agree that a human 
reading the code shouldn't be required to "infer" the types, and as such 
that types should be annotated), every language I know with type 
inference accepts explicit type annotations so you can always type 
explicitly every variable and you exactly get a strongly, explicitely 
typed language.

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Wed, 11 Aug 2010 01:04:29 +0200, _FrnchFrgg_ wrote:

> Le 10/08/2010 13:19, Dmitry A. Kazakov a �crit :
>> On Tue, 10 Aug 2010 13:06:58 +0200, _FrnchFrgg_ wrote:
> 
>>> Unification and pattern matching is independent of type inference.
>>
>> Did you mean the standard meaning of pattern matching instead of Standard
>> ML's Volap�k?
> 
> I meant pattern matching as a ML construct, which is in fact structural 
> unification. It can be done even without type inference, and only needs 
> some kind of polymorphism; essentially you have an object and an 
> expression made of (possibly nested) constructors, with leaves being 
> either constants or variables, and the unification engine answers
> 
> a) if the object could be obtained by the sequence of constructors, or not
> b) if yes, the content the variables would have had so that the sequence 
> of constructors would produce the object.
> 
> For convenience, you often have a list of expressions, and the engine 
> executes the code of the first which fits the object.

This translated into Ada terms, looks like an automated generation of
literals/aggregates. There is one step required for Ada, i.e.
interpretation of a text as code, since Ada is a compiled language.

Main objection to all this is that it is hard-coded and involves the
object's structure (not nominal). Ada has similar mechanism, also
hard-coded and also structural. One generates S'Input and S'Output
attributes for stream I/O of built-in container types (arrays and records).
Another generates record and array aggregates.

I don't like this either. I would like to see a more general mechanism that
would allow user-defined recursive non-generic implementations. Because
beyond literals and stream I/O there is an infinite number of cases where
this pattern apply. And secondly it should work for user-defined opaque
container types.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Structural unification (pattern matching) in Ada [was: Re: S-expression I/O in Ada]

[_FrnchFrgg_]

Le 11/08/2010 16:10, Dmitry A. Kazakov a �crit :
> On Wed, 11 Aug 2010 01:04:29 +0200, _FrnchFrgg_ wrote:
>
>> Le 10/08/2010 13:19, Dmitry A. Kazakov a �crit :
>>> On Tue, 10 Aug 2010 13:06:58 +0200, _FrnchFrgg_ wrote:
>>
>>>> Unification and pattern matching is independent of type inference.
>>>
>>> Did you mean the standard meaning of pattern matching instead of Standard
>>> ML's Volap�k?
>>
>> I meant pattern matching as a ML construct, which is in fact structural
>> unification. It can be done even without type inference, and only needs
>> some kind of polymorphism; essentially you have an object and an
>> expression made of (possibly nested) constructors, with leaves being
>> either constants or variables, and the unification engine answers
>>
>> a) if the object could be obtained by the sequence of constructors, or not
>> b) if yes, the content the variables would have had so that the sequence
>> of constructors would produce the object.
>>
>> For convenience, you often have a list of expressions, and the engine
>> executes the code of the first which fits the object.
>
> This translated into Ada terms, looks like an automated generation of
> literals/aggregates. There is one step required for Ada, i.e.
> interpretation of a text as code, since Ada is a compiled language.

Objective Caml is also a compiled language, I don't really follow your 
point.

> Main objection to all this is that it is hard-coded and involves the
> object's structure (not nominal). Ada has similar mechanism, also
> hard-coded and also structural. One generates S'Input and S'Output
> attributes for stream I/O of built-in container types (arrays and records).
> Another generates record and array aggregates.
>
> I don't like this either. I would like to see a more general mechanism that
> would allow user-defined recursive non-generic implementations. Because
> beyond literals and stream I/O there is an infinite number of cases where
> this pattern apply. And secondly it should work for user-defined opaque
> container types.

I don't understand what Streams have to do with ML pattern matching. 
Just to be sure we are talking about the same thing, I read one of you 
wishing Ada had a more powerful/generic "switch" construct, and I 
noticed that the description of such a "switch" looked like a subset of 
structural unification as you can find in most functionnal languages.

Perhaps the current Ada is powerful enough to write some function/code 
doing this switch machinery, but I was thinking it would be a new 
language construct (or an extension of "switch").

_FrnchFrgg_

Re: Structural unification (pattern matching) in Ada [was: Re: S-expression I/O in Ada]

[Dmitry A. Kazakov]

On Wed, 11 Aug 2010 19:51:23 +0200, _FrnchFrgg_ wrote:

> I don't understand what Streams have to do with ML pattern matching. 

You mentioned to constructors stream I/O deploys them.

> Just to be sure we are talking about the same thing, I read one of you 
> wishing Ada had a more powerful/generic "switch" construct, and I 
> noticed that the description of such a "switch" looked like a subset of 
> structural unification as you can find in most functionnal languages.

Can you explain why it looked so to you.

Actually I didn't wish a more generic "switch", I wished it as it is. What
I needed is an enumeration interface for non-discrete types in order to be
able to use them in case. String was just one example.

> Perhaps the current Ada is powerful enough to write some function/code 
> doing this switch machinery, but I was thinking it would be a new 
> language construct (or an extension of "switch").

That was my first thought, so I asked if you meant *normal* patterns, like
regular expressions etc. No, I certainly don't want them in Ada
case-statement. Alternatives of a case statement can be evaluated in any
order, they can be statically verified for being exhaustive and not
overlapping. These are most important features, which any more or less
usable pattern language lacks.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Structural unification (pattern matching) in Ada [was: Re: S-expression I/O in Ada]

[Robert A Duff]

_FrnchFrgg_ <frnchfrgg@free.fr> writes:

>...Just to be sure we are talking about the same thing, I read
> one of you wishing Ada had a more powerful/generic "switch" construct,
> and I noticed that the description of such a "switch" looked like a
> subset of structural unification as you can find in most functionnal
> languages.

Yes, generalizing Ada case statements pushes them roughly in the
direction of OCaml pattern matching, which is much more powerful.

But one of my favorite features of Ada is that case statements
are checked at compile time to make sure they cover all
possible cases ("full coverage") and don't overlap.
If I remember OCaml correctly, it doesn't do either.
OCaml compilers typically warn about full coverage violations,
but the language doesn't require full coverage.
And overlapping is not warned about, and is commonly used
(the alternatives are checked in order, and the first
one wins, like an Ada if/elsif/elsif.../else chain).

My problem with OCaml is that to understand a pattern,
you have to understand all the preceding ones simultaneously,
and negate them in your head.  I don't think it's easy to
get the best of both worlds (powerful pattern matching
with full coverage and overlap rules).

Note that "when others" in Ada turns off the full coverage rule.
Similar to "_" in OCaml -- it means "none of the above".

> Perhaps the current Ada is powerful enough to write some function/code
> doing this switch machinery, ...

Well, one could write an OCaml compiler or interpreter in Ada.  ;-)

>...but I was thinking it would be a new
> language construct (or an extension of "switch").

"case", please.  This isn't comp.lang.c.  ;-)

- Bob

Re: Structural unification (pattern matching) in Ada [was: Re: S-expression I/O in Ada]

[(see below)]

On 11/08/2010 20:43, in article wcc7hjxorzs.fsf@shell01.TheWorld.com,
"Robert A Duff" <bobduff@shell01.TheWorld.com> wrote:
 
> Note that "when others" in Ada turns off the full coverage rule.
> Similar to "_" in OCaml -- it means "none of the above".

What a lovely example of the FP concision fetish.

-- 
Bill Findlay
<surname><forename> chez blueyonder.co.uk

Re: Structural unification (pattern matching) in Ada

[Simon Wright]

"(see below)" <yaldnif.w@blueyonder.co.uk> writes:

> On 11/08/2010 20:43, in article wcc7hjxorzs.fsf@shell01.TheWorld.com,
> "Robert A Duff" <bobduff@shell01.TheWorld.com> wrote:
>  
>> Note that "when others" in Ada turns off the full coverage rule.
>> Similar to "_" in OCaml -- it means "none of the above".
>
> What a lovely example of the FP concision fetish.

I think it's more Unix-related. Consider the bash case statement .. it's
'*' there. And doesn't Perl have a similar construct?

Re: Structural unification (pattern matching) in Ada [was: Re: S-expression I/O in Ada]

[_FrnchFrgg_]

Le 11/08/2010 21:43, Robert A Duff a �crit :

> Yes, generalizing Ada case statements pushes them roughly in the
> direction of OCaml pattern matching, which is much more powerful.
>
> But one of my favorite features of Ada is that case statements
> are checked at compile time to make sure they cover all
> possible cases ("full coverage") and don't overlap.
> If I remember OCaml correctly, it doesn't do either.

OCaml checks for full coverage, and warns about it (I even think you can 
make it an error), so nothing would prevent such a feature in Ada to 
make lack of full coverage an error. Overlapping I don't really know, 
but if the check is feasible, then of course I'd expect it to be in Ada.

 > My problem with OCaml is that to understand a pattern,
 > you have to understand all the preceding ones simultaneously,
 > and negate them in your head.  I don't think it's easy to
 > get the best of both worlds (powerful pattern matching
 > with full coverage and overlap rules).

Why not ? The only way I use overlapping is when I want a special case 
of a bigger rule treated separately (like in: do something special when 
the root of the AST is an addition, and one of the leaves is an integer, 
then use the generic "operator handling" for every other mathematical 
operator). I never ever wrote cases where a rule overlaped an other 
without one being a subset of the other.

And subset-overlapping can be rewritten to not overlap by matching for 
the full case and using a nested "case" statement. So overlaping is not 
really needed.

> Note that "when others" in Ada turns off the full coverage rule.
> Similar to "_" in OCaml -- it means "none of the above".

Just as a side note, "_" in OCaml is just like any other variable (you 
can even write "let _ = 5 in" or "let (_,x) = 
function_returning_a_couple arg1 arg2 in"; its value is just never 
stored (and it is illegal to use it as a r-value)

>> Perhaps the current Ada is powerful enough to write some function/code
>> doing this switch machinery, ...
>
> Well, one could write an OCaml compiler or interpreter in Ada.  ;-)

I meant with a decent integration with the normal Ada :-)

>> ...but I was thinking it would be a new
>> language construct (or an extension of "switch").
>
> "case", please.  This isn't comp.lang.c.  ;-)

My bad. Sorry for that.

Julien

Re: S-expression I/O in Ada

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:tg5qbr47zjz4$.2ywa2b17s656.dlg@40tude.net...
...
>>> Most of modular types should have no + at all. There should be a 
>>> readable
>>> notation for X+1 instead of infernal T'Succ(X)

There was a proposal to support X'Succ (effectively as a procedure), X being 
an object name. That eliminates most of the noise, but isn't perfect as it 
can't be easily used in a larger expression. I was in favor of that (along 
with X'Image, which is something that GNAT has supported for years).

                    Randy.

Re: S-expression I/O in Ada

[Robert A Duff]

Georg Bauhaus <rm.dash-bauhaus@futureapps.de> writes:

> Is anything wrong with packed arrays of Booleans for status thingies?

No.

In fact, arrays of Booleans (packed or not) are better than modular
types in that they can have an arbitrary number of bits.
If you have 50 flags, and use a modular type, that's not
portable -- it works in GNAT, but not all Ada compilers.
And then if you change your mind and want 70 flags,
you have to rewrite all the code.

And if packed, they should be equally efficient.

- Bob

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 9, 5:14 pm, Georg Bauhaus <rm.dash-bauh...@futureapps.de>
wrote:
> On 09.08.10 16:38, Dmitry A. Kazakov wrote:
>
> > Just one example from a huge list, why "in" is not an operation?
>
> >    if 0 /= (Mode and Alarm) then  -- Isn't it awful?
>
> > why am I not allowed to have:
>
> >    if Alarm in Mode then
>
> Is anything wrong with packed arrays of Booleans for status thingies?

In C for such things I usually use a struct bitfield. I guess the Ada
equivalent would be a record of Booleans, with a pragma or something
to have them packed together. Would there be anything wrong with that?
It seems to me more readable than the above IFs, though from what I
understood from the rest of the thread it would make CASE on multiple
bit difficult, and it won't allow slicing (which I'm not sure is a bad
thing, readability-wise). Is there anything else?


Natacha

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/09/2010 11:32 AM, Natacha Kerensikova wrote:
>
> In C for such things I usually use a struct bitfield. I guess the Ada
> equivalent would be a record of Booleans, with a pragma or something
> to have them packed together. Would there be anything wrong with that?
> It seems to me more readable than the above IFs, though from what I
> understood from the rest of the thread it would make CASE on multiple
> bit difficult, and it won't allow slicing (which I'm not sure is a bad
> thing, readability-wise). Is there anything else?

Typically, I would address it using:

type Mode_ID is (Alarm, ...);

type Mode_Set is array (Mode_ID) of Boolean;
for Mode_Set'Component_Size use 1;

Mode : Mode_Set;

if Mode (Alarm) then ...

Of course, I'd usually leave off the 'Component_Size definition; memory is 
usually cheap and plentiful these days. Only if I had to match an existing 
bit-packed representation, or Mode_ID was very large [array (Integer) of 
Boolean, anyone?] would I worry about the representation.

-- 
Jeff Carter
"Run away! Run away!"
Monty Python and the Holy Grail
58

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Robert A Duff]

Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> writes:

> On 08/09/2010 11:32 AM, Natacha Kerensikova wrote:
>>
>> In C for such things I usually use a struct bitfield. I guess the Ada
>> equivalent would be a record of Booleans, with a pragma or something
>> to have them packed together. Would there be anything wrong with that?
>> It seems to me more readable than the above IFs, though from what I
>> understood from the rest of the thread it would make CASE on multiple
>> bit difficult, and it won't allow slicing (which I'm not sure is a bad
>> thing, readability-wise). Is there anything else?
>
> Typically, I would address it using:
>
> type Mode_ID is (Alarm, ...);
>
> type Mode_Set is array (Mode_ID) of Boolean;
> for Mode_Set'Component_Size use 1;
>
> Mode : Mode_Set;
>
> if Mode (Alarm) then ...
>
> Of course, I'd usually leave off the 'Component_Size definition; memory
> is usually cheap and plentiful these days. Only if I had to match an
> existing bit-packed representation, or Mode_ID was very large [array
> (Integer) of Boolean, anyone?] would I worry about the representation.

Well, if you're doing "and"s and "or"s (i.e. intersections and unions),
then the packed representation is likely more efficient.  Memory is
cheap in terms of money, but in terms of efficiency, using more
memory can make your program run slower (e.g. worse cache behavior).

I'd use "pragma Pack" instead of the rep clause in this case.
It's guaranteed to do the same thing, but the pragma has more
of a flavor of "for efficiency reasons".  I'd use the Component_Size
clause if I were interfacing to some existing representation,
i.e., "for correctness reasons".

- Bob

Re: S-expression I/O in Ada

[(see below)]

On 09/08/2010 19:32, in article
d8933d54-7fee-4923-8a0c-ce9c5b1b9383@14g2000yqa.googlegroups.com, "Natacha
Kerensikova" <lithiumcat@gmail.com> wrote:

> On Aug 9, 5:14�pm, Georg Bauhaus <rm.dash-bauh...@futureapps.de>
> wrote:
>> On 09.08.10 16:38, Dmitry A. Kazakov wrote:
>> 
>>> Just one example from a huge list, why "in" is not an operation?
>> 
>>> � �if 0 /= (Mode and Alarm) then �-- Isn't it awful?
>> 
>>> why am I not allowed to have:
>> 
>>> � �if Alarm in Mode then
>> 
>> Is anything wrong with packed arrays of Booleans for status thingies?

I define a function "/" that takes the types of Mode and Alarm and returns
Boolean, so I can say things like:

   if Alarm/Mode then ...

Or (real code)

   if INS.jump_target/starts_a_code_block  then ...

-- 
Bill Findlay
<surname><forename> chez blueyonder.co.uk

Re: S-expression I/O in Ada

[Robert A Duff]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> On Mon, 09 Aug 2010 09:48:02 -0400, Robert A Duff wrote:
>
>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
>> 
>>> The implementation or the idea? Would you agree that objects with some
>>> properties of modular integers have place in Ada programs which do not
>>> interface C?
>> 
>> No.  I do not like implicit "mod".  But that's the essence of
>> modular types.
>> 
>> If I ran the circus, then this:
>> 
>>     M : constant := 2**64;
>>     type My_Unsigned is range 0..M-1;
>>     pragma Assert(My_Unsigned'Size = 64);
>> 
>>     X : My_Unsigned := ...;
>>     Y : My_Unsigned := (X + 1) mod M;
>> 
>> would be legal, and would not raise any exceptions, even when
>> X = M-1.  (And I'd want "(X + 1) mod M" to be implemented
>> as a single "add" instruction on a typical 64-bit machine.)
>
> What does "+" above return?

A value of type My_Unsigned.  The values of this type are
the infinite set of all integers -- that's already the
case in Ada, except that the RM allows (not requires)
certain calculations to overflow and raise C_E.

>...Is "mod M" a required part of the notation or
> not?

No, not required -- my idea is that if you want to
perform some operation ("mod" or "+" or "*" or anything),
you write that explicitly in the code.  No change there
from the way signed integers already work.

>>>> Perhaps they _should_ be unordered, but I won't agree or disagree,
>>>> since I think in an ideal world they should be banished.
>>>
>>> I think they could be fixed.
>> 
>> How?  And having fixed them, when would you use them?
>> That is, when would you prefer them over signed integers?
>
> Ring buffer indexing,

I prefer an explicit "mod" there.

>...flags (by-value sets actually),

I prefer array-of-Boolean over modular.  Even better would
be a proper user-defined abstraction, with notations like "in".

>...cryptography,

Explicit "mod".

> interfacing, communication.

For these, you don't want modular semantics -- you just want
a data type whose representation matches what you're 
interfacing/communicating with, such as "type Octet is
range 0..2**8-1;"

> I think that all these usages require different types of modular types with
> different sets of operations. So I would prefer a language extension which
> would allow me construction of such types rather than built-in ones, which
> satisfy no one.

I certainly agree with that.  But it sounds like you are
agreeing with my point -- that modular types should not
be in the language.

> I agree. But it rather means that C is wrong defining size_t modular. On
> Ada side it must be Natural_32. Why Ada does not support that? Why is it
> impossible to declare 32-bit range 0..2**32-1 without wrapping (=with
> overflow check)?

Good question.

- Bob

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Mon, 09 Aug 2010 13:00:38 -0400, Robert A Duff wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
>> On Mon, 09 Aug 2010 09:48:02 -0400, Robert A Duff wrote:
>>
>>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
>>> 
>>>> The implementation or the idea? Would you agree that objects with some
>>>> properties of modular integers have place in Ada programs which do not
>>>> interface C?
>>> 
>>> No.  I do not like implicit "mod".  But that's the essence of
>>> modular types.
>>> 
>>> If I ran the circus, then this:
>>> 
>>>     M : constant := 2**64;
>>>     type My_Unsigned is range 0..M-1;
>>>     pragma Assert(My_Unsigned'Size = 64);
>>> 
>>>     X : My_Unsigned := ...;
>>>     Y : My_Unsigned := (X + 1) mod M;
>>> 
>>> would be legal, and would not raise any exceptions, even when
>>> X = M-1.  (And I'd want "(X + 1) mod M" to be implemented
>>> as a single "add" instruction on a typical 64-bit machine.)
>>
>> What does "+" above return?
> 
> A value of type My_Unsigned.  The values of this type are
> the infinite set of all integers -- that's already the
> case in Ada, except that the RM allows (not requires)
> certain calculations to overflow and raise C_E.

OK, but it is not modular, it is natural. And it is just a different case
where modular types should not be used in first place, like size_t.
Equivalent Ada types like Storage_Offset are not modular.

>>...Is "mod M" a required part of the notation or
>> not?
> 
> No, not required -- my idea is that if you want to
> perform some operation ("mod" or "+" or "*" or anything),
> you write that explicitly in the code.  No change there
> from the way signed integers already work.

Yes, but not for modular types! (:-))

>>>>> Perhaps they _should_ be unordered, but I won't agree or disagree,
>>>>> since I think in an ideal world they should be banished.
>>>>
>>>> I think they could be fixed.
>>> 
>>> How?  And having fixed them, when would you use them?
>>> That is, when would you prefer them over signed integers?
>>
>> Ring buffer indexing,
> 
> I prefer an explicit "mod" there.

This is will be a source of errors like forgetting mod or specifying wrong
modulus. The idea of modular types is to prevent such errors.

>>...flags (by-value sets actually),
> 
> I prefer array-of-Boolean over modular.  Even better would
> be a proper user-defined abstraction, with notations like "in".

Agree

>>...cryptography,
> 
> Explicit "mod".

Disagree.

>> interfacing, communication.
> 
> For these, you don't want modular semantics -- you just want
> a data type whose representation matches what you're 
> interfacing/communicating with, such as "type Octet is
> range 0..2**8-1;"

The problem is that these things require both array-of-Boolean view and
arithmetic. I agree that when arithmetic is used, then it has to be wide.
E.g. when interpreting sequences of octets as little/big endian numbers, we
never use modular arithmetic. But integer arithmetic is incompatible with
array/set view. I.e. once you added something it is another type integer:

   function "+" (Left, Right : Octet) return Universal_Integer;

>> I think that all these usages require different types of modular types with
>> different sets of operations. So I would prefer a language extension which
>> would allow me construction of such types rather than built-in ones, which
>> satisfy no one.
> 
> I certainly agree with that.  But it sounds like you are
> agreeing with my point -- that modular types should not
> be in the language.

Yes, but what I want will never become Ada, so I prefer that minimum I can
get. (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Robert A Duff]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> OK, but it is not modular, it is natural. And it is just a different case
> where modular types should not be used in first place, like size_t.

Right, the whole point of this part of the discussion is:  I was
trying to show how Ada's signed integer types could be
improved to cover cases where Ada's modular types are commonly
used.

> This is will be a source of errors like forgetting mod or specifying wrong
> modulus. The idea of modular types is to prevent such errors.

I see that point, which was also made by others.  I at least
half-agree with it.

> Yes, but what I want will never become Ada, so I prefer that minimum I can
> get. (:-))

;-)

- Bob

Re: S-expression I/O in Ada

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:1y1c8zzqmcer5.po56hkesa968.dlg@40tude.net...
...
>> For these, you don't want modular semantics -- you just want
>> a data type whose representation matches what you're
>> interfacing/communicating with, such as "type Octet is
>> range 0..2**8-1;"
>
> The problem is that these things require both array-of-Boolean view and
> arithmetic. I agree that when arithmetic is used, then it has to be wide.
> E.g. when interpreting sequences of octets as little/big endian numbers, 
> we
> never use modular arithmetic. But integer arithmetic is incompatible with
> array/set view.

What have you done with Dmitry?? You can't be the *real* Dmitry! :-)

Array-of-bytes views and arithmetic views are of clearly different types, 
with different sets of operations. These shouldn't be mixed, logically or 
any other way. If you need to go between these views, you need some sort of 
type conversion (or Unchecked_Conversion or Stream'Read or...). Thus, it is 
*never* necessary to do operations on both views at once, and it is 
irrelevant what the "math" operations for octets is. If Ada gets anything 
wrong about this, it is that it has math operations at all for 
stream_elements.

The Dmitry I now certainly understands this, so I wonder what you've done 
with him... :-)

                                    Randy.

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Mon, 9 Aug 2010 20:17:40 -0500, Randy Brukardt wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
> news:1y1c8zzqmcer5.po56hkesa968.dlg@40tude.net...
> ...
>>> For these, you don't want modular semantics -- you just want
>>> a data type whose representation matches what you're
>>> interfacing/communicating with, such as "type Octet is
>>> range 0..2**8-1;"
>>
>> The problem is that these things require both array-of-Boolean view and
>> arithmetic. I agree that when arithmetic is used, then it has to be wide.
>> E.g. when interpreting sequences of octets as little/big endian numbers, 
>> we
>> never use modular arithmetic. But integer arithmetic is incompatible with
>> array/set view.
> 
> What have you done with Dmitry?? You can't be the *real* Dmitry! :-)

Brainwashed me? (:-))

> Array-of-bytes views and arithmetic views are of clearly different types, 
> with different sets of operations. These shouldn't be mixed, logically or 
> any other way. If you need to go between these views, you need some sort of 
> type conversion (or Unchecked_Conversion or Stream'Read or...). Thus, it is 
> *never* necessary to do operations on both views at once, and it is 
> irrelevant what the "math" operations for octets is. If Ada gets anything 
> wrong about this, it is that it has math operations at all for 
> stream_elements.

Right, but there is no contradiction because it is more than one type
involved. What I meant is:

   type Octet is ...;

       -- Array interface to access bits of the octet (not Ada)
   type Bit_Index is range 1..8;
   function "()" (Unit : Octet; Bit : Bit_Index) return Boolean;
   procedure "()" (Unit : in out Octet; Bit : Bit_Index; Value : Boolean);

       -- Arithmetic interface, immediately leads out of octets (not Ada)
   function "+" (Left, Right : Octet) return Universal_Integer;
   function "-" (Left, Right : Octet) return Universal_Integer;
       ...
So when you write:

   Little_Endian_Value := Octet_1 + Octet_2 * 256;

There result is not an octet, as it would be with a modular type. Presently
it is just broken.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:1o6jah15584x1$.1arrrgog9jdk7.dlg@40tude.net...
> On Mon, 9 Aug 2010 20:17:40 -0500, Randy Brukardt wrote:
>
>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
>> news:1y1c8zzqmcer5.po56hkesa968.dlg@40tude.net...
>> ...
>>>> For these, you don't want modular semantics -- you just want
>>>> a data type whose representation matches what you're
>>>> interfacing/communicating with, such as "type Octet is
>>>> range 0..2**8-1;"
>>>
>>> The problem is that these things require both array-of-Boolean view and
>>> arithmetic. I agree that when arithmetic is used, then it has to be 
>>> wide.
>>> E.g. when interpreting sequences of octets as little/big endian numbers,
>>> we
>>> never use modular arithmetic. But integer arithmetic is incompatible 
>>> with
>>> array/set view.
>>
>> What have you done with Dmitry?? You can't be the *real* Dmitry! :-)
>
> Brainwashed me? (:-))
>
>> Array-of-bytes views and arithmetic views are of clearly different types,
>> with different sets of operations. These shouldn't be mixed, logically or
>> any other way. If you need to go between these views, you need some sort 
>> of
>> type conversion (or Unchecked_Conversion or Stream'Read or...). Thus, it 
>> is
>> *never* necessary to do operations on both views at once, and it is
>> irrelevant what the "math" operations for octets is. If Ada gets anything
>> wrong about this, it is that it has math operations at all for
>> stream_elements.
>
> Right, but there is no contradiction because it is more than one type
> involved. What I meant is:
>
>   type Octet is ...;
>
>       -- Array interface to access bits of the octet (not Ada)
>   type Bit_Index is range 1..8;
>   function "()" (Unit : Octet; Bit : Bit_Index) return Boolean;
>   procedure "()" (Unit : in out Octet; Bit : Bit_Index; Value : Boolean);
>
>       -- Arithmetic interface, immediately leads out of octets (not Ada)
>   function "+" (Left, Right : Octet) return Universal_Integer;
>   function "-" (Left, Right : Octet) return Universal_Integer;
>       ...
> So when you write:
>
>   Little_Endian_Value := Octet_1 + Octet_2 * 256;
>
> There result is not an octet, as it would be with a modular type. 
> Presently
> it is just broken.

I wouldn't mess with mixed "+" routines that return other types. I'd just 
convert the Octets to a suitable type and add them. That is, any mess should 
be in type conversions, not in operators.

Since we're inventing things, I would suggest:

   -- Conversion function (Not Ada):
   function "#" (Right : Octet) return Universal_Integer;

   Little_Endian_Value := #Octet_1 + #Octet_2 * 256;

And now you don't need any Octet math.

BTW, the "#" operator was suggested by Jean Ichbiah and others back during 
the design of Ada 83, to be used for conversions between types. It's a pity 
that it or something like it never made it into Ada. (It comes up each time 
and still is rejected.)

                                Randy.

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Tue, 10 Aug 2010 16:42:08 -0500, Randy Brukardt wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
> news:1o6jah15584x1$.1arrrgog9jdk7.dlg@40tude.net...
>> On Mon, 9 Aug 2010 20:17:40 -0500, Randy Brukardt wrote:
>>
>>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
>>> news:1y1c8zzqmcer5.po56hkesa968.dlg@40tude.net...
>>> ...
>>>>> For these, you don't want modular semantics -- you just want
>>>>> a data type whose representation matches what you're
>>>>> interfacing/communicating with, such as "type Octet is
>>>>> range 0..2**8-1;"
>>>>
>>>> The problem is that these things require both array-of-Boolean view and
>>>> arithmetic. I agree that when arithmetic is used, then it has to be 
>>>> wide.
>>>> E.g. when interpreting sequences of octets as little/big endian numbers,
>>>> we
>>>> never use modular arithmetic. But integer arithmetic is incompatible 
>>>> with
>>>> array/set view.
>>>
>>> What have you done with Dmitry?? You can't be the *real* Dmitry! :-)
>>
>> Brainwashed me? (:-))
>>
>>> Array-of-bytes views and arithmetic views are of clearly different types,
>>> with different sets of operations. These shouldn't be mixed, logically or
>>> any other way. If you need to go between these views, you need some sort 
>>> of
>>> type conversion (or Unchecked_Conversion or Stream'Read or...). Thus, it 
>>> is
>>> *never* necessary to do operations on both views at once, and it is
>>> irrelevant what the "math" operations for octets is. If Ada gets anything
>>> wrong about this, it is that it has math operations at all for
>>> stream_elements.
>>
>> Right, but there is no contradiction because it is more than one type
>> involved. What I meant is:
>>
>>   type Octet is ...;
>>
>>       -- Array interface to access bits of the octet (not Ada)
>>   type Bit_Index is range 1..8;
>>   function "()" (Unit : Octet; Bit : Bit_Index) return Boolean;
>>   procedure "()" (Unit : in out Octet; Bit : Bit_Index; Value : Boolean);
>>
>>       -- Arithmetic interface, immediately leads out of octets (not Ada)
>>   function "+" (Left, Right : Octet) return Universal_Integer;
>>   function "-" (Left, Right : Octet) return Universal_Integer;
>>       ...
>> So when you write:
>>
>>   Little_Endian_Value := Octet_1 + Octet_2 * 256;
>>
>> There result is not an octet, as it would be with a modular type. 
>> Presently
>> it is just broken.
> 
> I wouldn't mess with mixed "+" routines that return other types. I'd just 
> convert the Octets to a suitable type and add them. That is, any mess should 
> be in type conversions, not in operators.

Conversion mess is what we already have right now. The point is that "+" is
well-defined and meaningful for octets, but it is not closed in there. Why

   function "**" (Left : T; Right : Natural) return T;
   function S'Pos(Arg : S'Base) return universal_integer;
   ...

are OK and "+" is not?

> Since we're inventing things, I would suggest:
> 
>    -- Conversion function (Not Ada):
>    function "#" (Right : Octet) return Universal_Integer;
> 
>    Little_Endian_Value := #Octet_1 + #Octet_2 * 256;
> 
> And now you don't need any Octet math.

1. What is this else?

2. I see no advantage over

   Little_Endian_Value := Integer (Octet_1) + Integer (Octet_2) * 256;

> BTW, the "#" operator was suggested by Jean Ichbiah and others back during 
> the design of Ada 83, to be used for conversions between types. It's a pity 
> that it or something like it never made it into Ada. (It comes up each time 
> and still is rejected.)

It would be nice to have more operators  than now ("^", "|", "..", "in",
":=", "<>", "()", "[]", "{}", "<=>", "@", "~" and so on. Ah, Ada is Unicode
now, here is the list: http://www.unicode.org/charts/PDF/U2200.pdf ).

But what is so special in type conversions, aren't they just functions?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:iv3qwjogornz$.1s8kq0yfl2wxl.dlg@40tude.net...
...
> Conversion mess is what we already have right now. The point is that "+" 
> is
> well-defined and meaningful for octets, but it is not closed in there. Why
>
>   function "**" (Left : T; Right : Natural) return T;
>   function S'Pos(Arg : S'Base) return universal_integer;
>   ...
>
> are OK and "+" is not?

The first has the same operand and result type (the power operand is 
something else and not really part of the operator IMHO). The second is the 
long winded name for a conversion operator - it implies no semantic change. 
I'm arguing that operations like S'Pos would be better having a common name 
like "#" rather than a host of specialized names.

>> Since we're inventing things, I would suggest:
>>
>>    -- Conversion function (Not Ada):
>>    function "#" (Right : Octet) return Universal_Integer;
>>
>>    Little_Endian_Value := #Octet_1 + #Octet_2 * 256;
>>
>> And now you don't need any Octet math.
>
> 1. What is this else?

Huh? This doesn't parse.

> 2. I see no advantage over
>
>   Little_Endian_Value := Integer (Octet_1) + Integer (Octet_2) * 256;

"#" can be overridden and/or user-defined. "Integer" cannot (and could never 
be in a version of Ada, as any such attempt would be way too incompatible). 
In addition, "#" is more readable in the 98% of the cases where you don't 
need to specify a type. I would probably consider using predefined versions 
of "#" to replace other forms of explicit type conversion as well (so "#" 
would always work if defined - no decision about whether to call a function, 
use a built-in type conversion, or use some attribute - and no need to look 
up the name of the type).

>> BTW, the "#" operator was suggested by Jean Ichbiah and others back 
>> during
>> the design of Ada 83, to be used for conversions between types. It's a 
>> pity
>> that it or something like it never made it into Ada. (It comes up each 
>> time
>> and still is rejected.)
>
> It would be nice to have more operators  than now ("^", "|", "..", "in",
> ":=", "<>", "()", "[]", "{}", "<=>", "@", "~" and so on. Ah, Ada is 
> Unicode
> now, here is the list: http://www.unicode.org/charts/PDF/U2200.pdf ).
>
> But what is so special in type conversions, aren't they just functions?

That's the point: in Ada, type conversions are *not* functions, they're a 
built-in gizmo [including some attributed]; by naming them "#" we would 
allow unifying them with functions. Of course, this could be abused (by 
having some semantic change encoded in the function as well), but that of 
course can happen just as well with "+" or "-" or "*".

                                        Randy.

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Wed, 11 Aug 2010 18:18:48 -0500, Randy Brukardt wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
> news:iv3qwjogornz$.1s8kq0yfl2wxl.dlg@40tude.net...
> ...
>> Conversion mess is what we already have right now. The point is that "+" 
>> is
>> well-defined and meaningful for octets, but it is not closed in there. Why
>>
>>   function "**" (Left : T; Right : Natural) return T;
>>   function S'Pos(Arg : S'Base) return universal_integer;
>>   ...
>>
>> are OK and "+" is not?
> 
> The first has the same operand and result type (the power operand is 
> something else and not really part of the operator IMHO).

What is the difference between "**" and "+"? It must be a language one,
because semantically sum of octets is not an octet.

> The second is the 
> long winded name for a conversion operator - it implies no semantic change. 
> I'm arguing that operations like S'Pos would be better having a common name 
> like "#" rather than a host of specialized names.

Huh, and these must be a type conversions too:

   function S'Exponent (X : T) return universal_integer;
   S'Length
   A'Length (N)
   ...

>>> Since we're inventing things, I would suggest:
>>>
>>>    -- Conversion function (Not Ada):
>>>    function "#" (Right : Octet) return Universal_Integer;
>>>
>>>    Little_Endian_Value := #Octet_1 + #Octet_2 * 256;
>>>
>>> And now you don't need any Octet math.
>>
>> 1. What is this else?
> 
> Huh? This doesn't parse.

The above looks like math to me. You take octet arguments and compute some
results using arithmetic operations.

>> 2. I see no advantage over
>>
>>   Little_Endian_Value := Integer (Octet_1) + Integer (Octet_2) * 256;
> 
> "#" can be overridden and/or user-defined. "Integer" cannot (and could never 
> be in a version of Ada, as any such attempt would be way too incompatible). 
> In addition, "#" is more readable in the 98% of the cases where you don't 
> need to specify a type. I would probably consider using predefined versions 
> of "#" to replace other forms of explicit type conversion as well (so "#" 
> would always work if defined - no decision about whether to call a function, 
> use a built-in type conversion, or use some attribute - and no need to look 
> up the name of the type).

I see. No, that is a wrong way IMO. The right one is interfaces. If you
want S be like T, derive S from T. If you don't like the implementation of
T, inherit only the interface of. You should never have a need in explicit
type conversions in a properly designed program.

> That's the point: in Ada, type conversions are *not* functions, they're a 
> built-in gizmo [including some attributed]; by naming them "#" we would 
> allow unifying them with functions.

I would prefer to eliminate them altogether. Conversions are always bad.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:1spow9sibnv6l.ci558lff75c8$.dlg@40tude.net...
> On Wed, 11 Aug 2010 18:18:48 -0500, Randy Brukardt wrote:
>
>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
>> news:iv3qwjogornz$.1s8kq0yfl2wxl.dlg@40tude.net...
>> ...
>>> Conversion mess is what we already have right now. The point is that "+"
>>> is
>>> well-defined and meaningful for octets, but it is not closed in there. 
>>> Why
>>>
>>>   function "**" (Left : T; Right : Natural) return T;
>>>   function S'Pos(Arg : S'Base) return universal_integer;
>>>   ...
>>>
>>> are OK and "+" is not?
>>
>> The first has the same operand and result type (the power operand is
>> something else and not really part of the operator IMHO).
>
> What is the difference between "**" and "+"? It must be a language one,
> because semantically sum of octets is not an octet.

Octets don't have a sum, "sum of octets" is meaningless. It's like talking 
about the "sum of sand". They're just buckets. (By "octet" here I mean the 
same concept that Ada calls Stream_Elements, other than with a fixed size.)

>> The second is the
>> long winded name for a conversion operator - it implies no semantic 
>> change.
>> I'm arguing that operations like S'Pos would be better having a common 
>> name
>> like "#" rather than a host of specialized names.
>
> Huh, and these must be a type conversions too:
>
>   function S'Exponent (X : T) return universal_integer;
>   S'Length
>   A'Length (N)
>   ...

Nope, these have semantics. Of course, the boundary is a gray area.

...
>>> 1. What is this else?
>>
>> Huh? This doesn't parse.
>
> The above looks like math to me. You take octet arguments and compute some
> results using arithmetic operations.

The octets are converted to some other type that has math. There is no 
built-in relationship between the octet and the type with the math; it has 
to be explicitly defined.

...
> I see. No, that is a wrong way IMO. The right one is interfaces. If you
> want S be like T, derive S from T. If you don't like the implementation of
> T, inherit only the interface of. You should never have a need in explicit
> type conversions in a properly designed program.

That way lies madness. You would have to give types properties that they 
have no reason to have (like "sum of octets"). And if you do that, you now 
have a lot more chances for error (such as adding octets that are used to 
hold character values and not integers). Octets are by their nature an 
untyped bucket with no semantics; there has to be a conversion operation 
(like "#" above or S'Read in Ada) to some type with semantics for it to be 
meaningful.

>> That's the point: in Ada, type conversions are *not* functions, they're a
>> built-in gizmo [including some attributed]; by naming them "#" we would
>> allow unifying them with functions.
>
> I would prefer to eliminate them altogether. Conversions are always bad.

I strongly disagree. Some things are best modeled with little or no explicit 
semantics (such as a raw stream), and you must have conversions to get to 
real semantics. Indeed, *not* having using conversions in that case is 
misleading; you're applying improper semantics to the operation.

                        Randy.

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Thu, 12 Aug 2010 15:56:41 -0500, Randy Brukardt wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
> news:1spow9sibnv6l.ci558lff75c8$.dlg@40tude.net...
>> On Wed, 11 Aug 2010 18:18:48 -0500, Randy Brukardt wrote:
>>
>>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
>>> news:iv3qwjogornz$.1s8kq0yfl2wxl.dlg@40tude.net...
>>> ...
>>>> Conversion mess is what we already have right now. The point is that "+"
>>>> is
>>>> well-defined and meaningful for octets, but it is not closed in there. 
>>>> Why
>>>>
>>>>   function "**" (Left : T; Right : Natural) return T;
>>>>   function S'Pos(Arg : S'Base) return universal_integer;
>>>>   ...
>>>>
>>>> are OK and "+" is not?
>>>
>>> The first has the same operand and result type (the power operand is
>>> something else and not really part of the operator IMHO).
>>
>> What is the difference between "**" and "+"? It must be a language one,
>> because semantically sum of octets is not an octet.
> 
> Octets don't have a sum, "sum of octets" is meaningless. It's like talking 
> about the "sum of sand". They're just buckets.

The content of the bucket does have sum. Compare it with the elements an
integer array. According to the logic since array elements are buckets you
cannot sum integer elements.

> (By "octet" here I mean the 
> same concept that Ada calls Stream_Elements, other than with a fixed size.)

1. This is not the way octets are used in communication applications.

2. If the idea of opaque container is pushed to its end, octet must have no
"and", "or", bit extraction operations either. They also presume some way
octets can be manipulated and interact with other types (like Boolean).

>>> The second is the
>>> long winded name for a conversion operator - it implies no semantic change.
>>> I'm arguing that operations like S'Pos would be better having a common name
>>> like "#" rather than a host of specialized names.
>>
>> Huh, and these must be a type conversions too:
>>
>>   function S'Exponent (X : T) return universal_integer;
>>   S'Length
>>   A'Length (N)
>>   ...
> 
> Nope, these have semantics. Of course, the boundary is a gray area.

? semantics = meaning. Why do we need things without meaning = meaningless
things?

>>>> 1. What is this else?
>>>
>>> Huh? This doesn't parse.
>>
>> The above looks like math to me. You take octet arguments and compute some
>> results using arithmetic operations.
> 
> The octets are converted to some other type that has math. There is no 
> built-in relationship between the octet and the type with the math;

Built-in = language defined? But Ada does not define Octet.

> it has to be explicitly defined.

Yes, any operation has to be defined explicitly if not inherited.

>> I see. No, that is a wrong way IMO. The right one is interfaces. If you
>> want S be like T, derive S from T. If you don't like the implementation of
>> T, inherit only the interface of. You should never have a need in explicit
>> type conversions in a properly designed program.
> 
> That way lies madness. You would have to give types properties that they 
> have no reason to have (like "sum of octets").

The semantics is to be defined by the programmer.

> And if you do that, you now 
> have a lot more chances for error (such as adding octets that are used to 
> hold character values and not integers).

Yes, when implementing a communication protocol, there is no such things as
characters or integers, only octets. This is independent on the octet
operations. Compare it with address arithmetic. You can sum address of a
task with the address of an employee record. Does it mean that there has to
be no address arithmetic?

> Octets are by their nature an 
> untyped bucket with no semantics; there has to be a conversion operation 
> (like "#" above or S'Read in Ada) to some type with semantics for it to be 
> meaningful.

No, that is impossible, because 1) there is no such conversion in almost
any case. Other objects are represented by collections of octets
interlinked in a very complex way. A simple conversion operation is
absolutely unsuitable abstraction for this. 2) You are talking about an
interface, (where however there should be no octets visible at all, but
streams, files etc), I am talking about implementation of such an
interface. There should be no conversions at any abstraction level.

>>> That's the point: in Ada, type conversions are *not* functions, they're a
>>> built-in gizmo [including some attributed]; by naming them "#" we would
>>> allow unifying them with functions.
>>
>> I would prefer to eliminate them altogether. Conversions are always bad.
> 
> I strongly disagree. Some things are best modeled with little or no explicit 
> semantics (such as a raw stream), and you must have conversions to get to 
> real semantics. Indeed, *not* having using conversions in that case is 
> misleading; you're applying improper semantics to the operation.

   type Text_Stream is new Raw_Stream with private;

What is wrong with that?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:ayxycml2mo60$.1g3uh1fg6m3r0.dlg@40tude.net...
> On Thu, 12 Aug 2010 15:56:41 -0500, Randy Brukardt wrote:
...
>> Octets don't have a sum, "sum of octets" is meaningless. It's like 
>> talking
>> about the "sum of sand". They're just buckets.
>
> The content of the bucket does have sum. Compare it with the elements an
> integer array. According to the logic since array elements are buckets you
> cannot sum integer elements.

By "bucket" here I mean a semantic-less collection of bits. Integer elements 
can be summed because they are integers. Buckets of bits don't make sense to 
be summed.

>> (By "octet" here I mean the
>> same concept that Ada calls Stream_Elements, other than with a fixed 
>> size.)
>
> 1. This is not the way octets are used in communication applications.

No idea. I use Stream_Elements the way I describe here in *my* applications, 
but YMMV.

> 2. If the idea of opaque container is pushed to its end, octet must have 
> no
> "and", "or", bit extraction operations either. They also presume some way
> octets can be manipulated and interact with other types (like Boolean).

Correct, an octet should have no operations whatsoever. It's just a bucket 
of bits for transport; it needs to be converted to something real to use it, 
explicitly so it is obvious what is going on.

Besides, I don't believe in bit operations groups of bits in the first 
place; they should be restricted to Boolean logic. (Ada 83 got this right, 
IMHO.)

There is of course a certain expediency in copying bad ideas from other 
languages like C; that's whats going on with Ada modular types. But the 
whole thing is a bad idea from the start to the end.

>> And if you do that, you now
>> have a lot more chances for error (such as adding octets that are used to
>> hold character values and not integers).
>
> Yes, when implementing a communication protocol, there is no such things 
> as
> characters or integers, only octets. This is independent on the octet
> operations. Compare it with address arithmetic. You can sum address of a
> task with the address of an employee record. Does it mean that there has 
> to
> be no address arithmetic?

I'm a radical on this, too. There should be no address arithmetic in 
programming languages; leave that to us compiler-writers, we can do a better 
(and much safer) job. Exposing that low-level stuff is a recipe for 
disaster.

(Again, I realize that there are times, mostly in interfaces to low-level 
languages and hardware, where you need such things. But this is totally 
outside of the realm of anything that can be described with sane typing; it 
makes no sense to even try.)

>> Octets are by their nature an
>> untyped bucket with no semantics; there has to be a conversion operation
>> (like "#" above or S'Read in Ada) to some type with semantics for it to 
>> be
>> meaningful.
>
> No, that is impossible, because 1) there is no such conversion in almost
> any case. Other objects are represented by collections of octets
> interlinked in a very complex way. A simple conversion operation is
> absolutely unsuitable abstraction for this. 2) You are talking about an
> interface, (where however there should be no octets visible at all, but
> streams, files etc), I am talking about implementation of such an
> interface. There should be no conversions at any abstraction level.

I don't have any idea of what you are talking about; it makes no sense to me 
at all. As you say, you should never have octets visible in the first place 
above the transport layer. And in that layer, the only thing that makes 
sense is a conversion to a meaningful type (S'Read is just a fancy name for 
Unchecked_Conversion, after all).

 >>>> That's the point: in Ada, type conversions are *not* functions, 
they're a
>>>> built-in gizmo [including some attributed]; by naming them "#" we would
>>>> allow unifying them with functions.
>>>
>>> I would prefer to eliminate them altogether. Conversions are always bad.
>>
>> I strongly disagree. Some things are best modeled with little or no 
>> explicit
>> semantics (such as a raw stream), and you must have conversions to get to
>> real semantics. Indeed, *not* having using conversions in that case is
>> misleading; you're applying improper semantics to the operation.
>
>   type Text_Stream is new Raw_Stream with private;
>
> What is wrong with that?

It's applying misleading semantics to an entity; there can be no stream 
other than a raw stream; even if the stream carries typing information, that 
information has to be converted and interpreted before it can be used.

I don't think this discussion is going anywhere anyway.

                      Randy.

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/09/2010 06:48 AM, Robert A Duff wrote:
>
> Look at how unsigned types are used in C.  size_t is a good
> example.  It's used to count up the sizes of things.
> If I have 1000 objects of size 10_000_000, the total
> size is 1000*10_000_000 = 10_000_000_000.  If that
> calculation wraps around on a 32-bit machine, the
> answer is just plain wrong.  I'd rather get Constraint_Error.

It seems to me there are 2 orthogonal issues: signed or unsigned representation, 
and overflow checking on or off. In this case, we'd like an unsigned 
representation (because we don't need negative values but do need as large a 
range as possible) with overflow checking on (we almost always want overflow 
checking on).

If integer types had been approached this way, perhaps "wrap around" would only 
happen on the base type of types without overflow checking.

-- 
Jeff Carter
"Run away! Run away!"
Monty Python and the Holy Grail
58

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 8, 11:08 pm, Robert A Duff <bobd...@shell01.TheWorld.com>
wrote:
> "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de> writes:
> By the way, one defense of modular types I've heard is that
> they are used in mathematics.  True.  But mathematicians do
> not use _implicit_ mod.  They say things like "X = Y (mod N)",
> which is pronounced "X is congruent to Y (modulo N)".
> Congruent, not equal.

Actually past a certain level, they don't explicit mod anymore. I
don't remember hearing many "modulo" in Galois field classes. And
neither when dealing with stuff like an n-bit integer representing a
member of Z/2Z [X] quotiented by a n+1 degree polynomial (Reed-Solomon
stuff). And then you're overloading "+" with something very xor-like…


Hoping I'm not disrupting anything,
Natacha

Re: S-expression I/O in Ada

[Robert A Duff]

Natacha Kerensikova <lithiumcat@gmail.com> writes:

> On Aug 8, 11:08�pm, Robert A Duff <bobd...@shell01.TheWorld.com>
> wrote:
>> "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de> writes:
>> By the way, one defense of modular types I've heard is that
>> they are used in mathematics. �True. �But mathematicians do
>> not use _implicit_ mod. �They say things like "X = Y (mod N)",
>> which is pronounced "X is congruent to Y (modulo N)".
>> Congruent, not equal.
>
> Actually past a certain level, they don't explicit mod anymore.

Interesting.  I never reached that level of math!

>... I
> don't remember hearing many "modulo" in Galois field classes. And
> neither when dealing with stuff like an n-bit integer representing a
> member of Z/2Z [X] quotiented by a n+1 degree polynomial (Reed-Solomon
> stuff). And then you're overloading "+" with something very xor-like

- Bob

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/07/2010 12:23 AM, Natacha Kerensikova wrote:
>
> I heard people claiming that the first language shapes the mind of
> coders (and they continue saying a whole generation of programmers has
> been mind-crippled by BASIC). My first language happened to be 386
> assembly, that might explain things. Anyway, I genuinely tried OOP
> with C++ (which I dropped because it's way too complex for me (and I'm
> tempted to say way too complex for the average coder, it should be
> reserved to the few geniuses actually able to fully master it)), but I
> never felt the need of anything beyond what can be done with a C
> struct containing function pointers.

Capers Jones, the function-point person, collected statistics on function points 
in various languages. One statistic was the average number of LOC to implement a 
function point. Function points may or may not be a good metric, but the values 
fell into 3 groups, which he labeled low-level languages (assembler and C), 
mid-level (FORTRAN, Pascal), and high-level (Ada).

Thus, there is a big difference in abstraction between C and Ada. C is about 
translating the problem into the capabilities of the solution language; Ada is 
(or should be) about modeling the problem in the software. C is about coding: 
mapping everything onto a small set of predefined representations. Ada is about 
SW engineering: creating useful abstractions that represent important aspects of 
the problem.

Effectively using Ada requires a different way of thinking from using C or 
assembler. You'll more quickly gain that mindset by asking how to approach 
specific problems in Ada, rather than how to do something you did in C.

I think there may be an analogy between languages and data representation 
formats: everything can be implemented in machine code, but that doesn't mean 
it's a good language for development. Similarly, everything may be represented 
as a sequence of bytes, but that doesn't mean it's an appropriate representation 
for an application.

An S-expression library might be a useful thing at a low-level, but few 
applications should call it directly. Instead, there should probably be at least 
one layer of abstraction on top of the low-level storage representation, so that 
the application only deals with appropriate application-level representations of 
the data.

Here, your problem seems to be how to have a human-readable storage format for 
your applications. While there is merit to discussing the pros and cons of the 
many existing formats to use, the Ada approach is to use an application-specific 
abstraction, hiding the implementation detail of which such format is eventually 
chosen.

-- 
Jeff Carter
"I blow my nose on you."
Monty Python & the Holy Grail
03

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 7, 5:38 pm, Jeffrey Carter <spam.jrcarter....@spam.not.acm.org>
wrote:
> On 08/07/2010 12:23 AM, Natacha Kerensikova wrote:
> Thus, there is a big difference in abstraction between C and Ada. C is about
> translating the problem into the capabilities of the solution language; Ada is
> (or should be) about modeling the problem in the software. C is about coding:
> mapping everything onto a small set of predefined representations. Ada is about
> SW engineering: creating useful abstractions that represent important aspects of
> the problem.

Funnily, you're the first one shaking my resolve to learn Ada.

Let's get everything straight: I'm amateur. I'm coding for fun. Well,
I'll so be coding for a living too, but then I won't have a say in the
language chosen. I like coding in C, and I don't care how efficient it
is. There is not waste of time in a leisure activity. I'd rather have
fun with C rather than doing the same thing 10x faster without fun.
The only thing bothering me in C is that I often end up using
dangerous construct. For example, *(struct foo **)((char *)whatever +
bar.offset). While I'm perfectly fine with that, because I'm confident
in what I'm doing, but I can understand it looks sloppy from the
outside. My main motivation to learn Ada is publicize the concerns for
robustness and correctness that might not be obvious from my C code. I
was hoping to do Ada whatever I used to be doing in C: network
programming, DS homebrew, etc.

Am I misguided? Should I stop now?

> Effectively using Ada requires a different way of thinking from using C or
> assembler. You'll more quickly gain that mindset by asking how to approach
> specific problems in Ada, rather than how to do something you did in C.

Ok, so let's have a look at the grand picture. My main objective right
now is to code a webserver in Ada. Yes, that's reinventing the wheel,
but it does wonders for learning.

Here is how I intended to do it, admittedly exactly like I would do it
in C, could you please tell me how far I am from the Ada approach?

I start by dividing the work into "modules" or "components", each
containing a structure or a few related structures, along with
whatever functions or procedures to deal with them. I thought this
would map perfectly into Ada packages.

Configuration files are S-expressions, in the form of (key value)
pairs, gathered in sections like (section-name (key value) (key value))
… Webpage templates are also S-expressions, in the form "raw-
html" (function arg1 arg2 …) "raw-html" (function) "raw-html". The
interesting thing being that arg1 arg2 etc are S-expressions and thus
can be subtemplates too.

As I'm more comfortable using components already coded and tested, I
would code them from the lowest to the highest level:
 - first a component dealing with S-expression I/O, hence this topic.
 - then a component for configuration, which use the S-expression
library and is used by other components either for program-wide
configuration variables or for instance specific configuration
 - then a network component, gluing the rest of my program with
whatever socket library I will use (AdaSockets or GNAT.stuff or C
interfacing or whatever, don't know yet)
 - then a HTTP parsing component, taking data from the network
component and configuration
 - then a general page component, dispatching requests to the relevant
page objects
 - then a raw file component, a specific page responding to HTTP
request with data taken directly from a file
 - then a template component, interpreting the function calls from S-
expression templates
 - then a templated page component, another specialization of a page
object, dealing with HTTP response and containing instance-specific
data used by the template component.

And that should be about it, I might encounter the need for other
components, maybe for network I/O multiplexing or for logging or for
caching templates etc.

So, how bad is it?

> An S-expression library might be a useful thing at a low-level, but few
> applications should call it directly. Instead, there should probably be at least
> one layer of abstraction on top of the low-level storage representation, so that
> the application only deals with appropriate application-level representations of
> the data.

I have to admit in the above I don't really know what belongs to a
library and what belongs to the application. But indeed, a S-
expression package is a low-level thing, I'm well aware of that, I
just can begin with high-level stuff if I don't have strong and tested
low-level stuff to build upon.

However the point of coding so many separate components is to be able
to change the internals of one without having to touch everything
else. Should I find someday a format so much better than S-
expressions, I would only have one component to change. Should I want
different formats for configuration and templates, that's a component
to add, and maybe little modifications to configuration and/or
template modules. And so on…

> Here, your problem seems to be how to have a human-readable storage format for
> your applications. While there is merit to discussing the pros and cons of the
> many existing formats to use, the Ada approach is to use an application-specific
> abstraction, hiding the implementation detail of which such format is eventually
> chosen.

Is that so different than what I explained above?


Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/07/2010 10:01 AM, Natacha Kerensikova wrote:
>
> Let's get everything straight: I'm amateur. I'm coding for fun. Well,
> I'll so be coding for a living too, but then I won't have a say in the
> language chosen. I like coding in C, and I don't care how efficient it
> is. There is not waste of time in a leisure activity. I'd rather have
> fun with C rather than doing the same thing 10x faster without fun.
> The only thing bothering me in C is that I often end up using
> dangerous construct. For example, *(struct foo **)((char *)whatever +
> bar.offset). While I'm perfectly fine with that, because I'm confident
> in what I'm doing, but I can understand it looks sloppy from the
> outside. My main motivation to learn Ada is publicize the concerns for
> robustness and correctness that might not be obvious from my C code. I
> was hoping to do Ada whatever I used to be doing in C: network
> programming, DS homebrew, etc.
>
> Am I misguided? Should I stop now?

One can do anything you can do in C in Ada. Better, since creating buffer 
overflow and signed-integer overflow vulnerabilities takes effort in Ada, while 
they're the default in C. (Virtually every "important security update" I see for 
Linux is a buffer overflow or signed-integer overflow vulnerability. I doubt if 
people are creating these on purpose. My conclusion is that it is impossible in 
practice to use C without creating these errors.)

> Ok, so let's have a look at the grand picture. My main objective right
> now is to code a webserver in Ada. Yes, that's reinventing the wheel,
> but it does wonders for learning.
>
> Here is how I intended to do it, admittedly exactly like I would do it
> in C, could you please tell me how far I am from the Ada approach?

It's hard to comment meaningfully, since you mostly describe your intended 
implementation, not your requirements.

I'd use Ada Web Server (AWS), and perhaps you should try that, too. Using a 
significant, existing, high-level Ada application framework like that might help 
introduce you to how some experienced Ada people thought this kind of thing 
should be approached.

A "web server" can be a variety of things, from a simple page server that serves 
static files to a highly-dynamic system generating everything on the fly. It 
appears that you intend something that serves static files and expanded page 
templates.

Initially, I'd observe that the system talks to the network and to the permanent 
storage that stores the configuration information, static pages, and so on. So 
my initial decomposition would identify interface modules for communicating with 
these. (This is an "edges-in" approach.)

At a higher level, there is something that responds to incoming requests to 
serve the appropriate responses. There's something this uses to obtain the 
configuration from the permanent storage. This could make use of something that 
can serve a static page and something that can serve an expanded page template. 
There's also clearly a place for something that expands a page template.

I'm doing this off the top of my head, so I won't be surprised if I've missed 
something or otherwise screwed up.

This identifies the major high-level modules in the system. I could now define 
the package specifications for them and have the compiler check that they are 
syntactically correct and semantically consistent. Then I could pick one and 
design its implementation. It's likely at some point tasking would be involved, 
allowing the processing of multiple requests at once, so this would all have to 
be done keeping concurrency in mind.

At some point I'd get to a low enough level to start thinking about 
representations, which seems to be where you begin your thinking about the problem.

> As I'm more comfortable using components already coded and tested, I
> would code them from the lowest to the highest level:

In Ada, one can create package specifications, then create other units that make 
use of those specifications before they are implemented. This is an important 
concept in Ada called the separation of specification and body. Sometimes it is 
useful to create stub bodies for such packages, which can then be used to test 
the units that make use of these packages. Thus it is often possible to 
implement and test higher-level modules before lower-level modules that they use 
have been implemented. This may not be especially useful on a single-person 
project, but can be quite valuable in projects with more than one developer. 
This often seems to be a foreign concept to those used to C.

While your approach seems quite different to mine, many aspects of the final 
result seem to be similar. This probably bodes well for you being able to use 
Ada effectively.

-- 
Jeff Carter
"I blow my nose on you."
Monty Python & the Holy Grail
03

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 8, 8:52 am, Jeffrey Carter <spam.jrcarter....@spam.not.acm.org>
wrote:
> On 08/07/2010 10:01 AM, Natacha Kerensikova wrote:
> > Here is how I intended to do it, admittedly exactly like I would do it
> > in C, could you please tell me how far I am from the Ada approach?
>
> It's hard to comment meaningfully, since you mostly describe your intended
> implementation, not your requirements.

Well actually the requirements were presented in the beginning: I've
got a S-expression configuration file, a directory of static files, a
directory of S-expression templates and a directory of S-expression
data to populate the templates. And I want to respond to HTTP request
with either a static file or an expanded template. Now I might
misunderstanding the word "requirements", but what I actually consider
as requirements is the above without any "S-expression" occurrence,
the rest being implementation choices. "S-expression" might be part of
the requirements to use existing data, but then again another format
can be chosen provided a converted from S-expression is also coded.

I then proceeded to propose an implementation fitting these
requirements, which how I would do stuff, and asking how far it is
from a typical Ada-style implementation.

> I'd use Ada Web Server (AWS), and perhaps you should try that, too. Using a
> significant, existing, high-level Ada application framework like that might help
> introduce you to how some experienced Ada people thought this kind of thing
> should be approached.

Thanks for the pointer, however I'm already afraid learning only Ada
is too much for me (I've tried with C++ and it's a language much too
complex to fit in my head (especially the STL), and I hope Ada is
simple enough), learning both Ada and AWS at the same time will
probably be way too much. However I haven't decided yet whether I will
actually begin with this project, I might try to code something
simpler before that (which will also need S-expressions, hence my
beginning with that library). In that case, I might be confident
enough in my Ada before starting the web project, and then go for AWS.

> A "web server" can be a variety of things, from a simple page server that serves
> static files to a highly-dynamic system generating everything on the fly. It
> appears that you intend something that serves static files and expanded page
> templates.
>
> Initially, I'd observe that the system talks to the network and to the permanent
> storage that stores the configuration information, static pages, and so on. So
> my initial decomposition would identify interface modules for communicating with
> these. (This is an "edges-in" approach.)
>
> At a higher level, there is something that responds to incoming requests to
> serve the appropriate responses. There's something this uses to obtain the
> configuration from the permanent storage. This could make use of something that
> can serve a static page and something that can serve an expanded page template.
> There's also clearly a place for something that expands a page template.
>
> I'm doing this off the top of my head, so I won't be surprised if I've missed
> something or otherwise screwed up.

Actually, that's very similar to what I thought too. It's just that
I've already thought so much about this project that I'm further down
the road, and I tried to fit everything in the one-dimension of a
text.

That's actually a pretty strange thing I've never encountered yet in
other coders I know, I first spend a lot of time thinking before
writing the first line of code (or that particular case, before even
managing to install the compiler). I also spend much less time
debugging, though I can't tell whether they are related or whether one
of them is more efficient than the other.

> This identifies the major high-level modules in the system. I could now define
> the package specifications for them and have the compiler check that they are
> syntactically correct and semantically consistent. Then I could pick one and
> design its implementation.

I have done this a few times, however C compilation being weaker than
Ada's (from what I understood), knowing that something in C compiles
isn't that useful. Hence my stress on tests, which require an
implementation of every dependency. Hence my tendency to start with
lower-level code.

Moreover, I have quite a few times (though less often recently, it
might a beginner thing) realized during implementation that the
specification is poorly chosen. The less higher-level components are
already written, the cheaper interface redesign is.

> It's likely at some point tasking would be involved,
> allowing the processing of multiple requests at once, so this would all have to
> be done keeping concurrency in mind.

I'm unsure about this. In C I usually use a socket multiplexer call
(poll() or select()) along with memory buffers, which allows to serve
simultaneously multiple requests in a single thread. It scales
differently than a thread-based approach, but I'm nowhere near the
amount of traffic where it matters, so going for the simplest might be
the best.

Moreover, as the multiplexing might end up being one package (or being
integrated to the networking package, I don't know enough yet), there
is only one place to change should I want to switch between pure-
threaded (like apache), pure-multiplexed (like lighttpd), or a mixed
implementation (like nginx). Resources are largely independent and
read-only, so making everything thread-safe shouldn't be that
difficult anyway.

> At some point I'd get to a low enough level to start thinking about
> representations, which seems to be where you begin your thinking about the problem.

Actually I have already thought a lot before, I just didn't feel the
need of Ada-specific advice before thinking about the actual low-level
implementation.

> > As I'm more comfortable using components already coded and tested, I
> > would code them from the lowest to the highest level:
>
> In Ada, one can create package specifications, then create other units that make
> use of those specifications before they are implemented. This is an important
> concept in Ada called the separation of specification and body. Sometimes it is
> useful to create stub bodies for such packages, which can then be used to test
> the units that make use of these packages. Thus it is often possible to
> implement and test higher-level modules before lower-level modules that they use
> have been implemented. This may not be especially useful on a single-person
> project, but can be quite valuable in projects with more than one developer.
> This often seems to be a foreign concept to those used to C.

As I said, I have the feeling this is very close to what I'm already
doing in C, except that you don't get very far with stub in C, because
the C compiler doesn't prevent many errors beyond typos.

> While your approach seems quite different to mine, many aspects of the final
> result seem to be similar. This probably bodes well for you being able to use
> Ada effectively.

Unless I'm very misunderstanding and/or vary naive, I've got the
feeling our approaches are not that different, and differ mostly in
the order of implementation, which indeed doesn't change that much in
the final result.

I'm glad my future in Ada looks well. I'm still afraid of its
complexity, and of how intrusive the standard library is (e.g. DS is
very limited in memory, as much useless (and maybe not-so-unless)
stuff as possible should be trimmed away).


Thanks for your insights,
Natacha

Re: S-expression I/O in Ada

[Robert A Duff]

Natacha Kerensikova <lithiumcat@gmail.com> writes:

> Thanks for the pointer, however I'm already afraid learning only Ada
> is too much for me (I've tried with C++ and it's a language much too
> complex to fit in my head (especially the STL), and I hope Ada is
> simple enough), ...

Ada is more complicated than C, less complicated that C++.

There are also cases where Ada _seems_ less complicated.
Overloading, for example.  All you need to know is:

    - What it is.
    
    - It's allowed for subprograms and literals.  (That's slightly
      wrong, but you don't need to care.)

    - The compiler resolves overloading by looking at the types
      of things.

    - If you get an ambiguity, you can resolve it in one of three ways,
      in decreasing order of preference:

        - Change the names of some things.

        - Specify the type of some expression using a qualified
          expression.

        - Use Package_Name.Procedure_Name notation.

The actual overload resolution rules in Ada are quite complicated.
Dozens of pages of text, I'd guess.  But you don't need to
understand them -- just wait until the compiler complains,
and fix it.

In C++, on the other hand, you need to understand all kinds of
subtle interactions with implicit conversions, for example.

So Ada's overloading rules might be as complicated as the C++
rules, but Ada _seems_ simpler, because you can safely ignore
all the details.  Unless you're writing an Ada compiler, of
course.

>...learning both Ada and AWS at the same time will
> probably be way too much.

Well, I'm not sure.  To learn a programming language, you need to
write programs, and also read programs.  Jeff's point was that
reading AWS will help you learn Ada, and that's probably true.

By the way, I don't see anything fundamentally wrong with
your s-expression plans, with s-expressions containing
raw arrays-of-octets -- especially since you say you
already have a lot of stuff using s-exprs.  You can layer on
top of that less-general but more-typed abstractions, probably
using generics.

- Bob

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 8, 5:24 pm, Robert A Duff <bobd...@shell01.TheWorld.com> wrote:
> So Ada's overloading rules might be as complicated as the C++
> rules, but Ada _seems_ simpler, because you can safely ignore
> all the details.  Unless you're writing an Ada compiler, of
> course.

I remember I wasn't that happy with C++ overloading, especially
operator overloading. I wrote a 3D vector library only to find out
operator overloading was more obscuring than improving readability,
because too much was happening behind the scenes. So I came back to a
math-assembly-looking sequence of function calls, uglier but easier to
follow and maintain.

> >...learning both Ada and AWS at the same time will
> > probably be way too much.
>
> Well, I'm not sure.  To learn a programming language, you need to
> write programs, and also read programs.  Jeff's point was that
> reading AWS will help you learn Ada, and that's probably true.

For some strange reason it didn't even occurred to be that the advice
included reading AWS source and not only use it as a library. Still,
AWS looks like an overkill for my web needs.

> By the way, I don't see anything fundamentally wrong with
> your s-expression plans, with s-expressions containing
> raw arrays-of-octets

Thanks for the support, that's what I'm about to do, except I'm still
unsure about what array type to use: range octet, modular octet,
Storage_Element, Stream_Element, something else…

> -- especially since you say you
> already have a lot of stuff using s-exprs.  You can layer on
> top of that less-general but more-typed abstractions, probably
> using generics.

Actually I wasn't aware until very recently that the code I usually
put on top of my S-expression parser is indeed called a parser too. So
yeah, basically my S-expression library will be the common core of all
my specialized configuration/template/etc file parsers.


Natacha

Re: S-expression I/O in Ada

[Robert A Duff]

Natacha Kerensikova <lithiumcat@gmail.com> writes:

> Thanks for the support, that's what I'm about to do, except I'm still
> unsure about what array type to use: range octet, modular octet,
> Storage_Element, Stream_Element, something else

The truth (after all this discussion) is that it doesn't
matter a whole lot.

- Bob

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/08/2010 06:11 AM, Natacha Kerensikova wrote:
>
> Well actually the requirements were presented in the beginning: I've
> got a S-expression configuration file, a directory of static files, a
> directory of S-expression templates and a directory of S-expression
> data to populate the templates. And I want to respond to HTTP request
> with either a static file or an expanded template. Now I might
> misunderstanding the word "requirements", but what I actually consider
> as requirements is the above without any "S-expression" occurrence,
> the rest being implementation choices. "S-expression" might be part of
> the requirements to use existing data, but then again another format
> can be chosen provided a converted from S-expression is also coded.

Most of what you presented seem more like implementation decisions 
(S-expressions for the configuration information, S-expressions for the page 
templates) than requirements. Even discussion of files and directories might be 
premature, depending on the higher level requirements. A web server that can be 
used in an embedded system as the external interface as well as as a stand-alone 
system would need a more abstract view of permanent storage, for example.

Of course, sometimes implementation decisions are made by others and become part 
of one's requirements, but I wouldn't expect that in a personal project. And the 
existence of an existing library can be an important factor when masking such 
decisions.

I wasn't saying that implementing an S-expression library, or deciding to use it 
at a low level in your project, was a bad thing. I was viewing this project as a 
separate thing from your S-expression library.

> Thanks for the pointer, however I'm already afraid learning only Ada
> is too much for me (I've tried with C++ and it's a language much too
> complex to fit in my head (especially the STL), and I hope Ada is
> simple enough), learning both Ada and AWS at the same time will
> probably be way too much. However I haven't decided yet whether I will
> actually begin with this project, I might try to code something
> simpler before that (which will also need S-expressions, hence my
> beginning with that library). In that case, I might be confident
> enough in my Ada before starting the web project, and then go for AWS.

I think something like AWS would help you learn Ada. You need to look at 
existing code as well as write your own. Starting with the library has good and 
bad points. It's a fairly small, contained project, which is good. On the other 
hand, the first things you write in Ada are probably not going to be great. In a 
few months you'll probably look at the library and want to make significant 
changes to it.

Ada is simpler than C++. It's also fairly well designed, so most features are 
orthogonal, which simplifies learning the language.

> Actually, that's very similar to what I thought too. It's just that
> I've already thought so much about this project that I'm further down
> the road, and I tried to fit everything in the one-dimension of a
> text.

I'm glad to hear that. Perhaps I simply misinterpreted your presentation.

> That's actually a pretty strange thing I've never encountered yet in
> other coders I know, I first spend a lot of time thinking before
> writing the first line of code (or that particular case, before even
> managing to install the compiler). I also spend much less time
> debugging, though I can't tell whether they are related or whether one
> of them is more efficient than the other.

Excellent. The whole point of Ada is thinking before coding, so you'll probably 
find that Ada supports your way of doing things. It's a common experience that 
if you did think first, by the time you get your code to compile, it works 
correctly. Ada people don't do a lot of debugging; I can't remember the last 
time I used a debugger. When we do have problems, they tend to be logic errors 
rather than typos.

> Unless I'm very misunderstanding and/or vary naive, I've got the
> feeling our approaches are not that different, and differ mostly in
> the order of implementation, which indeed doesn't change that much in
> the final result.

You may well be right, in which case you'll likely find Ada to be a good fit for 
your way of approaching things.

-- 
Jeff Carter
"I wave my private parts at your aunties."
Monty Python & the Holy Grail
13

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 8, 10:34 pm, Jeffrey Carter
<spam.jrcarter....@spam.not.acm.org> wrote:
> On 08/08/2010 06:11 AM, Natacha Kerensikova wrote:
> Most of what you presented seem more like implementation decisions
> (S-expressions for the configuration information, S-expressions for the page
> templates) than requirements. Even discussion of files and directories might be
> premature, depending on the higher level requirements. A web server that can be
> used in an embedded system as the external interface as well as as a stand-alone
> system would need a more abstract view of permanent storage, for example.
>
> Of course, sometimes implementation decisions are made by others and become part
> of one's requirements, but I wouldn't expect that in a personal project. And the
> existence of an existing library can be an important factor when masking such
> decisions.

Technically, these are implementation decisions made by an earlier me:
the web server project is intended to be a replacement for an existing
web server, so either I use the same input files, or I write as a part
of the project a converter from the existing format into the new
server data. In both cases, I do have these S-expressions based files,
and I do want to serve HTTP responses based (directly or indirectly)
on them.

> I wasn't saying that implementing an S-expression library, or deciding to use it
> at a low level in your project, was a bad thing. I was viewing this project as a
> separate thing from your S-expression library.

I'm actually viewing it that way too: whatever project I will do next,
it will almost surely rely on some existing S-expression files, so it
make sense to start with general code dealing with S-expressions
before making the final choice about what to do next.

> > Unless I'm very misunderstanding and/or vary naive, I've got the
> > feeling our approaches are not that different, and differ mostly in
> > the order of implementation, which indeed doesn't change that much in
> > the final result.
>
> You may well be right, in which case you'll likely find Ada to be a good fit for
> your way of approaching things.

That's what I think after reading most posts from this threads,
exceptions being mostly Dmitry's posts which often make me feeling
Ada's world is a strange place (to my mind) where I don't belong. I
guess I'll have to dive deeper into Ada before knowing who's right.


Thanks for your advice,
Natacha

Re: S-expression I/O in Ada

[Simon Wright]

I'd disagree with Jeffrey here.

Nothing wrong with stating at the bottom! especially when you already
know that the component you're looking at is likely to be useful and to
fit into *your* way of thinking about things. Your plan already has
higher-level abstractions, so that if you get to the next layer up and
want to change your lowest layer (if only for experimentation's sake)
you will be able to do so.

Lots of people round here are responsible for component libraries at
various levels of abstraction which they've developed for their own
purposes and then pushed out to the community in the hope they'll help
someone else.

The only caution I'd add is that, at some point, when you're reading an
S-expression from an external file and you expect the next 4 bytes to
contain an integer in binary little-endian format, you're going to have
to trust _something_ to have got it right; if you wrote "*(struct foo
**)((char *)whatever + bar.offset)" in C you will have to write the
equivalent in Ada. Unless you were going to invent a sort of checked
S-expression? (I don't get that impression!)

--S

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 08 Aug 2010 11:26:11 +0100, Simon Wright wrote:

> The only caution I'd add is that, at some point, when you're reading an
> S-expression from an external file and you expect the next 4 bytes to
> contain an integer in binary little-endian format,

S-expressions are bad, but not that bad. They have binary data encoded as
hexadecimal strings.

> you're going to have
> to trust _something_ to have got it right; if you wrote "*(struct foo
> **)((char *)whatever + bar.offset)" in C you will have to write the
> equivalent in Ada.

Luckily, Ada makes it difficult to write C equivalents. In Ada a
"non-equivalent" could be:

with Ada.Streams;     use Ada.Streams;
with Interfaces;      use Interfaces;
with Ada.Exceptions;  use Ada.Exceptions;

function Get (Stream : access Root_Stream_Type'Class) return Unsigned_16 is
   Result : Unsigned_16 := 0;
begin
   if '#' /= Character'Input (Stream) then
      Raise_Exception (Syntax_Error'Identity, "Opening '#' is expected");
   end if;
   for Octet in 0..3 loop
      Result :=
         Result + Character'Pos (Character'Input (Stream)) * 2**Octet;
   end loop;
   if '#' /= Character'Input (Stream) then
      Raise_Exception (Syntax_Error'Identity, "Closing '#' is expected");
   end if;
   return Result;
end Get;

> Unless you were going to invent a sort of checked
> S-expression? (I don't get that impression!)

I don't think that were an invention. The program in C or Ada that does not
check the format of the input is broken to me.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 8 Aug 2010 13:44:48 +0200, Dmitry A. Kazakov wrote:

> On Sun, 08 Aug 2010 11:26:11 +0100, Simon Wright wrote:
> 
>> The only caution I'd add is that, at some point, when you're reading an
>> S-expression from an external file and you expect the next 4 bytes to
>> contain an integer in binary little-endian format,
> 
> S-expressions are bad, but not that bad. They have binary data encoded as
> hexadecimal strings.
> 
>> you're going to have
>> to trust _something_ to have got it right; if you wrote "*(struct foo
>> **)((char *)whatever + bar.offset)" in C you will have to write the
>> equivalent in Ada.
> 
> Luckily, Ada makes it difficult to write C equivalents. In Ada a
> "non-equivalent" could be:
> 
> with Ada.Streams;     use Ada.Streams;
> with Interfaces;      use Interfaces;
> with Ada.Exceptions;  use Ada.Exceptions;
> 
> function Get (Stream : access Root_Stream_Type'Class) return Unsigned_16 is

Unsigned_32

>    Result : Unsigned_16 := 0;

Unsigned_32

> begin
>    if '#' /= Character'Input (Stream) then
>       Raise_Exception (Syntax_Error'Identity, "Opening '#' is expected");
>    end if;
>    for Octet in 0..3 loop
>       Result :=
>          Result + Character'Pos (Character'Input (Stream)) * 2**Octet;

2**(Octet*8) of course

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 8, 12:26 pm, Simon Wright <si...@pushface.org> wrote:
> I'd disagree with Jeffrey here.
>
> Nothing wrong with stating at the bottom! especially when you already
> know that the component you're looking at is likely to be useful and to
> fit into *your* way of thinking about things. Your plan already has
> higher-level abstractions, so that if you get to the next layer up and
> want to change your lowest layer (if only for experimentation's sake)
> you will be able to do so.

Thanks a lot for the support \o/

> Lots of people round here are responsible for component libraries at
> various levels of abstraction which they've developed for their own
> purposes and then pushed out to the community in the hope they'll help
> someone else.

I indeed planned to share such a library (assuming I actually write
and finish it), should a generous soul accept to review it. However I
have long lost the hope of seeing my S-expression stuff used, I guess
I can't win against lisp-trauma.

> The only caution I'd add is that, at some point, when you're reading an
> S-expression from an external file and you expect the next 4 bytes to
> contain an integer in binary little-endian format, you're going to have
> to trust _something_ to have got it right; if you wrote "*(struct foo
> **)((char *)whatever + bar.offset)" in C you will have to write the
> equivalent in Ada. Unless you were going to invent a sort of checked
> S-expression? (I don't get that impression!)

Actually that ugly C expression is not a part of my S-expression code.
It's part of a generic self-balancing binary tree interface, supposed
to allow any algorithm as a back end. Because algorithms store
different data, I can't make assumption about the position of children
or balancing data inside the node structure. Therefore I allow the
back-end to provide the offset from the node structure where the
generic tree code can find stuff it needs. Here "whatever" is a void
pointer, pointing to the beginning of the node structure; "bar" is a
structure provided by the back-end; char* cast is needed to perform
byte-based pointer arithmetic, and then the struct foo** cast back to
the real type of the required element.

While I know perfectly what I'm doing with this, I guess it's not
obvious for the majority of C coders. My hope with Ada is that I
wouldn't ever need to write such dubious expressions. In that
particular case, I'm working around C's lack of generics, so it
shouldn't be a problem to express this in Ada. And should I ever need
to write dubious expressions, hope the Ada context would give me the
benefit of doubt long enough to have people read the documents or the
comments and understand there was no other choice; while in C people
wouldn't go further and just label my code and me as ugly and
dangerous.

Regarding the integer encoded as 4 little-endian bytes, I believe it's
pretty safe because S-expression atom are of known length, so if the
length is different than 4 bytes I know there is a problem, and
otherwise I need other constrains on the integer to know whether it's
valid or not. In any case, it doesn't disrupt the reading or
interpretation of the S-expression beyond that particular integer
value.


Thanks again for your support,
Natacha

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/08/2010 03:26 AM, Simon Wright wrote:
> I'd disagree with Jeffrey here.
>
> Nothing wrong with stating at the bottom! especially when you already
> know that the component you're looking at is likely to be useful and to
> fit into *your* way of thinking about things. Your plan already has
> higher-level abstractions, so that if you get to the next layer up and
> want to change your lowest layer (if only for experimentation's sake)
> you will be able to do so.
>
> Lots of people round here are responsible for component libraries at
> various levels of abstraction which they've developed for their own
> purposes and then pushed out to the community in the hope they'll help
> someone else.

As someone who is responsible for a library, I not sure we disagree. Certainly 
when one gets to the point of deciding on a representation, one should implement 
one's choice in a reusable manner if possible, or reuse an existing library; 
often the existence of a library will be an important factor in choosing a 
representation.

Where we may disagree is that I don't think one should begin a project such as 
this by choosing a representation.

-- 
Jeff Carter
"I wave my private parts at your aunties."
Monty Python & the Holy Grail
13

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Yannick Duchêne (Hibou57)]

Le Sat, 07 Aug 2010 09:23:01 +0200, Natacha Kerensikova  
<lithiumcat@gmail.com> a écrit:
> I think text-based format is very useful when the file has to be dealt
> with by both humans and programs. The typical example would be
> configuration files: read and written by humans, and used by the
> program. And that's where I believe XML is really poor, because it's
> too heavy for human use.
I use XML near to every day, I use to write in the raw text editor named  
PSPad, without any troubles. A lot of people write HTML (which like XML is  
a kind of SGML), for years some people wrote in Docbook document modal,  
which again come from SGML. XML can be as simple as S-Expressions are,  
providing you avoid using attributes (if you really feel you do not need  
such a thing). If you feel XML is too much heavy because it requires close  
tag, then just think about it as the Ada's “end if”, “end case”, “end  
loop” and so on. These closes tags enforce human readability and help  
reliability in the sence that one always know what is starting and ending,  
and a close-tag missmatch help to detect an error was done. These all miss  
 from S-Expression (do you know what LISP was named after by the way ? ;) )

I you still feel XML is too much heavy, then just read this :
http://quoderat.megginson.com/2007/01/03/all-markup-ends-up-looking-like-xml/

This demonstrate that as things goes, none of XML, LISP expressions or  
JSon, none is more simple than the other, and LISP expressions or JSon  
serializations just looks simple on very simple cases. When thing goes,  
this is another story.

Did not wanted to make you change your mind (this idea is far from my  
though), rather wanted to make you see there is no way to assert XML is  
too much heavy of LISP expressions are more human readable (I have never  
seen a readable LISP program by the way... I feel this is maid to be read  
by machines, not human).


-- 
There is even better than a pragma Assert: a SPARK --# check.
--# check C and WhoKnowWhat and YouKnowWho;
--# assert Ada;
--  i.e. forget about previous premises which leads to conclusion
--  and start with new conclusion as premise.

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 8/14/10 3:02 AM, Yannick Duchêne (Hibou57) wrote:
> Le Sat, 07 Aug 2010 09:23:01 +0200, Natacha Kerensikova <lithiumcat@gmail.com> a écrit:
>> The typical example would be
>> configuration files: read and written by humans, and used by the
>> program. And that's where I believe XML is really poor, because it's
>> too heavy for human use.
>  XML can be as simple as S-Expressions are, providing you avoid using attributes
>  (if you really feel you do not need such a thing).

>  a close-tag missmatch help to detect an error was done.

Yep, even some Lisps use different pairs of bracket.
These or tags can be read by programs and by humans, too.


> I you still feel XML is too much heavy, then just read this :
> http://quoderat.megginson.com/2007/01/03/all-markup-ends-up-looking-like-xml/

Thanks for the link.  (The authors make me feel less lonesome. ;-)
Just to start from a counterexample,

<tcp-connection host="foo" port="81"/>

Still heavy?

Or maybe XML tags look annoyingly cuspid. The sharp angles can
be smoothed in full SGML where () might supplant <>, thus

(tcp-connection host="foo" port="81" /)

Does this look o.K. to the fans of S-expressions?

Joining the two efforts, aren't the 7-bit ASCII data representation
of Rivest's S-expressions a good idea for safe external representation?
Put S-expression data (atoms) in element content and in attributes in
particular. This helps, I think, with the problem of programmers who
treat encoding with proper condescension.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 14, 3:02 am, Yannick Duchêne (Hibou57)
<yannick_duch...@yahoo.fr> wrote:
> If you feel XML is too much heavy because it requires close  
> tag, then just think about it as the Ada's “end if”, “end case”, “end  
> loop” and so on. These closes tags enforce human readability and help  
> reliability in the sence that one always know what is starting and ending,  
> and a close-tag missmatch help to detect an error was done.

Well, I don't have strong opinions about closing tags. They do clobber
the screen, which is a negative point for readability, but in some
situation knowing what is being closed does help readability. So in
general I won't dare judging whether closing tags are helpful or
harmful. And in some very specific cases, the answer is clear: for a
20-character content without sub-objects, the closing tag doesn't
really add much value, while for a 1000-line unindented complex object
its help surely vastly overpowers its cost.

Now on the ease-of-implementation side, remembering the tag stack and
checking closing-tag match is surely much more complex than matching
any closing parenthesis. Adding a recovery mechanism makes
implementation even more complex (and violate XML standard, which is
the main reason behind my (irrational) hate of XML). This triggers the
question whether the indubitable benefits outweigh the complexity cost
(and back when I chose S-expressions the answer was clearly "no", but
I'm fully aware of how the answer can be different on large and
complex projects unlike mine).

> I you still feel XML is too much heavy, then just read this :http://quoderat.megginson.com/2007/01/03/all-markup-ends-up-looking-l...
>
> This demonstrate that as things goes, none of XML, LISP expressions or  
> JSon, none is more simple than the other, and LISP expressions or JSon  
> serializations just looks simple on very simple cases. When thing goes,  
> this is another story.

Surprisingly, my first reaction to the article you linked was more
along the line "wow, they managed to make S-expression as unreadable
as XML." rather than improving my view of XML.

However with a little reformatting (more line breaks and indentation)
the S-expressions from the article can look much more readable (at
least to my eyes). While I can't do the same thing with his XML, it
doesn't mean it's impossible. Moreover it's probable most XML I've
seen was not designed to be extremely human readable, while S-
expression I've seen are almost all mine, designed to be me-readable.
I guess it goes a long way into making to prejudice that XML is
unreadable.

> Did not wanted to make you change your mind (this idea is far from my  
> though), rather wanted to make you see there is no way to assert XML is  
> too much heavy of LISP expressions are more human readable (I have never  
> seen a readable LISP program by the way... I feel this is maid to be read  
> by machines, not human).

I agree I don't know anymore which is more human readable between XML
and S-expression. Still, XML is heavier to use in programs than S-
expressions, and I still take it for a fact.


Thanks for opening my eyes,
Natacha

Re: S-expression I/O in Ada

[Simon Wright]

Natacha Kerensikova <lithiumcat@gmail.com> writes:

> This means that as far as the S-expression library is concerned, the
> byte sequence read from the file is not typed. Its typing actually has
> to be delayed until the application has enough context to interpret
> it.  Hence the need of a typeless chunk of data.
[...]
> Well, I imagined making such a new type to distinguish 'raw data
> coming from a S-expression to be interpreted' from other raw data
> types.  I thought this was the strong type safety to prevent
> (de)serialization procedure from trying to interpret just any chunk of
> memory.

This sounds really like an alternative implementation of Ada Stream
attributes - ARM 13.13,
http://www.adaic.com/standards/05rm/html/RM-13-13.html .

The idea of a stream is that the data is written (perhaps to a file,
perhaps over a network, perhaps to memory) in suce a way that it can be
retrieved later by the same or another program.

   type R is record
      I : Integer;
      F : Float;
   end record;
   procedure Read
     (Stream : not null access Ada.Streams.Root_Stream_Type'Class;
      Item : out R);
   procedure Write
     (Stream : not null access Ada.Streams.Root_Stream_Type'Class;
      Item : R);
   for R'Read use Read;
   for T'Write use Write;

then implement Read and Write as you wish.

But as you will see the medium - the Stream above - is just a bunch of
bytes in a file or a network packet. If you send a Foo and I read it
hoping it'll be a Bar, we're going to have a problem.

I guess, given the above, we could start the Sexp with "(R ...", then
the recipient would raise Constraint_Error if the type expected didn't
match the data being read.

This is like the mechanism standard Streams (in GNAT, at any rate) use
for transferring tagged and classwide types; the introductory segment
names the provided type and compiler magic creates the proper result.

[Team, I seem to remember an '05 feature that would support this? or was
that my imagination?]

Re: S-expression I/O in Ada

[Pascal Obry]

Simon,

> [Team, I seem to remember an '05 feature that would support this? or was
> that my imagination?]

Yes, I think you are referring to Ada.Tags.Generic_Dispatching_Constructor

For an usage example see:

http://www.adacore.com/2007/11/26/ada-gem-19/

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: S-expression I/O in Ada

[Simon Wright]

Pascal Obry <pascal@obry.net> writes:

> Simon,
>
>> [Team, I seem to remember an '05 feature that would support this? or was
>> that my imagination?]
>
> Yes, I think you are referring to Ada.Tags.Generic_Dispatching_Constructor
>
> For an usage example see:
>
> http://www.adacore.com/2007/11/26/ada-gem-19/
>
> Pascal.

That was it, thanks.

Am I right that this really comes into its own with classwide types?

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova <lithiumcat@gmail.com> writes:
> Hi,
>
> I'm trying to learn Ada, coming from a C background. The first thing I
> planned to code is a S-expression parser, because it's quite easy (at
> least in C, less than 1000 lines including comments and memory
> handling (dynamic arrays reinvented)) and very useful considering
> almost all my existing programs use S-expressions as a serialization
> format.
>
> To describe briefly S-expressions, I consider them to be the simplest
> existing data organization beyond raw sequences of bits. They are
> basically lists of elements, each element being either a list or an
> atom, and atoms being raw sequences of bits.
>
> While I'm still not deep enough into Ada to know how to represent the
> lists, I guess there won't be major issues, I think I can handle it
> myself (though pointers and hints would still be welcome).
>
> My question here is about how to represent the atoms in Ada. In C it
> was merely a void pointer and a size, but it seems more difficult in
> Ada because of the strong typing. Because it's up to the application
> to make sense (i.e. type) out of the raw sequences of bits in atoms,
> the S-expression library has to handle them as a sort of untyped
> memory chunk. Do you know of a way to handle that?
>
> Please correct me if I'm wrong, but my guess would be that the S-
> expression library would provide a Sexp_Atom type, which refers to the
> untyped memory chunks, and the application would have procedures to
> convert back and forth between Sexp_Atom and whatever types it
> internally uses (i.e. serialization and deserialization procedures).
> However the library would provide these procedures for the most common
> types (e.g. strings and numeric types).
>
> Though it looks like a fine Ada API (at least to my eyes), I have
> absolutely no idea about how to implement the library. How to define
> the application-opaque Sexp_Atom type? How to read Sexp_Atom objects
> from a file? How to build them from Ada types? How to write them back
> to disk?

In Ada, you normally model blobs with
System.Storage_Elements.Storage_Array; since arrays are first-class
citizens (as opposed to C's void pointers), you do not need to carry the
length of such an array separately.  Thus, a naive approach might be:

type Sexp_Atom is access System.Storage_Elements.Storage_Array;
type Sexp;
type Sexp_Access is access Sexp;
type Sexp is record
  Car : Sexp_Atom;
  Cdr : Sexp_Access;
end record;

However, the purpose of S-Expressions being to be read and written as
text, a blob may not be the most appropriate; you might be better off
with simply:

type Sexp;
type Sexp_Access is access Sexp;
type Sexp is
  Car : Ada.Strings.Unbounded.Unbounded_String;
  Cdr : Sexp_Access;
  Is_List : Boolean;
end record;

To write a sexp to disk and read back, you would leverage the Ada
streams as Dmitry pointed out.

You could then provide a generic package that serializes an arbitrary
type T back and forth to the unbounded_string.

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Ludovic Brenta]

Ludovic Brenta <ludovic@ludovic-brenta.org> writes on comp.lang.ada:
> Natacha Kerensikova <lithiumcat@gmail.com> writes:
>> Hi,
>>
>> I'm trying to learn Ada, coming from a C background. The first thing I
>> planned to code is a S-expression parser, because it's quite easy (at
>> least in C, less than 1000 lines including comments and memory
>> handling (dynamic arrays reinvented)) and very useful considering
>> almost all my existing programs use S-expressions as a serialization
>> format.
>>
>> To describe briefly S-expressions, I consider them to be the simplest
>> existing data organization beyond raw sequences of bits. They are
>> basically lists of elements, each element being either a list or an
>> atom, and atoms being raw sequences of bits.
>>
>> While I'm still not deep enough into Ada to know how to represent the
>> lists, I guess there won't be major issues, I think I can handle it
>> myself (though pointers and hints would still be welcome).
>>
>> My question here is about how to represent the atoms in Ada. In C it
>> was merely a void pointer and a size, but it seems more difficult in
>> Ada because of the strong typing. Because it's up to the application
>> to make sense (i.e. type) out of the raw sequences of bits in atoms,
>> the S-expression library has to handle them as a sort of untyped
>> memory chunk. Do you know of a way to handle that?
>>
>> Please correct me if I'm wrong, but my guess would be that the S-
>> expression library would provide a Sexp_Atom type, which refers to the
>> untyped memory chunks, and the application would have procedures to
>> convert back and forth between Sexp_Atom and whatever types it
>> internally uses (i.e. serialization and deserialization procedures).
>> However the library would provide these procedures for the most common
>> types (e.g. strings and numeric types).
>>
>> Though it looks like a fine Ada API (at least to my eyes), I have
>> absolutely no idea about how to implement the library. How to define
>> the application-opaque Sexp_Atom type? How to read Sexp_Atom objects
>> from a file? How to build them from Ada types? How to write them back
>> to disk?
>
> In Ada, you normally model blobs with
> System.Storage_Elements.Storage_Array; since arrays are first-class
> citizens (as opposed to C's void pointers), you do not need to carry the
> length of such an array separately.  Thus, a naive approach might be:
>
> type Sexp_Atom is access System.Storage_Elements.Storage_Array;
> type Sexp;
> type Sexp_Access is access Sexp;
> type Sexp is record
>   Car : Sexp_Atom;
>   Cdr : Sexp_Access;
> end record;
>
> However, the purpose of S-Expressions being to be read and written as
> text, a blob may not be the most appropriate; you might be better off
> with simply:
>
> type Sexp;
> type Sexp_Access is access Sexp;
> type Sexp is
>   Car : Ada.Strings.Unbounded.Unbounded_String;
>   Cdr : Sexp_Access;
>   Is_List : Boolean;
> end record;
>
> To write a sexp to disk and read back, you would leverage the Ada
> streams as Dmitry pointed out.
>
> You could then provide a generic package that serializes an arbitrary
> type T back and forth to the unbounded_string.

I pursued that idea a little further and actually wrote an embryonic
S-Expression library.  I'm not entirely satisfied with it because it
uses copy semantics instead of reference semantics and so is probably
inefficient.  But it does demonstrate how to read and write
S-Expressions on a stream (e.g. a file).  Also, it shows how to
hex-encode and hex-decode blobs, which I've modelled as Storage_Arrays.

You can browse the sources here:

http://green.ada-france.org:8081/branch/changes/org.ludovic-brenta.s_expressions

Enjoy.  (the license is the GPLv3 or later).

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 9, 8:49 pm, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:
> I pursued that idea a little further and actually wrote an embryonic
> S-Expression library.  I'm not entirely satisfied with it because it
> uses copy semantics instead of reference semantics and so is probably
> inefficient.  But it does demonstrate how to read and write
> S-Expressions on a stream (e.g. a file).  Also, it shows how to
> hex-encode and hex-decode blobs, which I've modelled as Storage_Arrays.
>
> You can browse the sources here:
>
> http://green.ada-france.org:8081/branch/changes/org.ludovic-brenta.s_...
>
> Enjoy.  (the license is the GPLv3 or later).

Interesting, though it seems your code won't like strings containing a
double-quote.

However, the principal conclusion I draw from reading this is that I'm
still far from having the Ada level required to write anything like
that (since I can't even understand everything while reading it).

Still, from what I understood, I'm not fond of the idea of having to
write type-to-atom conversion procedure while they already exist.

It occurred to me that most types already know how to serialize
themselves, through the Stream subsystem. Strings, Unbounded_strings,
Integers, Float, etc, can all be read from and written into a stream.
So the code to convert them into file-suitable Stream_Element_Array
exist. Won't be a pity not to reuse it?

Pursuing this idea leads to a library quite different from what I had
in mind when I started this thread. Let's call it Sexp_Stream, which
would be derived from Root_Stream_Type. Then Read and Write procedure
are called (from what I understand) by type'Read and type'Write after
said type has turned itself into a Stream_Element_Array. That would
cover atom I/O, and some procedures must be added to cover the list I/
O part.

I think I can imagine how the writing part would happen:
after having initialized a Sexp_Stream object with various
information, including an underlying stream where the S-expression
would be actually written, the application would use the Write
procedure to append an object as an atom to the current node, and the
newly appended atom would become the current node. List procedure
would be called something like List_Open, to open a new list and
append following nodes into it, and List_Close, to close the current
list and come back to the previous list.

Using my tcp-connect example, it would look like this (please excuse
Ada syntax error and focus on the idea):

Sexp_Stream.Open(SxStream, underlying_stream, ...);
Sexp_Stream.Open_List(SxStream);
String'Write(SxStream, "tcp-connect");
Sexp_Stream.Open_List(SxStream);
String'Write(SxStream, "host");
String'Write(SxStream, Hostname);
Sexp_Stream.Close_List(SxStream);
Sexp_Strean.Open_List(SxStream);
String'Write(SxStream, "port");
Integer'Write(SxStream, PortNumber);
Sexp_Stream.Close_List(SxStream);
Sexp_Stream.Close_List(SxStream);

Now I have to admit I don't know how to specify the reading part of
such a Sexp_Stream. I guess I would need the notion of a current node,
with a function telling the application whether the current node is a
list or an atom, type'Read converting the current atom into said type
(and raising an exception when the current atom is a list), and some
procedures to get to the next node, to the frist node following the
current list (i.e. up one level), and when the current node is a list
to go the first node of the list (i.e. one level deeper).

However such a library wouldn't cover all my S-expression needs,
because I sometimes need to keep S-expressions into memory, so there
would be another package for in-memory S-expressions, which would have
to interact nicely with Sexp_Stream.

So, how does it sound? Is it totally crazy? Is it totally not-Ada? Is
there something right in there?

For Jeffry Carter (and anybody interested in helping me understand the
Ada way): here is how it looks like when I haven't thought much about
it. Notice that all this is completely from the point of view of the
application using the package, with very few implementation choices.
If I knew Ada, wouldn't these explanations be pretty much the contents
of the specification file, with about everything from the body being
still to invent? How Ada-ish is that thought process?


Thanks in advance for your reviews of my ideas,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova <lithiumcat@gmail.com> writes:
> On Aug 9, 8:49 pm, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:
>> I pursued that idea a little further and actually wrote an embryonic
>> S-Expression library.  I'm not entirely satisfied with it because it
>> uses copy semantics instead of reference semantics and so is probably
>> inefficient.  But it does demonstrate how to read and write
>> S-Expressions on a stream (e.g. a file).  Also, it shows how to
>> hex-encode and hex-decode blobs, which I've modelled as Storage_Arrays.
>>
>> You can browse the sources here:
>>
>> http://green.ada-france.org:8081/branch/changes/org.ludovic-brenta.s_...
>>
>> Enjoy.  (the license is the GPLv3 or later).
>
> Interesting, though it seems your code won't like strings containing a
> double-quote.

Right; that's a minor detail and easy to correct by e.g. preceding
embedded double-quotes with a backslash.

> However, the principal conclusion I draw from reading this is that I'm
> still far from having the Ada level required to write anything like
> that (since I can't even understand everything while reading it).

I think I could simplify the library with some additional thought.  What
you see is but a first iteration :)

> Still, from what I understood, I'm not fond of the idea of having to
> write type-to-atom conversion procedure while they already exist.
>
> It occurred to me that most types already know how to serialize
> themselves, through the Stream subsystem. Strings, Unbounded_strings,
> Integers, Float, etc, can all be read from and written into a stream.
> So the code to convert them into file-suitable Stream_Element_Array
> exist. Won't be a pity not to reuse it?
>
> Pursuing this idea leads to a library quite different from what I had
> in mind when I started this thread. Let's call it Sexp_Stream, which
> would be derived from Root_Stream_Type. Then Read and Write procedure
> are called (from what I understand) by type'Read and type'Write after
> said type has turned itself into a Stream_Element_Array. That would
> cover atom I/O, and some procedures must be added to cover the list I/
> O part.
>
> I think I can imagine how the writing part would happen:
> after having initialized a Sexp_Stream object with various
> information, including an underlying stream where the S-expression
> would be actually written, the application would use the Write
> procedure to append an object as an atom to the current node, and the
> newly appended atom would become the current node. List procedure
> would be called something like List_Open, to open a new list and
> append following nodes into it, and List_Close, to close the current
> list and come back to the previous list.
>
> Using my tcp-connect example, it would look like this (please excuse
> Ada syntax error and focus on the idea):
>
> Sexp_Stream.Open(SxStream, underlying_stream, ...);
> Sexp_Stream.Open_List(SxStream);
> String'Write(SxStream, "tcp-connect");
> Sexp_Stream.Open_List(SxStream);
> String'Write(SxStream, "host");
> String'Write(SxStream, Hostname);
> Sexp_Stream.Close_List(SxStream);
> Sexp_Strean.Open_List(SxStream);
> String'Write(SxStream, "port");
> Integer'Write(SxStream, PortNumber);
> Sexp_Stream.Close_List(SxStream);
> Sexp_Stream.Close_List(SxStream);
>
> Now I have to admit I don't know how to specify the reading part of
> such a Sexp_Stream. I guess I would need the notion of a current node,
> with a function telling the application whether the current node is a
> list or an atom, type'Read converting the current atom into said type
> (and raising an exception when the current atom is a list), and some
> procedures to get to the next node, to the frist node following the
> current list (i.e. up one level), and when the current node is a list
> to go the first node of the list (i.e. one level deeper).
>
> However such a library wouldn't cover all my S-expression needs,
> because I sometimes need to keep S-expressions into memory, so there
> would be another package for in-memory S-expressions, which would have
> to interact nicely with Sexp_Stream.

That's exactly what my library is for: it lets you build S-Expressions
in memory and serialize them to a stream (adding any parentheses or
quoting necessary along the way) and, conversely, read from a stream and
build in-memory S-Expressions.  At this level, the value of any atom is
an unbounded_string; the semantics are left to the application.

The To_Atom and From_Atom functions are there to help you build a
high-level layer on top of the S-Expression layer.  The "test.adb"
program contains such a higher layer: it defines a (simple) record type
containing an integer and a boolean (enumerated type) and encodes it
into an S-Expression containing strings, and back.

I'll add a generic to encode arbitrary hashed_maps into S-Expressions of
the form ((key value) (key value) ...) later on, if you're interested.

> So, how does it sound? Is it totally crazy? Is it totally not-Ada? Is
> there something right in there?

I don't think "Ada" or "not Ada" should be the question; the proper
question should be "is it reusable and maintainable?"

> For Jeffry Carter (and anybody interested in helping me understand the
> Ada way): here is how it looks like when I haven't thought much about
> it. Notice that all this is completely from the point of view of the
> application using the package, with very few implementation choices.
> If I knew Ada, wouldn't these explanations be pretty much the contents
> of the specification file, with about everything from the body being
> still to invent? How Ada-ish is that thought process?
>
>
> Thanks in advance for your reviews of my ideas,
> Natacha

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Jeffrey Carter]

> Natacha Kerensikova<lithiumcat@gmail.com>  writes:
>
> For Jeffry Carter (and anybody interested in helping me understand the
> Ada way): here is how it looks like when I haven't thought much about
> it. Notice that all this is completely from the point of view of the
> application using the package, with very few implementation choices.
> If I knew Ada, wouldn't these explanations be pretty much the contents
> of the specification file, with about everything from the body being
> still to invent? How Ada-ish is that thought process?

In general, describing it from the point of view of "the application using the 
package" (often referred to as "the client") is the purpose of the 
specification. The body is for the implementation, usually best left until the 
specification is finished.

On the other hand, your proposed operations seem unnecessarily low level for the 
client. You have an internal representation, which is probably hidden from the 
client, and an external representation, and operations to write an object of the 
internal representation to that external representation and read the external 
representation to create an object of the internal representation. Thus, your 
client should probably invoke a single operation to write an object, and another 
single operation to read.

-- 
Jeff Carter
"Run away! Run away!"
Monty Python and the Holy Grail
58

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 10, 2:57 am, Jeffrey Carter
<spam.jrcarter....@spam.not.acm.org> wrote:
> > Natacha Kerensikova<lithium...@gmail.com>  writes:
>
> > For Jeffry Carter (and anybody interested in helping me understand the
> > Ada way): here is how it looks like when I haven't thought much about
> > it. Notice that all this is completely from the point of view of the
> > application using the package, with very few implementation choices.
> > If I knew Ada, wouldn't these explanations be pretty much the contents
> > of the specification file, with about everything from the body being
> > still to invent? How Ada-ish is that thought process?
>
> In general, describing it from the point of view of "the application using the
> package" (often referred to as "the client") is the purpose of the
> specification. The body is for the implementation, usually best left until the
> specification is finished.

So I was doing it the right way, right? (I still have almost no clue
about how to make the body, while the specification looks relatively
clear to me, even though I can't yet turn in into real Ada).

> On the other hand, your proposed operations seem unnecessarily low level for the
> client. You have an internal representation, which is probably hidden from the
> client, and an external representation, and operations to write an object of the
> internal representation to that external representation and read the external
> representation to create an object of the internal representation. Thus, your
> client should probably invoke a single operation to write an object, and another
> single operation to read.

I have to admit I can't imagine how to do it in any higher level than
this. Or are you suggesting to replace my sequence of procedure calls
by something like:

NewList(AtomFromString("tcp-connect",
     NewList(AtomFromString("host", AtomFromString("foo,example",
Nil)),
     NewList(AtomFromString("port", AtomFromPort(PortNb, Nil)), Nil)),
   Nil);

Or maybe we don't the same thing with "object"? Here I have five
objects to put into a S-expression as atoms, and they are indeed
fairly low-level objects. A higher-level object would be a record type
called e.g. TCP_Info, whose reading and writing primitives handle the
addition/matching of "host" and "port" and maybe "tcp-connect". So
there would be a single operation to read or write such a high-level
object by code using it. However the mid-level TCP_Whatever package
providing the high-level object and its operation would in turn use
the low-level Sexp_Stream (or anything else I might want to experiment
with, that's the whole point of modularity).

Would you mind hinting me about to do specify my library in a better
way?


Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/09/2010 11:47 PM, Natacha Kerensikova wrote:
>
> So I was doing it the right way, right? (I still have almost no clue
> about how to make the body, while the specification looks relatively
> clear to me, even though I can't yet turn in into real Ada).

Yes.

> I have to admit I can't imagine how to do it in any higher level than
> this. Or are you suggesting to replace my sequence of procedure calls
> by something like:
>
> NewList(AtomFromString("tcp-connect",
>       NewList(AtomFromString("host", AtomFromString("foo,example",
> Nil)),
>       NewList(AtomFromString("port", AtomFromPort(PortNb, Nil)), Nil)),
>     Nil);

I was commenting on your ideas about writing an S-expression to a stream, which 
included operations to open and close a list, and put an atom. I thought this 
was how you expected a client to output an S-expression. I think the client 
should just do

S : S_Expression := Some_Initialization;

Put (File => F, Expression => S);

> Or maybe we don't the same thing with "object"? Here I have five
> objects to put into a S-expression as atoms, and they are indeed
> fairly low-level objects. A higher-level object would be a record type
> called e.g. TCP_Info, whose reading and writing primitives handle the
> addition/matching of "host" and "port" and maybe "tcp-connect". So
> there would be a single operation to read or write such a high-level
> object by code using it. However the mid-level TCP_Whatever package
> providing the high-level object and its operation would in turn use
> the low-level Sexp_Stream (or anything else I might want to experiment
> with, that's the whole point of modularity).

Right. Here the program would call the single operation, and never look at an 
S-expression.

Your TCP_Info-handling pkg would convert the record into an S-expression, and 
call a single operation from your S-expression pkg to output the S-expression.

Your S-expression library would implement the writing of the expression as a 
series of steps.

The TCP_Info-handling pkg is the client of the S-expression library; the program 
is the client of the TCP_Info-handling pkg. I was referring to the former. You 
were apparently talking about the internals of the S-expression library.

-- 
Jeff Carter
"Strange women lying in ponds distributing swords
is no basis for a system of government."
Monty Python & the Holy Grail
66

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 10, 8:13 pm, Jeffrey Carter
<spam.jrcarter....@spam.not.acm.org> wrote:
> On 08/09/2010 11:47 PM, Natacha Kerensikova wrote:
> I was commenting on your ideas about writing an S-expression to a stream, which
> included operations to open and close a list, and put an atom. I thought this
> was how you expected a client to output an S-expression.

You thought right, except that due to my discussion with Dmitry I
ended up considering the possibility of splitting my first idea into
two different packages.

Sexp_Stream is supposed to perform S-expression I/O over streams,
without ever constructing a S-expression into memory. It is supposed
to do only the encoding and decoding of S-expression format, to expose
its clients only atoms and relations.

A further unnamed yet package would handle the memory representation
of S-expressions, which involve object creation, memory management and
things like that. It would be a client of Sexp_Stream for I/O, so the
parsing would only be done at one place (namely Sexp_Stream). As I
said Ludovic Brenta's code might take this place.

There are situations when a memory representation of S-expressions is
not needed, and the tcp-connect example here seems to be such a case.
That's why I imagined TCP_Info as a client of Sexp_Stream instead of a
client of the second package.

The stuff with operations to put an atom and open and close a list is
indeed how I think a client of Sexp_Stream should behave. And if I
want Sexp_Stream to have no in-memory representation of a S-
expression, I can't imagine another way of specifying it.

However I'm still undecided whether it's a good thing to have a
Sexp_Stream without memory representation, and whether packages like
TCP_Info should be a client of Sexp_Strean or of the in-memory S-
expression package.

> I think the client should just do
>
> S : S_Expression := Some_Initialization;
>
> Put (File => F, Expression => S);

To achieve such an interface, the client has to build an in-memory S-
expression object. In the tcp-connect example, there are eight nodes
(five atoms and three lists). They have to be somehow built, and it
doesn't look like a simple initialization.

The second interface I proposed, with a lot of nested calls, builds
the S-expression object with functions looking like:
function NewList(Contents, Next: S_node_access) return S_node_access;
function AtomFromWhatever(Contents: whatever, Next: S_node_access)
return S_node_access;

However, unless I'm building other higher-level functions, to build 8
nodes I need 8 function calls, exactly like the second example.

That 8-function-call example looks very C-ish to me, and honestly I
don't like it, I prefer the sequence of procedure calls from
Sexp_Stream example, but that's just a personal taste.

The problem is that I don't see how to do it otherwise. Any idea would
be welcome.

For the TCP_info stuff particular case, the only simplification I can
imagine is leveraging the fact that only 2 among the 8 nodes change.
So one could construct a global S-expression skeleton with the 6
constant nodes, and when actually writing data append to the "host"
and "port" atoms the relevant variable atoms. However that means
having a read/write global variable, which is bad for tasking, so one
might prefer a constant 6-node skeleton, duplicated in the declarative
part of the writing procedure, and then appending the variable nodes.
However that appending would probably end up with lines like:
root_sexp.FirstChild.Next.FirstChild.Append(host_node);
root_Sexp.FirstChild.Next.Next.FirstChild.Append(port_node);
which looks very ugly to me.

> Your TCP_Info-handling pkg would convert the record into an S-expression, and
> call a single operation from your S-expression pkg to output the S-expression.

That's the tricky part. At least so tricky that I can't imagine how to
do it properly.

> Your S-expression library would implement the writing of the expression as a
> series of steps.

Indeed, as a series of calls similar to my Sexp_Stream example. I'm
glad to have at least that part right.



Thanks for your comments,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova wrote on comp.lang.ada:
[...]
> Sexp_Stream is supposed to perform S-expression I/O over streams,
> without ever constructing a S-expression into memory. It is supposed
> to do only the encoding and decoding of S-expression format, to expose
> its clients only atoms and relations.

But how can it expose atoms and relations without an in-memory tree
representation? Honestly, I do not think that such a concept is
viable. Hence, the "S-Expression stream" must actually depend on the
in-memory tree representation, both for reading and for writing. That
is whay my package does.

> A further unnamed yet package would handle the memory representation
> of S-expressions, which involve object creation, memory management and
> things like that. It would be a client of Sexp_Stream for I/O, so the
> parsing would only be done at one place (namely Sexp_Stream). As I
> said Ludovic Brenta's code might take this place.

No, it replaces both of your hypothetical packages and I don't think
you can have the "stream" package without the "S-Expression" package.
You could, however, have an S-Expression package without any I/O.

> There are situations when a memory representation of S-expressions is
> not needed, and the tcp-connect example here seems to be such a case.
> That's why I imagined TCP_Info as a client of Sexp_Stream instead of a
> client of the second package.

If no S-Expressions are needed in memory, then your entire data is
just one big unstructured String, and you cannot parse it into a
TCP_Info record unless you re-introduce the S-Expression tree in
memory.

>> I think the client should just do
>>
>> S : S_Expression := Some_Initialization;
>>
>> Put (File => F, Expression => S);
>
> To achieve such an interface, the client has to build an in-memory S-
> expression object. In the tcp-connect example, there are eight nodes
> (five atoms and three lists). They have to be somehow built, and it
> doesn't look like a simple initialization.
>
> The second interface I proposed, with a lot of nested calls, builds
> the S-expression object with functions looking like:
> function NewList(Contents, Next: S_node_access) return S_node_access;
> function AtomFromWhatever(Contents: whatever, Next: S_node_access)
> return S_node_access;

That's what I do in the test.adb example (using Cons () and To_Atom ()
respectively).

> However, unless I'm building other higher-level functions, to build 8
> nodes I need 8 function calls, exactly like the second example.
>
> That 8-function-call example looks very C-ish to me, and honestly I
> don't like it, I prefer the sequence of procedure calls from
> Sexp_Stream example, but that's just a personal taste.
>
> The problem is that I don't see how to do it otherwise. Any idea would
> be welcome.

Here is one idea: embed the S-Expression "language" in Ada:

function To_Sexp (S : in String) return S_Expression.T is
  String_Stream : constant Ada.Streams.Stream_IO.Stream_Access :=
     Stream_Reading_From (S); -- implementation left as an exercise :)
begin
   return Result : S_Expression.T do
      S_Expression.T'Read (String_Stream, Result);
   end return;
end To_Sexp;

TCP_Info : constant String := "(tcp-connect (host foo.bar) (port
80))";
TCP_Info_Structured := constant To_TCP_Info (To_Sexp (TCP_Info));

(this is based on my implementation).

>> Your TCP_Info-handling pkg would convert the record into an S-expression, and
>> call a single operation from your S-expression pkg to output the S-expression.
>
> That's the tricky part. At least so tricky that I can't imagine how to
> do it properly.

Yes, the S_Expression.Read operation is quite involved. But it can be
done, and has been done :)

--
Ludovic Brenta.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 12, 12:55 pm, Ludovic Brenta <ludo...@ludovic-brenta.org>
wrote:
> Natacha Kerensikova wrote on comp.lang.ada:
> [...]
>
> > Sexp_Stream is supposed to perform S-expression I/O over streams,
> > without ever constructing a S-expression into memory. It is supposed
> > to do only the encoding and decoding of S-expression format, to expose
> > its clients only atoms and relations.
>
> But how can it expose atoms and relations without an in-memory tree
> representation? Honestly, I do not think that such a concept is
> viable.

I have already showing how to write S-expressions to a stream without
in-memory representations of the S-expression (thought I still need an
in-memory representation of atoms, and I believe this can't be worked
around), using an interface like this:

procedure OpenList(sstream: in out Sexp_Stream);
  -- more or less equivalent to output a '(' into the underlying
stream
procedure CloseList(sstream: in out Sexp_Stream);
  -- more or less equivalent to output a ')' into the underlying
stream
procedure PutAtom(sstream: in out Sexp_Stream, atom: in octet_array);
  -- encode the atom and send it into the underlying stream

I guess it will also need some functions to convert usual types to and
from Octet_Array.

The reading part is a bit more tricky, and I admitted when I proposed
Sexp_Stream I didn't know how to make it. Having thought (maybe too
much) since then, here is what the interface might look like:

type Node_Type is (S_None, S_List, S_Atom);

function Current_Node_Type(sstream: in Sexp_Stream) return Atom_Type;

procedure Get_Atom(sstream: in Sexp_Stream, contents: out
octet_array);
  -- raises an exception when Current_Node_Type is not S_Atom
  -- not sure "out octet_array" works, but that's the idea
  --   maybe turn it into a function for easier atom-to-object
conversion

procedure Move_Next(sstream: in out Sexp_Stream);
  -- when the client is finished with the current node, it calls this
  -- procedure to update stream internal state to reflect the next
node in
  -- list

procedure Move_Lower(sstream: in out Sexp_Stream);
  -- raises an exception when Current_Node_Type is not S_List
  -- update the internal state to reflect the first child of the
current list

procedure Move_Upper(sstream: in out Sexp_Stream);
  -- update the internal state to reflect the node following the list
  -- containing the current node. sortof "uncle" node
  -- the implementation is roughly skipping whatever nodes exist in
the
  -- current list until reading its ')' and reading the following node


Such an interface support data streams, i.e. reading new data without
seek or pushback. The data would probably be read octet-by-octet
(unless reading a verbatim encoded atom), hoping for an efficient
buffering in the underlying stream. If that's a problem I guess some
buffering can be transparently implemented in the private part of
Sexp_Stream.

Of course atoms are to be kept in memory, which means Sexp_Stream will
have to contain a dynamic array (Vector in Ada terminology, right?)
populated from the underlying stream until the atom is completely
read. Get_Atom would hand over a (static) array built from the private
dynamic array.

The implementation would for example rely on a procedure to advance
the underlying to the next node (i.e. skipping white space), and from
the first character know whether it's the beginning of a new list
(when '(' is encountered) and update the state to S_List, and it's
finished; whether it's the end of the list (when ')' is encountered)
and update the state to S_None; or whether it's the beginning of an
atom, so read it completely, updating the internal dynamic array and
setting the state to S_Atom.

Well actually, it's probably not the best idea, I'm not yet clear
enough on the specifications and on stream I/O to clearly think about
implementation, but that should be enough to make you understand the
idea of S-expression reading without S-expression objects.


Now regarding the actual use of this interface, I think my previous
writing example is enough, so here is the reading part.

My usual way of reading S-expression configuration file is to read
sequentially, one at a time, a list whose first element is an atom.
Atoms, empty lists and lists beginning with a list are considered as
comments and silently dropped. "(tcp-connect …)" is meant to be one of
these, processed by TCP_Info's client, which will hand over only the
"…" part to TCP_Info.Read (or other initialization subprogram). So
just like TCP_Info's client processes something like "(what-ever-
config …) (tcp-connect …) (other-config …)", TCP_Info.Read will
process only "(host foo.example) (port 80)".

So TCP_Info's client, after having read "tcp-connect" atom, will call
Move_Next on the Sexp_Stream and pass it to TCP_Info.Read. Then
TCP_Info.Read proceeds with:

loop
  case Current_Atom_Type(sstream) is
     when S_None => return;  -- TCP_Info configuration is over
     when S_Atom => null;    -- silent atom drop
     when S_List =>
       Move_Lower(sstream);
       Get_Atom(sstream, atom);
       -- make command of type String from atom
       -- if Get_Atom was successful
       Move_Next(sstream);
       if command = "host" then
         -- Get_Atom and turn it into host string
       elif command = "port" then
         -- Get_Atom and turn it into port number
       else
         -- complain about unrecognized command
       end if;
       Move_Upper(sstream);
  end case;
end loop;

TCP_Info's client S-expression parsing would be very similar, except
if command = … would be followed by a call to TCP_Info.Read rather
than a Get_Atom.

So where are the problems with my Sexp_Stream without in memory
object? What am I missing?

Or is it so ugly and insane that it should be kept in C's realm?

> > A further unnamed yet package would handle the memory representation
> > of S-expressions, which involve object creation, memory management and
> > things like that. It would be a client of Sexp_Stream for I/O, so the
> > parsing would only be done at one place (namely Sexp_Stream). As I
> > said Ludovic Brenta's code might take this place.
>
> No, it replaces both of your hypothetical packages and I don't think
> you can have the "stream" package without the "S-Expression" package.

Yes, I indeed realized this mistake short after having send the
message. My bad.

> You could, however, have an S-Expression package without any I/O.

Indeed, though I have to admit I have trouble imagining what use it
can have.

> TCP_Info : constant String := "(tcp-connect (host foo.bar) (port
> 80))";
> TCP_Info_Structured := constant To_TCP_Info (To_Sexp (TCP_Info));

That's an interesting idea, which conceptually boils down to serialize
by hand the S-expression into a String, in order to unserialize it as
an in-memory object, in order to serialize back into a Stream.

Proofreading my post, the above might sound sarcastic though actually
it is not. It's a twist I haven't thought of, but it might indeed turn
out to be the simplest practical way of doing it.

Actually for a very long time I used to write S-expressions to file
using only string facilities and a special sx_print_atom() function to
handle escaping of unsafe data. By then I would have handled
TCP_Info.Write with the following C fragment (sorry I don't know yet
how to turn it into Ada, but I'm sure equivalent as simple exists):

fprintf(outfile, "(tcp-connect\n\t(host ");
sx_print_atom(outfile, host);
fprintf(outfile, ")\n\t(port %d)\n)\n", port);

> >> Your TCP_Info-handling pkg would convert the record into an S-expression, and
> >> call a single operation from your S-expression pkg to output the S-expression.
>
> > That's the tricky part. At least so tricky that I can't imagine how to
> > do it properly.
>
> Yes, the S_Expression.Read operation is quite involved. But it can be
> done, and has been done :)

Actually what I called "tricky" was the "single operation" part of
TCP_Info's job. As I said, 8 nodes to build and send, it's not that
easy to fit in a single operation, though your tick performs it quite
well.

By comparison the S-expression reading from a stream feels less tricky
to me, though a bit more tiring, and has indeed been done, even by me
(if you can read C, that the second half (lines 417 to the end) of
http://git.instinctive.eu/cgit/libnathandbag/tree/csexp.c ).


Thanks for your comments and your implementation,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova wrote on comp.lang.ada:
> On Aug 12, 12:55 pm, Ludovic Brenta <ludo...@ludovic-brenta.org>
> wrote:
>
>> Natacha Kerensikova wrote on comp.lang.ada:
>> [...]
>
>>> Sexp_Stream is supposed to perform S-expression I/O over streams,
>>> without ever constructing a S-expression into memory. It is supposed
>>> to do only the encoding and decoding of S-expression format, to expose
>>> its clients only atoms and relations.
>
>> But how can it expose atoms and relations without an in-memory tree
>> representation? Honestly, I do not think that such a concept is
>> viable.
[...]
> The reading part is a bit more tricky, and I admitted when I proposed
> Sexp_Stream I didn't know how to make it. Having thought (maybe too
> much) since then, here is what the interface might look like:
>
> type Node_Type is (S_None, S_List, S_Atom);
>
> function Current_Node_Type(sstream: in Sexp_Stream) return Atom_Type;
>
> procedure Get_Atom(sstream: in Sexp_Stream, contents: out
> octet_array);
>   -- raises an exception when Current_Node_Type is not S_Atom
>   -- not sure "out octet_array" works, but that's the idea
>   --   maybe turn it into a function for easier atom-to-object
> conversion
>
> procedure Move_Next(sstream: in out Sexp_Stream);
>   -- when the client is finished with the current node, it calls this
>   -- procedure to update stream internal state to reflect the next
> node in
>   -- list
>
> procedure Move_Lower(sstream: in out Sexp_Stream);
>   -- raises an exception when Current_Node_Type is not S_List
>   -- update the internal state to reflect the first child of the
> current list
>
> procedure Move_Upper(sstream: in out Sexp_Stream);
>   -- update the internal state to reflect the node following the list
>   -- containing the current node. sortof "uncle" node
>   -- the implementation is roughly skipping whatever nodes exist in
> the
>   -- current list until reading its ')' and reading the following node
>
> Such an interface support data streams, i.e. reading new data without
> seek or pushback. The data would probably be read octet-by-octet
> (unless reading a verbatim encoded atom), hoping for an efficient
> buffering in the underlying stream. If that's a problem I guess some
> buffering can be transparently implemented in the private part of
> Sexp_Stream.
>
> Of course atoms are to be kept in memory, which means Sexp_Stream will
> have to contain a dynamic array (Vector in Ada terminology, right?)
> populated from the underlying stream until the atom is completely
> read. Get_Atom would hand over a (static) array built from the private
> dynamic array.
>
> The implementation would for example rely on a procedure to advance
> the underlying to the next node (i.e. skipping white space), and from
> the first character know whether it's the beginning of a new list
> (when '(' is encountered) and update the state to S_List, and it's
> finished; whether it's the end of the list (when ')' is encountered)
> and update the state to S_None; or whether it's the beginning of an
> atom, so read it completely, updating the internal dynamic array and
> setting the state to S_Atom.
>
> Well actually, it's probably not the best idea, I'm not yet clear
> enough on the specifications and on stream I/O to clearly think about
> implementation, but that should be enough to make you understand the
> idea of S-expression reading without S-expression objects.
>
> Now regarding the actual use of this interface, I think my previous
> writing example is enough, so here is the reading part.
>
> My usual way of reading S-expression configuration file is to read
> sequentially, one at a time, a list whose first element is an atom.
> Atoms, empty lists and lists beginning with a list are considered as
> comments and silently dropped. "(tcp-connect …)" is meant to be one of
> these, processed by TCP_Info's client, which will hand over only the
> "…" part to TCP_Info.Read (or other initialization subprogram). So
> just like TCP_Info's client processes something like "(what-ever-
> config …) (tcp-connect …) (other-config …)", TCP_Info.Read will
> process only "(host foo.example) (port 80)".
>
> So TCP_Info's client, after having read "tcp-connect" atom, will call
> Move_Next on the Sexp_Stream and pass it to TCP_Info.Read. Then
> TCP_Info.Read proceeds with:
>
> loop
>   case Current_Atom_Type(sstream) is
>      when S_None => return;  -- TCP_Info configuration is over
>      when S_Atom => null;    -- silent atom drop
>      when S_List =>
>        Move_Lower(sstream);
>        Get_Atom(sstream, atom);
>        -- make command of type String from atom
>        -- if Get_Atom was successful
>        Move_Next(sstream);
>        if command = "host" then
>          -- Get_Atom and turn it into host string
>        elif command = "port" then
>          -- Get_Atom and turn it into port number
>        else
>          -- complain about unrecognized command
>        end if;
>        Move_Upper(sstream);
>   end case;
> end loop;
>
> TCP_Info's client S-expression parsing would be very similar, except
> if command = … would be followed by a call to TCP_Info.Read rather
> than a Get_Atom.
>
> So where are the problems with my Sexp_Stream without in memory
> object? What am I missing?

The "problem" is that, without admitting it, you have reintroduced a
full S-Expression parser. Most of it is hidden in the Sexp_Stream
implementation, but it has to be there. Otherwise, how can the
Move_Lower, Advance, and Move_Upper operations work, keeping track of
how many levels deep you are at all times?

Note also that your TCP_Info.Read looks quite similar to mine, except
that mine takes an S-Expression as the input, rather than a stream.
Afterwards, it traverses the S-Expression using pretty much the same
algorithm as yours. The S-Expression itself comes from the stream. So,
the only difference between your concept and my implementation is that
I expose the S-Expression memory tree and you don't.

The reason why I prefer to expose the S-Expression is because, in the
general (arbitrarily complex) case, you cannot traverse an S-
Expresssion linearly; you need to traverse it as what it really is, a
tree. A stream suggests linear traversal only.

[...]
>> TCP_Info : constant String := "(tcp-connect (host foo.bar) (port 80))";
>> TCP_Info_Structured := constant To_TCP_Info (To_Sexp (TCP_Info));
>
> That's an interesting idea, which conceptually boils down to serialize
> by hand the S-expression into a String, in order to unserialize it as
> an in-memory object, in order to serialize back into a Stream.
>
> Proofreading my post, the above might sound sarcastic though actually
> it is not. It's a twist I haven't thought of, but it might indeed turn
> out to be the simplest practical way of doing it.

Right. I was not being sarcastic either. The Cons (), To_Atom () and
Append () operations are needed only when creating arbitrary and
dynamic S-Expressions. For simple cases where most of the expression
is hardcoded, the textual representation of the S-Expression is much
more compact, readable and maintainable than the Ada procedural
representation. In fact, you could also conceivably write something
like:

TCP_Info_Sexp : S_Expression := To_Sexp ("(tcp-info (host *) (port
*))");

and programmatically change the values of the atoms containing the
actual data. Once you have the in-memory S-Expression, there is no
limit to what you can do with it. You can *change* the S-Expression as
you traverse it, deleting, replacing or adding nodes as you wish. You
cannot do that with a Sexp_Stream.

> Actually for a very long time I used to write S-expressions to file
> using only string facilities and a special sx_print_atom() function to
> handle escaping of unsafe data. By then I would have handled
> TCP_Info.Write with the following C fragment (sorry I don't know yet
> how to turn it into Ada, but I'm sure equivalent as simple exists):
>
> fprintf(outfile, "(tcp-connect\n\t(host ");
> sx_print_atom(outfile, host);
> fprintf(outfile, ")\n\t(port %d)\n)\n", port);

Sure, that can also be done just as easily in Ada. You can write S-
Expressions as easily as any blob or string; it is reading them back
and understanding their structure that is tricky.

--
Ludovic Brenta.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 12, 2:46 pm, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:
> Natacha Kerensikova wrote on comp.lang.ada:
> > So where are the problems with my Sexp_Stream without in memory
> > object? What am I missing?
>
> The "problem" is that, without admitting it, you have reintroduced a
> full S-Expression parser. Most of it is hidden in the Sexp_Stream
> implementation, but it has to be there.

Actually, I *do* admit it. That's even the whole point of Sexp_Stream:
it does the parsing, the whole parsing, and nothing more.

As soon as I realized (thanks to Dmitry) that the parser can be
meaningfully split from everything else (memory management, object
construction, etc), I was seduced by the idea.

> Otherwise, how can the
> Move_Lower, Advance, and Move_Upper operations work, keeping track of
> how many levels deep you are at all times?

In my idea it doesn't keep track of level depth, though if needed (for
example, to assert Move_Lower and Move_Upper are correctly balanced)
it can be easily added, using and integer in the Sexp_Stream object
that is incremented when moving up and decremented when moving down.
That way it can trigger an exception when encountering an unmatched
')' (the other mismatch being already handled by end-of-stream
exception), though I prefer silently dropping them (though it's a
personal preference, it's probably better design to raise the
exception in the library and catch and drop it in the application if
I'm so inclined).

> Note also that your TCP_Info.Read looks quite similar to mine, except
> that mine takes an S-Expression as the input, rather than a stream.
> Afterwards, it traverses the S-Expression using pretty much the same
> algorithm as yours. The S-Expression itself comes from the stream. So,
> the only difference between your concept and my implementation is that
> I expose the S-Expression memory tree and you don't.

Exactly. It's also very similar to my C clients, which all work on in-
memory S-expression objects.

> The reason why I prefer to expose the S-Expression is because, in the
> general (arbitrarily complex) case, you cannot traverse an S-
> Expresssion linearly; you need to traverse it as what it really is, a
> tree. A stream suggests linear traversal only.

Yes, linear traversal only is indeed a limitation of Sexp_Stream.
That's the reason why I had in mind another package of object and
memory handling, which would rely on Sexp_Stream for the parsing.

I prefer (for now) the Sexp_Stream/memory-object split because 1. it
makes thing more modular, which means more possibilities of localized
changes and of implementation overhaul without changing specification,
and 2. in my real-life application the linear traversal is often
enough. Far from always though, especially when one counts cases when
application can traverse linearly but when of the subobjects needs to
store its sub-S-expression in memory.

> > Actually for a very long time I used to write S-expressions to file
> > using only string facilities and a special sx_print_atom() function to
> > handle escaping of unsafe data. By then I would have handled
> > TCP_Info.Write with the following C fragment (sorry I don't know yet
> > how to turn it into Ada, but I'm sure equivalent as simple exists):
>
> > fprintf(outfile, "(tcp-connect\n\t(host ");
> > sx_print_atom(outfile, host);
> > fprintf(outfile, ")\n\t(port %d)\n)\n", port);
>
> Sure, that can also be done just as easily in Ada. You can write S-
> Expressions as easily as any blob or string;

And even nowadays I still wonder, all things considered, whether it
might be the best way of doing it after all.

As far as I know the three lines above is the simplest way of writing
an S-expression, and often have a hard time finding justifications of
using a more complex way. The only added value I found to S-expression
writing facilities over String I/O is ensuring the written S-
expression is actually what I meant, with parenthesis correctly
matching and no stray character or anything messing up the parsing. As
it never happened to me despite my quite heavy use of S-expressions, I
often doubt the necessity of such extra checks, especially compared to
their complexity-cost.

> it is reading them back
> and understanding their structure that is tricky.

I completely agree.


Thanks for your discussion,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova wrote on comp.lang.ada:
>> The reason why I prefer to expose the S-Expression is because, in the
>> general (arbitrarily complex) case, you cannot traverse an S-
>> Expresssion linearly; you need to traverse it as what it really is, a
>> tree. A stream suggests linear traversal only.
>
> Yes, linear traversal only is indeed a limitation of Sexp_Stream.
> That's the reason why I had in mind another package of object and
> memory handling, which would rely on Sexp_Stream for the parsing.
>
> I prefer (for now) the Sexp_Stream/memory-object split because 1. it
> makes thing more modular, which means more possibilities of localized
> changes and of implementation overhaul without changing specification,
> and 2. in my real-life application the linear traversal is often
> enough. Far from always though, especially when one counts cases when
> application can traverse linearly but when of the subobjects needs to
> store its sub-S-expression in memory.

OK, let's assume this can work technically. Let's do a cost-benefit
analysis of your preferred solution.

Cost: you limit the use of the S-Expression parser to linear traversal
only; this makes it unsuitable for a general-purpose library, which I
thought was one of your goals.

Cost: while the S-Expression parser has to construct S-Expressions in
memory anyway, you need artificial measures to hide the tree structure
from clients, thereby cluttering the implementation of the S-
Expression parser and making any future changes more difficult.

Benefit: in memory-constrained systems, you may deallocate memory soon
(after the client calls Move_Up, e.g.) and allocate it late (only when
the client calls Get_Atom or Move_Down), thereby holding only one node
at a time in memory. But if you're planning to write a web server, I
don't think you were concerned about the ~100 bytes necessary for the
tree of "(tcp-connect (host foo.bar) (port 80))", were you?

Benefit: I can't see any for the client, honestly, since parsing your
stream is not easier than parsing the S-Expression tree directly, as
you can see when comparing our respective implementations of
TCP_Info.Read.

So, I do think the more general solution is better.

--
Ludovic Brenta.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 12, 6:19 pm, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:
> Natacha Kerensikova wrote on comp.lang.ada:
> >> The reason why I prefer to expose the S-Expression is because, in the
> >> general (arbitrarily complex) case, you cannot traverse an S-
> >> Expresssion linearly; you need to traverse it as what it really is, a
> >> tree. A stream suggests linear traversal only.
>
> > Yes, linear traversal only is indeed a limitation of Sexp_Stream.
> > That's the reason why I had in mind another package of object and
> > memory handling, which would rely on Sexp_Stream for the parsing.
>
> > I prefer (for now) the Sexp_Stream/memory-object split because 1. it
> > makes thing more modular, which means more possibilities of localized
> > changes and of implementation overhaul without changing specification,
> > and 2. in my real-life application the linear traversal is often
> > enough. Far from always though, especially when one counts cases when
> > application can traverse linearly but when of the subobjects needs to
> > store its sub-S-expression in memory.
>
> OK, let's assume this can work technically. Let's do a cost-benefit
> analysis of your preferred solution.

This is turning a bit too much into a childish "my idea is better than
yours" for my taste.

However please note that I'm afraid your misjudging my project by
taking into consideration only what I call Sexp_Stream. I've never
thought Sexp_Stream without the other (still unnamed) package I keep
talking about that handles the representations in memory, because
Sexp_Stream alone won't fit all my needs. These two packages are
functionally equivalent to what I wanted initially, and they are also
equivalent to the package your writing. So from a client point of
view, there wouldn't be much (if any) differences between my second
package and yours. The possibility of using directly Sexp_Stream is
just an option for the particular cases where it is as easy as using
in-memory objects. So I'd say ex-aequo on functionality and
generality.

Now from an implementation point of view, my two-package idea is also
equivalent to my original plans in that the glue between the two
packages is so thin coding both packages takes about as much effort as
my original idea. I suspect your package is very similar to my
original idea, which means ex-eaquo too on implementation efforts and
complexity.

So what are the differences then?

My two-package idea allows to make use of the memory-efficient
Sexp_Stream interface instead of building objects. Honestly I don't
expect ever encountering a situation where it matters, so let's say it
doesn't.

My two-package idea implies a well-defined interface between what
would be two halves of your packages. Well it boils to whether two
independent packages are better or not than a single double-sized
package. I personally have a clear preference of two halves rather
than one equally size; but again, that's just my taste, I'm not sure
there is really any objective technical advantage.

My two-package idea would be implemented under an ISC license, while
your package is GPL. I don't want to start a license troll or
anything, my point here is just that at this point of the analysis
both projects looks so similar to me that license-preferences might be
the only thing left to tip the balance.

My two-package idea is so far just an idea. Your package already has
lots of real code. I won't be surprised if I'm so busy here learning
about what people think about my ideas and learning about Ada that you
end up finishing your package before I actually write my first line in
Ada. We've got a clear winner here.

Thanks for your attention,
Natacha

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/12/2010 02:26 AM, Natacha Kerensikova wrote:
>
> There are situations when a memory representation of S-expressions is
> not needed, and the tcp-connect example here seems to be such a case.
> That's why I imagined TCP_Info as a client of Sexp_Stream instead of a
> client of the second package.

Not needed, perhaps, but it makes things clearer and more reusable if it is used 
anyway. The result might not be acceptable for a memory-constrained system, but 
general, reusable pkgs are usually not acceptable when you have special 
requirements.

>> S : S_Expression := Some_Initialization;
>>
>> Put (File =>  F, Expression =>  S);
>
> To achieve such an interface, the client has to build an in-memory S-
> expression object. In the tcp-connect example, there are eight nodes
> (five atoms and three lists). They have to be somehow built, and it
> doesn't look like a simple initialization.

Perhaps not. But for each kind of thing you want to store using this 
representation, it need be done only once, and then reused:

S : S_Expression := To_S_Expression (TCP_Info);

> The second interface I proposed, with a lot of nested calls, builds
> the S-expression object with functions looking like:
> function NewList(Contents, Next: S_node_access) return S_node_access;
> function AtomFromWhatever(Contents: whatever, Next: S_node_access)
> return S_node_access;

An aside on Ada style: Ada is not case sensitive, and many of us reformat code 
to make it easier for us to read. The resulting identifiers such as Newlist and 
Atomfromwidget are not very readable. This is why the convention is to use 
underlines between words.

> For the TCP_info stuff particular case, the only simplification I can
> imagine is leveraging the fact that only 2 among the 8 nodes change.
> So one could construct a global S-expression skeleton with the 6
> constant nodes, and when actually writing data append to the "host"
> and "port" atoms the relevant variable atoms. However that means
> having a read/write global variable, which is bad for tasking, so one
> might prefer a constant 6-node skeleton, duplicated in the declarative
> part of the writing procedure, and then appending the variable nodes.
> However that appending would probably end up with lines like:
> root_sexp.FirstChild.Next.FirstChild.Append(host_node);
> root_Sexp.FirstChild.Next.Next.FirstChild.Append(port_node);
> which looks very ugly to me.

Then I would suggest that your client interface for the TCP-info stuff would 
only deal with (host, port) pairs.

I would suggest a constant in the pkg body that you would use to initialize an 
S-expression variable, then substitute the host and port values into the 
variable before writing. On reading, you would read the S-expression, then 
extract the host and port values and return them to the client.

Ugly is relative; having code like that sprinkled throughout your application is 
ugly; having it occur once in a pkg body is less so. Having it be less ugly and 
occur once in a pkg body would be even better, though.

>> Your TCP_Info-handling pkg would convert the record into an S-expression, and
>> call a single operation from your S-expression pkg to output the S-expression.
>
> That's the tricky part. At least so tricky that I can't imagine how to
> do it properly.

I'm not sure what you think is tricky about it. Clearly you see how to output an 
S-expression:

>> Your S-expression library would implement the writing of the expression as a
>> series of steps.
>
> Indeed, as a series of calls similar to my Sexp_Stream example. I'm
> glad to have at least that part right.

-- 
Jeff Carter
"What I wouldn't give for a large sock with horse manure in it."
Annie Hall
42

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 12, 8:51 pm, Jeffrey Carter
<spam.jrcarter....@spam.not.acm.org> wrote:
> On 08/12/2010 02:26 AM, Natacha Kerensikova wrote:
> > There are situations when a memory representation of S-expressions is
> > not needed, and the tcp-connect example here seems to be such a case.
> > That's why I imagined TCP_Info as a client of Sexp_Stream instead of a
> > client of the second package.
>
> Not needed, perhaps, but it makes things clearer and more reusable if it is used
> anyway.

Well indeed, I do consider readability and simplicity (they often come
together) as one of the most important criteria when choosing an
interface. I think I didn't explained the whole thought process above,
my bad.

When I wrote this I couldn't think of any way to write clearer or
simpler code with any of the two proposed packages (stream or memory
based), because of the basic argument that 8 nodes have to be somehow
created anyway and couldn't think of any way of doing that except by
creating them one by one (be it with a sequence of procedure calls or
a bunch of nested function calls). So when complexity and readability
are equal, I go for the least amount of dependencies.

Of course finding a way to make S-expression building much clearer
with a given interface would be a huge argument in favor of the
interface, no matter its level.

> > To achieve such an interface, the client has to build an in-memory S-
> > expression object. In the tcp-connect example, there are eight nodes
> > (five atoms and three lists). They have to be somehow built, and it
> > doesn't look like a simple initialization.
>
> Perhaps not. But for each kind of thing you want to store using this
> representation, it need be done only once, and then reused:
>
> S : S_Expression := To_S_Expression (TCP_Info);

Unless I missed something important, it looks like it only moves the
problem around. While a To_S_Expression function does make a
TCP_Info'Write simple and one-operation and all, complexity is only
transferred to To_S_Expression which will still have to do the dirty
job of creating 8 nodes.

> > The second interface I proposed, with a lot of nested calls, builds
> > the S-expression object with functions looking like:
> > function NewList(Contents, Next: S_node_access) return S_node_access;
> > function AtomFromWhatever(Contents: whatever, Next: S_node_access)
> > return S_node_access;
>
> An aside on Ada style: Ada is not case sensitive, and many of us reformat code
> to make it easier for us to read. The resulting identifiers such as Newlist and
> Atomfromwidget are not very readable. This is why the convention is to use
> underlines between words.

Yes, I have not managed yet to get that habit. I try to use a
Underscore_And_Capitalization style (not sure whether it's the usual
Ada idiom or not), but sometimes in the heat of action I forgot to do
it (funny thing that identifier that slip end up to be camel cased,
while my C habit is underscore and lower case).

> >> Your TCP_Info-handling pkg would convert the record into an S-expression, and
> >> call a single operation from your S-expression pkg to output the S-expression.
>
> > That's the tricky part. At least so tricky that I can't imagine how to
> > do it properly.
>
> I'm not sure what you think is tricky about it. Clearly you see how to output an
> S-expression:

You're not the only one having trouble understanding that part. I'm
not good at expressing myself :-(

What I found tricky is the "single operation" part. Building 8 nodes
in a single operation does look very difficult, and while Ludovic's
trick of building them from an encoded string is nice, it makes we
wonder (again) about the point of building a S-expression object
before writing it while it's simpler and clearer to write strings
containing hand-encoded S-expressions.



Thanks for your remarks,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova <lithiumcat@gmail.com> writes on comp.lang.ada:
> What I found tricky is the "single operation" part. Building 8 nodes
> in a single operation does look very difficult, and while Ludovic's
> trick of building them from an encoded string is nice, it makes we
> wonder (again) about the point of building a S-expression object
> before writing it while it's simpler and clearer to write strings
> containing hand-encoded S-expressions.

The only use case for constructing an S-Expression tree in memory and
then writing it out is when the S-Expression is very dynamic.  For
example, when you want to save the state of your application, e.g. the
cache of a web browser, in a structured way.  Or when you want to save a
tree, the contents and structure of which are unknown in advance.

If the S-Expression is static or mostly so, then by all means write it
by hand.

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Jeffrey R. Carter]

On 08/13/2010 02:32 AM, Natacha Kerensikova wrote:
>
> When I wrote this I couldn't think of any way to write clearer or
> simpler code with any of the two proposed packages (stream or memory
> based), because of the basic argument that 8 nodes have to be somehow
> created anyway and couldn't think of any way of doing that except by
> creating them one by one (be it with a sequence of procedure calls or
> a bunch of nested function calls). So when complexity and readability
> are equal, I go for the least amount of dependencies.
>
> Of course finding a way to make S-expression building much clearer
> with a given interface would be a huge argument in favor of the
> interface, no matter its level.
>
> Unless I missed something important, it looks like it only moves the
> problem around. While a To_S_Expression function does make a
> TCP_Info'Write simple and one-operation and all, complexity is only
> transferred to To_S_Expression which will still have to do the dirty
> job of creating 8 nodes.

Sure, but it will be hidden and reused, rather than appearing frequently.

Let's see if I can make what I'm talking about clear. We want to build an 
application that does something. It will need to manipulate widgets to do so, so 
we package the widget manipulation:

with Ada.Text_IO;

package Widgets is
    type Widget is ...;

    procedure Put (File : in Ada.Text_IO.File_Type; Item : in Widget);
    function Get (File : in Ada.Text_IO.File_Type) return Widget;
end Widgets;

Now we can go off and work on the rest of the application. Eventually we'll need 
to implement the body of Widgets. We decide that we want use S-expressions for 
the external storage syntax, so we'll need (or already have) a low-level 
S-expression pkg:

with Ada.Text_IO;

package Sexps is
    type Atom is private;

    -- Operations to convert things to/from Atoms.

    type Sexp is private; -- An Atom or a list of Sexp.

    function To_Sexp (From : in Atom);

    function Empty_List return Sexp;

    Not_A_List : exception;

    function Append (To : in Sexp; Item : in Sexp) return Sexp;
    -- Appends Item to the list To. Raises Not_A_List is To is not a list.

    procedure Put (File : in Ada.Text_IO.File_Type; Item : in Sexp);
    function Get (File : in Ada.Text_IO.File_Type) return Sexp;

    -- Operations to extract information from a Sexp.
    -- Other declarations as needed.
end Sexps;

I've only listed the operations to construct a Sexp and for I/O. This is 
low-level and somewhat messy to use, but would never be accessed directly from 
the code that implements the functionality of the application.

Now we can implement the body of Widgets:

with Sexps;

package body Widgets is
    function To_Sexp (From : in Widget) return Sexps.Sexp is separate;
    function To_Widget (From : in Sexps.Sexp) return Widget is separate;
    -- These are left as an exercise for the reader :)

    procedure Put (File : in Ada.Text_IO.File_Type; Item : in Widget) is
       -- null;
    begin -- Put
       Sexps.Put (File => File, Item => To_Sexp (Item) );
    end Put;

    function Get (File : in Ada.Text_IO.File_Type) return Widget is
       -- null;
    begin -- Get
       return To_Widget (Sexps.Get (File) );
    end Get;
end Widgets;

The body of Sexps will be messy, but it will be written once and mostly ignored 
from then on. The bodies of To_Sexp and To_Widget will be sort of messy, but 
will be written once (for each type of interest) and mostly ignored from then 
on. The code that implements the functionality of the application will be clear.

> Yes, I have not managed yet to get that habit. I try to use a
> Underscore_And_Capitalization style (not sure whether it's the usual
> Ada idiom or not), but sometimes in the heat of action I forgot to do
> it (funny thing that identifier that slip end up to be camel cased,
> while my C habit is underscore and lower case).

Initial_Caps is the Ada convention. Your C convention would be better than 
CamelCase.

>
>>>> Your TCP_Info-handling pkg would convert the record into an S-expression, and
>>>> call a single operation from your S-expression pkg to output the S-expression.
>
> What I found tricky is the "single operation" part. Building 8 nodes
> in a single operation does look very difficult, and while Ludovic's
> trick of building them from an encoded string is nice, it makes we
> wonder (again) about the point of building a S-expression object
> before writing it while it's simpler and clearer to write strings
> containing hand-encoded S-expressions.

"Single operation" referred to output of an S-expression. If you can write this 
a sequence of low-level steps, then you can put those steps in a higher-level 
procedure which becomes a single operation to output an S-expression.

Converting something into an S-expression is similar. Leaving aside 
considerations of limitedness, anything you can build up in a series of steps in 
Ada can be wrapped in a function that returns the thing after it has been built; 
this function is then a single operation.

Using the Sexps pkg I sketched earlier, one can build an arbitrary S-expression 
in a single statement with lots of calls to Empty_List, To_Atom, and Append. But 
one could also build it up in steps using intermediaries. Either way would serve 
as an implementation of Widgets.To_Sexp; To_Sexp is a single operation to 
convert a Widget to an S-expression.

HTH.

-- 
Jeff Carter
"Have you gone berserk? Can't you see that that man is a ni?"
Blazing Saddles
38

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 14, 12:53 am, "Jeffrey R. Carter"
<spam.jrcarter....@spam.acm.org> wrote:
> The body of Sexps will be messy, but it will be written once and mostly ignored
> from then on. The bodies of To_Sexp and To_Widget will be sort of messy, but
> will be written once (for each type of interest) and mostly ignored from then
> on. The code that implements the functionality of the application will be clear.

Ok I understand know. And actually the only difference with what I had
in mind is that I thought of what you call To_Sexp and To_Widget as
part of respectively Widgets.Put and Widgets.Get, because I didn't
imagine any other place where To_Sexp or To_Widget would be used, so I
would have "hand-inlined" them into Put and Get.

Of course, I do follow the "don't repeat yourself" principle, and as
soon as I would feel the need of another place to have a S-expression
<--> Widget conversion I would have built To_Sexp and To_Widget
procedures (though probably with another name) from what would have
used to be Widgets.Put and Widgets.Get bodies.

> Using the Sexps pkg I sketched earlier, one can build an arbitrary S-expression
> in a single statement with lots of calls to Empty_List, To_Atom, and Append. But
> one could also build it up in steps using intermediaries. Either way would serve
> as an implementation of Widgets.To_Sexp; To_Sexp is a single operation to
> convert a Widget to an S-expression.

OK

Thanks for your explanations,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova <lithiumcat@gmail.com> writes:
> On Aug 9, 8:49 pm, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:
>> I pursued that idea a little further and actually wrote an embryonic
>> S-Expression library.  I'm not entirely satisfied with it because it
>> uses copy semantics instead of reference semantics and so is probably
>> inefficient.  But it does demonstrate how to read and write
>> S-Expressions on a stream (e.g. a file).  Also, it shows how to
>> hex-encode and hex-decode blobs, which I've modelled as Storage_Arrays.
>>
>> You can browse the sources here:
>>
>> http://green.ada-france.org:8081/branch/changes/org.ludovic-brenta.s_...
>>
>> Enjoy.  (the license is the GPLv3 or later).
>
> Interesting, though it seems your code won't like strings containing a
> double-quote.

I've corrected that and rewritten test.adb to use your example:

(tcp-connection ((host foo.bar) (port 80)))

My parser needs still the above syntax and will not accept the
theoretically equivalent

(tcp-connection (host foo.bar) (port 80))

yet.

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 10.08.10 17:48, Ludovic Brenta wrote:

> I've corrected that and rewritten test.adb to use your example:
> 
> (tcp-connection ((host foo.bar) (port 80)))
> 
> My parser needs still the above syntax and will not accept the
> theoretically equivalent
> 
> (tcp-connection (host foo.bar) (port 80))
> 
> yet.

How does it handle
(tcp-connection ((host foo.bar) (port 80))
(tcp-connection ((host abc.xyz) (port 80)))

Sorry, CNR ;-)


Georg

Re: S-expression I/O in Ada

[Ludovic Brenta]

Georg Bauhaus wrote:
> On 10.08.10 17:48, Ludovic Brenta wrote:
>
>> I've corrected that and rewritten test.adb to use your example:
>>
>> (tcp-connection ((host foo.bar) (port 80)))
>>
>> My parser needs still the above syntax and will not accept the
>> theoretically equivalent
>>
>> (tcp-connection (host foo.bar) (port 80))
>>
>> yet.
>
> How does it handle
> (tcp-connection ((host foo.bar) (port 80))
> (tcp-connection ((host abc.xyz) (port 80)))

I made some fixes and I now consider my S-Expression parser[1] feature-
complete as of revision b13ccabbaf227bad264bde323138910751aa2c2b.
There may still be some bugs though, and the error reporting (to
diagnose syntax errors in the input) is very primitive.

Highlights:
* the procedure S_Expression.Read is a quasi-recursive descent
parser.  "Quasi" because it only recurses when encountering an opening
parenthesis, but processes atoms without recursion, in the same finite
state machine.
* the parser reads each character exactly once; there is no push_back
or backtracking involved.  This makes the parser suitable to process
standard input on the fly.
* to achive this, I had to resort to using exceptions instead of
backtracking; this happens when the parser encounters a ')'
immediately after an S-Expression (atom or list).
* the parser also supports lists of the form (a b c) (more than 2
elements) and properly translates them to (a (b c)).  The Append()
procedure that does this is also public and available to clients.
* the parser does not handle incomplete input well.  If it gets
Ada.IO_Exceptions.End_Error in the middle of an S-Expression, it will
return an incomplete, possibly empty, S-Expression rather than report
the error.  I'll try to improve that.
* the test.adb program demonstrates how to construct an S-Expression
tree in memory (using cons()) and then sending it to a stream (using
'Write).
* the test.adb program also demonstrates how to read an S-Expression
from a stream (using 'Read) and then traverse the in-memory tree
(using car(), cdr()).

[1] http://green.ada-france.org:8081/branch/changes/org.ludovic-brenta.s_expressions

I have not yet tested the parser on your proposed input (IIUC,
consisting of two S-Expressions with a missing closing parenthesis).
I think this will trigger the bug where End_Error in the middle of an
S-Expression is not diagnosed.

I also still need to add the proper GPLv3 license text on each file.

I'll probably add support for Lisp-style comments (starting with ';'
and ending at end of line) in the future.

--
Ludovic Brenta.

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/12/2010 12:53 AM, Ludovic Brenta wrote:
> * the parser also supports lists of the form (a b c) (more than 2
> elements) and properly translates them to (a (b c)).  The Append()
> procedure that does this is also public and available to clients.

The example was something like

(TCP-something (host abc) (port 42) )

So I think the grammar is

S-expression = '(' Element {' ' Element} ')'

Element = Atom | S-expression

and you've implemented something slightly different.

-- 
Jeff Carter
"What I wouldn't give for a large sock with horse manure in it."
Annie Hall
42

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Ludovic Brenta]

Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> writes:
> On 08/12/2010 12:53 AM, Ludovic Brenta wrote:
>> * the parser also supports lists of the form (a b c) (more than 2
>> elements) and properly translates them to (a (b c)).  The Append()
>> procedure that does this is also public and available to clients.
>
> The example was something like
>
> (TCP-something (host abc) (port 42) )
>
> So I think the grammar is
>
> S-expression = '(' Element {' ' Element} ')'
>
> Element = Atom | S-expression
> 
> and you've implemented something slightly different.

If I understand your quasi-BNF correctly, an S-Expression in your
grammar can have only one or two, but not three elements.  This
contradicts the "(TCP-something (host abc) (port 42) )" example, which
has three elements.

In reality, an S-Expression (a "cons pair" in Lisp parlance) can indeed
not have three elements; the notation (a b c) is, really, shorthand for
(a (b c)); I was keen to implement that in my parser.  See

http://en.wikipedia.org/wiki/S-expression#Definition

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 12, 9:59 pm, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:
> In reality, an S-Expression (a "cons pair" in Lisp parlance) can indeed
> not have three elements; the notation (a b c) is, really, shorthand for
> (a (b c)); I was keen to implement that in my parser.  See
>
> http://en.wikipedia.org/wiki/S-expression#Definition

According to that page, cons pairs are noted with an extra dot, like
(x . y), right? So (a b c) would be (a . (b . (c . nil))), right?
Then my example would be:
(tcp-connect . ((host . (foo.example . nil)) . ((port . (80 . nil)) .
nil)))

For the record, I'm just discovering the concept of cons pairs, I
based my previous S-expression work only on Rivest's proposed standard
http://people.csail.mit.edu/rivest/Sexp.txt


Hoping this helps,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova writes on comp.lang.ada:
> On Aug 12, 9:59 pm, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:
>> In reality, an S-Expression (a "cons pair" in Lisp parlance) can indeed
>> not have three elements; the notation (a b c) is, really, shorthand for
>> (a (b c)); I was keen to implement that in my parser.  See
>>
>> http://en.wikipedia.org/wiki/S-expression#Definition
>
> According to that page, cons pairs are noted with an extra dot, like
> (x . y), right? So (a b c) would be (a . (b . (c . nil))), right?
> Then my example would be:
> (tcp-connect . ((host . (foo.example . nil)) . ((port . (80 . nil)) .
> nil)))

Correct. The dot notation is from Lisp; (x . y) really denotes a cons
pair while (x y) is shorthand for (x . (y . nil)).

However, (y . nil) doesn't really make any sense; since y is an atom,
you don't need to wrap it in a cons pair, so just y is sufficient (you
only need a way to distinguish cons pairs from atoms and say that atoms
don't have a car and cdr but only a value; that's what the Atom
discriminant is for in my implementation).  Replacing (y . nil) with
just y, your example then becomes:

(tcp-connect . ((host . foo.example) . (port . 80)))

which is exactly what my parser builds in memory: 4 cons pairs, 5 atoms.
The other notations:

(tcp-connect ((host foo.example) (port 80)))

and

(tcp-connect (host foo.example) (port 80))

are shorthand for that.

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 12, 10:45 pm, Ludovic Brenta <ludo...@ludovic-brenta.org>
wrote:
> The other notations:
>
> (tcp-connect ((host foo.example) (port 80)))
>
> and
>
> (tcp-connect (host foo.example) (port 80))
>
> are shorthand for that.

What bother me in this is that I feel those are two different objects.

I won't go into writing explicit cons pairs again, this is quite
unreadable. So I'll go for an ASCII-art representation, I hope it
won't be scrambled (if so please tell me, I'll try to come up with a
textual representation). Let L and A represent nodes, respectively
lists and atoms, vertical lines for "next in list", "brother" or
"cons" relations, and horizontal lines for "first child of list" or
"atom contents" relations.

Consider these two objects:

L--A--tcp-connect
   |
   L--A--host
   |  |
   |  A--foo.example
   |
   L--A--port
      |
      A--80

and:

L--A--tcp-connect
   |
   L--L--A--host
      |  |
      |  A--foo.example
      |
      L--A--port
         |
         A--80

I'm sure we can agree on the fact that these are two different and non-
equivalent objects: not the same topology and not even the same number
of nodes. So I'd say a proper S-expression library has to be able to
deal with both of them without mixing them up.

However, by considering as equivalent the two representations quoted
above, I'm afraid you might mixing these two object during encoding or
decoding.

Or have I misunderstood again?



Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova wrote on comp.lang.ada:
> On Aug 12, 10:45 pm, Ludovic Brenta <ludo...@ludovic-brenta.org>
> wrote:
>> The other notations:
>>
>> (tcp-connect ((host foo.example) (port 80)))
>>
>> and
>>
>> (tcp-connect (host foo.example) (port 80))
>>
>> are shorthand for that.
>
> What bother me in this is that I feel those are two different objects.
>
> I won't go into writing explicit cons pairs again, this is quite
> unreadable. So I'll go for an ASCII-art representation, I hope it
> won't be scrambled (if so please tell me, I'll try to come up with a
> textual representation). Let L and A represent nodes, respectively
> lists and atoms, vertical lines for "next in list", "brother" or
> "cons" relations, and horizontal lines for "first child of list" or
> "atom contents" relations.
>
> Consider these two objects:
>
> L--A--tcp-connect
>    |
>    L--A--host
>    |  |
>    |  A--foo.example

This part cannot exist: the second L cannot have more than 2 children;
here you gave it three (A, A and the next L). The dot notation would
have prevented that mistake :)

>    |
>    L--A--port
>       |
>       A--80
>
> and:
>
> L--A--tcp-connect
>    |
>    L--L--A--host
>       |  |
>       |  A--foo.example
>       |
>       L--A--port
>          |
>          A--80

This one is the only correct topology.

--
Ludovic Brenta.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 13, 11:08 am, Ludovic Brenta <ludo...@ludovic-brenta.org>
wrote:
> Natacha Kerensikova wrote on comp.lang.ada:
> > Consider these two objects:
>
> > L--A--tcp-connect
> >    |
> >    L--A--host
> >    |  |
> >    |  A--foo.example
>
> This part cannot exist: the second L cannot have more than 2 children;
> here you gave it three (A, A and the next L). The dot notation would
> have prevented that mistake :)

No, it has only two children: the A connected to "host" and the L you
haven't quoted. All the nodes in both my graphs have exactly one
"right child", which is their value, and zero or one (but not more)
"lower child" being the next element in the list, i.e. the second
argument of cons.

So if you really want a dot notation, here is the first object:
(tcp-connect .
   (
       (host . (foo.example . nil))
   .
      ((port . (80          . nil)) . nil)
   )
)

while the second one is:

(tcp-connect .
   (
      (
          (host . (foo.example . nil))
      .
         ((port . (80          . nil)) . nil)
      )
   .
      nil
   )
)

I hope my vertical alignment will make these cons pairs clear enough.


Hoping to finally understand,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova <lithiumcat@gmail.com> writes:
> On Aug 13, 11:08 am, Ludovic Brenta <ludo...@ludovic-brenta.org>
> wrote:
>> Natacha Kerensikova wrote on comp.lang.ada:
>> > Consider these two objects:
>>
>> > L--A--tcp-connect
>> >    |
>> >    L--A--host
>> >    |  |
>> >    |  A--foo.example
>>
>> This part cannot exist: the second L cannot have more than 2 children;
>> here you gave it three (A, A and the next L). The dot notation would
>> have prevented that mistake :)
>
> No, it has only two children: the A connected to "host" and the L you
> haven't quoted. All the nodes in both my graphs have exactly one
> "right child", which is their value, and zero or one (but not more)
> "lower child" being the next element in the list, i.e. the second
> argument of cons.

OK, so your A nodes weren't really atoms after all, they were cons
pairs, the car of which was the actual atom.  I misunderstood that.

> So if you really want a dot notation, here is the first object:
> (tcp-connect .
>    (
>        (host . (foo.example . nil))
>    .
>       ((port . (80          . nil)) . nil)
>    )
> )
>
> while the second one is:
>
> (tcp-connect .
>    (
>       (
>           (host . (foo.example . nil))
>       .
>          ((port . (80          . nil)) . nil)
>       )
>    .
>       nil
>    )
> )
> 
> I hope my vertical alignment will make these cons pairs clear enough.
>
>
> Hoping to finally understand,
> Natacha

OK, but these two are equivalent if you remember the axiom that (y
. nil) is really just y.  If you apply that transformation to both of
your examples you reach the same topology:

  (tcp-connect . ((host . (foo.example . nil)) . ((port . (80 . nil)) . nil)))
= (tcp-connect . ((host .     foo.example    ) . ((port .    80)      . nil)))
= (tcp-connect . ((host .     foo.example    ) .                (port . 80)))


  (tcp-connect . (((host . (foo.example . nil)) . ((port . (80 . nil)) . nil)) . nil))
= (tcp-connect . (((host .       foo.example  ) . ((port .    80)      . nil)) . nil))
= (tcp-connect . (((host .       foo.example  ) .                (port . 80))  . nil))
= (tcp-connect . ((host .        foo.example  ) .                (port . 80)))

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-14, Ludovic Brenta <ludovic@ludovic-brenta.org> wrote:
> OK, but these two are equivalent if you remember the axiom that (y
> . nil) is really just y.

Ok, that's the part I have trouble with. When thinking in list terms,
that cons feels like it's the operation of prepending an item to an
existing list, right?

Here is my understanding, please correct me where I'm wrong. Oh, I'm
going to have to type that.

Let's assume we have a Node type, which can be either a List or an Atom
(little Ada question as an aside, Node would then be a parametrized
type, and the two cases would be called Node(List) and Node(Atom),
right? or have I misunderstood something?).

The cons operation would then take a Node as its first parameter (before
the dot) and a List as its second, and return a List which is
semantically the input list prepended by the first parameter taken as a
list item. Right?

So nil is of type List and represents the empty list. Then (y . nil) is
a List containing a single node, which is the Atom y. So I feel that y
is of type Atom while (y . nil) is of type List.

(Then (x . (y .nil)) is the previous list with x prepended, therefore
the two-item list containing x then y.
And then (z . (x . (y . nil))) would be a three-item list.
But then what of ((x . (y. nil)) . (z . nil))? It looks to me like
a two-item list whose first item is a list, followed by the atom z. But
then how would reverse list be written?
Maybe (z . ((x . (y . nil)) . nil))? But that doesn't fit with your
derivation pattern on tcp-connect, so I'm lost.)

Now I can understand that in an untyped environment, an Atom can be
automagically cast into a single-item list. In such a case, (x . y) is
first translated into (x . LY) where LY is the List containing the Atom
y a its only item (i.e. (y . nil)). And now that cons is correctly typed
like (Node . List), it ends up with the two-item List containing the
Atom x followed by the Atom y.

But then how would you make a list ending with LY? My guess would be
(x . (LY . nil)), but then it is not equivalent to (x . LY), so I'm lost
again.

Or are single-item lists prohibited in lisp?


Thanks for your patience,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natasha Kerensikova writes on comp.lang.ada:
> On 2010-08-14, Ludovic Brenta <ludovic@ludovic-brenta.org> wrote:
>> OK, but these two are equivalent if you remember the axiom that (y
>> . nil) is really just y.
>
> Ok, that's the part I have trouble with. When thinking in list terms,
> that cons feels like it's the operation of prepending an item to an
> existing list, right?
>
> Here is my understanding, please correct me where I'm wrong. Oh, I'm
> going to have to type that.
>
> Let's assume we have a Node type, which can be either a List or an Atom
> (little Ada question as an aside, Node would then be a parametrized
> type, and the two cases would be called Node(List) and Node(Atom),
> right? or have I misunderstood something?).

Correct, see my implementation:

   type Access_T is access T;
   type Internal_T (Atom : Boolean := False) is record
      case Atom is
         when True  => Value : Ada.Strings.Unbounded.Unbounded_String;
         when False =>
            Car : Access_T;
            Cdr : Access_T;
      end case;
   end record;
   type Internal_Access_T is access Internal_T;
   type T is new Ada.Finalization.Controlled with record
      Internal : Internal_Access_T;
   end record;

> The cons operation would then take a Node as its first parameter (before
> the dot) and a List as its second, and return a List which is
> semantically the input list prepended by the first parameter taken as a
> list item. Right?

Correct:

   function Cons (Car, Cdr : in T) return T is
      Result : T;
   begin
      Result.Internal := new Internal_T (Atom => False);
      Result.Internal.Car := new T'(Car);
      Result.Internal.Cdr := new T'(Cdr);
      return Result;
   end Cons;

> So nil is of type List and represents the empty list. Then (y . nil) is
> a List containing a single node, which is the Atom y. So I feel that y
> is of type Atom while (y . nil) is of type List.

If you see nil as a list, then it has a car and a cdr, so nil is defined
recursively as (nil . nil), so you then end up with infinite recursion.
So, I'd rather see nil as an atom that, by definition, does not have a
car or cdr and only a value (in this case, the value is empty).  Again
from my implementation:

Nil : constant T := (Ada.Finalization.Controlled with Internal => null);

Now let me explain the (y . nil) = y axiom.

Definition: a list node is a cons pair consisting of a car and a cdr
that point to other nodes.

Definition: an atom is a node consisting only of a value (i.e. it has no
car and no cdr).

Now let us look at (y . nil), where y is an arbitrary node.

This (y . nil) is a cons pair where the car points to an actual node but
the cdr points to nil.  If you traverse this node, you will find y and
then stop.  So essentially it is a node with a single value, y.

If you look at just y and you traverse that, you will find y and then
stop.  This is exactly the same result as traversing (y . nil).  Hence
the equivalence.

> (Then (x . (y .nil)) is the previous list with x prepended, therefore
> the two-item list containing x then y.

Correct, and it is also equivalent to just (x . y).

> And then (z . (x . (y . nil))) would be a three-item list.
> But then what of ((x . (y. nil)) . (z . nil))? It looks to me like
> a two-item list whose first item is a list, followed by the atom z.

((x . (y. nil)) . (z . nil)) is not terminated by nil; it is equivalent
to
(((x . (y. nil)) . (z . nil)) . nil)
and to
((x . y) . z)

> But then how would reverse list be written?

(z . (x . y)), I suppose.

> Maybe (z . ((x . (y . nil)) . nil))? But that doesn't fit with your
> derivation pattern on tcp-connect, so I'm lost.)
>
> Now I can understand that in an untyped environment, an Atom can be
> automagically cast into a single-item list. In such a case, (x . y) is
> first translated into (x . LY) where LY is the List containing the Atom
> y a its only item (i.e. (y . nil)). And now that cons is correctly typed
> like (Node . List), it ends up with the two-item List containing the
> Atom x followed by the Atom y.
>
> But then how would you make a list ending with LY? My guess would be
> (x . (LY . nil)), but then it is not equivalent to (x . LY), so I'm lost
> again.
>
> Or are single-item lists prohibited in lisp?

I guess so, because if nil is a list, then it must be defined as (nil
. nil); the definition would be infinitely recursive.  So a single-item
list is, by definition, an atom.

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Yannick Duchêne (Hibou57)]

Le Sat, 14 Aug 2010 15:13:51 +0200, Ludovic Brenta  
<ludovic@ludovic-brenta.org> a écrit:
>    type Access_T is access T;
>    type Internal_T (Atom : Boolean := False) is record
>       case Atom is
>          when True  => Value : Ada.Strings.Unbounded.Unbounded_String;
>          when False =>
>             Car : Access_T;
>             Cdr : Access_T;
>       end case;

Just a detail : what about an enumerated type instead of a boolean ?


    type Item_Kinds is (An_Atom, A_List);

    type Internal_T (Kind : Item_KInds := A_List) is record
       case Kind is
          when An_Atom  => Value : Ada.Strings.Unbounded.Unbounded_String;
          when A_List =>
             Car : Access_T;
             Cdr : Access_T;
       end case;

Perhaps more expressive (refer to things using their name instead of  
predicates) and more resilient (just imagine you want to later have a  
special handing of nil).

Personal note: I do not agree with the use of a Boolean every where there  
is a two element enumeration. Boolean are made for Boolean arithmetic and  
to represent items of the Boolean set.

-- 
There is even better than a pragma Assert: a SPARK --# check.
--# check C and WhoKnowWhat and YouKnowWho;
--# assert Ada;
--  i.e. forget about previous premises which leads to conclusion
--  and start with new conclusion as premise.

Re: S-expression I/O in Ada

[Jeffrey R. Carter]

On 08/12/2010 12:59 PM, Ludovic Brenta wrote:
>
> If I understand your quasi-BNF correctly, an S-Expression in your
> grammar can have only one or two, but not three elements.  This
> contradicts the "(TCP-something (host abc) (port 42) )" example, which
> has three elements.

No, {} means zero or more, so an S-expression can have one or more elements.

Kerensikova (or is it Port�?) provided a more detailed grammar (using the 
trailing '*' from regexps rather than {}) with some additional options.

> http://en.wikipedia.org/wiki/S-expression#Definition

But maybe we're both wrong.

-- 
Jeff Carter
"What I wouldn't give for a large sock with horse manure in it."
Annie Hall
42

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 13, 12:25 am, "Jeffrey R. Carter"
<spam.jrcarter....@spam.acm.org> wrote:
> Kerensikova (or is it Porté?)

It's both ;-)  One of them is my real-life last name, as can be found
on my ID or on copyright notices, while the other is a pseudonymous
last name I've been using over the internet for something like a
decade now. While it was never intended to more than a thin pseudonym,
it was supposed to require a little more research to make the
connection, here I just messed up with Google accounts (I really hate
these browser interfaces, and now it makes me hate them even more, I
wish I could find a NNTP server and a real user-friendly text-mode
news reader).

> provided a more detailed grammar (using the
> trailing '*' from regexps rather than {}) with some additional options.

Is one of them more standard than the other? While I'm very familiar
with regex, I only have a few theoretical notions about grammar, and
almost nothing as practical experience with real grammar formats.

> >http://en.wikipedia.org/wiki/S-expression#Definition
>
> But maybe we're both wrong.

I can assure you what I'm doing is completely conform with Rivest's
standard proposition. I don't know how come it has not reached the
status of full-fledged RFC (while real RFCs like SPKI do depend on
it), but I'm using it as if it had. I only dropped the "display hint"
stuff from the format, because I find it little to no use, while it
introduces quite a bit of complexity.

However I have no idea how things look like from a lisp point of view.


Hoping this helps,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Fri, 13 Aug 2010 02:10:23 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 13, 12:25�am, "Jeffrey R. Carter"

>> provided a more detailed grammar (using the
>> trailing '*' from regexps rather than {}) with some additional options.
> 
> Is one of them more standard than the other?

{} is from EBNF

   http://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_Form

used in the meaning "repeat or nothing". However earlier, at least I
remember it so, {} was rather used as mere grouping brackets of required
alternatives, and the number of repetitions where written as a superscript
number on the closing bracket. So the newish {P} were denoted as [P]*. And
[P] itself was an abbreviation of {P|} etc.

> I can assure you what I'm doing is completely conform with Rivest's
> standard proposition. I don't know how come it has not reached the
> status of full-fledged RFC (while real RFCs like SPKI do depend on
> it),

Probably because there already was a standard, the Abstract Syntax
Notation:

   http://en.wikipedia.org/wiki/ASN.1

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 13, 11:51 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Fri, 13 Aug 2010 02:10:23 -0700 (PDT), Natacha Kerensikova wrote:
> > I can assure you what I'm doing is completely conform with Rivest's
> > standard proposition. I don't know how come it has not reached the
> > status of full-fledged RFC (while real RFCs like SPKI do depend on
> > it),
>
> Probably because there already was a standard, the Abstract Syntax
> Notation:
>
>    http://en.wikipedia.org/wiki/ASN.1

But this standard has nothing to do with S-expressions.

And even when comparing their use cases (which will anyway be of no
help when it comes to re-using existing S-expressions or implementing
SPKI), ASN.1 doesn't seem to compete with S-expressions, as they
represent different trade-offs and design choices.

You can even add JSON and YAML to the mix and end up with 4 different
formats covering 4 different sets of needs. And while these 4 sets
probably intersect, I'm sure you can find examples of needs covered by
only one of them.


Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sat, 14 Aug 2010 03:36:34 -0700 (PDT), Natacha Kerensikova wrote:

> On Aug 13, 11:51�am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> On Fri, 13 Aug 2010 02:10:23 -0700 (PDT), Natacha Kerensikova wrote:
>>> I can assure you what I'm doing is completely conform with Rivest's
>>> standard proposition. I don't know how come it has not reached the
>>> status of full-fledged RFC (while real RFCs like SPKI do depend on
>>> it),
>>
>> Probably because there already was a standard, the Abstract Syntax
>> Notation:
>>
>> � �http://en.wikipedia.org/wiki/ASN.1
> 
> But this standard has nothing to do with S-expressions.

The intended application area, that is why S-expressions presumably didn't
make it.

> ASN.1 doesn't seem to compete with S-expressions, as they
> represent different trade-offs and design choices.

Since I am an unfortunate to deal a lot with the standards directly or
indirectly based on it, I would say it achieves the goal: these standards
are absolutely incomprehensible so that you could not get a slightest idea
what should be on the wire.

> You can even add JSON and YAML to the mix and end up with 4 different
> formats covering 4 different sets of needs. And while these 4 sets
> probably intersect, I'm sure you can find examples of needs covered by
> only one of them.

That depends on your definition of "needs". According to mine the *union*
of these sets is empty.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/13/2010 02:10 AM, Natacha Kerensikova wrote:
> On Aug 13, 12:25 am, "Jeffrey R. Carter"
> <spam.jrcarter....@spam.acm.org>  wrote:
>> Kerensikova (or is it ?)
>
> It's both ;-)  One of them is my real-life last name, as can be found
> on my ID or on copyright notices, while the other is a pseudonymous
> last name I've been using over the internet for something like a
> decade now. While it was never intended to more than a thin pseudonym,
> it was supposed to require a little more research to make the
> connection, here I just messed up with Google accounts (I really hate
> these browser interfaces, and now it makes me hate them even more, I
> wish I could find a NNTP server and a real user-friendly text-mode
> news reader).

Interesting. IIUC, "-ova" is a Bulgarian feminine patronymic suffix. Port� 
appears to be French ("carried"). "Natacha" might also be the French 
transliteration of what is usually transliterated in English as "Natasha".

I don't know about a text-mode news reader, but you can point a news reader at 
freenews.netfront.net without registering.

I seem to be getting seriously OT.

>> provided a more detailed grammar (using the
>> trailing '*' from regexps rather than {}) with some additional options.
>
> Is one of them more standard than the other? While I'm very familiar
> with regex, I only have a few theoretical notions about grammar, and
> almost nothing as practical experience with real grammar formats.

Both seem to be standards. {} is used in the ARM, Annex P, with its standard 
meaning of "zero or more", which is why I used it, and am surprised Brenta 
didn't understand it.

> I can assure you what I'm doing is completely conform with Rivest's
> standard proposition. I don't know how come it has not reached the
> status of full-fledged RFC (while real RFCs like SPKI do depend on
> it), but I'm using it as if it had. I only dropped the "display hint"
> stuff from the format, because I find it little to no use, while it
> introduces quite a bit of complexity.

Yes, things are murkier than I realized. S-expressions seem to have originated 
with Lisp, long before Rivest's proposal, but the cons-pair is an implementation 
detail, not an inherent part of the syntax, so lists can have zero or more elements.

-- 
Jeff Carter
"C++ is like giving an AK-47 to a monk, shooting him
full of crack and letting him loose in a mall and
expecting him to balance your checking account
'when he has the time.'"
Drew Olbrich
52

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Fri, 13 Aug 2010 12:23:38 -0700, Jeffrey Carter wrote:

> Interesting. IIUC, "-ova" is a Bulgarian feminine patronymic suffix.

in all Slavic languages, actually.

> "Natacha" might also be the French 
> transliteration of what is usually transliterated in English as "Natasha".

Natasha is Russian's diminutive of Natalia.

Compare Sasha-Alexander, Masha-Maria, Pasha-Paul, Grisha-Gregory,
Misha-Michael and so on.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Yannick Duchêne (Hibou57)]

Le Fri, 13 Aug 2010 21:23:38 +0200, Jeffrey Carter  
<spam.jrcarter.not@spam.not.acm.org> a écrit:
> Porté appears to be French ("carried").
Right
> "Natacha" might also be the French transliteration of what is usually  
> transliterated in English as "Natasha".
Right too

> I don't know about a text-mode news reader, but you can point a news  
> reader at freenews.netfront.net without registering.
I did not knew this one. However, I could also suggest the one I known :  
nntp.aioe.org/ (or to visit http://www.aioe.org/ )


-- 
There is even better than a pragma Assert: a SPARK --# check.
--# check C and WhoKnowWhat and YouKnowWho;
--# assert Ada;
--  i.e. forget about previous premises which leads to conclusion
--  and start with new conclusion as premise.

Re: S-expression I/O in Ada

[Randy Brukardt]

"Jeffrey Carter" <spam.jrcarter.not@spam.not.acm.org> wrote in message 
news:i4463s$hus$1@adenine.netfront.net...
...
> Both seem to be standards. {} is used in the ARM, Annex P, with its 
> standard meaning of "zero or more", which is why I used it, and am 
> surprised Brenta didn't understand it.

The grammar notation of Ada is described in section 1.1.4 
(http://www.adaic.com/standards/05rm/html/RM-1-1-4.html). One rather 
presumes that Ada programmers are at least somewhat familar with it, so it 
makes a good choice to use for grammars on this newgroup.

                       Randy.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 13, 9:23 pm, Jeffrey Carter
<spam.jrcarter....@spam.not.acm.org> wrote:
> On 08/13/2010 02:10 AM, Natacha Kerensikova wrote:
> > I can assure you what I'm doing is completely conform with Rivest's
> > standard proposition. I don't know how come it has not reached the
> > status of full-fledged RFC (while real RFCs like SPKI do depend on
> > it), but I'm using it as if it had. I only dropped the "display hint"
> > stuff from the format, because I find it little to no use, while it
> > introduces quite a bit of complexity.
>
> Yes, things are murkier than I realized. S-expressions seem to have originated
> with Lisp, long before Rivest's proposal, but the cons-pair is an implementation
> detail, not an inherent part of the syntax, so lists can have zero or more elements.

That's exactly how I feel towards this whole S-expression deal.

I didn't know anything about lisp or cons pairs (except that lisp
involves lots of nested parentheses), I just stumbled on Rivest's de
facto standard when I was looking for a simple and easy-to-use
external data format.

And since then I've always been alone with that format. Usually it was
because people freaked out because of parentheses, now it's because
I'm doing everything wrong (i.e. not according to the one true cons
pair idea), but the end result is the same.

Anyway, I knew from the beginning switching from C to Ada won't
increase the number of people using, or even only reading, my code.
Starting from 1 (me), the result was very predictable.


Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Simon Wright]

Natacha Kerensikova <lithiumcat@gmail.com> writes:

> (I really hate these browser interfaces, and now it makes me hate them
> even more, I wish I could find a NNTP server and a real user-friendly
> text-mode news reader).

news.eternal-september.org, and emacs.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 12, 8:55 pm, Jeffrey Carter
<spam.jrcarter....@spam.not.acm.org> wrote:
> The example was something like
>
> (TCP-something (host abc) (port 42) )
>
> So I think the grammar is
>
> S-expression = '(' Element {' ' Element} ')'
>
> Element = Atom | S-expression

Actually a S-expression list can be empty, and a S-expression can be a
single atom, or a list of atom, without parentheses.

The grammar would be more something like:

SP = TAB | SPACE | CR | LF  -- whitespace

S-expression = SP* (Atom | List)* SP*

List = '(' S-expression* ')'

Atom = Token-Atom | Quoted-String-Atom | Hexadecimal-Atom | Base64-
Atom | Verbatim-Atom

And the rest I'm not sure can be described by a grammar (especially
verbatim encoded atoms), and that's without entering into details like
brace encoding or display hints (which sucks anyway). The whitespace
being optional because generally atoms don't need separators, token-
encoded atoms being the exception (unfortunately all atoms from my
example are token-encoded).


Hoping it helps,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

I wrote on comp.lang.ada:
> I made some fixes and I now consider my S-Expression parser[1] feature-
> complete as of revision b13ccabbaf227bad264bde323138910751aa2c2b.
> There may still be some bugs though, and the error reporting (to
> diagnose syntax errors in the input) is very primitive.
>
> Highlights:
> * the procedure S_Expression.Read is a quasi-recursive descent
> parser.  "Quasi" because it only recurses when encountering an opening
> parenthesis, but processes atoms without recursion, in the same finite
> state machine.
> * the parser reads each character exactly once; there is no push_back
> or backtracking involved.  This makes the parser suitable to process
> standard input on the fly.
> * to achive this, I had to resort to using exceptions instead of
> backtracking; this happens when the parser encounters a ')'
> immediately after an S-Expression (atom or list).
> * the parser also supports lists of the form (a b c) (more than 2
> elements) and properly translates them to (a (b c)).  The Append()
> procedure that does this is also public and available to clients.
> * the parser does not handle incomplete input well.  If it gets
> Ada.IO_Exceptions.End_Error in the middle of an S-Expression, it will
> return an incomplete, possibly empty, S-Expression rather than report
> the error.  I'll try to improve that.
> * the test.adb program demonstrates how to construct an S-Expression
> tree in memory (using cons()) and then sending it to a stream (using
> 'Write).
> * the test.adb program also demonstrates how to read an S-Expression
> from a stream (using 'Read) and then traverse the in-memory tree
> (using car(), cdr()).
>
> [1] http://green.ada-france.org:8081/branch/changes/org.ludovic-brenta.s_expressions
>
> I have not yet tested the parser on your proposed input (IIUC,
> consisting of two S-Expressions with a missing closing parenthesis).
> I think this will trigger the bug where End_Error in the middle of an
> S-Expression is not diagnosed.
>
> I also still need to add the proper GPLv3 license text on each file.
>
> I'll probably add support for Lisp-style comments (starting with ';'
> and ending at end of line) in the future.


As of revision b60f80fba074431aeeffd95aa273a1d4fc81bf41, I now handle
end-of-stream in all situations and (I believe) react appropriately.

I have now tested the parser against this sample input file:

$ cat test_input
(tcp-connect (host foo.bar) (port 80))

(tcp-connect ((host foo.bar) (port 80))
(tcp-connect (host foo.bar) (port 80)))

$ ./test < test_input
(tcp-connect ((host foo.example) (port 80)))
Parsing the S-Expression: (tcp-connect ((host foo.bar) (port 80)))
Writing the S-Expression: (tcp-connect ((host foo.bar) (port 80)))
Parsing the S-Expression: Exception name: TEST.SYNTAX_ERROR
Message: Expected atom with value 'host'

Writing the S-Expression: (tcp-connect ((((host foo.bar) (port 80)) (tcp-connect (host foo.bar))) (port 80)))

raised S_EXPRESSION.SYNTAX_ERROR : Found ')' at the start of an expression



The very first line of output is the result of 'Write of an S-Expression
constructed from a hardcoded TCP_Connect record.

"Parsing" refers to the high-level part of the parsing that traverses
the in-memory S-Expression tree and converts it to a TCP_Connect_T
record.

"Writing" refers to both halves of the low-level parsing: reading the
character stream, producing the in-memory S-Expression tree, and
converting it back to a character stream.

The TEST.SYNTAX_ERROR is because the high-level parser found a list
instead of the expected atom "host"; this is because of the extra '('
before "host" at line 3 in the input.

The S_EXPRESSION.SYNTAX_ERROR is because the low-level parser found an
extra ')' at the very end of line 4 in the input; it coalesced lines 3
and 4 into a single, valid, S-Expression, and was expecting a new
S-Expression starting with '('.

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/01/2010 05:17 AM, Natacha Kerensikova wrote:
>
> To describe briefly S-expressions, I consider them to be the simplest
> existing data organization beyond raw sequences of bits. They are
> basically lists of elements, each element being either a list or an
> atom, and atoms being raw sequences of bits.

You might very well be able to use something like:

package Byte_Lists is new Ada.Containers.Vectors (Index_Type => Positive, 
Element_Type => System.Storage_Elements.Storage_Element);

type Content_ID is (Atom, List);

type S_Expression;
type S_Expression_Ptr is access all S_Expression;

type S_Expression_Element (Content : Content_ID := Atom) is record
    case Content is
    when Atom =>
       Byte : Byte_Lists.Vector;
    when List =>
       Ptr : S_Expression_Ptr;
    end case;
end record;

package S_Expression_Lists is new Ada.Containers.Doubly_Linked_Lists 
(Element_Type => S_Expression_Element);

type S_Expression is new S_Expression_Lists.List;

If you can use unbounded strings as Brenta suggested, instead of an unbounded 
array of bytes (Storage_Element), then this would be even simpler.

-- 
Jeff Carter
"People called Romanes, they go the house?"
Monty Python's Life of Brian
79

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 1, 8:25 pm, Jeffrey Carter <spam.jrcarter....@spam.not.acm.org>
wrote:
> You might very well be able to use something like:
>
> [snip]

Thanks a lot for the example, it really looks like what I'm used to
(at least in C).

I have to admit I don't really grasp the difference between the vector
you use and the Storage_Array, but it seems I have to research it by
myself before asking here.

> If you can use unbounded strings as Brenta suggested, instead of an unbounded
> array of bytes (Storage_Element), then this would be even simpler.

Actually I'm a bit reluctant to use Ada strings for atoms, because I'm
afraid it might somehow interpret the data (e.g. locale or encoding
related stuff, or trouble with NUL inherited from C). I occasionally
include binary data in S-expressions, and while I keep it on the disk
as a text file (using hexadecimal or base-64 encoding), it is the S-
expression library's responsibility to load it into memory as the
original binary data.

On the other hand, most of my atoms are indeed strings, and Character
definition from A.1.35 looks exactly like a perfect mapping to bytes.
So if Ada strings have no issue with embedded NUL or non-graphics
character, and if binary data can be losslessly recovered once stored
into an Ada string, it could be the best type for atoms. It will
probably be a while before I reach the level of reading the Reference
Manual cover-to-cover, does anyone knows whether those "if"s are
guaranteed by the standard?


Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natacha Kerensikova writes:
> On Aug 1, 8:25 pm, Jeffrey Carter <spam.jrcarter....@spam.not.acm.org>
> wrote:
>> You might very well be able to use something like:
>>
>> [snip]
>
> Thanks a lot for the example, it really looks like what I'm used to
> (at least in C).
>
> I have to admit I don't really grasp the difference between the vector
> you use and the Storage_Array, but it seems I have to research it by
> myself before asking here.

The vector manages its own memory, you can grow and shrink it at will.
With a Storage_Array you must do that manually by allocating,
reallocating and freeing as needed.

>> If you can use unbounded strings as Brenta suggested, instead of an
>> unbounded array of bytes (Storage_Element), then this would be even
>> simpler.
>
> Actually I'm a bit reluctant to use Ada strings for atoms, because I'm
> afraid it might somehow interpret the data (e.g. locale or encoding
> related stuff, or trouble with NUL inherited from C).

No, they won't.  Ada does not need NUL since it has proper arrays.

> I occasionally include binary data in S-expressions, and while I keep
> it on the disk as a text file (using hexadecimal or base-64 encoding),
> it is the S- expression library's responsibility to load it into
> memory as the original binary data.

I would still store the S-expressions in memory as Unbouded_Strings,
read straight from the file (i.e. in the hexadecimal or base-64
encoding).  Then I would provide conversion subprograms for each data
type as needed on top of that.

> On the other hand, most of my atoms are indeed strings, and Character
> definition from A.1.35 looks exactly like a perfect mapping to bytes.
> So if Ada strings have no issue with embedded NUL or non-graphics
> character, and if binary data can be losslessly recovered once stored
> into an Ada string, it could be the best type for atoms. It will
> probably be a while before I reach the level of reading the Reference
> Manual cover-to-cover, does anyone knows whether those "if"s are
> guaranteed by the standard?

While Ada strings indeed have no problems with embedded NULs or
non-ASCII characters (the character set is Latin-1, not ASCII), it is
unwise to use character strings to store things that are not characters.

Like I said, you should see the character representation as the
low-level (storage) representation, then provide conversion subprograms
for the few non-String data types that you need.

-- 
Ludovic Brenta.

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 01 Aug 2010 21:53:58 +0200, Ludovic Brenta wrote:

> I would still store the S-expressions in memory as Unbouded_Strings,
> read straight from the file (i.e. in the hexadecimal or base-64
> encoding).

Strings is preferable to Unbouded_Strings in all cases when string content
is not changed, once the string is created. This is 90% of all cases,
including this one.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/01/2010 12:43 PM, Natacha Kerensikova wrote:
>
> I have to admit I don't really grasp the difference between the vector
> you use and the Storage_Array, but it seems I have to research it by
> myself before asking here.

Storage_Array is a simple array; all instances of the type have a fixed size. So 
to have a dynamic size, you'd have to do dynamic allocation and memory 
management yourself. Vectors is an unbounded array abstraction, with all 
allocation and memory management hidden from the client, and tested by many users.

> On the other hand, most of my atoms are indeed strings, and Character
> definition from A.1.35 looks exactly like a perfect mapping to bytes.
> So if Ada strings have no issue with embedded NUL or non-graphics
> character, and if binary data can be losslessly recovered once stored
> into an Ada string, it could be the best type for atoms. It will
> probably be a while before I reach the level of reading the Reference
> Manual cover-to-cover, does anyone knows whether those "if"s are
> guaranteed by the standard?

String is just an array of Character; there's nothing special about it. No 
characters have special significance. Unbounded_String is just an unbound 
variant of String, as the vector would be an unbounded variant of Storage_Array. 
However, if you store non-Character data, then it would be more appropriate to 
use a vector of Storage_Element. Possibly you might want to recognize your 
extensive use of strings by having 3 variants, one for String, one for List, and 
one for anything else.

-- 
Jeff Carter
"People called Romanes, they go the house?"
Monty Python's Life of Brian
79

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 8/1/10 2:17 PM, Natacha Kerensikova wrote:

> My question here is about how to represent the atoms in Ada. In C it
> was merely a void pointer and a size, but it seems more difficult in
> Ada because of the strong typing. Because it's up to the application
> to make sense (i.e. type) out of the raw sequences of bits in atoms,
> the S-expression library has to handle them as a sort of untyped
> memory chunk. Do you know of a way to handle that?

Maybe there is a reasonably simple way of a different
kind---one that emphasizes the type system and leaves
distinguishing types to the compiler and the plain old
run-time system.

First, to represent an atom, I'd choose Ada's limited types,
since they imply no copying and no predefined equality
operators:  Each object of such a type will be unique
by definition of "limited". Then I'd derive types for symbols,
numbers, ..., from this type.  And use construction functions
to make symbols, numbers, ...

Second, to perform I/O of objects of such a type (Atom
or derived from Atom; also List) via textual representation
of S-expressions, Ada's Generic_Dispatching_Constructor seems
ideal: textual representations of S-expressions will include
a tag that guides the I/O routines. This is completely portable,
and transparent too, unlike 'Input and 'Output attributes or
similar.

A.k.a. "Object factory functions", there is more in the Rationale
http://www.adaic.com/standards/05rat/html/Rat-2-6.html

Some ad hoc definitions and a test case for the Lisp stuff,
just to illustrate the idea involving limited types
(I couldn't resist):


with Ada.Finalization;  use Ada;

package S_Expressions is
    
    pragma Preelaborate (S_Expressions);
    
    --
    --  Provide definitions typical of Lisp (ad hoc)
    --
    
    type EQ_Type is limited Interface;
    --   Force EQ of Common Lisp.
    
    function EQ (Left, Right : EQ_Type) return Boolean is abstract;
    --  Objects of the same kind may be compared in specific ways,
    --  e.g. by numeric value.

    function "=" (Left, Right: EQ_Type'Class) return Boolean;
    -- General comparison operation for each object in the system,
    -- dispatches to EQ if Left'Tag = Right'Tag. False otherwise.
    
    
    subtype Defined is Finalization.Limited_Controlled;
    --  Every object in the system is unique and has no "=",
    --  since all objects are limited.  Also adds hooks for
    --  garbage collection.
    
    
    --
    --  Root of everything not a list
    --
    
    type Atom is abstract new Defined and EQ_Type with private;
    --  Symbols, Numbers, Character strings, ...
    
    function EQ (Left, Right : Atom) return Boolean is abstract;


    -- Kinds of atoms, with construction functions:
    
    type Symbol(<>) is new Atom with private;
    
    function Make_Symbol (Name : String) return Symbol;
    
    
    type Numeric is range 0 .. 666;   -- or something better suited
    type Number(<>) is new Atom with private;
    
    function Make_Number (Initial : Numeric) return Number;
    
    
    --
    --  Lists, construction and access to elements:
    --
    
    type List is new Defined and EQ_Type with private;
    
    Nil: constant Symbol;
    
    function Cons (Car, Cdr : Defined'Class) return List;
    
    type Ptr is access constant Defined'Class;
    -- read-only access to elements of lists
    function Car (L : List) return Ptr;
    function Cdr (L : List) return Ptr;
    
private
    --
    --  Equality functions of all types that need to implement EQ_Type
    --
    function EQ (Left, Right : Symbol) return Boolean;
    function EQ (Left, Right : Number) return Boolean;
    function EQ (Left, Right : List) return Boolean;
    -- ...

    type Atom is abstract new Defined and EQ_Type with null record;
    
    
    type Symbol_Chars is access constant String;
    type Symbol(Name : Symbol_Chars) is new Atom with null record;
    
    Nil_Chars : aliased constant String := "";
    Nil: constant Symbol := Symbol'(Atom with Name => Nil_Chars'Access);
    
    type Number(Value : Numeric) is new Atom with null record;
    
    
    type List is new Defined and EQ_Type with
       record
	 Info : Ptr;
	 Next : Ptr;
       end record;
    
    
end S_Expressions;


with S_Expressions; use S_Expressions;
procedure S_Test is
    
    A: Symbol := Make_Symbol("Foo");
    B: Symbol := Make_Symbol("Foo");
    
    M: Number := Make_Number(42);
    N: Number := Make_Number(42);
    
    S : List := Cons(Make_Number(1),
			  Cons(Make_Number(2),
			       Cons(Make_Number(3), Nil)));
    Sx : List := Cons(Make_Number(1),
			  Cons(Make_Number(2),
			       Cons(Make_Number(3), Nil)));
begin
    if A /= B then   -- same name!!
       raise Program_Error;
    end if;
    
    if M /= N then   -- same value!!
       raise Program_Error;
    end if;
    
    if A = N then   -- different kinds of atoms!!
       raise Program_Error;
    end if;
    
    if S = Sx then  -- different lists (even though same values)!!
       raise Program_Error;
    end if;
  
    if S /= S then   --  same object!!
       raise Program_Error;
    end if;
  
    declare
       Cadr : constant Ptr := Car (List( Cdr (S).all));
    begin
       if Number(Cadr.all) /= Make_Number(2) then  -- same value!!
	 raise Program_Error;
       end if;
    end;
end S_Test;



with Ada.Tags;
with System;
package body S_Expressions is
    
    -- Comparison
    
    function EQ (Left, Right : Symbol) return Boolean is
    begin
       return Left.Name.all = Right.Name.all;
    end EQ;
    
    function EQ (Left, Right : Number) return Boolean is
    begin
       return Left.Value = Right.Value;
    end EQ;
    
    function EQ (Left, Right : List) return Boolean is
       use type System.Address;
    begin
       return Left'Address = Right'Address;
    end EQ;
    
    function "=" (Left, Right: EQ_Type'Class) return Boolean is
       use type Ada.Tags.Tag;
    begin
       if Left'Tag /= Right'Tag then
	 return False;
       else
	 return EQ (Left, Right);
       end if;
    end "=";
    
    -- Making atoms
    
    function Make_Number (Initial : Numeric) return Number is
    begin
       return Number'(Defined with
		       Value => Initial);
    end Make_Number;

    function Make_Symbol (Name : String) return Symbol is
    begin
       return Symbol'(Defined with
		       Name => new String'(Name));
    end Make_Symbol;
    
    
    -- Lists
    
    function Cons (Car, Cdr : Defined'Class) return List is
    begin
       return List'(Defined with
		     Info => Car'Unchecked_Access,
		   Next => Cdr'Unchecked_Access);
    end Cons;
    
    function Car (L : List) return Ptr is
    begin
       return L.Info;
    end Car;
    
    function Cdr (L : List) return Ptr is
    begin
       return L.Next;
    end Cdr;
    
end S_Expressions;


-- Georg

Re: S-expression I/O in Ada

[Georg Bauhaus]

(The same again, if you can be bothered, untabified and after
ada-remove-trailing-spaces.)

with Ada.Finalization;  use Ada;

package S_Expressions is

    pragma Preelaborate (S_Expressions);

    --
    --  Provide definitions typical of Lisp (ad hoc; no Lisp expert
    --  here.)
    --

    type EQ_Type is limited Interface;

    function EQ (Left, Right : EQ_Type) return Boolean is abstract;
    -- objects of the same kind may be compared in specific ways

    function "=" (Left, Right: EQ_Type'Class) return Boolean;
    -- General comparison operation for each object in the system,
    -- dispatches to EQ if Left'Tag = Right'Tag.


    subtype Defined is Finalization.Limited_Controlled;
    --  Every object in the system is unique and has no "=",
    --  since all objects are limited.  Also adds hooks for
    --  garbage collection.


    --
    --  Root of everything not a list
    --

    type Atom is abstract new Defined and EQ_Type with private;
    --  Symbols, Numbers, Character strings, ...


    function EQ (Left, Right : Atom) return Boolean is abstract;

    -- Kinds of atoms, with construction functions:

    type Symbol(<>) is new Atom with private;

    function Make_Symbol (Name : String) return Symbol;


    type Numeric is range 0 .. 666;   -- or something better suited
    type Number(<>) is new Atom with private;

    function Make_Number (Initial : Numeric) return Number;


    --
    --  Lists, construction and access to elements:
    --

    type List is new Defined and EQ_Type with private;

    Nil: constant Symbol;

    function Cons (Car, Cdr : Defined'Class) return List;

    type Ptr is access constant Defined'Class;
    -- read-only access to elements of lists
    function Car (L : List) return Ptr;
    function Cdr (L : List) return Ptr;

private
    --
    --  Equality functions of all types that need to implement EQ_Type
    --
    function EQ (Left, Right : Symbol) return Boolean;
    function EQ (Left, Right : Number) return Boolean;
    function EQ (Left, Right : List) return Boolean;
    -- ...

    type Atom is abstract new Defined and EQ_Type with null record;


    type Symbol_Chars is access constant String;
    type Symbol(Name : Symbol_Chars) is new Atom with null record;

    Nil_Chars : aliased constant String := "";  -- (static)
    Nil: constant Symbol := Symbol'(Atom with Name => Nil_Chars'Access);

    type Number(Value : Numeric) is new Atom with null record;


    type List is new Defined and EQ_Type with
       record
          Info : Ptr;
          Next : Ptr;
       end record;


end S_Expressions;


with S_Expressions; use S_Expressions;
procedure S_Test is

    A: Symbol := Make_Symbol("Foo");
    B: Symbol := Make_Symbol("Foo");

    M: Number := Make_Number(42);
    N: Number := Make_Number(42);

    S : List := Cons(Make_Number(1),
                           Cons(Make_Number(2),
                                Cons(Make_Number(3), Nil)));
    Sx : List := Cons(Make_Number(1),
                           Cons(Make_Number(2),
                                Cons(Make_Number(3), Nil)));
begin
    if A /= B then   -- same name!!
       raise Program_Error;
    end if;

    if M /= N then   -- same value!!
       raise Program_Error;
    end if;

    if A = N then   -- different kinds of atoms!!
       raise Program_Error;
    end if;

    if S = Sx then  -- different lists (even though same values)
       raise Program_Error;
    end if;

    if S /= S then
       raise Program_Error;
    end if;

    declare
       Cadr : constant Ptr := Car (List( Cdr (S).all));
    begin
       if Number(Cadr.all) /= Make_Number(2) then  -- same value!!
          raise Program_Error;
       end if;
    end;
end S_Test;



with Ada.Tags;
with System;
package body S_Expressions is

    -- Comparison

    function EQ (Left, Right : Symbol) return Boolean is
    begin
       return Left.Name.all = Right.Name.all;
    end EQ;

    function EQ (Left, Right : Number) return Boolean is
    begin
       return Left.Value = Right.Value;
    end EQ;

    function EQ (Left, Right : List) return Boolean is
       use type System.Address;
    begin
       return Left'Address = Right'Address;
    end EQ;

    function "=" (Left, Right: EQ_Type'Class) return Boolean is
       use type Ada.Tags.Tag;
    begin
       if Left'Tag /= Right'Tag then
          return False;
       else
          return EQ (Left, Right);
       end if;
    end "=";

    -- Making atoms

    function Make_Number (Initial : Numeric) return Number is
    begin
       return Number'(Defined with
                        Value => Initial);
    end Make_Number;

    function Make_Symbol (Name : String) return Symbol is
    begin
       return Symbol'(Defined with
                        Name => new String'(Name));
    end Make_Symbol;


    -- Lists

    function Cons (Car, Cdr : Defined'Class) return List is
    begin
       return List'(Defined with
                      Info => Car'Unchecked_Access,
                    Next => Cdr'Unchecked_Access);
    end Cons;

    function Car (L : List) return Ptr is
    begin
       return L.Info;
    end Car;

    function Cdr (L : List) return Ptr is
    begin
       return L.Next;
    end Cdr;

end S_Expressions;

Re: S-expression I/O in Ada

[anon]

In <547afa6b-731e-475f-a7f2-eaefefb25861@k8g2000prh.googlegroups.com>, Natacha Kerensikova <lithiumcat@gmail.com> writes:
>Hi,
>
>I'm trying to learn Ada, coming from a C background. The first thing I
>planned to code is a S-expression parser, because it's quite easy (at
>least in C, less than 1000 lines including comments and memory
>handling (dynamic arrays reinvented)) and very useful considering
>almost all my existing programs use S-expressions as a serialization
>format.
>
>To describe briefly S-expressions, I consider them to be the simplest
>existing data organization beyond raw sequences of bits. They are
>basically lists of elements, each element being either a list or an
>atom, and atoms being raw sequences of bits.
>
>While I'm still not deep enough into Ada to know how to represent the
>lists, I guess there won't be major issues, I think I can handle it
>myself (though pointers and hints would still be welcome).
>
>My question here is about how to represent the atoms in Ada. In C it
>was merely a void pointer and a size, but it seems more difficult in
>Ada because of the strong typing. Because it's up to the application
>to make sense (i.e. type) out of the raw sequences of bits in atoms,
>the S-expression library has to handle them as a sort of untyped
>memory chunk. Do you know of a way to handle that?
>
>Please correct me if I'm wrong, but my guess would be that the S-
>expression library would provide a Sexp_Atom type, which refers to the
>untyped memory chunks, and the application would have procedures to
>convert back and forth between Sexp_Atom and whatever types it
>internally uses (i.e. serialization and deserialization procedures).
>However the library would provide these procedures for the most common
>types (e.g. strings and numeric types).
>
>Though it looks like a fine Ada API (at least to my eyes), I have
>absolutely no idea about how to implement the library. How to define
>the application-opaque Sexp_Atom type? How to read Sexp_Atom objects
>from a file? How to build them from Ada types? How to write them back
>to disk?
>
>Thanks for your help,
>Natacha


In creating the LISP interpreter, most use:

For outside environment to internal:

  Word or Numeric Text (Text_IO) --> Tokenizer --> Internal Text tree
                                               --> Internal List tree

If the I/O is to become a VS to your program use the Direct_IO package 
to create a Indexed binary file where the words and string phases would 
be stored in one file while the internal access pointers would be stored 
in the second.

  Internal Text tree      <-->  Direct_IO ( Text_Sexp ) 
  Internal List tree      <-->  Direct_IO ( List_Sexp ) 

In this design, short term (online) memory storage the access pointer 
are still valid. In long term storage you would need to remap the 
List tree to map the Text tree using indexes.
 
So, for long time storage it is better to use the Text_IO, and rebuild the 
outside environment version of the structures.

  Internal Word tree  --> unTokenizer --> Word or Numeric Text (Text_IO)
  Internal List tree  --> 

Re: S-expression I/O in Ada

[Shark8]

Natacha,

I took a bit of a stab at writing an SExpression handler, note that
it's not a parser [text-file -> SExpression] but rather what (I think)
you were terming in-memory. It has stream-support so you can directly
save/load an SExpression's structure & contents and it should be easy
to make it "parenthesis preservative" in the parser if you so desire.

I [semi-]plan on making a child-package for the parser "package
SExpression.Text" or somesuch for storing/loading the SExpressions
from text [that is, parsing them].

---------------------------------------------
-- SExpression.ads
---------------------------------------------

With
 Ada.Streams,
 Ada.Streams.Stream_IO;


Package SExpression is

   Type Byte is mod 2**8;
    For Byte'Size use 8;

   Type Binary_Data_Type is Array( Positive Range <> ) Of Byte;
     Pragma Pack(Binary_Data_Type);

   Type Node_Data_Type is ( Binary_Data, String_Data, Number_Data );
   Type Node_Type( Data_Type : Node_Data_Type ) is private;
     Procedure Create( Node: out Node_Type; Data: in
String           );
     Procedure Create( Node: out Node_Type; Data: in
Binary_Data_Type );
     Procedure Create( Node: out Node_Type; Data: in
Long_Integer     );

   Type List_Type;
   Type SExpression_Type(List_Size : Natural) is private;
     Function Is_List ( SExp : in SExpression_Type) Return Boolean;
     Function Create  ( Node : Node_Type ) Return SExpression_Type;
     Procedure Append ( SExp : in out SExpression_Type; Node :
Node_Type );
     Procedure Append ( Front: in out SExpression_Type;
                        Back : in     SExpression_Type );
     Procedure Prepend( SExp : in out SExpression_Type; Node :
Node_Type );
     Procedure Prepend( Front: in     SExpression_Type;
                        Back : in out SExpression_Type );

   Type List_Type is Array(Positive Range <>) of
        Not Null Access SExpression_Type;

     Procedure Create( SExp : out SExpression_Type; List : in
List_Type );

   Package Stream is
      Use Ada.Streams;

      Procedure Write_Bin ( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : in Binary_Data_Type );
      Procedure Read_Bin  ( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : out Binary_Data_Type );

      Procedure Write_Node( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : in Node_Type );
      Procedure Read_Node ( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : out Node_Type );

      Procedure Write_Sexp( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : in SExpression_Type );
      Procedure Read_SExp ( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : out SExpression_Type );

      Procedure Write_List( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : in List_Type );
      Procedure Read_List ( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : out List_Type );

   end Stream;


private


   Type Node_Type( Data_Type : Node_Data_Type ) is Record
      case Data_Type is
         when Binary_Data => B : Access Binary_Data_Type;
         when String_Data => S : Access String;
         when Number_Data => N : Long_Integer;
      end case;
   end record;

    For Node_Type'Write use Stream.Write_Node;
    For Node_Type'Read  use Stream.Read_Node;

   Type SExpression_Type(List_Size : Natural) is Record
      case List_Size is
         when 0      => Data: Not Null Access Node_Type;
         when others => List: List_Type(1..List_Size);
      end case;
   end record;

    For SExpression_Type'Read  use Stream.Read_SExp;
    For SExpression_Type'Write use Stream.Write_SExp;

    For List_Type'Read  use Stream.Read_List;
    For List_Type'Write use Stream.Write_List;

    For Binary_Data_Type'Read  use Stream.Read_Bin;
    For Binary_Data_Type'Write use Stream.Write_Bin;
end SExpression;

---------------------------------------------
-- SExpression.adb
---------------------------------------------

Package body SExpression is

   -- Internal Node-Creation Functions.
   Function Make_Node( Data: String ) Return Node_Type is
   begin
      Return Result: Node_Type( Data_Type => String_Data ) do
         Result.S:= New String'(Data);
      end Return;
   end Make_Node;

   Function Make_Node( Data: Binary_Data_Type ) Return Node_Type is
   begin
      Return Result: Node_Type( Data_Type => Binary_Data ) do
         Result.B:= New Binary_Data_Type'(Data);
      end Return;
   end Make_Node;

   Function Make_Node( Data: Long_Integer ) Return Node_Type is
   begin
      Return Result: Node_Type( Data_Type => Number_Data ) do
         Result.N:= Data;
      end Return;
   end Make_Node;


   -- Public Node-Creation Procedures {wraps node-creation functions}
   Procedure Create ( Node: out Node_Type; Data: in String ) is
   begin
      Node:= Make_Node(Data);
   end Create;

   Procedure Create ( Node: out Node_Type; Data: in Binary_Data_Type )
is
   begin
      Node:= Make_Node( Data );
   end Create;

   Procedure Create ( Node: out Node_Type; Data: in Long_Integer ) is
   begin
      Node:= Make_Node(Data);
   end Create;

Pragma Inline( Create );

   Function Is_List(SExp : in SExpression_Type) Return Boolean is
   begin
      Return SExp.List_Size > 0;
   end Is_List;
   Pragma Inline(Is_List);

   Function Create( List : in List_Type ) Return SExpression_Type is
   begin
      Return Result: SExpression_Type( List_Size => List'Length ) do
         Result.List:= List;
      end return;
   end Create;

   Procedure Create( SExp : out SExpression_Type; List : in
List_Type ) is
   begin
      SExp:= Create( List );
   end Create;

   Function Create( Node : Node_Type ) Return SExpression_Type is
   begin
      Return Result : SExpression_Type( List_Size => 0 ) do
         Result.Data:= New Node_Type'( Node );
      end return;
   end Create;

   Procedure Append ( SExp : in out SExpression_Type; Node :
Node_Type ) is
   begin
      if Is_List(SExp) then
         SExp:= Create( Sexp.List & New
SExpression_Type'(Create(Node)) );
      else
         SExp:= Create(   New
SExpression_Type'(Create(SExp.Data.All))
			& New SExpression_Type'(Create(Node))
		      );
      end if;
   end Append;

   Procedure Prepend( SExp : in out SExpression_Type; Node :
Node_Type ) is
   begin
      if Is_List(SExp) then
         SExp:= Create( New SExpression_Type'(Create(Node)) &
Sexp.List);
      else
         SExp:= Create(   New SExpression_Type'(Create(Node))
			& New SExpression_Type'(Create(SExp.Data.All))
		      );
      end if;
   end Prepend;

   Procedure Append ( Front: in out SExpression_Type;
		      Back : in     SExpression_Type ) is
   begin
      if Is_List(Back) then
         Front:= Create( Front.List & Back.List);
      else
         Append( SExp => Front, Node => Back.Data.All );
      end if;
   end Append;

   Procedure Prepend( Front: in     SExpression_Type;
		      Back : in out SExpression_Type ) is
   begin
      if Is_List(Front) then
         Back:= Create( Front.List & Back.List);
      else
         Prepend( SExp => Back, Node => Front.Data.All );
      end if;
   end Prepend;

   package body Stream is
      Procedure Write_Bin ( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : in Binary_Data_Type ) is
      begin
         Positive'Write( Stream, Item'Length );
         For Index in Item'Range loop
            Byte'Write( Stream, Item(Index) );
         end loop;
      end Write_Bin;

      Procedure Read_Bin  ( Stream: Not Null Access
Root_Stream_Type'Class;
                           Item : out Binary_Data_Type ) is

         Function Get_Elements( Length: Positive ) Return
Binary_Data_Type is
         begin
            Return Result: Binary_Data_Type(1..Length) do
               For Index in Result'Range loop
                  Byte'Read( Stream, Result(Index) );
               end loop;
            end Return;
         end Get_Elements;

         Length: Positive;
      begin
         Positive'Read( Stream, Length );
         Item:= Get_Elements( Length );
      end Read_Bin;


      Procedure Write_Node( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : in Node_Type ) is
      begin
         Node_Data_Type'Write( Stream, Item.Data_Type );

         Case Item.Data_Type is
            When Binary_Data => Binary_Data_Type'Write( Stream,
Item.B.All );
            When String_Data => String'Write( Stream, Item.S.All );
            When Number_Data => Long_Integer'Write( Stream, Item.N );
         end Case;
      end Write_Node;

      Procedure Read_Node ( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : out Node_Type ) is
         Data_Type : Node_Data_Type;
      begin
         Node_Data_Type'Read( Stream, Data_Type );

            Case Data_Type is
            When Binary_Data =>
               Item.B:= New Binary_Data_Type'( 1..0 => <> );
               Declare
                  Binary_Temp  : Access Binary_Data_Type:=
                    New Binary_Data_Type'( 1..0 => 16#FF# );
               Begin
                  Binary_Data_Type'Read( Stream, Binary_Temp.All );
                  Item:= Make_Node( Data => Binary_Temp.All );
               End;
            When String_Data =>
               declare
                  String_Temp  : Access String:= New String'("");
               begin
                  String'Read( Stream, String_Temp.All);
                  Item:= Make_Node( Data => String_Temp.All );
               end;
            When Number_Data =>
               declare
                  Temp : Long_Integer;
               begin
                  Long_Integer'Read( Stream, Temp );
                  Item:= Make_Node( Temp );
               end;
            end Case;
      end Read_Node;




      Procedure Write_SExp( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : in SExpression_Type ) is
      begin
         Natural'Write( Stream, Item.List_Size );
         if Item.List_Size = 0 then
            Node_Type'Write( Stream, Item.Data.All );
         else
            List_Type'Write( Stream, Item.List );
         end if;
      end Write_SExp;

      Procedure Read_SExp ( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : out SExpression_Type ) is
         List_Size : Natural;
      begin
         Natural'Read( Stream, List_Size );
         if List_Size = 0 then
            Declare
               Temp : Access Node_Type:= New
Node_Type'( Make_Node(0) );
            Begin
               Node_Type'Read( Stream, Temp.All );
               Item:= Create( Temp.All );
            End;
         else
            Declare
               Dummy: Aliased SExpression_Type:=
Create(Make_Node(0));
               Temp : List_Type:= (1..List_Size =>
Dummy'Unchecked_Access );
            Begin
               List_Type'Read( Stream, Temp );
               Item:= Create( Temp );
            End;
         end if;
      end Read_SExp;


      Procedure Write_List( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : in List_Type ) is
      begin
         Positive'Write(Stream, Item'Length);
         For Index in Item'Range loop
            SExpression_Type'Write( Stream, Item(Index).All );
         end loop;
      end Write_List;

      Procedure Read_List ( Stream: Not Null Access
Root_Stream_Type'Class;
			    Item : out List_Type ) is

         Function Get_Elements( Length: Positive ) Return List_Type is
            Working : Access SExpression_Type;
         begin
            Working:= New SExpression_Type'( Create(Make_Node(0)) );
            Return Result: List_Type(1..Length) do
               For Index in Result'Range loop
                  SExpression_Type'Read( Stream, Working.All );
                  Result(Index):= New
SExpression_Type'( Working.All );
               end loop;
            end Return;
         end Get_Elements;

         Length : Positive;
      begin
         Positive'Read(Stream, Length);
         Item:= Get_Elements( Length );
      end Read_List;


   end Stream;

end SExpression;

Re: S-expression I/O in Ada

[Shark8]

Reading about the Cons-es I figure I should show how the constructs I
made can be made exactly equivalent to a series of Cons-pairs.
Unless I'm completely misunderstanding/misinterpreting the Lisp idea
of a list being an item or an item followed by a list.

     Function Construct_Cons_Pair( Head, Tail : in Node_Type )
                                  Return SExpression_Type is
   begin
      Return Result: SExpression_Type (List_Size => 2) do
         Result.List(1):= New SExpression_Type'( Create(Head) );
         Result.List(2):= New SExpression_Type'( Create(Tail) );
      end return;
   end Construct_Cons_Pair;

     Function Construct_Cons_Pair( Head : in Node_Type;
				   Tail : in SExpression_Type)
                                  Return SExpression_Type is
   begin
      Case Tail.List_Size is
         When 0 => Return Construct_Cons_Pair( Head,
Tail.Data.All    );
         When 1 => Return Construct_Cons_Pair( Head,
Tail.List(1).All );
         When Others =>
            Return Result: SExpression_Type (List_Size => 2) do
               Result.List(1):= New SExpression_Type'( Create(Head) );
               Result.List(2):= New SExpression_Type'( Tail );
            end return;
      end case;
   end Construct_Cons_Pair;


   Function Is_Cons_Pair_Construct( S: in SExpression_Type ) Return
Boolean is
    -- Perhaps this would be better named Is_Lisp_Construct....
   begin
      Return ((S.List_Size = 2) And Then
		((Not Is_List(S.List(1).All)) And
		  Is_Cons_Pair_Construct( S.List(2).All )))
              OR  S.List_Size = 0;
   end Is_Cons_Pair_Construct;

Maybe I should make a Construct_Cons_Pair( Head : in SExpression_Type;
Tail : in Node_Type) for orthogonality's sake... but that would go
against the Lisp-way.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 13, 1:26 am, Shark8 <onewingedsh...@gmail.com> wrote:
> I took a bit of a stab at writing an SExpression handler, note that
> it's not a parser [text-file -> SExpression] but rather what (I think)
> you were terming in-memory.

Indeed, that's what my idea of my still-unnamed in-memory S-expression
object package would look like.

What I find surprising is that you seem to include type information in
the Node type (which actually represent a S-expression atom, while S-
expression nodes usually include lists too). Is it a typical Ada way
of doing it?

I find it surprising because the underlying S-expression format (as
standardized by Rivest, I know almost nothing about cons pairs or
lisp) cannot encoding that information. I would expect a S-expression
object in memory to reflect only what can be encoded to and decoded
from S-expression files.

In fact, one can consider S-expressions as a heterogeneous container,
in that each atom can represent an object of any type (while losing
the type information, which has to be retrieved from somewhere else),
in contrast to thing like vectors, whose items are all of the same
type. Does anybody know an example of heterogeneous container in Ada?
I'm not sure how Ada generics can be leveraged in such a case.

Another interesting point is that you chose arrays to represent S-
expression lists. Your code seems currently unable to represent empty
lists (which are legal at least in Rivest's S-expressions) but I guess
I can't be too difficult to correct. But it raises the question of
array vs linked list for a S-expression object implementation.

I'm not sure S-expressions would ever be used in a time-critical
application, so the usual argument of cache-friendliness of arrays is
pretty weak here. S-expressions are meant to be used for I/O where the
bottleneck is most likely to be.

Ada arrays are statically sized, which can be an issue for parsing,
because S-expression format doesn't encode list length, so a list has
to be built by pushing nodes one after the other until encountering
the end-of-list marker. But I think that should be easily worked
around by using vectors for unfinished S-expression lists, and once
the list is completely parsed build the array from the vector, and
reset the vector for further list parsing.

However for S-expression objects dynamically created into memory, the
static size of arrays might make some operations much less efficient
than with linked lists.

Any idea I'm missing about this implementation choice? Would it be
worth to try and make such a package generic over the container type
(array vs linked list vs something else?)?


Thanks for your code example,
Natacha

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 13.08.10 10:56, Natacha Kerensikova wrote:

> Ada arrays are statically sized,

Not quite, but I think you mean that once an array is allocated, you
can't add more components than are declared by the index constraint?
Example:

procedure Uses_Array (N : Positive; ...) is

   X : array (1 .. N) of Character;  -- now fixed to N compoments

begin...

But Ada's Container packages can provide for the desired flexibility.
There are containers of homogeneous components and containers of
heterogeneous components.  The latter have _Indefinite in their
package's names.  Example:

 package Vecs is new Ada.Containers.Indefinite_Vectors
     (Element_Type => Types.Heterogenous'Class,
      Index_Type => Positive,
      "=" => Types.Eq);

(Vecs.Vector objects will grow as needed.)

Another option is to instantiate a "definite" container generic with
a pointer type, which is definite.  Its access values will point
to any object in some hierarchy.  (This can mean pointers to
any objects that share the same interface (Ada 2005 keyword),
so types need not even be rooted at the same one parent type,
I think). For example

   package S_Lists is new Ada.Containers.Doubly_Linked_Lists
     (Element_Type => Some_Hier_Ptr,
      "=" => ...);


Georg

Re: S-expression I/O in Ada

[Shark8]

On Aug 13, 2:56 am, Natacha Kerensikova <lithium...@gmail.com> wrote:
> On Aug 13, 1:26 am, Shark8 <onewingedsh...@gmail.com> wrote:
>
> > I took a bit of a stab at writing an SExpression handler, note that
> > it's not a parser [text-file -> SExpression] but rather what (I think)
> > you were terming in-memory.
>
> Indeed, that's what my idea of my still-unnamed in-memory S-expression
> object package would look like.
>
> What I find surprising is that you seem to include type information in
> the Node type (which actually represent a S-expression atom, while S-
> expression nodes usually include lists too). Is it a typical Ada way
> of doing it?
>
> I find it surprising because the underlying S-expression format (as
> standardized by Rivest, I know almost nothing about cons pairs or
> lisp) cannot encoding that information. I would expect a S-expression
> object in memory to reflect only what can be encoded to and decoded
> from S-expression files.
>
> In fact, one can consider S-expressions as a heterogeneous container,
> in that each atom can represent an object of any type (while losing
> the type information, which has to be retrieved from somewhere else),
> in contrast to thing like vectors, whose items are all of the same
> type. Does anybody know an example of heterogeneous container in Ada?
> I'm not sure how Ada generics can be leveraged in such a case.
>
> Another interesting point is that you chose arrays to represent S-
> expression lists. Your code seems currently unable to represent empty
> lists (which are legal at least in Rivest's S-expressions) but I guess
> I can't be too difficult to correct. But it raises the question of
> array vs linked list for a S-expression object implementation.
>
> I'm not sure S-expressions would ever be used in a time-critical
> application, so the usual argument of cache-friendliness of arrays is
> pretty weak here. S-expressions are meant to be used for I/O where the
> bottleneck is most likely to be.
>
> Ada arrays are statically sized, which can be an issue for parsing,
> because S-expression format doesn't encode list length, so a list has
> to be built by pushing nodes one after the other until encountering
> the end-of-list marker. But I think that should be easily worked
> around by using vectors for unfinished S-expression lists, and once
> the list is completely parsed build the array from the vector, and
> reset the vector for further list parsing.
>
> However for S-expression objects dynamically created into memory, the
> static size of arrays might make some operations much less efficient
> than with linked lists.
>
> Any idea I'm missing about this implementation choice? Would it be
> worth to try and make such a package generic over the container type
> (array vs linked list vs something else?)?
>
> Thanks for your code example,
> Natacha

Re: S-expression I/O in Ada

[Shark8]

Sorry about the empty-reply earlier... I think I double-tapped the
reply/send button

Earlier you wrote this:
Consider these two objects:

L--A--tcp-connect
   |
   L--A--host
   |  |
   |  A--foo.example
   |
   L--A--port
      |
      A--80

and:

L--A--tcp-connect
   |
   L--L--A--host
      |  |
      |  A--foo.example
      |
      L--A--port
         |
         A--80

"I'm sure we can agree on the fact that these are two different and
non-
equivalent objects: not the same topology and not even the same number
of nodes. So I'd say a proper S-expression library has to be able to
deal with both of them without mixing them up."


These aren't valid representations of the LISP-like structure --
Remember that it [a list] is defined as a single item followed my a
list OR a single item alone. Therefore, *all* internal nodes in the
previous drawings should be A-L type and the leaf-nodes should be the
[terminal] A-type node.


> What I find surprising is that you seem to include type information in
> the Node type (which actually represent a S-expression atom, while S-
> expression nodes usually include lists too). Is it a typical Ada way
> of doing it?

I'm tempted to say 'yes' but I'm a newcomer to the language (from a
Pascal/Delphi background); so take that with a grain of salt. In my
opinion types are your friends {they give you information about what
your datum/variable *is*} and you shouldn't strip type-information
without good reason. That was my main reasoning for making the node-
type carry information about its contents; you *could* use arrays-of-
bytes/bits but, unless you need to, why?

> I find it surprising because the underlying S-expression format (as
> standardized by Rivest, I know almost nothing about cons pairs or
> lisp) cannot encoding that information. I would expect a S-expression
> object in memory to reflect only what can be encoded to and decoded
> from S-expression files.


My data-type can exactly represent any encoding that is in an S-
expression text-file [excepting an null-list], which by the LISP-
definition doesn't exist. An empty-list could be simulated with a
single-node containing the null-string "array (1..0) of Character." As
it stands you can think of my SExpression_type, when the List_Size
discriminant is /= 0, as being "everything within that [balanced]
parentheses-pair."

>
> In fact, one can consider S-expressions as a heterogeneous container,
> in that each atom can represent an object of any type (while losing
> the type information, which has to be retrieved from somewhere else),
> in contrast to thing like vectors, whose items are all of the same
> type. Does anybody know an example of heterogeneous container in Ada?
> I'm not sure how Ada generics can be leveraged in such a case.

You can simulate them, as I have done, by isolating "all the types it
could be" and making a variant-record type that ANY Ada container
could then hold. The "big disadvantage" is that the elements cannot
have the same name; that is to say that you cannot have a record with
an element named 'Data' which changes it's type on what discriminant
was passed.


> Another interesting point is that you chose arrays to represent S-
> expression lists. Your code seems currently unable to represent empty
> lists (which are legal at least in Rivest's S-expressions) but I guess
> I can't be too difficult to correct. But it raises the question of
> array vs linked list for a S-expression object implementation.

Well, I showed you how it could be simulated.

> I'm not sure S-expressions would ever be used in a time-critical
> application, so the usual argument of cache-friendliness of arrays is
> pretty weak here. S-expressions are meant to be used for I/O where the
> bottleneck is most likely to be.

*nod* - I didn't choose arrays for I/O cache at all; I chose them
because the number of elements is known [we've either parsed it into
memory or are building it in-place], and that all elements of that
list would have the same type -- basically a non-null pointer to some
other SExpression_type --and that SExpression could be either a list
itself or a terminal node. Add to that that Ada has nice array
manipulation facilities and I think that's enough justification to use
them.


> Ada arrays are statically sized, which can be an issue for parsing,
> because S-expression format doesn't encode list length,

That can be handled by the parser; after all it has to figure out list
lengths in order to produce the list-object, right?
Using the streams provided is a way to cut out the parser altogether,
if that's an issue, and you can just load the SExpressions directly...
think of it as being able to save/load the parse-structure that the
GCC back-end takes, you would then have a way to "compile" a program
without having to parse its source code again. {Not a very useful
feature when the program text is oft changed and would have to be re-
parsed, but for a configuration file which might remain unchanged over
the life of the application it's attendant to the story's a bit
different.}

> so a list has
> to be built by pushing nodes one after the other until encountering
> the end-of-list marker.

You're confusing parsing the list with the finished/internal list-
structure itself, I think.
Parsing is just a way of saying "I'm reading this [in order to produce
this structure]."

> But I think that should be easily worked
> around by using vectors for unfinished S-expression lists, and once
> the list is completely parsed build the array from the vector, and
> reset the vector for further list parsing.

Sure you could do that too. The Append & Prepend procedures do just
that -- without involving a vector as an intermediate.

> However for S-expression objects dynamically created into memory, the
> static size of arrays might make some operations much less efficient
> than with linked lists.

I doubt that; like you just mentioned you could wait until the entire
list has been read into-memory before converting it to a static array.

> Any idea I'm missing about this implementation choice? Would it be
> worth to try and make such a package generic over the container type
> (array vs linked list vs something else?)?

I don't know; like I said, I'm a newcomer to Ada and while I really
like it and the underlying ideology it would be foolish of me to claim
that I have the deep-understanding of a lot of the nuances.

> Thanks for your code example,
> Natacha

You're welcome.

Re: S-expression I/O in Ada

[Natacha Kerensikova]

On Aug 13, 11:48 pm, Shark8 <onewingedsh...@gmail.com> wrote:
> These aren't valid representations of the LISP-like structure --
> Remember that it [a list] is defined as a single item followed my a
> list OR a single item alone. Therefore, *all* internal nodes in the
> previous drawings should be A-L type and the leaf-nodes should be the
> [terminal] A-type node.

So you're saying what you call "item" can't be a list? Then you can't
make nested lists, right? That sure makes lisp much simpler…

> > What I find surprising is that you seem to include type information in
> > the Node type (which actually represent a S-expression atom, while S-
> > expression nodes usually include lists too). Is it a typical Ada way
> > of doing it?
>
> I'm tempted to say 'yes' but I'm a newcomer to the language (from a
> Pascal/Delphi background); so take that with a grain of salt. In my
> opinion types are your friends {they give you information about what
> your datum/variable *is*} and you shouldn't strip type-information
> without good reason. That was my main reasoning for making the node-
> type carry information about its contents; you *could* use arrays-of-
> bytes/bits but, unless you need to, why?

Because the external format defined by the S-expression standard has
no room for type information. While you can indeed create typed nodes
from in-memory objects, whose type is well known, when you'll have to
create a node from a stream/file/whatever following the S-expression
standard, you won't have any way to retrieve the type information.

> > Ada arrays are statically sized, which can be an issue for parsing,
> > because S-expression format doesn't encode list length,
>
> That can be handled by the parser; after all it has to figure out list
> lengths in order to produce the list-object, right?
> Using the streams provided is a way to cut out the parser altogether,
> if that's an issue, and you can just load the SExpressions directly...

Yes but that wouldn't help using existing S-expression files, which is
the whole point of my S-expression library project.

> > so a list has
> > to be built by pushing nodes one after the other until encountering
> > the end-of-list marker.
>
> You're confusing parsing the list with the finished/internal list-
> structure itself, I think.
> Parsing is just a way of saying "I'm reading this [in order to produce
> this structure]."

Yes, except "this" has a well-defined on-disk representation, which
surely is not the same as any internal in-memory representation of any
GCC object.

This well-defined on-disk representation happens to give no clue about
a list size, so it has to be deduced from the list contents read until
the end-of-list marker is reached. Such an on-disk representation
means that the in-memory representation used by the parser has to be
able to grow on demand. I don't know whether Ada arrays can be tricked
into that, but from the little Ada I know vectors are more indicated
here. That's what I meant, do you still see any confusion?

> > However for S-expression objects dynamically created into memory, the
> > static size of arrays might make some operations much less efficient
> > than with linked lists.
>
> I doubt that; like you just mentioned you could wait until the entire
> list has been read into-memory before converting it to a static array.

Yes, and at that paragraphed I had already moved on to in-memory S-
expression creation from nothing, having finished with S-expression
reading from parsing a stream (or whatever).


Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Natasha Kerensikova]

Hello,

here is my first try at a S_Expressions package, which is supposed to
handle the in-memory representations. I first planned to begin with the
Sexp_Stream I imagined, but it proved much more difficult than I
thought, so I went for the easier package first.

Even though I'm much more used to C than to Ada, I have the feeling it's
horribly ugly and that using access types all over the place like I did
is extremely poor. Yet I just can't find out exactly how it's wrong, nor
how to get it right.

Would any other you be kind enough to have a look at it, and point me
where I did wrong and explain me how wrong it is, be it on high-level
package design to low-level implementation choice and anything in
between including code style.


Thanks in advance for your help,
Natacha




with Ada.Finalization;

package S_Expressions is

   -- atom related types
   -- they are public to allow atom <--> object converters anywhere
   type Octet is range 0 .. 255;
   type Atom_Data is array (Integer range <>) of Octet;

   function To_String (From: in Atom_Data) return String;

   -- S-expression private types
   type Node_Content is (Atom, List);
   type Node (Content: Node_Content) is
      new Ada.Finalization.Controlled with private;
   type Access_Node is access Node;

   -- Node accessors
   function Get_Atom(node: not null Access_Node) return Atom_Data;
   function Get_Child(node: not null Access_Node) return Access_Node;
   function Get_Next(node: not null Access_Node) return Access_Node;
   function Get_Node_Type(node: not null Access_Node) return Node_Content;

   -- Node constructors
   function New_Atom_Node(contents: Atom_Data; next: Access_Node)
     return Access_Node;
   function New_List_Node(child, next: Access_Node)
     return Access_Node;

private

   type Access_Atom_Data is access Atom_Data;

   type Node (Content: Node_Content) is
      new Ada.Finalization.Controlled with record
         Next: Access_Node;
         case Content is
            when Atom =>
               Atom: Access_Atom_Data;
            when List =>
               Child: Access_Node;
         end case;
      end record;

   overriding procedure Adjust(object: in out Node);
   overriding procedure Finalize(object: in out Node);

end S_Expressions;




with Ada.Unchecked_Deallocation;

package body S_Expressions is

   procedure Free is
      new Ada.Unchecked_Deallocation (Node, Access_Node);
   procedure Free is
      new Ada.Unchecked_Deallocation (Atom_Data, Access_Atom_Data);

   -- Atom data duplication
   function Duplicate(object: in Access_Atom_Data) return Access_Atom_Data is
      new_object: Access_Atom_Data;
   begin
      if object /= Null then
         new_object := new Atom_Data(object'Range);
         new_object.All := object.All;
      end if;
      return new_object;
   end Duplicate;


   -- Deep node duplication
   function Duplicate(object: in Access_Node) return Access_Node is
      new_object: Access_Node;
   begin
      if object /= Null then
         case object.Content is
            when Atom =>
               new_object := new Node (Atom);
               new_object.Atom := Duplicate(object.Atom);
            when List =>
               new_object := new Node (List);
               new_object.Child := Duplicate(object.Child);
         end case;
         new_object.Next := Duplicate(object.Next);
      end if;
      return new_object;
   end Duplicate;


   -- atom to string converter
   function To_String(from: Atom_Data) return String is
      to: String(from'Range);
   begin
      for i in from'Range loop
         to(i) := Character'Val(from(i) + 1);
      end loop;
      return to;
   end To_String;


   -- deep copy of node objects
   overriding procedure Adjust(object: in out Node) is
   begin
      case object.Content is
         when Atom =>
            object.Atom := Duplicate(object.Atom);
         when List =>
            object.Child := Duplicate(object.Child);
      end case;
      object.Next := Duplicate(object.Next);
   end Adjust;


   -- deep release of node objects
   overriding procedure Finalize(object: in out Node) is
   begin
      case object.Content is
         when Atom => Free(object.Atom);
         when List => Free(object.Child);
      end case;
      Free(object.Next);
   end;


   -- Node acessors
   function Get_Node_Type(node: not null Access_Node) return Node_Content is
   begin
      return node.Content;
   end;

   function Get_Atom(node: not null Access_Node) return Atom_Data is
   begin
      return node.Atom.All;
   end;

   function Get_Child(node: not null Access_Node) return Access_Node is
   begin
      return node.Child;
   end;

   function Get_Next(node: not null Access_Node) return Access_Node is
   begin
      return node.Next;
   end;


   -- Node constructors
   function New_Atom_Node(contents: Atom_Data; next: Access_Node)
     return Access_Node is
      new_node: Access_Node;
   begin
      new_node := new Node (Atom);
      new_node.Atom := Duplicate (contents'Access);
      new_node.Next := next;
      return new_node;
   end;

   function New_List_Node(child, next: Access_Node)
     return Access_Node is
      new_node: Access_Node;
   begin
      new_node := new Node (List);
      new_node.Child := Duplicate (child);
      new_node.Next := next;
      return new_node;
   end;

end S_Expressions;

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/17/2010 10:01 AM, Natasha Kerensikova wrote:
>
> Even though I'm much more used to C than to Ada, I have the feeling it's
> horribly ugly and that using access types all over the place like I did
> is extremely poor. Yet I just can't find out exactly how it's wrong, nor
> how to get it right.

I would agree. Access types and values should not appear in the visible part of 
a package specification if at all possible.

> Would any other you be kind enough to have a look at it, and point me
> where I did wrong and explain me how wrong it is, be it on high-level
> package design to low-level implementation choice and anything in
> between including code style.

It seemed to me that you could implement this without any access types or 
values, so I gave it a shot. The following has been compiled:

private with Ada.Containers.Indefinite_Vectors;
private with Ada.Containers.Vectors;

package S_Expressions is
    type Atom is private;

    function To_Atom (Value : in String) return Atom;
    function To_String (Value : in Atom) return String;

    generic -- Object_Conversions
       type Element (<>) is limited private;
    package Object_Conversions is
       function To_Atom (Value : in Element) return Atom;
       function To_Object (Value : in Atom) return Element;
    end Object_Conversions;

    type S_Expression (<>) is tagged private; -- An Atom or a list of S_Expression.

    function Make (Item : in Atom) return S_Expression;

    function Is_Atom (Expression : in S_Expression) return Boolean;

    Not_An_Atom : exception;

    function To_Atom (Expression : in S_Expression) return Atom;
    -- Raises Not_An_Atom if not Is_Atom (Expression);

    Empty_List : constant S_Expression;

    Not_A_List : exception;

    function Append (Onto : in S_Expression; Value : in S_Expression) return 
S_Expression;
    -- Returns a list consisting of the exisiting list Onto with Value appended 
to it.
    -- Raises Not_A_List if Is_Atom (Onto);

    procedure Iterate
       (Over : in S_Expression; Process : not null access procedure (Expression 
: in S_Expression; Continue : in out Boolean) );
    -- Passes each element of Over to Process in turn, with Continue => True.
    -- Returns immediately if Process sets Continue to False; remaining elements 
will not be processed.
    -- Raises Not_A_List if Is_Atom (Over).
private -- S_Expressions
    Byte_Size : constant := 8;

    type Byte_Value is mod 2 ** Byte_Size;

    package Byte_Lists is new Ada.Containers.Vectors (Index_Type => Positive, 
Element_Type => Byte_Value);

    type Atom is record
       Value : Byte_Lists.Vector;
    end record;

    type Root is tagged null record;

    package Lists is new Ada.Containers.Indefinite_Vectors (Index_Type => 
Positive, Element_Type => Root'Class);

    type S_Expression (Is_Atom : Boolean) is new Root with record
       case Is_Atom is
       when False =>
          List : Lists.Vector;
       when True =>
          Value : Atom;
       end case;
    end record;

    Empty_List : constant S_Expression := (Is_Atom => False, List => 
Lists.Empty_Vector);
end S_Expressions;

with Ada.Unchecked_Conversion;

package body S_Expressions is
    function To_Atom (Value : in String) return Atom is
       Result : Atom;
    begin -- To_Atom
       All_Characters : for I in Value'range loop
          Result.Value.Append (New_Item => Character'Pos (Value (I) ) );
       end loop All_Characters;

       return Result;
    end To_Atom;

    function To_String (Value : in Atom) return String is
       Result : String (Value.Value.First_Index .. Value.Value.Last_Index);
    begin -- To_String
       All_Bytes : for I in Result'range loop
          Result (I) := Character'Val (Value.Value.Element (I) );
       end loop All_Bytes;

       return Result;
    end To_String;

    package body Object_Conversions is
       type Byte_List is array (Positive range <>) of Byte_Value;

       function To_Atom (Value : in Element) return Atom is
          Num_Bytes : constant Positive := (Value'Size + Byte_Size - 1) / Byte_Size;

          subtype Element_List is Byte_List (1 .. Num_Bytes);

          function Convert is new Ada.Unchecked_Conversion (Source => Element, 
Target => Element_List);

          Byte : Element_List renames Convert (Value);

          Result : Atom;
       begin -- To_Atom
          All_Bytes : for I in Byte'range loop
             Result.Value.Append (New_Item => Byte (I) );
          end loop All_Bytes;

          return Result;
       end To_Atom;

       function To_Object (Value : in Atom) return Element is
          subtype Element_List is Byte_List (Value.Value.First_Index .. 
Value.Value.Last_Index);

          function Convert is new Ada.Unchecked_Conversion (Source => 
Element_List, Target => Element);

          Byte : Element_List;
       begin -- To_Object
          All_Bytes : for I in Byte'range loop
             Byte (I) := Value.Value.Element (I);
          end loop All_Bytes;

          return Convert (Byte);
       end To_Object;
    end Object_Conversions;

    function Make (Item : in Atom) return S_Expression is
       -- null;
    begin -- Make
       return S_Expression'(Is_Atom => True, Value => Item);
    end Make;

    function Is_Atom (Expression : in S_Expression) return Boolean is
       -- null;
    begin -- Is_Atom
       return Expression.Is_Atom;
    end Is_Atom;

    function To_Atom (Expression : in S_Expression) return Atom is
       -- null;
    begin -- To_Atom
       if not Expression.Is_Atom then
          raise Not_An_Atom;
       end if;

       return Expression.Value;
    end To_Atom;

    function Append (Onto : in S_Expression; Value : in S_Expression) return 
S_Expression is
       Result : S_Expression (Is_Atom => False);
    begin -- Append
       if Onto.Is_Atom then
          raise Not_A_List;
       end if;

       Result.List := Onto.List;
       Result.List.Append (New_Item => Value);

       return Result;
    end Append;

    procedure Iterate
       (Over : in S_Expression; Process : not null access procedure (Expression 
: in S_Expression; Continue : in out Boolean) )
    is
       Continue : Boolean := True;
    begin -- Iterate
       if Over.Is_Atom then
          raise Not_A_List;
       end if;

       All_Expressions : for I in Over.List.First_Index .. Over.List.Last_Index loop
          Process (Expression => S_Expression (Over.List.Element (I) ), Continue 
=> Continue);

          exit All_Expressions when not Continue;
       end loop All_Expressions;
    end Iterate;
end S_Expressions;

I think this includes all necessary functionality to build and process 
S-expressions, though you might have some additional operations you might like 
to add. Reading it might prove instructional, whether you like this approach or not.

-- 
Jeff Carter
"I'm a lumberjack and I'm OK."
Monty Python's Flying Circus
54

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-17, Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> wrote:
> It seemed to me that you could implement this without any access types or 
> values, so I gave it a shot. The following has been compiled:
>
> private with Ada.Containers.Indefinite_Vectors;
> private with Ada.Containers.Vectors;

I have indeed forgotten about Ada containers. However after having read
A.18.2 and A.18.3, it seems to me that Doubly_Linked_Lists are more
appropriate than vectors: the recursive structure of S-expressions can
make assignment (i.e. deep copies) quite expensive, and
Doubly_Linked_Lists seem designed primarily to avoid copy of contained
objects (though I may be mistaken). Usual operations on S-expressions
are append and traversal, on which Vectors don't provide a significant
improvement (unlike e.g. random access).

Does it make sense or am I missing something?

Btw reading A.18 I discovered the idea of Cursor types, which I find
very nice: I'm used (in C) to hand over Nodes because they also contain
enough information to progress in the S-expression structure, but it
looks like Cursor would do it in a more Ada-ish way.

>     generic -- Object_Conversions
>        type Element (<>) is limited private;
>     package Object_Conversions is
>        function To_Atom (Value : in Element) return Atom;
>        function To_Object (Value : in Atom) return Element;
>     end Object_Conversions;

I'm not fond of that part, because it's actually a memory dump, while
atom contents are supposed to be a serialized representation of
(atomic) objects.

I'm aware that it's best to hide as much as possible about type
definitions, but I think in that particular case having Atom definition
public is justified.

The idea is that my other objects, clients of S_Expressions, will be
able to make a S-expression representing themselves. So they will need
subprograms to make S-expressions. Complex objects will only have to use
the container aspect of S-expressions, assembling S-expression
representation provided by their sub-objects. But at some point some
objects will be simple enough to be represented as atoms, and knowledge
about how to serialize these atomic objects into atom objects can't be
part of S_Expressions. Which means S_Expressions has to expose details
of atom type.

On the other hand, there will be relatively few atomic objects compared
to complex objects. Would it possible to hide atom type definition in a
subpackage or something, that would only be with'ed in atomic object
packages?

Maybe something like:
private with S_Expressions.Atoms;
package S_Expressions is
   type Atom is private;
   ...

It doesn't seem to work due to a circular dependency. But there anything
to do along that line of thoughts?

>     type Root is tagged null record;
>
>     package Lists is new Ada.Containers.Indefinite_Vectors (Index_Type => 
> Positive, Element_Type => Root'Class);
>
>     type S_Expression (Is_Atom : Boolean) is new Root with record
>        case Is_Atom is
>        when False =>
>           List : Lists.Vector;
>        when True =>
>           Value : Atom;
>        end case;
>     end record;

I don't understand why you need Indefinite_Vectors here. They have been
presented to me as heterogeneous vector containers, but here Lists would
be homogeneous, containing only S_Expression items.
I'm not sure about the need of Root either, but my guess is that it's to
provide a common root to all objects contained in the
Indefinite_Vectors.

But what makes S_Expression an indefinite type?



Thanks a lot for your review,
Natacha

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natasha Kerensikova wrote on comp.lang.ada:
> I have indeed forgotten about Ada containers. However after having read
> A.18.2 and A.18.3, it seems to me that Doubly_Linked_Lists are more
> appropriate than vectors: the recursive structure of S-expressions can
> make assignment (i.e. deep copies) quite expensive, and
> Doubly_Linked_Lists seem designed primarily to avoid copy of contained
> objects (though I may be mistaken).

I think you are mistaken:

ARM A.18.3(159/2):
  The execution of an assignment_statement for a list shall have the
effect of copying the elements from the source list object to the
target list object.

However, if the elements contain access values, then whether _they_
perform a deep or shallow copy is your decision; the Element_Type may
be controlled or not.

[...]

>>     generic -- Object_Conversions
>>        type Element (<>) is limited private;
>>     package Object_Conversions is
>>        function To_Atom (Value : in Element) return Atom;
>>        function To_Object (Value : in Atom) return Element;
>>     end Object_Conversions;
>
> I'm not fond of that part, because it's actually a memory dump, while
> atom contents are supposed to be a serialized representation of
> (atomic) objects.
>
> I'm aware that it's best to hide as much as possible about type
> definitions, but I think in that particular case having Atom definition
> public is justified.
>
> The idea is that my other objects, clients of S_Expressions, will be
> able to make a S-expression representing themselves. So they will need
> subprograms to make S-expressions. Complex objects will only have to use
> the container aspect of S-expressions, assembling S-expression
> representation provided by their sub-objects. But at some point some
> objects will be simple enough to be represented as atoms, and knowledge
> about how to serialize these atomic objects into atom objects can't be
> part of S_Expressions. Which means S_Expressions has to expose details
> of atom type.
>
> On the other hand, there will be relatively few atomic objects compared
> to complex objects. Would it possible to hide atom type definition in a
> subpackage or something, that would only be with'ed in atomic object
> packages?

Yes; in my implementation I resolved that problem by providing a
generic package Serialization containing the To_Atom and From_Atom
operations, for any discrete type. For other, arbitrary types, I
provide From_Blob and To_Blob; a client of my S_Expression package can
thus hex-encode any object by overlaying it with a Storage_Array, e.g.

function To_Atom (Object : Arbitrary_Type) return S_Expression.T is
   Blob : Storage_Array (1 .. Object'Size / System.Storage_Unit);
   for Blob'Address use Object'Address;
begin
   return S_Expression.To_Atom (Blob);
end To_Atom;

All this without exposing the internals of an atom.

In my implementation, it is also possible to turn any kind of object
into an atom by providing a pair of To_String, From_String operations,
but these operations actually perform the serialization you were
trying to hide ;/

> Maybe something like:
> private with S_Expressions.Atoms;
> package S_Expressions is
>    type Atom is private;
>    ...
>
> It doesn't seem to work due to a circular dependency. But there anything
> to do along that line of thoughts?
>
>>     type Root is tagged null record;
>>
>>     package Lists is new Ada.Containers.Indefinite_Vectors (Index_Type =>
>> Positive, Element_Type => Root'Class);
>>
>>     type S_Expression (Is_Atom : Boolean) is new Root with record
>>        case Is_Atom is
>>        when False =>
>>           List : Lists.Vector;
>>        when True =>
>>           Value : Atom;
>>        end case;
>>     end record;
>
> I don't understand why you need Indefinite_Vectors here. They have been
> presented to me as heterogeneous vector containers, but here Lists would
> be homogeneous, containing only S_Expression items.
> I'm not sure about the need of Root either, but my guess is that it's to
> provide a common root to all objects contained in the
> Indefinite_Vectors.
>
> But what makes S_Expression an indefinite type?

The type S_Expression is definite but Root'Class, being class-wide, is
indefinite. (It is considered to have at least one unknown
discriminant, which is the tag).

This is also the reason why Lists.Vector, in the example above, is
really heterogeneous; it can contain objects of different types that
extend Root.

--
Ludovic Brenta.

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-18, Ludovic Brenta <ludovic@ludovic-brenta.org> wrote:
> Natasha Kerensikova wrote on comp.lang.ada:
>> I have indeed forgotten about Ada containers. However after having read
>> A.18.2 and A.18.3, it seems to me that Doubly_Linked_Lists are more
>> appropriate than vectors: the recursive structure of S-expressions can
>> make assignment (i.e. deep copies) quite expensive, and
>> Doubly_Linked_Lists seem designed primarily to avoid copy of contained
>> objects (though I may be mistaken).
>
> I think you are mistaken:
>
> ARM A.18.3(159/2):
>   The execution of an assignment_statement for a list shall have the
> effect of copying the elements from the source list object to the
> target list object.
>
> However, if the elements contain access values, then whether _they_
> perform a deep or shallow copy is your decision; the Element_Type may
> be controlled or not.

Actually that's not exactly what I meant. What I meant is that in the
usual operations on Vectors (e.g. increasing storage space to complete
an Append call) there are more hidden assignments (and therefore, deep
copies) than on Doubly_Linked_Lists.

But having another look at A.18, it seems I was indeed mistaken.

Then I don't really understand the point of having both Vectors and
Doubly_Linked_Lists. The interface of Vectors is a strict superset of
the interface of Doubly_Linked_Lists, and the only difference in
complexity (which is advice anyway, so non-guaranteed) is on Prepend.

The only notable difference I've found is that
Doubly_Linked_Lists.Cursor is more resistant to container changes than
Vectors.Cursor, but I find it a bit weak to justify the existence of
both types.

What I am missing there?

>> On the other hand, there will be relatively few atomic objects compared
>> to complex objects. Would it possible to hide atom type definition in a
>> subpackage or something, that would only be with'ed in atomic object
>> packages?
>
> Yes; in my implementation I resolved that problem by providing a
> generic package Serialization containing the To_Atom and From_Atom
> operations, for any discrete type. For other, arbitrary types, I
> provide From_Blob and To_Blob; a client of my S_Expression package can
> thus hex-encode any object by overlaying it with a Storage_Array, e.g.

Obviously, discrete type serialization alone is not enough, as I will
have non-discrete atomic object to somehow turn into atoms. And your
_Blob functions suffer from the same issue as Jeffrey Carter's
implementation in that it's just a memory dump.

> In my implementation, it is also possible to turn any kind of object
> into an atom by providing a pair of To_String, From_String operations,
> but these operations actually perform the serialization you were
> trying to hide ;/

Actually I'm not trying to hide the serialization, I only want to have
it happen in the client object package, because I assume the object is
better qualified to know how to serialize itself rather than the
S-expression package.

I'm aware we don't agree on the S-expression concept, but my (and
Rivest's) atoms are only a sequence of octets. In that case From_String
and From_Blob actually perform the same operation, which is taking an
array of Ada things which are basically (on most platforms) octets, and
turn them into the internal Atom type. That's why I'm not at ease with
From_String and From_Blob, I feel it should be one function.

Let's take for example a Wide_Wide_String (e.g. because that's how my
application handles strings internally), which is not so good as an
example because a memory dump wouldn't be so much of an issue (except
maybe for endianness). Let's further assume I want it serialized in
UTF-8. How do I do that?

From_String is not good, because Ada Strings are supposed to represent
iso-8859-1 code points, which a UTF-8 string is not. From_Blob is not
good because the point of using UTF-8 is to still be in a certain text
representation, and because it involves turning the Wide_Wide_String
into a UTF-8 string type, then into a Storage_Array, then at last into
an Atom; while UTF-8 is already supposed be a sequnce-of-octet encoding.
See my point?

Now of course I could hide Atom type away and allow conversions to and
from Storage_Element_Array or Stream_Element_Array, which would amount
to a (IMHO useless) direct copy.

As I said, I understand the point of hiding type definitions, and the
main reason I think it's a valid exception here it that the Atom type is
already defined by the S-expression standard I follow, so there is
little to no chance of ever having to change it.

>> I don't understand why you need Indefinite_Vectors here. They have been
>> presented to me as heterogeneous vector containers, but here Lists would
>> be homogeneous, containing only S_Expression items.
>> I'm not sure about the need of Root either, but my guess is that it's to
>> provide a common root to all objects contained in the
>> Indefinite_Vectors.
>>
>> But what makes S_Expression an indefinite type?
>
> The type S_Expression is definite but Root'Class, being class-wide, is
> indefinite. (It is considered to have at least one unknown
> discriminant, which is the tag).

Yes, I understood that (and I was trying in vain to mean that when I
explained my perception of Root use). What I don't understand is why
using a Root'Class indefinite vector instead of a S_Expression vector?


Sorry for my poor expression and thanks for your comments,
Natacha Porté

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natasha Kerensikova wrote on comp.lang.ada:
> Then I don't really understand the point of having both Vectors and
> Doubly_Linked_Lists. The interface of Vectors is a strict superset of
> the interface of Doubly_Linked_Lists, and the only difference in
> complexity (which is advice anyway, so non-guaranteed) is on Prepend.

No, there are similar differences in complexity in Delete, Insert and
Append (the Vector occasionally has to reallocate its internal storage
and copy all existing elements over; the Doubly_Linked_List not).
Vector provides random access, Doubly_Linked_List does not. So, while
the two interfaces are similar, Vectors and Lists are for different
kinds of problems with different time complexities.

> Obviously, discrete type serialization alone is not enough, as I will
> have non-discrete atomic object to somehow turn into atoms. And your
> _Blob functions suffer from the same issue as Jeffrey Carter's
> implementation in that it's just a memory dump.

Why is that a problem? The only reason I can think of is that your
type contains access values, in which case it's simply not an atom but
rather a cons pair (or a list).

>> In my implementation, it is also possible to turn any kind of object
>> into an atom by providing a pair of To_String, From_String operations,
>> but these operations actually perform the serialization you were
>> trying to hide ;/
>
> Actually I'm not trying to hide the serialization, I only want to have
> it happen in the client object package, because I assume the object is
> better qualified to know how to serialize itself rather than the
> S-expression package.

Then there is still no problem:

with S_Expression;
procedure Client is
   type Complex_Atom is record ... end record;
   function To_String (A : Complex_Atom) return String is separate;
   -- does the difficult part of the serialization to string
   function To_Atom (A : Complex_Atom) return S_Expression.T is
   begin
      return S_Expression.To_Atom (To_String (A));
   end To_Atom;
begin
   ...
end Client;

Here, the serialization is indeed in the client.

> I'm aware we don't agree on the S-expression concept, but my (and
> Rivest's) atoms are only a sequence of octets. In that case From_String
> and From_Blob actually perform the same operation, which is taking an
> array of Ada things which are basically (on most platforms) octets, and
> turn them into the internal Atom type. That's why I'm not at ease with
> From_String and From_Blob, I feel it should be one function.

I differ. The Ada way is to treat octets as octets (specifically
Storage_Elements or Stream_Elements), and characters as characters
(specifically [[Wide_]Wide_]Character). (I am surprised you should say
that, since in the next paragraph you say that a String should not
contain UTF-8 octets).

Therefore, as String is not a blob. The blob needs to be encoded into
ASCII characters, the String does not because it already consists of
characters. Therefore From_Blob hex-encodes the blob into a String.

> Let's take for example a Wide_Wide_String (e.g. because that's how my
> application handles strings internally), which is not so good as an
> example because a memory dump wouldn't be so much of an issue (except
> maybe for endianness). Let's further assume I want it serialized in
> UTF-8. How do I do that?

By modifying my implementation a little:

1) You replace S_Expression.From_String and To_String with
From_Wide_Wide_String and To_Wide_Wide_String, and the value of any
atom with a Wide_Wide_Unbounded_String.
2) In S_Expression.T'Read and 'Write, you convert every
Wide_Wide_Character to and from its UTF-8 octet representation (i.e.
one or more Stream_Elements). Hidden from the client, of course.

Having said that, I'll add that, in almost all cases, you really don't
want Wide_Wide_Strings; it is better to deal with UTF-8-encoded
Strings. OK, not all elements of such a string is a Character, but you
don't generally care; when parsing the external UTF-8 representation,
the only Characters you really care about are '(', ')', '"', '\' and
'' (space) which are all in the 7-bit ASCII range (as are '0' .. '9'
and 'A'.. 'F' if you use hex-encoding of blobs). You treat all other
octets as parts of characters you don't care about and copy them
verbatim into the atoms. Oh, and if you use GtkAda, (or GTK+ in C), it
expects all character strings to be UTF-8 and does not use
[Wide_]Wide_Strings.

I like to define
subtype UTF8_String is String;

to make it clear when a String really contains UTF-8.

--
Ludovic Brenta.

Re: S-expression I/O in Ada

[J-P. Rosen]

Le 18/08/2010 14:31, Ludovic Brenta a �crit :
> Oh, and if you use GtkAda, (or GTK+ in C), it
> expects all character strings to be UTF-8 and does not use
> [Wide_]Wide_Strings.
> 
> I like to define
> subtype UTF8_String is String;
> 
> to make it clear when a String really contains UTF-8.
> 
Slightly OT, but you (and others) might be interested to know that Ada
2012 will include string encoding packages to the various UTF-X
encodings. These will be (are?) provided very soon by GNAT.

See AI05-137-2
(http://www.ada-auth.org/cgi-bin/cvsweb.cgi/ai05s/ai05-0137-2.txt?rev=1.2)

-- 
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-18, Ludovic Brenta <ludovic@ludovic-brenta.org> wrote:
> Natasha Kerensikova wrote on comp.lang.ada:
>> Then I don't really understand the point of having both Vectors and
>> Doubly_Linked_Lists. The interface of Vectors is a strict superset of
>> the interface of Doubly_Linked_Lists, and the only difference in
>> complexity (which is advice anyway, so non-guaranteed) is on Prepend.
>
> No, there are similar differences in complexity in Delete, Insert and
> Append (the Vector occasionally has to reallocate its internal storage
> and copy all existing elements over; the Doubly_Linked_List not).

Usually, and probably also in reality, yes. It's just that the ARM
doesn't mention them, so we can't rely on them unless we first define
the platform.

ARM only mentions worst-case time complexity, as advice on top of that,
and they are all O(log N) for Vectors.Element, Vectors.Append,
D_L_Lists.Element, D_L_Lists.Insert and D_L_Lists.Delete; O(N log N) for
Vectors.Prepend and Vectors.Delete_First; and O(N**2) for sort.
These are quite conservative requirements anyways.

> Vector provides random access, Doubly_Linked_List does not. So, while
> the two interfaces are similar, Vectors and Lists are for different
> kinds of problems with different time complexities.

So as I said in another post, considering I don't anything more than the
common interface and that I won't use expensive operations on any of
them, which one do I choose?

>> Obviously, discrete type serialization alone is not enough, as I will
>> have non-discrete atomic object to somehow turn into atoms. And your
>> _Blob functions suffer from the same issue as Jeffrey Carter's
>> implementation in that it's just a memory dump.
>
> Why is that a problem? The only reason I can think of is that your
> type contains access values, in which case it's simply not an atom but
> rather a cons pair (or a list).

Serialization isn't only about access values. Or maybe I'm misusing the
word.

Anyway, my (and Rivest's) S-expressions being meant for storage and
transport, there are other issues with memory dumps than access values.
For example, endianness, space-efficiency, or human-readability.

>>> In my implementation, it is also possible to turn any kind of object
>>> into an atom by providing a pair of To_String, From_String operations,
>>> but these operations actually perform the serialization you were
>>> trying to hide ;/
>>
>> Actually I'm not trying to hide the serialization, I only want to have
>> it happen in the client object package, because I assume the object is
>> better qualified to know how to serialize itself rather than the
>> S-expression package.
>
> Then there is still no problem:
>
> with S_Expression;
> procedure Client is
>    type Complex_Atom is record ... end record;
>    function To_String (A : Complex_Atom) return String is separate;
>    -- does the difficult part of the serialization to string
>    function To_Atom (A : Complex_Atom) return S_Expression.T is
>    begin
>       return S_Expression.To_Atom (To_String (A));
>    end To_Atom;
> begin
>    ...
> end Client;
>
> Here, the serialization is indeed in the client.

Yes, and that's exactly what I would do if I was forced to hide Atom
type definition; except I wouldn't use String as an intermediate value,
rather Storage_Element_Array or Stream_Element_Array, because I allow both
string and binary contents in atoms.

Which then raises the question, why not use array-of-octet directly, to
avoid issues when Storage_Element or Stream_Element are not octets.

But then I could make a public array-of-octets type, and provide
functions to "convert" back and forth between the private Atom type and
public array-of-octets type. Would that be acceptable?

> Therefore, as String is not a blob. The blob needs to be encoded into
> ASCII characters, the String does not because it already consists of
> characters. Therefore From_Blob hex-encodes the blob into a String.

My (and Rivest's) atoms can contain any octet sequence, which makes the
hex encoding irrelevant. So I do treat strings as blobs, they are both
data only a type away from being an atom.

You could somehow say that strings and blobs need only to be "cast" into
an atom, while other objects need to be serialized first (usually into
strings or machine-independant blobs).

>> Let's take for example a Wide_Wide_String (e.g. because that's how my
>> application handles strings internally), which is not so good as an
>> example because a memory dump wouldn't be so much of an issue (except
>> maybe for endianness). Let's further assume I want it serialized in
>> UTF-8. How do I do that?
>
> By modifying my implementation a little:

I think you're missing the point. I don't want to modify any
s-expression package implementation whenever I use a new type in an
application.

Let's say for example I have a thousand of applications using Strings,
and one using Wide_Wide_Strings, I can't see how it can be justified to
change (or fork) the S-expression package to take it into account.

It seems much more natural, at least to my C-used eyes, to ask that one
application to provide whatever is needed to serialize Wide_Wide_Strings
into atoms.


Thanks for your tought-prvoking comments,
Natacha

Re: S-expression I/O in Ada

[J-P. Rosen]

Le 18/08/2010 15:55, Natasha Kerensikova a �crit :

> So as I said in another post, considering I don't anything more than the
> common interface and that I won't use expensive operations on any of
> them, which one do I choose?
> 
Pick any of them (or try both). That's exactly why the specifications
have so much in common: it allows you to switch without disturbing your
program - especially if you have properly used "use" clauses (shameless
troll for the use-allergics).

-- 
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: S-expression I/O in Ada

[Yannick Duchêne (Hibou57)]

Le Wed, 18 Aug 2010 16:40:05 +0200, J-P. Rosen <rosen@adalog.fr> a écrit:
> especially if you have properly used "use" clauses (shameless
> troll for the use-allergics).
Or else package renaming

Re: S-expression I/O in Ada

[Ludovic Brenta]

Natasha Kerensikova wrote on comp.lang.ada:
>>> Obviously, discrete type serialization alone is not enough, as I will
>>> have non-discrete atomic object to somehow turn into atoms. And your
>>> _Blob functions suffer from the same issue as Jeffrey Carter's
>>> implementation in that it's just a memory dump.
>
>> Why is that a problem? The only reason I can think of is that your
>> type contains access values, in which case it's simply not an atom but
>> rather a cons pair (or a list).
>
> Serialization isn't only about access values. Or maybe I'm misusing the
> word.
>
> Anyway, my (and Rivest's) S-expressions being meant for storage and
> transport, there are other issues with memory dumps than access values.
> For example, endianness, space-efficiency, or human-readability.

Endianness only affects discrete types, which the generic To_Atom and
From_Atom for discrete types convert to textual representation. And my
implementation theoretically also works with multi-byte characters
since it simply calls 'Image and 'Value.

Space efficiency and human readability are antagonistic goals. S-
Expressions are not meant to be space-efficient. If you want space
efficiency, pipe the S-Expressions through zlib :) OTOH, a raw memory
dump is not human-readable, hence the optional hex-encoding.

> Yes, and that's exactly what I would do if I was forced to hide Atom
> type definition; except I wouldn't use String as an intermediate value,
> rather Storage_Element_Array or Stream_Element_Array, because I allow both
> string and binary contents in atoms.
>
> Which then raises the question, why not use array-of-octet directly, to
> avoid issues when Storage_Element or Stream_Element are not octets.

Sure, that's an option. I used Storage_Element but I might as well
have used Interfaces.Unsigned_8.

> But then I could make a public array-of-octets type, and provide
> functions to "convert" back and forth between the private Atom type and
> public array-of-octets type. Would that be acceptable?

Sure. The only problem would be that you probably want the external
representation to be human-readable, so you'd still need some sort of
conversion.

>> Therefore, as String is not a blob. The blob needs to be encoded into
>> ASCII characters, the String does not because it already consists of
>> characters. Therefore From_Blob hex-encodes the blob into a String.
>
> My (and Rivest's) atoms can contain any octet sequence, which makes the
> hex encoding irrelevant. So I do treat strings as blobs, they are both
> data only a type away from being an atom.
>
> You could somehow say that strings and blobs need only to be "cast" into
> an atom, while other objects need to be serialized first (usually into
> strings or machine-independant blobs).

OK. I was working under the assumption that S-Expression files must be
human-readable and only contain ASCII or UTF-8 in order to work with
any text editor. But if you lift that assumption, then your solution
works.

>>> Let's take for example a Wide_Wide_String (e.g. because that's how my
>>> application handles strings internally), which is not so good as an
>>> example because a memory dump wouldn't be so much of an issue (except
>>> maybe for endianness). Let's further assume I want it serialized in
>>> UTF-8. How do I do that?
>>
>> By modifying my implementation a little:
>
> I think you're missing the point. I don't want to modify any
> s-expression package implementation whenever I use a new type in an
> application.

Ah but the fact that you changed the application-level type is
irrelevant. The relevant fact was that you changed the external
encoding, which is hidden in the body of the S_Expression package, to
UTF-8 (and which simply does not exist in your implementation). But I
see your point.

> Let's say for example I have a thousand of applications using Strings,
> and one using Wide_Wide_Strings, I can't see how it can be justified to
> change (or fork) the S-expression package to take it into account.

No, but you could hex-encode the Wide_Wide_String instead of using
UTF-8 in that one package. Or, for an alternative, see below...

> It seems much more natural, at least to my C-used eyes, to ask that one
> application to provide whatever is needed to serialize Wide_Wide_Strings
> into atoms.

i.e.
subtype UTF8_String is String;
function To_UTF8 (S : Wide_Wide_String) return UTF8_String;
function To_Atom (S : Wide_Wide_String) return S_Expression.T is
begin
   return To_Atom (To_UTF8 (S));
end To_Atom;

?

which is a specialization of the example I used to demonstrate that
you can move the serialization of any arbitrary type into the client
if that's what you need.

--
Ludovic Brenta.

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-18, Ludovic Brenta <ludovic@ludovic-brenta.org> wrote:
> Space efficiency and human readability are antagonistic goals. S-
> Expressions are not meant to be space-efficient. If you want space
> efficiency, pipe the S-Expressions through zlib :) OTOH, a raw memory
> dump is not human-readable, hence the optional hex-encoding.

Actually a pretty good space efficiency can be attained with Rivest's
S-expressions, in what he defined as the canonical form: the overhead of
each list is two bytes (the opening and closing parentheses), there is
no per-element overhead in lists, and the overhead in atom encoding is
logarithmic (the raw binary data is inserted into the stream as-is, with
only a header containing its size in ASCII-encoded decimal, followed by
a colon). Of course that's assuming the atom content is already
space-efficient.

I think space-efficiency vs human readability should not be a choice at
the S_Expressions package level. It should rather be a choice at the
object-level, or maybe better at the application level, choosing between
different representations provided by the objects.

>> Yes, and that's exactly what I would do if I was forced to hide Atom
>> type definition; except I wouldn't use String as an intermediate value,
>> rather Storage_Element_Array or Stream_Element_Array, because I allow both
>> string and binary contents in atoms.
>>
>> Which then raises the question, why not use array-of-octet directly, to
>> avoid issues when Storage_Element or Stream_Element are not octets.
>
> Sure, that's an option. I used Storage_Element but I might as well
> have used Interfaces.Unsigned_8.

So are Storage_Element or Interfaces.Unsigned_8 better in some way then
the user-defined Octet in my S_Expression package?

>> But then I could make a public array-of-octets type, and provide
>> functions to "convert" back and forth between the private Atom type and
>> public array-of-octets type. Would that be acceptable?
>
> Sure. The only problem would be that you probably want the external
> representation to be human-readable, so you'd still need some sort of
> conversion.

As I said, in my opinion it should be an application-level choice.

However Rivest's standard does provide way of encoding atoms in
different more or less human-readable formats.

I described above the canonical form, which uses the so-called verbatim
encoding (raw data prepended by the size).

When the atom content is limited to a certain subset of octets, which
is basically ASCII letters, digits and a few punctuation marks, there is
the "token encoding" which is putting the contents directly, separated
by whitespace. That's the encoding of all atoms in my example (even
though technically Rivet's standard forbids tokens beginning with a
digit, all my parsers handled them well).

When the atom content is more complicated text, there is the
quoted-string encoding, which allows almost anything, but backslash and
double-quote have to be escaped.

When the atom content is binary, but the S-expression has to be
text-based for some reason (be it readability, or transport over a
test-based protocol (e.g. SMTP)), there are hexadecimal and base-64
encodings, which should be self-describing enough.

And  when you want the space-efficiency or unicity or ease-of-parse of
the canonical form but have to send it over a text-based channel, there
is the brace-encoding, which is a full S-expression (not a single atom)
encoded in base-64 and surrounded by braces.

However all these encodings allowed by Rivest's standard represent the
same atom contents. So design-wise, it seems best for the package to
expose an octet-sequence interface for atoms, and the encoding and
decoding being handled internally along with I/O. There would only
options set by the application to guide the package into the preferred
choice of encoding (e.g. Use_Token_When_Possible, Canonical_Form,
Brace_Encode, etc).

While the S-expression decoding part is tedious but quite easy, I found
it quite fun to write the S-expression "pretty printer" (and it was no
small part of the fun to define what is "pretty").


Thanks for your comments,
Natacha

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 18.08.10 13:59, Natasha Kerensikova wrote:

> Then I don't really understand the point of having both Vectors and
> Doubly_Linked_Lists. The interface of Vectors is a strict superset of
> the interface of Doubly_Linked_Lists, and the only difference in
> complexity (which is advice anyway, so non-guaranteed) is on Prepend.

The difference in complexity, in both time and space, is likely
present in other operations. The Implementation Advice in the
LRM stipulates that Vector is likely an array, internally. It is
in GNAT's Vectors, last time I checked.  In addition to Cursor based
operations, Vectors have Index based operations, like
arrays. Lists don't have them; I trust that Vectors have the
lowest overhead for directly accessing specific elements of any
container, e.g.  "the elements at index 5 and 5 plus offset".
If you wanted the same for a list then this could mean linear
traversal. (At least when there are no auxiliary tables internal
to the list implementation, which might in turn be implemented
using a vector kind of thing, exposing a need for a vector
abstraction...)


Georg

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-18, Georg Bauhaus <rm.dash-bauhaus@futureapps.de> wrote:
> The difference in complexity, in both time and space, is likely
> present in other operations. The Implementation Advice in the
> LRM stipulates that Vector is likely an array, internally. It is
> in GNAT's Vectors, last time I checked.  In addition to Cursor based
> operations, Vectors have Index based operations, like
> arrays. Lists don't have them; I trust that Vectors have the
> lowest overhead for directly accessing specific elements of any
> container, e.g.  "the elements at index 5 and 5 plus offset".
> If you wanted the same for a list then this could mean linear
> traversal. (At least when there are no auxiliary tables internal
> to the list implementation, which might in turn be implemented
> using a vector kind of thing, exposing a need for a vector
> abstraction...)

Yes, I do clearly see the advantage of Vectors over Doubly_Linked_Lists.
What I have trouble with is the point of having Doubly_Linked_Lists.
They must somehow have a specific use, otherwise they wouldn't have been
included and everybody would use Vectors instead.

Well, there is the Splice and Swap_Links stuff, and they probably make a
significant difference when they are called a lot of times on large
objects. That still look very specialized, and therefore very weak.
Hence my impression of missing something more important.

There is probably a difference in complexity for all operations, but
they are not specified, and generally not predictable. Well, Vectors
have cache-friendliness, D_L_Lists have (probably) more memory overhead
per element, non-tail insertion should be less efficient on Vectors,
etc.

But these are pretty small and platform-specific details, I believe it's
way too early in my project to care about them. So I'm basically face
the question of Vectors vs D_L_Lists in a situation where they are about
as efficient (mostly appends and linear traversal) and where I only need
the interface common to both containers. That's a tough choice.

(What is sure on the other hand is that I will rely on Cursor, and
probably even make a S_Expressions.Cursor, to make switching as easy
as possible.)


Thanks for your explanations,
Natacha

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 18.08.10 15:24, Natasha Kerensikova wrote:

> There is probably a difference in complexity for all operations, but
> they are not specified, and generally not predictable.

Complexity is, in fact, specified, in Implementation Advice,
which is generally trustworthy. It is written by implementers,
and complexity has been part of the design of the containers.
Predictability is quite important, I assume it will be even
more important with the new bounded forms of the containers,
to be used in computers with limits on memory, and where
timing matters much, hence predictable performance of
container operations is required.
(They may not be a good choice for storing arbitrarily sized
S_Expressions, but illustrate the point: different complexities
are one important criterion for choosing among the containers.)

> But these are pretty small and platform-specific details, I believe it's
> way too early in my project to care about them.

A Yes to ignoring container differences for one reason,
and a No for one and a half reasons, IMO:

Yes-1. It is certainly o.K. to not think about the specific
  container for an application if both meet the needs---from
  an abstract point of view.  In fact, the adapter patterns
  of template libraries can be interpreted to be a manifestation
  of this view, I think.

No-1. The differences in execution can be significant, even
   from a feasibility point of view.  O(log N) versus
   O(1) or O(N log N) can matter much, sometimes preventing
   choice of an otherwise convenient container; the LRM has advice
   (limits, actually) as to which complexity bounds a specific
   container's operations must not exceed.
   (I fail to see why Ada.Containers are platform-specific?
   (Insofar as the complexity measures are not given WRT some
    computer architecture.))

   An example is a filter that must process loads of
   different data items coming in quickly trough some wire.
   A Set can be used to see whether an item is good
   or bad: generate a key from an item and perform look-up.
   A List or Vector can be used, too. But, if many keys are
   possible, Sets can be searched sufficiently faster
   than searching a list or vector for the key. The latter
   might actually be so slow that the program cannot process
   enough items per second.

   The filter program will exclude the use of any container
   but Map or Set, then.

No-1.5. At some point, memory consumption or sufficient
   speed might matter: I cannot, of course, tell but users
   of S_Expression might have rather large expressions
   or need to manipulate them in rather small memories.
   Operations related to S_Expression may need to be fast
   enough, etc.

The good news is in the common set of container operations:
One can indeed switch containers fairly easily when a
change is needed because of changing requirements, or because
of new requirements. If you can start your implementation with
a small subset of operations, lists seems fine, then.


Georg

Re: S-expression I/O in Ada

[Randy Brukardt]

"Natasha Kerensikova" <lithiumcat@gmail.com> wrote in message 
news:slrni6nioh.1efq.lithiumcat@sigil.instinctive.eu...
...
> Then I don't really understand the point of having both Vectors and
> Doubly_Linked_Lists. The interface of Vectors is a strict superset of
> the interface of Doubly_Linked_Lists, and the only difference in
> complexity (which is advice anyway, so non-guaranteed) is on Prepend.
>
> The only notable difference I've found is that
> Doubly_Linked_Lists.Cursor is more resistant to container changes than
> Vectors.Cursor, but I find it a bit weak to justify the existence of
> both types.
>
> What I am missing there?

You've obviously missed the Splice operations (which allow you to move nodes 
from one container to another without copying); those of course are 
impossible for Vectors. Similarly with the Swap_Links operations.

And the performance of Insert is O(log N) for lists while it is not 
specificed for vectors (typically it would O(N) or worse as all of the 
elements will need to moved to make room).

It's a subtle difference in fact; we argued about whether we needed both 
along the lines that you have, but eventually decided that the data 
structures are so fundamental that it would be weird to not to provide both.

                         Randy.


                                 Randy.

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 18.08.10 12:49, Natasha Kerensikova wrote:

>>     type S_Expression (Is_Atom : Boolean) is new Root with record
>>        case Is_Atom is
>>        when False =>
>>           List : Lists.Vector;
>>        when True =>
>>           Value : Atom;
>>        end case;
>>     end record;
> 
> I don't understand why you need Indefinite_Vectors here. They have been
> presented to me as heterogeneous vector containers, but here Lists would
> be homogeneous, containing only S_Expression items.
> I'm not sure about the need of Root either, but my guess is that it's to
> provide a common root to all objects contained in the
> Indefinite_Vectors.
> 
> But what makes S_Expression an indefinite type?

The type S_Expression is not constrained: Is_Atom has no fixed value.
It is an unknown discriminant.  (LRM 3.3 says---in order to give a
reason I guess---"An indefinite subtype does not by itself provide
enough information to create an object".)
An S_Expression object that has Is_Atom = True can have a layout
different from that of an object with Is_Atom = False, and has
different components.  The objects are heterogeneous in this sense.

(Imagine creating two subtypes:
  subtype SA is S_Expression (Is_Atom => True);
  subtype SL is S_Expression (Is_Atom => False);
Then,
  A : SA;
  L : SL;
will be objects that might or might not take up the same amount
of storage, but more importantly, A has a Value component, L
does not, it has a List component, perhaps unlike C unions, cast.)

When instantiating a container generic without Indefinite_ in its name,
a definite subtype is needed.


Georg

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-18, Georg Bauhaus <rm.dash-bauhaus@futureapps.de> wrote:
>> But what makes S_Expression an indefinite type?
>
> The type S_Expression is not constrained: Is_Atom has no fixed value.
> It is an unknown discriminant.  (LRM 3.3 says---in order to give a
> reason I guess---"An indefinite subtype does not by itself provide
> enough information to create an object".)
> An S_Expression object that has Is_Atom = True can have a layout
> different from that of an object with Is_Atom = False, and has
> different components.  The objects are heterogeneous in this sense.

Oh ok, thanks a lot for the explanation.

I thought discriminant really was like C's union, allocating enough room
for any possible sub-object to fit (and adding an overhead to explicit
which sub-object is currently used, to prevent C union's unfamous
undefined behaviour issues). Actually it's much more than a union.


Thanks a lot for your clarity,
Natacha

Re: S-expression I/O in Ada

[Yannick Duchêne (Hibou57)]

> I thought discriminant really was like C's union, allocating enough room
> for any possible sub-object to fit

Depends if it has a default discriminant or not. If it has, and you  
declare an object of such a discriminated type somewhere, without  
initialization, you will got nearly the same, except that in Ada you will  
not be allowed to change arbitrary members without regards to the actual  
discriminant. If you want to change it, you will have to assign the object  
as a whole, to give both the new discriminant and the value of the members  
which will comes into live with it.

If on the other hand, if an object is declared using a constrained  
subtype, either via initialization (this gives the object the subtype of  
the value used to initialize it) at declaration or because it provides a  
discriminant at the declaration (which is required if there is no default  
value for the discriminant), then this have nothing in common with C's  
union.

The structure of the object may be frozen or not (will not in the first  
case, will be in the second case). And if it is frozen, you cannot change  
it, so cannot make (and need not) make assumption about its actual  
internal layout.

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-20, Yannick Duchêne <yannick_duchene@yahoo.fr> wrote:
> Depends if it has a default discriminant or not. If it has, and you  
> declare an object of such a discriminated type somewhere, without  
> initialization, you will got nearly the same, except that in Ada you will  
> not be allowed to change arbitrary members without regards to the actual  
> discriminant. If you want to change it, you will have to assign the object  
> as a whole, to give both the new discriminant and the value of the members  
> which will comes into live with it.

This is very interesting, it will help when I write the parser (with
per-state context information).

But then it means the following type allow some sort of dynamic array?

type Dynamic_Whatever_Array (Size : Natural := 0) is
   array (1 .. Size) of Whatever;


Then is there a simpler way to replace the object than :

procedure Foo is
   object : Dynamic_Whatever_Array;
   Computed_Size : Natural;
begin
   ...
   Computed_Size := ...;
   ...

   declare
      new_object : Dynamic_Whatever_Array(Computed_Size);
   begin
      -- initialize new_object contents
      object := new_object;
   end;

   ...
end Foo;


Or is it complete nonsense?


Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Simon Wright]

Natasha Kerensikova <lithiumcat@gmail.com> writes:

> But then it means the following type allow some sort of dynamic array?
>
> type Dynamic_Whatever_Array (Size : Natural := 0) is
>    array (1 .. Size) of Whatever;
>
>
> Then is there a simpler way to replace the object than :
>
> procedure Foo is
>    object : Dynamic_Whatever_Array;

Unfortunately the way Ada does this is by allocating enough room for
'object' to hold any possible size -- 2**31 bytes -- this will break
your stack!

You could use a sensible max size and allocate on the heap for larger
objects.

Or you may be able to hold Dynamic_Whatever_Array instances in an
Ada.Containers.Indefinite_<something>.

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/22/2010 03:28 AM, Simon Wright wrote:
>
> Unfortunately the way Ada does this is by allocating enough room for
> 'object' to hold any possible size -- 2**31 bytes -- this will break
> your stack!

Actually, 2 ** 31 * Whatever'Size / 8 bytes.

However, this is not technically true. The language does not specify how such 
things are stored. Some compilers (Janus Ada, for example) only use enough 
memory to store the current value of the object; others (GNAT, for example) 
allocate enough memory for the largest possible value. Of course, portable code 
has to work with both.

-- 
Jeff Carter
"Sheriff murdered, crops burned, stores looted,
people stampeded, and cattle raped."
Blazing Saddles
35

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 22 Aug 2010 10:21:08 +0000 (UTC), Natasha Kerensikova wrote:

> Then is there a simpler way to replace the object than�:
> 
> procedure Foo is
>    object : Dynamic_Whatever_Array;
>    Computed_Size : Natural;
> begin
>    ...
>    Computed_Size := ...;
>    ...
> 
>    declare
>       new_object : Dynamic_Whatever_Array(Computed_Size);
>    begin
>       -- initialize new_object contents
>       object := new_object;
>    end;
> 
>    ...
> end Foo;

procedure Foo is
    Computed_Size : Natural;
begin
    ...
    Computed_Size := ...;
    ... 
    declare
        Object : Dynamic_Whatever_Array (Computed_Size);
    begin
        -- use it Object
        ...
    end; -- Drop object
    ...
end Foo;

You don't need varying arrays in 70% cases.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Yannick Duchêne (Hibou57)]

Le Wed, 18 Aug 2010 14:02:48 +0200, Natasha Kerensikova  
<lithiumcat@gmail.com> a écrit:
> I thought discriminant really was like C's union, allocating enough room
> for any possible sub-object to fit (and adding an overhead to explicit
> which sub-object is currently used, to prevent C union's unfamous
> undefined behaviour issues). Actually it's much more than a union.

Another point to tell you (if ever you like this new point) : you may  
think of it a what is also named “Tagged Union”, but while Tagged has a  
more particular meaning in Ada, better not to confuse with this and use  
the expression “Disjoint Union” instead.

Do not really like WikiPedia, but some time I do not have any other source  
to suggest to read:
http://en.wikipedia.org/wiki/Disjoint_union

And to be complete, about Tagged Union also (not to be confused with Ada  
tagged type, while these are similar in some ways):
http://en.wikipedia.org/wiki/Tagged_union

You may also think that the discriminant is part of the object value,  
which a main topic and is something C strictly do not know about (well,  
Ada also have unchecked union, but pretty sure you do not need it here...  
this is to be used in interfacing with the outside world only).

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/18/2010 03:49 AM, Natasha Kerensikova wrote:
>
> I have indeed forgotten about Ada containers. However after having read
> A.18.2 and A.18.3, it seems to me that Doubly_Linked_Lists are more
> appropriate than vectors: the recursive structure of S-expressions can
> make assignment (i.e. deep copies) quite expensive, and
> Doubly_Linked_Lists seem designed primarily to avoid copy of contained
> objects (though I may be mistaken). Usual operations on S-expressions
> are append and traversal, on which Vectors don't provide a significant
> improvement (unlike e.g. random access).
>
> Does it make sense or am I missing something?

You're mistaken about Doubly_Linked_Lists not doing a deep copy; on the other 
hand, adding an element to a Vector may cause reallocation with an internal 
copy, which lists avoid.

I used Vector here because I thought it would be clearer what is happening, 
since it's an unbounded array. Iteration thus uses "for" loops and indexing to 
access the elements, similar to arrays; I thought this would be clearer to a 
beginner (in Ada). Lists and arrays are both sequences and do the same things, 
so either could be used.

> I'm not fond of that part, because it's actually a memory dump, while
> atom contents are supposed to be a serialized representation of
> (atomic) objects.
>
> I'm aware that it's best to hide as much as possible about type
> definitions, but I think in that particular case having Atom definition
> public is justified.

OK, I misunderstood that bit. Making the definition of Atom public would be 
fine, then, allowing clients to create their own. You could also replace the 
definition of Atom with

subtype Atom is Byte_Lists.Vector;

which would simplify the implementation a bit.

> I don't understand why you need Indefinite_Vectors here. They have been
> presented to me as heterogeneous vector containers, but here Lists would
> be homogeneous, containing only S_Expression items.
> I'm not sure about the need of Root either, but my guess is that it's to
> provide a common root to all objects contained in the
> Indefinite_Vectors.

Yes, ideally we want a list of S_Expression. However, the language doesn't allow 
us to create that directly. I can't instantiate Vectors with S_Expression before 
I declare S_Expression, and I can't declare S_Expression until I've instantiated 
Vectors for lists.

But I can instantiate Indefinite_Vectors for Root'Class, then derive 
S_Expression from Root and store S_Expression in the Vector.

Root'Class is indefinite; you can't declare

V : Root'Class;

S_Expression is also indefinite, because tagged types can't have a default on 
their discriminants, so you can't declare

V : S_Expression;

If there were a way to instantiate Vectors with S_Expression directly, then I 
could make S_Expression non-tagged, and have a default for the discriminant; 
S_Expression would then be definite.

-- 
Jeff Carter
"There's no messiah here. There's a mess all right, but no messiah."
Monty Python's Life of Brian
84

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-18, Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> wrote:
> On 08/18/2010 03:49 AM, Natasha Kerensikova wrote:
>>
>> I have indeed forgotten about Ada containers. However after having read
>> A.18.2 and A.18.3, it seems to me that Doubly_Linked_Lists are more
>> appropriate than vectors: the recursive structure of S-expressions can
>> make assignment (i.e. deep copies) quite expensive, and
>> Doubly_Linked_Lists seem designed primarily to avoid copy of contained
>> objects (though I may be mistaken). Usual operations on S-expressions
>> are append and traversal, on which Vectors don't provide a significant
>> improvement (unlike e.g. random access).
>>
>> Does it make sense or am I missing something?
>
> You're mistaken about Doubly_Linked_Lists not doing a deep copy; on the other
> hand, adding an element to a Vector may cause reallocation with an internal
> copy, which lists avoid.

Actually I was mixing things up (at least twice).

When I wrote the paragraph above my concern was both reallocation *and*
deep copy. More precisely I thought the reallocation, triggered e.g. by
Vectors.Append, would not only cause the internal array to be copied,
but all items in the Vector to be deep-copied from the old array to the
newly allocated array.

Then I thought I was mistaken, and just like C's realloc() doesn't mess
with internal pointers pointing to outside of the reallocated area, the
Vector could do a mere shallow copy.

And now I have just had a look at GNAT FSF (v4.4) implementation, and
there is a normal array assignment, which I guess triggers an assignment
of each item, which in our case (record containing a Vector) triggers a
deep copy. Or am I wrong here?

So it seems a simple Vector.Append call might trigger a copy of the
entire S-expression, which makes a strong point for the linked-list,
doesn't it?

> I used Vector here because I thought it would be clearer what is
> happening, since it's an unbounded array. Iteration thus uses "for"
> loops and indexing to access the elements, similar to arrays; I
> thought this would be clearer to a beginner (in Ada). Lists and arrays
> are both sequences and do the same things, so either could be used.

Ok, I understand the motive.

And indeed, your implementation was the first I understood almost
completely, but I can't tell how much is due to your clarity intend and
how much is due to having already tried to write one.

Anyway, thanks for your efforts in pedagogy.

> OK, I misunderstood that bit. Making the definition of Atom public would be
> fine, then, allowing clients to create their own. You could also replace the
> definition of Atom with
>
> subtype Atom is Byte_Lists.Vector;
>
> which would simplify the implementation a bit.

I'm still wondering which is better between vector and array. I thought
a vector could be easily be turned into an array, hence my initial
choice of arrays, but it seems there is no such vector primitive in
A.18.2. Another tough choice...

> Yes, ideally we want a list of S_Expression. However, the language
> doesn't allow us to create that directly. I can't instantiate Vectors
> with S_Expression before I declare S_Expression, and I can't declare
> S_Expression until I've instantiated Vectors for lists.

OK, I understand the catch-22 problem here. I guess it will be the same
with linked lists instead of vectors.

Is there any other (perhaps more difficult for a beginner) way to
address that issue?



Thanks a lot for your help,
Natacha

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-19, Natasha Kerensikova <lithiumcat@gmail.com> wrote:
> On 2010-08-18, Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> wrote:
>> Yes, ideally we want a list of S_Expression. However, the language
>> doesn't allow us to create that directly. I can't instantiate Vectors
>> with S_Expression before I declare S_Expression, and I can't declare
>> S_Expression until I've instantiated Vectors for lists.
>
> OK, I understand the catch-22 problem here. I guess it will be the same
> with linked lists instead of vectors.
>
> Is there any other (perhaps more difficult for a beginner) way to
> address that issue?

And if there is no other way, would it make sense to get rid of the
discriminant with something like:

type S_Expression is tagged null record;

package Lists is new Ada.Containers.Indefinite_Doubly_Linked_List
                     (Element_Type => S_Expression'Class);

type Atom_Node is new S_Expression with record
   Value : Atom;
end record;

type List_Node is new S_Expression with record
   List : Lists.Vector;
end record;


There definitions would of course be private, though I guess
S_Expression, Atom_Node, List_Node and their lineage would be public.
That would allow subprogram to explicit whether they work on/return
atoms, lists, or any kind of S_Expression.

Does it make sense? Is it better or worse or just different? Would
conversion from *_Node to S_Expression be to difficult or too heavy to
make this realistic?


Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Thu, 19 Aug 2010 10:16:00 +0000 (UTC), Natasha Kerensikova wrote:

> On 2010-08-19, Natasha Kerensikova <lithiumcat@gmail.com> wrote:
>> On 2010-08-18, Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> wrote:
>>> Yes, ideally we want a list of S_Expression. However, the language
>>> doesn't allow us to create that directly. I can't instantiate Vectors
>>> with S_Expression before I declare S_Expression, and I can't declare
>>> S_Expression until I've instantiated Vectors for lists.
>>
>> OK, I understand the catch-22 problem here. I guess it will be the same
>> with linked lists instead of vectors.
>>
>> Is there any other (perhaps more difficult for a beginner) way to
>> address that issue?
> 
> And if there is no other way, would it make sense to get rid of the
> discriminant with something like:
> 
> type S_Expression is tagged null record;

The type tag in S_Expression'Class is a kind of discriminant. You replaced
one with another.

There could be other solutions how to build graphs, but in any case you
need a common type or a set of types (=class) for all nodes of the graph.

> Does it make sense?

Yes.

> Is it better or worse or just different?

It is open ended. When using a discrete type for a discriminant you need to
know all types of the nodes in advance. Using a type tag instead you can
add new types of nodes later. It is a less fragile design in the end.

> Would
> conversion from *_Node to S_Expression be to difficult or too heavy to
> make this realistic?

For graphs, node is also a graph rooted in it. So you would need no
conversion. When dealing with doubly-linked lists you inherit their
referential semantics. It means that the node can be only in one list,
though there might be many references to it.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-19, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
> On Thu, 19 Aug 2010 10:16:00 +0000 (UTC), Natasha Kerensikova wrote:
>> And if there is no other way, would it make sense to get rid of the
>> discriminant with something like:
>> 
>> type S_Expression is tagged null record;
>
> The type tag in S_Expression'Class is a kind of discriminant. You replaced
> one with another.

That's more or less the idea: it seems I can't escape using a tag, and I
thought having only a tag might be simpler and cleaner than having both
a tag and a real discriminant, hence my proposition.

Was I misled there?


Thanks for your help,
Natacha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 22 Aug 2010 10:24:16 +0000 (UTC), Natasha Kerensikova wrote:

> On 2010-08-19, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
>> On Thu, 19 Aug 2010 10:16:00 +0000 (UTC), Natasha Kerensikova wrote:
>>> And if there is no other way, would it make sense to get rid of the
>>> discriminant with something like:
>>> 
>>> type S_Expression is tagged null record;
>>
>> The type tag in S_Expression'Class is a kind of discriminant. You replaced
>> one with another.
> 
> That's more or less the idea: it seems I can't escape using a tag, and I
> thought having only a tag might be simpler and cleaner than having both
> a tag and a real discriminant, hence my proposition.

You often add discriminants later. One possible way to implement a tree:

   type Abstract_Node is
      abstract new Ada.Finalization.Limited_Controlled null record;

   type Leaf is new Abstract_Node with ...;

   type Branch (Cardinality : Positive) is new Abstract_Node with record
       Children : Array_Of_Node_Ptr (1..Cardinality);
   end record;

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/19/2010 03:16 AM, Natasha Kerensikova wrote:

Now it's "Natasha".

> And if there is no other way, would it make sense to get rid of the
> discriminant with something like:
>
> type S_Expression is tagged null record;
>
> package Lists is new Ada.Containers.Indefinite_Doubly_Linked_List
>                       (Element_Type =>  S_Expression'Class);
>
> type Atom_Node is new S_Expression with record
>     Value : Atom;
> end record;
>
> type List_Node is new S_Expression with record
>     List : Lists.Vector;
> end record;

This sort of approach would work, though I don't much like it. There's no 
representation of a complete S-expression, just for the bits one is made up of.

This replaces an explicit discriminant with an invisible discriminant; the 
explicit discriminant is generally clearer.

You could also make the explicit discriminant visible, eliminating the Is_Atom 
function, and declaring subtypes, which would eliminate the need for the 
explicit exceptions and tests. That might be a good idea.

> Natacha

So which is it, Natasha or Natacha?

-- 
Jeff Carter
"That was the most fun I've ever had without laughing."
Annie Hall
43

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-19, Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> wrote:
>> And if there is no other way, would it make sense to get rid of the
>> discriminant with something like:
>>
>> type S_Expression is tagged null record;
>>
>> package Lists is new Ada.Containers.Indefinite_Doubly_Linked_List
>>                       (Element_Type =>  S_Expression'Class);
>>
>> type Atom_Node is new S_Expression with record
>>     Value : Atom;
>> end record;
>>
>> type List_Node is new S_Expression with record
>>     List : Lists.Vector;
>> end record;
>
> This sort of approach would work, though I don't much like it. There's
> no representation of a complete S-expression, just for the bits one is
> made up of.

Well technically, any S-expression node is also a full-fledged
S-expression. A subprogram taking any kind of node would look like:

procedure Make_Bar (From : S_Expression'Class);

> explicit discriminant is generally clearer.

Thinking about it, I start believing you're right.

> You could also make the explicit discriminant visible, eliminating the
> Is_Atom function, and declaring subtypes, which would eliminate the
> need for the explicit exceptions and tests. That might be a good idea.

Actually I find an enumerated discriminant clearer than a boolean one,
at least for that particular case.

However I'm still unsure about the pros and cons for having a public
discriminant rather than a private one.

>> Natacha
>
> So which is it, Natasha or Natacha?

They are both transliterations of Наташа, the former being English while
the latter is French. In an English context I consider both of them
equally valid, the first has a better integration in the English flow
while the latter is what is actually written on my ID.

Do I really have to pick one?


Thanks for your help,
Natasha

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/22/2010 03:43 AM, Natasha Kerensikova wrote:
>
> They are both transliterations of Наташа, the former being English while
> the latter is French. In an English context I consider both of them
> equally valid, the first has a better integration in the English flow
> while the latter is what is actually written on my ID.
>
> Do I really have to pick one?

Consistency is nice (or possibly the hobgoblin of a little mind). If your name 
is "Наташа", then I'd think the English transliteration would be better for an 
English newsgroup. But if your name is "Natacha", then that's probably what you 
should use.

-- 
Jeff Carter
"Sheriff murdered, crops burned, stores looted,
people stampeded, and cattle raped."
Blazing Saddles
35

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/19/2010 01:09 AM, Natasha Kerensikova wrote:
>
> And now I have just had a look at GNAT FSF (v4.4) implementation, and
> there is a normal array assignment, which I guess triggers an assignment
> of each item, which in our case (record containing a Vector) triggers a
> deep copy. Or am I wrong here?
>
> So it seems a simple Vector.Append call might trigger a copy of the
> entire S-expression, which makes a strong point for the linked-list,
> doesn't it?

Yes, an Append to a Vector might trigger several recursive calls to Vector 
assignment. If using a Vector is clearer than using a list, that may offset any 
advantage of using a list. Since this is a general, reusable package, 
"efficiency" is not a real concern: projects with special requirements usually 
shouldn't use general packages and need special versions of such things to meet 
their requirements.

> And indeed, your implementation was the first I understood almost
> completely, but I can't tell how much is due to your clarity intend and
> how much is due to having already tried to write one.

Having tried to write one probably helped.

> I'm still wondering which is better between vector and array. I thought
> a vector could be easily be turned into an array, hence my initial
> choice of arrays, but it seems there is no such vector primitive in
> A.18.2. Another tough choice...

Yes, that was a mistake, IMO. An unbounded-array abstraction should have 
operations to convert to and from the fixed equivalent, like To_String and 
To_Unbounded_String in Ada.Strings.Unbounded (which is a special case of 
Vectors; Ada.Characters.Vectors, anyone?).

> Is there any other (perhaps more difficult for a beginner) way to
> address that issue?

That's hardly simple for a beginner. Another way is to use explicit access 
values (and types, though some would prefer the more C-like use of anonymous 
access types). I prefer to avoid explicit access types and values whenever 
possible, and it's clearly possible here.

-- 
Jeff Carter
"That was the most fun I've ever had without laughing."
Annie Hall
43

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-19, Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> wrote:
>> Is there any other (perhaps more difficult for a beginner) way to
>> address that issue?
>
> That's hardly simple for a beginner. Another way is to use explicit access 
> values (and types, though some would prefer the more C-like use of anonymous 
> access types). I prefer to avoid explicit access types and values whenever 
> possible, and it's clearly possible here.

So there is no way of declaring such a S_Expression type without tag and
without access (which also implies tagging it for finalization)?


Thanks for your help,
Natasha

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/22/2010 03:45 AM, Natasha Kerensikova wrote:
>
> So there is no way of declaring such a S_Expression type without tag and
> without access (which also implies tagging it for finalization)?

Well, there's always the C way: public access values, leaving it up to the 
client to do memory management.

That's so bad I think I'll say: No, there's no way to do it without a tag.

-- 
Jeff Carter
"Sheriff murdered, crops burned, stores looted,
people stampeded, and cattle raped."
Blazing Saddles
35

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natasha Kerensikova]

Hello,

Here is my second try at a in-memory S-expression package. I used
Doubly_Linked_Lists, but I did my best to make it as easy as possible to
switch to Vectors.

I managed to remove all explicit access, except for callbacks.

I face the difficulty of trying to make as few list assignments as
possible, because they trigger deep-copy. The problem is that the
package is supposed to be fed directly from external data, which might
be unsafe. So I don't want deep copyings all over the place, because it
might become very expensive on specially-crafted deep input.

Inspired from Ada containers interface (i.e. Cursors), I think the
interface here is usable for read a S-expression object.

However I couldn't imagine a way to design S-expression construction
interface without triggering a deep copy. Ada containers use
Update_Element, which requires passing a read-write container. So in
order to Append a node to a List (let's call it L), I need to pass L's
parent in read-write mode. However in order to get a read-write access
to L's parent, I need to pass L's grandparent to something, which
requires L's grand-grandparent, and so on until the unknown root list.
Obviously this doesn't work.

Using the Append procedure implemented here, I can insert a List as a
node at the end of an existing List, but that makes a deep copy of the
inserted List. So using this interface, if we imagine more and more
complex objects containing less complex object, it means making a simple
list, deep-copying into a large list, itself deep-copied into a larger
list, itself deep-copied into an even larger list, and so-on. While it's
bounded by the application rather than as user input, which is less bad,
it seems sounds horribly inefficient.

Does anybody have an idea about how to solve this issue?


Another point I discovered while writing this implementation, is that I
end up writing a lot of wrappers over the container's functions. Would
it be possible to somehow define List and Cursors directly as instances
of the container respective types, and to expose directly container's
functions? All this while maintaining the type hiding, I don't want to
expose the container, and just want to specify "I expose this and that
functions" and tell in the body "oh by the way, this and that functions
are just public names of this and that container functions". This way
the code would be lighter but still as flexible (whenever I'm not
satisfied with container functions, or whenever I want to change the
container, I just have to (re)write the publicly exposed functions
without changing the specification).

Or is it just completely insane?


Also, I find this implementation much more complex than I expected
for a conceptually simple object. I'm afraid I got caught up into some
design hell and came up with something much more complex than needed.
However I can't see how I can make it simpler without giving up to the
deep-copy issue. Is it because I'm a noob or because it really has to be
that complex?


I believe I would be much more at ease with a custom access-based
singly-linked list implementation, but I'm afraid this is due to my
excessive use of C pointers. I still can't estimate how bad or inelegant
or ugly are explicit access. I guess even in Ada they have some niche
use, and avoiding expensive copy can be one them, right?


Another thing I noticed with this relatively large package is Ada
verbosity. Now I see the difference between using words versus {}. The
problem I see with words for everything is that identifiers don't come
out as well (even with syntax highlighting, differently-colored words
have less contrast than words and punctuation). I hope it's only because
I'm still misusing whitespace.

I'm used to 8-column tab for indentation in C, is it completely
unrealistic in Ada? I followed the recommendation of 3-space
indentation, and that's very flat. Again, do I need only time to get
used to it, or are there other lighter whitespace styles that I haven't
encountered yet?


Last point, I had trouble finding names. I read the Ada 95 Quality and
Style Guide, and I find it difficult to follow. For example, I was
tempted to call "Atom" the public array representation, the internal
node type, and the function taking a Cursor and returning the public
array representation. I was tempted to call "Node_Type" the public
enumeration type, the function taking a Cursor and returning the type of
the pointed node, and (not in this implementation but in some other
attempts) the internal object discriminant name.

Does it become easier with practice? Would you mind reviewing my name
choices to progress faster?


As usual, all comments on my code are welcome, especially those that can
make me progress.
Thanks in advance for your help,
Natasha





with Ada.Containers;
use  Ada.Containers;

private with Ada.Containers.Indefinite_Doubly_Linked_Lists;


package S_Expressions is

   -----------
   -- Types --
   -----------

   type Octet is range 0 .. 255;
   type Atom_Data is array (Integer range <>) of Octet;
      -- Atom_Data is public to allow Object <--> Atom conversion anywhere

   type List is tagged private;
      -- a S-experssion is actually a List object
      -- theoretically any List tail should be a valid S-expression,
      --    but it seems to difficult and not useful enough
      -- each element is either an Atom or a List

   type Cursor is private;
      -- designates an element in a List

   type Node_Kind is (Node_Atom, Node_List);



   ---------------
   -- Constants --
   ---------------

   Empty_List : constant List;
   No_Element : constant Cursor;


   ------------------------------------
   -- String vs Atom_Data converters --
   ------------------------------------

   procedure String_To_Atom(S : in String; A : out Atom_Data);
   procedure Atom_To_String(A : in Atom_Data; S : out String);
   function To_Atom(S : in String) return Atom_Data;
   function To_String(A : in Atom_Data) return String;


   ---------------------
   -- List inspection --
   ---------------------

   procedure Counts(sexp : in List; Atom_Count, List_Count : out Count_Type);
   function Atom_Count(sexp : in List) return Count_Type;
   function List_Count(sexp : in List) return Count_Type;
   function Node_Count(sexp : in List) return Count_Type;


   ---------------
   -- Iterators --
   ---------------

   procedure Iterate(sexp         : in List;
                     Process_Atom : access procedure(Atom : in Atom_Data);
                     Process_List : access procedure(Sublist : in List));
   procedure Iterate_Over_Atoms(sexp    : in List;
                                Process : not null access procedure
                                             (Atom : in Atom_Data));
   procedure Iterate_Over_Lists(sexp    : in List;
                                Process : not null access procedure
                                             (Sublist : in List));


   -----------------------
   -- Cursor management --
   -----------------------

   function First(sexp : in List) return Cursor;
   function Last(sexp : in List) return Cursor;

   procedure Next(Position : in out Cursor);
   procedure Previous(Position : in out Cursor);
   function Next(Position : in Cursor) return Cursor;
   function Previous(Position : in Cursor) return Cursor;

   function Is_Atom(Element : in Cursor) return Boolean;
   function Is_List(Element : in Cursor) return Boolean;
   function Kind_Of(Element : in Cursor) return Node_Kind;


   --------------------
   -- Atom accessors --
   --------------------

   procedure Query_Atom(Element : in Cursor;
                        Process : not null access procedure
                                       (Atom : in Atom_Data));
   function Atom_Size(Element : in Cursor) return Natural;
   function Atom_Contents(Element : in Cursor) return Atom_Data;
   function Atom_To_String(Element : in Cursor) return String;


   --------------------
   -- List accessors --
   --------------------

   procedure Query_List(Element : in Cursor;
                        Process : not null access procedure
                                       (Sub_Sexp : in List));
   function Sublist_First(Element : in Cursor) return Cursor;
   function Sublist_Last(Element : in Cursor) return Cursor;


   -----------------------
   -- Node constructors --
   -----------------------


   procedure Append(Target : in out List; Data : in Atom_Data);
      -- Append a new atom to the target list
   procedure Append(Target : in out List; Data : in String);
      -- Append a new atom built from a String to the target list
   procedure Append(Target : in out List; Data : in List);
      -- Append a copy of the given list to the target list



private

   ----------------------
   -- Type definitions --
   ----------------------

   type Node is tagged null record;

   package Lists is new Ada.Containers.Indefinite_Doubly_Linked_Lists
                        (Element_Type => Node'Class);

   type Atom (Size : Natural) is new Node with
      record
         Internal : Atom_Data (1 .. Size);
      end record;

   type List is new Node with
      record
         Internal : Lists.List;
      end record;

   type Cursor is
      record
         Internal : Lists.Cursor;
      end record;


   ---------------
   -- Constants --
   ---------------

   Empty_List : constant List := List'(Internal => Lists.Empty_List);
   No_Element : constant Cursor := Cursor'(Internal => Lists.No_Element);

end S_Expressions;




with Ada.Tags;

package body S_Expressions is

   -----------------------------------
   -- Tag related private functions --
   -----------------------------------


   function Is_Atom(N : in Node'Class) return Boolean is
   begin
      return Ada.Tags."="(N'Tag, Atom'Tag);
   end;

   function Is_List(N : in Node'Class) return Boolean is
   begin
      return Ada.Tags."="(N'Tag, List'Tag);
   end;

   function Kind_Of(N : in Node'Class) return Node_Kind is
      ret : Node_Kind;
   begin
      if Is_Atom(N) then
         ret := Node_Atom;
      elsif Is_List(N) then
         ret := Node_List;
      else
         raise Program_Error;
      end if;
      return ret;
   end Kind_Of;

   pragma Inline(Is_Atom, Is_List, Kind_Of);



   ------------------------------------
   -- String vs Atom_Data converters --
   ------------------------------------

   procedure String_To_Atom(S : in String; A : out Atom_Data) is
   begin
      for i in S'Range loop
         A(i - S'First + A'First) := Character'Pos(S(i)) - 1;
      end loop;
   end String_To_Atom;



   procedure Atom_To_String(A : in Atom_Data; S : out String) is
   begin
      for i in A'Range loop
         S(i - A'First + S'First) := Character'Val(A(i) + 1);
      end loop;
   end Atom_To_String;



   function To_Atom(S : in String) return Atom_Data is
      ret : Atom_Data(1 .. S'Length);
   begin
      String_To_Atom(S, ret);
      return ret;
   end To_Atom;



   function To_String(A : in Atom_Data) return String is
      ret : String(A'Range);
   begin
      Atom_To_String(A, ret);
      return ret;
   end To_String;


   ---------------------
   -- List inspection --
   ---------------------


   procedure Counts(sexp : in List; Atom_Count, List_Count : out Count_Type) is

      procedure Process(Element : in Node'Class) is
      begin
         case Kind_Of(Element) is
            when Node_Atom =>
               Atom_Count := Atom_Count + 1;
            when Node_List =>
               List_Count := List_Count + 1;
         end case;
      end;

      procedure Process(Position : in Lists.Cursor) is
      begin
         Lists.Query_Element(Position, Process'Access);
      end Process;

   begin
      Atom_Count := 0;
      List_Count := 0;
      Lists.Iterate(sexp.Internal, Process'Access);
   end Counts;



   function Atom_Count(sexp : in List) return Count_Type is
      ret, discard : Count_Type;
   begin
      Counts(sexp, ret, discard);
      return ret;
   end;



   function List_Count(sexp : in List) return Count_Type is
      ret, discard : Count_Type;
   begin
      Counts(sexp, discard, ret);
      return ret;
   end;



   function Node_Count(sexp : in List) return Count_Type is
   begin
      return Lists.Length(sexp.Internal);
   end;



   ---------------
   -- Iterators --
   ---------------

   procedure Iterate(sexp         : in List;
                     Process_Atom : access procedure(Atom : in Atom_Data);
                     Process_List : access procedure(Sublist : in List)) is
      procedure Process(N : in Node'Class) is
      begin
         case Kind_Of(N) is
            when Node_Atom => Process_Atom(Atom(N).Internal);
            when Node_List => Process_List(List(N));
         end case;
      end Process;

      procedure Read_Element(C : in Lists.Cursor) is
      begin
         Lists.Query_Element(C, Process'Access);
      end;

   begin
      if Process_Atom = null then
         if Process_List /= null then
            Iterate_Over_Lists(sexp, Process_List);
         end if;
      else
         if Process_List = null then
            Iterate_Over_Atoms(sexp, Process_Atom);
         else
            Lists.Iterate(sexp.Internal, Read_Element'Access);
         end if;
      end if;
   end Iterate;



   procedure Iterate_Over_Atoms(sexp    : in List;
                                Process : not null access procedure
                                             (Atom : in Atom_Data)) is
      procedure Internal_Process(N : Node'Class) is
      begin
         if Is_Atom(N) then
            Process(Atom(N).Internal);
         end if;
      end Internal_Process;

      procedure Read_Element(C : in Lists.Cursor) is
      begin
         Lists.Query_Element(C, Internal_Process'Access);
      end;

   begin
      Lists.Iterate(sexp.Internal, Read_Element'Access);
   end Iterate_Over_Atoms;



   procedure Iterate_Over_Lists(sexp    : in List;
                                Process : not null access procedure
                                             (Sublist : in List)) is
      procedure Internal_Process(N : Node'Class) is
      begin
         if Is_List(N) then
            Process(List(N));
         end if;
      end Internal_Process;

      procedure Read_Element(C : in Lists.Cursor) is
      begin
         Lists.Query_Element(C, Internal_Process'Access);
      end;

   begin
      Lists.Iterate(sexp.Internal, Read_Element'Access);
   end Iterate_Over_Lists;



   -----------------------
   -- Cursor management --
   -----------------------


   function First(sexp : in List) return Cursor is
      ret : Cursor := (Internal => Lists.First(sexp.Internal));
   begin
      return ret;
   end First;



   function Last(sexp : in List) return Cursor is
      ret : Cursor := (Internal => Lists.Last(sexp.Internal));
   begin
      return ret;
   end Last;



   procedure Next(Position : in out Cursor) is
   begin
      Lists.Next(Position.Internal);
   end Next;



   procedure Previous(Position : in out Cursor) is
   begin
      Lists.Previous(Position.Internal);
   end Previous;



   function Next(Position : in Cursor) return Cursor is
      ret : Cursor := (Internal => Lists.Next(Position.Internal));
   begin
      return ret;
   end Next;



   function Previous(Position : in Cursor) return Cursor is
      ret : Cursor := (Internal => Lists.Previous(Position.Internal));
   begin
      return ret;
   end Previous;



   function Is_Atom(Element : in Cursor) return Boolean is
   begiN
      return Element /= No_Element
         and then Lists.Has_Element(Element.Internal)
         and then Kind_Of(Element) = Node_Atom;
   end Is_Atom;



   function Is_List(Element : in Cursor) return Boolean is
   begiN
      return Element /= No_Element
         and then Lists.Has_Element(Element.Internal)
         and then Kind_Of(Element) = Node_List;
   end Is_List;



   function Kind_Of(Element : in Cursor) return Node_Kind is
      ret : Node_Kind;
      procedure Process(Element : in Node'Class) is
      begin
         ret := Kind_Of(Element);
      end;
   begin
      Lists.Query_Element(Element.Internal, Process'Access);
      return ret;
   end Kind_Of;



   --------------------
   -- Atom accessors --
   --------------------


   procedure Query_Atom(Element : in Cursor;
                        Process : not null access procedure
                                       (Atom : in Atom_Data)) is
      procedure Internal_Process(Element : in Node'Class) is
      begin
         Process(Atom(Element).Internal);
      end Internal_Process;
   begin
      Lists.Query_Element(Element.Internal, Internal_Process'Access);
   end Query_Atom;



   function Atom_Size(Element : in Cursor) return Natural is
      ret : Natural;
      procedure Record_Size(Element : in Node'Class) is
      begin
         ret := Atom(Element).Size;
      end Record_Size;
   begin
      Lists.Query_Element(Element.Internal, Record_Size'Access);
      return ret;
   end Atom_Size;



   function Atom_Contents(Element : in Cursor) return Atom_Data is
      ret : Atom_Data(1 .. Atom_Size(Element));
      procedure Copy_Atom_Data(Element : in Node'Class) is
      begin
         ret := Atom(Element).Internal;
      end;
   begin
      Lists.Query_Element(Element.Internal, Copy_Atom_Data'Access);
      return ret;
   end Atom_Contents;



   function Atom_To_String(Element : in Cursor) return String is
      ret : String(1 .. Atom_Size(Element));
      procedure To_String(Element : in Node'Class) is
         input : Atom_Data renames Atom(Element).Internal;
      begin
         Atom_To_String(Atom(Element).Internal, ret);
      end;
   begin
      Lists.Query_Element(Element.Internal, To_String'Access);
      return ret;
   end Atom_To_String;



   --------------------
   -- List accessors --
   --------------------


   procedure Query_List(Element : in Cursor;
                        Process : not null access procedure
                                       (Sub_Sexp : in List)) is
      procedure Internal_Process(Element : in Node'Class) is
      begin
         Process(List(Element));
      end Internal_Process;
   begin
      Lists.Query_Element(Element.Internal, Internal_Process'Access);
   end Query_List;



   function Sublist_First(Element : in Cursor) return Cursor is
      ret : Cursor;
      procedure Make_First(Element : in Node'Class) is
      begin
         ret.Internal := Lists.First(List(Element).Internal);
      end Make_First;
   begin
      Lists.Query_Element(Element.Internal, Make_First'Access);
      return ret;
   end Sublist_First;



   function Sublist_Last(Element : in Cursor) return Cursor is
      ret : Cursor;
      procedure Make_Last(Element : in Node'Class) is
      begin
         ret.Internal := Lists.Last(List(Element).Internal);
      end Make_Last;
   begin
      Lists.Query_Element(Element.Internal, Make_Last'Access);
      return ret;
   end Sublist_Last;


   -----------------------
   -- Node constructors --
   -----------------------


   procedure Append(Target : in out List; Data : in Atom_Data) is
      Neo_Node : Atom(Data'Length);
   begin
      Neo_Node.Internal(1 .. Data'Length) := Data(Data'Range);
      Lists.Append(Target.Internal, Neo_Node);
   end Append;



   procedure Append(Target : in out List; Data : in String) is
      Neo_Node : Atom(Data'Length);
   begin
      String_To_Atom(Data, Neo_Node.Internal);
      Lists.Append(Target.Internal, Neo_Node);
   end Append;



   procedure Append(Target : in out List; Data : in List) is
   begin
      Lists.Append(Target.Internal, Data);
   end Append;



end S_Expressions;

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/24/2010 04:41 AM, Natasha Kerensikova wrote:
>
> Does anybody have an idea about how to solve this issue?

What issue? You haven't demonstrated that a deep copy is a problem anywhere 
outside your mind. If you implement this and find that actual use exhausts 
memory or takes longer than the user is willing to wait (~2 sec) then you have 
something to worry about.

> Another point I discovered while writing this implementation, is that I
> end up writing a lot of wrappers over the container's functions. Would
> it be possible to somehow define List and Cursors directly as instances
> of the container respective types, and to expose directly container's
> functions? All this while maintaining the type hiding, I don't want to
> expose the container, and just want to specify "I expose this and that
> functions" and tell in the body "oh by the way, this and that functions
> are just public names of this and that container functions". This way
> the code would be lighter but still as flexible (whenever I'm not
> satisfied with container functions, or whenever I want to change the
> container, I just have to (re)write the publicly exposed functions
> without changing the specification).

This is called "renaming as body": you implement the declaration with a renames 
of another subprogram:

function F (I : Integer) return Integer;

and later

package P is new Something (...);

function F renames P.F;

This requires that both subprograms have the same parameter and result type profile.

> I still can't estimate how bad or inelegant
> or ugly are explicit access. I guess even in Ada they have some niche
> use, and avoiding expensive copy can be one them, right?

They are needed to implement things like Ada.Containers.Doubly_Linked_Lists.

> Last point, I had trouble finding names. I read the Ada 95 Quality and
> Style Guide, and I find it difficult to follow. For example, I was
> tempted to call "Atom" the public array representation, the internal
> node type, and the function taking a Cursor and returning the public
> array representation. I was tempted to call "Node_Type" the public
> enumeration type, the function taking a Cursor and returning the type of
> the pointed node, and (not in this implementation but in some other
> attempts) the internal object discriminant name.

One thing about choosing names: the fewer names you have to choose, the less of 
an issue it is. If you use a Boolean for the discriminant, for example, then you 
don't need to choose names for the type of the discriminant or its values. 
Overloading can also help reduce the number of unique names. Your procedures 
String_To_Atom and Atom_To_String could be named To_Atom and To_String, same as 
your functions, just as you did later with procedures and functions named Next 
and Previous for Cursor.

A and S are terrible parameter names for these procedures. I would probably use 
Value and Image, respectively.

> package S_Expressions is
>
>     -----------
>     -- Types --
>     -----------

This is a worthless comment; we can tell that these are type declarations.

It's also not a good way to structure your code. Logically related things should 
be grouped together, not grouped by an incidental relationship such as "these 
are all types". You might want to group all declarations relating to Atoms, for 
example, then all relating to Lists, and so on.

>     type Node_Kind is (Node_Atom, Node_List);

These seem backward from an English usage point of view. Both are nodes, so the 
rest is a modifier that indicates the kind of node. In English, modifiers 
usually come before the thing they modify, so Atom_Node and List_Node would be 
more natural. It also puts the important difference between the 2 first, where 
it is immediately apparent.

>     procedure String_To_Atom(S : in String; A : out Atom_Data);
>     procedure Atom_To_String(A : in Atom_Data; S : out String);

These are only partly specified. What happens if S'Length /= A'Length?

>     procedure Counts(sexp : in List; Atom_Count, List_Count : out Count_Type);
>     function Atom_Count(sexp : in List) return Count_Type;
>     function List_Count(sexp : in List) return Count_Type;
>     function Node_Count(sexp : in List) return Count_Type;

These are problematical. I'd use Length rather than Node_Count, but I'm not sure 
what I'd name the others.

>
>     function Is_Atom(Element : in Cursor) return Boolean;
>     function Is_List(Element : in Cursor) return Boolean;
>     function Kind_Of(Element : in Cursor) return Node_Kind;

Is_* is good, but I'd just use Kind for the 3rd.

>     function Atom_Size(Element : in Cursor) return Natural;
>     function Atom_Contents(Element : in Cursor) return Atom_Data;
>     function Atom_To_String(Element : in Cursor) return String;

Again, Length, Contents, and To_String (or Image). What happens if Element isn't 
an Atom?

>     procedure Query_List(Element : in Cursor;
>                          Process : not null access procedure
>                                         (Sub_Sexp : in List));
>     function Sublist_First(Element : in Cursor) return Cursor;
>     function Sublist_Last(Element : in Cursor) return Cursor;

What happens when Element isn't a List?

-- 
Jeff Carter
"Help! Help! I'm being repressed!"
Monty Python & the Holy Grail
67

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-25, Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> wrote:
> On 08/24/2010 04:41 AM, Natasha Kerensikova wrote:
>> Does anybody have an idea about how to solve this issue?
>
> What issue? You haven't demonstrated that a deep copy is a problem anywhere 
> outside your mind. If you implement this and find that actual use exhausts 
> memory or takes longer than the user is willing to wait (~2 sec) then
> you have something to worry about.

Well, if it were about performances I would completely agree with you:
this is way too early in the development process to care about
performances or optimization.

However it is more about security. I guess it's a bad habit from C, but
I can't make a design or write some code without considering how it can
be abused. And I acknowledge the Ada compiler does a lot to free the
programmer from such considerations.

The deep copy won't be a problem for all the real-life cases I've ever
encountered so far, which is still quite a decent amount. However it
requires only one specially-crafted input from a single attacker to
bring the system on its knees (well, honestly I don't know how much an
Ada deep copy costs, but I do know how much a memcpy() costs and I'm
sure Ada deep copy is not cheaper).

> This is called "renaming as body": you implement the declaration with
> a renames 
>
> This requires that both subprograms have the same parameter and result
> type profile.

Thanks a lot for letting me know of the at possibility.

However I'm afraid the type profile is the issue here. Maybe I did it
wrong, but I didn't manage to get to compile any of my attempts at using
a package-provided type as the private implementation of a public type.

And if the public type doesn't match the package-provided type, none of
the package-provided subprograms can be used as publicly exposed
subprograms.

>> I still can't estimate how bad or inelegant
>> or ugly are explicit access. I guess even in Ada they have some niche
>> use, and avoiding expensive copy can be one them, right?
>
> They are needed to implement things like Ada.Containers.Doubly_Linked_Lists.

Yes, and I have the feeling the package I'm trying to write can thought
of as the same level as Ada standard containers: it's just a kind of
branching simply-linked list, with a predefined Element_Type instead of
a generic one.

That's why I came to think using accesses here might be legitimate, but
then again I still don't know how biased I am from my C pointer
juggling past. Hence my asking for external, hopefully less biased,
opinions.

>>     procedure String_To_Atom(S : in String; A : out Atom_Data);
>>     procedure Atom_To_String(A : in Atom_Data; S : out String);
>
> These are only partly specified. What happens if S'Length /= A'Length?

Maybe I should have kept them private.

If the source is shorter than the destination, the destination is
partially filled an everything continues; if the destination is shorter,
an exception is raised.

Should this kind of information be put as comments after the procedure
declaration?

> What happens if Element isn't an Atom?
>
> What happens when Element isn't a List?

Constraint_Error is raised, which felt like the right thing to do. At
first I used to explicitly check the node type, and explicitly raised the
exception, until I discovered that the compiler already does this when
casting the input Node'Class object into Atom or List.



Thanks a lot for your comments (even those I snipped here, have been
read and acknowledged),
Natasha

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/25/2010 05:18 AM, Natasha Kerensikova wrote:
>
> However it is more about security. I guess it's a bad habit from C, but
> I can't make a design or write some code without considering how it can
> be abused. And I acknowledge the Ada compiler does a lot to free the
> programmer from such considerations.

Thinking about security is hardly a bad habit

> The deep copy won't be a problem for all the real-life cases I've ever
> encountered so far, which is still quite a decent amount. However it
> requires only one specially-crafted input from a single attacker to
> bring the system on its knees (well, honestly I don't know how much an
> Ada deep copy costs, but I do know how much a memcpy() costs and I'm
> sure Ada deep copy is not cheaper).

I'm not sure what you mean by "the system". The attacker might be able to 
exhaust memory in your program, causing Storage_Error to be raised, but that 
probably won't affect the rest of the system, such as other programs or the OS. 
If you're writing OS code, it would be a concern, but a general, reusable 
package is probably not appropriate for such a use.

> If the source is shorter than the destination, the destination is
> partially filled an everything continues; if the destination is shorter,
> an exception is raised.
>
> Should this kind of information be put as comments after the procedure
> declaration?

My questions were intended to point out information that the client needs from 
the package spec and you weren't providing.

Comments after the subprogram declaration are a good and common way to provide 
this kind of information. You need to tell the client what exception the 
operation might raise. You might also want to look at

Ada.Text_IO.Get_Line (Item : out String; Last : out Natural);

to see how it handles similar cases

(http://www.adaic.org/standards/05rm/html/RM-A-10-1.html)

>
>> What happens if Element isn't an Atom?
>>
>> What happens when Element isn't a List?
>
> Constraint_Error is raised, which felt like the right thing to do. At
> first I used to explicitly check the node type, and explicitly raised the
> exception, until I discovered that the compiler already does this when
> casting the input Node'Class object into Atom or List.

Again, I wasn't asking what the implementation does, but pointing out 
information the client needs the package spec to provide.

The Ada term is "[type] conversion" (ARM 4.6), so we'd say "when converting the 
input ..."

-- 
Jeff Carter
"What I wouldn't give for a large sock with horse manure in it."
Annie Hall
42

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 8/24/10 1:41 PM, Natasha Kerensikova wrote:


> Using the Append procedure implemented here, I can insert a List as a
> node at the end of an existing List, but that makes a deep copy of the
> inserted List. So using this interface, if we imagine more and more
> complex objects containing less complex object, it means making a simple
> list, deep-copying into a large list, itself deep-copied into a larger
> list, itself deep-copied into an even larger list, and so-on. While it's
> bounded by the application rather than as user input, which is less bad,
> it seems sounds horribly inefficient.
>
> Does anybody have an idea about how to solve this issue?

Part of the issue (if it should be one) can be addressed by by
looking at S-Expressions as the treesthey are.
To store data, keep the List type, but store objects inside a map.
Conceptually,


   _________________________________________
   | ... | atom_1 atom_2 � list_1 � atom 3  |
   | ... | atom_4 � list_2 �   ...         |


such that � list_1 � is really a key, not the list itself.
Thus,

    key_X  ->  atom_1 atom_2 key_Y atom 3
    key_Y  ->  atom_4 key_Z ...
  

This structure replace pointers and memory management
of your original package with the use of a Map in the
implementation.  (Some) copying is avoided.  The programming
task will be to hide this structure and continue to expose
S-Expression subprograms as in your  latest package.

If you'd have a package that was build around a list type
that uses pointers to connect atom nodes and list nodes,
would you provide subprograms that either insert copies of
lists or just connect the existing list to the new list?
If there was only the latter and no copying, clients would
have to be very careful not to touch the data structures,
since they are shared between your implementation and the
client programs.


> Another thing I noticed with this relatively large package is Ada
> verbosity. Now I see the difference between using words versus {}. The
> problem I see with words for everything is that identifiers don't come
> out as well (even with syntax highlighting, differently-colored words
> have less contrast than words and punctuation). I hope it's only because
> I'm still misusing whitespace.

(ASCII symbols work well as long as they aren't overloaded;
when you work with a few languages, the sparse symbolism
of languages that use punctuation for essential things gets
in the way.  ASCII punctuation also triggers compiler diagnostics
that non-experts in ;,: parsing find startling.)


> Last point, I had trouble finding names. [...]
>
> Does it become easier with practice?

Good names really help.  My goal is to read a name and
then know a fair bit about the named thing. A fair bit
is when I don't have to consult lots of comments, or try
to remember what this object was about, and why it was here.


A few quick remarks:

>     ---------------------
>     -- List inspection --
>     ---------------------

These could be in another package, if all they need in their bodies
is the iteration subprograms and Kind.  Makes the S_Expression package
smaller and separates concerns.

>     procedure Counts(sexp : in List; Atom_Count, List_Count : out Count_Type);
>     function Atom_Count(sexp : in List) return Count_Type;
>     function List_Count(sexp : in List) return Count_Type;
>     function Node_Count(sexp : in List) return Count_Type;
>


I'd use a type test here, not 'Tag,

      function Is_Atom(N : in Node'Class) return Boolean is
      begin
         --return Ada.Tags."="(N'Tag, Atom'Tag);
         return N in Atom;
      end;

and here:
>     function Is_List(N : in Node'Class) return Boolean is
>     begin
>        return Ada.Tags."="(N'Tag, List'Tag);
>     end;

The new DbC-like facilities of Ada use identifier Result when
referring to a function result.  I'd use Result in place of "ret".

>     function To_Atom(S : in String) return Atom_Data is
>        ret : Atom_Data(1 .. S'Length);
>     begin
>        String_To_Atom(S, ret);
>        return ret;
>     end To_Atom;



Just to mention a possibility, a renaming of "ret" might or might
not clarify its two roles, being both set in Record_Size, and returned.

      function Atom_Size(Element : in Cursor) return Natural is
         size : Natural;
         procedure Record_Size(Element : in Node'Class) is
         begin
            size := Atom(Element).Size;
         end Record_Size;
         ret : Natural renames size;
      begin
         Lists.Query_Element(Element.Internal, Record_Size'Access);
         return ret;
      end Atom_Size;



Georg

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-25, Georg Bauhaus <rm-host.bauhaus@maps.futureapps.de> wrote:
> On 8/24/10 1:41 PM, Natasha Kerensikova wrote:
>>
>> Does anybody have an idea about how to solve this issue?
>
> Part of the issue (if it should be one) can be addressed by by
> looking at S-Expressions as the treesthey are.
> To store data, keep the List type, but store objects inside a map.

That's a very interesting idea. I'm not familiar enough with maps to
imagine how to implement this, but I'll study them. Another nice
facility that I had to live without in C.

> If you'd have a package that was build around a list type
> that uses pointers to connect atom nodes and list nodes,
> would you provide subprograms that either insert copies of
> lists or just connect the existing list to the new list?
> If there was only the latter and no copying, clients would
> have to be very careful not to touch the data structures,
> since they are shared between your implementation and the
> client programs.

Actually as an exercise I started writing an access-based version of the
package, and I planned to forbid any direct action of the client to the
data structure.

There is an opaque Container object, which destroys everything it
contains when finalized, and opaque Cursor objects, which are
semantically like access to nodes: many Cursors can refer to the same
Node, and copying a Cursors doesn't copy any data. The only difference
with Ada.Containers is that I need a read-write Cursor for modifications
around that point, but I will probably wrap it into a record to follow
Ada.Containers.

Then the public subprograms only allow to append a new node at the end
of a Cursor-designated list, said node being internally constructed, the
client only provides data to fill it in; and to cut off whatever lies
after a Cursor-designed node, either including or excluding said node.

That sounds limited enough to prevent the client from reaching a
problematic state.


> I'd use a type test here, not 'Tag,
>
>       function Is_Atom(N : in Node'Class) return Boolean is
>       begin
>          --return Ada.Tags."="(N'Tag, Atom'Tag);
>          return N in Atom;
>       end;

I wasn't aware of that kind of tests, it does indeed look much better
than my tag idea.

> The new DbC-like facilities of Ada use identifier Result when
> referring to a function result.  I'd use Result in place of "ret".

Indeed, thanks for the pointer. In a C context "ret" used to be a pretty
long name of a 10-line scoped variable. Bad habits are difficult to get
rid of.

> Just to mention a possibility, a renaming of "ret" might or might
> not clarify its two roles, being both set in Record_Size, and returned.
>
>       function Atom_Size(Element : in Cursor) return Natural is
>          size : Natural;
>          procedure Record_Size(Element : in Node'Class) is
>          begin
>             size := Atom(Element).Size;
>          end Record_Size;
>          ret : Natural renames size;
>       begin
>          Lists.Query_Element(Element.Internal, Record_Size'Access);
>          return ret;
>       end Atom_Size;

Well actually here my tastes would rather go for "return size;" I
typically use "ret" when I have no other more meaningful name for that
object.



Thanks a lot for your remarks,
Natasha

Re: S-expression I/O in Ada

[Simon Wright]

Georg Bauhaus <rm-host.bauhaus@maps.futureapps.de> writes:

> To store data, keep the List type, but store objects inside a map.

This would mean that the order in which data is accessed isn't the order
in which it was received.

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 25.08.10 20:55, Simon Wright wrote:
> Georg Bauhaus <rm-host.bauhaus@maps.futureapps.de> writes:
> 
>> To store data, keep the List type, but store objects inside a map.
> 
> This would mean that the order in which data is accessed isn't the order
> in which it was received.

Why not? If reading appends to a list stored at
the current key which lives on top of a stack?

(A B (C D E) F)

'(' ~> push key_1     -- First key will be returned

key_1 : A B key_2     -- '(' ~> push key_2
key_2 : C D E         -- ')' ~> pop (key_1 on top)
 |
 v
key_1 : A B key_2 F   -- ')' ~> pop (stack empty)
key_2 : C D E

First_Key := key_1;
return;

Georg

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 25.08.10 21:19, Georg Bauhaus wrote:
> On 25.08.10 20:55, Simon Wright wrote:
>> Georg Bauhaus <rm-host.bauhaus@maps.futureapps.de> writes:
>>
>>> To store data, keep the List type, but store objects inside a map.
>>
>> This would mean that the order in which data is accessed isn't the order
>> in which it was received.
> 
> Why not?

I meant, why? Sorry.

Re: S-expression I/O in Ada

[Simon Wright]

Georg Bauhaus <rm.dash-bauhaus@futureapps.de> writes:

> On 25.08.10 20:55, Simon Wright wrote:
>> Georg Bauhaus <rm-host.bauhaus@maps.futureapps.de> writes:
>> 
>>> To store data, keep the List type, but store objects inside a map.
>> 
>> This would mean that the order in which data is accessed isn't the order
>> in which it was received.
>
> Why not? If reading appends to a list stored at
> the current key which lives on top of a stack?
>
> (A B (C D E) F)
>
> '(' ~> push key_1     -- First key will be returned
>
> key_1 : A B key_2     -- '(' ~> push key_2
> key_2 : C D E         -- ')' ~> pop (key_1 on top)
>  |
>  v
> key_1 : A B key_2 F   -- ')' ~> pop (stack empty)
> key_2 : C D E
>
> First_Key := key_1;
> return;

Oh. I had quite misunderstood your meaning.

There are issues - like copying a sexp and then deleting or changing
part of one of them, how to know when to delete or duplicate.

Seems to me that the key is a disguised pointer to a shared structure?

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 8/26/10 12:38 AM, Simon Wright wrote:

>> (A B (C D E) F)
>>
>> '(' ~>  push key_1     -- First key will be returned
>>
>> key_1 : A B key_2     -- '(' ~>  push key_2
>> key_2 : C D E         -- ')' ~>  pop (key_1 on top)
>>   |
>>   v
>> key_1 : A B key_2 F   -- ')' ~>  pop (stack empty)
>> key_2 : C D E
>>
>> First_Key := key_1;
>> return;
>
> Oh. I had quite misunderstood your meaning.
>
> There are issues - like copying a sexp and then deleting or changing
> part of one of them, how to know when to delete or duplicate.
>
> Seems to me that the key is a disguised pointer to a shared structure?

key_N is a pointer in disguise, and modifications of key_N entries
anywhere need care, recursively.  When all Map operations are
hidden behind the container operations of package S_Expressions,
can copying (in order to prevent structural sharing) still be
avoided by implementing copy-on-write?

In any case, an optimized version might use a variation of skip lists
for Element_Type instead of List, I think. Its pointers point to
any key_N is the list, simplifying depth first tree operations
(like removing a subtree).

Georg

Re: S-expression I/O in Ada

[Natasha Kerensikova]

Hello,

Here is my third attempt, hopefully avoiding the mistakes I did before.

I wanted to have a go at an access-based implementation, and while it
seems to end up without roughly the same complexity as my previous
Doubly_Linked_List-based attempt, I felt incredibly much more at home
with it. Is it genuinely more simple in some way, or is it just my
pointer-juggling experience from C?

The interface is relatively limited, but it's enough for all the needs I
ever had. 

The limited interface long with the Container objects make me pretty
confident in the correctness of the memory management.

I'm still not satisfied with the indentation, it still feels much too
shallow for my tastes, and the general look of my sources feels it needs
much more spacing out.


As usual, all comments that can help me writing better Ada code and/or
becoming a better Ada noob are warmly welcome.

Thanks in advance for your help,
Natasha




with Ada.Finalization;

private with Ada.Containers.Vectors;



package Sexp is

   type Container is tagged limited private;
      -- container that keeps tracks of allocated data

   type Cursor is private;
      -- object that refers to a S-expression node
      -- it's the only way a client can interact with S-expression contents

   type Node_Kind is (Atom_Node, List_Node);
      -- a node is either an atom or a list

   type Octet is range 0 .. 255;
   type Atom_Data is array (Positive range <>) of Octet;
      -- publicly defined to allow object conversion anywhere



   ---------------
   -- Constants --
   ---------------

   No_Element : constant Cursor;



   ---------------------
   -- Atom converters --
   ---------------------

   procedure Atom_To_String(Data : in Atom_Data; Image : out String);
   procedure String_To_Atom(Image : in String; Data : out Atom_Data);
      -- conversion between existing objects, assumed to be of equal length

   function To_String(Data : in Atom_Data) return String;
   function To_Atom(Image : in String) return Atom_Data;
      -- constructing one type from the other



   -----------------------
   -- Container methods --
   -----------------------

   procedure Reset(Universe : in out Container);
      -- reset the container using an empty list node as the new root
   procedure Reset(Universe : in out Container; Root_Atom : in Atom_Data);
      -- reset the container using the given atom data as root
   procedure Reset(Universe : in out Container; Root_Atom : in String);
      -- reset the container using the given string as root atom



   -----------------------
   -- Cursor management --
   -----------------------

   function Root(From : in Container) return Cursor;

   function Next(Position : in Cursor) return Cursor;
   procedure Next(Position : in out Cursor);



   --------------------
   -- Node accessors --
   --------------------

   function Is_Atom(Position : in Cursor) return Boolean;
   function Is_List(Position : in Cursor) return Boolean;

   function Kind(Position : in Cursor) return Node_Kind;
      -- raise Constraint_Error when Position is No_Element



   --------------------
   -- Atom accessors --
   --------------------
      -- They all raise Constraint_Error when the given Cursor does not refer
      -- to an atom node.

   function To_Atom(Position : in Cursor) return Atom_Data;
   function To_String(Position : in Cursor) return String;

   procedure Query_Atom
     (Position : in Cursor;
      Process  : not null access procedure(Data : in Atom_Data));

   function Atom_Length(Position : in Cursor) return Natural;


   -------------------
   -- List accessor --
   -------------------

   function Sublist(Position : in Cursor) return Cursor;
      -- raise Constraint_Error when Position does not refer to a list



   -----------------------
   -- Node constructors --
   -----------------------
      -- They all raise Constraint_Error when Position is No_Element

   procedure Append_Empty_List
     (Universe : in out Container;
      Position : in     Cursor);

   procedure Append
     (Universe : in out Container;
      Position : in     Cursor;
      Atom     : in     Atom_Data);

   procedure Append
     (Universe : in out Container;
      Position : in     Cursor;
      Image    : in     String);



   ---------------
   -- Iterators --
   ---------------

   procedure Iterate
     (Start        : in Cursor;
      Process_Atom : access procedure(Data : in Atom_Data);
      Process_List : access procedure(First : in Cursor));

   procedure Iterate_Over_Atoms
     (Start   : in Cursor;
      Process : not null access procedure(Data : in Atom_Data));

   procedure Iterate_Over_Lists
     (Start   : in Cursor;
      Process : not null access procedure(First : in Cursor));

   procedure Iterate_Over_Commands
     (Start   : in Cursor;
      Execute : not null access procedure(Command   : in String;
                                          Arguments : in Cursor));



private

   ----------------------
   -- Type definitions --
   ----------------------

   type Node(<>);
   type Atom_Access is access Atom_Data;
   type Node_Access is access Node;

   type Node (Kind : Node_Kind) is
      record
         Next : Node_Access;
         case Kind is
            when Atom_Node => Atom  : Atom_Access;
            when List_Node => Child : Node_Access;
         end case;
      end record;

   package Nodes is
      new Ada.Containers.Vectors
        (Index_Type   => Positive,
         Element_Type => Node_Access);

   package Atoms is
      new Ada.Containers.Vectors
        (Index_Type   => Positive,
         Element_Type => Atom_Access);

   type Container is
      new Ada.Finalization.Limited_Controlled with record
         List_Node : Nodes.Vector;
         Atom_List : Atoms.Vector;
         Root      : Node_Access;
      end record;

   type Cursor is
      record
         Parent : Node_Access;
         Node   : Node_Access;
      end record;



   -----------------------
   -- Container methods --
   -----------------------

   procedure Clean_Atoms(Universe : in out Container);
   procedure Clean_Nodes(Universe : in out Container);

   overriding
   procedure Finalize(This : in out Container);

   procedure Make_Atom
     (Maker : in out Container;
      Atom  :    out Atom_Access;
      Data  : in     Atom_Data);

   procedure Make_Atom
     (Maker : in out Container;
      Atom  :    out Atom_Access;
      Image : in     String);

   procedure Make_Node
     (Maker : in out Container;
      Node  :    out Node_Access;
      Kind  : in     Node_Kind);

   procedure Make_Node_Atom
     (Maker : in out Container;
      Node  :    out Node_Access;
      Data  : in     Atom_Data);

   procedure Make_Node_Atom
     (Maker : in out Container;
      Node  :    out Node_Access;
      Image : in     String);

   procedure Make_Node_List
     (Maker  : in out Container;
      Node   :    out Node_Access;
      Child  : in     Node_Access := null);



   ---------------
   -- Constants --
   ---------------

   No_Element : constant Cursor := (Parent => null, Node => null);

end Sexp;





with Ada.Unchecked_Deallocation;



package body Sexp is

   procedure Free is
      new Ada.Unchecked_Deallocation(Node, Node_Access);

   procedure Free is
      new Ada.Unchecked_Deallocation(Atom_Data, Atom_Access);



   ------------------------------------
   -- String vs Atom_Data converters --
   ------------------------------------

   procedure String_To_Atom(Image : in String; Data : out Atom_Data) is
   begin
      for i in Image'Range loop
         Data(i - Image'First + Data'First) := Character'Pos(Image(i));
      end loop;
   end String_To_Atom;



   procedure Atom_To_String(Data : in Atom_Data; Image : out String) is
   begin
      for i in Data'Range loop
         Image(i - Data'First + Image'First) := Character'Val(Data(i));
      end loop;
   end Atom_To_String;



   function To_Atom(Image : in String) return Atom_Data is
      Data : Atom_Data(1 .. Image'Length);
   begin
      String_To_Atom(Image, Data);
      return Data;
   end To_Atom;



   function To_String(Data : in Atom_Data) return String is
      Image : String(Data'Range);
   begin
      Atom_To_String(Data, Image);
      return Image;
   end To_String;



   -------------------------------
   -- Container private methods --
   -------------------------------


   procedure Clean_Atoms(Universe : in out Container) is
      Current : Atoms.Cursor := Atoms.First(Universe.Atom_List);
      Atom : Atom_Access;
   begin
      while Atoms.Has_Element(Current) loop
         Atom := Atoms.Element(Current);
         Free(Atom);
         Atoms.Next(Current);
      end loop;
      Atoms.Clear(Universe.Atom_List);
   end Clean_Atoms;



   procedure Clean_Nodes(Universe : in out Container) is
      Current : Nodes.Cursor := Nodes.First(Universe.List_Node);
      Node : Node_Access;
   begin
      while Nodes.Has_Element(Current) loop
         Node := Nodes.Element(Current);
         Free(Node);
         Nodes.Next(Current);
      end loop;
      Nodes.Clear(Universe.List_Node);
   end Clean_Nodes;



   overriding
   procedure Finalize(This : in out Container) is
   begin
      Clean_Nodes(This);
      Clean_Atoms(This);
   end Finalize;



   procedure Make_Atom
     (Maker : in out Container;
      Atom  :    out Atom_Access;
      Data  : in     Atom_Data) is
   begin
      Atom := new Atom_Data(Data'Range);
      Atom.All := Data;
      Atoms.Append(Maker.Atom_List, Atom);
   end Make_Atom;



   procedure Make_Atom
     (Maker : in out Container;
      Atom  :    out Atom_Access;
      Image : in     String) is
   begin
      Atom := new Atom_Data(1 .. Image'Length);
      String_To_Atom(Image, Atom.All);
      Atoms.Append(Maker.Atom_List, Atom);
   end Make_Atom;



   procedure Make_Node
     (Maker : in out Container;
      Node  :    out Node_Access;
      Kind  : in     Node_Kind) is
   begin
      Node := new Sexp.Node(Kind);
      Nodes.Append(Maker.List_Node, Node);
   end Make_Node;



   procedure Make_Node_Atom
     (Maker : in out Container;
      Node  :    out Node_Access;
      Data  : in     Atom_Data) is
   begin
      Make_Node(Maker, Node, Atom_Node);
      Make_Atom(Maker, Node.Atom, Data);
   end Make_Node_Atom;



   procedure Make_Node_Atom
     (Maker : in out Container;
      Node  :    out Node_Access;
      Image : in     String) is
   begin
      Make_Node(Maker, Node, Atom_Node);
      Make_Atom(Maker, Node.Atom, Image);
   end Make_Node_Atom;



   procedure Make_Node_List
     (Maker  : in out Container;
      Node   :    out Node_Access;
      Child  : in     Node_Access := null) is
   begin
      Make_Node(Maker, Node, List_Node);
      Node.Child := Child;
   end Make_Node_List;



   ------------------------------
   -- Container public methods --
   ------------------------------


   procedure Reset(Universe : in out Container) is
   begin
      Clean_Nodes(Universe);
      Clean_Atoms(Universe);
      Make_Node_List(Universe, Universe.Root);
   end Reset;



   procedure Reset(Universe : in out Container; Root_Atom : in Atom_Data) is
   begin
      Clean_Nodes(Universe);
      Clean_Atoms(Universe);
      Make_Node_Atom(Universe, Universe.Root, Root_Atom);
   end Reset;



   procedure Reset(Universe : in out Container; Root_Atom : in String) is
   begin
      Clean_Nodes(Universe);
      Clean_Atoms(Universe);
      Make_Node_Atom(Universe, Universe.Root, Root_Atom);
   end Reset;



   -----------------------
   -- Cursor management --
   -----------------------

   function Root(From : in Container) return Cursor is
      Result : Cursor := (Parent => null, Node => From.Root);
   begin
      return Result;
   end Root;



   function Next(Position : in Cursor) return Cursor is
      Result : Cursor := No_Element;
   begin
      if Position.Node /= null and then Position.Node.Next /= null then
         Result.Parent := Position.Node;
         Result.Node   := Position.Node.Next;
      end if;
      return Result;
   end Next;



   procedure Next(Position : in out Cursor) is
   begin
      if Position.Node /= null then
         if Position.Node.Next /= null then
            Position.Parent := Position.Node;
            Position.Node   := Position.Parent.Next;
         else
            Position := No_Element;
         end if;
      end if;
   end Next;



   --------------------
   -- Node accessors --
   --------------------


   function Is_Atom(Position : in Cursor) return Boolean is
   begin
      return Position.Node /= null and then Position.Node.Kind = Atom_Node;
   end Is_Atom;



   function Is_List(Position : in Cursor) return Boolean is
   begin
      return Position.Node /= null and then Position.Node.Kind = List_Node;
   end Is_List;



   function Kind(Position : in Cursor) return Node_Kind is
   begin
      if Position.Node = null then
         raise Constraint_Error with "Position cursor has no element";
      end if;
      return Position.Node.Kind;
   end Kind;



   --------------------
   -- Atom accessors --
   --------------------


   function To_Atom(Position : in Cursor) return Atom_Data is
   begin
      if not Is_Atom(Position) then
         raise Constraint_Error with "Position cursor is not an atom";
      end if;
      return Position.Node.Atom.all;
   end To_Atom;



   function To_String(Position : in Cursor) return String is
   begin
      if not Is_Atom(Position) then
         raise Constraint_Error with "Position cursor is not an atom";
      end if;
      return To_String(Position.Node.Atom.all);
   end To_String;



   procedure Query_Atom
     (Position : in Cursor;
      Process  : not null access procedure(Data : in Atom_Data)) is
   begin
      if not Is_Atom(Position) then
         raise Constraint_Error with "Position cursor is not an atom";
      end if;
      Process(Position.Node.Atom.all);
   end Query_Atom;



   function Atom_Length(Position : in Cursor) return Natural is
   begin
      if not Is_Atom(Position) then
         raise Constraint_Error with "Position cursor is not an atom";
      end if;
      return Position.Node.Atom'Length;
   end Atom_Length;



   -------------------
   -- List accessor --
   -------------------


   function Sublist(Position : in Cursor) return Cursor is
      Result : Cursor;
   begin
      if not Is_List(Position) then
         raise Constraint_Error with "Position cursor is not a list";
      end if;
      Result.Parent := Position.Node;
      Result.Node   := Position.Node.Child;
      return Result;
   end Sublist;



   -----------------------
   -- Node constructors --
   -----------------------


   procedure Append_Empty_List
     (Universe : in out Container;
      Position : in     Cursor)
   is
      Current : Node_Access := Position.Node;
   begin
      if Current /= null then
         raise Constraint_Error with "Position cursor has no element";
      end if;
      while Current.Next /= null loop
         Current := Current.Next;
      end loop;
      Make_Node_List(Universe, Current.Next);
   end Append_Empty_List;



   procedure Append
     (Universe : in out Container;
      Position : in     Cursor;
      Atom     : in     Atom_Data)
   is
      Current : Node_Access := Position.Node;
   begin
      if Current /= null then
         raise Constraint_Error with "Position cursor has no element";
      end if;
      while Current.Next /= null loop
         Current := Current.Next;
      end loop;
      Make_Node_Atom(Universe, Current.Next, Atom);
   end Append;



   procedure Append
     (Universe : in out Container;
      Position : in     Cursor;
      Image    : in     String)
   is
      Current : Node_Access := Position.Node;
   begin
      if Current /= null then
         raise Constraint_Error with "Position cursor has no element";
      end if;
      while Current.Next /= null loop
         Current := Current.Next;
      end loop;
      Make_Node_Atom(Universe, Current.Next, Image);
   end Append;



   ---------------
   -- Iterators --
   ---------------

   procedure Iterate
     (Start        : in Cursor;
      Process_Atom : access procedure(Data : in Atom_Data);
      Process_List : access procedure(First : in Cursor))
   is
      Current : Node_Access := Start.Node;
      First   : Cursor;
   begin
      if Process_Atom = null then
         if Process_List /= null then
            Iterate_Over_Lists(Start, Process_List);
         end if;
      elsif Process_List = null then
         Iterate_Over_Atoms(Start, Process_Atom);
      else
         while Current /= null loop
            case Current.Kind is
               when Atom_Node =>
                  Process_Atom(Current.Atom.all);
               when List_Node =>
                  First.Parent := Current;
                  First.Node   := Current.Child;
                  Process_List(First);
            end case;
            Current := Current.Next;
         end loop;
      end if;
   end Iterate;



   procedure Iterate_Over_Atoms
     (Start   : in Cursor;
      Process : not null access procedure(Data : in Atom_Data))
   is
      Current : Node_Access := Start.Node;
   begin
      while Current /= null loop
         if Current.Kind = Atom_Node then
            Process(Current.Atom.all);
         end if;
      end loop;
   end Iterate_Over_Atoms;



   procedure Iterate_Over_Lists
     (Start   : in Cursor;
      Process : not null access procedure(First : in Cursor))
   is
      Current : Node_Access := Start.Node;
      First   : Cursor;
   begin
      while Current /= null loop
         if Current.Kind = List_Node then
            First.Parent := Current;
            First.Node   := Current.Child;
            Process(First);
         end if;
      end loop;
   end Iterate_Over_Lists;



   procedure Iterate_Over_Commands
     (Start   : in Cursor;
      Execute : not null access procedure(Command   : in String;
                                          Arguments : in Cursor))
   is
      Current : Node_Access := Start.Node;
      Arg : Cursor;
   begin
      while Current /= null loop
         if Current.Kind = Atom_Node then
            Execute(To_String(Current.Atom.all), No_Element);
         elsif Current.Child.Kind = Atom_node then
            Arg.Parent := Current.Child;
            Arg.Node   := Current.Child.Next;
            Execute(To_String(Current.Child.Atom.all), Arg);
         end if;
      end loop;
   end Iterate_Over_Commands;

end Sexp;

Re: S-expression I/O in Ada

[Georg Bauhaus]

[-- Attachment #1: Type: text/plain, Size: 868 bytes --]

On 27.08.10 15:19, Natasha Kerensikova wrote:
> Hello,
> 
> Here is my third attempt, hopefully avoiding the mistakes I did before.

Just a few quick comments on style, more or less.

The first is to use -gnatwa every now and then, or a similar
feature of your Ada compiler or analysis tool.  It will tell
you which variables are in effect constants, for example.

Second, if you assign entire aggregates instead of performing
several assignment component by component, this ensures you
will never miss a component, should the definition of the
type change.

Finally, some variables (like First in Iterate_Over_Lists)
may or may not serve some purpose, and can be omitted.
OTOH, I guess one might want to weigh this against having
a meaningful name for a larger construct or consider a
more locally declared constant instead (in a declare block).

diff attached
Georg

[-- Attachment #2: sexp.diff --]
[-- Type: text/plain, Size: 4913 bytes --]

*** sexp.ada	2010-08-27 15:44:20.000000000 +0200
--- ../sexp.ada	2010-08-27 15:36:52.000000000 +0200
***************
*** 450,454 ****
  
     function Root(From : in Container) return Cursor is
!       Result : constant Cursor := (Parent => null, Node => From.Root);
     begin
        return Result;
--- 450,454 ----
  
     function Root(From : in Container) return Cursor is
!       Result : Cursor := (Parent => null, Node => From.Root);
     begin
        return Result;
***************
*** 461,466 ****
     begin
        if Position.Node /= null and then Position.Node.Next /= null then
!          Result := (Parent => Position.Node,
!                     Node   => Position.Node.Next);
        end if;
        return Result;
--- 461,466 ----
     begin
        if Position.Node /= null and then Position.Node.Next /= null then
!          Result.Parent := Position.Node;
!          Result.Node   := Position.Node.Next;
        end if;
        return Result;
***************
*** 473,478 ****
        if Position.Node /= null then
           if Position.Node.Next /= null then
!             Position := (Parent => Position.Node,
!                          Node   => Position.Parent.Next);
           else
              Position := No_Element;
--- 473,478 ----
        if Position.Node /= null then
           if Position.Node.Next /= null then
!             Position.Parent := Position.Node;
!             Position.Node   := Position.Parent.Next;
           else
              Position := No_Element;
***************
*** 570,575 ****
           raise Constraint_Error with "Position cursor is not a list";
        end if;
!       Result := (Parent => Position.Node,
!                  Node   => Position.Node.Child);
        return Result;
     end Sublist;
--- 570,575 ----
           raise Constraint_Error with "Position cursor is not a list";
        end if;
!       Result.Parent := Position.Node;
!       Result.Node   := Position.Node.Child;
        return Result;
     end Sublist;
***************
*** 659,664 ****
                    Process_Atom(Current.Atom.all);
                 when List_Node =>
!                   First := (Parent => Current,
!                             Node   => Current.Child);
                    Process_List(First);
              end case;
--- 659,664 ----
                    Process_Atom(Current.Atom.all);
                 when List_Node =>
!                   First.Parent := Current;
!                   First.Node   := Current.Child;
                    Process_List(First);
              end case;
***************
*** 674,678 ****
        Process : not null access procedure(Data : in Atom_Data))
     is
!       Current : constant Node_Access := Start.Node;
     begin
        while Current /= null loop
--- 674,678 ----
        Process : not null access procedure(Data : in Atom_Data))
     is
!       Current : Node_Access := Start.Node;
     begin
        while Current /= null loop
***************
*** 689,702 ****
        Process : not null access procedure(First : in Cursor))
     is
!       Current : constant Node_Access := Start.Node;
!       --First   : Cursor;
     begin
        while Current /= null loop
           if Current.Kind = List_Node then
!             --First := (Parent => Current,
!             --          Node   => Current.Child);
!             --Process(First);
!             Process(First => Cursor'(Parent => Current,
!                                      Node   => Current.Child));
           end if;
        end loop;
--- 689,700 ----
        Process : not null access procedure(First : in Cursor))
     is
!       Current : Node_Access := Start.Node;
!       First   : Cursor;
     begin
        while Current /= null loop
           if Current.Kind = List_Node then
!             First.Parent := Current;
!             First.Node   := Current.Child;
!             Process(First);
           end if;
        end loop;
***************
*** 710,714 ****
                                            Arguments : in Cursor))
     is
!       Current : constant Node_Access := Start.Node;
        Arg : Cursor;
     begin
--- 708,712 ----
                                            Arguments : in Cursor))
     is
!       Current : Node_Access := Start.Node;
        Arg : Cursor;
     begin
***************
*** 717,722 ****
              Execute(To_String(Current.Atom.all), No_Element);
           elsif Current.Child.Kind = Atom_node then
!             Arg := (Parent => Current.Child,
!                     Node   => Current.Child.Next);
              Execute(To_String(Current.Child.Atom.all), Arg);
           end if;
--- 715,720 ----
              Execute(To_String(Current.Atom.all), No_Element);
           elsif Current.Child.Kind = Atom_node then
!             Arg.Parent := Current.Child;
!             Arg.Node   := Current.Child.Next;
              Execute(To_String(Current.Child.Atom.all), Arg);
           end if;

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-27, Georg Bauhaus <rm.dash-bauhaus@futureapps.de> wrote:
> On 27.08.10 15:19, Natasha Kerensikova wrote:
>> Hello,
>> 
>> Here is my third attempt, hopefully avoiding the mistakes I did before.
>
> Just a few quick comments on style, more or less.
>
> The first is to use -gnatwa every now and then, or a similar
> feature of your Ada compiler or analysis tool.  It will tell
> you which variables are in effect constants, for example.

Actually in C I'm used to always compile with the maximum level of
warning (pedantic) and treat them as errors. I didn't do that because I
had not found the documentation of my compiler (fsf gnat 4.4 from
FreeBSD port tree).

So I guess that would be "-gnatwae -gnaty" (I do like the idea of style
checks too). But this raises the question, what is the rationale for
excluding some warning from -gnatwa? I'm also curious about excluded
style checks. Would anybody happen to know where I can find such
rationale? Google doesn't seem very helpful, but I'm not convinced by my
keyword choice either.

By the way, about style issues, do you all here really use the
"standard" (or so it seems to me) indentation of 3 spaces for new blocs
and 2 spaces for line continuations? This is still way too shallow for my
tastes.

The reason why I'm so keen on getting whatever messages the compiler can
provide is that they might be helpful indication of a programming error
that results in a syntactically correct code. For example here, -gnatwa
complains about Current not being constant, but it's not supposed to be,
as it's a pointer to the node being currently processed in a iterator.
So this warning reveals that I have actually forgot to increment Current
(hence never write it), which ends up in an infinite loop.

I find compiler issues to be the weakest point of Ada: documentation is
pretty well hidden, installation is a mess, and there is that
project-file stuff which seems almost as complicated as Ada language
itself, while being compiler-dependant. Coming for a C world where
everything is simpler and well documented in manual pages...

> Second, if you assign entire aggregates instead of performing
> several assignment component by component, this ensures you
> will never miss a component, should the definition of the
> type change.

I didn't know about this syntax, but it does look very seducing. Thanks
a lot for pointing it out.

> Finally, some variables (like First in Iterate_Over_Lists)
> may or may not serve some purpose, and can be omitted.
> OTOH, I guess one might want to weigh this against having
> a meaningful name for a larger construct or consider a
> more locally declared constant instead (in a declare block).

Well actually it doesn't serve any purpose, I used such a variable
because I wasn't aware a record could be created anonymously.
Considering the parameter of the callback is also named First, there is
not additional information for the human reader by using an
intermediate.

Even

function Root(From : in Container) return Cursor is
   Result : constant Cursor := (Parent => null, Node => From.Root);
begin
   return Result;
end Root;

wouldn't be simpler and clearer to build the Cursor anonymously:

function Root(From : in Container) return Cursor is
begin
   return Cursor'(Parent => null, Node => From.Root);
end Root;

Or am I confusing terseness with simplicity?



Thanks a lot for your remarks,
Natasha

Re: S-expression I/O in Ada

[Simon Wright]

Natasha Kerensikova <lithiumcat@gmail.com> writes:

> So I guess that would be "-gnatwae -gnaty" (I do like the idea of
> style checks too). But this raises the question, what is the rationale
> for excluding some warning from -gnatwa? I'm also curious about
> excluded style checks. Would anybody happen to know where I can find
> such rationale? Google doesn't seem very helpful, but I'm not
> convinced by my keyword choice either.

'gnatmake -h' will tell you about the options. On the other hand, as to
_why_ AdaCore made those choices, I know no more than you; I guess they
thought they were reasonable without being too picky.

I always use -gnatwaL (no warnings for missing elaboration pragmas)
because I use GNAT's default elaboration rules & you (usually) don't
need the full range of Elaborate, Elaborate_All (and, having been a GNAT
user forever, I don't understand them anyway).

> By the way, about style issues, do you all here really use the
> "standard" (or so it seems to me) indentation of 3 spaces for new
> blocs and 2 spaces for line continuations? This is still way too
> shallow for my tastes.

All I can say is, works for me!

> I find compiler issues to be the weakest point of Ada: documentation
> is pretty well hidden, installation is a mess, and there is that
> project-file stuff which seems almost as complicated as Ada language
> itself, while being compiler-dependant. Coming for a C world where
> everything is simpler and well documented in manual pages...

The project file stuff is GNAT, not Ada. That said, I have to agree with
you about it.

> Even
>
> function Root(From : in Container) return Cursor is
>    Result : constant Cursor := (Parent => null, Node => From.Root);
> begin
>    return Result;
> end Root;
>
> wouldn't be simpler and clearer to build the Cursor anonymously:
>
> function Root(From : in Container) return Cursor is
> begin
>    return Cursor'(Parent => null, Node => From.Root);
> end Root;
>
> Or am I confusing terseness with simplicity?

I think the generated code would probably be the same.

I would write the second. I'd look at the first and wonder why you
needed to name the result ... & then realise you didn't.

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-29, Simon Wright <simon@pushface.org> wrote:
> Natasha Kerensikova <lithiumcat@gmail.com> writes:
>
>> So I guess that would be "-gnatwae -gnaty" (I do like the idea of
>> style checks too). But this raises the question, what is the rationale
>> for excluding some warning from -gnatwa? I'm also curious about
>> excluded style checks. Would anybody happen to know where I can find
>> such rationale? Google doesn't seem very helpful, but I'm not
>> convinced by my keyword choice either.
>
> 'gnatmake -h' will tell you about the options. On the other hand, as to
> _why_ AdaCore made those choices, I know no more than you; I guess they
> thought they were reasonable without being too picky.

I have to admit I'm not clear about the relationship between AdaCore and
FSF about GNAT. The only thing I know is that AdaCore's GNAT doesn't
compile on 64-bit FreeBSD, so choosing between them is a no-brainer.

>> I find compiler issues to be the weakest point of Ada: documentation
>> is pretty well hidden, installation is a mess, and there is that
>> project-file stuff which seems almost as complicated as Ada language
>> itself, while being compiler-dependant. Coming for a C world where
>> everything is simpler and well documented in manual pages...
>
> The project file stuff is GNAT, not Ada. That said, I have to agree with
> you about it.

Considering I'm a poor hobbist, I don't have many choices outside of
GNAT and GNAT. But I would welcome a way of using GNAT with only good
old makefiles and no project file thingies.

> I would write the second. I'd look at the first and wonder why you
> needed to name the result ... & then realise you didn't.

Well in my case you have the answer: because I'm still very low in the
learning ladder.


Thanks for your remarks,
Natasha

Re: S-expression I/O in Ada

[Niklas Holsti]

Natasha Kerensikova wrote:
> But I would welcome a way of using GNAT with only good
> old makefiles and no project file thingies.

I have managed quite well with just ADA_INCLUDE_PATH and gnatmake (and 
no makefile -- what a relief, compared to C!), and sometimes a pragma 
Linker_Options. Of course I use the GNAT convention for source-file 
names (.ads, .adb).

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: S-expression I/O in Ada

[Robert A Duff]

Natasha Kerensikova <lithiumcat@gmail.com> writes:

>...But this raises the question, what is the rationale for
> excluding some warning from -gnatwa?

They would generate too much noise for too many people.
But feel free to turn them on, and see if you like them.

> By the way, about style issues, do you all here really use the
> "standard" (or so it seems to me) indentation of 3 spaces for new blocs
> and 2 spaces for line continuations?

This is the AdaCore standard, not any official Ada standard.
I prefer 4 spaces, but since I work for AdaCore, of course
I use the AdaCore coding conventions.

> I find compiler issues to be the weakest point of Ada: documentation is
> pretty well hidden, ...

Heh?  Have you read the GNAT User's Guide, and the GNAT Reference
Manual?  They should be under some 'doc' directory in your
GNAT installation.  Or a quick google sends me to:

http://gcc.gnu.org/onlinedocs/index.html#dir

How are these "well hidden"?

> function Root(From : in Container) return Cursor is
> begin
>    return Cursor'(Parent => null, Node => From.Root);
> end Root;

Nothing wrong with that.  I would probably write
"return (Parent => null, Node => From.Root);" in this case,
because the type is obvious by looking just two lines
previous.  But if you want to emphasize the type of
an expression, you can always qualify it as you did.

- Bob

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/29/2010 06:23 AM, Robert A Duff wrote:
>
> Heh?  Have you read the GNAT User's Guide, and the GNAT Reference
> Manual?  They should be under some 'doc' directory in your
> GNAT installation.  Or a quick google sends me to:
>
> http://gcc.gnu.org/onlinedocs/index.html#dir
>
> How are these "well hidden"?

What Dewar calls the secret documentation.

-- 
Jeff Carter
"We call your door-opening request a silly thing."
Monty Python & the Holy Grail
17

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Britt Snodgrass]

On Aug 29, 8:23 am, Robert A Duff <bobd...@shell01.TheWorld.com>
wrote:

>
> > I find compiler issues to be the weakest point of Ada: documentation is
> > pretty well hidden, ...
>
> Heh?  Have you read the GNAT User's Guide, and the GNAT Reference
> Manual?  They should be under some 'doc' directory in your
> GNAT installation.  Or a quick google sends me to:
>
> http://gcc.gnu.org/onlinedocs/index.html#dir
>
> How are these "well hidden"?
>

Of course thay are not hidden since they are also located in a "...
\share\doc\..." subdirectory within the compiler installation.  THe
GNAT Programming System (GPS) Help menu also provides sub-menus for
all the documentation.

I find the GNAT manuals to be quite complete and generally well
written.  I appreciate the fact that the are supplied in PDF format as
well as HTML.  I like PDF because I often use the "find all
occurences" search feature in Adobe Reader and Acrobat. I also
sometime print a reference section from one of the PDF manuals.

If I do have a complaint it is that there are possibly too many
different manuals.  I deal with that by using Adobe Acrobat to create
an index of _all_ the GNAT PDF manuals so that I can quickly search
all of them at once.

The GNAT Project (GPR file) information is a pretty complete but
scattered among several manuals. In addition to the Users Guide, it is
important to look in the GNAT Reference Manual as well, and also in
the GPS and GPRbuild manuals for a few things.


- Britt

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-29, Robert A Duff <bobduff@shell01.TheWorld.com> wrote:
> Natasha Kerensikova <lithiumcat@gmail.com> writes:
>> I find compiler issues to be the weakest point of Ada: documentation is
>> pretty well hidden, ...
>
> Heh?  Have you read the GNAT User's Guide, and the GNAT Reference
> Manual?  They should be under some 'doc' directory in your
> GNAT installation.  Or a quick google sends me to:
>
> http://gcc.gnu.org/onlinedocs/index.html#dir
>
> How are these "well hidden"?

Well, let's relatively well hidden compared to its C counterpart, where
I'm only "man gcc" away from the exhaustive list of command-line
switches, or "man whatever_function" away from a standard library
function documentation.

But then it might be more FreeBSD's fault than GNAT's, considering
provided GNAT man pages don't cover Ada stuff, and provided GNAT info
pages are not in info search path. But then calling the user manual info
file "gnat_ugn.info" is quite unintuitive, and the organization of the
info file looks pretty messy (but then again I'm not used to info
pages, and man pages tend to be shorter and more specialized).

>> function Root(From : in Container) return Cursor is
>> begin
>>    return Cursor'(Parent => null, Node => From.Root);
>> end Root;
>
> Nothing wrong with that.  I would probably write
> "return (Parent => null, Node => From.Root);"

I wasn't aware of that even-shorter form. I should really take the time
to read the reference manual (hoping now I've reached the level needed
to understand it).


Thanks for your remarks,
Natasha

Re: S-expression I/O in Ada

[Robert A Duff]

Natasha Kerensikova <lithiumcat@gmail.com> writes:

> Well, let's relatively well hidden compared to its C counterpart, where
> I'm only "man gcc" away from the exhaustive list of command-line
> switches, or "man whatever_function" away from a standard library
> function documentation.

I hate man pages, because they tend to require that I already
know what I'm looking for.  Interestingly, usually when I
do "man <something>" these days, I often get a man page that tells
me to go look at the "info" docs.

Anyway, man pages wouldn't make a lot of sense for windows users.

> But then it might be more FreeBSD's fault than GNAT's, considering
> provided GNAT man pages don't cover Ada stuff, and provided GNAT info
> pages are not in info search path. But then calling the user manual info
> file "gnat_ugn.info" is quite unintuitive, and the organization of the
> info file looks pretty messy (but then again I'm not used to info
> pages, and man pages tend to be shorter and more specialized).

OK, but now you know where to find the GNAT docs.  I suggest you
print out the .pdf files and read them.  Or look at the .txt
files in your favorite text editor (and search for things).
Or look at the .html files in your browser.  Or use info.
Whatever you're most comfortable with.

>>> function Root(From : in Container) return Cursor is
>>> begin
>>>    return Cursor'(Parent => null, Node => From.Root);
>>> end Root;
>>
>> Nothing wrong with that.  I would probably write
>> "return (Parent => null, Node => From.Root);"
>
> I wasn't aware of that even-shorter form.

"(Parent => null, Node => From.Root)" is an aggregate.
It's type is determined from context (in this
case, the expected type of the expression is Cursor,
because that's what the function returns).

"T'(X)" is a qualified expression -- it tells you that the
type of X is T.  This is necessary when the type is ambiguous.
You can also use it for clarity when it's not necessary.

Combining the two, you have "Cursor'((Parent => null, Node =>
From.Root))", but there's a special syntax rule that allows you to leave
out one set of parentheses, whenever an aggregate is used as the
argument of a qualified expression.

>...I should really take the time
> to read the reference manual (hoping now I've reached the level needed
> to understand it).

Well, the RM is pretty tough going.  I suggest a text book
instead.  The Barnes book, for example -- it's fun to read
because of John's wry sense of humour.  There are also some
on-line tutorials you can find via google.

> Thanks for your remarks,

You're welcome.

- Bob

Re: S-expression I/O in Ada

[Randy Brukardt]

"Robert A Duff" <bobduff@shell01.TheWorld.com> wrote in message 
news:wccoccltpui.fsf@shell01.TheWorld.com...
...
>>...I should really take the time
>> to read the reference manual (hoping now I've reached the level needed
>> to understand it).
>
> Well, the RM is pretty tough going.  I suggest a text book
> instead.  The Barnes book, for example -- it's fun to read
> because of John's wry sense of humour.  There are also some
> on-line tutorials you can find via google.

Or via the Learn page at AdaIC:
    http://www.adaic.com/learn/index.html

                      Randy.

Re: S-expression I/O in Ada

[Jeffrey Carter]

On 08/29/2010 03:45 AM, Natasha Kerensikova wrote:
>
> By the way, about style issues, do you all here really use the
> "standard" (or so it seems to me) indentation of 3 spaces for new blocs
> and 2 spaces for line continuations? This is still way too shallow for my
> tastes.

I indent 3, with the convention that an indentation level = a nesting level. I 
indent continuation lines based on the context.

> I find compiler issues to be the weakest point of Ada: documentation is
> pretty well hidden, installation is a mess, and there is that
> project-file stuff which seems almost as complicated as Ada language
> itself, while being compiler-dependant. Coming for a C world where
> everything is simpler and well documented in manual pages...


Make files are simple? I still don't understand them completely ...

> function Root(From : in Container) return Cursor is
> begin
>     return Cursor'(Parent =>  null, Node =>  From.Root);
> end Root;

This is what I do. Some prefer the other way so they have a name to look at in 
the debugger, but since I haven't used a debugger in years that's not an issue 
fort me.

-- 
Jeff Carter
"We call your door-opening request a silly thing."
Monty Python & the Holy Grail
17

--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

Re: S-expression I/O in Ada

[Natasha Kerensikova]

On 2010-08-29, Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> wrote:
> On 08/29/2010 03:45 AM, Natasha Kerensikova wrote:
>>
>> I find compiler issues to be the weakest point of Ada: documentation is
>> pretty well hidden, installation is a mess, and there is that
>> project-file stuff which seems almost as complicated as Ada language
>> itself, while being compiler-dependant. Coming for a C world where
>> everything is simpler and well documented in manual pages...
>
> Make files are simple? I still don't understand them completely ...

I depends on what you're trying to with them. I agree that makefiles can
be pretty obscure, but that usually happen when you're trying to make it
do more than it was meant to.

Now honestly I have not looked into project files yet, so I have no idea
of their expressive power or clarity.

However project files are specific to Ada, while makefiles have a much
broader use, which makes them more useful as a standard and a more
rewarding time investment.



Thanks for the discussion,
Natasha

Re: S-expression I/O in Ada

[Dmitry A. Kazakov]

On Sun, 29 Aug 2010 14:34:41 +0000 (UTC), Natasha Kerensikova wrote:

> Now honestly I have not looked into project files yet, so I have no idea
> of their expressive power or clarity.

Project files supersede both makefile and configure. For example, we
compile our stuff (quite low-level-messy networking things) for either
Windows or VxWorks using just one project. Try this with makefiles.

> However project files are specific to Ada, while makefiles have a much
> broader use, which makes them more useful as a standard and a more
> rewarding time investment.

No, project files aren't specific to Ada. GPRbuild is designed to support C
and other languages.

As for makefile "standard," did you try a gnu-make file with nmake (MSVC).
The converse?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: S-expression I/O in Ada

[Robert A Duff]

Natasha Kerensikova <lithiumcat@gmail.com> writes:

> On 2010-08-29, Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> wrote:
>> On 08/29/2010 03:45 AM, Natasha Kerensikova wrote:
>>>
>>> I find compiler issues to be the weakest point of Ada: documentation is
>>> pretty well hidden, installation is a mess, and there is that
>>> project-file stuff which seems almost as complicated as Ada language
>>> itself, while being compiler-dependant. Coming for a C world where
>>> everything is simpler and well documented in manual pages...
>>
>> Make files are simple? I still don't understand them completely ...

The 'make' language is pretty simple.  Programs written in this
language (i.e. make files) are often complicated.  The gcc make
files are a nightmare.  Textual macro substitution is just a
wrong way to design a language.

I once had a bug in a make file, where some macro SOURCE_FILES
was set to a list of files in some directory.  Turns out some
program had evilly put a file called .#something in that
directory, so the macro expansion was something like:

SOURCE_FILES=this.ada that.ada .#something other.ada

which is equivalent to:

SOURCE_FILES=this.ada that.ada .

because # starts a comment in 'make'.  It took me a long
time to figure out why other.ada was being ignored.  Yuck!

And combine that with the idiotic rule that 'ls' doesn't
show you files starting with ".", by default.

> I depends on what you're trying to with them. I agree that makefiles can
> be pretty obscure, but that usually happen when you're trying to make it
> do more than it was meant to.
>
> Now honestly I have not looked into project files yet, so I have no idea
> of their expressive power or clarity.

You really don't need to understand project files.  If you're
comfortable with 'make', you can write a very simple make file,
like:

    .PHONY: my_program
    my_program:
        gnatmake -g -O0 -gnata -gnato my_program.adb

Put all your Ada code in the same directory, and follow the
*.ads/*.adb convention.

Maybe put the object files in a subdirectory:

    .PHONY: my_program
    my_program:
        mkdir -p obj
        cd obj ; gnatmake -g -O0 -gnata -gnato -I.. ../my_program.adb

(Above code not tested!)

Or just write a one-line shell script with the gnatmake command in it.

> However project files are specific to Ada, while makefiles have a much
> broader use, which makes them more useful as a standard and a more
> rewarding time investment.

Actually, gprbuild can use project files to build Ada, C, C++, etc.

- Bob

Re: S-expression I/O in Ada

[Georg Bauhaus]

On 8/29/10 12:45 PM, Natasha Kerensikova wrote:

> So I guess that would be "-gnatwae -gnaty" (I do like the idea of style
> checks too). But this raises the question, what is the rationale for
> excluding some warning from -gnatwa?

A programmer might know more than he or she can express  as an
instruction to a compiler, and still be portable among compilers.
Example:

overriding
procedure P (Item : in out Some_Type; Unused : Another_Type) is
   ...

Full warnings might list a message about "Unused" being unused.
You might not want to see it all the time. One possible reason is
that the parameter will not be used, since the algorithm does
not need it. I think that there is a GNAT specific pragma to
suppress the warning (which possibly is reflected somewhere in
the sequence of letters following -gnatw...).

On the other hand, other ways to deliberately ignore a warning are
possible, more portable, and less error prone than the pragma, IMHO.
For just as well, if the procedure P is currently being worked
on, parameter Unused might become used, and then the additional pragma
needs to be dealt with instead.  Other compilers might warn about a
pragma they don't recognize and will ignore. This defeats the purpose.



Georg

Re: S-expression I/O in Ada

[Simon Wright]

Georg Bauhaus <rm-host.bauhaus@maps.futureapps.de> writes:

> Example:
>
> overriding
> procedure P (Item : in out Some_Type; Unused : Another_Type) is
>   ...
>
> Full warnings might list a message about "Unused" being unused.  You
> might not want to see it all the time. One possible reason is that the
> parameter will not be used, since the algorithm does not need it. I
> think that there is a GNAT specific pragma to suppress the warning
> (which possibly is reflected somewhere in the sequence of letters
> following -gnatw...).

pragma Unreferenced (Unused);