Re: very specific question on Ada syntax

Re: very specific question on Ada syntax

[Paul Graham]

Mario Amado Alves wrote:
> 
> >    len := Color'image(Color'first)'length;
> 
> Yes, of course. Thanks. Just found an example in the RM itself:
> 
>   Rainbow'Base'First           4.1.4(16)
> 
> This was because I was wondering if a character literal could be tokenized
> without context. I guess not. In
> 
>   A'B'C
> 
> 'B' has the form of a character literal, but it is not it. I could perhaps
> rely on the fact (?) that there is no attribute of only one letter, but
> that is twisted to say the least.

VHDL, which is based on Ada, allows user defined attributes, so it is
possible to have, say

    x'some_user_attr'length

and there is no rule preventing a user defined attribute from being one
character
in length, so you can have:

    x'y'length

A VHDL parser won't get confused for the same reason that an Ada parser
wouldn't: a character literal cannot immediately follow an identifier.

Paul

RE: very specific question on Ada syntax

[Robert Dewar]

In article <mailman.980504596.2748.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:
> This was because I was wondering if a character literal could
> be tokenized without context. I guess not. In
>
>   A'B'C
>
> 'B' has the form of a character literal, but it is not it. I
> could perhap rely on the fact (?) that there is no attribute
> of only one letter, but that is twisted to say the least.

And, as per my previous comment, wrong, for example, assuming
you accept the brackets notation used for example in the ACATS
tests, you should accept '["0041"]' as equivalent to 'A'.

I strongly suggest reading scn.adb from the GNAT sources when
you have questions about lexical scanning, it is easy reading!


>
> | |,| | | |RuaFranciscoTaborda24RcD 2815-249CharnecaCaparica
351+939354005
> |M|A|R|I|O|
> |A|M|A|D|O|DepartmentoDeInformaticaFCT/UNL 2825-114 Caparica
351+212958536
> |A|L|V|E|S|
fax 212948541
> | | | | | |                 maa@di.fct.unl.pt
FCT 212948300
>
>


Sent via Deja.com
http://www.deja.com/

RE: very specific question on Ada syntax

[Mario Amado Alves]

On Fri, 26 Jan 2001, Christoph Grein wrote:
> There are no one-character attributes I'm aware of; I do not know of a
> requirement in the RM to not have an implementation defined one either.
> But three lexers I know (OpenToken, the one in AdaGIDE, and my one)
> rely on this fact.

Humm... so we have:

Restriction: no one-character attributes.

Conjecture: that restriction is necessary in order to tokenize a
character_literal without semantic or sintactical analysis.

Proof: (left as a home assignment ;-)

Thanks.

| |,| | | |RuaFranciscoTaborda24RcD 2815-249CharnecaCaparica 351+939354005
|M|A|R|I|O|
|A|M|A|D|O|DepartmentoDeInformaticaFCT/UNL 2825-114 Caparica 351+212958536
|A|L|V|E|S|                                                  fax 212948541
| | | | | |                 maa@di.fct.unl.pt                FCT 212948300

Re: very specific question on Ada syntax

[Florian Weimer]

Robert Dewar <robert_dewar@my-deja.com> writes:

> Seeing as the text between the quotes can be more than one
> character in the case of wide character encodings, I would
> guess that you are not handling wide characters :-)

At least in Emacs, even a wide character consists of a single
character. ;-)

Re: very specific question on Ada syntax

[Paul Graham]

Christoph Grein wrote:
> 
> There are no one-character attributes I'm aware of; I do not know of a
> requirement in the RM to not have an implemetation defined one either. But three
> lexers I know (OpenToken, the one in AdaGIDE, and my one) rely on this fact.

I'm curious what the basis is for this reliance on attributes being more
than one character long.  You don't check the preceding token's type to
determine if a character literal is allowed?

Paul

Re: very specific question on Ada syntax

[Robert Dewar]

In article <87bssu2h6w.fsf@deneb.enyo.de>,
  Florian Weimer <fw@deneb.enyo.de> wrote:
> Robert Dewar <robert_dewar@my-deja.com> writes:
>
> > Seeing as the text between the quotes can be more than one
> > character in the case of wide character encodings, I would
> > guess that you are not handling wide characters :-)
>
> At least in Emacs, even a wide character consists of a single
> character. ;-)

NO! That's wrong. Or more accurately, that would be a nasty
restriction. You definitely want to be able to edit texts
using their underlying representation, e.g. seeing ["0041"]
as those 8 characters, rather than as an upper case A.

Incidentally, a bug report was filed internally for Ada mode
in EMACS, since indeed it does not handle this case correctly
:-)


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Robert Dewar]

In article <3A71FD83.281DFC3E@cadence.com>,
  Paul Graham <pgraham@cadence.com> wrote:

> I'm curious what the basis is for this reliance on attributes
> being more than one character long.

Well that's easy enough to see! It's easier to do it this way.

> You don't check the preceding token's type to
> determine if a character literal is allowed?

Read the thread, I already posted the details of this
approach.


Sent via Deja.com
http://www.deja.com/

RE: very specific question on Ada syntax

[Robert Dewar]

In article <mailman.980529197.22106.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:
> Conjecture: that restriction is necessary in order to
> tokenize a character_literal without semantic or sintactical
> analysis.

Since the truth of this depends on your arbitrary definition
of what is and what is not sintactical analysis (I leave the
misspelling to emaphasize that in fact this is NOT a standard
precisely defined technical term :-), this is not a well
formed conjecture.

In fact properly written lexical analyzers for Ada solve this
in the manner I previously posted, WITHIN the lexical analyzer
(this is perfectly possible either in a hand written lexer, or
in an automatically built one). The method is simple and
probably the only reason it is not used is simply a matter of
not being aware of this standard technique which has been well
known for 20 years (i.e. check previous token).





>
> Proof: (left as a home assignment ;-)
>
> Thanks.
>
> | |,| | | |RuaFranciscoTaborda24RcD 2815-249CharnecaCaparica
351+939354005
> |M|A|R|I|O|
> |A|M|A|D|O|DepartmentoDeInformaticaFCT/UNL 2825-114 Caparica
351+212958536
> |A|L|V|E|S|
fax 212948541
> | | | | | |                 maa@di.fct.unl.pt
FCT 212948300
>
>


Sent via Deja.com
http://www.deja.com/

RE: very specific question on Ada syntax

[Mario Amado Alves]

"Restriction:
   no one-character attribute designators.
Conjecture:
   that restriction is necessary in order to tokenize a
   character_literal without semantic or syntactical analysis."
(Alves, revised)

"Since the truth of this depends on your arbitrary definition
of what is and what is not syntactical analysis [...]"
(Dewar)

Syntactical: ranging more than one lexical element. A lexer may be purely
morphological, i.e. based solely on the individual form of each element.
But that is rare. An Ada lexer must have some syntax---so the conjecture is
true!

"[..] properly written lexical analyzers for Ada solve this" (Dewar)

It is nice to know my personal lexer is "proper" apud Dewar :) Excerpt:

          when ''' =>
            if Previous_Kind = Identifier
            or (Previous_Kind = Delimiter and To_String(Previous_Form)=")")
            or Previous_Kind = Reserved_Word_All
            then
              Send(Delimiter, C);
            else
              State := Character_Literal_1;
            end if;

And I was surprised much hyped systems (OpenToken, AdaGIDE...) fail this!

Thanks.

| |,| | | |RuaFranciscoTaborda24RcD 2815-249CharnecaCaparica 351+939354005
|M|A|R|I|O|
|A|M|A|D|O|DepartmentoDeInformaticaFCT/UNL 2825-114 Caparica 351+212958536
|A|L|V|E|S|                                                  fax 212948541
| | | | | |                 maa@di.fct.unl.pt                FCT 212948300

RE: very specific question on Ada syntax

[Ted Dennison]

In article <mailman.980766440.11889.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:

> And I was surprised much hyped systems (OpenToken, AdaGIDE...) fail
> this!

Actually, I was quite suprised to read that as well.

I just tried the test_ada_lexer from the OpenToken examples directory on
the following input:

A'L;

and got the output

Found IDENTIFIER_T A
Found TICK_T '
Found IDENTIFIER_T L
Found SEMICOLON_T ;
Found END_OF_FILE_T *

(actually, the last wasn't an "*" but an EOF char. I didn't want to
paste that in because I don't know what kind of havoc that would cause
to everyone's newsreaders).

That looks like the right output to me. Am I missing some parameter of
the problem?

--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com
http://www.deja.com/

RE: very specific question on Ada syntax

[Ted Dennison]

In article <9543p4$u4u$1@nnrp1.deja.com>,
  Ted Dennison <dennison@telepath.com> wrote:

> I just tried the test_ada_lexer from the OpenToken examples directory
> on the following input:
>
> A'L;
...
> That looks like the right output to me. Am I missing some parameter of
> the problem?

Ahhh, yes I was. I tried it on:

A'L'M;

and got:

Found IDENTIFIER_T A
Found CHARACTER_T 'L'
Found IDENTIFIER_T M
Found SEMICOLON_T ;
Found END_OF_FILE_T *

I suppose the original author of the Ada syntax (Christop, I think) was
more or less forced to do it that way because the OpenToken lexical
analyzer keeps no record of what the last token was. That can certainly
be fixed. It will add a bit of processing time on each token. But
(thanks to Robert Dewar) I now have some ideas for optimizing the
scanning that ought to speed things up significantly, so the processing
time delta in the next release ought to still be in the good direction.

Note that OpenToken is not all about analyzing Ada; its a general
lexical analysis/parsing utility along the lines of lex/yacc and ANTLR
(except that it is totally API-based). In fact, it was initially
developed to help parse configuration files, and all my commercial work
with it still centers around that. The Ada tokenizer is just one of the
many user-submitted extensions that are included in the distribution. I
won't complain if it gets to the point where most users are downloading
OpenToken just to get the the Ada analyzer. However, there's a lot more
to it than just that.

I'd like to give kudos to Christoph for investigating this issue in the
first place. Many developers I know would have just let it slide, since
it works fine with all current Ada compilers.

--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com
http://www.deja.com/

RE: very specific question on Ada syntax

[Mario Amado Alves]

> I just tried the test_ada_lexer from the OpenToken examples directory on
> the following input:
> 
> A'L;
> 
> and got the output
> 
> Found IDENTIFIER_T A
> Found TICK_T '
> Found IDENTIFIER_T L
> Found SEMICOLON_T ;
> Found END_OF_FILE_T *
> 
> That looks like the right output to me. Am I missing some parameter of
> the problem?

It is the right output. The problem is tokenisation of character literals
relying on inexistence of one-character attribute designator. Someone said
OpenToken did that. Try input

  A'B'C

(attribute C of attribute B of something A) on the OpenToken lexer. If what
that someone said is true, the output will ressemble

  Identifier_T         A
  Character_Literal_T  B
  Identifier_T         C

(which is incorrect).

Thanks.

| |,| | | |RuaFranciscoTaborda24RcD 2815-249CharnecaCaparica 351+939354005
|M|A|R|I|O|
|A|M|A|D|O|DepartmentoDeInformaticaFCT/UNL 2825-114 Caparica 351+212958536
|A|L|V|E|S|                                                  fax 212948541
| | | | | |                 maa@di.fct.unl.pt                FCT 212948300

RE: very specific question on Ada syntax

[Robert Dewar]

In article <9543p4$u4u$1@nnrp1.deja.com>,
  Ted Dennison <dennison@telepath.com> wrote:
> In article
<mailman.980766440.11889.comp.lang.ada@ada.eu.org>,
>   comp.lang.ada@ada.eu.org wrote:
> I just tried the test_ada_lexer from the OpenToken examples
> directory on the following input:
>
> A'L;
>
> and got the output
>
> Found IDENTIFIER_T A
> Found TICK_T '
> Found IDENTIFIER_T L
> Found SEMICOLON_T ;
> Found END_OF_FILE_T *

Well of course, this is bound to work, and has nothing to do
with the case we are discussing, which is A'B'C. I think you
missed something :-)


Sent via Deja.com
http://www.deja.com/

RE: very specific question on Ada syntax

[Christoph Grein]

Ted Dennison wrote:

> A'L'M;
> 
> and got:
> 
> Found IDENTIFIER_T A
> Found CHARACTER_T 'L'
> Found IDENTIFIER_T M
> Found SEMICOLON_T ;
> Found END_OF_FILE_T *
> 
> I suppose the original author of the Ada syntax (Christop, I think) was

Ted, you're doing me too much favour, I'm definitely NOT the author of the Ada 
syntax ;>) only of the syntax analyser.

> more or less forced to do it that way because the OpenToken lexical
> analyzer keeps no record of what the last token was. That can certainly
> be fixed. It will add a bit of processing time on each token. But

I think this isn't worth it as long as we do not definitely know of a compiler 
having a one letter attribute that also can appear as an attribute prefix (since 
A'L is correctly lexed).

And as long as there are no such compilers, A'L'M is just a syntax error because 
it should perhaps be A & 'L' & M. So, as Robert Dewar said, Ada lexing is not so 
simple as it might look.

BTW: I'm a physicist and have never studied (computer) language theory. I 
assumed that lexical analysis need not care for the previous token. Now I stand 
corrected. [OK, I already knew about the difference of access and 'Access, but I 
ignored this deliberately. Also here OpenToken will fail.]

I'm more worried about how to lex wide characters. I gather the best way in 
OpenToken would be to leave the character recognizer alone and create a new 
wide_character recognizer, which then could handle the different encodings. We 
can postpone this until the need arises.


Christoph Grein

Re: very specific question on Ada syntax

[Jean-Pierre Rosen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 657 bytes --]


"Christoph Grein" <christoph.grein@eurocopter.de> a �crit dans le message news: mailman.980836879.792.comp.lang.ada@ada.eu.org...
> I think this isn't worth it as long as we do not definitely know of a compiler
> having a one letter attribute that also can appear as an attribute prefix (since
> A'L is correctly lexed).
>
Also try this one:
Character'('a')

It's: Identifier-Apostrophe-Left par-character litteral-Right par
and not: Identifier-character litteral-Apostrophe-left par

--
---------------------------------------------------------
           J-P. Rosen (Rosen.Adalog@wanadoo.fr)
Visit Adalog's web site at http://pro.wanadoo.fr/adalog

Re: very specific question on Ada syntax

[Jean-Pierre Rosen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 718 bytes --]


"Jean-Pierre Rosen" <rosen.adalog@wanadoo.fr> a �crit dans le message news: ...
>
> "Christoph Grein" <christoph.grein@eurocopter.de> a �crit dans le message news: mailman.980836879.792.comp.lang.ada@ada.eu.org...
> > I think this isn't worth it as long as we do not definitely know of a compiler
> > having a one letter attribute that also can appear as an attribute prefix (since
> > A'L is correctly lexed).
> >
> Also try this one:
> Character'('a')
>
Well, to make things more interesting, this should be:
subtype S is character;
S'('a')

--
---------------------------------------------------------
           J-P. Rosen (Rosen.Adalog@wanadoo.fr)
Visit Adalog's web site at http://pro.wanadoo.fr/adalog

Re: very specific question on Ada syntax

[Florian Weimer]

Robert Dewar <robert_dewar@my-deja.com> writes:

> > At least in Emacs, even a wide character consists of a single
> > character. ;-)
> 
> NO! That's wrong. Or more accurately, that would be a nasty
> restriction. You definitely want to be able to edit texts
> using their underlying representation, e.g. seeing ["0041"]
> as those 8 characters, rather than as an upper case A.

Ah, you were talking about the unusual ACVC/GNAT representation of
wide characters using non-wide characters!  Emacs has its own internal
(and different) representation of multibyte characters, and an Emacs
multibyte character is displayed as a single character.

Re: very specific question on Ada syntax

[Christoph Grein]

Jean-Pierre Rosen wrote:
> Also try this one:
> Character'('a')
> 
> It's: Identifier-Apostrophe-Left par-character litteral-Right par
> and not: Identifier-character litteral-Apostrophe-left par

Thanks for the hint. I have to change my mind.

Christoph Grein
--

Re: very specific question on Ada syntax

[Mario Amado Alves]

> Character'('a')

This is the killer one. Well formed on all accounts, and still problematic.
How well does the OpenToken-base lexer on this?

| |,| | | |RuaFranciscoTaborda24RcD 2815-249CharnecaCaparica 351+939354005
|M|A|R|I|O|
|A|M|A|D|O|DepartmentoDeInformaticaFCT/UNL 2825-114 Caparica 351+212958536
|A|L|V|E|S|                                                  fax 212948541
| | | | | |                 maa@di.fct.unl.pt                FCT 212948300

Re: very specific question on Ada syntax

[Tucker Taft]

Mario Amado Alves wrote:
> 
> >    len := Color'image(Color'first)'length;
> 
> Yes, of course. Thanks. Just found an example in the RM itself:
> 
>   Rainbow'Base'First           4.1.4(16)
> 
> This was because I was wondering if a character literal could be tokenized
> without context. I guess not. In
> 
>   A'B'C
> 
> 'B' has the form of a character literal, but it is not it. I could perhaps
> rely on the fact (?) that there is no attribute of only one letter, but
> that is twisted to say the least.

Anyone who has built a production-quality Ada compiler will
know that distinguishing the attribute "'" from the character literal "'"..."'"
requires some special finagling.  The technique most of us have
adopted is to keep track of whether the immediately prior token
can be the last token of a "name", and if so, interpret
the "'" as an attribute "tic" rather than as a character-literal "quote".
In particular, if the prior token is:
    identifier
    )
    string-literal
    ALL
    character-literal

then a "'" should be interpreted as an attribute "tic"; otherwise
presume it is the beginning of a character literal.

> 
> | |,| | | |RuaFranciscoTaborda24RcD 2815-249CharnecaCaparica 351+939354005
> |M|A|R|I|O|
> |A|M|A|D|O|DepartmentoDeInformaticaFCT/UNL 2825-114 Caparica 351+212958536
> |A|L|V|E|S|                                                  fax 212948541
> | | | | | |                 maa@di.fct.unl.pt                FCT 212948300

-- 
-Tucker Taft   stt@avercom.net   http://www.averstar.com/~stt/
Chief Technology Officer, AverCom, Inc. (A Titan Company) Burlington, MA  USA
(AverCom was formed 1/1/01 from the Commercial Division of AverStar)
(http://www.averstar.com/services/ebusiness_applications.html)

Re: very specific question on Ada syntax

[Robert Dewar]

In article <mailman.980871715.11959.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:
> > Character'('a')
>
> This is the killer one.

Yes, this is well known, and as Tuck says, all production
quality Ada lexers have known about this for two decades,
and handle it with no problem (apostrophe after identifier
is always a tic). Any lexer not able to do such computations
will have trouble. Note that if you are using a tool like
flex, there is absolutely no problem in dealing with this.
I do not know about open token.


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Robert Dewar]

In article <8766ixe89y.fsf@deneb.enyo.de>,
  Florian Weimer <fw@deneb.enyo.de> wrote:

> Ah, you were talking about the unusual ACVC/GNAT
> representation of wide characters using non-wide characters!

Nothing unusual about that. GNAT supports six different
representations of wide characters, including the widely
used JIS representation usually used in Japan for such
texts, and the ISO standard UTF representation.

>  Emacs has its own internal
> (and different) representation of multibyte characters, and
> an Emacs multibyte character is displayed as a single
> character.

Well yes of course your editor is expected to understand
the encoding sequence you use. But the compiler does not
look at an EMACS screen, it looks at the underlying
representation, and this will typically be a multi-character
representation.

Yes, you can imagine a world in which 16 bit characters are
used uniformly, but that world is not today's world (I am
talking about common usage in Japan, Korea and China -- all
of which tend to use different representation methods, all
common ones of which are supported by GNAT.


Sent via Deja.com
http://www.deja.com/

RE: very specific question on Ada syntax

[Robert Dewar]

In article <mailman.980836879.792.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:
> OK, I already knew about the difference of access and
> 'Access, but I  ignored this deliberately.

You may be confused here, when the 'Access attribute is used,
the word access here is the reserved keyword access, not an
identifier (go look at the grammar in the RM).

> Also here OpenToken will fail.]

How? It should return access as a reserved keyword, surely
it does ...

> I'm more worried about how to lex wide characters. I gather
> the best way in  OpenToken would be to leave the character
> recognizer alone and create a new wide_character recognizer,
> which then could handle the different encodings. We
> can postpone this until the need arises.

There are quite a few tricky issues here, have a look at the
GNAT scanner for information.


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Ted Dennison]

In article <mailman.980871715.11959.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:
> > Character'('a')
>
> This is the killer one. Well formed on all accounts, and still
> problematic. How well does the OpenToken-base lexer on this?

As expected, it doesn't handle it properly. Defininitely a bug. It
should be fixed in the next version. :-)

--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Ted Dennison]

In article <956vav$e7q$1@nnrp1.deja.com>,
  Robert Dewar <robert_dewar@my-deja.com> wrote:
> will have trouble. Note that if you are using a tool like
> flex, there is absolutely no problem in dealing with this.

To the best of my knowledge, flex has no "last token" function in its
API. It can do *character* context, but not token context. Thus to
properly handle this situation with lex, you'd have to hack it in
manually with inserted C code (most likely by passing info back from the
parser).

I just checked the Ada95 lex spec available from the AdaIC site
(http://www.adaic.org/standards/95lrm_rat/lexer9x.l
). It indeed contains no context information in the token definitions,
and thus should have the same problem. The relevent entries are:
-------------------------------------------
"'"                     return(TIC);
"'"."'"                 return(char_lit);
-------------------------------------------


--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Florian Weimer]

Robert Dewar <robert_dewar@my-deja.com> writes:

> Yes, you can imagine a world in which 16 bit characters are
> used uniformly, but that world is not today's world (I am
> talking about common usage in Japan, Korea and China -- all
> of which tend to use different representation methods, all
> common ones of which are supported by GNAT.

BTW, Unicode is now a 21.something bit character set, and the
Wide_Character type is no longer appropriate for representing Unicode
(UCS-4) characters.  Will Ada be adapted accordingly?

RE: very specific question on Ada syntax

[Christoph Grein]

Robert Dewar Wrote:

>> OK, I already knew about the difference of access and
>> 'Access, but I  ignored this deliberately.
>
>You may be confused here, when the 'Access attribute is used,
>the word access here is the reserved keyword access, not an
>identifier (go look at the grammar in the RM).
>

OK, you are right, but also the RM does not print these attributes in bold lower 
case.

So if a lexer returns them as reserved words when used as attributes, it 
complicates (hm, "complicates" seems too strong) the job of analysing source 
code. IMHO it's rather irrelevant for lexical analysis if an attribute is 
treated as a reserved word or an identifier.

So please what is the most appropriate treatment (i.e. preventing extra checks 
AFTER lexical analysis in most cases) for 'Access, 'Delta, 'Digits, 'Range? Tick 
Identifier or Tick Reserved_Word?

I would tend to say, it's better to put the extra check for these four into the 
lexer and treat them as identifiers afterwards.

>> Also here OpenToken will fail.]
>
>How? It should return access as a reserved keyword, surely
>it does ...>

Of course is does. So in the light of the above, it does not fail.

RE: very specific question on Ada syntax

[Robert Dewar]

In article <mailman.980917762.19771.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:
> So please what is the most appropriate treatment (i.e.
> preventing extra checks  AFTER lexical analysis in most
> cases) for 'Access, 'Delta, 'Digits, 'Range? Tick
> Identifier or Tick Reserved_Word?

Either would work, someone has to list special cases here. In
GNAT we find it preferable not to distort the lexical rules of
the language (which clearly say that it is a reserved word),
and do the special tests in the parser, following the official
grammar in the RM.

But you could alter the lexical structure and alter the
grammar, and do the special checks in the lexical analyzer
if you want.

GNAT always attempts to keep its internal data structures as
close as possible to the formal definition. This has proved
a good choice since it means that the RM acts as a kind of
manual for the compiler.


>
> I would tend to say, it's better to put the extra check for
these four into the
> lexer and treat them as identifiers afterwards.
>
> >> Also here OpenToken will fail.]
> >
> >How? It should return access as a reserved keyword, surely
> >it does ...>
>
> Of course is does. So in the light of the above, it does not
fail.
>
>


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Robert Dewar]

In article <877l3ck7nb.fsf@deneb.enyo.de>,
  Florian Weimer <fw@deneb.enyo.de> wrote:
> Robert Dewar <robert_dewar@my-deja.com> writes:
>
> > Yes, you can imagine a world in which 16 bit characters are
> > used uniformly, but that world is not today's world (I am
> > talking about common usage in Japan, Korea and China -- all
> > of which tend to use different representation methods, all
> > common ones of which are supported by GNAT.
>
> BTW, Unicode is now a 21.something bit character set, and the
> Wide_Character type is no longer appropriate for representing
Unicode
> (UCS-4) characters.  Will Ada be adapted accordingly?

I think in practice that the 16-bit subset of Unicode, the BMP
page of 10646, will be much more widely used in practice than
the full 21-bit set, and it was a quite deliberate decision
in the design of Ada to restrict Wide_Chararacter to the BMP.

Of course there is certainly nothing to stop the addition of
a Wide_Wide_Character type to the language, and at the very
least it would be worth an ARG ruling permitting this usage.

Actually our experience is that very few people are even using
the BMP in most Ada applications. There was certainly a
requirement to support it in Ada 95 (it in fact was a deal that
had been made in passing the ISO Ada 85 standard, where it was
noted that the standard was deficient with respect to character
set support, and it was agred that this would be addressed
in the next revision of the language).

Of course if Ada gets used more in international commerce,
perhaps making use of the IS annex, then the provision of
Wide_Character is important.

So far, the most significant program I know of using
Wide_Character and Wide_String is the ASIS definition
itself, where it is used to represent source texts.



Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Robert Dewar]

In article <9573th$isv$1@nnrp1.deja.com>,
  Ted Dennison <dennison@telepath.com> wrote:
> To the best of my knowledge, flex has no "last token"
> function in its API. It can do *character* context, but not
> token context. Thus to properly handle this situation with
> lex, you'd have to hack it in manually with inserted C code
> (most likely by passing info back from the parser).

That's not the way I would do it at all. I would do it the
same way that the GNAT lexer does things, which is to keep
a global variable which remembers the previous token. That's
useful for posting error messages anyway.

There is absolutely NO problem in coding this in lex, remember
that in lex, the whole idea is that each recognized token
causes the execution of "inserted C code", this C code is
not a hack, it is the essence of how flex works. I definitely
would NOT do it by passing information back from the parser.

See scans.ads and the entry to scn.adb in the GNAT sources
for details on how this is done in GNAT. The issue of whether
the lexer is written by hand or generated by flex is quite
irrelevant in considering how to handle this situation.


>
> I just checked the Ada95 lex spec available from the AdaIC
site
> (http://www.adaic.org/standards/95lrm_rat/lexer9x.l
> ). It indeed contains no context information in the token
definitions,
> and thus should have the same problem. The relevent entries
are:
> -------------------------------------------
> "'"                     return(TIC);
> "'"."'"                 return(char_lit);
> -------------------------------------------
>
> --
> T.E.D.
>
> http://www.telepath.com/~dennison/Ted/TED.html
>
> Sent via Deja.com
> http://www.deja.com/
>


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Robert Dewar]

In article <9573c4$iav$1@nnrp1.deja.com>,
  Ted Dennison <dennison@telepath.com> wrote:

> As expected, it doesn't handle it properly. Defininitely a
> bug. It should be fixed in the next version. :-)

I would definitely suggest looking through the source of
scn.adb in GNAT to check on peculiarities of this kind.


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Mario Amado Alves]

> > > Character'('a')
> >
> > This is the killer one.
> 
> Yes, this is well known, and as Tuck says, all production
> quality Ada lexers have known about this for two decades,

Good. I suspected that, but only come into contact with Ada two years ago.

> and handle it with no problem (apostrophe after identifier
> is always a tic). Any lexer not able to do such computations
> will have trouble. Note that if you are using a tool like
> flex, there is absolutely no problem in dealing with this.

I write my lexers "by hand" (I dont understand flex), but without problem
anyway. I already posted the relevant excerpt. Here it is again:

  63   when ''' =>
  64     if Previous_Kind = Identifier
  65     or (Previous_Kind = Delimiter and To_String(Previous_Form)=")")
  66     or Previous_Kind = Reserved_Word_All
  67     then
  68       Send(Delimiter, C);
  69     else
  70       State := Character_Literal_1;
  71     end if;

According to Dewar and Taft (this thread) I should add String_Literal (Taft
only) and Character_Literal to the list of apostrophe-enabling previous
tokens (lines 64-66). But those are syntax errors, no?

I tried

  "123"'Length

on GNAT. It says "binary operator expected" on the apostrophe (error).

Thanks.

| |,| | | |RuaFranciscoTaborda24RcD 2815-249CharnecaCaparica 351+939354005
|M|A|R|I|O|
|A|M|A|D|O|DepartmentoDeInformaticaFCT/UNL 2825-114 Caparica 351+212958536
|A|L|V|E|S|                                                  fax 212948541
| | | | | |                 maa@di.fct.unl.pt                FCT 212948300

Re: very specific question on Ada syntax

[Florian Weimer]

Robert Dewar <robert_dewar@my-deja.com> writes:

> > BTW, Unicode is now a 21.something bit character set, and the
> > Wide_Character type is no longer appropriate for representing
> Unicode
> > (UCS-4) characters.  Will Ada be adapted accordingly?
> 
> I think in practice that the 16-bit subset of Unicode, the BMP
> page of 10646, will be much more widely used in practice than
> the full 21-bit set, and it was a quite deliberate decision
> in the design of Ada to restrict Wide_Chararacter to the BMP.

Things have changed rather dramatically.  Just two years ago, it
seemed that the other planes would be used for rarely-used scripts,
most of them only interesting scholars (e.g. Linear B).  Now there are
several proposals which could become pretty relevant as soon as
Unicode is more widely used (if this really happens some day), for
example the alphanumeric math characters.

> Of course there is certainly nothing to stop the addition of
> a Wide_Wide_Character type to the language, and at the very
> least it would be worth an ARG ruling permitting this usage.

Hmm, I'll try to figure out the formal requirements for an ARG
submission.  Which is the correct address?

http://www.ada-auth.org/arm-html/RM-0-3.html mentions the address
ada-comment@ada-auth.org, but the ARG procedures
http://www.ada-auth.org/ai-files/minutes/arg-proc.txt request to use
ada-comment@sw-eng.falls-church.va.us.

Re: very specific question on Ada syntax

[Marc A. Criley]

Florian Weimer wrote:
> 
> Hmm, I'll try to figure out the formal requirements for an ARG
> submission.  Which is the correct address?
> 
> http://www.ada-auth.org/arm-html/RM-0-3.html mentions the address
> ada-comment@ada-auth.org, but the ARG procedures

This ^^ is the correct one to use.

Marc A. Criley
Senior Staff Engineer
Quadrus Corporation
www.quadruscorp.com

Re: very specific question on Ada syntax

[Florian Weimer]

Mario Amado Alves <maa@di.fct.unl.pt> writes:

> According to Dewar and Taft (this thread) I should add String_Literal (Taft
> only) and Character_Literal to the list of apostrophe-enabling previous
> tokens (lines 64-66). But those are syntax errors, no?

A character literal is a prefix, taking attributes is allowed.  In
contrast, a string literal is not a prefix, see RM 4.1(2) and 4.1.4.

Re: very specific question on Ada syntax

[Robert Dewar]

In article <mailman.980940984.1270.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:
 (I dont understand flex)

really? I can't imagine a simpler concept than the one behind
flex, and even reading the entire manual with all the gory
details is pretty easy, it is not that long. Even if you want
to write lexical analyzers by hand (a decision I certainly find
reasonable in some cases), it is worth making the very minimal
effort to understand how tools like flex are used.


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Robert Dewar]

In article <87k87bs7az.fsf@deneb.enyo.de>,
  Florian Weimer <fw@deneb.enyo.de> wrote:

> A character literal is a prefix, taking attributes is
allowed.  In
> contrast, a string literal is not a prefix, see RM 4.1(2) and
4.1.4.


No, but "and" is!


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Robert Dewar]

In article <87k87bs7az.fsf@deneb.enyo.de>,
  Florian Weimer <fw@deneb.enyo.de> wrote:

> A character literal is a prefix, taking attributes is
allowed.  In
> contrast, a string literal is not a prefix, see RM 4.1(2) and
4.1.4.


No, but "and" is!
so a."and"'a'b
is syntactically fine :-)


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Christoph Grein]

Florian Weimer wrote:

> A character literal is a prefix, taking attributes is allowed.  In
> contrast, a string literal is not a prefix, see RM 4.1(2) and 4.1.4.

Compiling the following with Gnat 3.13p gives:


package P is

  type  T  is ('a', None, 'Z');
  type F_T is access function return T;

  X: F_T := 'a''Access;
  Y: F_T := None'Access;

end P;

     6.   X: F_T := 'a''Access;
                    |
        >>> prefix of "Access" attribute must be aliased

Is this then a bug?

Re: very specific question on Ada syntax

[Robert Dewar]

In article <mailman.981007013.11542.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:
> Florian Weimer wrote:
>
> > A character literal is a prefix, taking attributes is
allowed.  In
> > contrast, a string literal is not a prefix, see RM 4.1(2)
and 4.1.4.
>
> Compiling the following with Gnat 3.13p gives:
>
> package P is
>
>   type  T  is ('a', None, 'Z');
>   type F_T is access function return T;
>
>   X: F_T :=3D 'a''Access;
>   Y: F_T :=3D None'Access;
>
> end P;
>
>      6.   X: F_T :=3D 'a''Access;
>                     |
>         >>> prefix of "Access" attribute must be aliased
>
> Is this then a bug?

No, this is obviously a correct error message. What would
make you think that this particular usage was legal?


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Christoph Grein]

Robert Dewar wrote:

> > package P is
> >
> >   type  T  is ('a', None, 'Z');
> >   type F_T is access function return T;
> >
> >   X: F_T := 'a''Access;
> >   Y: F_T := None'Access;
> >
> > end P;
> >
> >      6.   X: F_T := 'a''Access;
> >                     |
> >         >>> prefix of "Access" attribute must be aliased
> >
> > Is this then a bug?
> 
> No, this is obviously a correct error message. What would
> make you think that this particular usage was legal?

A prefix is a name, and a character literal is a name. So taking an attribute is 
legal. An enumeration literal is a parameterless function, its convention is 
intrinsic. The Access attribute is not allowed for intrinsic subprograms.

So I see two bugs (the second being a misleading error message :-).

Christoph Grein

Re: very specific question on Ada syntax

[Keith Thompson]

Robert Dewar <robert_dewar@my-deja.com> writes:
> In article <mailman.981007013.11542.comp.lang.ada@ada.eu.org>,
>   comp.lang.ada@ada.eu.org wrote:
> > Florian Weimer wrote:
> >
> > > A character literal is a prefix, taking attributes is
> allowed.  In
> > > contrast, a string literal is not a prefix, see RM 4.1(2)
> and 4.1.4.
> >
> > Compiling the following with Gnat 3.13p gives:
> >
> > package P is
> >
> >   type  T  is ('a', None, 'Z');
> >   type F_T is access function return T;
> >
> >   X: F_T := 'a''Access;
> >   Y: F_T := None'Access;
> >
> > end P;
> >
> >      6.   X: F_T := 'a''Access;
> >                     |
> >         >>> prefix of "Access" attribute must be aliased
> >
> > Is this then a bug?
> 
> No, this is obviously a correct error message. What would
> make you think that this particular usage was legal?

I think the error message would be clearer if it said that the prefix
must not be an intrinsic subprogram.  RM95-6.3.1 says that the calling
convention for an enumeration literal is Intrinsic; 3.10.2 says that
the Access attribute cannot be applied to an intrinsic subprogram.

One source of confusion is that GNAT doesn't complain about
None'Access; presumably this is a bug.  I don't have a recent version
of Gnat installed, so I'll let someone else submit a bug report
(unless it's already been fixed).

-- 
Keith Thompson (The_Other_Keith) kst@cts.com  <http://www.ghoti.net/~kst>
San Diego Supercomputer Center           <*>  <http://www.sdsc.edu/~kst>
MAKE MONEY FAST!!  DON'T FEED IT!!

Re: very specific question on Ada syntax

[Christoph Grein]

''''A'B => character literal '''
           'A attribute
           'B attribute
and not character literal '''
        character literal 'A'
        identifier        B

And I always thought Ada is not cryptic :-)

Re: very specific question on Ada syntax

[Robert Dewar]

In article <mailman.981026222.22976.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:
> ''''A'B => character literal '''
>            'A attribute
>            'B attribute
> and not character literal '''
>         character literal 'A'
>         identifier        B
>
> And I always thought Ada is not cryptic :-)

Anyone can write anything in any language in cryptic form
if they try hard enough :-)

In practice of course it is very unlikely that any
implementation would introduce a single character attribute,
it would be very unada to do so.

Furthermore there are almost no attributes that can apply to
a character literal anyway.


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Mario Amado Alves]

> > (I dont understand flex) (Alves)
> 
> really? I can't imagine a simpler concept than the one behind flex
> (Dewar)

I chose the word "understand" wrongly. I know very well the concepts behind
it, as per e.g. the "Modern Compiler Design with Java" book. I simply never
utilized flex/yacc. I should have said "I dont _talk_ flex". Never saw a
flex manual worth that name. I am glad to know one exists that is not very
long ;-) I will try to locate it someday. Thanks.

| |,| | | |RuaFranciscoTaborda24RcD 2815-249CharnecaCaparica 351+939354005
|M|A|R|I|O|
|A|M|A|D|O|DepartmentoDeInformaticaFCT/UNL 2825-114 Caparica 351+212958536
|A|L|V|E|S|                                                  fax 212948541
| | | | | |                 maa@di.fct.unl.pt                FCT 212948300

Re: very specific question on Ada syntax

[Robert Dewar]

In article <mailman.981051603.8824.comp.lang.ada@ada.eu.org>,
  comp.lang.ada@ada.eu.org wrote:

> I chose the word "understand" wrongly. I know very well the
> concepts behind it, as per e.g. the "Modern Compiler Design
> with Java" book. I simply never utilized flex/yacc. I should
> have said "I dont _talk_ flex". Never saw  flex manual worth
> that name. I am glad to know one exists that is not very
> long ;-) I will try to locate it someday. Thanks.

From my course page, try this link

http://cs.nyu.edu/courses/spring01/G22.2130-001/flex/flex_toc.h
tml

The course page may be of interest to some people here, it is
a graduate compilers course which is based on looking at the
front end of GNAT closely, and the class project is to write
a code generator for a subset of Ada compiled using the GNAT
front end.


Sent via Deja.com
http://www.deja.com/

Re: very specific question on Ada syntax

[Christoph Grein]

Robert Dewar wrote:
> 
> In practice of course it is very unlikely that any
> implementation would introduce a single character attribute,
> it would be very unada to do so.
> 
> Furthermore there are almost no attributes that can apply to
> a character literal anyway.

OK, I know that, but if I have learnt anything valuable from this, it's that 
from a syntax point of view, it's irrelevant whether such an attribute exists or 
not.

It's just fun to see something like (C'('C')'C) :-)
which is an illegal expression, whereas (C'('(')) is legal.

Christoph Grein

Re: very specific question on Ada syntax

[Florian Weimer]

Robert Dewar <robert_dewar@my-deja.com> writes:

> > A character literal is a prefix, taking attributes is allowed.  In
> > contrast, a string literal is not a prefix, see RM 4.1(2) and
> > 4.1.4.
> 
> 
> No, but "and" is!  so a."and"'a'b is syntactically fine :-)

Oops, Ada syntax is a lot more obscure than I thought. ;-)