Is there a lex utility for Ada that handles unicode?

Is there a lex utility for Ada that handles unicode?

[brian.b.mcguinness]

Is there some equivalent of the lex utility that produces
Ada code rather than C code, and is capable of handling
any character in the Unicode basic code plane?  I am
thinking of using it on strings read from a GUI created
with GtkAda, so it would probably be best if it accepted
UTF-8 strings, but I could convert the input to a wide
string if necessary.

Thanks.

--- Brian

Re: Is there a lex utility for Ada that handles unicode?

[Martin Dowie]

brian.b.mcguinness@lmco.com wrote:
> Is there some equivalent of the lex utility that produces
> Ada code rather than C code, and is capable of handling
> any character in the Unicode basic code plane?  I am
> thinking of using it on strings read from a GUI created
> with GtkAda, so it would probably be best if it accepted
> UTF-8 strings, but I could convert the input to a wide
> string if necessary.

Alex and Ayacc:
http://www.iste.uni-stuttgart.de/ps/ada-software/html/tools.html

<Warning>
I've never even looked at them! :-)
</Warning>

Cheers

-- Martin

Re: Is there a lex utility for Ada that handles unicode?

[Frank J. Lhota]

Martin Dowie wrote:
> 
> 
> Alex and Ayacc:
> http://www.iste.uni-stuttgart.de/ps/ada-software/html/tools.html
> 
> <Warning>
> I've never even looked at them! :-)
> </Warning>
> 
> Cheers

I've used Alex and Ayacc. They are rather close to Lex / Yacc. Lex and 
Yacc, however, are designed to work with 8 bit characters, and Alex / 
Ayacc inherit that design. The parsing of Unicode sources require a 
different set of tools.

> -- Martin


-- 
"All things extant in this world,
Gods of Heaven, gods of Earth,
Let everything be as it should be;
Thus shall it be!"
- Magical chant from "Magical Shopping Arcade Abenobashi"

"Drizzle, Drazzle, Drozzle, Drome,
Time for the this one to come home!"
- Mr. Lizard from "Tutor Turtle"

Re: Is there a lex utility for Ada that handles unicode?

[Dmitry A. Kazakov]

On 27 Oct 2005 10:17:15 -0700, brian.b.mcguinness@lmco.com wrote:

> Is there some equivalent of the lex utility that produces
> Ada code rather than C code, and is capable of handling
> any character in the Unicode basic code plane?  I am
> thinking of using it on strings read from a GUI created
> with GtkAda, so it would probably be best if it accepted
> UTF-8 strings, but I could convert the input to a wide
> string if necessary.

Why do you wish to convert it to wide? You can parse UTF-8 encoded text
as-is. After all that was the idea behind UTF-8. For example, my unit
compiler parses directly UTF-8. The advantage is that I can use the same
parser for units spelt both in pure ASCII and in full UTF-8. I simply flag
UTF-8 tokens from the table if I don't want to recognize them. There is a
trick that 8-bit tokes need to be replaced with 2-characters UTF-8
equivalents. But they are rare. BTW, the parser is table-driven, so I don't
need lex.

For UTF-8 handing in Ada you can take a look at:
http://www.dmitry-kazakov.de/ada/strings_edit.htm

It and table-driven parsers in Ada are included in components:
http://www.dmitry-kazakov.de/ada/components.htm

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Is there a lex utility for Ada that handles unicode?

[Steve]

<brian.b.mcguinness@lmco.com> wrote in message 
news:1130433435.410224.186300@g49g2000cwa.googlegroups.com...
> Is there some equivalent of the lex utility that produces
> Ada code rather than C code, and is capable of handling
> any character in the Unicode basic code plane?  I am
> thinking of using it on strings read from a GUI created
> with GtkAda, so it would probably be best if it accepted
> UTF-8 strings, but I could convert the input to a wide
> string if necessary.
>
> Thanks.
>
> --- Brian
>
I don't know if it helps, but if I were looking (based on what I have seen 
in the past) I would look at:
  Aflex
  Ayacc
  OpenToken

Source code is available.  It probably wouldn't be that much work to make a 
unicode version.

Steve
(The Duck)