Wide Character Problem in Ada.Numerics

Wide Character Problem in Ada.Numerics

[david.smith]

Hi, I'm new here and to Ada.  I'm having a problem compiling code
using greek letters as identifiers.  The problem arises when I need to
use the Ada.Numerics package.  My source code is UTF-8, and I'm
entering the characters using a greek keymap in vim.  I'm using gnat/
gcc-4.1 and the -gnatiw and -gnatW8 switches.

An example code I wrote compiles fine:
---
with Ada.Wide_Text_IO; use Ada.Wide_Text_IO;
with Ada.Float_Text_IO; use Ada.Float_Text_IO;
procedure WCTest is
   Σ : Wide_Character := 'π';
   μ : Float := 0.01;
begin
   Put(Σ);
   New_Line;
   Put(μ);
end WCTest;
---
Output:
π
 1.00000E-02
---

When I next compile my real code with greek letters *and* using the
Ada.Numerics package, I get the following error:

a-numeri.ads:25:04: illegal wide character

This is the line that has the bracket notation definition of pi in
it.

Does anyone know what is going on?  Why is there an error in a system
file?

Thanks!

Re: Wide Character Problem in Ada.Numerics

[Jean-Pierre Rosen]

Martin Krischik a écrit :
>> This is the line that has the bracket notation definition of pi in
>> it.
>>
>> Does anyone know what is going on?  Why is there an error in a system
>> file?
> 
> Yes - I saw that to - GNAT produces this error if utf-8 (-gnatW8) is 
> used for source code. Best is to file a bug report.
> 
Since, with the source model, specifications are recompiled every time 
you need them, so is Ada.Numerics. With -gnatW8, you are telling the 
compiler you are using utf-8, but the source of Ada.Numerics uses 
bracket notation - hence the error.

Not pretty elegant, I admit.

-- 
---------------------------------------------------------
            J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: Wide Character Problem in Ada.Numerics

[Georg Bauhaus]

On Tue, 2007-04-24 at 08:59 +0200, Martin Krischik wrote:
> david.smith@gmail.com schrieb:

> > When I next compile my real code with greek letters *and* using the
> > Ada.Numerics package, I get the following error:
> > 
> > a-numeri.ads:25:04: illegal wide character
> > 
> > This is the line that has the bracket notation definition of pi in
> > it.
> > 
> > Does anyone know what is going on?  Why is there an error in a system
> > file?
> 
> Yes - I saw that to - GNAT produces this error if utf-8 (-gnatW8) is 
> used for source code. Best is to file a bug report.

Wouldn't it be best in this case to do what everyone else does
and, as an option, have the compiler determine the default character
encoding using a BOM? http://unicode.org/faq/utf_bom.html#22

   Bytes Encoding Form
   00 00 FE FF  UTF-32, big-endian
   FF FE 00 00  UTF-32, little-endian
   FE FF        UTF-16, big-endian
   FF FE        UTF-16, little-endian
   EF BB BF     UTF-8

If there is no BOM, there is ambiguity anyway for compilers
with circuits for interpreting 8 bits in many ways.

Re: Wide Character Problem in Ada.Numerics

[Adam Beneschan]

On Apr 24, 1:03 am, Jean-Pierre Rosen <r...@adalog.fr> wrote:
> Martin Krischik a écrit :>> This is the line that has the bracket notation definition of pi in
> >> it.
>
> >> Does anyone know what is going on?  Why is there an error in a system
> >> file?
>
> > Yes - I saw that to - GNAT produces this error if utf-8 (-gnatW8) is
> > used for source code. Best is to file a bug report.
>
> Since, with the source model, specifications are recompiled every time
> you need them, so is Ada.Numerics. With -gnatW8, you are telling the
> compiler you are using utf-8, but the source of Ada.Numerics uses
> bracket notation - hence the error.

Wait a minute... are you saying that in GNAT, you cannot WITH a
package unless the source of the WITH'ed package uses the same
encoding as the source of the package doing the WITH'ing?  Ouch.  This
somehow seems to run counter to the whole philosophy of abstraction
that packages are supposed to provide.

                                                   -- Adam

Re: Wide Character Problem in Ada.Numerics

[Pascal Obry]

david.smith@gmail.com a écrit :
> Hi, I'm new here and to Ada.  I'm having a problem compiling code
> using greek letters as identifiers.  The problem arises when I need to
> use the Ada.Numerics package.  My source code is UTF-8, and I'm
> entering the characters using a greek keymap in vim.  I'm using gnat/
> gcc-4.1 and the -gnatiw and -gnatW8 switches.
> 
> An example code I wrote compiles fine:
> ---
> with Ada.Wide_Text_IO; use Ada.Wide_Text_IO;
> with Ada.Float_Text_IO; use Ada.Float_Text_IO;
> procedure WCTest is
>    Σ : Wide_Character := 'π';
>    μ : Float := 0.01;
> begin
>    Put(Σ);
>    New_Line;
>    Put(μ);
> end WCTest;
> ---
> Output:
> π
>  1.00000E-02
> ---
> 
> When I next compile my real code with greek letters *and* using the
> Ada.Numerics package, I get the following error:
> 
> a-numeri.ads:25:04: illegal wide character

Did you try -gnatif option ?

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|              http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: Wide Character Problem in Ada.Numerics

[Maciej Sobczak]

Adam Beneschan wrote:

> Wait a minute... are you saying that in GNAT, you cannot WITH a
> package unless the source of the WITH'ed package uses the same
> encoding as the source of the package doing the WITH'ing?  Ouch.  This
> somehow seems to run counter to the whole philosophy of abstraction
> that packages are supposed to provide.

It just shows that GNAT's 'with' is about the same as '#include' in C.

My personal opinion: allowing arbitrary encodings in source code is a 
bad idea anyway.

-- 
Maciej Sobczak : http://www.msobczak.com/
Programming    : http://www.msobczak.com/prog/

Re: Wide Character Problem in Ada.Numerics

[Jean-Pierre Rosen]

Adam Beneschan a �crit :
> Wait a minute... are you saying that in GNAT, you cannot WITH a
> package unless the source of the WITH'ed package uses the same
> encoding as the source of the package doing the WITH'ing?  Ouch.  This
> somehow seems to run counter to the whole philosophy of abstraction
> that packages are supposed to provide.
> 
That's the most likely explanation of what you describe. Of course, ACT 
people should know better. But there are so few people using 
wide_character (at least in literals), and it is likely that most of 
them use bracket notation, therefore the issue may have escaped until now...

Bug report seems in order.
-- 
---------------------------------------------------------
            J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: Wide Character Problem in Ada.Numerics

[Markus E Leypold]

Adam Beneschan <adam@irvine.com> writes:

> On Apr 24, 1:03 am, Jean-Pierre Rosen <r...@adalog.fr> wrote:
>> Martin Krischik a �crit :>> This is the line that has the bracket notation definition of pi in
>> >> it.
>>
>> >> Does anyone know what is going on?  Why is there an error in a system
>> >> file?
>>
>> > Yes - I saw that to - GNAT produces this error if utf-8 (-gnatW8) is
>> > used for source code. Best is to file a bug report.
>>
>> Since, with the source model, specifications are recompiled every time
>> you need them, so is Ada.Numerics. With -gnatW8, you are telling the
>> compiler you are using utf-8, but the source of Ada.Numerics uses
>> bracket notation - hence the error.
>
> Wait a minute... are you saying that in GNAT, you cannot WITH a
> package unless the source of the WITH'ed package uses the same
> encoding as the source of the package doing the WITH'ing?  Ouch.  This
> somehow seems to run counter to the whole philosophy of abstraction
> that packages are supposed to provide.

Wow, super. One would have to expected that the encoding problem is
already solved in the file reader and the rest of the compiler would
work on more or less the same representation anyway.

Looks like a quick and dirty hack.

Regards -- Markus

Re: Wide Character Problem in Ada.Numerics

[Brian May]

>>>>> "Jean-Pierre" == Jean-Pierre Rosen <rosen@adalog.fr> writes:

    >> Wait a minute... are you saying that in GNAT, you cannot WITH a
    >> package unless the source of the WITH'ed package uses the same
    >> encoding as the source of the package doing the WITH'ing?
    >> Ouch.  This somehow seems to run counter to the whole
    >> philosophy of abstraction that packages are supposed to
    >> provide.

How do you expect the compiler to know what encoding is used for each
source file? I think it could only know if the file was compiled
first.

    Jean-Pierre> Bug report seems in order.

Yes. I agree.
-- 
Brian May <bam@snoopy.apana.org.au>

Re: Wide Character Problem in Ada.Numerics

[David Smith]

> Did you try -gnatif option ?

That worked, but I don't understand why.

Re: Wide Character Problem in Ada.Numerics

[Adam Beneschan]

On Apr 25, 5:27 pm, Brian May <b...@snoopy.apana.org.au> wrote:
> >>>>> "Jean-Pierre" == Jean-Pierre Rosen <r...@adalog.fr> writes:
>
>     >> Wait a minute... are you saying that in GNAT, you cannot WITH a
>     >> package unless the source of the WITH'ed package uses the same
>     >> encoding as the source of the package doing the WITH'ing?
>     >> Ouch.  This somehow seems to run counter to the whole
>     >> philosophy of abstraction that packages are supposed to
>     >> provide.
>
> How do you expect the compiler to know what encoding is used for each
> source file? I think it could only know if the file was compiled
> first.

Well, the original example had a problem with a language-defined
package that was WITH'ed.  So surely that file must have been compiled
first?  By *somebody*???  I hope they're not releasing runtime
packages that they've never compiled!!!!!!

But in a more general case, the compiler, when it does compile a unit
that contains characters not in the 7-bit ASCII range, probably ought
to be generating some sort of information indicating what the source
encoding was, so that it would know how to reread the source when it's
WITH'ed.  (And, in particular, whatever file contains that information
for Ada.Numerics would be part of the GNAT release.)  And in a case
where you want to WITH a unit without compiling it first (although I
don't really understand why), it should be trivial to provide some
mechanism to generate that information without compiling.

In any case, this is not a difficult problem to solve.  And IMHO a
solution is necessary, if the alternative is to require that every
source in your environment, including sources delivered as part of the
runtime and sources that you might have downloaded from Sourceforge or
somewhere else on the web, use the same source encoding.  (Actually,
that might not even be bad if they all used UTF-8.  But requiring that
every source use some nonstandard encoding method---well, yuck.)

                                         -- Adam

Re: Wide Character Problem in Ada.Numerics

[Pascal Obry]

David Smith a �crit :
>> Did you try -gnatif option ?
> 
> That worked, but I don't understand why.
> 

GNAT User's Guide section 3.2.10 Character Set Control :

-gnati<c>

   f
       Full upper-half codes allowed in identifiers

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|              http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: Wide Character Problem in Ada.Numerics

[Georg Bauhaus]

On Thu, 2007-04-26 at 21:03 +0200, Pascal Obry wrote:

> GNAT User's Guide section 3.2.10 Character Set Control :
> 
> -gnati<c>
> 
>    f
>        Full upper-half codes allowed in identifiers

Shouldn't this be(come) an implication of -gnat05?

Re: Wide Character Problem in Ada.Numerics

[Pascal Obry]

Georg Bauhaus a �crit :
> On Thu, 2007-04-26 at 21:03 +0200, Pascal Obry wrote:
> 
>> GNAT User's Guide section 3.2.10 Character Set Control :
>>
>> -gnati<c>
>>
>>    f
>>        Full upper-half codes allowed in identifiers
> 
> Shouldn't this be(come) an implication of -gnat05?

Don't think so, this option is there since a long time...

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|              http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: Wide Character Problem in Ada.Numerics

[Brian May]

>>>>> "Adam" == Adam Beneschan <adam@irvine.com> writes:

    Adam> a case where you want to WITH a unit without compiling it
    Adam> first (although I don't really understand why), it should be
    Adam> trivial to provide some mechanism to generate that
    Adam> information without compiling.

For the general case you can have complicated dependencies, IIRC, for
example two packages depending on each other. In which case it might
not be possible to ensure each one is compiled first.
-- 
Brian May <bam@snoopy.apana.org.au>

Re: Wide Character Problem in Ada.Numerics

[Adam Beneschan]

On Apr 26, 12:03 pm, Pascal Obry <pas...@obry.net> wrote:
> David Smith a écrit :
>
> >> Did you try -gnatif option ?
>
> > That worked, but I don't understand why.
>
> GNAT User's Guide section 3.2.10 Character Set Control :
>
> -gnati<c>
>
>    f
>        Full upper-half codes allowed in identifiers

I still don't understand why this would work...  upper half of what?
I'd assume "upper-half" refers to characters in the 16#A0#..16#FF#
range, but the original poster had problems when it processed a Greek
"pi" in a runtime source, which isn't in the upper half of anything
(unless you count ISO-8859-7, the Greek alphabet!).  It's certainly
not anywhere in Latin-1 (ISO-8859-1).  And I doubt that "upper-half"
means the upper half of Unicode (16#8000#..16#FFFD#?), which I believe
would start somewhere in the middle of the Chinese character set.  So
why would -gnatif magically make pi work?  Mighty strange.........

                                       -- Adam

Re: Wide Character Problem in Ada.Numerics

[David Smith]

> I still don't understand why this would work...  upper half of what?
> I'd assume "upper-half" refers to characters in the 16#A0#..16#FF#
> range, but the original poster had problems when it processed a Greek
> "pi" in a runtime source, which isn't in the upper half of anything
> (unless you count ISO-8859-7, the Greek alphabet!).  It's certainly
> not anywhere in Latin-1 (ISO-8859-1).  And I doubt that "upper-half"
> means the upper half of Unicode (16#8000#..16#FFFD#?), which I believe
> would start somewhere in the middle of the Chinese character set.  So
> why would -gnatif magically make pi work?  Mighty strange.........

This is really what I was asking.

-Dave

Re: Wide Character Problem in Ada.Numerics

[Jean-Pierre Rosen]

Adam Beneschan a �crit :
> On Apr 25, 5:27 pm, Brian May <b...@snoopy.apana.org.au> wrote:
>>>>>>> "Jean-Pierre" == Jean-Pierre Rosen <r...@adalog.fr> writes:
>>     >> Wait a minute... are you saying that in GNAT, you cannot WITH a
>>     >> package unless the source of the WITH'ed package uses the same
>>     >> encoding as the source of the package doing the WITH'ing?
>>     >> Ouch.  This somehow seems to run counter to the whole
>>     >> philosophy of abstraction that packages are supposed to
>>     >> provide.
>>
>> How do you expect the compiler to know what encoding is used for each
>> source file? I think it could only know if the file was compiled
>> first.
> 
> Well, the original example had a problem with a language-defined
> package that was WITH'ed.  So surely that file must have been compiled
> first?  By *somebody*???  I hope they're not releasing runtime
> packages that they've never compiled!!!!!!
> 
I think you missed the point about the source model used by Gnat. A 
specification never needs to be compiled (unless it is a bodyless 
package). It is really treated like a #include: it is read everytime you 
"with" the package.

So I assume it is recompiled with the current compilation options - 
including the encoding scheme, which causes the problem.

-- 
---------------------------------------------------------
            J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: Wide Character Problem in Ada.Numerics

[Adam Beneschan]

On Apr 27, 5:08 am, Jean-Pierre Rosen <r...@adalog.fr> wrote:
> Adam Beneschan a écrit :
>
> > On Apr 25, 5:27 pm, Brian May <b...@snoopy.apana.org.au> wrote:
> >>>>>>> "Jean-Pierre" == Jean-Pierre Rosen <r...@adalog.fr> writes:
> >>     >> Wait a minute... are you saying that in GNAT, you cannot WITH a
> >>     >> package unless the source of the WITH'ed package uses the same
> >>     >> encoding as the source of the package doing the WITH'ing?
> >>     >> Ouch.  This somehow seems to run counter to the whole
> >>     >> philosophy of abstraction that packages are supposed to
> >>     >> provide.
>
> >> How do you expect the compiler to know what encoding is used for each
> >> source file? I think it could only know if the file was compiled
> >> first.
>
> > Well, the original example had a problem with a language-defined
> > package that was WITH'ed.  So surely that file must have been compiled
> > first?  By *somebody*???  I hope they're not releasing runtime
> > packages that they've never compiled!!!!!!
>
> I think you missed the point about the source model used by Gnat. A
> specification never needs to be compiled (unless it is a bodyless
> package). It is really treated like a #include: it is read everytime you
> "with" the package.

Even so, I believe that I'd want to compile a specification that I
wrote before trying to WITH it, just to make sure there are no errors.

This wouldn't apply to a specification that's part of someone else's
distribution (either the GNAT runtime, or some other Ada software
downloaded from the web).  Now, I might not feel a need to compile a
specification myself since I could trust that someone else already did
that and made it work.  But I presume that whoever wrote the software
compiled the spec themselves (or it was "compiled" when it was WITH'ed
into some other package), and at that point the compiler would have
been told what the source encoding was---and that information, about
the source encoding, should somehow be saved and included in the
distribution, so that it's available to the compiler when other users
who download the distribution WITH its packages.  At least that's the
way I'd do it.

                                        -- Adam

Re: Wide Character Problem in Ada.Numerics

[Ray Blaak]

Adam Beneschan <adam@irvine.com> writes:
> Even so, I believe that I'd want to compile a specification that I
> wrote before trying to WITH it, just to make sure there are no errors.

If I recall correctly, GNAT uses a source based model for its package library,
in that the "library interface" files for package specs are simply the sources
themselves.

This means that package specs are always loaded/compiled on the fly whenever
they are WITH'd.

This turns out to have all sorts of advantages and simplifications. I don't
know if GNAT is still doing this.

There should still be a smarter way to handle source encodings however. XML
files have a starting <?xml that can be analyzed to determine the
encoding. Perhaps GNAT could do something similar based on the first possible
tokens of a spec (e.g. --, package, with, etc.)

-- 
Cheers,                                        The Rhythm is around me,
                                               The Rhythm has control.
Ray Blaak                                      The Rhythm is inside me,
rAYblaaK@STRIPCAPStelus.net                    The Rhythm has my soul.

Re: Wide Character Problem in Ada.Numerics

[Randy Brukardt]

"Ray Blaak" <rAYblaaK@STRIPCAPStelus.net> wrote in message
news:uwszx93ce.fsf@STRIPCAPStelus.net...
...
> If I recall correctly, GNAT uses a source based model for its package
library,
> in that the "library interface" files for package specs are simply the
sources
> themselves.
>
> This means that package specs are always loaded/compiled on the fly
whenever
> they are WITH'd.
>
> This turns out to have all sorts of advantages and simplifications. I
don't
> know if GNAT is still doing this.

I wouldn't call it "advantages". It's more like trade-offs. Recompiling in
this way means you don't need to invent a compiled symboltable form. But it
does lose a lot of control over how the specs are compiled. This character
encoding issue is one of many similar issues: you lose control over how the
spec is compiled. If you want the spec to be compiled differently than the
units that use it, that becomes hard. Another disadvantage is that you end
up with two compilation modes for a spec, one that generates elaboration
code and one that does not. That's clearly going to complicate things.

I don't think the choice has much impact on compiler performance: on most
programs, GNAT and Janus/Ada compile at similar speeds. (There might have
been a difference visible years ago when computers were much slower, but it
surely isn't evident now, even on my relatively slow work machine.)

So it's just a trade-off, one of the many choices that you have to make when
designing a compiler. This is one area where I see no real issue with the
choice that we made in Janus/Ada - there are lots of things where an
alternative approach is appealing, but this isn't one of them.

                                                   Randy.