Re: Interpretation of extensions different from Unix/Linux?

["Randy Brukardt" <randy@rrsoftware.com>]

"Adam Beneschan" <adam@irvine.com> wrote in message 
news:6f80c882-fa03-4ca9-a53e-fae34cea160d@b15g2000yqd.googlegroups.com...
On Aug 17, 3:28 pm, "Randy Brukardt" <ra...@rrsoftware.com> wrote:

>> The problem here is that String really is not the right type, but since 
>> you
>> can't have string literals for private types in Ada, you can't make it a
>> private type. (And if you could have string literals, it still couldn't 
>> be
>> used with the existing I/O packages, it would be way too incompatible.)
>
>That wouldn't even be an issue if UTF-8 were strictly a "storage
>format" as you called it above.  If that were the case, you wouldn't
>need string literals for it.  I think the problem is that UTF-8 is
>something of a hybrid.  If all characters in the string are in the
>32..126 range, the "sequence of octets" stored in the UTF-8 string is
>identical to the graphic characters stored in a String.  (UTF-8 was
>designed purposefully so that would happen.)  In cases like that, it
>makes sense to use a string literal.

Well, the problem here is that it *always* makes sense to use a string 
literal. That's how you specify what you want in storage in Ada.

I think Dmitry's point is that he'd rather always see explicit conversions. 
The problem is that they don't work well -- exhibit A is unbounded strings. 
That's especially true for the use-adverse like me. I hate having to write:

    A_Str := Ada.Strings.Unbounded.To_Unbounded_String ("ABC");

and surely UTF8 would be worse:

   A_Str := Ada.Strings.Unbounded_UTF_8.To_Unbounded_UTF_8_String ("ABC");

..
>Also, I'm afraid that using String can backfire.  If I understand it
>correctly, the decision was that the Name parameter of Text_IO.Open
>should be interpreted as a UTF-8 octet sequence even though it's a
>String.  But the intent is to allow string literals.  At some point,
>though, some poor innocent programmer in Germany or Spain is going to
>try to use a string literal (or a Latin-1 string variable) with an
>umlaut or an accented vowel in it and get totally screwed up since
>those characters don't represent themselves in UTF-8 encoding, and
>they'll end up puzzling over how their program created a file with a
>Chinese character in the middle of the name.  (Yeah, I know, that's
>very unlikely; most likely the UTF-8 encoding will simply be invalid.)

I've been presuming that UTF-8 encoding started with a BOM or something like 
that, else you couldn't tell it from regular Latin-1 encoding. It would be 
hard to insert a BOM into a string literal by accident!

But I do agree that this issue needs some discussion.

(Also note that a major reason for this package is to make ASIS work; there 
[as with I/O], we're stuck with existing routines that return Wide_Strings 
that are not enough to handle all possible text.)

                                        Randy.

                                              -- Adam