Re: Why no Ada.Wide_Directories?

[Adam Beneschan <adam@irvine.com>]

On Oct 18, 12:55 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:
> On Mon, 17 Oct 2011 18:10:35 -0700 (PDT), Adam Beneschan wrote:
> > I have a feeling you're fundamentally confused about what UTF-8 is, as
> > compared to "Latin-1".  Latin-1 is a character mapping.  It defines,
> > for all integers in the range 0..255, what character that integer
> > represents (e.g. 77 represents 'M', etc.).  Unicode is a character
> > mapping that defines characters for a much larger integer range.
>
> No, Unicode is a standard describes character mappings. Both UTF-8 and
> Latin-1 are encodings. Latin-1 as an encoding has a property that there is
> 1-1 octet to code point correspondence, at the cost that some (most) of
> code points cannot be represented by the encoding. UTF-8 lacks this
> property, but is capable to represent all code points.

Sigh... I guess you're right about the term "Latin-1".  It appears to
be *both* a character mapping *and* an encoding, based on a bit of
Wikipedia research.  The problem for me is this: what does that make
Latin-2, Latin-3, KOI8-R, etc.?  Those seem to describe the same
encoding mechanism as Latin-1 (each code represented as one 8-bit
byte), but with different meanings for the codes in the 16#A0#..16#FF#
range.  So the same encoding scheme seems to have multiple different
names.  That's very confusing to me.

I've tended to look at character-set issues as having two independent
parts: part 1 is how do we define the correspondence between integers
and the character symbols [or other "characters" with special meanings
like control characters]; and part 2 is, once we have a sequence of
integers that correspond to those characters, how do we represent that
sequence in memory, in a file, when sending bits over a wire, etc.
The two parts appear completely independent to me, which is why I get
confused when a term like "Latin-1" is used that straddles both
parts.  (Unless we decree that Unicode is the only mapping in
existence, and things like Latin-2 or KOI8-R are encodings in which
bytes in the 16#A0#..16#FF# range represent integers which are totally
different and which are defined by the Unicode standard?)

I guess I'll have to learn what people mean by their terms.  I had
some misimpressions.

And I think we could solve a lot by making String a more abstract type
defined by its operations rather than by its representation (array of
character).  For a new language, as opposed to one in which we're
trying to maintain backward compatibility with a language designed in
the 1980s, that would be a great idea.  (I *don't* think it was a good
idea to define UTF8_String as a subtype of String, and to decide that
a String could be used as a sequence of bytes that had no direct
correspondence to any characters from a character set.  That seems
like a big compromise.  On the other hand, doing it "right" would have
been a lot of work which I wouldn't have had to do, most of it
unpaid.  So I'm hesitant to complain too much.)

                            -- Adam