Re: System.WCh_Cnv

["Björn Persson" <spam-away@nowhere.nil>]

Martin Krischik wrote:
> Bjï¿œrn Persson wrote:
> 
>> Martin Krischik wrote:
>>> I wonder about that. UCS character set are fixed length and UTF
>>> character sets are variable lengt. So is it rigth  to say that  UCS-4
>>> is UTF-32?
>> I believe every possible text will be encoded identically in UCS-4BE and
>> UTF-32BE, as well as in UCS-4LE and UTF-32LE. If you have a
>> counter-example then I would like to see it. What character could take
>> up more than one code unit in UTF-32?
> 
> A few years ago you could have said the same replacing all '32' with '16'.
> Many programmers relied on UTF-16 and UCS-2 being the the same. There where
> no counter-examples at the time either. But one fine day in 2001 the
> unicode authority(s) defined the 65537'th character...
> 
> I know that currently only 21 bits are actually used and the unicode
> authority(s) have given up on using more codepoints. Still I am unsure of
> just declaring them both the same.

How about a *hypothetical* counter-example? If you had a character with
the code point 100000000 hexadecimal, how would you encode it in UTF-32?
I believe it's impossible; I believe UTF-32 is a fixed-width encoding.

I found what looks like the definition of UTF-32 in
http://www.unicode.org/versions/Unicode4.0.0/ch03.pdf, page 76 (actually
page 23 in the file). It says:

"UTF-32 encoding form: The Unicode encoding form which assigns each
Unicode scalar value to a single unsigned 32-bit code unit with the same
numeric value as the Unicode scalar value."

Note "single".

Also, in Unicode Technical Report #17, at
http://www.unicode.org/reports/tr17/, UTF-32 is listed under "Examples
of fixed-width encoding forms", while UTF-16 is listed under "Examples
of variable-width encoding forms".

Of course, should the Unicode consortium make the unwise decision to
change the definition of UTF-32, then it might no longer be equivalent
to UCS-4, but then it would no longer be UTF-32. I would be a different
encoding, and would deserve a different name.

-- 
Bjï¿œrn Persson                              PGP key A88682FD
                    omb jor ers @sv ge.
                    r o.b n.p son eri nu