Re: strange behaviour of utf-8 files

[Georg Bauhaus <rm.dash-bauhaus@futureapps.de>]

On 23.11.13 05:14, Randy Brukardt wrote:
> "Shark8" <onewingedshark@gmail.com> wrote
>> Not a lot of demand for UTF-8, or not a lot of demand for Ada-2012 [from
>> the customers]?
>
> Not a lot of demand for UTF-8 or wide characters in general. As far as Ada
> 2012 goes, if I want to use a feature, it somehow gets in the compiler. :-)
> Customer demand not required (but it always helps).

Actually, programmers seem to suppress existing demand.

Equating "customers" to "consumers" of software for the moment
(who pays?), customers suffer from ASCII-fied communication in
ways that would not be accepted if written on paper.
I got a terribly malformed computer generated messages from no lesser
company than DHL (inspiring this follow up).

"??" in the mail text quoted below has obviously been put in place
of what was perfectly UTF-8 encoded character data. (In the mail's
source text, to be sure.)  The non-ASCII character is 'ü' (16#FC#)
in both cases (L.8, L.10):

+======================================================================+
Subject: Ihre Sendung wurde in eine FILIALE umgeleitet
MIME-Version: 1.0
Content-Type: text/plain; charset=ANSI_X3.4-1968
Content-Transfer-Encoding: 7bit

Guten Tag Herr Georg Bauhaus,

leider konnte Ihre Sendung  NICHT in die gew??nschte PACKSTATION eingestellt werden.

Die Sendung liegt f??r Sie in der FILIALE (...)
+======================================================================+

Ironically, the messages are produced using an industry standard Java
framework while Java's char data are not 7bit ASCII:
  
Message-ID: <...48667.JavaMail.ypqbson@HANPQ021>

These messages used to be o.K. in the past. Judging by the count of excess
spaces and long and empty lines in the message, I guess they are having some
competitive programming shop streamline their software.

Character set support can be a real issue when the use of ASCII leads
to misprints of addresses, or to ambiguity in legal documents. Consider
families
    Joseph Müller (16#FC#)
and
   Joseph Möller (16#F6#)
each owning a flat in the same house. If rendered

   Fam. Joseph M??ller
   X Str. 15
   ...

and

   Fam. Joseph M??ller
   X Str. 15
   ...

respectively, what is the postman to do?

Proper support for encoding all characters is a necessity!