From mboxrd@z Thu Jan  1 00:00:00 1970
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me
X-Spam-Level: 
X-Spam-Status: No, score=-0.3 required=5.0 tests=BAYES_00,
	REPLYTO_WITHOUT_TO_CC autolearn=no autolearn_force=no version=3.4.4
X-Google-Thread: 103376,36b39757d8f8763e
X-Google-NewGroupId: yes
X-Google-Attributes: gida07f3367d7,domainid0,public,usenet
X-Google-Language: ENGLISH,ASCII-7-bit
Received: by 10.68.212.232 with SMTP id nn8mr5246194pbc.1.1323962243790;
        Thu, 15 Dec 2011 07:17:23 -0800 (PST)
Path: 
 lh20ni25668pbb.0!nntp.google.com!news1.google.com!npeer02.iad.highwinds-media.com!news.highwinds-media.com!feed-me.highwinds-media.com!nntp.club.cc.cmu.edu!feeder.erje.net!news2.arglkargh.de!news.mixmin.net!aioe.org!.POSTED!not-for-mail
From: "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de>
Newsgroups: comp.lang.ada
Subject: Re: Does OpenToken support Unicode
Date: Thu, 15 Dec 2011 16:16:58 +0100
Organization: cbb software GmbH
Message-ID: <8hyeoq2x5hph$.1bkxhpw0z9oh8$.dlg@40tude.net>
References: <2652647e-ef0a-4440-b127-4ddc59620707@4g2000yqu.googlegroups.com>
Reply-To: mailbox@dmitry-kazakov.de
NNTP-Posting-Host: FbOMkhMtVLVmu7IwBnt1tw.user.speranza.aioe.org
Mime-Version: 1.0
X-Complaints-To: abuse@aioe.org
User-Agent: 40tude_Dialog/2.0.15.1
X-Notice: Filtered by postfilter v. 0.8.2
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Date: 2011-12-15T16:16:58+01:00
List-Id: <comp.lang.ada>

On Thu, 15 Dec 2011 06:09:13 -0800 (PST), mtrenkmann wrote:

> I am a student from Bauhaus University Weimar (Germany) currently
> writing my Master thesis where I implement an ASN.1 to Ada compiler
> and runtime codec. For the parsing part I am using the OpenToken
> library. Now, as some aspects of ASN.1 deal with Unicode I want to ask
> if there is any build-in support for that or could be added by the
> user in some way?
> 
> For example, can it somehow be made possible that
> 
> procedure OpenToken.Recognizer.Analyze
>    (The_Token : in out Instance;
>     Next_Char : in       Character;
>     Verdict      : out     Analysis_Verdict) is abstract;
> 
> does support Wide_Wide_Character for the Next_Char parameter?
> 
> Thanks in advance for any advice.

I don't use OpenToken for my parsing projects, but normally encoding should
play almost no any role whatever tool you are using. Recode the input into
UTF-8, if it is not already is, and process it as if it were character
strings.

P.S. My condolences regarding ASN.1. We had a pair or two parsers/protocols
implemented, for which the documentation was in ASN.1. I still remember how
dreadful it was.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de