From mboxrd@z Thu Jan  1 00:00:00 1970
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me
X-Spam-Level: 
X-Spam-Status: No, score=-0.9 required=5.0 tests=BAYES_00,FORGED_GMAIL_RCVD,
	FREEMAIL_FROM autolearn=no autolearn_force=no version=3.4.4
X-Google-Thread: 103376,4a07e5fd38e94bef
X-Google-NewGroupId: yes
X-Google-Attributes: gida07f3367d7,domainid0,public,usenet
X-Google-Language: ENGLISH,ASCII-7-bit
Path: 
 g2news1.google.com!news3.google.com!feeder.news-service.com!85.214.198.2.MISMATCH!eternal-september.org!feeder.eternal-september.org!.POSTED!not-for-mail
From: Natasha Kerensikova <lithiumcat@gmail.com>
Newsgroups: comp.lang.ada
Subject: Re: Tentative design of markdown parser
Date: Mon, 30 May 2011 18:07:15 +0000 (UTC)
Organization: A noiseless patient Spider
Message-ID: <slrniu7n6j.i18.lithiumcat@sigil.instinctive.eu>
References: <slrniu1to9.i18.lithiumcat@sigil.instinctive.eu>
 <4de3a128$0$7613$9b4e6d93@newsspool1.arcor-online.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Injection-Date: Mon, 30 May 2011 18:07:15 +0000 (UTC)
Injection-Info: mx04.eternal-september.org;
 posting-host="Mda950WjNwNLAFOE7yJXQw";
	logging-data="32627"; mail-complaints-to="abuse@eternal-september.org";
	posting-account="U2FsdGVkX197WfOxTL2qqh0c5qkyv8PD"
User-Agent: slrn/0.9.9p1 (FreeBSD)
Cancel-Lock: sha1:yyAzt2LFp/XfFomgQUftXsXhwsY=
Xref: g2news1.google.com comp.lang.ada:19591
Date: 2011-05-30T18:07:15+00:00
List-Id: <comp.lang.ada>

Hello,

On 2011-05-30, Georg Bauhaus <rm.dash-bauhaus@futureapps.de> wrote:
> On 28.05.11 15:22, Natasha Kerensikova wrote:
>
>> This is actually the only point I don't like about the design: there
>> should be a way to check at compile time that in a Span_Element the
>> Renderer_Callback can actually handle what is produced by the
>> Lexer_Callback. However I can't find any.
>
> When I know all the types of objects that Lexer_Callback
> can produce, I like to list them in an enumeration type,
> type Token_Kind is (A, B, C, ...).

Yes, I do like that too, and then using a variant record to store the
token-specific parameters rather than a tagged type.

However, that means knowing statically all kinds of tokens, which
prevents any extension that adds new tokens. That is how my C library
was designed, but then I found out that not all Markdown extensions fit
in the Markdown-strict set of tokens. One common example is tables. So I
ended up adding tokens for tables, which means some of the features are
inside the parser while some others are completely outside, and I really
don't like that.

So with that experience and the emphasis on extensibility from various
people in my previous thread, I was trying to design a parser that would
be extensible both by adding new tokens and new ways of rendering a set
of tokens. It does seem like a much harder problem to solve. If it turns
out impossible, or unreasonably complex, to solve, then I will fall back
on the fixed set of tokens.

Here I used a tagged type for Token only to have what is effectively a
variant record whose discriminant and variations would be extensible
(using the tag instead of a discriminant). It feels a bit like an abuse
of the tagged type system, but I haven't been able to come up with
anything better.


Thanks for your comment,
Natasha