Re: Data table text I/O package?

["Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de>]

On Tue, 21 Jun 2005 23:01:59 +0200, Georg Bauhaus wrote:

> Dmitry A. Kazakov wrote:
> 
> Let me first guess that many here have their largely
> regular and homogenuous data in mind. I'm not talking
> about this. We went off from what to do if you
> don't have atomic, homogenous, unambigous data, sent
> around.
> 
> 1) If you have a nice arrangement of exactly one set of
>    array-like data of guaranteed quality, there is little
>    to win by using XML.

OK, that is a big difference. Tables representing tree-like structures are
awful.

>> Sorry, but the thread's subject reads "Data table text I/O package". Text =
>> rendered data.
> 
> Notice that the thread title has I/O. I/O can mean pretty printing,
> and it can mean a reliable and robust data input-output facility,
> working well in the face of erroneous input.

But for data exchange there are better techniques than XML. Even if you
mean [far stretched] objects brokering and active agents performed over a
stream or printable characters, even then I wouldn't take XML.

>>>The accuracy is well defined and most importantly,
>>>it is up to the application, yours and mine repectively.
>  
>> This is a wrong approach of course.
> 
> There is no more accurate representation of 3.15 than the text "3.15",
> right under our noses. In a text data stream, tabular, XML, whatever.

The text "3.15" represents what? Everything of course depends on the OSI
layer we are talking about. (:-))

[...]
> The accuracy of the data may not be defined at all, IN THE DATA
> STREAM. (The again, some peoply may try, adding a schema.)

Then you cannot talk about numbers transferred. You said "3.15" is a text.
So let it be a text. "3.1 5" is also a text, as valid as "3.15" [at this
level of abstraction.]

BTW, again there are better ways to send texts than XML offers.

>> [3.1499, 3.1600]
> 
> Well, someone will ask you, 'and what exactly is 3.1499?' on
> *our* machine?

3.1499 is the lower bound. So on your machine you can represent it by any
number less or equal to 3.1499. You loose precision, but retain
correctness. The true value is always within the bounds. There is still a
problem, but a much lesser one.

> Now consider separated key=value lines. They will be longer,
> but you can scan the line looking for the key strings. A big
> step up. XML isn't worse in my view.

Unfortunately in our case it is not that simple. key=value does not help.
The problem is that data need to be sorted and filtered using various
criteria. In other words a value has more than one key. A relational DB
would probably help, but to load that amount of data would take too long.
So it ends up with a specialized tool chain, integrated diagnostic etc.

BTW, 80% of that would probably be unnecessary if Ada were used! (:-)) But
the customer wished otherwise...

> A formatted table just isn't that robust.
> Consider the case where the headline
> gets lost. The missing redundancy will leave you with a
> puzzle, not a robust set of self describing text.

It is a bad idea to correct I/O error using syntax anyway. Relevant errors
are only ones made by humans. It is very unlikely that somebody would
forget to read a table header [I don't talk about writing, because to write
in XML is beyond anybody's capability anyway.]

Humans are unbeatable in pattern recognition. This is whole idea behind
tables. Tab stops and lines are very easy patterns to detect and any error
becomes immediately visible long before inspecting the table contents.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de