ANNOUNCE: DTraq First Public Release

ANNOUNCE: DTraq First Public Release

[Marc A. Criley]

McKae Technologies announces the first public release of DTraq,
version 0.960.

DTraq is a data logging and playback debugging tool providing near
realtime data logging and analysis to aid debugging and validation.
Captured, or 'tapped' data from a program can be viewed live while the
program is running or, since it is being logged to a file, played back
or printed out later for off-line review and analysis.

DTraq differs from other logging and playback tools in that no data
layout maps or byte interpretations or "data dumpers" need to be
manually created. Nor is the application responsible for converting
the raw binary data to text form before logging it.  DTraq handles all
conversion automatically by scanning the application's source code,
identifying tapped data items, and extracting the information it needs
to properly convert and display the logged items-simple scalar items
as well as arrays and records.  When the layout of data items change,
rescanning automatically picks up the changes.

The DTraq binary distribution,comprehensive documentation, and source
code is available for download at http://www.mckae.com/dtraq.html.

DTraq is currently available for Linux systems running GNAT 3.15p.

DTraq operation has been verified on both RedHat 8.0 and RedHat 9.0
Linux distributions (albeit the user needs to set an environment
variable when using the latter distribution--a README.RedHat9 file in
the distribution covers this).  Because DTraq utilizes ASIS-for-GNAT,
which is tightly bound to a corresponding compiler version (in this
case 3.15p), DTraq has currently been validated only against GNAT
3.15p.

For more information or questions, please contact marc 'at' mckae.com.

Marc A. Criley
Mckae Technologies
1 Sep 2003

Re: ANNOUNCE: DTraq First Public Release

[Pascal Obry]

"Marc A. Criley" <marc@mckae.com> writes:

> McKae Technologies announces the first public release of DTraq,
> version 0.960.
> 
> DTraq is a data logging and playback debugging tool providing near
> realtime data logging and analysis to aid debugging and validation.
> Captured, or 'tapped' data from a program can be viewed live while the
> program is running or, since it is being logged to a file, played back
> or printed out later for off-line review and analysis.

I've never used nor seen such a tool. Can you provides more information
about what it can be used for ? How can it helps debugging ?

Thanks,
Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|         http://perso.wanadoo.fr/pascal.obry
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: ANNOUNCE: DTraq First Public Release

[Florian Weimer]

Pascal Obry <p.obry@wanadoo.fr> writes:

> I've never used nor seen such a tool. Can you provides more information
> about what it can be used for ? How can it helps debugging ?

You can review the execution history that led to the misbehavior you
are concerned with.

This is particularly useful if you are trying to track down a bug
which corrupts the stack, for example.

Re: ANNOUNCE: DTraq First Public Release

[Pascal Obry]

Florian Weimer <fw@deneb.enyo.de> writes:

> You can review the execution history that led to the misbehavior you
> are concerned with.

One more question then, what is recorded into the "execution history" ?
Procedure names, parameters, start time... ?

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|         http://perso.wanadoo.fr/pascal.obry
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: ANNOUNCE: DTraq First Public Release

[Florian Weimer]

Pascal Obry <p.obry@wanadoo.fr> writes:

> Florian Weimer <fw@deneb.enyo.de> writes:
>
>> You can review the execution history that led to the misbehavior you
>> are concerned with.
>
> One more question then, what is recorded into the "execution history" ?
> Procedure names, parameters, start time... ?

There are systems which track register and memory contents.

(I haven't looked at DTraq and don't know if it goes this far.  I just
wanted to stress that such tools are useful.)

Re: ANNOUNCE: DTraq First Public Release

[Guillaume Foliard]

Pascal Obry wrote:
> "Marc A. Criley" <marc@mckae.com> writes:
>> DTraq is a data logging and playback debugging tool providing near
>> realtime data logging and analysis to aid debugging and validation.
>> Captured, or 'tapped' data from a program can be viewed live while the
>> program is running or, since it is being logged to a file, played back
>> or printed out later for off-line review and analysis.
> 
> I've never used nor seen such a tool. Can you provides more information
> about what it can be used for ? How can it helps debugging ?

I'm using such a tool at work, which has been developped in-house. It allows
you to record during execution data messages (effective parameters, data
going through a network connection, etc...) and to analyse them afterwards. 
The important part there is not to record the data but to interpret it
without having to type millions line of Put procedure by hand. This is a
two step process. First you need an Ada parser to analyse the data
structure of what you want to record and to generate all the Ada code
needed to access and display those structures. Then this code will be used
in other applications for displaying, plotting, etc...
From my own experience with such a tool, and with the help of a good
software design, more than 90 percent of the defects are resolved by just
reading the inputs and the outputs and the involded code, without any
debugger. This is indeed possible because of the natural safety of Ada.
When bug occur we can generally safely ignore all the low-level bugs C
programmers are familiar with to focus on the functionnal stuff.

By the way, http://www.mckae.com/dtraq.html is unreachable.

Re: ANNOUNCE: DTraq First Public Release

[Jeffrey Creem]

"Pascal Obry" <p.obry@wanadoo.fr> wrote in message
news:ubru3cfxa.fsf@wanadoo.fr...
>
> I've never used nor seen such a tool. Can you provides more information
> about what it can be used for ? How can it helps debugging ?
>

I have not looked a the details about what this tool provide but at a high
level I can say that this
could be really useful for "system level" debugging/analysis information.
What I mean is that in many systems, one ends
up creating some sort of data logging interface that keeps track of either
inputs to some top level module and/or
intermediate calculations to better understand how the system is operating.
Sounds like this would be a quick way
to get this information out more easily. It probably will not be as quick as
some of the hand/rolled solutions (which
dump data in a raw binary format for later offline expansion to ASCII) but
still useful in some cases.

Re: ANNOUNCE: DTraq First Public Release

[Marc A. Criley]

Jeffrey Creem wrote:
> "Pascal Obry" <p.obry@wanadoo.fr> wrote in message
> news:ubru3cfxa.fsf@wanadoo.fr...
> 
>>I've never used nor seen such a tool. Can you provides more information
>>about what it can be used for ? How can it helps debugging ?
>>
> 
> 
> I have not looked a the details about what this tool provide but at a high
> level I can say that this
> could be really useful for "system level" debugging/analysis information.
> What I mean is that in many systems, one ends
> up creating some sort of data logging interface that keeps track of either
> inputs to some top level module and/or
> intermediate calculations to better understand how the system is operating.
> Sounds like this would be a quick way
> to get this information out more easily. It probably will not be as quick as
> some of the hand/rolled solutions (which
> dump data in a raw binary format for later offline expansion to ASCII) but
> still useful in some cases.

Pretty close :-)

DTraq does log data in binary form, though, so you get the performance 
benefits of doing that.  And it also does both near simultaneous 
conversion to text for live monitoring, and post-run conversion for 
playback, printing, and analysis.  (The logging and text conversion is 
performed by a logging server, which can run on a totally different 
machine so as to minimally interfere with the system under test.  Data 
is transferred from app to server via TCP/IP.)

Marc A. Criley
McKae Technologies

Re: ANNOUNCE: DTraq First Public Release

[Jeffrey Creem]

"Marc A. Criley" <marc@mckae.com> wrote in message
news:q_a5b.17012$EW1.1279@newsread1.news.atl.earthlink.net...
> Jeffrey Creem wrote:
> > "Pascal Obry" <p.obry@wanadoo.fr> wrote in message
> > news:ubru3cfxa.fsf@wanadoo.fr...
> Pretty close :-)
>
> DTraq does log data in binary form, though, so you get the performance
> benefits of doing that.  And it also does both near simultaneous
> conversion to text for live monitoring, and post-run conversion for
> playback, printing, and analysis.  (The logging and text conversion is
> performed by a logging server, which can run on a totally different
> machine so as to minimally interfere with the system under test.  Data
> is transferred from app to server via TCP/IP.)
>

Cool.. It makes it even more exciting than I had originally thought. (And I
was
already pretty excited about it!)

Re: ANNOUNCE: DTraq First Public Release

[Marc A. Criley]

Pascal Obry wrote:
> 
> I've never used nor seen such a tool. Can you provides more information
> about what it can be used for ? How can it helps debugging ?

Many projects of any significant size have some sort of "logger", a
package or class or file of logging functions.  As the program runs,
information that the developer has decided would be useful to record
for subsequent analysis or monitoring is logged.  (Actually, specific
logging requirements may also be placed on an application by its
customer, especially for military systems.)

So such loggers basically record to one or more log files the
information that is submitted to the logging service while the app
runs, along with a timestamp and perhaps other useful information.

There are two general approaches regarding the form of the logged
data: One is that the application preconverts the data to text and
then logs that text, so the application has the responsibility of
properly interpreting and formatting the data that is then logged.

The second approach is to more-or-less provide an address and a number
of bytes to the logger, and the logger then writes those raw bytes to
the log file.  This address/size combination can sometimes be dressed
up as somehow simply specifying the object to log, or utilizing a
suitably instantiated generic (which is what DTraq does).

The problem with the first approach is that text representations
usually take up more bytes than the raw data, which means added I/O or
network traffic, plus the text conversion impacts application
performance.

The problem with the second approach is that the raw bytes still have
to be converted to text at some point.  So there are a couple typical
ways of doing that: defining layout templates that are overlayed on
the data to break them up and that describe how to interpret each
piece--scalar, record component, array element, string, integer,
float, character, etc.  Another is to create a "data dumper" that
programmatically converts a data stream into text, which is then
invoked by the logging system's "viewer".  And of course whenever a
data item's format changes, the corresponding template or dumper needs
to be updated, and when there's a need to log a new type of data, a
new template/dumper has to be created.

For the basic logging part, DTraq is like most other loggers--you
identify the data item you want to log, instantiate a generic
procedure with the type of that data item, and then add invocations of
that instantiation wherever you want to "tap" the value of that item.

Something like:

    type Color_Type is (Red, Green, Blue);

    Current_Color : Color_Type := Red;

    procedure Tap_Color is new DTraq.Tap
            (Color_Type, 121, "Color_Type");
    -- Tap's parameters are the type to tap, a numeric ID you pick
    -- to associate with it, and an optional string that will be
    -- later conveyed to the viewer.

Then wherever you want to record the current value insert:

    Tap_Color(Current_Color);

At this point the current value of Current_Color goes out to the log
file.

Now what you can do with DTraq is monitor your taps live while the
program is running, and verify that they're taking on the values you
expected as the program is put through its paces.

After you've done your run, you can then reload a log file into DTraq
and have it play it back for you, single or multiple step through the
logged data, forward or backward, reviewing how the key values changed
as the program executed.  And of course you can print out all or
portions of the logged data.


The key area where DTraq differs from other loggers is in the
conversion from raw bytes to text.  DTraq scans your Ada source code,
and with the aid of the Ada Semantic Information Specification (ASIS)
automatically locates the "Tap" invocations, identifies each type being
tapped, and analyzes it (including any components it may have) to
understand how to convert instances of the data type into
corresponding text, whether that item is a number, a character,
string, enumeration, record, or array.

This is all automatically done by running the "mkdtq" component of
DTraq over your code base--you NEVER have to manually convert, or
describe how to convert, raw data to text.  When a data item changes
or a new one is added, another invocation of mkdtq automatically picks
up the changes.

Makes things _much_ less tedious :-)

Hope this helps, there's detailed info in the DTraq user manual, and a
fully working example accompanies the distribution and is gone over
step by step in the manual.

Marc
marc 'at' mckae.com

Re: ANNOUNCE: DTraq First Public Release

[Pascal Obry]

"Marc A. Criley" <marc@mckae.com> writes:

[long message removed]

Thanks to you and all people who have replied to help me understand what is
DTraq about.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|         http://perso.wanadoo.fr/pascal.obry
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595