comp.lang.ada
 help / color / mirror / Atom feed
* Efficiently setting up large quantities of constant data
@ 2004-12-13 21:48 Michael Mounteney
  2004-12-13 22:09 ` Stephen Leake
                   ` (6 more replies)
  0 siblings, 7 replies; 25+ messages in thread
From: Michael Mounteney @ 2004-12-13 21:48 UTC (permalink / raw)


I want to set up about 100 variables which somehow reference
variable-length arrays of other things.  The other things, for a given
variable, are fixed.  Simplifying:

     type component is
          record
               X : character;
          end record;

     type component_list is array (positive range <>) of component;

     type component_ref is access all component_list;

     type structure is
          record
               part : component_ref;
          end record;

So I will have lots of structures scattered about, which will be
completely constant.  I would like to enforce this constancy, but Ada
(as far as I know) does not (unlike C++;  that should get some hackles
rising) allow `constant' to be splashed about in type definitions. 
This is on GNAT, which does not support pragma read_only.

Is there any way to ensure that these data are not modified, other
than by forbidding direct access to structure.part etc?



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-13 21:48 Efficiently setting up large quantities of constant data Michael Mounteney
@ 2004-12-13 22:09 ` Stephen Leake
  2004-12-13 22:15 ` Luke A. Guest
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 25+ messages in thread
From: Stephen Leake @ 2004-12-13 22:09 UTC (permalink / raw)
  To: comp.lang.ada

gate02@wrandyke.demon.co.uk (Michael Mounteney) writes:

> I want to set up about 100 variables which somehow reference
> variable-length arrays of other things.  The other things, for a given
> variable, are fixed.  Simplifying:
> 
>      type component is
>           record
>                X : character;
>           end record;
> 
>      type component_list is array (positive range <>) of component;
> 
>      type component_ref is access all component_list;
> 
>      type structure is
>           record
>                part : component_ref;
>           end record;
> 
> So I will have lots of structures scattered about, which will be
> completely constant.  I would like to enforce this constancy, but Ada
> (as far as I know) does not (unlike C++;  that should get some hackles
> rising) allow `constant' to be splashed about in type definitions. 

Yes it does; any object, and any pointer to object, can be declared
constant. 

Please show an actual object that you attempted to declare as
constant, but where unable to.

-- 
-- Stephe




^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-13 21:48 Efficiently setting up large quantities of constant data Michael Mounteney
  2004-12-13 22:09 ` Stephen Leake
@ 2004-12-13 22:15 ` Luke A. Guest
  2004-12-14  0:20 ` Jeffrey Carter
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 25+ messages in thread
From: Luke A. Guest @ 2004-12-13 22:15 UTC (permalink / raw)


On Mon, 13 Dec 2004 13:48:49 -0800, Michael Mounteney wrote:

>      type structure is
>           record
>                part : component_ref;
>           end record;
> 
> So I will have lots of structures scattered about, which will be
> completely constant.  I would like to enforce this constancy, but Ada

Well, firstly, it seems like overkill to have a record which has only a
pointer to an array. Surely, just creating your variables *as* instances
of these arrays would be better?

> (as far as I know) does not (unlike C++;  that should get some hackles
> rising) allow `constant' to be splashed about in type definitions. This
> is on GNAT, which does not support pragma read_only.

By doing the above, you can of course do the following:

part : constant component_list := (...);  -- Initialise me here!!
 
> Is there any way to ensure that these data are not modified, other than
> by forbidding direct access to structure.part etc?

Do this with a package that *has memory* and it should work nicely. Also,
make it private if you want and add an accessor that can only read from
the arrays.

Luke.




^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-13 21:48 Efficiently setting up large quantities of constant data Michael Mounteney
  2004-12-13 22:09 ` Stephen Leake
  2004-12-13 22:15 ` Luke A. Guest
@ 2004-12-14  0:20 ` Jeffrey Carter
  2004-12-14  8:43 ` Martin Krischik
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 25+ messages in thread
From: Jeffrey Carter @ 2004-12-14  0:20 UTC (permalink / raw)


Michael Mounteney wrote:

>      type component is
>           record
>                X : character;
>           end record;
> 
>      type component_list is array (positive range <>) of component;
> 
>      type component_ref is access all component_list;

type Component_Ref is access constant Component_List;

> 
>      type structure is
>           record
>                part : component_ref;
>           end record;
> 
> So I will have lots of structures scattered about, which will be
> completely constant.  I would like to enforce this constancy, but Ada
> (as far as I know) does not (unlike C++;  that should get some hackles
> rising) allow `constant' to be splashed about in type definitions. 
> This is on GNAT, which does not support pragma read_only.

It might raise some hackles if it were true.

It sounds, though, as though you can simply write

A_Constant : constant Component_List := (...);

to declare your constants.

-- 
Jeff Carter
"Sir Lancelot saves Sir Gallahad from almost certain temptation."
Monty Python & the Holy Grail
69



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-13 21:48 Efficiently setting up large quantities of constant data Michael Mounteney
                   ` (2 preceding siblings ...)
  2004-12-14  0:20 ` Jeffrey Carter
@ 2004-12-14  8:43 ` Martin Krischik
  2004-12-14 12:18 ` Simon Wright
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 25+ messages in thread
From: Martin Krischik @ 2004-12-14  8:43 UTC (permalink / raw)


Michael Mounteney wrote:

> I want to set up about 100 variables which somehow reference
> variable-length arrays of other things.  The other things, for a given
> variable, are fixed.  Simplifying:
> 
>      type component is
>           record
>                X : character;
>           end record;

Since you come from C: Ada "subtype" is C "typedef" - type is a different
concept. You only need:

type component is new Character;

if you need Character type without implicit convertions.

>      type component_list is array (positive range <>) of component;

>      type component_ref is access all component_list;

Even repeating myself: don't use "access all" until you need it. "access
all" might need additional runtime checks to be performed. The GNAT
compiler will actualy isue an error message telling you when "access all"
is needed.

>      type structure is
>           record
>                part : component_ref;
>           end record;

You could join that:

type component_list is array (positive range <>) of component;

type structure  (size : positive)  is
   record
      part : component_list (1 .. size);
   end record;

Save you the use of heap memory.

> So I will have lots of structures scattered about, which will be
> completely constant.  I would like to enforce this constancy, but Ada
> (as far as I know) does not (unlike C++;  that should get some hackles
> rising) allow `constant' to be splashed about in type definitions.
> This is on GNAT, which does not support pragma read_only.

Somehow true. But almost any named type can be made constant.

See http://en.wikibooks.org/wiki/Programming:Ada:Subtypes#named_subtype for
what a named type is.

> Is there any way to ensure that these data are not modified, other
> than by forbidding direct access to structure.part etc?

GNAT has an interesting optimization:

type String_Contant is access constant String;
My_Constant : contant String_Contant := new String'("Some Test");

will not call malloc (GNAT uses C malloc and free internaly) but uses static
memory instead. Since this optimization is suggested by the RM other
compilers are likely to behave the same.

Martin

-- 
mailto://krischik@users.sourceforge.net
http://www.ada.krischik.com



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-13 21:48 Efficiently setting up large quantities of constant data Michael Mounteney
                   ` (3 preceding siblings ...)
  2004-12-14  8:43 ` Martin Krischik
@ 2004-12-14 12:18 ` Simon Wright
  2004-12-15 21:10   ` Brian May
  2004-12-16 12:36 ` Dr. Adrian Wrigley
  2004-12-16 18:29 ` Alex R. Mosteo
  6 siblings, 1 reply; 25+ messages in thread
From: Simon Wright @ 2004-12-14 12:18 UTC (permalink / raw)



In a spec ...

   type String_P is access constant String;

   type Stream_Element_Array_P is
     access constant Ada.Streams.Stream_Element_Array;

   type Format is
     (HTML,
      Plain,
      JPEG,
      GIF,
      PNG,
      OCTET_STREAM);

   type URL_Info is record
      URL : String_P;
      Doc : Stream_Element_Array_P;
      Form : Format;
   end record;

   type URL_Info_Array is array (Positive range <>) of URL_Info;
   type URL_Info_Array_P is access constant URL_Info_Array;

in a body ...

   Url_1 : aliased constant String := "/index.html";
   Doc_1 : aliased constant Stream_Element_Array :=
     (
      60, 33, 45, 45, 32, 36, 73, 100, 58, 32, 105, 110,
      100, 101, 120, 46, 104, 116, 109, 108, 44, 118, 32, 49,
.......
      108, 62
     );
   Documents : aliased constant Url_Info_Array :=
     (
      1 => (Url => Url_1'Access, Doc => Doc_1'Access, Form => HTML)
     );
   function Static_Urls return Url_Info_Array_P is
   begin
      return Documents'Access;
   end Static_Urls;


(on looking at this, I'm a bit surprised that I didn't need to say
'all' ..)

-- 
Simon Wright                               100% Ada, no bugs.



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-14 12:18 ` Simon Wright
@ 2004-12-15 21:10   ` Brian May
  2004-12-16 23:18     ` Nick Roberts
  0 siblings, 1 reply; 25+ messages in thread
From: Brian May @ 2004-12-15 21:10 UTC (permalink / raw)


>>>>> "Simon" == Simon Wright <simon@pushface.org> writes:

    Simon> (on looking at this, I'm a bit surprised that I didn't need
    Simon> to say 'all' ..)

What compiler? Could this be a compiler bug?

I thought the whole idea of requiring "all" is that it wouldn't work
in the above case because the memory wasn't dynamically allocated.

Unless I am somehow mistaken...
-- 
Brian May <bam@snoopy.apana.org.au>



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-13 21:48 Efficiently setting up large quantities of constant data Michael Mounteney
                   ` (4 preceding siblings ...)
  2004-12-14 12:18 ` Simon Wright
@ 2004-12-16 12:36 ` Dr. Adrian Wrigley
  2004-12-16 13:50   ` Marc A. Criley
                     ` (3 more replies)
  2004-12-16 18:29 ` Alex R. Mosteo
  6 siblings, 4 replies; 25+ messages in thread
From: Dr. Adrian Wrigley @ 2004-12-16 12:36 UTC (permalink / raw)


A similar question I have is how to link in regular files and access
their contents in Ada.

For example, suppose I have a binary file "distances.dat" containing
the (road) distances between 1000 different cities.  This could be a 2MB
file computed by another application. How can I build a single executable
file containing the data and access it from the Ada code?

I presume this needs to be done at the linker stage, but I don't
see any options to link in non-object files.  The obvious alternative
of distributing the data files separately and reading them in
is slower and much less tidy than having a single self-contained binary.

(I tried Google, but probably didn't use the right search terms...)
-- 
Adrian





^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 12:36 ` Dr. Adrian Wrigley
@ 2004-12-16 13:50   ` Marc A. Criley
  2004-12-17  2:32     ` John B. Matthews
  2004-12-16 14:06   ` rien
                     ` (2 subsequent siblings)
  3 siblings, 1 reply; 25+ messages in thread
From: Marc A. Criley @ 2004-12-16 13:50 UTC (permalink / raw)


"Dr. Adrian Wrigley" <amtw@linuxchip.demon.co.uk.uk.uk> wrote:

> A similar question I have is how to link in regular files and access
> their contents in Ada.
>
> For example, suppose I have a binary file "distances.dat" containing
> the (road) distances between 1000 different cities.  This could be a 2MB
> file computed by another application. How can I build a single executable
> file containing the data and access it from the Ada code?
>
> I presume this needs to be done at the linker stage, but I don't
> see any options to link in non-object files.  The obvious alternative
> of distributing the data files separately and reading them in
> is slower and much less tidy than having a single self-contained binary.

I don't know the answer to your real question about linking in such files,
but the way we dealt with this problem _many_ years ago with some very large
tables of math constants was to write a little tool that read in the file
and then generated Ada source code that initialized a suitably defined data
structure with the constants.  (And I don't think we used big aggregates,
because they were...well...too big.  I think we had the tool generate
thousands of assignment statements.)

The generated file was quite large and took awhile to compile (it was 1985
after all :-), but it worked just fine for us.  Of course it was a one-shot
effort as well, since math constants don't change very often, so YMMV if
you're dealing with changing data.

Marc A. Criley
McKae Technologies
www.mckae.com





^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 12:36 ` Dr. Adrian Wrigley
  2004-12-16 13:50   ` Marc A. Criley
@ 2004-12-16 14:06   ` rien
  2004-12-16 14:24   ` Vinzent 'Gadget' Hoefler
  2004-12-16 17:11   ` Andre
  3 siblings, 0 replies; 25+ messages in thread
From: rien @ 2004-12-16 14:06 UTC (permalink / raw)


Dr. Adrian Wrigley wrote:
> A similar question I have is how to link in regular files and access
> their contents in Ada.
>
> For example, suppose I have a binary file "distances.dat" containing
> the (road) distances between 1000 different cities.  This could be a
2MB
> file computed by another application. How can I build a single
executable
> file containing the data and access it from the Ada code?
>
> I presume this needs to be done at the linker stage, but I don't
> see any options to link in non-object files.  The obvious alternative
> of distributing the data files separately and reading them in
> is slower and much less tidy than having a single self-contained
binary.

i did that once with a C++ application and an XML resource file.

i wrote some XSL files which converted the XML resource to C++ code,
and this code was linked into the application. the transformation was
applied at build time using Xalan (which distributes with a pretty
small executable for doing this) and triggered from the makefile.
-- 
rien




^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 12:36 ` Dr. Adrian Wrigley
  2004-12-16 13:50   ` Marc A. Criley
  2004-12-16 14:06   ` rien
@ 2004-12-16 14:24   ` Vinzent 'Gadget' Hoefler
  2004-12-16 18:12     ` Dr. Adrian Wrigley
                       ` (2 more replies)
  2004-12-16 17:11   ` Andre
  3 siblings, 3 replies; 25+ messages in thread
From: Vinzent 'Gadget' Hoefler @ 2004-12-16 14:24 UTC (permalink / raw)


Dr. Adrian Wrigley wrote:

> I presume this needs to be done at the linker stage, but I don't
> see any options to link in non-object files.

You could give "objcopy" a try. It can convert a binary file into an
object file which could then be linked normally.

See the man page for the --binary-architecture option. You just should
be _very_ sure about what the binary file contains and how to map it
onto the appropriate Ada-type(s).


Vinzent.



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 12:36 ` Dr. Adrian Wrigley
                     ` (2 preceding siblings ...)
  2004-12-16 14:24   ` Vinzent 'Gadget' Hoefler
@ 2004-12-16 17:11   ` Andre
  2004-12-16 21:52     ` Larry Kilgallen
  3 siblings, 1 reply; 25+ messages in thread
From: Andre @ 2004-12-16 17:11 UTC (permalink / raw)


Dr. Adrian Wrigley wrote:
> A similar question I have is how to link in regular files and access
> their contents in Ada.
> 
> For example, suppose I have a binary file "distances.dat" containing
> the (road) distances between 1000 different cities.  This could be a 2MB
> file computed by another application. How can I build a single executable
> file containing the data and access it from the Ada code?
> 
> I presume this needs to be done at the linker stage, but I don't
> see any options to link in non-object files.  The obvious alternative
> of distributing the data files separately and reading them in
> is slower and much less tidy than having a single self-contained binary.
> 
> (I tried Google, but probably didn't use the right search terms...)

If you are writting for the Windows platform you have two options:

1. Put it in a resource file and use a resource compiler to add it to 
your executable. With WIN32 the functions Find_Resource and 
Lock_Resource you can get a pointer to the data.

2. Use a memory mapped file. This way you also get a pointer to the data 
and Windows is taking care of loading it into the memory. Data reading 
using a memory mapped file is just as fast as putting it in the executable.

Andr�


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 14:24   ` Vinzent 'Gadget' Hoefler
@ 2004-12-16 18:12     ` Dr. Adrian Wrigley
  2004-12-16 23:25       ` Nick Roberts
  2004-12-17  5:20     ` tmoran
  2004-12-17 13:53     ` Dr. Adrian Wrigley
  2 siblings, 1 reply; 25+ messages in thread
From: Dr. Adrian Wrigley @ 2004-12-16 18:12 UTC (permalink / raw)


On Thu, 16 Dec 2004 14:24:01 +0000, Vinzent 'Gadget' Hoefler wrote:

> Dr. Adrian Wrigley wrote:
> 
>> I presume this needs to be done at the linker stage, but I don't
>> see any options to link in non-object files.
> 
> You could give "objcopy" a try. It can convert a binary file into an
> object file which could then be linked normally.

This is exactly what I've been looking for!  Thanks!

It gives the necessary symbols to access the data from Ada with
a pragma import, AFAICT.  It looks like the equivalent on Windows
is to use a Resource Compiler.  Getting a "constant" view of the
data from Ada should be simple (but I haven't tested it).

(extract of objcopy man page on Linux):

--binary-architecture=bfdarch
  Useful when transforming a raw binary input file into an object file.
  In  this  case  the  output architecture  can  be  set  to  bfdarch. 
  This option will be ignored if the input file has a known bfdarch. You can
  access this binary data inside a program by referencing the special
  symbols  that are   created   by  the  conversion  process. These  symbols
  are  called  _binary_objfile_start, _binary_objfile_end and
  _binary_objfile_size.  e.g. you can transform a picture file into an
  object file and then access it in your code using these symbols.
-- 
Adrian




^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-13 21:48 Efficiently setting up large quantities of constant data Michael Mounteney
                   ` (5 preceding siblings ...)
  2004-12-16 12:36 ` Dr. Adrian Wrigley
@ 2004-12-16 18:29 ` Alex R. Mosteo
  6 siblings, 0 replies; 25+ messages in thread
From: Alex R. Mosteo @ 2004-12-16 18:29 UTC (permalink / raw)


Michael Mounteney wrote:
> I want to set up about 100 variables which somehow reference
> variable-length arrays of other things.  The other things, for a given
> variable, are fixed.  Simplifying:
> 
>      type component is
>           record
>                X : character;
>           end record;
> 
>      type component_list is array (positive range <>) of component;
> 
>      type component_ref is access all component_list;
> 
>      type structure is
>           record
>                part : component_ref;
>           end record;
> 
> So I will have lots of structures scattered about, which will be
> completely constant.  I would like to enforce this constancy, but Ada
> (as far as I know) does not (unlike C++;  that should get some hackles
> rising) allow `constant' to be splashed about in type definitions. 
> This is on GNAT, which does not support pragma read_only.
> 
> Is there any way to ensure that these data are not modified, other
> than by forbidding direct access to structure.part etc?

Reading the other posts I think you may be interested in the resource 
management of AWS [Ada Web Server].



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 17:11   ` Andre
@ 2004-12-16 21:52     ` Larry Kilgallen
  2004-12-16 23:23       ` Nick Roberts
  2004-12-16 23:47       ` Dr. Adrian Wrigley
  0 siblings, 2 replies; 25+ messages in thread
From: Larry Kilgallen @ 2004-12-16 21:52 UTC (permalink / raw)


In article <cpsfjh$fg3$1@news4.zwoll1.ov.home.nl>, Andre <avsaway@hotmail.com> writes:
> Dr. Adrian Wrigley wrote:

>> For example, suppose I have a binary file "distances.dat" containing
>> the (road) distances between 1000 different cities.  This could be a 2MB
>> file computed by another application. How can I build a single executable
>> file containing the data and access it from the Ada code?

> 2. Use a memory mapped file. This way you also get a pointer to the data 
> and Windows is taking care of loading it into the memory. Data reading 
> using a memory mapped file is just as fast as putting it in the executable.

But since he did not say the purpose of putting it in the application
was for speed, I think we have to presume it was for religious reasons.



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-15 21:10   ` Brian May
@ 2004-12-16 23:18     ` Nick Roberts
  0 siblings, 0 replies; 25+ messages in thread
From: Nick Roberts @ 2004-12-16 23:18 UTC (permalink / raw)


Brian May wrote:

>>>>>>"Simon" == Simon Wright <simon@pushface.org> writes:
> 
> 
>     Simon> (on looking at this, I'm a bit surprised that I didn't need
>     Simon> to say 'all' ..)
> 
> What compiler? Could this be a compiler bug?
> 
> I thought the whole idea of requiring "all" is that it wouldn't work
> in the above case because the memory wasn't dynamically allocated.
> 
> Unless I am somehow mistaken...

Well, yes, in that the use of 'constant' in 'type ... is access
constant ...' implies 'all' (as if you had said 'type ... is access
all constant ...').

-- 
Nick Roberts



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 21:52     ` Larry Kilgallen
@ 2004-12-16 23:23       ` Nick Roberts
  2004-12-16 23:47       ` Dr. Adrian Wrigley
  1 sibling, 0 replies; 25+ messages in thread
From: Nick Roberts @ 2004-12-16 23:23 UTC (permalink / raw)


Larry Kilgallen wrote:

> But since he did not say the purpose of putting it in the application
> was for speed, I think we have to presume it was for religious reasons.

The Holy Church of the Single Almighty Executable? Excellent!

"There is but one executable, and PE is its format." Hehe.

-- 
Nick Roberts



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 18:12     ` Dr. Adrian Wrigley
@ 2004-12-16 23:25       ` Nick Roberts
  0 siblings, 0 replies; 25+ messages in thread
From: Nick Roberts @ 2004-12-16 23:25 UTC (permalink / raw)


Dr. Adrian Wrigley wrote:

>>You could give "objcopy" a try. It can convert a binary file into an
>>object file which could then be linked normally.
> 
> This is exactly what I've been looking for!  Thanks!

I was going to say, this is a capability that compilers /should/
support (one way or another).

-- 
Nick Roberts



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 21:52     ` Larry Kilgallen
  2004-12-16 23:23       ` Nick Roberts
@ 2004-12-16 23:47       ` Dr. Adrian Wrigley
  2004-12-19 14:34         ` Simon Wright
  1 sibling, 1 reply; 25+ messages in thread
From: Dr. Adrian Wrigley @ 2004-12-16 23:47 UTC (permalink / raw)


On Thu, 16 Dec 2004 15:52:28 -0600, Larry Kilgallen wrote:

> In article <cpsfjh$fg3$1@news4.zwoll1.ov.home.nl>, Andre <avsaway@hotmail.com> writes:
>> Dr. Adrian Wrigley wrote:
> 
>>> For example, suppose I have a binary file "distances.dat" containing
>>> the (road) distances between 1000 different cities.  This could be a 2MB
>>> file computed by another application. How can I build a single executable
>>> file containing the data and access it from the Ada code?
> 
>> 2. Use a memory mapped file. This way you also get a pointer to the data 
>> and Windows is taking care of loading it into the memory. Data reading 
>> using a memory mapped file is just as fast as putting it in the executable.
> 
> But since he did not say the purpose of putting it in the application
> was for speed, I think we have to presume it was for religious reasons.

It seems like a good idea in some circumstances:

Atomic upgrade - replace the binary in one step may be useful,
particularly in a real-time system (think cron job running single
executable avoids possible inconsistency of old data files during upgrade)

obscures internal structures a bit - makes it a bit harder for users to
mess around with/view internal structure (eg substituting their own data
files)

simpler distribution - can put executable in /usr/bin without messy
directory of data files to go somewhere.  No need for .tar.gz archive.
Also simpler to remove and to verify.

I like the objcopy route over generating and compiling Ada source
(particularly for large data), since it avoids potential compiler limits,
and doesn't risk unexpected efficiency problems (stack space/elaboration
time/code space, compilation time etc.).

but mostly it was religious, yes...
-- 
Adrian




^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 13:50   ` Marc A. Criley
@ 2004-12-17  2:32     ` John B. Matthews
  0 siblings, 0 replies; 25+ messages in thread
From: John B. Matthews @ 2004-12-17  2:32 UTC (permalink / raw)


In article <32di5rF3lj4ggU1@individual.net>,
 "Marc A. Criley" <mcNOSPAM@mckae.com> wrote:

> "Dr. Adrian Wrigley" <amtw@linuxchip.demon.co.uk.uk.uk> wrote:
> 
> > A similar question I have is how to link in regular files and access
> > their contents in Ada.
> >
> > For example, suppose I have a binary file "distances.dat" containing
> > the (road) distances between 1000 different cities.  This could be a 2MB
> > file computed by another application. How can I build a single executable
> > file containing the data and access it from the Ada code?
> >
> > I presume this needs to be done at the linker stage, but I don't
> > see any options to link in non-object files.  The obvious alternative
> > of distributing the data files separately and reading them in
> > is slower and much less tidy than having a single self-contained binary.
> 
> I don't know the answer to your real question about linking in such files,
> but the way we dealt with this problem _many_ years ago with some very large
> tables of math constants was to write a little tool that read in the file
> and then generated Ada source code that initialized a suitably defined data
> structure with the constants.  (And I don't think we used big aggregates,
> because they were...well...too big.  I think we had the tool generate
> thousands of assignment statements.)
> 
> The generated file was quite large and took awhile to compile (it was 1985
> after all :-), but it worked just fine for us.  Of course it was a one-shot
> effort as well, since math constants don't change very often, so YMMV if
> you're dealing with changing data.
> 
> Marc A. Criley
> McKae Technologies
> www.mckae.com

For several projects, we've used a script (SQL*Plus, PL/SQL, Java) 
to read from a database and generate executable code (Ada, Java). 
The script was part of the build, so each new release had the latest 
data.

-- 
John
jmatthews at wright dot edu
www dot wright dot edu/~john.matthews/



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 14:24   ` Vinzent 'Gadget' Hoefler
  2004-12-16 18:12     ` Dr. Adrian Wrigley
@ 2004-12-17  5:20     ` tmoran
  2004-12-17  8:38       ` Vinzent 'Gadget' Hoefler
  2004-12-17 13:53     ` Dr. Adrian Wrigley
  2 siblings, 1 reply; 25+ messages in thread
From: tmoran @ 2004-12-17  5:20 UTC (permalink / raw)


>> I presume this needs to be done at the linker stage, but I don't
>> see any options to link in non-object files.
>
>You could give "objcopy" a try. It can convert a binary file into an
>object file which could then be linked normally.
  You could also create an asm (remember that?) file with the data
and link the resultant object file.  In either case, you can use
MS Link to combine the obj files into an exe for Windows.

>See the man page for the --binary-architecture option. You just should
>be _very_ sure about what the binary file contains and how to map it
>onto the appropriate Ada-type(s).
  What's wrong with just creating an Ada source file with all the constant
data?  That would give you the advantage of the compiler doing the
worrying about the data representation and layout and doing some checking
that you are accessing things correctly.



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-17  5:20     ` tmoran
@ 2004-12-17  8:38       ` Vinzent 'Gadget' Hoefler
  0 siblings, 0 replies; 25+ messages in thread
From: Vinzent 'Gadget' Hoefler @ 2004-12-17  8:38 UTC (permalink / raw)


tmoran@acm.org wrote:

>>You could give "objcopy" a try. It can convert a binary file into an
>>object file which could then be linked normally.
>
>   You could also create an asm (remember that?) file with the data
> and link the resultant object file.

Yes. BTDT.

>   What's wrong with just creating an Ada source file with all the
>   constant data?

It's nothing wrong with it. But sometimes it just may seem a little bit
overkill to create source code from a binary to be compiled by a
compiler to get an object file to be linked by the linker when you can
create the needed object file in just one single, fast step with
bin2obj/objcopy. :)

> That would give you the advantage of the compiler doing the
> worrying about the data representation and layout and doing some
> checking that you are accessing things correctly.

Yes, agreed. If you really need this.

But if the file is essentially just an array of basic types I see not
much sense in doing all the steps mentioned above just to link a
lookup-table into the executable.

I mean, what kind of checking can the compiler do on a simple array of
integers?
Especially in the case when the program that writes the binary data uses
the same packages with the type declaration as the program where the
precomputed result is linked into?

Oh, and floating point data might be a little bit funny, if you convert
the binary result to textual representation, and then let the compiler
convert it back to its binary equivalent. ;-)


Vinzent.



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 14:24   ` Vinzent 'Gadget' Hoefler
  2004-12-16 18:12     ` Dr. Adrian Wrigley
  2004-12-17  5:20     ` tmoran
@ 2004-12-17 13:53     ` Dr. Adrian Wrigley
  2004-12-17 14:16       ` Alex R. Mosteo
  2 siblings, 1 reply; 25+ messages in thread
From: Dr. Adrian Wrigley @ 2004-12-17 13:53 UTC (permalink / raw)


OK I have just tried this.

I have a plain text file "NigeriaScam4.txt", which I turn into
an object file "nigeria.o", and link in. (Source below)

I encountered one or two problems:

I can't access the "..._size" value.  This is an absolute value
in the object file, not the location where the value is stored.
If I try to Import the size, the program crashes at runtime,
because the size is incorrectly interpreted as the location of
the variable storing the size.

The code works, but seems a bit messy.  Calculating the size
is a nuisance, and I can't "import" the Message (local type
is not constant).
Aside from these issues, the code works.  The program
outputs the contents of the text file as
built into the executable.

Any ideas for:

1) Importing the value of the ..._size symbol
2) Making the contents constant
3) Tidying it up a bit

note that writing into the Message is possible, but probably
shouldn't be. (is there an option to make the data read-only?)

Of course, the Message could be copied into another value
which was constant, but that would be wasteful.  Or an
access to constant value could be used.

Thanks
-- 
Adrian (using GNAT 3.15p on Athlon/Linux)

with Text_IO;
with System.Storage_Elements;

-- Start out with two files, this source file, and a simple
-- text file, NigeriaScam4.txt
--
-- Build executable by creating object file containing txt data
-- using objcopy.  Include object file in link.
--
-- objcopy --input-target=binary --binary-architecture=i386 --output-target=elf32-i386 NigeriaScam4.txt nigeria.o
-- gnatmake demo

procedure Demo is

   pragma Linker_Options ("nigeria.o");

   Name : constant String := "NigeriaScam4_txt";

   use System.Storage_Elements;

   MessageStart, MessageEnd :  Character;
   pragma Import (C, MessageStart, "_binary_" & Name & "_start");
   pragma Import (C, MessageEnd,   "_binary_" & Name & "_end");

   Message :  String (1 .. Integer (MessageEnd'Address-MessageStart'Address));
   for Message'Address use MessageStart'Address;

begin

   Text_IO.Put_Line ("Length was : " & Integer'Image (Message'Length));
   Text_IO.Put_Line ("Message was : " & Message);

end Demo;




^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-17 13:53     ` Dr. Adrian Wrigley
@ 2004-12-17 14:16       ` Alex R. Mosteo
  0 siblings, 0 replies; 25+ messages in thread
From: Alex R. Mosteo @ 2004-12-17 14:16 UTC (permalink / raw)


Dr. Adrian Wrigley wrote:
> OK I have just tried this.
> 
> I have a plain text file "NigeriaScam4.txt", which I turn into
> an object file "nigeria.o", and link in. (Source below)

You should check AWS resources. I think it can be more "ada-ish". It 
seems perfectly suited for this. The only problem I've encountered is 
for very large files (over 4MB) which require huge quantities of memory 
to be compiled with Gnat. However, the latest AWS version supports 
compressed resources which may ease this problem a bit.

> 
> I encountered one or two problems:
> 
> I can't access the "..._size" value.  This is an absolute value
> in the object file, not the location where the value is stored.
> If I try to Import the size, the program crashes at runtime,
> because the size is incorrectly interpreted as the location of
> the variable storing the size.
> 
> The code works, but seems a bit messy.  Calculating the size
> is a nuisance, and I can't "import" the Message (local type
> is not constant).
> Aside from these issues, the code works.  The program
> outputs the contents of the text file as
> built into the executable.
> 
> Any ideas for:
> 
> 1) Importing the value of the ..._size symbol
> 2) Making the contents constant
> 3) Tidying it up a bit
> 
> note that writing into the Message is possible, but probably
> shouldn't be. (is there an option to make the data read-only?)
> 
> Of course, the Message could be copied into another value
> which was constant, but that would be wasteful.  Or an
> access to constant value could be used.
> 
> Thanks



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: Efficiently setting up large quantities of constant data
  2004-12-16 23:47       ` Dr. Adrian Wrigley
@ 2004-12-19 14:34         ` Simon Wright
  0 siblings, 0 replies; 25+ messages in thread
From: Simon Wright @ 2004-12-19 14:34 UTC (permalink / raw)


"Dr. Adrian Wrigley" <amtw@linuxchip.demon.co.uk.uk.uk> writes:

> On Thu, 16 Dec 2004 15:52:28 -0600, Larry Kilgallen wrote:
> > But since he did not say the purpose of putting it in the
> > application was for speed, I think we have to presume it was for
> > religious reasons.
> 
> It seems like a good idea in some circumstances:

Another one might be a target which, for whatever reason, doesn't have
an easily-accessible filesystem .. in our case that would be VxWorks
where someone else has decided that there can be no more than 8 files
on the flash filesystem, and there is no disk-based FS.

-- 
Simon Wright                               100% Ada, no bugs.



^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2004-12-19 14:34 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-12-13 21:48 Efficiently setting up large quantities of constant data Michael Mounteney
2004-12-13 22:09 ` Stephen Leake
2004-12-13 22:15 ` Luke A. Guest
2004-12-14  0:20 ` Jeffrey Carter
2004-12-14  8:43 ` Martin Krischik
2004-12-14 12:18 ` Simon Wright
2004-12-15 21:10   ` Brian May
2004-12-16 23:18     ` Nick Roberts
2004-12-16 12:36 ` Dr. Adrian Wrigley
2004-12-16 13:50   ` Marc A. Criley
2004-12-17  2:32     ` John B. Matthews
2004-12-16 14:06   ` rien
2004-12-16 14:24   ` Vinzent 'Gadget' Hoefler
2004-12-16 18:12     ` Dr. Adrian Wrigley
2004-12-16 23:25       ` Nick Roberts
2004-12-17  5:20     ` tmoran
2004-12-17  8:38       ` Vinzent 'Gadget' Hoefler
2004-12-17 13:53     ` Dr. Adrian Wrigley
2004-12-17 14:16       ` Alex R. Mosteo
2004-12-16 17:11   ` Andre
2004-12-16 21:52     ` Larry Kilgallen
2004-12-16 23:23       ` Nick Roberts
2004-12-16 23:47       ` Dr. Adrian Wrigley
2004-12-19 14:34         ` Simon Wright
2004-12-16 18:29 ` Alex R. Mosteo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox