Re: A little trouble with very large arrays.

[Shark8 <onewingedshark@gmail.com>]

On Friday, October 5, 2018 at 12:20:34 AM UTC-6, Niklas Holsti wrote:
> On 18-10-05 00:38 , Shark8 wrote:
> > I'm trying to implement a FITS library for work -- see
> > https://fits.gsfc.nasa.gov/standard40/fits_standard40aa.pdf --
> > and have come across some rather interesting problems implementing it.
> >
> > The main-problem right now is the "Primary Data Array" which can
> > have  a dimensionality in 1..999, each itself with some non-zero range.
> > (In the files these are specified by keywords in the file like
> > NAXIS = n, NAXIS1 = n_1, NAXIS2 = n_2, and so on until the NAXISn = n_n
> > keyword/value pair is encountered.)
> >
> > Relatively straightforward, no?
> 
> No. Handling arrays with a variable number of dimensions is not simple.
> 
> > Well, I'd thought I could handle  everything with a dimensionality-array
> > and generic like:
> >
> >     Type Axis_Count is range 0..999 with Size => 10;
> >     Type Axis_Dimensions is Array (Axis_Count range <>) of Positive
> >       with Default_Component_Value => 1;
> > ...
> > Generic
> > 	Type Element is (<>);
> > 	Dim : Axis_Dimensions:= (1..999 => 1);
> > Package FITS.Data with Pure is
> >
> > 	Type Data_Array is not null access Array(
> > 			1..Dim( 1),1..Dim( 2),1..Dim( 3),1..Dim( 4),
> >   			1..Dim( 5),1..Dim( 6),1..Dim( 7),1..Dim( 8),
> >   			--...
> > 			1..Dim( 997),1..Dim( 998),1..Dim( 999)
> > 		) of Element
> 
> Give it some thought. Even if each dimension would have the smallest 
> sensible length, which is two index values, the total number of elements 
> in that array would be 2**999, somewhat larger than the memories of 
> current computers.

No, the smallest sensible number of indices is 1, for everything except maybe the first two or three dimensions: eg Image data from a camera, or perhaps topological data from a map (longitude, latitude, elevation).


FITS was developed for handling "image" transport by the astronomy world, back when there were 9-bit bytes and such.