From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-0.9 required=5.0 tests=BAYES_00,FORGED_GMAIL_RCVD, FREEMAIL_FROM autolearn=no autolearn_force=no version=3.4.4 X-Received: by 2002:a6b:7012:: with SMTP id l18-v6mr9038223ioc.24.1538768948367; Fri, 05 Oct 2018 12:49:08 -0700 (PDT) X-Received: by 2002:a9d:618e:: with SMTP id g14-v6mr158714otk.3.1538768948237; Fri, 05 Oct 2018 12:49:08 -0700 (PDT) Path: eternal-september.org!reader01.eternal-september.org!reader02.eternal-september.org!feeder.eternal-september.org!news.gegeweb.eu!gegeweb.org!usenet-fr.net!proxad.net!feeder1-2.proxad.net!209.85.166.216.MISMATCH!x98-v6no105956ita.0!news-out.google.com!l70-v6ni98itb.0!nntp.google.com!z5-v6no104950ite.0!postnews.google.com!glegroupsg2000goo.googlegroups.com!not-for-mail Newsgroups: comp.lang.ada Date: Fri, 5 Oct 2018 12:49:07 -0700 (PDT) In-Reply-To: Complaints-To: groups-abuse@google.com Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=76.18.64.97; posting-account=lJ3JNwoAAAAQfH3VV9vttJLkThaxtTfC NNTP-Posting-Host: 76.18.64.97 References: <3f2828df-d54a-4427-bc3c-dc5ef0dc8069@googlegroups.com> User-Agent: G2/1.0 MIME-Version: 1.0 Message-ID: <13fa128f-5ec7-453e-9231-2260dd4adcbc@googlegroups.com> Subject: Re: A little trouble with very large arrays. From: Shark8 Injection-Date: Fri, 05 Oct 2018 19:49:08 +0000 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Xref: reader02.eternal-september.org comp.lang.ada:54495 Date: 2018-10-05T12:49:07-07:00 List-Id: > > No, the smallest sensible number of indices is 1, for everything except > > maybe the first two or three dimensions: eg Image data from a camera, o= r > > perhaps topological data from a map (longitude, latitude, elevation). >=20 > FITS images can have more dimensions than that. Further dimensions might= =20 > be the frequency of the light (spectral imaging); polarisation; time=20 > when image was taken; and perhaps a couple more that don't come to mind= =20 > immediately. Sure; but even those are a fairly small dimensionality than what the standa= rd allows. =20 > I understand what you tried to do, including having length-one=20 > dimensions, but I don't think that it is a sensible approach to handling= =20 > up to 999 dimensions. I agree with the flattening approach that Dmitry=20 > suggested. That would be rather unfortunate, to be honest. I'd much rather rely on the= compiler translating the indexes than have to do so manually. I trust the = compiler more than myself; plus letting it take care of keeping track of th= e mapping (ie FORTRAN convention) is nice. My ultimate goal was to have some FITS_OBJECT type that had the appropriate= data-members be able to simply "write itself to a stream" to output the pr= oper FITS format file. >=20 > If your FITS files are not much larger than your RAM, the fastest=20 > approach is probably to "mmap" the file into your virtual address space= =20 > and then compute the address of any given image pixel with the=20 > flattening method. If your FITS files are larger than your RAM, your=20 > program should process the file as a stream, which may or may not be=20 > practical, depending on what the program should output. Most of the anticipated usage for where I am right now would be producing F= ITS files, likely in something that would boil down to a coupling like this= : Count : Positive :=3D 1; Today : Ada.Calendar.Time renames Ada.Calendar.Clock; New_Image : Camera_Image renames Normalize( Get_Camera_Image ); New_Object : FITS.Object :=3D FITS.Create_w_Defaults( New_Image ); --.. -- Writes data out to "Observation(YYYY-MM-DD)_00X.FITS". New_Object.Write( Base =3D> "Observation", Date =3D> Today, Count =3D> X = ); I'd rather not tie things to a memory-mapped file at a high level, but it m= ay be that my ideal abstraction is non-tenable. > > FITS was developed for handling "image" transport by the astronomy > > world, back when there were 9-bit bytes and such. >=20 > I know, I used to work in astronomy. What's your point about 9-bit=20 > bytes? FITS standard version 4.0 defines "byte" as 8 bits, and allows=20 > only 8, 16, 32 and 64-bit pixels. No 9-bit pixels. Sorry, that was more about Dmitry's suggestion to pretend representation-cl= auses don't exist; I haven't done anything at a bit-level at all. (And I do= n't think I need to, except perhaps to mark the Primary-Data array elements= as Big-endian [IIRC].)