basic question on nested packages

basic question on nested packages

[Nasser M. Abbasi]

I'd like to improve the structure of the Ada lapack binding
to make it easier to use.

The official Fortran lapack is structured as follows

                                lapack
                                  +
                                  |
                 +----------------+-------------------+
                 |                |                   |
              driver routines    computational     auxiliary

The current Ada Lapack binding has the following 4 packages
that maps to the above, and a package for IO, like this:

labase.ads  (provides only data types)

ladrv.ads (the drivers API's, WITH labase;)
laaux.ads (the auxillary API's, WITH labase;)
lacmp.ads (the computional API's, WITH labase;)

then an IO package
labaseio.ads, labaseio.adb (WITH labase;) provides IO routines
for the lapack Ada types.

So, a client Ada program that wants to use lapack's driver
routine say SGESV() would do

-------------------
with labase; use labase;
with ladrv;
....
     A    : labase.Fortran_Real_Matrix (1 .. 3, 1 .. 3);
     ladrv.SGESV(...)
------------------------

What I think would be better is to have ONE lapack package and
with the above packages as nested packages. So that a client
program would do

-------------------
with lapack;

....
     A    : lapack.Fortran_Real_Matrix (1 .. 3, 1 .. 3);
     lapack.driver.SGESV(...)
------------------------

i.e. a client would only need to WITH one package, and
access the other packages using a dot from that one base package.

Hence it will be, for the API's

    lapack.driver.foo()
    lapack.comp.foo()
    lapack.aux.foo()
    lapack.IO.foo()

and for the datatypes, it will just be

    A : lapack.type_name;

The 4 ada lapack packages (driver, comp, aux, IO) all do
WITH labase;  So putting them as nested packages to labase
seems to make sense.

I do not know much about Ada packaging and such, so wanted to ask
the experts what they think. Woukd nested packages be the answer,
or some other stucture? or should the API be left as is?

There will be no code changes in the packages itself, just
a bit of shuffling of the code location in the files to make it
easier and more clear to use the binding. I think the client will
be more clear this way.

Any thoughts?

--Nasser

Re: basic question on nested packages

[Nasser M. Abbasi]

On 7/27/2012 10:22 PM, Nasser M. Abbasi wrote:

>
> What I think would be better is to have ONE lapack package and
> with the above packages as nested packages.

ref (me)

I think may be I need to use 'child packages', not 'nested pacakges',
but not sure yet.

I am doing a crash course now reading an ada book to learn more
the difference between these and which one to use....

--Nasser

Re: basic question on nested packages

[Shark8]

On Friday, July 27, 2012 11:27:31 PM UTC-6, Nasser M. Abbasi wrote:
> On 7/27/2012 10:22 PM, Nasser M. Abbasi wrote:
> 
> >
> > What I think would be better is to have ONE lapack package and
> > with the above packages as nested packages.
> ref (me)
> 
> I think may be I need to use 'child packages', not 'nested pacakges',
> but not sure yet.
> 
> I am doing a crash course now reading an ada book to learn more
> the difference between these and which one to use....

You could likely use nested packages; though child packages will likely be better for maintainability, but if they're short/simple child packages make good sense.

Re: basic question on nested packages

[Nasser M. Abbasi]

On 7/28/2012 12:36 AM, Shark8 wrote:
> On Friday, July 27, 2012 11:27:31 PM UTC-6, Nasser M. Abbasi wrote:
>> On 7/27/2012 10:22 PM, Nasser M. Abbasi wrote:
>>
>>>
>>> What I think would be better is to have ONE lapack package and
>>> with the above packages as nested packages.
>> ref (me)
>>
>> I think may be I need to use 'child packages', not 'nested pacakges',
>> but not sure yet.
>>
>> I am doing a crash course now reading an ada book to learn more
>> the difference between these and which one to use....
>

  
> You could likely use nested packages; though child packages will likely be better
>for maintainability, but if they're short/simple child packages make good sense.
>

Yes, this is what I ended up doing. It was easier that I thought.

Now the lapack client has this:

----------------------------
with Ada.Text_IO; use Ada.Text_IO;
with Interfaces.Fortran; use Interfaces.Fortran;
with lapack, lapack.driver;

procedure mysolve is
     A    : lapack.Fortran_Real_Matrix (1 .. 3, 1 .. 3)
  begin
    lapack.driver.SGESV(...)
    ....
-----------------------

>gnatmake -I../ada mysolve.adb
           -largs
           -L/usr/lib/atlas-base/atlas
           -L/usr/lib/atlas-base/
           -lblas
           -llapack

>./mysolve
-1.31250E+00
  3.50000E+00
  1.12500E+00
>

done!

The only surprise for me, was that I had to WITH the 'driver' which is a
child package of the parent lapack package explicitly to use it from the
client even though I withed the parent package.

I thought, since it is a child package of lapack, then it will be pulled in
automatically by just WITH'ing lapack. But no.

i.e. I thought by inviting the parent in, the children will come along
with the parent and not be left alone outside.

No problem.

I think now the API is much more clean. I rebuild lapack
with this new change and all is well. I think when Lapack binding was made,
child packages did not exist yet in Ada?

I also removed all the pragma linker options that was hard-coded in the
source code. I do not like to see hard-coded values for build in source
code. I think all of this belong to Makefile or gpr and not in source code.

Making more documentations and diagrams and will update all this on
my Ada page soon with a new snapshot tar file in case someone wants to try
it.

--Nasser

Re: basic question on nested packages

[AdaMagica]

On Saturday, July 28, 2012 11:01:25 AM UTC+2, Nasser M. Abbasi wrote:
> The only surprise for me, was that I had to WITH the 'driver' which is a
> child package of the parent lapack package explicitly to use it from the
> client even though I withed the parent package.
>
> I thought, since it is a child package of lapack, then it will be pulled in
> automatically by just WITH'ing lapack. But no.
> i.e. I thought by inviting the parent in, the children will come along
> with the parent and not be left alone outside.

It's just the other way round. Withing the child implicitly withs the parent:

with Parent.Child;  -- no need to with Parent

It would indeed be confusing if you could see the child, but not the parent.

Re: basic question on nested packages

[J-P. Rosen]

Le 28/07/2012 11:53, AdaMagica a �crit :
>> I thought, since it is a child package of lapack, then it will be pulled in
>> automatically by just WITH'ing lapack. But no.
>> i.e. I thought by inviting the parent in, the children will come along
>> with the parent and not be left alone outside.
> 
> It's just the other way round. Withing the child implicitly withs the parent:
> 
> with Parent.Child;  -- no need to with Parent
> 
> It would indeed be confusing if you could see the child, but not the parent.
> 
It is possible:

with Parent.Child;
package Parent_Child renames Parent_Child;

with Parent_Child;  -- brings in only the child

Not confusing: you can access the declarations of the child, not those
of the parent.

This is actually useful if you want to provide a package to users, and
you want to make it a child (for implementation reasons), but don't want
your users to know that.

-- 
J-P. Rosen
Adalog
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
Tel: +33 1 45 29 21 52, Fax: +33 1 45 29 25 00
http://www.adalog.fr

Re: basic question on nested packages

[Niklas Holsti]

On 12-07-28 06:22 , Nasser M. Abbasi wrote:
> I'd like to improve the structure of the Ada lapack binding
> to make it easier to use.
> 
> The official Fortran lapack is structured as follows
> 
>                                lapack
>                                  +
>                                  |
>                 +----------------+-------------------+
>                 |                |                   |
>              driver routines    computational     auxiliary
> 
> The current Ada Lapack binding has the following 4 packages
> that maps to the above, and a package for IO, like this:
> 
> labase.ads  (provides only data types)
> 
> ladrv.ads (the drivers API's, WITH labase;)
> laaux.ads (the auxillary API's, WITH labase;)
> lacmp.ads (the computional API's, WITH labase;)
> 
> then an IO package
> labaseio.ads, labaseio.adb (WITH labase;) provides IO routines
> for the lapack Ada types.
> 
> So, a client Ada program that wants to use lapack's driver
> routine say SGESV() would do
> 
> -------------------
> with labase; use labase;
> with ladrv;
> ....
>     A    : labase.Fortran_Real_Matrix (1 .. 3, 1 .. 3);
>     ladrv.SGESV(...)
> ------------------------
> 
> What I think would be better is to have ONE lapack package and
> with the above packages as nested packages. So that a client
> program would do
> 
> -------------------
> with lapack;
> 
> ....
>     A    : lapack.Fortran_Real_Matrix (1 .. 3, 1 .. 3);
>     lapack.driver.SGESV(...)
> ------------------------
> 
> i.e. a client would only need to WITH one package, and
> access the other packages using a dot from that one base package.
> 
> Hence it will be, for the API's
> 
>    lapack.driver.foo()
>    lapack.comp.foo()
>    lapack.aux.foo()
>    lapack.IO.foo()

I'm going to play devil's advocate and ask why you don't put it all in
one package, lapack.

Is there any benefit to writing a call as "lapack.driver.SGESV" instead
of directly as "lapack.SGESV"? Is there some risk of ambiguity, with the
name SGESV defined with different meaning in the different packages?

If there is no risk of ambiguity, and a client program typically needs
to "with" both lapack and lapack.driver, it seems much simpler to me to
put all the stuff that a typical client will use directly in package
lapack, and not to divide it into some nested or child packages like
lapack.driver.

Note that the body of package lapack can still delegate the
implementation of an operation like SGESV to a child package (using a
renaming as body, for example), if the package body otherwise risks
becoming very large. On the other hand, this code is probably not
rapidly evolving, any more, so even a large body is unlikely to become a
maintenance problem.

I'm not a LAPACK user, so I don't know anything about its internal
architecture. If there are some internal items that are not used by
normal client applications, you can put them in child packages, e.g.
lapack.com, lapack.aux. An unusual client that needs to use that stuff
directly can "with" these child packages (unless they are declared
"private").

> I do not know much about Ada packaging and such, so wanted to ask
> the experts what they think. Woukd nested packages be the answer,
> or some other stucture? or should the API be left as is?

A hierarchy of child packages is definitely nicer than the old set of
four unrelated packages, IMO. But I would seriously consider using just
one package, at least just one "front" package that contains all the
things a normal client will use.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
      .      @       .

Re: basic question on nested packages

[Nasser M. Abbasi]

On 7/28/2012 7:19 AM, Niklas Holsti wrote:
> On 12-07-28 06:22 , Nasser M. Abbasi wrote:
>> I'd like to improve the structure of the Ada lapack binding
>> to make it easier to use.
>>
>> The official Fortran lapack is structured as follows
>>
>>                                 lapack
>>                                   +
>>                                   |
>>                  +----------------+-------------------+
>>                  |                |                   |
>>               driver routines    computational     auxiliary
>>

....
>>
>> Hence it will be, for the API's
>>
>>     lapack.driver.foo()
>>     lapack.comp.foo()
>>     lapack.aux.foo()
>>     lapack.IO.foo()

>
> I'm going to play devil's advocate and ask why you don't put it all in
> one package, lapack.
>
> Is there any benefit to writing a call as "lapack.driver.SGESV" instead
> of directly as "lapack.SGESV"? Is there some risk of ambiguity, with the
> name SGESV defined with different meaning in the different packages?
>

Main reason is just to keep things organized as in the Fortran tradition
of the original lapack:

http://www.netlib.org/lapack/lug/node22.html

"
The subroutines in LAPACK are classified as follows:
     driver routines, ....
     computational routines.....
     auxiliary routines....
"

So someone used to using lapack would find it easier if there are 3
packages that correspond to those 3 areas.

I assume that is the reason Wasu Chaopanon (who made the binding)
did it this way as you can see here (one of hosting sites for
the binding)

http://www.mirrorservice.org/sites/ftp.cs.nyu.edu/pub/gnat/OLD/contrib/lapack-ada/README

So, I think it is better to keep this layout? But putting all
the packages all under one package name "lapack" in order to reduce
name space is all what I did.

thanks,
--Nasser

Re: basic question on nested packages

[Niklas Holsti]

On 12-07-28 15:52 , Nasser M. Abbasi wrote:
> On 7/28/2012 7:19 AM, Niklas Holsti wrote:
>> On 12-07-28 06:22 , Nasser M. Abbasi wrote:
>>> I'd like to improve the structure of the Ada lapack binding
>>> to make it easier to use.
>>>
>>> The official Fortran lapack is structured as follows
>>>
>>>                                 lapack
>>>                                   +
>>>                                   |
>>>                  +----------------+-------------------+
>>>                  |                |                   |
>>>               driver routines    computational     auxiliary
>>>
> 
> ....
>>>
>>> Hence it will be, for the API's
>>>
>>>     lapack.driver.foo()
>>>     lapack.comp.foo()
>>>     lapack.aux.foo()
>>>     lapack.IO.foo()
> 
>>
>> I'm going to play devil's advocate and ask why you don't put it all in
>> one package, lapack.
>>
>> Is there any benefit to writing a call as "lapack.driver.SGESV" instead
>> of directly as "lapack.SGESV"? Is there some risk of ambiguity, with the
>> name SGESV defined with different meaning in the different packages?
>>
> 
> Main reason is just to keep things organized as in the Fortran tradition
> of the original lapack:
> 
> http://www.netlib.org/lapack/lug/node22.html
> 
> "
> The subroutines in LAPACK are classified as follows:
>     driver routines, ....
>     computational routines.....
>     auxiliary routines....
> "

Ok, but compare Ada.Text_IO: there are subprograms to input data, to
output data, and to open/close files. These can be considered to be of
different "classes", but the Ada authors did not define separate
packages for them, so that we would have, for example,
Ada.Text_IO.Inputs.Get_Line, Ada.Text_IO.Files.Open, and so on.
(Granted, child packages did not exist at the time time Text_IO was
defined, but still I would consider such a child-package system to be
overdesign.)

In other words, while this classification of LAPACK subroutines no doubt
makes it easier to understand and learn to use LAPACK, I still don't see
why it would be good to embody this logical classification as a package
hierarchy. If a user has already decided to use the SGESV subprogram, I
don't see any benefit in forcing the user to "with" lapack.driver in
addition to lapack, and to write lapack.driver.SGESV instead of the
shorter name lapack.SGESV.

It could be useful to put the non-driver subprograms in child packages,
to emphasize that users should prefer to use the driver-level
subprograms (as suggested in the link you gave). So I would put all the
types and driver subprograms in package lapack, and the "computational"
and "auxiliary" subprograms in lapack.comp and lapack.aux, respectively.
I'm not sure about the I/O subprograms, but if most clients need them, I
would put them directly in package lapack and not in a child package.
This has the advantage that the I/O subprograms become primitive
operations for the LAPACK types and are overridably inherited in derived
types.

All this is IMO, of course, and tastes will differ.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
      .      @       .

Re: basic question on nested packages

[Nasser M. Abbasi]

On 7/28/2012 8:34 AM, Niklas Holsti wrote:

>
> In other words, while this classification of LAPACK subroutines no doubt
> makes it easier to understand and learn to use LAPACK, I still don't see
> why it would be good to embody this logical classification as a package
> hierarchy.

I see what you mean and I understand it. It will indeed make it simpler
to use if there is only one package to include so one does not
have to figure which child to include.

I looked more at the packages, and it will be possible to put all in
one lapack package.  I guess if someone wanted to find which function
belong to which category, they can always look at the original lapack
web site for that.

I'll see if I can make the changes to use one package since all the
other packages do is just add more API's. Nothing more.  It will be
easier to use this way for a client. (There are so many functions in
Lapack, it is scary to think how long it took to write all of this and
do it right. I remember reading somewhere (can't find the link now) that
lapack is about 700,000 lines of Fortran code.

I think though, the large size is due to that each function is broken to 4
functions (one for each flavor of the input type (single, double, complex single,
complex double). In addition the same function is further broken into based
on the storage of the matrix itself (normal, symmetric, banded, etc...)

So, that is why there are so many functions and code. i.e. for the same
one function, there will be may be 30 functions (all different by permutations
of the starting letters of the name of the function and/or the ending letter)
but they all do the same thing pretty much.

Well, they did not have generics in Fortran when they wrote Lapack!  So the
above was the only way to do what they did using Fortran at the time.

--Nasser

Re: basic question on nested packages

[Niklas Holsti]

On 12-07-29 04:19 , Nasser M. Abbasi wrote:
> On 7/28/2012 8:34 AM, Niklas Holsti wrote:
> 
>>
>> In other words, while this classification of LAPACK subroutines no doubt
>> makes it easier to understand and learn to use LAPACK, I still don't see
>> why it would be good to embody this logical classification as a package
>> hierarchy.
> 
> I see what you mean and I understand it. It will indeed make it simpler
> to use if there is only one package to include so one does not
> have to figure which child to include.
> 
> I looked more at the packages, and it will be possible to put all in
> one lapack package.  I guess if someone wanted to find which function
> belong to which category, they can always look at the original lapack
> web site for that.

The declarations in the single package can be divided into a text
section for each category, with comment headings separating the sections
and explaining what the categories mean. That would serve the
pedagogical purpose without introducting more packages.

> I think though, the large size is due to that each function is broken to 4
> functions (one for each flavor of the input type (single, double,
> complex single,
> complex double). In addition the same function is further broken into based
> on the storage of the matrix itself (normal, symmetric, banded, etc...)
> 
> So, that is why there are so many functions and code. i.e. for the same
> one function, there will be may be 30 functions (all different by
> permutations
> of the starting letters of the name of the function and/or the ending
> letter)
> but they all do the same thing pretty much.

This could be a more logical reason for having some nested or child
packages in the Ada binding: there could be a nested or child package
for each elementary type, with a further nested or child package for
each storage form. For example, the package lapack.single.symmetric
could hold the subprograms for operations on symmetric matrices of
single-precision numbers.

The naming of the operations would be changed correspondingly so that
the Fortran subroutine SGEBRD, for example, would become
lapack.single.general.BRD.

This scheme could make it easy to change a client from single-precision
to double precision, for example. The single precision version would do
this:

   with lapack.single.general;
   use  lapack.single.general;
   ...
   begin
      BRD (...);
   end;

The double-precision version would change just the context clauses to

   with lapack.double.general;
   use  lapack.double.general;

The call to BRD would be unchanged, although it would now call a
different Fortran subroutine.

An alternative would be to just change the operation names for Ada to
omit the "type and format" letters and instead rely on overloading by
parameter type to select the intended subprogram, while keeping all
declarations in a single "lapack" package. The client would do:

   with lapack;
   ...
   declare
      matrix : lapack.<some lapack matrix type>;
   begin
      lapack.BRD (... matrix, ... );
   end;

The Ada compiler would pick the correct "BRD" subprogram for this matrix
type, e.g. SGEBRD if the matrix type is a general matrix of
single-precision numbers.

> Well, they did not have generics in Fortran when they wrote Lapack!  So the
> above was the only way to do what they did using Fortran at the time.

Generics could have helped a bit, for those operations where the
algorithm (source code) is the same for different types. But in
numerical SW one often has to use different algorithms for different
precisions or matrix formats, and then generics (in the Ada style, at
least) do not help.

Overloading by parameter/result types could have kept the name-space
smaller, but I think it would still have been necessary to implement
most subprograms separately for each precision and matrix format,
because the algorithms are different.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
      .      @       .