Substrings as argument to procedures/functions

Substrings as argument to procedures/functions

[reinkor]

Dear All,

I believed that if a substring is an argument to a subroutine,
then its "range" would start at 1 (and not inherited from the calling program).
But this seems not to be the case. 

Given the following Ada source code:

with Text_IO;
use  Text_IO;
procedure t3 is
   package Int_Io is new Text_IO.Integer_Io (Integer);
   use Int_Io;
   procedure string_test1(S : String) is
   begin
      New_Line;
      Put(" In string_test1: ");
      Put(S'First,4);
      Put(S'Last,4);
   end string_test1;
   Str : String := "ABCDEF";
begin
      string_test1(Str);
      string_test1(Str(3..5));
end t3;

The second call to "string_test1" gives:

 In string_test1:    3   5

(at least on my computer).

Is this correct? And in case, is it "good" ? :-)

reinert

Re: Substrings as argument to procedures/functions

[Dmitry A. Kazakov]

On 12/04/2016 10:25, reinkor wrote:

> I believed that if a substring is an argument to a subroutine,
> then its "range" would start at 1 (and not inherited from the calling program).
> But this seems not to be the case.
>
> Given the following Ada source code:
>
> with Text_IO;
> use  Text_IO;
> procedure t3 is
>     package Int_Io is new Text_IO.Integer_Io (Integer);
>     use Int_Io;
>     procedure string_test1(S : String) is
>     begin
>        New_Line;
>        Put(" In string_test1: ");
>        Put(S'First,4);
>        Put(S'Last,4);
>     end string_test1;
>     Str : String := "ABCDEF";
> begin
>        string_test1(Str);
>        string_test1(Str(3..5));
> end t3;
>
> The second call to "string_test1" gives:
>
>   In string_test1:    3   5
>
> (at least on my computer).
>
> Is this correct? And in case, is it "good" ? :-)

Yes, it is good.

Consider a program that uses an index to walk through a string/array. 
When you pass a substring of the string and an index into it (usually 
in-out) down to a subprogram, then the index remains valid.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Substrings as argument to procedures/functions

[reinkor]

On Tuesday, April 12, 2016 at 11:02:03 AM UTC+2, Dmitry A. Kazakov wrote:
> On 12/04/2016 10:25, reinkor wrote:
> 
> > I believed that if a substring is an argument to a subroutine,
> > then its "range" would start at 1 (and not inherited from the calling program).
> > But this seems not to be the case.
> >
> > Given the following Ada source code:
> >
> > with Text_IO;
> > use  Text_IO;
> > procedure t3 is
> >     package Int_Io is new Text_IO.Integer_Io (Integer);
> >     use Int_Io;
> >     procedure string_test1(S : String) is
> >     begin
> >        New_Line;
> >        Put(" In string_test1: ");
> >        Put(S'First,4);
> >        Put(S'Last,4);
> >     end string_test1;
> >     Str : String := "ABCDEF";
> > begin
> >        string_test1(Str);
> >        string_test1(Str(3..5));
> > end t3;
> >
> > The second call to "string_test1" gives:
> >
> >   In string_test1:    3   5
> >
> > (at least on my computer).
> >
> > Is this correct? And in case, is it "good" ? :-)
> 
> Yes, it is good.
> 
> Consider a program that uses an index to walk through a string/array. 
> When you pass a substring of the string and an index into it (usually 
> in-out) down to a subprogram, then the index remains valid.
> 
> -- 
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de


But as far as I remember, several other program languages do not bring the initial indexing down to subroutines like this?  Anyway, one may for example copy strings in soubroutines if one really want to start indexing at one.

Sometimes I want the soubroutines not to "know" too much :-)

Re: Substrings as argument to procedures/functions

[Dmitry A. Kazakov]

On 2016-04-12 18:55, reinkor wrote:
> On Tuesday, April 12, 2016 at 11:02:03 AM UTC+2, Dmitry A. Kazakov wrote:
>> On 12/04/2016 10:25, reinkor wrote:
>>
>>> I believed that if a substring is an argument to a subroutine,
>>> then its "range" would start at 1 (and not inherited from the calling program).
>>> But this seems not to be the case.
>>>
>>> Given the following Ada source code:
>>>
>>> with Text_IO;
>>> use  Text_IO;
>>> procedure t3 is
>>>      package Int_Io is new Text_IO.Integer_Io (Integer);
>>>      use Int_Io;
>>>      procedure string_test1(S : String) is
>>>      begin
>>>         New_Line;
>>>         Put(" In string_test1: ");
>>>         Put(S'First,4);
>>>         Put(S'Last,4);
>>>      end string_test1;
>>>      Str : String := "ABCDEF";
>>> begin
>>>         string_test1(Str);
>>>         string_test1(Str(3..5));
>>> end t3;
>>>
>>> The second call to "string_test1" gives:
>>>
>>>    In string_test1:    3   5
>>>
>>> (at least on my computer).
>>>
>>> Is this correct? And in case, is it "good" ? :-)
>>
>> Yes, it is good.
>>
>> Consider a program that uses an index to walk through a string/array.
>> When you pass a substring of the string and an index into it (usually
>> in-out) down to a subprogram, then the index remains valid.
>
> But as far as I remember, several other program languages do not
> bring the initial indexing down to subroutines like this?

I don't remember any other language implementing arrays in a consistent way.

> Anyway, one may for example copy strings in soubroutines if one really want to start indexing at one.

Why would you want it to start at a certain number? There is S'First 
that tells where it begins.

In Ada 2012 you can constrain a string subtype to always begin at 1, 
which is not a great idea. Though it will crash your program sooner than 
otherwise.

The rule is that the preconditions must be as weak as possible. If you 
require the index to begin at 1 you strengthen the subprogram's 
precondition. If you don't require it, but silently slide indices, you 
have two different types outside the call and inside it. This is even 
worse, because it breaks the substitutablity rule in so many cases.

I think that one problem is confusing index with position. Index and 
string bounds are absolute, they are not positions or offsets. It would 
be nice if the language separated them more cleanly. Note that it does 
this in some cases. For example S'Length is not index, it is a position. 
So its type is not the type of the index, e.g. Stream_Element_Offset for 
Stream_Element_Array, but universal integer. The type of index is a 
specific type. The type of position is universal integer. The first 
position is always 1. The first index is any.

> Sometimes I want the soubroutines not to "know" too much :-)

Knowing that the index type and the string type bound to the same object 
remain consistent with the slicing operation is not "too much", it is 
knowing that the language design is sane.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Substrings as argument to procedures/functions

[reinkor]

Yes, I may somehow have confused index and position. 
However, there may be some security/paranoia reasons to minimize the information handed over to a subroutine?

Re: Substrings as argument to procedures/functions

[Dmitry A. Kazakov]

On 13/04/2016 06:44, reinkor wrote:
> Yes, I may somehow have confused index and position.
> However, there may be some security/paranoia reasons to minimize the
> information handed over to a subroutine?

How is that insecure not to know array bounds? It is all about contracts.

Consider Ada.Stream_IO. It has the procedure Read

    procedure Read (File : in  File_Type;
                     Item : out Stream_Element_Array;
                     Last : out Stream_Element_Offset);

With indices sliding there would be no way to implement it. Consider a call:

    Read (Buffer (From..Buffer'Last), To);

How could Read know what was From at the caller side?

You probably could redesign all interfaces in terms of positions, but 
then it will be heavy burden on the client's side, and very error-prone 
for the client to translate returned positions back to indices.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Substrings as argument to procedures/functions

[Brian Drummond]

On Tue, 12 Apr 2016 21:44:01 -0700, reinkor wrote:

> Yes, I may somehow have confused index and position.
> However, there may be some security/paranoia reasons to minimize the
> information handed over to a subroutine?

Rather the reverse.

It's the absence of such information - and its consequences - that makes 
buffer overflow attacks possible.

-- Brian

Re: Substrings as argument to procedures/functions

[reinkor]

On Wednesday, April 13, 2016 at 12:19:28 PM UTC+2, Brian Drummond wrote:
> On Tue, 12 Apr 2016 21:44:01 -0700, reinkor wrote:
> 
> > Yes, I may somehow have confused index and position.
> > However, there may be some security/paranoia reasons to minimize the
> > information handed over to a subroutine?
> 
> Rather the reverse.
> 
> It's the absence of such information - and its consequences - that makes 
> buffer overflow attacks possible.
> 
> -- Brian

I have no strong opinion, but just trying arguments: 

If a complex subroutine is made by a subcontract (in the country far far away),
then it is in general a point to provide only the necessary information
to make the subroutine? I mean, if paranoia is in charge ? :-)

Re: Substrings as argument to procedures/functions

[Pascal Obry]

Le mercredi 13 avril 2016 à 04:43 -0700, reinkor a écrit :
> If a complex subroutine is made by a subcontract (in the country far
> far away),
> then it is in general a point to provide only the necessary
> information
> to make the subroutine? I mean, if paranoia is in charge ? :-)

No the point is for programmers to stop assuming anything about the
data passed as argument. And here Ada shine, it gives you 'First,
'Last, 'Range...

As pointed by others this is one very strong point to avoid buffer
overflows.

-- 
  Pascal Obry /  Magny Les Hameaux (78)

  The best way to travel is by means of imagination

  http://v2p.fr.eu.org
  http://www.obry.net

  gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: Substrings as argument to procedures/functions

[Mart van de Wege]

Pascal Obry <pascal@obry.net> writes:

> Le mercredi 13 avril 2016 à 04:43 -0700, reinkor a écrit :
>> If a complex subroutine is made by a subcontract (in the country far
>> far away),
>> then it is in general a point to provide only the necessary
>> information
>> to make the subroutine? I mean, if paranoia is in charge ? :-)
>
> No the point is for programmers to stop assuming anything about the
> data passed as argument. And here Ada shine, it gives you 'First,
> 'Last, 'Range...

I do have a minor question here though: That does not work if you have
any reason to try and modify the first character of a String. S(1) :=
To_Upper(S(1)) works, but S'First := To_Upper(S'First) does not.

So how do I handle that particular case?

(In my case I just threw in a localised 'Pragma Warnings' to turn off
the GNAT warning, but that's a workaround, not a solution, IMO).

Mmart

-- 
"We will need a longer wall when the revolution comes."
    --- AJS, quoting an uncertain source.

Re: Substrings as argument to procedures/functions

[Egil H H]

On Wednesday, April 13, 2016 at 2:42:04 PM UTC+2, Mart van de Wege wrote:
> I do have a minor question here though: That does not work if you have
> any reason to try and modify the first character of a String. S(1) :=
> To_Upper(S(1)) works, but S'First := To_Upper(S'First) does not.
> 
> So how do I handle that particular case?

S(S'First) := To_Upper(S(S'First));

-- 
~egilhh

Re: Substrings as argument to procedures/functions

[Mart van de Wege]

Egil H H <ehh.public@gmail.com> writes:

> On Wednesday, April 13, 2016 at 2:42:04 PM UTC+2, Mart van de Wege wrote:
>> I do have a minor question here though: That does not work if you have
>> any reason to try and modify the first character of a String. S(1) :=
>> To_Upper(S(1)) works, but S'First := To_Upper(S'First) does not.
>> 
>> So how do I handle that particular case?
>
> S(S'First) := To_Upper(S(S'First));

D'uh. Imagine me slapping my hand against my forehead right now.

-- 
"We will need a longer wall when the revolution comes."
    --- AJS, quoting an uncertain source.

Re: Substrings as argument to procedures/functions

[Jeffrey R. Carter]

On 04/13/2016 05:30 AM, Mart van de Wege wrote:
>
> I do have a minor question here though: That does not work if you have
> any reason to try and modify the first character of a String. S(1) :=
> To_Upper(S(1)) works, but S'First := To_Upper(S'First) does not.

S is an array. 1 is an index into that array. S (1) is the component of S at 
index 1, which is a Character. So your 1st statement passes a Character to a 
function and assigns the result to a Character.

S'First is an index into S. It is also constant. So your 2nd statement attempts 
to pass an Integer to a function that expects a Character, and tries to assign 
the result to a constant.

If you use the index S'First in the same way as you used the index 1, things 
will probably work better for you.

-- 
Jeff Carter
"Violence is the last refuge of the incompetent."
Foundation
151

Re: Substrings as argument to procedures/functions

[Robert A Duff]

reinkor <reinkor@gmail.com> writes:

> I believed that if a substring is an argument to a subroutine,
> then its "range" would start at 1 (and not inherited from the calling program).
> But this seems not to be the case. 
>
> Given the following Ada source code:
>
> with Text_IO;
> use  Text_IO;
> procedure t3 is
>    package Int_Io is new Text_IO.Integer_Io (Integer);
>    use Int_Io;
>    procedure string_test1(S : String) is
>    begin
>       New_Line;
>       Put(" In string_test1: ");
>       Put(S'First,4);
>       Put(S'Last,4);
>    end string_test1;
>    Str : String := "ABCDEF";
> begin
>       string_test1(Str);
>       string_test1(Str(3..5));
> end t3;
>
> The second call to "string_test1" gives:
>
>  In string_test1:    3   5
>
> (at least on my computer).
>
> Is this correct?

Yes.

>...And in case, is it "good" ? :-)

No.  It breaks abstraction -- the procedure shouldn't have to know where
the string came from (e.g. a slice).  And it either introduces bugs
(because the code assumes S'First = 1, which is almost always true),
or it complicates the code (because you have to do arithmetic on
the bounds).

But Dmitry makes a good point.

- Bob

Re: Substrings as argument to procedures/functions

[Jeffrey R. Carter]

On 04/12/2016 01:25 AM, reinkor wrote:
>
> I believed that if a substring is an argument to a subroutine,
> then its "range" would start at 1 (and not inherited from the calling program).
> But this seems not to be the case.

What would you expect for a slice of a 1D array other than String? What would 
you expect for a slice of a 1D array with a non-numeric index type? Remember 
that the declaration of type String is

type String is array (Positive range <>) of Character with Pack;

(ARM A.1, http://www.ada-auth.org/standards/rm12_w_tc1/html/RM-A-1.html);

it's no different than any other 1D array type. And it's possible to declare 
your own string types (ARM 3.6.3), so you could declare

type Str is array (Positive range <>) of Character;

and do anything with it that you can do with String. This isn't very useful, but 
a possibly useful example would be

subtype Hex_Range is Natural range 0 .. 15;

type Hex_Image_List is array (Hex_Range) of Character;

Hex_Image : constant Hex_Image_List := "0123456789ABCDEF";

which allows you to map a hex-digit value onto its Character representation.

Note also that when you call a subprogram with a parameter of type String, the 
actual parameter is always a subtype of String, with the subtype determined by 
its bounds. During the call, the formal parameter takes on the subtype of the 
actual parameter. While one normally declares a subtype of String of length 3 as

S : String (1 .. 3);

there's no reason one can't declare

S : String (3 ..5);

and sometimes there are good reasons for such a declaration. Should the subtype 
and the good reasons for its declaration disappear if S is passed to a 
subprogram? A slice has a different subtype than the array being sliced, and 
sometimes there are good reasons for it to have a different subtype.

> Is this correct? And in case, is it "good" ? :-)

Yes, it's good. For example, parsing a simple CSV file (no quoted fields) using 
an Index function from Ada.Strings.Fixed looks like

Source : constant String := Get_Line (CSV_File);

Start : Positive := Source'First;
Stop  : Natural;

Parse : loop
    Stop := Index (Source (Start .. Source'Last), ",");

    if Stop = 0 then
       Stop := Source'Last;
    else
       Stop := Stop - 1;
    end if;

    -- Do something with Source (Start .. Stop)

    exit Parse when Stop = Source'Last;

    Start := Stop + 2;
end loop Parse;

This works because the 1st formal parameter of Index retains the subtype of its 
actual parameter, which is the slice Source (Start .. Source'Last). If it 
didn't, the values it returned would always be relative to the start of the 
slice passed to it, and the example would be much more complicated.

Finally, in Ada a string type with the lower bound always 1 looks like

type String_1 (Length : Natural) is record
    Value : String (1 .. Length);
end record;

and you can define slicing for it:

function Slice (Source : String_1; Low : Positive; High : Natural)
return String_1;

-- 
Jeff Carter
"Apart from the sanitation, the medicine, education, wine,
public order, irrigation, roads, the fresh water system,
and public health, what have the Romans ever done for us?"
Monty Python's Life of Brian
80

Re: Substrings as argument to procedures/functions

[Randy Brukardt]

"Jeffrey R. Carter" <spam.jrcarter.not@spam.not.acm.org> wrote in message 
news:nejeuc$57d$1@dont-email.me...
> ... And it's possible to declare your own string types (ARM 3.6.3), so you 
> could declare
>
> type Str is array (Positive range <>) of Character;
>
> and do anything with it that you can do with String. This isn't very 
> useful, but a possibly useful example would be

Probably a better example is to remember that the range of Positive is 
implementation-defined, and that the language only requires the upper bound 
to be at least 32767. So if you need strings that have potentially more 
characters than that (to read an entire text file, for instance), and you 
want the code to be unconditionally portable (to steal someone else's line), 
you need to declare a type yourself (here, assuming that a million 
characters are enough):

     type Big_Natural is range 0 .. 1_000_000;
     subtype Big_Positive is Big_Natural range 1 .. Big_Natural'Last;
     type Big_String is array (Big_Positive range <>) of Character;

You can do almost anything you can do with a String with a Big_String (and 
you can convert a Big_String to a String so you can use Put_Line and the 
like), but the index type is guaranteed to support up to a million 
characters.

                                Randy.

Re: Substrings as argument to procedures/functions

[rieachus]

On Wednesday, April 13, 2016 at 5:29:29 PM UTC-4, Randy Brukardt wrote:

> Probably a better example is to remember that the range of Positive is 
> implementation-defined, and that the language only requires the upper bound 
> to be at least 32767. So if you need strings that have potentially more 
> characters than that (to read an entire text file, for instance), and you 
> want the code to be unconditionally portable (to steal someone else's line), 
> you need to declare a type yourself (here, assuming that a million 
> characters are enough)...

Is it time to "fix" this?  Or are there still Ada compilers around that use 16-bit String indexes by default? I remember when (in the early days of Ada83) the question of whether String should use 16 or 32 bit indexes was a major implementation decision.  Today, compilers that support hardware indexes shorter than 32-bits are probably mapping to some hardware defined offset field in instructions. Using such indexes when the subtype allows is obviously an optimization worth supporting.  But in Ada defining "subtype Short_String is String range (1..32767);" or whatever allows using such a hardware type if available.  But there is no need to make Short_String a type.

Yes, programs may have records with character string fields limited such that the index can fit into a 16-bit field in the record.  But I can't imagine doing that without an explicit layout for the record or at least a size for the index field. Back to the original topic here, you want/need sliding to work such that

    type Rec is
      Name_Length : Integer range 0..32_767; 
      Name: String;
    end;
    for Rec use ...
    ...
    Name_Length := Param'Length;
    Name(1..Name_Length) := Param;

 works as expected.

Re: Substrings as argument to procedures/functions

[Randy Brukardt]

<rieachus@comcast.net> wrote in message 
news:518c7059-14c6-4344-a251-129cca2621af@googlegroups.com...
On Wednesday, April 13, 2016 at 5:29:29 PM UTC-4, Randy Brukardt wrote:

>> Probably a better example is to remember that the range of Positive is
>> implementation-defined, and that the language only requires the upper 
>> bound
>> to be at least 32767. So if you need strings that have potentially more
>> characters than that (to read an entire text file, for instance), and you
>> want the code to be unconditionally portable (to steal someone else's 
>> line),
>> you need to declare a type yourself (here, assuming that a million
>> characters are enough)...
>
>Is it time to "fix" this?

Use of predefined types is evil, so who cares?

>Or are there still Ada compilers around that use 16-bit String indexes by 
>default?

Janus/Ada for one.

All versions of Janus/Ada have had Integer as 16-bit. Changing that would 
destroy compatability of binary files (Sequential_IO, Direct_IO, Stream_IO) 
and of course would have other effects as well.

I've considered having some sort of optional way to change the definition of 
Integer (it's defined in a single place), but the problem is that 
*everything* depends upon that, so one would end up with two different 
incompatible compilers/runtimes. The maintenance headaches would be immense 
(any mix-up would cause bizarre internal errors).

Good code doesn't depend on predefined types in the first place, so it's 
mainly a problem dealing with the predefined packages. (And that's mainly a 
problem with the predefined packages depending on type String.)

                           Randy.

Re: Substrings as argument to procedures/functions

[Georg Bauhaus]

On 26.04.16 00:07, Randy Brukardt wrote:
> Use of predefined types is evil, so who cares?

Anyone coming from just about any other language, since they
have no idea that there could be such a fine thing as user defined
fundamental types! ;-)

So, how does one introduce them to programming in Ada without using
library types but still using strings, and text I/O?

Re: Substrings as argument to procedures/functions

[Randy Brukardt]

"Georg Bauhaus" <bauhaus@futureapps.invalid> wrote in message 
news:nfn0mh$abh$1@dont-email.me...
> On 26.04.16 00:07, Randy Brukardt wrote:
>> Use of predefined types is evil, so who cares?
>
> Anyone coming from just about any other language, since they
> have no idea that there could be such a fine thing as user defined
> fundamental types! ;-)
>
> So, how does one introduce them to programming in Ada without using
> library types but still using strings, and text I/O?

Using Root_String'Class, of course. Which Ada doesn't have, unfortunately. 
I'd rather fix that rather than noddling with Standard types - the 
Wide_Wide_Unbounded_UTF8_String nonsense doesn't make sense (but maybe it's 
UTF8_Unbounded_Wide_Wide_String? Bah hambug.)

                                                   Randy.

Re: Substrings as argument to procedures/functions

[G.B.]

On 12.04.16 10:25, reinkor wrote:
> Dear All,
>
> I believed that if a substring is an argument to a subroutine,
> then its "range" would start at 1 (and not inherited from the calling program).
> But this seems not to be the case.

> Is this correct? And in case, is it "good" ? :-)

It is a good idea, I think, not to be mislead by programming
languages that don't have expressions for iteration at all,
nor "sequence types", only emulations. That these are prone to
errors has been mentioned in this thread.  Consider

    for K in A'Range loop
       ... A(K) ...
    end loop;

or, more recently found in Ada,

    for El of A loop
       ... El ...
    end loop;

This most frequently used loop that does sequence processing
does not refer to any specific bound on A. No assumptions, or
checks are being programmed. It is safe and efficient.

Similarly, if I were using a recursive LISP style approach,
positions, or index values, do not matter either:

    function Count (A : String; X : Character) return Integer is

       function Count_Loop (K : Positive) return Natural is
       begin
          if K <= A'Last then
             return "+" (Boolean'Pos ("=" (A(K), X)),
                         Count_Loop (Positive'Succ (K)));
          else
             return 0;
          end if;
       end Count_Loop;

    begin
       return Count_Loop (A'First);
    end Count;