GNATCOLL JSON Parsing

GNATCOLL JSON Parsing

[eduardsapotski]

I try understand parsing JSON in Ada. 

For example:
Have web-api that gives simple JSON: http://api.exmo.com/v1/trades/?pair=BTC_USD&limit=10

I need to save this data to database.
Created type:

   type Money is delta 0.00000001 range 0.0 .. 9_999_999_999.9;
   type UTC_Date is range 1_500_000_000 .. 3_000_000_000;

   type Trade is record

      Trade_Id   : Integer;
      Pair       : Unbounded_String;
      Trade_Type : Unbounded_String;
      Price      : Money;
      Quantity   : Money;
      Amount     : Money;
      Date       : UTC_Date;
      Saved      : Boolean;

   end record;

Created collection:

    package Vector_Trades is new Ada.Containers.Vectors(Natural, Trade);

    Trades : Vector_Trades.Vector;

Receive data:

JSON : Unbounded_String;

JSON := To_Unbounded_String(AWS.Response.Message_Body (AWS.Client.Get (URL => "http://api.exmo.com/v1/trades/?pair=BTC_USD&limit=10")));

What to do next? How to get list of objects from the JSON-text?

How to save data to database already understood.

Thanks.

Re: GNATCOLL JSON Parsing

[Björn Lundin]

On 2018-06-08 07:52, eduardsapotski@gmail.com wrote:
> I try understand parsing JSON in Ada. 
> 
> For example:
> Have web-api that gives simple JSON: http://api.exmo.com/v1/trades/?pair=BTC_USD&limit=10
> 
> I need to save this data to database.
> Created type:
> 
>    type Money is delta 0.00000001 range 0.0 .. 9_999_999_999.9;
>    type UTC_Date is range 1_500_000_000 .. 3_000_000_000;
> 
>    type Trade is record
> 
>       Trade_Id   : Integer;
>       Pair       : Unbounded_String;
>       Trade_Type : Unbounded_String;
>       Price      : Money;
>       Quantity   : Money;
>       Amount     : Money;
>       Date       : UTC_Date;
>       Saved      : Boolean;
> 
>    end record;
> 
> Created collection:
> 
>     package Vector_Trades is new Ada.Containers.Vectors(Natural, Trade);
> 
>     Trades : Vector_Trades.Vector;
> 
> Receive data:
> 
> JSON : Unbounded_String;
> 
> JSON := To_Unbounded_String(AWS.Response.Message_Body (AWS.Client.Get (URL => "http://api.exmo.com/v1/trades/?pair=BTC_USD&limit=10")));
> 
> What to do next? How to get list of objects from the JSON-text?


something like (not tested):
declare
   use gnatcoll.Json;
   current_item, Reply : JSON_Value := Create;
   BTC_Array : json_array := empty_array;
begin
  Reply := Read(Strm => Aws.Response.Message_Body(AWS_Reply),Filename =>
"");

  if reply.has_Field("BTC_USD") then
     btc_array := reply.Get("BTC_USD");
     if length(btc_array) > 0 then
        for i in 1 .. length(btc_array) loop
           Current_Item := Get(btc_array, i);
           declare
             tradeid : integer := 0;
           begin
             if Current_item.Has_Field("tradeid") then
               tradeid := current_item.Get(""tradeid);
             end if;
               ...
           end;
        end loop;
     end if;
  end if;
end;





> 
> How to save data to database already understood.
> 
> Thanks.
> 
> 
> 
> 
> 
> 


-- 
--
Björn

Re: GNATCOLL JSON Parsing

[eduardsapotski]

OMG I writing homemade parser all morning. 
It turns out everything is simple.
Thank you!

Re: GNATCOLL JSON Parsing

[Per Sandberg]

On Friday, June 8, 2018 at 7:52:56 AM UTC+2, eduards...@gmail.com wrote:
> I try understand parsing JSON in Ada. 
> 
> For example:
> Have web-api that gives simple JSON: http://api.exmo.com/v1/trades/?pair=BTC_USD&limit=10
> 
> I need to save this data to database.
> Created type:
> 
>    type Money is delta 0.00000001 range 0.0 .. 9_999_999_999.9;
>    type UTC_Date is range 1_500_000_000 .. 3_000_000_000;
> 
>    type Trade is record
> 
>       Trade_Id   : Integer;
>       Pair       : Unbounded_String;
>       Trade_Type : Unbounded_String;
>       Price      : Money;
>       Quantity   : Money;
>       Amount     : Money;
>       Date       : UTC_Date;
>       Saved      : Boolean;
> 
>    end record;
> 
> Created collection:
> 
>     package Vector_Trades is new Ada.Containers.Vectors(Natural, Trade);
> 
>     Trades : Vector_Trades.Vector;
> 
> Receive data:
> 
> JSON : Unbounded_String;
> 
> JSON := To_Unbounded_String(AWS.Response.Message_Body (AWS.Client.Get (URL => "http://api.exmo.com/v1/trades/?pair=BTC_USD&limit=10")));
> 
> What to do next? How to get list of objects from the JSON-text?
> 
> How to save data to database already understood.
> 
> Thanks.

You might find the library
 https://github.com/persan/gnatcoll-json
Usefull it contains JSON suport for most Ada.Containers.* packages and som examples.
/P

Re: GNATCOLL JSON Parsing

[eduardsapotski]

Thanks. I saw that. Libraries are not the problem. The problem is that the authors do not provide practical examples of using libraries. It is difficult for a beginner to understand what to do.
björn lundin showed an example and everything became clear.

I've been figuring out how to work with PostgreSQL for a week. 
Unfortunately in addition to this document, virtually no information: https://docs.adacore.com/gnatcoll-docs/sql.html
There are no elementary examples of how to do simple CRUD-operations!
There is no example of how to call a function and get the return value! 
Maybe for Ada-programmers it's obvious.But after working with C#/Java, it's kind of unusual.
Ada language is good! But it is very difficult to approach it.

Re: GNATCOLL JSON Parsing

[Björn Lundin]

On 2018-06-08 17:19, eduardsapotski@gmail.com wrote:

> I've been figuring out how to work with PostgreSQL for a week. 
You should ask earlier.

John Marino did a good job at
http://jrmarino.github.io/AdaBase/

there is also an interface at
https://www.dropbox.com/s/h5gdakfr5uf0n82/pg_ifc.tar.gz?dl=0
which I wrote on top of Samuel Tardieu's Pgada

sql_test.adb shows how to use it,
just put all files in a src directory and link with libpq


> Unfortunately in addition to this document, virtually no information: https://docs.adacore.com/gnatcoll-docs/sql.html
Hmm, I was never fond of the sql-interface of gnatcoll. I know that the
authors argue with safety, but I prefer embedded SQL.



-- 
--
Björn

Re: GNATCOLL JSON Parsing

[eduardsapotski]

Once again many thanks! I will study this. 

Re: GNATCOLL JSON Parsing

[eduardsapotski]

björn lundin, if it does not complicate you, could you answer one more question:
I often have to parse html-documents.
In Java, there is Jsoup: https://jsoup.org/
Maybe there is a similar library in the Ada-language?
Thanks.

Re: GNATCOLL JSON Parsing

[Björn Lundin]

On 2018-06-08 18:35, eduardsapotski@gmail.com wrote:
> björn lundin, if it does not complicate you, could you answer one more question:
> I often have to parse html-documents.
> In Java, there is Jsoup: https://jsoup.org/
> Maybe there is a similar library in the Ada-language?
> Thanks.
> 

I know of no such library in Ada.
The only one I can think of is xml/ada given that the html is valid
xhtml, which I guess is not the case in most situations.

Others might know though.

-- 
--
Björn

Re: GNATCOLL JSON Parsing

[gautier_niouzes]

Le vendredi 8 juin 2018 18:35:28 UTC+2, eduards...@gmail.com a écrit :
> björn lundin, if it does not complicate you, could you answer one more question:
> I often have to parse html-documents.
> In Java, there is Jsoup: https://jsoup.org/
> Maybe there is a similar library in the Ada-language?
> Thanks.

Here is a rudimentary "real-world" HTML parser:
https://sourceforge.net/projects/wasabee/
It is tolerant with faulty HTML (which is the case of 99.999% of Web pages ;-) ) and parses CSS as well.

Re: GNATCOLL JSON Parsing

[gautier_niouzes]

>    type Money is delta 0.00000001 range 0.0 .. 9_999_999_999.9;

Wait... you have a 10 Billion limit? Only ;-) ?

Re: GNATCOLL JSON Parsing

[eduardsapotski]

> Here is a rudimentary "real-world" HTML parser:
https://sourceforge.net/projects/wasabee/
> It is tolerant with faulty HTML (which is the case of 99.999% of Web pages ;-) ) and parses CSS as well. 

Thanks. That's interesting. Will it work on Linux?

Re: GNATCOLL JSON Parsing

[eduardsapotski]

> https://sourceforge.net/projects/wasabee/
Thanks! What I need I found! It remains to check how it works.

Re: GNATCOLL JSON Parsing

[eduardsapotski]

> Wait... you have a 10 Billion limit? Only ;-) ? 
Yes. I'm modest. ))

Re: GNATCOLL JSON Parsing

[R Srinivasan]

On Saturday, June 9, 2018 at 1:31:09 AM UTC-4, eduards...@gmail.com wrote:
> > https://sourceforge.net/projects/wasabee/
> Thanks! What I need I found! It remains to check how it works.

simple example at:

https://gitlab.com/RajaSrinivasan/orientation/tree/master/jsonutil

Re: GNATCOLL JSON Parsing

[eduardsapotski]

Again problem ...
I'm trying parse such result: https://api.exmo.com/v1/pair_settings/

function Get_Pair_Settings return Pair_Settings_Vector_Package.Vector is
      JSON    : Unbounded_String := Null_Unbounded_String;
      URL_Str : Unbounded_String := To_Unbounded_String("https://api.exmo.com/v1/pair_settings/");
      
      Current_JSON_Item, JSON_Main_Node : JSON_Value := Create;
      JSON_Result_Array : JSON_Array := Empty_Array; 
      
      Result  : Pair_Settings_Vector_Package.Vector;
   begin
      JSON := To_Unbounded_String(Message_Body(Get(URL => To_String(URL_Str))));
      JSON_Main_Node := Read(JSON);              
      JSON_Result_Array := Get(JSON_Main_Node);
 
      if Length(JSON_Result_Array) > 0 then
         for i in 1 .. Length(JSON_Result_Array) loop
            Put_Line("123");
         end loop;
      end if;
      
      return Result;
      
   end;

...

raised CONSTRAINT_ERROR : gnatcoll-json.adb:1328 discriminant check failed

Björn, hope only on you :))

Re: GNATCOLL JSON Parsing

[Björn Lundin]

On 2018-11-26 07:42, eduardsapotski@gmail.com wrote:
> Again problem ...
> I'm trying parse such result: https://api.exmo.com/v1/pair_settings/
> 
> 
> raised CONSTRAINT_ERROR : gnatcoll-json.adb:1328 discriminant check failed
> 
> Björn, hope only on you :))

I doubt that.

Anyway, I re-wrote towith Gnatcoll.Json; use Gnatcoll.Json;
with Aws;
with Aws.Client;
with Aws.Response;
with Ada.Text_IO; use Ada.Text_IO;
with Gnatcoll.Json; use Gnatcoll.Json;
procedure Test_Json is
  Json_Main_Node    : Json_Value := Create;
  Json_Result_Array : Json_Array := Empty_Array;
  Aws_Reply         : Aws.Response.Data;
begin
  Aws_Reply := Aws.Client.Get
  (Url          => "https://api.exmo.com/v1/pair_settings");

  Json_Main_Node := Read
  (Strm     => Aws.Response.Message_Body(Aws_Reply),
   Filename => "");
  Json_Result_Array := Get(Json_Main_Node);

  if Length(Json_Result_Array) > 0 then
    for I in 1 .. Length(Json_Result_Array) loop
      Put_Line( "123"  );
    end loop;
  end if;
end Test_Json;

But I get a
bnl@sebjlun-deb64:~/bnlbot/botstart/bot-1-0/target/bin$ ./test_json

Execution terminated by unhandled exception
raised PROGRAM_ERROR : aws-client.adb:391 finalize/adjust raised exception
Call stack traceback locations:
0x5efe79 0x407823 0x4094a8 0x7fa59ae762af 0x407208 0xfffffffffffffffe
bnl@sebjlun-deb64:~/bnlbot/botstart/bot-1-0/target/bin$ addr2line -e
test_json 0x5efe79 0x407823 0x4094a8 0x7fa59ae762af 0x407208
0xfffffffffffffffe
??:?
/home/bnl/bnlbot/botstart/bot-1-0/source/ada/local/utils/test_json.adb:11
/home/bnl/bnlbot/botstart/bot-1-0/target/objects/bot/b__test_json.adb:1051
??:0
??:?
??:0
bnl@sebjlun-deb64:~/bnlbot/botstart/bot-1-0/target/bin$


So my version crashes on AWS.Client.Get
I cannot really help you further here


Where does you client crash? what line is 1328?


-- 
--
Björn

Re: GNATCOLL JSON Parsing

[eduardsapotski]

Try not to use https: http://api.exmo.com/v1/pair_settings/

Re: GNATCOLL JSON Parsing

[eduardsapotski]

gnatcoll-json.adb:1328
-----------------------------------------------------
function Get (Val : JSON_Value) return JSON_Array is
   begin
      return Val.Data.Arr_Value.Arr;
   end Get;

Re: GNATCOLL JSON Parsing

[Simon Wright]

Björn Lundin <b.f.lundin@gmail.com> writes:

> So my version crashes on AWS.Client.Get
> I cannot really help you further here

I think this is because of SSL not being supported (I'm using the
unpatched CE 2018).

For me, GDB says

Catchpoint 1, PROGRAM_ERROR (SSL not supported.) at 0x00000001000018fe in test_json () at /Users/simon/tmp/cla/test_json.adb:12
12	   Aws_Reply := Aws.Client.Get
(gdb) c
Continuing.

Catchpoint 1, PROGRAM_ERROR (aws-client.ads:592 finalize/adjust raised exception) at 0x00000001000018fe in test_json ()
    at /Users/simon/tmp/cla/test_json.adb:12
12	   Aws_Reply := Aws.Client.Get

I triet getting the JSON file and using a file:// URL to get it locally,
but now I get a constraint error

Catchpoint 1, CONSTRAINT_ERROR (aws-url.adb:374 range check failed) at 0x00000001000018de in test_json () at /Users/simon/tmp/cla/test_json.adb:12
12	   Aws_Reply := Aws.Client.Get

which seems to be something to do with a port number?

Unfortunately the AWS in GNAT CE seems to have been compiled without
debugging symbols, so I can't get any further.

Re: GNATCOLL JSON Parsing

[eduardsapotski]

понедельник, 26 ноября 2018 г., 18:56:36 UTC+3 пользователь Simon Wright написал:
> Björn Lundin <b.f.lundin@gmail.com> writes:
> 
> > So my version crashes on AWS.Client.Get
> > I cannot really help you further here
> 
> I think this is because of SSL not being supported (I'm using the
> unpatched CE 2018).
> 
> For me, GDB says
> 
> Catchpoint 1, PROGRAM_ERROR (SSL not supported.) at 0x00000001000018fe in test_json () at /Users/simon/tmp/cla/test_json.adb:12
> 12	   Aws_Reply := Aws.Client.Get
> (gdb) c
> Continuing.
> 
> Catchpoint 1, PROGRAM_ERROR (aws-client.ads:592 finalize/adjust raised exception) at 0x00000001000018fe in test_json ()
>     at /Users/simon/tmp/cla/test_json.adb:12
> 12	   Aws_Reply := Aws.Client.Get
> 
> I triet getting the JSON file and using a file:// URL to get it locally,
> but now I get a constraint error
> 
> Catchpoint 1, CONSTRAINT_ERROR (aws-url.adb:374 range check failed) at 0x00000001000018de in test_json () at /Users/simon/tmp/cla/test_json.adb:12
> 12	   Aws_Reply := Aws.Client.Get
> 
> which seems to be something to do with a port number?
> 
> Unfortunately the AWS in GNAT CE seems to have been compiled without
> debugging symbols, so I can't get any further.

Yes, most likely a problem with ssl. I build AWS separately with support openssl. Everything is working.
But json can be obtained without ssl. Try this: http://api.exmo.com/v1/pair_settings/ 

Re: GNATCOLL JSON Parsing

[eduardsapotski]

I have already "manually" parse this json. 
But would like to have a standard beautiful solution.

Re: GNATCOLL JSON Parsing

[Simon Wright]

eduardsapotski@gmail.com writes:

> Yes, most likely a problem with ssl. I build AWS separately with
> support openssl. Everything is working.
> But json can be obtained without ssl. Try this:
> http://api.exmo.com/v1/pair_settings/

OK, a little further. I think the problem may be with

   function Get (Val : JSON_Value) return JSON_Array is
   begin
      return Val.Data.Arr_Value.Arr;           <<<< exception here
   end Get;

which is expecting Val to be of type JSON_Array_Type, i.e. an array, but
it isn't; it appears to be some sort of nested array-like structure, and
I can recognise the names from the pair_settings. You need someone who
understands GNATCOLL.JSON (there don't seem to be any examples in the
source).

Re: GNATCOLL JSON Parsing

[Björn Lundin]

On 2018-11-26 19:01, Simon Wright wrote:
> eduardsapotski@gmail.com writes:
> 
>> Yes, most likely a problem with ssl. I build AWS separately with
>> support openssl. Everything is working.

Hmm I though I had build mine with ssl. Never mind it works with http.
However I am a bit uncertain on the validity of the JSON served by the
server.

IF it is an array, it should be enclosed by '[' ']'

So - I added that , and a name for it, and then it works.
Still, my version gives it all as ONE element, so you need to fiddle a
bit with the JSON syntax. I looked at it and it looks right,
but only one element.

code below

with Aws;
with Aws.Client;
with Aws.Response;
with Ada.Text_IO; use Ada.Text_IO;
with Gnatcoll.Json; use Gnatcoll.Json;
procedure Test_Json is
  Json_Reply    : Json_Value := Create;
  Json_Item    : Json_Value := Create;
  Json_Result_Array : Json_Array := Empty_Array;
  Aws_Reply         : Aws.Response.Data;
begin
  Aws_Reply := Aws.Client.Get
  (Url          => "http://api.exmo.com/v1/pair_settings");

--  Put_Line ("{" & '"' & "list" & '"' & ":[" &
Aws.Response.Message_Body(Aws_Reply) & "]}");

  Json_Reply := Read
  (Strm     => "{" & '"' & "list" & '"' & ":[" &
Aws.Response.Message_Body(Aws_Reply) & "]}",
   Filename => "");

  if Json_Reply.Has_Field("list") then
     Json_Result_Array := Json_Reply.Get("list");
  end if;

  Put_Line ("Length(Json_Result_Array)" & Length(Json_Result_Array)'img);

  if Length(Json_Result_Array) > 0 then
    for I in 1 .. Length(Json_Result_Array) loop
      Json_Item := Get(Json_Result_Array,I);
      Put_Line( Kind(Json_Item)'img  );
    end loop;
  end if;
end Test_Json;


-- 
--
Björn

Re: GNATCOLL JSON Parsing

[eduardsapotski]

There is no limit to human stupidity. 
I spent two days studying different ada-libs to parse json.
It ended up by myself wrote function in 40 lines. Spent one hour.

   function Get_Pair_Settings return Pair_Settings_Vector_Package.Vector is
      package List_Strings is new Vectors(Count_Type, Unbounded_String);
      JSON_Text : Unbounded_String := Message_Body(Get("http://api.exmo.com/v1/pair_settings"));
      S         : Unbounded_String := Null_Unbounded_String;
      List      : List_Strings.Vector;
      Count     : Count_Type := 0;
      Result    : Pair_Settings_Vector_Package.Vector;
   begin     
      for i in 1 .. Length(JSON_Text) loop
         if Element(JSON_Text, i) /= ' ' 
           and Element(JSON_Text, i) /= '{'
           and Element(JSON_Text, i) /= '}'
           and Element(JSON_Text, i) /= '"'
         then
            if Element(JSON_Text, i) = ',' or Element(JSON_Text, i) = ':' then              
               List.Append(S);
               S := Null_Unbounded_String;
            else
               Append(S, Element(JSON_Text, i));                   
            end if;            
         end if; 
      end loop;
      List.Append(S);
      
      while Count <= List.Length - 1 loop
         declare
            PS : Pair_Setting;
         begin
            PS.Pair_ID      := List.Element(Count);
            PS.Min_Quantity := Money_Type'Value(To_String(List.Element(Count + 2)));
            PS.Max_Quantity := Money_Type'Value(To_String(List.Element(Count + 4)));
            PS.Min_Price    := Money_Type'Value(To_String(List.Element(Count + 6)));
            PS.Max_Price    := Money_Type'Value(To_String(List.Element(Count + 8)));
            PS.Max_Amount   := Money_Type'Value(To_String(List.Element(Count + 10)));
            PS.Min_Amount   := Money_Type'Value(To_String(List.Element(Count + 12)));
            
            Result.Append(PS);
         end;
         Count := Count + 13;
      end loop;
     
      return Result; 
   end;

Re: GNATCOLL JSON Parsing

[Per Sandberg]

You may have a look at:
https://github.com/persan/gnatcoll-json/tree/master/examples/web-trader

The repo not in a 100% stable state at the moment but i think you will 
get the idea.


/P



On 6/8/18 6:35 PM, eduardsapotski@gmail.com wrote:
> björn lundin, if it does not complicate you, could you answer one more question:
> I often have to parse html-documents.
> In Java, there is Jsoup: https://jsoup.org/
> Maybe there is a similar library in the Ada-language?
> Thanks.
> 

Re: GNATCOLL JSON Parsing

[Olivier Henley]

>> The problem is that the authors do not provide practical examples of using 
>> libraries. It is difficult for a beginner to understand what to do.

I totally agree. It is like some authors

a) do not want others to use their stuff
b) think that Ada does not need new kids around
c) think you need to pass some kind of implicit test to be 'blessed' in using their stuff
d) want some form of 'extended special attention' from strangers

... It is always related to some form of God complex ...

I cant stand that kind of 'counter-productive' attitude. It is not a competence issue, its just people want to get going, period. Personally I don't have the time to get in the head of some authors, don't have the time to dive in their code (from the lib perspective I'm a client not a contributor) and frankly not actually interested to understand until I know at least what they publish actually works. So many projects are broken right from the start.

Ada, being nowhere on the popularity charts should really streamline its 'tutorials' offering for surviving. Some of the old guard seems to be completely disconnected from that fact. No wonder something like Rust is taking over the world... when we already have Ada.

Re: GNATCOLL JSON Parsing

[Dmitry A. Kazakov]

On 2018-12-03 16:02, Olivier Henley wrote:
>>> The problem is that the authors do not provide practical examples of using
>>> libraries. It is difficult for a beginner to understand what to do.
> 
> I totally agree. It is like some authors
> 
> a) do not want others to use their stuff
> b) think that Ada does not need new kids around
> c) think you need to pass some kind of implicit test to be 'blessed' in using their stuff
> d) want some form of 'extended special attention' from strangers
> 
> ... It is always related to some form of God complex ...

Authors simply cannot provide any good getting started samples:

1. Their perspective is massively distorted by the level of knowledge of 
the software internals. Things that appear them obvious can turn a total 
mystery for a beginner.

2. Authors know surprisingly little about the potential application of 
their software. They have some general guesses about the purpose, but 
nothing substantial about concrete uses. They usually get quite 
astonished (horrified? (:-)) the ways end users apply their software.

3. Writing introductory material requires a great deal of teaching 
talent to write. Most authors lack it. It is a different job and 
different qualification.

At best an average author can write a decent reference manual, nothing 
more. Ergo, it must be a third person to write samples and introductory 
stuff.
> I cant stand that kind of 'counter-productive' attitude. It is not a competence issue, its just people want to get going, period. Personally I don't have the time to get in the head of some authors, don't have the time to dive in their code (from the lib perspective I'm a client not a contributor) and frankly not actually interested to understand until I know at least what they publish actually works. So many projects are broken right from the start.
> 
> Ada, being nowhere on the popularity charts should really streamline its 'tutorials' offering for surviving. Some of the old guard seems to be completely disconnected from that fact. No wonder something like Rust is taking over the world... when we already have Ada.

They have a much bigger community with much more people available to 
write independent manuals, samples, blogs, make youtube videos. They 
have lost of people eager to describe their experience with deploying 
particular software for particular projects, we do not.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: GNATCOLL JSON Parsing

[briot.emmanuel]

> I cant stand that kind of 'counter-productive' attitude. It is not a competence issue, its just people want to get going, period. Personally I don't have the time to get in the head of some authors, don't have the time to dive in their code (from the lib perspective I'm a client not a contributor) and frankly not actually interested to understand until I know at least what they publish actually works. So many projects are broken right from the start.

Interesting rant. Could you point us to some of the libraries you had made available to others and that have great documentation ? You have a github account with no repository... Critics are easy.

Writing documentation is really a difficult things. Some things are obvious to anyone who has used Ada before. Should we keep repeating them in documentation ? For instance, one of the difficult things for new Ada users are functions returning string. Should we always have examples how to use declare blocks ? Definitely not.

So you are not a contributor in the sense "please provide a patch", but you are definitely a client in the sense of "please kindly provide additional information on how to use such and such API". That's the price to pay to use open-source software.

That said, there certainly are libraries that could use some higher-level documentation. Just ask their authors to do it, explaining your difficulties.
(hint: if you start by telling them they have a God complex, you might as well not bother sending a message).

Emmanuel

Author of a large part of GNATCOLL, but not GNATCOLL.JSON itself... And I am sure you could have similar critics for those other parts.

Re: GNATCOLL JSON Parsing

[Simon Wright]

eduardsapotski@gmail.com writes:

> Again problem ...
> I'm trying parse such result: https://api.exmo.com/v1/pair_settings/
>
>    function Get_Pair_Settings return Pair_Settings_Vector_Package.Vector is
>       JSON    : Unbounded_String := Null_Unbounded_String;
>       URL_Str : Unbounded_String := To_Unbounded_String("https://api.exmo.com/v1/pair_settings/");
>
>       Current_JSON_Item, JSON_Main_Node : JSON_Value := Create;
>       JSON_Result_Array : JSON_Array := Empty_Array;
>
>       Result  : Pair_Settings_Vector_Package.Vector;
>    begin
>       JSON := To_Unbounded_String(Message_Body(Get(URL => To_String(URL_Str))));
>       JSON_Main_Node := Read(JSON);
>       JSON_Result_Array := Get(JSON_Main_Node);
>
>       if Length(JSON_Result_Array) > 0 then
>          for i in 1 .. Length(JSON_Result_Array) loop
>             Put_Line("123");
>          end loop;
>       end if;
>
>       return Result;
>
>    end;
>
> ...
>
> raised CONSTRAINT_ERROR : gnatcoll-json.adb:1328 discriminant check failed

I finally managed to make this work.

It turns out that the JSON returned from that query (actually, the http:
version) is a single JSON object, where the contents are fields, not
array elements (I may have the nomenclature wrong there).

This code

with Aws;
with Aws.Client;
with Aws.Response;
with Ada.Text_IO; use Ada.Text_IO;
with GNATCOLL.JSON; use GNATCOLL.JSON;
with Ada.Strings.Unbounded; use Ada.Strings.Unbounded;
procedure Test_JSON is
   Aws_Reply         : Aws.Response.Data;
   JSON_Main_Node    : JSON_Value := Create;

   procedure Report_Outer_Field_Name (Name : Utf8_String; Value : JSON_Value);
   procedure Report_Inner_Field_Name (Name : Utf8_String; Value : JSON_Value);

   procedure Report_Outer_Field_Name (Name : Utf8_String; Value : JSON_Value)
   is
   begin
      Put_Line ("field: " & Name);
      Map_JSON_Object (Value, Report_Inner_Field_Name'Access);
   end Report_Outer_Field_Name;

   procedure Report_Inner_Field_Name (Name : Utf8_String; Value : JSON_Value)
   is
   begin
      Put_Line ("    field: " & Name & ", kind: " & Kind (Value)'Image);
   end Report_Inner_Field_Name;

begin
   Aws_Reply := Aws.Client.Get
     (Url          => "http://api.exmo.com/v1/pair_settings");

   JSON_Main_Node := Read
     (Strm     => Unbounded_String'(Aws.Response.Message_Body(Aws_Reply)),
      Filename => "");

   Map_JSON_Object (JSON_Main_Node, Report_Outer_Field_Name'Access);

end Test_JSON;

results in

$ ./test_json 
field: BTC_USD
    field: min_quantity, kind: JSON_STRING_TYPE
    field: max_quantity, kind: JSON_STRING_TYPE
    field: min_price, kind: JSON_STRING_TYPE
    field: max_price, kind: JSON_STRING_TYPE
    field: max_amount, kind: JSON_STRING_TYPE
    field: min_amount, kind: JSON_STRING_TYPE
field: BTC_EUR
    field: min_quantity, kind: JSON_STRING_TYPE
    field: max_quantity, kind: JSON_STRING_TYPE
    field: min_price, kind: JSON_STRING_TYPE
    field: max_price, kind: JSON_STRING_TYPE
    field: max_amount, kind: JSON_STRING_TYPE
    field: min_amount, kind: JSON_STRING_TYPE
...
field: BTG_ETH
    field: min_quantity, kind: JSON_STRING_TYPE
    field: max_quantity, kind: JSON_STRING_TYPE
    field: min_price, kind: JSON_STRING_TYPE
    field: max_price, kind: JSON_STRING_TYPE
    field: max_amount, kind: JSON_STRING_TYPE
    field: min_amount, kind: JSON_STRING_TYPE

Struggles:

As noted several times, this was all a bit mystifying

The default build of GNATCOLL has no debug symbols (both static & shared
libraries)

Re: GNATCOLL JSON Parsing

[Olivier Henley]

First, yeah I may have been too harsh but still...

> 1. Their perspective is massively distorted by the level of knowledge of 
> the software internals. Things that appear them obvious can turn a total 
> mystery for a beginner.

You are right. But hey, you need to be sleeping under a rock for not providing meaningful example using JSON reader/writer. The thing is ubiquitous for any backend server these days... and for the past 10 years.  

> 2. Authors know surprisingly little about the potential application of 
> their software. They have some general guesses about the purpose, but 
> nothing substantial about concrete uses. They usually get quite 
> astonished (horrified? (:-)) the ways end users apply their software.

I suppose.

> 3. Writing introductory material requires a great deal of teaching 
> talent to write. Most authors lack it. It is a different job and 
> different qualification.

Ok. To those in needs, write to me, I have plenty of ideas.
 
> They have a much bigger community with much more people available to 
> write independent manuals, samples, blogs, make youtube videos. They 
> have lost of people eager to describe their experience with deploying 
> particular software for particular projects, we do not.

Well, Rust is much younger than Ada. The traction comes from 1) 'modern/trendy' ubiquitous projects and 2) Good doc, nice 'tutorials'. Something like LearnAda, which is great by the way, exists in those new languages for a long time now.  

Re: GNATCOLL JSON Parsing

[Olivier Henley]

>> Interesting rant. Could you point us to some of the libraries you had made >> available to others and that have great documentation ? You have a github >> account with no repository... Critics are easy.

1) Responsibilities comes with providing; it is a statement. If I did not 'published' much is exactly because I prefer to not share, than share half-baked stuff (missing meaningful useage contexts). There is also the fact that I am doing Ada during the week-ends, when I can, and I do not find myself fluent enough to tackle the kind of projects I wan to undertake.

2) Man, I would practice due diligence before biting. My https://github.com/ohenley has 12 repos.

i. The most complete Awesome Ada page ... by far is my last baby, WIP.
ii. A simple native Mouse Capture in Ada. Toy project grabbing the coords of a PC mouse and outputting it to the terminal. Code in is very straightforward, clean and naive... on purpose. Not using any 'mad' dependencies.
iii. Aglw, a replacement for GLFW3 in Ada, with a demo, the git project Aglw-demos. The demo is the doc/tutorial... sir.
iv. Los33, a cookbook to build a crypto mining rig on Linux
v. A readme template, to help people who wants to have a 'streamlined' readme with most sections appropriate for a newcomer to understand the project

*All my code repos are in Ada, to show a commitment.

>> Writing documentation is really a difficult things. Some things are obvious >> to anyone who has used Ada before. Should we keep repeating them in 
>> documentation ? For instance, one of the difficult things for new Ada users >> are functions returning string.

No but you could definitely point out a clear resource that does it (a url staring by http://...), what about that?

>> So you are not a contributor in the sense "please provide a patch", but you >> are definitely a client in the sense of "please kindly provide additional >> information on how to use such and such API". That's the price to pay to 
>> use open-source software.

Look, if you want to put your head in the sand fine. Many language communities are more structured than Ada and THIS is exactly why they flourish. The new efforts by Adacore are perfect (the blog, LearnAda, Make with Ada, a modern look of their pages etc)

But if you want to bring back Ada strong we will have to push harder. GoF Patterns in Ada, games, more blogs, super robust JSON, bindings to Graph databse (eg DGraph, Noe4j) etc

>> That said, there certainly are libraries that could use some higher-level >> documentation. Just ask their authors to do it, explaining your 
>> difficulties.

Now, I am fine, I surf without help most of the time. But when I got in Ada I was pissed! 

>> (hint: if you start by telling them they have a God complex, you might as >> well not bother sending a message).

Seriously!? I really though I could approach people this way... 

> Emmanuel

I said SOME of the old guard, so if you feel concerned it is on you, sorry.

But, all that said, I know I was a bit too harsh. I would really like to have Ada praised everywhere. That is why I am shaking the tree.

Re: GNATCOLL JSON Parsing

[briot.emmanuel]

> 2) Man, I would practice due diligence before biting. My https://github.com/ohenley has 12 repos.

I was talking about repositories with some doc.
"amoc" -> no comment, except for copyright header
"aglw-demos" -> no comment either.
The other Ada repositories are all forked from existing projects.

I did check before I commented...


> ii. A simple native Mouse Capture in Ada. Toy project grabbing the coords of a PC mouse and outputting it to the terminal. Code in is very straightforward, clean and naive... on purpose. Not using any 'mad' dependencies.

In that single sentence, there are more comment than in all the repository.
You actually fell to the same criticism you made earlier: "code is straightforward". For you as the author, I have no doubt. For someone looking at you repository => no idea what it does, except it has something to do with "mouse". There is no high-level description, no code example,...

I understand this is work-in-progress, and I am glad to see more Ada repositories. My point (and I believe Dmitry's, both of us having shared a number of repositories) is exactly that it is so easy for authors to assume something is "straightforward" (to quote you).

There is always someone for whom this isn't the case. That person needs to report on that (and this thread was initially a good example of that: a problem with GNATCOLL.JSON, which might eventually get reported to the authors), or ask questions (more Ada questions on stackoverflow cannot hurt, though it will be more work for Simon).
The author of the library of course get to decide where to draw the line between "obvious, doesn't need doc", and "yes, let me add some doc".


> No but you could definitely point out a clear resource that does it (a url staring by http://...), what about that?

For the particular example I sent that resource is called the Ada Reference Manual, though any book on Ada also have that info. I must admit that I am not current on getting-started resources for Ada.

> But if you want to bring back Ada strong we will have to push harder. GoF Patterns in Ada, games, more blogs, super robust JSON, bindings to Graph databse (eg DGraph, Noe4j) etc

Every little progress will be nice. Don't think though that Ada will become mainstream again, that time has passed, and a lot of people just prefer to look at the new and shiny.

> Seriously!? I really though I could approach people this way... 

I know this is ironic, but you would not believe the number of request for help that have exactly that tone.

> I said SOME of the old guard, so if you feel concerned it is on you, sorry.

I don't mind being the old, middle-aged or new guard, doesn't matter to me.

> But, all that said, I know I was a bit too harsh. I would really like to have Ada praised everywhere. That is why I am shaking the tree.

That won't work. What works is publishing things (Code, tutorials, doc,...) Why don't you start collecting the information from comp.lang.ada threads and make that available in a nice form, for instance ?
Or publish resources for Ada beginners (the "old guard" cannot help here, we just assume some of the things are obvious).

Your list of awesome-ada seems like it could be a good starting point. But if you don't advertise it, this is wasted effort. And as soon as you advertise it, you will get comments from people "your list is stupid because it doesn't include such and such"... Then you will become part of the middle-aged guard.

Emmanuel

Re: GNATCOLL JSON Parsing

[Olivier Henley]

>> I was talking about repositories with some doc.
>> "amoc" -> no comment, except for copyright header

Ok, we are not on the same page here. 
The Amoc readme is super clear, no need for orthodox doc! 

- Amoc has a screenshot, what you see is what you get.
- Amoc has build procedure using gprbuild. The exact command line to build!
- Amoc states the prerequisites, win32 and a Gnat compiler.
- Amoc states a usage. "Launch the exe from the build". Cannot be clearer.
- Amoc acknowlegde others work as a lead for further study.

So you refer the ARM for people struggling with strings, but my repo is open to 'interpretation'... you really want to cover this ground. Please.

>> "aglw-demos" -> no comment either.

You know that the github readmes as being doc is a thing these days..? Now you know. Everything is there in the readme of Aglw-demos... for a WIP. The src is 3 files, and the demo goal is exactly to offer a 'complete real case usage' of the aglw lib API.

I never said I should completely hold the hand of a newbie. You refer to obvious things, underline the tricky parts, you handout some meaningful use case, in real code, and a normal person should be fine.

I do not believe in 90's doc, a la MySQL, where you have to read dozens and dozens of official doc to make a simple query or just to install it. No, what is proper doc these days, is hands-on toy demo. You build it, inspect a couple of files, you get the picture, c&p the basic code and expand from that. You can keep rolling.

Eg, gprbuild is awesome, but the doc is somewhat too much. I learn more from inspecting others use of it that from the official doc.

Do you see where I go? That was the point of the OP, I think, he just needed an 'almost' complete real use case, showed off. He would have picked from there not problem. 

You want a good example, check Imgui (https://github.com/ocornut/imgui)

>> I did check before I commented...

To me, not acknowledging the role of the readmes as the 'official doc' or proper 'comments' is bad faith from your part.

>> In that single sentence, there are more comment than in all the repository.
>> You actually fell to the same criticism you made earlier.

Again, use the readmes. You know that, I know your repos. Some are dead clear others are just empty. Look in your yard before picking on me.

>> There is always someone for whom this isn't the case. That person needs to >> report on that (and this thread was initially a good example of that: a 
>> problem with GNATCOLL.JSON,"

Talking of which. I started a project using GNATCOLL.JSON to unmarshal server infos comming from Binance, one of, if not the main crypto exchange. 

To my surprise I kept getting, errors, crash or empty data (I do not recall exactly), with no leads what so ever. After some endless search on where the culprit was I decided to dig in GNATCOLL.JSON code. You bet, GNATCOLL.JSON did not handle word starting with a capital letter... yeah! And you bet it did not report what is going wrong. 

As you say we cannot pinpoint where ones knowledge starts/ends! 

>> For the particular example I sent that resource is called the Ada Reference >> Manual.

A bit of disconnection here. Learning ways of bread and butter Ada code (strings manip) from the ARM is like studying quantum mechanics to understand why you should refrigerate milk. I'm joking ... almost not.

>> Every little progress will be nice. Don't think though that Ada will become >> mainstream again, that time has passed, and a lot of people just prefer to >> look at the new and shiny.

C++ has never been so hyped! They work for the hype, they believe in it, they play the 'real man' game (vs the script kiddies culture) without being mundane and it works. 

>> I know this is ironic, but you would not believe the number of request for >> help that have exactly that tone.

Tell them to go to hell.

>> That won't work. What works is publishing things (Code, tutorials, doc,...) 

I know publishing things is the key, but a reality check is always good. The OP is right.  

>> Your list of awesome-ada seems like it could be a good starting point. But >> if you don't advertise it, this is wasted effort. 

I did not advertise it because it misses obvious things, see the TODO at the end.  

>> And as soon as you advertise it, you will get comments from people "your 
>> list is stupid because it doesn't include such and such"... Then you will >> become part of the middle-aged guard.

I have no problem handling such people, and it is no argument to not properly engage in 'demonstrating' how our projects/libs etc works in a modern way.

Re: GNATCOLL JSON Parsing

[Dmitry A. Kazakov]

On 2018-12-03 19:45, Olivier Henley wrote:
> First, yeah I may have been too harsh but still...
> 
>> 1. Their perspective is massively distorted by the level of knowledge of
>> the software internals. Things that appear them obvious can turn a total
>> mystery for a beginner.
> 
> You are right. But hey, you need to be sleeping under a rock for not providing meaningful example using JSON reader/writer. The thing is ubiquitous for any backend server these days... and for the past 10 years.

Why on Earth do you need JSON parser or any examples for? It is trivial 
to parse JSON per hand. Whatever Ada interface will be far more complex 
than direct parsing since Ada has no tuples. And it will be incredible 
inefficient requiring dynamic memory allocation. Why GNATCOLL has this 
stuff in it is beyond me.

>> 3. Writing introductory material requires a great deal of teaching
>> talent to write. Most authors lack it. It is a different job and
>> different qualification.
> 
> Ok. To those in needs, write to me, I have plenty of ideas.

You are welcome to contribute, really.

>> They have a much bigger community with much more people available to
>> write independent manuals, samples, blogs, make youtube videos. They
>> have lost of people eager to describe their experience with deploying
>> particular software for particular projects, we do not.
> 
> Well, Rust is much younger than Ada. The traction comes from 1) 'modern/trendy' ubiquitous projects and 2) Good doc, nice 'tutorials'. Something like LearnAda, which is great by the way, exists in those new languages for a long time now.

It comes from advertising, desire to try something "new", lack of 
fundamental education and common sense, brainwashing in the colleges...

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: GNATCOLL JSON Parsing

[Olivier Henley]

> Again, use the readmes. You know that, I know your repos. Some are dead clear others are just empty. Look in your yard before picking on me.

Oh, sorry! I was talking about Fabien Chouteau repos. I mixed you both, my bad.

Re: GNATCOLL JSON Parsing

[Olivier Henley]

>> It comes from advertising, desire to try something "new", lack of 
>> fundamental education and common sense, brainwashing in the colleges...

I like your style. ;)

Re: GNATCOLL JSON Parsing

[briot.emmanuel]

> I do not believe in 90's doc, a la MySQL, where you have to read dozens and dozens of official doc to make a simple query or just to install it. No, what is proper doc these days, is hands-on toy demo.

Indeed, we have a very serious disconnect here.
Documentation is nowhere near toy demo for me.

Re: GNATCOLL JSON Parsing

[gautier_niouzes]

On Monday, December 3, 2018 at 9:48:59 PM UTC+1, Olivier Henley wrote:

> I do not believe in 90's doc, a la MySQL, where you have to read dozens and dozens of official doc to make a simple query or just to install it. No, what is proper doc these days, is hands-on toy demo. You build it, inspect a couple of files, you get the picture, c&p the basic code and expand from that. You can keep rolling.

IMHO, both (doc and demos) are necessary.
I elaborate a bit (experience gathered over years, including from more or less polite but mostly helpful feedback):

1) the demo tree in 2 layers:
1.1) a bunch of minimal toy demos (max 50 lines) and pointers to 1.2
1.2) a few larger applications (ideally, useful)

2) the doc tree in 3 layers:
2.1) readme, with a short summary and a pointer to 2.2
2.2) a larger doc in one file with build instructions and pointers to 2.3
2.3) technical documentation

The demos are crucial if you want to have people have any interest at your project.
Even in the 80's (so, before the Web and GitHub ;-) ), successful software had lots of demos.
The demos need to build and run out-of-the-box (or the author(s) need to explain very clearly what are the dependencies to other libraries).
The test process includes packaging the project, copying to another computer, unpack, build and run as if you (the author) were a newcomer - of course this is very difficult to do it yourself, but better than not trying it (there are some tricks to improve randomness in that phase)...

The demos show much of a project's health and portability - outside the computer it has been developed on!

Disclaimer: probably I forget to follow these rules. Feedback is always welcome...

Re: GNATCOLL JSON Parsing

[Olivier Henley]

>> Indeed, we have a very serious disconnect here.
>> Documentation is nowhere near toy demo for me.

I understand the need for 'in-depth' documentation for projects like MySQL, Gprbuild, PolyORB etc. Nevertheless, some demos, from toy to complex are essential. They need to work and I do not need to fight my way to learn from them.

Do not get me wrong: the response should be proportional to the signal.

Re: GNATCOLL JSON Parsing

[Olivier Henley]

> IMHO, both (doc and demos) are necessary.
> 1) the demo tree in 2 layers:
> 1.1) a bunch of minimal toy demos (max 50 lines) and pointers to 1.2
> 1.2) a few larger applications (ideally, useful)

> 2) the doc tree in 3 layers:
> 2.1) readme, with a short summary and a pointer to 2.2
> 2.2) a larger doc in one file with build instructions and pointers to 2.3
> 2.3) technical documentation

This is ideal.