ZLIB_ERROR with AWS

ZLIB_ERROR with AWS

[Okasu]

Hello, i'm getting "raised ZLIB.ZLIB_ERROR : DATA_ERROR: incorrect header check"
with this code:

-----
with AWS.Client;   use AWS.Client;
with AWS.Response; use AWS.Response;

procedure Test
is
  Limit   : Content_Range := (1, 3000);
  Content : Data := Get (URL => "http://www.reddit.com/", Data_Range => Limit);
begin
  null;
end Test;
-----

With other(Non https) links it works fine, but not with reddit.

This code works fine with reddit:
-----
with AWS.Client;   use AWS.Client;
with AWS.Response; use AWS.Response;

procedure Test
is
  Content : Data := Get ("http://www.reddit.com/");
begin
  null;
end Test;
-----

Questions:
1) How can i use Get with HTTPS
   I'm getting "raised CONSTRAINT_ERROR : aws-url.adb:338 range check
   failed" after attempt to get certificate from host and use as Get
   Certificate argument. Code:
   -----
   Cert : AWS.Net.SSL.Certificate.Object :=
     Get_Certificate (Create (AWS.URL.Host (AWS.URL.Parse (Link))));
   -----
2) How can i use Data_Range with redditlike resources?

Re: ZLIB_ERROR with AWS

[Okasu]

On 2012-10-20, Okasu <oka.sux@gmail.com> wrote:
> 1) How can i use Get with HTTPS
>    I'm getting "raised CONSTRAINT_ERROR : aws-url.adb:338 range check
>    failed" after attempt to get certificate from host and use as Get
>    Certificate argument. Code:
>    -----
>    Cert : AWS.Net.SSL.Certificate.Object :=
>      Get_Certificate (Create (AWS.URL.Host (AWS.URL.Parse (Link))));
>    -----

I'm trying to get certificate because plain Get ("https://url")
produces "PROGRAM_ERROR: aws-client.adb:408 finalize/adjust raised
exception".
But not sure if it help.

openssl 1.0.1

Re: ZLIB_ERROR with AWS

[Dmitry A. Kazakov]

On Sat, 20 Oct 2012 07:23:54 +0000 (UTC), Okasu wrote:

> On 2012-10-20, Okasu <oka.sux@gmail.com> wrote:
>> 1) How can i use Get with HTTPS
>>    I'm getting "raised CONSTRAINT_ERROR : aws-url.adb:338 range check
>>    failed" after attempt to get certificate from host and use as Get
>>    Certificate argument. Code:
>>    -----
>>    Cert : AWS.Net.SSL.Certificate.Object :=
>>      Get_Certificate (Create (AWS.URL.Host (AWS.URL.Parse (Link))));
>>    -----
> 
> I'm trying to get certificate because plain Get ("https://url")
> produces "PROGRAM_ERROR: aws-client.adb:408 finalize/adjust raised
> exception".
> But not sure if it help.

No, it looks like a typical problem of the clean up involving controlled
types. That is when an exception is propagated amidst of some process
involving controlled objects in possibly not a coherent state. So that when
the propagation starts kicking them out of existence a call Finalize may
fail. That turns the original exception into Program_Error. This stuff is
barely testable, because one cannot foresee and emulate all possible
exceptions at all possible places. I bet that almost any Ada program has
issues of this kind. So, it is a bug, but not the original problem.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: ZLIB_ERROR with AWS

[Okasu]

On 2012-10-20, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
> No, it looks like a typical problem of the clean up involving controlled
> types. That is when an exception is propagated amidst of some process
> involving controlled objects in possibly not a coherent state. So that when
> the propagation starts kicking them out of existence a call Finalize may
> fail. That turns the original exception into Program_Error. This stuff is
> barely testable, because one cannot foresee and emulate all possible
> exceptions at all possible places. I bet that almost any Ada program has
> issues of this kind. So, it is a bug, but not the original problem.

What bug are you talking about? Issue with zlib or finalize/adjust with HTTPS
issue? Can you suggest workaround (without involving another
language)?

Re: ZLIB_ERROR with AWS

[Dmitry A. Kazakov]

On Sat, 20 Oct 2012 08:32:02 +0000 (UTC), Okasu wrote:

> On 2012-10-20, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
>> No, it looks like a typical problem of the clean up involving controlled
>> types. That is when an exception is propagated amidst of some process
>> involving controlled objects in possibly not a coherent state. So that when
>> the propagation starts kicking them out of existence a call Finalize may
>> fail. That turns the original exception into Program_Error. This stuff is
>> barely testable, because one cannot foresee and emulate all possible
>> exceptions at all possible places. I bet that almost any Ada program has
>> issues of this kind. So, it is a bug, but not the original problem.
> 
> What bug are you talking about? Issue with zlib or finalize/adjust with HTTPS
> issue? 

Yes, that should never happen.

> Can you suggest workaround (without involving another
> language)?

It is likely not the original problem, it is an induced one:

A) There was an exception #1 which started winding the stack up (the
original issue)

B) Upon that in a Finalize of a controlled object happened an exception #2
(which is a secondary bug). Quite frequently this happens very lately when
the compiler kills strayed controlled objects at the library level. In that
case there is would be no stack trace at all.

C) #2 was not handled in Finalize, so the compiler converted it to the
exception #3, Program_Error, obscuring the things completely.

As for workarounds that depends on whether #1 was legitimate and how you
could prevent it from happening. Your first step could be to track #1 down.

If you want that and have access to the sources, you could modify the code
of the offending Finalize (where #2 propagates from) by adding at its
beginning something like:

   procedure Finalize (...) is
   begin
      Ada.Text_IO.Put_Line
      (  "#1 was:"
      &  Ada.Exceptions.Exception_Information
         (  GNAT.Most_Recent_Exception.Occurrence
      )  );
      ...

at the end of the Finalize you add:

      ...
   exception
      when Error : others =>
         Ada.Text_IO.Put_Line
         (  "#2 was:"
         & Ada.Exceptions.Exception_Information (Error)
         );
         raise;
   end Finalize;

Now in the output there should be lines "#1 was:" followed by "#2 was:",
which could give a hint what is going on.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: ZLIB_ERROR with AWS

[Okasu]

On 2012-10-20, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
> If you want that and have access to the sources, you could modify the code
> of the offending Finalize (where #2 propagates from) by adding at its
> beginning something like:
>
>    procedure Finalize (...) is
>    begin
>       Ada.Text_IO.Put_Line
>       (  "#1 was:"
>       &  Ada.Exceptions.Exception_Information
>          (  GNAT.Most_Recent_Exception.Occurrence
>       )  );
>       ...
>
> at the end of the Finalize you add:
>
>       ...
>    exception
>       when Error : others =>
>          Ada.Text_IO.Put_Line
>          (  "#2 was:"
>          & Ada.Exceptions.Exception_Information (Error)
>          );
>          raise;
>    end Finalize;
>
> Now in the output there should be lines "#1 was:" followed by "#2 was:",
> which could give a hint what is going on.

I made it (took ~1 hour to recompile AWS),  and here is output:
-----
#1 was:Exception name: PROGRAM_ERROR
Message: SSL not supported.

#2 was:Exception name: PROGRAM_ERROR
Message: SSL not supported.


raised PROGRAM_ERROR : aws-client.adb:290 finalize/adjust raised exception
-----

Why it tells me that SSL not supported?
I have openssl installed.

Re: ZLIB_ERROR with AWS

[Pascal Obry]

Le 20/10/2012 22:46, Okasu a �crit :

> Why it tells me that SSL not supported?
> I have openssl installed.

It must be selected at setup time:

   $ make SOCKET=openssl setup
   $ make

Otherwise AWS is built without SSL support.

-- 

  Pascal Obry /  Magny Les Hameaux (78)

  The best way to travel is by means of imagination

  http://v2p.fr.eu.org
  http://www.obry.net

  gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: ZLIB_ERROR with AWS

[Okasu]

On 2012-10-20, Pascal Obry <pascal@obry.net> wrote:
> Le 20/10/2012 22:46, Okasu a écrit :
>
>> Why it tells me that SSL not supported?
>> I have openssl installed.
>
> It must be selected at setup time:
>
>    $ make SOCKET=openssl setup
>    $ make
>
> Otherwise AWS is built without SSL support.
>

Oh, thanks a lot!
Maybe there is some catch like this with zlib issue too?

Re: ZLIB_ERROR with AWS

[Pascal Obry]

Le 20/10/2012 23:15, Okasu a écrit :
> Oh, thanks a lot!
> Maybe there is some catch like this with zlib issue too?

With Zlib you can tell if the one from the system is to be used or not.

   $ make ZLIB=false setup

Will force AWS to build it's own version of Zlib.

-- 

  Pascal Obry /  Magny Les Hameaux (78)

  The best way to travel is by means of imagination

  http://v2p.fr.eu.org
  http://www.obry.net

  gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: ZLIB_ERROR with AWS

[Okasu]

On 2012-10-20, Pascal Obry <pascal@obry.net> wrote:
> Le 20/10/2012 23:15, Okasu a écrit :
>> Oh, thanks a lot!
>> Maybe there is some catch like this with zlib issue too?
>
> With Zlib you can tell if the one from the system is to be used or not.
>
>    $ make ZLIB=false setup
>
> Will force AWS to build it's own version of Zlib.
>
Great, thanks again.

Re: ZLIB_ERROR with AWS

[Okasu]

On 2012-10-20, Okasu <oka.sux@gmail.com> wrote:
> On 2012-10-20, Pascal Obry <pascal@obry.net> wrote:
>> Le 20/10/2012 23:15, Okasu a écrit :
>>> Oh, thanks a lot!
>>> Maybe there is some catch like this with zlib issue too?
>>
>> With Zlib you can tell if the one from the system is to be used or not.
>>
>>    $ make ZLIB=false setup
>>
>> Will force AWS to build it's own version of Zlib.
>>
> Great, thanks again.

I've make AWS with "make SOCKET=openssl ZLIB=false setup" and SSL works
fine now. But "raised ZLIB.ZLIB_ERROR : DATA_ERROR: incorrect header check"
still persists. Any suggestions?

>    procedure Finalize (...) is
>    begin
>       Ada.Text_IO.Put_Line
>       (  "#1 was:"
>       &  Ada.Exceptions.Exception_Information
>          (  GNAT.Most_Recent_Exception.Occurrence
>       )  );
>       ...
>
> at the end of the Finalize you add:
>
>       ...
>    exception
>       when Error : others =>
>          Ada.Text_IO.Put_Line
>          (  "#2 was:"
>          & Ada.Exceptions.Exception_Information (Error)
>          );
>          raise;
>    end Finalize;

With zlib case gives nothing but same error.

Re: ZLIB_ERROR with AWS

[Pascal Obry]

On which platform?

If Windows are you using Cygwin toolset?

In this case you want to make sure the Zlib DLL is not used to run the
application. If not on Windows, no idea, I've never seen this error on
GNU/Linux for example.

-- 

  Pascal Obry /  Magny Les Hameaux (78)

  The best way to travel is by means of imagination

  http://v2p.fr.eu.org
  http://www.obry.net

  gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: ZLIB_ERROR with AWS

[Marius Amado-Alves]

I have AWS working on Windows sans zlib errors so far, but no ssl. I have published the install process here, maybe you'll find it useful.

http://mariusamadoalves.blogspot.pt/2012/09/installing-aws-on-windows.html

Re: ZLIB_ERROR with AWS

[Okasu]

I'm using Linux and by now i have to use this ugly workaround:

-----
      function Get_Body (URL : Unbounded_String) return String
      is
         Limit : constant Content_Range := (1, 3000);
      begin
         return Message_Body
           (Get (URL => To_String (URL), Data_Range => Limit));
      exception
         when Zlib_Error =>
            return Message_Body (Get (To_String (URL)));
      end Get_Body;
-----

It's miserable.

Re: ZLIB_ERROR with AWS

[Pascal Obry]

Le 21/10/2012 22:44, Okasu a �crit :
> It's miserable.

Maybe because the Web site does not support partial content downloading.
Actually if you remove Data_Range it is working, right?

-- 

  Pascal Obry /  Magny Les Hameaux (78)

  The best way to travel is by means of imagination

  http://v2p.fr.eu.org
  http://www.obry.net

  gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: ZLIB_ERROR with AWS

[Okasu]

On 2012-10-22, Pascal Obry <pascal@obry.net> wrote:
> Le 21/10/2012 22:44, Okasu a écrit :
>> It's miserable.
>
> Maybe because the Web site does not support partial content downloading.
> Actually if you remove Data_Range it is working, right?
>

Right.

Re: ZLIB_ERROR with AWS

[Pascal Obry]

Le 22/10/2012 10:08, Okasu a écrit :
> Right.

So that's just expected :)

A better error message could be given though.

-- 

  Pascal Obry /  Magny Les Hameaux (78)

  The best way to travel is by means of imagination

  http://v2p.fr.eu.org
  http://www.obry.net

  gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: ZLIB_ERROR with AWS

[darkestkhan]

On Saturday, October 20, 2012 8:46:31 PM UTC, Okasu wrote:
> On 2012-10-20, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
> 
> > If you want that and have access to the sources, you could modify the code
> 
> > of the offending Finalize (where #2 propagates from) by adding at its
> 
> > beginning something like:
> 
> >
> 
> >    procedure Finalize (...) is
> 
> >    begin
> 
> >       Ada.Text_IO.Put_Line
> 
> >       (  "#1 was:"
> 
> >       &  Ada.Exceptions.Exception_Information
> 
> >          (  GNAT.Most_Recent_Exception.Occurrence
> 
> >       )  );
> 
> >       ...
> 
> >
> 
> > at the end of the Finalize you add:
> 
> >
> 
> >       ...
> 
> >    exception
> 
> >       when Error : others =>
> 
> >          Ada.Text_IO.Put_Line
> 
> >          (  "#2 was:"
> 
> >          & Ada.Exceptions.Exception_Information (Error)
> 
> >          );
> 
> >          raise;
> 
> >    end Finalize;
> 
> >
> 
> > Now in the output there should be lines "#1 was:" followed by "#2 was:",
> 
> > which could give a hint what is going on.
> 
> 
> 
> I made it (took ~1 hour to recompile AWS),  and here is output:
> 
> -----
> 
> #1 was:Exception name: PROGRAM_ERROR
> 
> Message: SSL not supported.
> 
> 
> 
> #2 was:Exception name: PROGRAM_ERROR
> 
> Message: SSL not supported.
> 
> 
> 
> 
> 
> raised PROGRAM_ERROR : aws-client.adb:290 finalize/adjust raised exception
> 
> -----
> 
> 
> 
> Why it tells me that SSL not supported?
> 
> I have openssl installed.

IIRC, OpenSSL isn't supported by default because of licence incompatibility.

Also for exception propagated too deep - you can attach gdb to the process ("catch exception" - Ada specific command for gdb which will catch every exception, before they get propagated). Though I understand not wanting to use debugger.