Performance techniques used in AWS

Performance techniques used in AWS

[R. Tyler Croy]

I asked this in #Ada and couldn't find any decent documentation on some of 
the techniques used in AWS to make it perform as well as it apparently 
does (at least according to Thomas' silly benchmarks: <http://wiki.ada-
dk.org/index.php/AWS_vs_node.js>)

Surely beyond simple tasking there's some interesting approaches AWS must 
be using to make it fast (I don't have the time nor expertise to figure it 
out on my own)


Cheers
-- 
- R. Tyler Croy
--------------------------------------
    Code: http://github.com/rtyler

Re: Performance techniques used in AWS

[Thomas Løcke]

On 06/13/2011 07:48 AM, R. Tyler Croy wrote:
> ...(at least according to Thomas' silly benchmarks:<http://wiki.ada-
> dk.org/index.php/AWS_vs_node.js>)


LOL.. Thanks.  :D

-- 
Thomas Løcke

Email: tl at ada-dk.org
Web: http//:ada-dk.org
http://identi.ca/thomaslocke

Re: Performance techniques used in AWS

[Pascal Obry]

Le 13/06/2011 07:48, R. Tyler Croy a écrit :
> I asked this in #Ada and couldn't find any decent documentation on some of
> the techniques used in AWS to make it perform as well as it apparently
> does (at least according to Thomas' silly benchmarks:<http://wiki.ada-
> dk.org/index.php/AWS_vs_node.js>)
>
> Surely beyond simple tasking there's some interesting approaches AWS must
> be using to make it fast (I don't have the time nor expertise to figure it
> out on my own)

Sadly I fear there is no magic. I have never ever tried to build 
something fast in fact and never tried to optimized things a lot. I've 
always been concentrated to provide a simple and sound design. The heart 
of AWS has almost not been changed since it inception. I was the first 
surprised to see it being so fast.

Note that Dmitriy Anisimkov already reported long time ago very good 
response time.

I've seen so many applications molested to gain some speed without 
testing properly first... AWS is not in this category.

The heart of the web server design is quite simple: a single queue for 
the incoming requests. Those requests are dispatched to a free task 
taken from a a poll. That's pretty all there is about it.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: Performance techniques used in AWS

[Pascal Obry]

 > The heart of AWS has almost not been changed since it inception.

Hum, I'm wrong about one thing... Dmitriy had done some work to optimize 
the heart of AWS by using proper select/poll call. The very first 
implementation I did long long ago was just "stupid". Sorry about that!

Yet this was in the first year of AWS IIRC, so it was almost in the 
inception time :)

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: Performance techniques used in AWS

[J-P. Rosen]

Le 13/06/2011 18:28, Pascal Obry a écrit :
> Sadly I fear there is no magic. I have never ever tried to build
> something fast in fact and never tried to optimized things a lot. I've
> always been concentrated to provide a simple and sound design.
Well, that's the magic ! ;-)

-- 
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Adalog a déménagé / Adalog has moved:
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
Tel: +33 1 45 29 21 52, Fax: +33 1 45 29 25 00

Re: Performance techniques used in AWS

[R. Tyler Croy]

On Mon, 13 Jun 2011 18:28:28 +0200, Pascal Obry wrote:

> Le 13/06/2011 07:48, R. Tyler Croy a écrit :
>> I asked this in #Ada and couldn't find any decent documentation on some
>> of the techniques used in AWS to make it perform as well as it
>> apparently does (at least according to Thomas' silly
>> benchmarks:<http://wiki.ada- dk.org/index.php/AWS_vs_node.js>)
>>
>> Surely beyond simple tasking there's some interesting approaches AWS
>> must be using to make it fast (I don't have the time nor expertise to
>> figure it out on my own)
> 
> Sadly I fear there is no magic. I have never ever tried to build
> something fast in fact and never tried to optimized things a lot. I've
> always been concentrated to provide a simple and sound design. The heart
> of AWS has almost not been changed since it inception. I was the first
> surprised to see it being so fast.
> 
> Note that Dmitriy Anisimkov already reported long time ago very good
> response time.
> 
> I've seen so many applications molested to gain some speed without
> testing properly first... AWS is not in this category.
> 
> The heart of the web server design is quite simple: a single queue for
> the incoming requests. Those requests are dispatched to a free task
> taken from a a poll. That's pretty all there is about it.


What I'm trying to work out in my head is how to build an application (in 
Ada) that can compete[0] effectively with a libev-based program written in 
C. The advantage that libev brings to the table is that this program (in 
C) can very easily handle and respond to tens of thousands of concurrent 
connections.

Unlike AWS or most web servers however, these connections are not HTTP-
based and do not follow the traditional Request->Response->Close pattern 
of most of the HTTP world, but rather sockets are held open for minutes 
even hours at a time.

Can you conceive of a variation of your queue->task pool technique that 
might be able to effectively manage such a workload?

The closest I can think of at the moment is hooking into the epoll(2) 
Linux system interface from a "main" task and use the event triggers from 
epoll(2) to execute "work" in a task-pool. I've not come up with an idea 
that could perform well using pure GNAT libraries.


Any suggestions? :)

Cheers

[0] Whereas competition means response-time and number of connections that 
it can handle are roughly equal

-- 
- R. Tyler Croy
--------------------------------------
    Code: http://github.com/rtyler

Re: Performance techniques used in AWS

[Natasha Kerensikova]

Hello,

On 2011-06-14, R. Tyler Croy <tyler@linux.com> wrote:
> What I'm trying to work out in my head is how to build an application (in 
> Ada) that can compete[0] effectively with a libev-based program written in 
> C. The advantage that libev brings to the table is that this program (in 
> C) can very easily handle and respond to tens of thousands of concurrent 
> connections.

I don't think that really counts as competing, but I was planning at
some point in the future to write a thin Ada binding to libevent2 -- not
so much for performance, which is only a happy side-effect, but for the
simplicity and homogeneity of the callback-based multiplexing it offers.
However I still feel the need for a better mastery of the language
before starting such a project.

Does anyone have any objection to such an idea?

Or would anyone have any hint I could use for such a thin binding,
and/or for thickening it?


Thanks in advance,
Natasha

Re: Performance techniques used in AWS

[Simon Wright]

"R. Tyler Croy" <tyler@linux.com> writes:

> What I'm trying to work out in my head is how to build an application
> (in Ada) that can compete[0] effectively with a libev-based program
> written in C. The advantage that libev brings to the table is that
> this program (in C) can very easily handle and respond to tens of
> thousands of concurrent connections.
>
> Unlike AWS or most web servers however, these connections are not
> HTTP- based and do not follow the traditional Request->Response->Close
> pattern of most of the HTTP world, but rather sockets are held open
> for minutes even hours at a time.

Nowadays most HTTP connections are persistent, either because of
HTTP/1.0's "Connection: Keep-Alive" or the absence of the newer
"Connection: close".

I guess the performance will depend heavily on select(2) vs poll(2) or
epoll.

Re: Performance techniques used in AWS

[Yannick Duchêne (Hibou57)]

Le Tue, 14 Jun 2011 10:28:11 +0200, Natasha Kerensikova  
<lithiumcat@gmail.com> a écrit:
> I don't think that really counts as competing, but I was planning at
> some point in the future to write a thin Ada binding to libevent2 -- not
> so much for performance, which is only a happy side-effect, but for the
> simplicity and homogeneity of the callback-based multiplexing it offers.
> However I still feel the need for a better mastery of the language
> before starting such a project.
Thanks to your message, I've just learn about libEvent which I did not  
knew before. A good unification, I like it. Thanks again.

-- 
“Syntactic sugar causes cancer of the semi-colons.”  [Epigrams on  
Programming — Alan J. — P. Yale University]
“Structured Programming supports the law of the excluded muddle.” [Idem]
“c++; /* this makes c bigger but returns the old value */” [Anonymous]

Re: Performance techniques used in AWS

[Yannick Duchêne (Hibou57)]

Le Tue, 14 Jun 2011 12:40:15 +0200, Simon Wright <simon@pushface.org> a  
écrit:
> I guess the performance will depend heavily on select(2) vs poll(2) or
> epoll.
According to this
http://monkey.org/~provos/libevent/libevent-benchmarks.jpg
and that
http://monkey.org/~provos/libevent/libevent-benchmark2s.jpg
epoll and kqueue are the more efficient


-- 
“Syntactic sugar causes cancer of the semi-colons.”  [Epigrams on  
Programming — Alan J. — P. Yale University]
“Structured Programming supports the law of the excluded muddle.” [Idem]
“c++; /* this makes c bigger but returns the old value */” [Anonymous]

Re: Performance techniques used in AWS

[Yannick Duchêne (Hibou57)]

Le Tue, 14 Jun 2011 12:48:56 +0200, Yannick Duchêne (Hibou57)  
<yannick_duchene@yahoo.fr> a écrit:
> According to this
> http://monkey.org/~provos/libevent/libevent-benchmarks.jpg
> and that
> http://monkey.org/~provos/libevent/libevent-benchmark2s.jpg

Sorry, I gave links to thumbnails. Look these instead (without the final s)
http://monkey.org/~provos/libevent/libevent-benchmark.jpg
http://monkey.org/~provos/libevent/libevent-benchmark2.jpg

-- 
“Syntactic sugar causes cancer of the semi-colons.”  [Epigrams on  
Programming — Alan J. — P. Yale University]
“Structured Programming supports the law of the excluded muddle.” [Idem]
“c++; /* this makes c bigger but returns the old value */” [Anonymous]

Re: Performance techniques used in AWS

[Simon Wright]

"Yannick Duchêne (Hibou57)" <yannick_duchene@yahoo.fr> writes:

> Le Tue, 14 Jun 2011 12:40:15 +0200, Simon Wright <simon@pushface.org>
> a écrit:
>> I guess the performance will depend heavily on select(2) vs poll(2) or
>> epoll.
> According to this
> http://monkey.org/~provos/libevent/libevent-benchmarks.jpg
> and that
> http://monkey.org/~provos/libevent/libevent-benchmark2s.jpg
> epoll and kqueue are the more efficient

I see that libev[ent] abstracts away the actual polling mechanism, which
is good. Of course, what mechanism you can actually use depends on the
OS: I think epoll is a Linux-specific service? - certainly no sign of it
on this Mac.

Re: Performance techniques used in AWS

[R. Tyler Croy]

On Tue, 14 Jun 2011 12:18:11 +0100, Simon Wright wrote:

> "Yannick Duchêne (Hibou57)" <yannick_duchene@yahoo.fr> writes:
> 
>> Le Tue, 14 Jun 2011 12:40:15 +0200, Simon Wright <simon@pushface.org> a
>> écrit:
>>> I guess the performance will depend heavily on select(2) vs poll(2) or
>>> epoll.
>> According to this
>> http://monkey.org/~provos/libevent/libevent-benchmarks.jpg and that
>> http://monkey.org/~provos/libevent/libevent-benchmark2s.jpg epoll and
>> kqueue are the more efficient
> 
> I see that libev[ent] abstracts away the actual polling mechanism, which
> is good. Of course, what mechanism you can actually use depends on the
> OS: I think epoll is a Linux-specific service? - certainly no sign of it
> on this Mac.



Correct, you're options on Linux 2.6+ are epoll(2), on *BSD and Mac OS X 
the kqueue(2) facility is available. For Windows users..well, you get what 
you pay for ;)

Both libev and libevent both abstract away the underlying polling 
mechanism.



-- 
- R. Tyler Croy
--------------------------------------
    Code: http://github.com/rtyler

Re: Performance techniques used in AWS

[Yannick Duchêne (Hibou57)]

Le Tue, 14 Jun 2011 17:10:12 +0200, R. Tyler Croy <tyler@linux.com> a  
écrit:
> Correct, you're options on Linux 2.6+ are epoll(2), on *BSD and Mac OS X
> the kqueue(2) facility is available. For Windows users..well, you get  
> what
> you pay for ;)

On Windows, all included in the original license fees (at no additional  
cost), you have…

WaitForSingleObject(Ex):  
http://msdn.microsoft.com/en-us/library/ms687032(v=vs.85).aspx
WaitForMultipleObjects(Ex):  
http://msdn.microsoft.com/en-us/library/ms687025(v=vs.85).aspx
Select: http://msdn.microsoft.com/en-us/library/ms740141(v=vs.85).aspx
WSAAsyncSelect:  
http://msdn.microsoft.com/en-us/library/ms741540(v=vs.85).aspx
WSAEventSelect:  
http://msdn.microsoft.com/en-us/library/ms741576(v=vs.85).aspx
There was also something like WaitForEvents if my mind is right, but I  
could not retrieve the relevant reference.

-- 
“Syntactic sugar causes cancer of the semi-colons.”  [Epigrams on  
Programming — Alan J. — P. Yale University]
“Structured Programming supports the law of the excluded muddle.” [Idem]
“c++; /* this makes c bigger but returns the old value */” [Anonymous]