Re: Distributed Systems Annex, data sharing between programs

[Maciej Sobczak <see.my.homepage@gmail.com>]

On 8 Cze, 04:11, Shark8 <onewingedsh...@gmail.com> wrote:

> Could you give examples of what sorts of tasks are better-suited/easier in DSA and YAMI4?

Pascal already provided example where DSA is attractive due to its
integration with the language. Note the pattern: Ada is a synchronous
sequential[*] language, so if your distributed problem is of the same
nature, then the DSA solution will look natural.

[*] this means that your problem can be described as a sequence of
steps where each step has to finish before the next step begins.

Now, the contrary example.

You have a system with several thousand machines and you want to send
them new configuration or something. You know the locations of those
systems, but the little problem is that with this scale some of them
are not working at all, some are hanging and some are overloaded and
therefore process everything very slowly. And you cannot do much about
it. And you have to send them the new data within very short time
frame. How short? Let's say several seconds.
The synchronous-sequental solution has a loop over the targets and a
remote call of the kind that Pascal has shown. This does not work. It
would not work even if all the targets were in a perfect shape.
A naive extension of this solution involves creating some tasks to
make things go in parallel. This is bad design, as the parallelism
already exists in huge amounts - remember, there are thousands of
machines over there, right? So why do we have to create tasks locally,
if the parallel resources are elsewhere? OK, let's shove this
conceptual issue under the carpet, but then - how many tasks should we
create? This is where DSA (or CORBA or ICE or similar solutions from
the same paradigm) proves to be actually very low-level instead of
high-level and forces us to think in terms that are very distant from
the actual problem at hand.

The solution is to stop thinking in terms of RPC (remote procesure
calls), and start thinking in terms of messages. If you lift the
concept of the message to the level of a design entity (yes,
distribution is *not* transparent and it *cannot* be an afterthought),
then some cool things are possible - just create thousands of messages
and let them go. No need to create additional tasks and no need to
wait for consecutive steps, as everything can happen in parallel
naturally. The original problem becomes feasible even in the presence
of partial failures, because separate message do not have to see their
delays and their individual communication problems.

Another example of this paradigm shift is a trading system, where you
want to inform all interested participants about a price change. Say,
there is a new price of the IBM share and you want to send it to
everybody. The difference from the above example is that this time you
don't really care (and therefore don't need to know) who is
interested. But you still have to publish this data somehow.
The DSA (CORBA/ICE/etc.) solution is lots of coding. Lots of low-level
code that has very little connection to the original, high-level
problem ("publish new price").
The messaging system allows to solve it differently, exactly thanks to
the fact that a message becomes a design-level entity. Create a new
message and give it to the broker - he will take care of it.
Hey, if the message is a first-class entity, then you might try to do
even more funny things with it - why not store it? Why not "record"
the stream of messages and "replay" them later? Why not attach some
security-related stuff to it, like digital signatures or access
control lists? Or maybe route tracing? This is just the tip of the
iceberg.

YAMI4 is a messaging system, where the programmer deals with messages.
This is the fundamental difference from DSA, where the programmer
deals with calls. Ironically, the lack of language integration is both
a disadvantage and a big advantage of messaging systems - the
advantage being that if there is no integration, then the language
rules do not limit what you can do.

More here:

http://www.inspirel.com/articles/RPC_vs_Messaging.html

Also, in the last section titled "Message-oriented middleware":

http://www.inspirel.com/articles/Types_Of_Middleware.html

--
Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com