This site is archived.

parallel processing

How to do parallel processing and map/reduce with Gearman.

Giuseppe Maxia 14 June 2010
Type:  Not planned session

Parallel processing, in any programming language, is hard to achieve and easy to get wrong.
However, you can leverage a distributed client/server paradigm to create parallel processed tasks quite easily, with a scalable and fail-resistant system. With Gearman you can run tasks in parallel, no matter where they should be performed.

A common scenario in web development is a delay in serving a page, due to
the need of performing several unrelated operations in sequence.
Sequential development is the norm in every major programming language,
and, although parallel processing is possible, implementing it in your
application is often painful and results in complex and debug unfriendly code.
Enters Gearman, a distributed client/server framework, which can be easily
integrated in any programming language, including SQL (through a MySQL User
Defined Function), and makes parallel processing a trivial task. This

An introduction to Gearman

Giuseppe Maxia 14 June 2010
Type:  Not planned session

Gearman is a distributed client/server system that works across different operating systems, with different programming languages, in a scalable and fault-tolerant mode.
Add to it that it is incredibly easy to use, and you will wonder why you haven't used it yet.

In the age of ubiquitous connectivity, distributed servers have become
readily available and usable. Gearman, a friendly infrastructure for
distributed tasks, gives developers and DBAs a large degree of flexibility
in their applications. Using Gearman, developers can access libraries
written in multiple languages, even if they are different from the ones
used in their main application. DBAs will benefit from Gearman in their
daily tasks, by accessing external features from within the database server
itself. This session introduces Gearman principles and shows some simple