Concurrent Data Processing in Elixir (Pragmatic Bookshelf)

PragmaticBookshelf
Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or fault-tolerance. Leverage concurrency by using OTP, GenStage, Flow, and Broadway to build high-performance applications and data processing pipelines.

Svilen Gospodinov @svilen

edited by Jacquelyn Carter @jkcarter

Learn different ways of writing concurrent code in Elixir and increase your application’s performance, without sacrificing scalability or fault-tolerance. Most projects benefit from running background tasks and processing data concurrently, but the world of OTP and various libraries can be challenging. Which Supervisor and what strategy to use? What about GenServer? Maybe you need back-pressure, but is GenStage, Flow, or Broadway a better choice? You will learn everything you need to know to answer these questions, start building highly concurrent applications in no time, and write code that’s not only fast, but also resilient to errors and easy to scale.

Whether you are building a high-frequency stock trading application or a consumer web app, you need to know how to leverage concurrency to build applications that are fast and efficient. Elixir and the OTP offer a range of powerful tools, and this guide will show you how to choose the best tool for each job, and use it effectively to quickly start building highly concurrent applications.

Learn about Tasks, supervision trees, and the different types of Supervisors available to you. Understand why processes and process linking are the building blocks of concurrency in Elixir. Get comfortable with the OTP and use the GenServer behaviour to maintain process state for long-running jobs. Easily scale the number of running processes using the Registry. Handle large volumes of data and traffic spikes with GenStage, using back-pressure to your advantage. Create your first multi-stage data processing pipeline using producer, consumer, and producer-consumer stages. Process large collections with Flow, using MapReduce and more in parallel. Thanks to Broadway, you will see how easy it is to integrate with popular message broker systems, or even existing GenStage producers.

Start building the high-performance and fault-tolerant applications Elixir is famous for today.


Don’t forget you can get 35% off with your Devtalk discount! Just use the coupon code “devtalk.com" at checkout :+1:

78 3397 25

Latest Threads About This Book Top

Cichacz
@svilen The book says: Currently, async_stream/3 assumes we want the results in the same order as they were originally. This order pre...
0 487 0
New
sprite2005
@svilen warning: Using a mapper operation, such as map/filter/reject, after reduce/3 is deprecated. Use Flow.on_trigger/2 instead (flow...
0 757 2
New
markthequark
@svilen def via(id) do {:via, Registry, {ProducerConsumerRegistry, id} end is missing a }. This is also the case in the downloadable...
0 616 0
New
smahi
Spining multiple PageConsumer using the book provided code, raise an error children = [ PageProducer, Supervisor.child_sp...
1 756 0
New
pgeraghty
@svilen Book version: P1.0 (August 2021) At the end of the definition of init, I believe GenStage.ParitionDispatcher should be GenStage...
0 821 0
New
axelson
Hello Svilen Gospodinov @svilen, The book currently states: When using GenServer.cast/2, we always get :ok as a reply. In fact, the r...
1 792 0
New
mafinar
Concurrent Data Processing in Elixir is now content complete and I finally found the time I’ve been looking for to dedicate behind readin...
70 1849 17
New
New
kunna
On page 147. the book says “Using Task.async_stream/3 is a great way to process large collections of data, and provide back-pressure at t...
0 817 0
New
New

Most Active This Week Top

Most Active This Month Top

Most Active This Year Top

Most Active Last Three Years Top

mafinar
Concurrent Data Processing in Elixir is now content complete and I finally found the time I’ve been looking for to dedicate behind readin...
70 1849 17
New
sprite2005
@svilen warning: Using a mapper operation, such as map/filter/reject, after reduce/3 is deprecated. Use Flow.on_trigger/2 instead (flow...
0 757 2
New
pgeraghty
@svilen Book version: P1.0 (August 2021) At the end of the definition of init, I believe GenStage.ParitionDispatcher should be GenStage...
0 821 0
New
markthequark
@svilen def via(id) do {:via, Registry, {ProducerConsumerRegistry, id} end is missing a }. This is also the case in the downloadable...
0 616 0
New
Cichacz
@svilen The book says: Currently, async_stream/3 assumes we want the results in the same order as they were originally. This order pre...
0 487 0
New

Most Active Over Three Years Top

New
PragmaticBookshelf
Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or...
78 3397 25
New
herminiotorres
Hi @Margaret , On page VII the book tells us the example and snippets will be all using Elixir version 1.11 But on page 3 almost the en...
15 1540 13
New
kunna
After finishing Chapter 2, Long-Running Processes Using GenServer, I was able to successfully run the code. but if I see the process obse...
2 901 4
New
mattjg908
Title: Concurrent Data Processing in Elixir: typo (31-32) I believe (page 31-32): def handle_cast({:send, email}, state) do . . {:no...
1 887 1
New
New
chriseyre2000
The first handle_cast({:send, email}, state) does not update the state so the example will not work as described. It’s missing: state =...
2 857 1
New
axelson
Hello Svilen Gospodinov @svilen, The book currently states: When using GenServer.cast/2, we always get :ok as a reply. In fact, the r...
1 792 0
New
New
mratliff
Not a big deal but might want to mention that you should add :crypto to the extra_applications portion of mix like so: def application d...
1 812 0
New
kunna
On page 147. the book says “Using Task.async_stream/3 is a great way to process large collections of data, and provide back-pressure at t...
0 817 0
New
New
smahi
Spining multiple PageConsumer using the book provided code, raise an error children = [ PageProducer, Supervisor.child_sp...
1 756 0
New
lancejjohnson
You will also discover a range new of new techniques, that will help you simplify your product, Delete “new” after “range”.
1 775 0
New
lancejjohnson
Page 3 We are going to use the Task module later to develop some if its functionality. s/if/of/
1 759 0
New

Book Info

Publisher: Pragmatic Bookshelf

Are you this book's author? To be notified of all posts made about it, sign up an account then click here, then select 'watching' from the bell icon in the top right corner.

Get money off!

The Pragmatic Bookshelf

35% off any eBook

Simply use coupon code "devtalk.com" at checkout. See full details here.