Reactive Streaming with Akka Streams
A lot of data processing can be decomposed into sequential stages of iterating and aggregating. The usual pattern is to retrieve data, apply some function to it, and store the result away for the next processing stage. In practice, this is usually complicated by the need to handle errors, to execute parts asynchronously or in parallel, to merge data streams, and to wait for blocking I/O. As the amount of data increases, resilience against congestion, overflow and concurrency problems becomes more and more important, and at the same time harder to manage.
Akka-Streams offers a powerful framework for asynchronous pipeline processing with memory-safety through backpressure. Data flow is modelled as a graph of Source, Sink and Flow elements that compose in a type-safe way. This approach hides all the details of rate limiting, buffering, pipelining, and asynchronicity, while retaining their explicit semantics. And it profits from the large ecosystem that already exists around the Akka framework!
This session is held on