Amit Ramesh, Qui Nguyen - Building Stream Processing Applications

Amit Ramesh, Qui Nguyen - Building Stream ProcessingĀ Applications

Do you have a stream of data that you would like to process in real time? There are many components with Python APIs that you can put together to build a stream processing application. We will go through some common design patterns, tradeoffs and available components / frameworks for designing such systems. We will solve an example problem during the presentation to make these points concrete. Much of what will be presented is based on experience gained from building production pipelines for the real-time processing of ad streams at Yelp. This talk will cover topics such as consistency, availability, idempotency, scalability, etc.

https://us.pycon.org/2017/schedule/presentation/392/

Bde70c0ba031a765ff25c19e6b7d6d23?s=128

PyCon 2017

June 05, 2017
Tweet