Building Reactive Pipelines

Building Reactive Pipelines: How to go from scalable apps to (ridiculously) scalable systems

Going from imperative, blocking code to a reactive programming model enables us to scale our apps in ways that aren't possible with a thread scale-out approach, which is a good thing! But as with all optimizations, one must examine & address the system holistically or all we accomplish is moving bottlenecks around, creating or finding new chokepoints as we tune certain applications/services. This is *not* so good.

In this session, the presenter discusses & demonstrates:

* How Project Reactor builds on reactive streams to help you create performant & scalable reactive microservices
* Message brokers & streaming platforms like RabbitMQ & Apache Kafka
* How Spring Cloud Stream leverages Reactor to provide fully reactive pipelines for system-wide (ridiculous!) scalability

The presenter will code all examples using 100% open source software live and in real time. This is not an abstract discussion, come to gain real, practical knowledge!

Transcript

1. Building Reactive Pipelines How to go from scalable apps to (ridiculously) scalable systems Mark Heckler Spring Developer & Advocate www.thehecklers.com mark@thehecklers.com mheckler@pivotal.io @mkheck
2. @mkheck www.thehecklers.com “Please do LESS with MORE!”
3. @mkheck www.thehecklers.com Why are we here? Scaling systems: traditional approaches What to do when we reach the limits? Sounds good, but how does it work?
4. @mkheck www.thehecklers.com Who am I? • Author • Architect & Developer • Java Champion, Rockstar • Professional Problem Solver • Spring Developer & Advocate • Creador y curador de
5. @mkheck www.thehecklers.com Scaling systems: off to a good start Microservices for independent scaling Messaging platforms Spring Cloud Stream for productivity + versatility (+ resilience, etc.)
6. @mkheck www.thehecklers.com Sink Processor For example… Source
7. @mkheck www.thehecklers.com We’ve redlined, now what? Change approach to scaling Scaling (connections) vs. Performance (parallelization) Reactor Integration with messaging platforms…any synergies to be had here?
8. @mkheck www.thehecklers.com –Rossen Stoyanchev, Reactor team member “In a nutshell reactive programming is about non-blocking, event-driven applications that scale with a small number of threads with backpressure as a key ingredient that aims to ensure producers do not overwhelm consumers.”
9. @mkheck www.thehecklers.com Reactive Streams: 4 interfaces Publisher<T> Subscriber<T> Subscription Processor<T,R>
10. @mkheck www.thehecklers.com Reactive Streams in Context Publisher<T> Subscriber<T> Subscription Processor<T,R> Spring Cloud Stream parallel Source Sink (n/a) Processor
11. @mkheck www.thehecklers.com Let’s code!
12. @mkheck www.thehecklers.com
13. @mkheck www.thehecklers.com Resources https://github.com/mkheck/building-reactive-pipelines https://cloud.spring.io/spring-cloud-stream/ https://projectreactor.io mark@thehecklers.com, mheckler@pivotal.io @mkheck on Twitter