Building Reactive Pipelines: How to go from scalable apps to (ridiculously) scalable systems

Going from imperative, blocking code to a reactive programming model enables us to scale our apps in ways that aren't possible with a thread scale-out approach, which is a good thing! But as with all optimizations, one must examine & address the system holistically or all we accomplish is moving bottlenecks around, creating or finding new chokepoints as we tune certain applications/services. This is *not* so good.

In this session, the presenter discusses & demonstrates:

* How Project Reactor builds on reactive streams to help you create performant & scalable reactive microservices
* Message brokers & streaming platforms like RabbitMQ & Apache Kafka
* How Spring Cloud Stream leverages Reactor to provide fully reactive pipelines for system-wide (ridiculous!) scalability

The presenter codes all examples using 100% open source software live and in real time.

Transcript

1. #ReactivePipelines @mkheck Building Reactive Pipelines Mark Heckler @mkheck, @springnoticias springnoticias.io mark@thehecklers.com mheckler@pivotal.io
2. Building Reactive Pipelines How to go from scalable apps to (ridiculously) scalable systems Mark Heckler Professional Problem Solver, Spring Developer & Advocate www.thehecklers.com mark@thehecklers.com mheckler@pivotal.io @mkheck, @springnoticias
3. @mkheck www.thehecklers.com “Please do LESS with MORE!”
4. @mkheck www.thehecklers.com Takeaways Message brokers & streaming platforms What is Spring Cloud Stream? Why use it? Reactive streams & Project Reactor Spring Cloud Stream + Reactor = system-wide (ridiculous!) scalability Turning it up to 11: going even further
5. @mkheck www.thehecklers.com Who am I? • Author • Architect & Developer • Java Champion, Groundbreaker • Professional Problem Solver • Spring Developer & Advocate
6. @mkheck www.thehecklers.com Takeaways Message brokers & streaming platforms What is Spring Cloud Stream? Why use it? Reactive streams & Project Reactor Spring Cloud Stream + Reactor = system-wide (ridiculous!) scalability
7. @mkheck www.thehecklers.com –Rossen Stoyanchev, Project Reactor team “In a nutshell reactive programming is about non-blocking, event-driven applications that scale with a small number of threads with backpressure as a key ingredient that aims to ensure producers do not overwhelm consumers.”
8. @mkheck www.thehecklers.com – https://github.com/reactive-streams/reactive-streams-jvm “The purpose of Reactive Streams is to provide a standard for asynchronous stream processing with non-blocking backpressure.”
9. @mkheck @projectreactor Reactive Streams: 4 interfaces Publisher<T> Subscriber<T> Subscription Processor<T,R>
10. @mkheck www.thehecklers.com Sink Processor Streaming Context Source
11. @mkheck www.thehecklers.com Streaming with Reactive Streams Publisher<T> Subscriber<T> Subscription Processor<T,R> Spring Cloud Stream parallel Source Sink (n/a) Processor
12. @mkheck www.thehecklers.com Let’s code!
13. @mkheck www.thehecklers.com
14. @mkheck www.thehecklers.com Takeaways Message brokers & streaming platforms What is Spring Cloud Stream? Why use it? Reactive streams & Project Reactor Spring Cloud Stream + Reactor = system-wide (ridiculous!) scalability Turning it up to 11: going even further
15. @mkheck www.thehecklers.com Resources https://github.com/mkheck/building-reactive-pipelines https://cloud.spring.io/spring-cloud-stream/ https://projectreactor.io mark@thehecklers.com, mheckler@pivotal.io @mkheck on Twitter