Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cloud-native Streaming and Event-Driven Microse...

Cloud-native Streaming and Event-Driven Microservices

Talk delivered at Spring IO in Barcelona (May 20, 2016)

The talk introduces Spring Cloud Stream as a framework for developing event-driven microservices, as well as Spring Cloud Data Flow as an orchestration and deployment layer.

Marius Bogoevici

May 20, 2016

Transcript

22 slides
  1. CLOUD-NATIVE STREAMING AND EVENT- DRIVEN MICROSERVICES • Use cases: •

    predictive maintenance, fraud detection, QoS, IoT • enterprise integration meets ‘Big Data’ • Specific requirements • High throughput, low latency • Grouping, ordering, windowing • Middleware adaptations: beyond JMS, Rabbit MQ • Kafka, Amazon Kinesis, Google Pub/Sub, Azure Event Hubs • Brokerless: Aeron, ZeroMQ Spring I/O 2016
  2. CLOUD-NATIVE STREAMING AND EVENT- DRIVEN MICROSERVICES • Architectural style advantages:

    • bounded context, development agility • Process-specific resource tuning: • memory, CPU, instance count • Messaging vs. HTTP • Discoverability • Availability • Consistency across heterogenous resources Spring I/O 2016
  3. CLOUD-NATIVE STREAMING AND EVENT- DRIVEN MICROSERVICES • Modern platforms •

    Resource management • memory, CPU, instance count • Scaling up/down • Health monitoring and failover • Routing and load balancing Spring I/O 2016 Apache YARN Apache Mesos Kubernetes
  4. Spring Cloud Stream Spring Cloud Data Flow Spring I/O 2016

    Spring Cloud Data Flow Spring Cloud Stream Spring Cloud Task 2015
  5. Spring Cloud Stream • Event-driven microservice framework • Built on

    battle-tested components • Spring Boot: full-stack standalone applications • Spring Integration: Messaging, EIP patterns, connectors • Opinionated primitives • Pluggable middleware abstractions Spring I/O 2016
  6. Consumer Groups • Borrowed from Kafka, applied to all binders

    • Groups of competing consumers within the pub-sub destination • Used for scaling and partitioning Spring I/O 2016
  7. Partitioning • Required for stateful processing • Outputs specify a

    partitioning strategy • Inputs consuming from specific partition Spring I/O 2016
  8. Spring Cloud Data Flow • Orchestration: • DSL for Stream

    topologies • REST API • Shell • UI • Portable Deployment SPI • OOTB apps for common integration use- cases Spring I/O 2016
  9. Stream DSL: pipes and filters Stream definition Boot Apps build

    on top of Spring Ecosystem httpfile = http | file | Spring I/O 2016 ‘pipe’, or how to communicate Rabbit, Kafka, Gemfire
  10. Spring Cloud Deployer • SPI for deploying applications to modern

    runtimes • Local (for testing) • Cloud Foundry • YARN • Kubernetes • Mesos + Marathon • Different resources supported • Spring Boot Uberjars • Docker Images • Different locations • File system, HDFS, HTTP, Maven, Docker Hub Spring I/O 2016
  11. Timeline • Spring Cloud Stream • 1.0.0.GA May 10, 2016

    • 1.1.0.M1 July 2016 • 1.1.0.GA Q3 2016 • Spring Cloud Data Flow • 1.0.0.M3 May 7, 2016 • 1.0.0.RC1 End of May, 2016 Spring I/O 2016

More Decks by Marius Bogoevici

Other Decks in Technology