Tweet
Tweet

Slide 1

Slide 1 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Cloud Native Event-Driven and Streaming Microservices Marius Bogoevici, Pivotal @mariusbogoevici

Slide 2

Slide 2 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Streaming • Processing large quantities of data as fast as possible (near real-time) • Use cases: • predictive maintenance, fraud detection, QoS, IoT • enterprise integration meets ‘Big Data’ • High throughput / low latency • Grouping, ordering, windowing • Transition in messaging middleware design: • Traditional (JMS-based, RabbitMQ) • To log-style: Kafka, Amazon Kinesis, Google Pub/Sub, Azure Event Hubs

Slide 3

Slide 3 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Microservices • Preferred model for distributed data processing applications • Failure isolation • Independent scaling • Process-specific resource tuning: • memory/CPU/Instance count/network • Typical benefits and drawbacks of microservices: • bounded context as cohesive units • development agility • added complexity

Slide 4

Slide 4 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Event-driven • Messaging vs HTTP/“fire-and-forget” vs. request-reply • Decoupling • Discoverability • Availability • Eventual consistency vs distributed transactions • especially over heterogenous resources • Strong similarities between: • data streaming - ‘big data’: ingestion, analytics • event streaming - async interaction, event sourcing, etc

Slide 5

Slide 5 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Data streaming vs Event sourcing

Slide 6

Slide 6 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Cloud Native • Focus on modern platform capabilities • Scaling up/down • Resource management (memory, CPU, instance count) • Health monitoring and failover • Routing and load balancing Apache YARN Apache Mesos Kubernetes

Slide 7

Slide 7 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream In a single sentence … dataflow:> stream create demo --definition “http | file”

Slide 8

Slide 8 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Stream 2015 Spring XD Spring Cloud Data Flow Spring Cloud Task

Slide 9

Slide 9 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Stream • Event-driven microservice framework • Built on battle-tested components • Spring Boot: full-stack standalone applications • Spring Integration: Messaging, EIP patterns, connectors • Focusing on scalable, structured distributed applications • Opinionated primitives • Pluggable middleware abstractions

Slide 10

Slide 10 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Stream in a nutshell Application Core Spring Boot Spring Integration Spring Messaging Reactive APIs inputs outputs

Slide 11

Slide 11 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream and a 10000 ft nutshell …

Slide 12

Slide 12 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream The binder abstraction

Slide 13

Slide 13 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Programming model @EnableBinding + Binder Implementation Apache Kafka JMS Google PubSub Production-ready: Experimental

Slide 14

Slide 14 text

Demo @mariusbogoevici @springcloud #Devoxx #SpringCloudStream

Slide 15

Slide 15 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Durable Pub Sub

Slide 16

Slide 16 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Consumer Groups

Slide 17

Slide 17 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Partitioning

Slide 18

Slide 18 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Data Flow • Orchestration: • DSL for Stream topologies • REST API • Shell • UI • Portable Deployment SPI • OOTB apps for common integration use-cases

Slide 19

Slide 19 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Stream DSL Stream Definition Spring Boot applications built on top of Spring ecosystem

Slide 20

Slide 20 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Data Flow Deployment Pla$orm

Slide 21

Slide 21 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Deployer • SPI for deploying applications to modern runtimes • Local (for testing) • Cloud Foundry • YARN • Kubernetes • Mesos + Marathon • Different resources supported • Spring Boot Uberjars • Docker Images • Different resource location supported • File system, HDFS, HTTP, Maven, Docker Hub

Slide 22

Slide 22 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Data Flow Streams

Slide 23

Slide 23 text

@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Deployment and orchestration

Slide 24

Slide 24 text

Demo @mariusbogoevici @springcloud #Devoxx #SpringCloudStream