Building Cloud-Native Data-Intensive Applications with Spring Soby Chacko [@sobychacko] Sabby Anandan [@sabbyanandan]

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Disclaimer 2

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Safe Harbor Statement The following is intended to outline the general direction of Pivotal's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding Pivotal's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for Pivotal's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation. 3

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Plot Events in the Enterprise Events and Data-intensive Workloads A Practical Take on Data-Intensive Applications in Cloud Q+A 4

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Events in the Enterprise 1 Day 15 Million Rides 3 Million Drivers 78 Countries 5

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Event-centric Thinking 6 Profound Customer Engagement

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Real-time Matching Efficient Pickups/ Drops Lower Prices Improved Ride Opportunities More Income More Tips Customer: Driver Convenient Costs Less Flexible Customer: Rider(s) 7

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Events in Reality Unpredictable Traffic Patterns Resource Contention Faults Rewind / Replay Ordering Guarantees Slow Processing 8

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Events in Reality Unpredictable Traffic Patterns Resource Contention Faults Rewind / Replay Ordering Guarantees Slow Processing 1 Day 15 Million Rides! 9

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Data Intensity Unpredictable Traffic Patterns Resource Contention Faults Rewind / Replay Ordering Guarantees Slow Processing 1 Day 15 Million Rides! Maintainability Scalability Portability Reliability 10

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ “We call an application data- intensive if data is its primary challenge—the quantity of data, the complexity of data, or the speed at which it is changing.” 11

!12

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ You have data from disparate systems 13

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 14 You have data of different types

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 15 You have data of varying speed, size and shape

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 16 You have data that evolves

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ But, we are not Uber! 17

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Every company must find ways to learn, adapt, and grow. 18

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Stream a event-driven microservices framework Pluggable Binder Implementations Stream Partitions Consumer Groups Message Headers Testing Framework Content-type Negotiation Imperative + Functional Programming Model public class TransferServiceImpl implements TransferService { public TransferServiceImpl(AccountRepository ar) { this.accountRepository = ar; } 19

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Stream a event-driven microservices framework Pluggable Binder Implementations Opportunities: Same code; Same tests; Drop-in replacement for a variety of Message Brokers 20 Rabbit MQ Apache Kafka Google PubSub Amazon Kinesis Azure Event Hubs Solace

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Stream a event-driven microservices framework Pluggable Binder Implementations Stream Partitions Consumer Groups Message Headers Testing Framework Content-type Negotiation Imperative + Functional Programming Model public class TransferServiceImpl implements TransferService { public TransferServiceImpl(AccountRepository ar) { this.accountRepository = ar; } 21

Demo #1 A practical take on Events in Data Intensive Applications. 22

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ :UserCreated :UserCreated :UserNameChanged :UserActivated 23

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Driver’s nearby Surge Pricing: Passengers headed to the same event Split fare with friends 24

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 25 ? John Doe User userCreated() userActivated() userDeactivated() Commands // passenger getUserByGeoLocation() // drivers getUsersNearby() Queries :Eve Append-only Log Application State Event Storming CQRS Event Sourcing Architecture Methods / Patterns

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 26 Use Case: Trending User Activity by Region

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Stream + Kafka Streams 27 Bindings support not only for Channels, but also for Kafka Streams types. Hide the infrastructure and configuration overhead from the developers. Stateless and Stateful as options.

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Stream + Kafka Streams Topic = foo Partition n Partition 1 Partition 0 Spring Cloud Stream App Instance 0 KTable KStream GlobalKTable 28 Spring Cloud Stream App Instance 1

Demo #2 Let’s interact with the Data Intensive Applications. 29

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 30 Use Case: Trending User Activity by Region on a Moving Window

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Unpredictable Traffic Patterns Resource Contention Faults Rewind / Replay Ordering Guarantees Slow Processing 1 Day 15 Million Rides! Maintainability Scalability Portability Reliability 31 Data Intensity

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Maintainability Build, test, and iterate as frequently as needed. CI/CD as first-class thinking for data-centric workloads. Data processing guarantees in the event of rolling-upgrades. Scalability Auto-scale up/down based on throughput demands.
 Linear throughput characteristics as you scale applications. Same app runs locally in the laptop or in any cloud platform where there’s JVM. Portability Reliability Focus on the business logic and the unit-level, integration, and acceptance tests.
 Depend on platform runtime (Kubernetes or Cloud Foundry) for reliability and resiliency guarantees.
 32 Data Intensive Design

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 33 A toolkit for building data integration, real- time streaming, and batch data processing pipelines. WHAT IS SPRING CLOUD DATA FLOW?

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Data pipelines consist of Spring Boot apps, using Spring Cloud Stream for event-streaming or Spring Cloud Task for batch processes. Ready for Data Integration with >60 out-of-the- box streaming and batch Apps. DSL, GUI, and REST-APIs to build and orchestrate data pipelines onto platforms like Kubernetes and Cloud Foundry. Continuous delivery for streaming data pipelines using Spring Cloud Skipper. Cron-job scheduler for batch data pipelines using Spring Cloud Scheduler. 34 A toolkit for building data integration, real- time streaming, and batch data processing pipelines. WHAT IS SPRING CLOUD DATA FLOW?

Demo #3 Run all the things in the Cloud. 35

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 1 3 2 36 Putting it all together 1 2 3 Websocket Connection REST request/response REST request/response

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Next First-class support for Apps with Multiple Input & Output Channels (eg: KStream) Apps as Composable Functions Audit Trail: Who did what and when? 37

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Resources Spring Cloud Stream Samples | Gitter | StackOverflow Spring Cloud Data Flow Samples | Gitter | StackOverflow Demo: https://github.com/sabbyanandan/eda 38

Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Q+A 39