How to modernize your stack with cloud solutions and integration patterns

Related presentation: https://youtu.be/vTdP2mLXiHg

This deck covers a brief explanation of an opinionated solution pattern aimed to those who seek to modernize their technology stack by augmenting and extending their services.

By adopting the change data capture integration pattern and event driven architectures on top of a hybrid cloud, this architecture and its actual implementation proves how organizations can adopt cloud and modern technologies to bring innovation to their solutions.

"Modernize your stack by adopting Change Data Capture".
A Solution Pattern for Cloud Native Architectures.

For more information see
https://redhat-solution-patterns.github.io/solution-pattern-modernization-cdc

Transcript

1. CONFIDENTIAL Designator 1 With cloud solutions and integration patterns How to modernize your stack Karina Varela Principal Technical Marketing Manager Red Hat Application Services Solution Patterns Bernard Tison Sr. Principal Technical Marketing Manager
2. V0000000 2 Ways companies define modernization Application Modernization Source: Application Modernization Report, Konveyor Community, 2022 Top reasons for modernization Adopting Change Data Capture for stack modernization
3. Common Modernization Patterns 3 Lift and Shift Leave the architecture alone but modernize the deployment platform. Can be used for performance increases by allowing for deploying to better hardware. Can be used to accelerate deployments and improve processes by leveraging platform automation. Fast Monoliths. Refactor and Augment/Extend Find parts of the architecture that are sources of pain - refactor. Build new capability in microservices with well-defined APIs. Wrap legacy software too brittle to change with adapter layers. Rewrite/Replace Create new functionality to replace existing functionality. Likely expensive and time-consuming. Typically only recommended when legacy vendors go away or a major skills gap forces it. HYBRID APP Phased approach in migration APIs Microservices Serverless/ functions Adopting Change Data Capture for stack modernization
4. How to modernize by augmenting and extending with low to zero impact on the legacy stack? Adopting Change Data Capture for stack modernization 4
5. CONFIDENTIAL Designator Extend and augment with enterprise integration patterns and modern cloud native technologies Adopting Change Data Capture for stack modernization 5 The adoption of the Change Data Capture integration pattern allows the immersion of legacy services within an event-driven architecture. The architecture evolves with new functionality being added to the existing technology stack using new cloud-native microservices. This solution allows multiple deployment models for the new services - going from full on-premise model to a hybrid cloud model.
6. The story and challenges of the Greenworld store Adopting Change Data Capture for stack modernization 6
7. CONFIDENTIAL Designator The Greenworld business A plant's store story Adopting Change Data Capture for stack modernization 7 We are a leading plants retail store acting in the market for over 15 years. We managed to start digital technologies adoption many years ago by acquiring a solution for our inventory and sales registrations. Our biggest challenge is to join competition with high-tech competitors. “” Alex Tizon CEO, Greenworld Our first step into modernization is developing a new cashback system for our customers. Adding to that, our current service lacks good searching capabilities so we need better ways for customers to find what they need. “” Avery Smith CTO, Greenworld
8. CONFIDENTIAL Designator Greenworld's goals and requirements: Adopting Change Data Capture for stack modernization 8 There is no code available for the current production software and its usage can't be interrupted or the store would need to halt its sales. Cashback functionality Every new purchase is rewarded with a proportional amount of cashback. The customer should have a cashback wallet and be able to track transactions history. The customer can use the cashback to get discount in upcoming purchases in our store. Full text search capabilities We need a new way for customers to find the products they need. Full-text search is a good starting point, but the inventory data is and should still be maintained via the legacy application. Modern cloud technology and architecture We seek to modernize our stack and enable our staff progressively on cloud technologies. We are open to adopt new architectures and technologies so we can then expand cloud adoption.
9. The existing architecture Adopting Change Data Capture for stack modernization 9
10. Architect [Person] Defines how systems are developed and integrated based on good practices. Retail Application [Software System] Legacy application used to control inventory and sales of a retail store. 10 Database [Software System] Persists inventory and sales related data. Developer [Person] Responsible for extending the application capabilities. Fixes bugs Persists execution data in Customer [Person] The user who buys products from the retail store Diagram Legend: Dark Red: Person Light Red: Software System Grey: External Systems "How can we enhance search capabilities?" "How can we offer a new cashback functionality?" Buys items using Technical Expert [Person] Works in the retail org. Knows well the domain and legacy applications. Maintains Adopting Change Data Capture for stack modernization Existing Architecture Legacy application architectural context diagram
11. CONFIDENTIAL Designator Legacy code can't be changed Legacy services need to be extended or enhanced but, it is not a possibility to implement the changes in the existing application. New services depend on data managed by legacy apps New applications or services must leverage existing data from an existing stack of services. Legacy and new services' data must be consistent Legacy applications are now extended and complemented by capabilities delivered by new microservices or services (e.g. search index, cache) but the data should always be synchronized. Increased cloud adoption The organization seeks to move towards cloud environments and comply to best practices on cloud-native architectures but must also guarantee that any production-active legacy system data is also synchronized to the new services; A next step is needed but.. Adopting Change Data Capture for stack modernization 11
12. Modernization Challenges Adopting Change Data Capture for stack modernization 12
13. Antipattern for data access and management in a distributed architecture: multiple services changes data directly in databases owned by other services. Multiple services relying on the same data 13 Retail Database Retail Service Cashback Service Legacy New service Cashback Database Adopting Change Data Capture for stack modernization
14. Dual-write anti-pattern: A service inserts and/or changes data in two or more different data stores potentially causing data inconsistency A service needs to persist data in multiple data stores 14 Retail Service Adopting Change Data Capture for stack modernization Retail Database Cache Search Index (e.g. elasticsearch)
15. Architect [Person] Defines how systems are developed and integrated based on good practices. Adopt a composable architecture, where independent event-driven services adds capabilities with zero impact to the legacy stack. 15 Architectural Solution Modernize your stack with Change Data Capture Adopting Change Data Capture for stack modernization
16. Cashback Application [Software System] New cloud-native microservice for interaction with users' cashback obtained by buying products. Allows buying products using earned cashback. Adopting CDC for stack modernization 16 Database [Software System] Persists inventory and sales related data. Buys items using Customer [Person] The user who buys products from the retail store Technical Expert [Person] Works in the retail org. Knows well the domain and legacy applications. Developer [Person] Responsible for extending the application capabilities. Architect [Person] Defines how systems are developed and integrated based on good practices. Diagram Legend: Dark Red: Person Light Red: Software System Grey: External Systems Database [Software System] Persists cashback related data. Search Index [Software System] ElasticSearch : Provides an efficient indexing and search mechanism. Persists data in Uses Cashback Searches items information Solution's Architectural Overview Proposed solution architectural context Maintains Persists data in Update and consumes index data from Fixes bugs Develops Develops Retail Application [Software System] Legacy application used to control inventory and sales of a retail store. Search Application [Software System] New cloud-native microservice for enhanced search capabilities including full text search.
17. Architect [Person] Defines how systems are developed and integrated based on good practices. 17 Legacy Application Database Event-driven microservice Debezium Kafka Event streaming 1. Captures all Database Changes 2. Pushes all captured database events as Kafka events 3. Consumes relevant events from legacy app Using Change Data Capture we can track every change in legacy database as a stream of events. New services can react and execute logic as needed. Architectural Solution Modernize your stack with Change Data Capture Adopting Change Data Capture for stack modernization
18. • Ensures that all data changes are captured; • Can produce change events with very low delay (ms); • Requires no changes to existing data model; • Can capture include, update and deletes; • Captures old record state plus additional metadata; 18 Legacy Application Database Event-driven microservice Debezium Kafka Event streaming 1. Captures all Database Changes 2. Pushes all captured database events as Kafka events 3. Consumes relevant events from legacy app Architect [Person] Defines how systems are developed and integrated based on good practices. What are the benefits of using Debezium in this architecture? Architectural Solution Modernize your stack with Change Data Capture Adopting Change Data Capture for stack modernization
19. 19 Legacy Application Event-driven Microservice (Camel) Retail Database Debezium Kafka Event streaming 1. Captures all Database Changes 2. Pushes all captured database events as Kafka events 3. Consumes relevant events from legacy app Architect [Person] Defines how systems are developed and integrated based on good practices. All in one simplified architecture representation Architectural Solution Modernize your stack with Change Data Capture Adopting Change Data Capture for stack modernization Search Index (Elasticsearch) 4. Indexes the relevant data Cashback Database Microservice (Kafka Streams) Event-driven Cashback Microservice (Quarkus) 3. Consumes relevant events from legacy app 4.Syncs expenses data Search Service UI Microservice (Quarkus) 5. Consumes and exposes data Cashback service UI Microservice (Quarkus) 5. Consumes and exposes data 4.Syncs cashback data 3. Consumes relevant events from legacy app
20. 20 Legacy Application Database Event-driven microservice Debezium Kafka Event streaming 1. Captures all Database Changes 2. Pushes all captured database events as Kafka events 3. Consumes relevant events from legacy app Architectural Solution Modernize your stack with Change Data Capture Adopting Change Data Capture for stack modernization Architect [Person] Defines how systems are developed and integrated based on good practices. How does Debezium support the integration between the legacy and search index tools like elasticsearch?
21. 21 Legacy Application Database Event-driven microservice Debezium Kafka Event streaming 1. Captures all Database Changes 2. Pushes all captured database events as Kafka events 3. Consumes relevant events from legacy app Architectural Solution Modernize your stack with Change Data Capture Adopting Change Data Capture for stack modernization Architect [Person] Defines how systems are developed and integrated based on good practices. How does Debezium support the integration between the legacy and search index tools like elasticsearch? Search Index (Elasticsearch) 4. Indexes the relevant data
22. 22 Architect [Person] Defines how systems are developed and integrated based on good practices. Which technologies can we use to create and run the new event-driven microservices? Architectural Solution Modernize your stack with Change Data Capture Adopting Change Data Capture for stack modernization Event-driven microservice
23. Apache Camel • Integration solutions with out-of-the-box best practices; • 340+ components such as databases, message queues, APIs; • Translate messages in multiple formats, and industry standard formats; • Simple, self-explained Domain Specific Language. Java, YAML or XML; • Commonly used patterns like: Content-based router, Splitter, Aggregator, etc • Biggest and most active community for open source integration software • Camel Extensions for Quarkus; 23 Architect [Person] Defines how systems are developed and integrated based on good practices. Which technologies can we use to create and run the new event-driven microservices? Architectural Solution Modernize your stack with Change Data Capture Adopting Change Data Capture for stack modernization Quarkus Camel Camel Extensions for Quarkus ▸ Latest Camel v3 (lighter, faster) ▸ Container-based environments ▸ Flash boot time ▸ Minimal memory footprint ▸ Standalone & Kubernetes ▸ Long-term support commitment Event-driven microservice
24. 24 Legacy Application Database Event-driven microservice Debezium 1. Captures all Database Changes 2. Pushes all captured database events as Kafka events 3. Consumes relevant events from legacy app Architect [Person] Defines how systems are developed and integrated based on good practices. Leverage cloud services for event streaming with Red Hat OpenShift Streams for Apache Kafka Adopting Change Data Capture for stack modernization OpenShift Streams for Apache Kafka • Reduce your operational cost and complexity of delivering real-time applications across hybrid-cloud environments; • CLI tool available for interaction with the hosted Kafka (rhoas) • Provides an unified experience across all clouds • Begin developing immediately and continuously respond to change • Fully hosted and managed Kafka service by Red Hat: ◦ 24x7 global support and a 99.95% service-level agreement (SLA); Architectural Solution Modernize your stack with Change Data Capture
25. • Receive data from input streams, execute complex operations (e.g. mapping, filtering, joining) and write the results into output streams; • Can be combined with the power of Quarkus runtime, including native compilation for serverless architecture; • Abstracts changing event data sets in Kafka clusters allowing for high throughput and scalability. • Simplify your services integration with out-of-the-box apis for processing and transferring data ; OpenShift Streams for Apache Kafka 25 Legacy Application Database Debezium 1. Captures all Database Changes 2. Pushes all captured database events as Kafka events 3. Consumes relevant events from legacy app Architect [Person] Defines how systems are developed and integrated based on good practices. Leverage cloud services for event streaming with Red Hat OpenShift Streams for Apache Kafka Architectural Solution Modernize your stack with Change Data Capture Adopting Change Data Capture for stack modernization Kafka streams processing microservice
26. 26 Legacy Application Event-driven Microservice (Camel) Retail Database Debezium Kafka Event streaming 1. Captures all Database Changes 2. Pushes all captured database events as Kafka events 3. Consumes relevant events from legacy app Architect [Person] Defines how systems are developed and integrated based on good practices. All in one simplified architecture representation Architectural Solution Modernize your stack with Change Data Capture Adopting Change Data Capture for stack modernization Search Index (Elasticsearch) 4. Indexes the relevant data Cashback Database Microservice (Kafka Streams) Event-driven Cashback Microservice (Quarkus) 3. Consumes relevant events from legacy app 4.Syncs expenses data Search Service UI Microservice (Quarkus) 5. Consumes and exposes data Cashback service UI Microservice (Quarkus) 5. Consumes and exposes data 4.Syncs cashback data 3. Consumes relevant events from legacy app
27. CONFIDENTIAL Designator Red Hat Application Foundation brings together enterprise solutions for building, integrating, deploying, and managing apps from development to production. Solid foundation for a reliable and long-living solution Adopting Change Data Capture for stack modernization 27 Linux (container host operating system) Kubernetes (orchestration) Physical* Virtual Private cloud Public cloud Edge Red Hat OpenShift Networking :: DevTools :: CI/CD :: ServiceMesh :: Serverless :: Observability :: Metering :: Cost Management Build and Modernize apps Connect and Extend apps Runtime frameworks App Connectivity & Composition Real time Data streaming API Management Application Platform App Foundations + Cloud Services
28. CONFIDENTIAL Designator Learn more: 28 Adopting Change Data Capture for stack modernization Red Hat OpenShift Streams ▸ Try for free: https://red.ht/TryKafka About the presented topic: ▸ Documentation: ▸ https://red.ht/modernize-with-cdc ▸ Implementation: https://red.ht/modernize-with-cdc-demo
29. CONFIDENTIAL Designator linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat 29 Red Hat is the world’s leading provider of enterprise open source software solutions. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. Thank you