Upgrade to Pro — share decks privately, control downloads, hide ads and more …

State Machine Saga Design Pattern

State Machine Saga Design Pattern

Building Cloud-Native App Series - Part 7 of 15
Microservices Architecture Series
FSM, Spring State Machine
States, Transitions, Events
Actions, Guards
Saga Design Pattern,
Enterprise Integration Patterns

Araf Karsh Hamid

August 30, 2023
Tweet

More Decks by Araf Karsh Hamid

Other Decks in Technology

Transcript

  1. @arafkarsh arafkarsh 8 Years Network & Security 6+ Years Microservices

    Blockchain 8 Years Cloud Computing 8 Years Distributed Computing Architecting & Building Apps a tech presentorial Combination of presentation & tutorial ARAF KARSH HAMID Co-Founder / CTO MetaMagic Global Inc., NJ, USA @arafkarsh arafkarsh 1 Finite State Machine Saga, EIP Building Cloud Native Apps FSM, Spring State Machine States, Transitions, Events Actions, Guards Saga Design Pattern, Enterprise Integration Patterns Part 7 of 15
  2. @arafkarsh arafkarsh 2 Slides are color coded based on the

    topic colors. Finite State Machine Spring State Machine 1 State Machine Demo States, Transitions, Events, Actions, Guards 2 Distributed Tx Saga Design Pattern Orchestrator / Choreography 3 Enterprise Integration Patterns Spring Integration 4
  3. @arafkarsh arafkarsh 0 Code Setup o SpringBoot 2.7.2 (MVC) /

    SpringBoot 3.1.0 (WebFlux) o Java 8 for Compile and Java 17 to run o H2 DB or PostgreSQL database 3
  4. @arafkarsh arafkarsh Package Structure 4 Source: https://github.com/arafkarsh Threads / Rx

    Java 2: https://github.com/arafkarsh/fusion-sky-threads-rx-java Spring WebFlux Reactive Programming: https://github.com/arafkarsh/ms-springboot-310-webflux-r2dbc Spring MVC & Security: https://github.com/arafkarsh/ms-springboot-272-vanilla
  5. @arafkarsh arafkarsh Finite State Machine – States / Transitions 7

    1. States: A state represents a condition or status of the system at a point in time. For instance, in a traffic light system, the states could be 'Green,' 'Orange,’ or 'Red.’ In a software system, the state might represent whether the system is in 'start-up,' 'operation,’ or 'shutdown' mode. Each finite state machine must have at least one state and one of those states is designated as the starting state or initial state. 2. Transitions: Transitions are the paths from one state to another. Events or conditions trigger them. For example, a timer reaching zero might trigger a transition from 'Green' to 'Orange' in a traffic light system. Each transition is associated with a specific condition that must be fulfilled to move from one state to another.
  6. @arafkarsh arafkarsh Understanding Finite State Machine 8 S1 S2 S3

    T1 T2 T3 T4 States Transitions Transitions
  7. @arafkarsh arafkarsh Deterministic / Non-Deterministic State Machines 9 DFA (Deterministic

    Finite Automata): • A real-world example of a DFA could be a turnstile at a train station or amusement park. The turnstile has two states: Locked and Unlocked. When you insert a ticket or token (input), the state changes from Locked to Unlocked. After you pass through (another input), it changes back to the Locked state. At any point, the turnstile has a deterministic next state for a given input. NFA (Non-deterministic Finite Automata): • A real-world example of an NFA is less common because NFAs allow a system to be in multiple states at once or make transitions without specific inputs, which only sometimes occurs in everyday physical systems. However, in computer science, we often use NFAs while designing algorithms or systems, such as pattern-matching algorithms used in text editors, compilers, and the design of certain types of computer hardware (Network Routers, FPGAs).
  8. @arafkarsh arafkarsh Spring State Machine o States o Transitions: Triggers

    (Events, Timers, Choices) o Guards / Actions o Extended State o Choice o Entry / Exit States 10
  9. @arafkarsh arafkarsh Spring State Machine 11 State Machine: The main

    entity that drives a collection of states, together with regions, transitions, and events. State: A state models a situation during which some invariant condition holds. The state is the main entity of a state machine where events drive state changes. Event: An entity sent to a state machine drives various state changes. Initial State: A unique state in which the state machine starts. The initial state is always bound to a particular state machine or region. A state machine with multiple regions may have multiple initial states
  10. @arafkarsh arafkarsh Spring State Machine 12 Extended State: An extended

    state is a special set of variables kept in a state machine to reduce the number of needed states. Transition: A transition is a relationship between a source and target states. It may be part of a compound transition, which takes the state machine from one state configuration to another, representing the state machine’s complete response to an event of a particular type. End State: (Also called a final state.) A special kind of state signifying that the enclosing region is completed. If the enclosing region is directly contained in a state machine and all other regions in the state machine are also completed, the entire state machine is completed.
  11. @arafkarsh arafkarsh Spring State Machine 13 Guard: A boolean expression

    evaluated dynamically based on the value of extended state variables and event parameters. Guard conditions affect the behavior of a state machine by enabling actions or transitions only when they evaluate to TRUE and disabling them when they evaluate to FALSE. Action: A action is a behavior run during the triggering of the transition. Choice State: A pseudo state that allows for making a transition choice based on (for example) event headers or extended state variables. Region: A region is an orthogonal part of either a composite state or a state machine. It contains states and transitions. Junction State: A pseudo state that is relatively similar to choice state but allows multiple incoming transitions, while choice allows only one incoming transition.
  12. @arafkarsh arafkarsh Spring State Machine 14 Fork State: A pseudo

    state that gives controlled entry into a region. Join State: A pseudo state that gives controlled exit from a region. Entry Point: A pseudo state that allows controlled entry into a submachine. Exit Point: A pseudo state that allows controlled exit from a submachine. History State: A pseudo state that lets a state machine remember its last active state. Two types of history state exists: shallow (which remembers only top level state) and deep (which remembers active states in sub-machines).
  13. @arafkarsh arafkarsh 2 State Machine Demo o Order Processing State

    Machine Configuration o Order Processing using State Machine o Order Entities and States, Events, Results 15
  14. @arafkarsh arafkarsh Order State Machine Config 16 Order State Machine

    Config Order State Machine Listener Adapter Order State Change Interceptor Order State Details Order History Service Define States and Transitions Handles State Changes Handles Post State Transitions Keeps Order States in Request Scope Service to Persist Order State Transition History Setting up the State Machine Factory
  15. @arafkarsh arafkarsh Order Processing 17 Order State Machine Manager Order

    State Machine Actions Order State Machine Error Handler Order State Machine Guards Restores State Machine and Send Messages to State Machine Globally Handling Error Actions for Every State Guards to Validate the States and Control the Flow Order Service Implementation Handles Messages to State Machine Manager Order Process Controller Handles all the REST Endpoints State Machine is handled in stateless Architecture. State Machine is Restored using OSM Manager.
  16. @arafkarsh arafkarsh Order Entity 18 Order Entity Order Items Entity

    1 * Shipping Address (VO) Order State History Entity 1 1 1 * Order Entities / Repository Order State Order Event Order Result Order Notes Order Constants Enumerations
  17. @arafkarsh arafkarsh State Machine Code base o States and Transitions

    o Actions / Guards o Restore State Machine o Error Handling 21 Source: https://github.com/arafkarsh/ms-springboot-272-java-8
  18. @arafkarsh arafkarsh 3 Distributed Transactions 37 • Saga Design Pattern

    • Features • Handling Invariants • Forward recovery • Local Saga Feature • Distributed Saga • Use Case: Distributed Saga
  19. @arafkarsh arafkarsh Distributed Transactions: 2 Phase Commit 2 PC or

    not 2 PC, Wherefore Art Thou XA? 38 How does 2PC impact scalability? • Transactions are committed in 2 phases. • This involves communicating with every database (XA: eXtended Architecture Resources) involved to determine if the transaction will commit (Prepare) in the 1st phase. • During the 2nd phase, each database is asked to complete the Commit. • While all of this coordination is going on, locks in all of the data sources are being held. • The longer duration locks create the risk of higher contention. • Additionally, the two phases require more database processing time than a single-phase commit. • The result is lower overall TPS in the system. Transaction Manager XA Resources Request to Prepare Commit Prepared Prepare Phase Commit Phase Done Source : Pat Helland (Amazon) : Life Beyond Distributed Transactions Distributed Computing : http://dancres.github.io/Pages/ Solution : Resilient System • Event Based • Design for failure • Asynchronous Recovery • Make all operations idempotent. • Each DB operation is a 1 PC 1 2
  20. @arafkarsh arafkarsh Distributed Tx: SAGA Design Pattern instead of 2PC

    39 Long Lived Transactions (LLTs) hold on to DB resources for relatively long periods of time, significantly delaying the termination of shorter and more common transactions. Source: SAGAS (1987) Hector Garcia Molina / Kenneth Salem, Dept. of Computer Science, Princeton University, NJ, USA T1 T2 Tn Local Transactions C1 C2 Cn-1 Compensating Transaction Divide long–lived, distributed transactions into quick local ones with compensating actions for recovery. Travel : Flight Ticket & Hotel Booking Example BASE (Basic Availability, Soft State, Eventual Consistency) Room Reserved T1 Room Payment T2 Seat Reserved T3 Ticket Payment T4 Cancelled Room Reservation C1 Cancelled Room Payment C2 Cancelled Ticket Reservation C3
  21. @arafkarsh arafkarsh SAGA Design Pattern Features 40 1. Backward Recovery

    (Rollback) T1 T2 T3 T4 C3 C2 C1 Order Processing, Banking Transactions, Ticket Booking Examples Updating individual scores in a Team Game. 2. Forward Recovery with Save Points T1 (sp) T2 (sp) T3 (sp) • To recover from Hardware Failures, SAGA needs to be persistent. • Save Points are available for both Forward and Backward Recovery. Type Source: SAGAS (1987) Hector Garcia Molina / Kenneth Salem, Dept. of Computer Science, Princeton University, NJ, USA
  22. @arafkarsh arafkarsh Handling Invariants – Monolithic to Micro Services 41

    In a typical Monolithic App Customer Credit Limit info and the order processing is part of the same App. Following is a typical pseudo code. Order Created T1 Order Microservice Credit Reserved T2 Customer Microservice In Micro Services world with Event Sourcing, it’s a distributed environment. The order is cancelled if the Credit is NOT available. If the Payment Processing is failed then the Credit Reserved is cancelled. Payment Microservice Payment Processed T3 Order Cancelled C1 Credit Cancelled due to payment failure C2 Begin Transaction If Order Value <= Available Credit Process Order Process Payments End Transaction Monolithic 2 Phase Commit https://en.wikipedia.org/wiki/Invariant_(computer_science)
  23. @arafkarsh arafkarsh 42 Use Case : Restaurant – Forward Recovery

    Domain The example focus on a concept of a Restaurant which tracks the visit of an individual or group to the Restaurant. When people arrive at the Restaurant and take a table, a table is opened. They may then order drinks and food. Drinks are served immediately by the table staff, however food must be cooked by a chef. Once the chef prepared the food it can then be served. Payment Billing Dining Source: http://cqrs.nu/tutorial/cs/01-design Soda Cancelled Table Opened Juice Ordered Soda Ordered Appetizer Ordered Soup Ordered Food Ordered Juice Served Food Prepared Food Served Appetizer Served Table Closed Aggregate Root : Dinning Order Billed Order T1 Payment CC T2 Payment Cash T3 T1 (sp) T2 (sp) T3 (sp) Event Stream Aggregate Root : Food Bill Transaction doesn't rollback if one payment method is failed. It moves forward to the NEXT one. sp Network Error C1 sp
  24. @arafkarsh arafkarsh Choreography Vs. Orchestration 43 1. Choreography: In this

    model, each local transaction publishes domain events that trigger local transactions in other services. There is no central coordination. The logic is distributed among the participants, which means that it must be a collaborative process. 2. Orchestration: In this model, an orchestrator (object) tells the participants what local transactions to execute. The logic for the distributed transaction is centralized in the orchestrator. Comparison of Event Choreography and Orchestration Techniques in Microservices Architecture Chaitanya K Rudrabhatla, Executive Director, Solution Architect, Media & Entertainment Domain, LA USA
  25. @arafkarsh arafkarsh Choreography: Local SAGA Features 44 1. Part of

    the Microservices 2. Local Transactions and Compensation Transactions 3. SAGA State is persisted 4. All the Local transactions are based on Single Phase Commit (1 PC) 5. Developers need to ensure that appropriate compensating transactions are Raised in the event of a failure. API Examples @StartSaga(name=“HotelBooking”) public void reserveRoom(…) { } @EndSaga(name=“HotelBooking”) public void payForTickets(…) { } @AbortSaga(name=“HotelBooking”) public void cancelBooking(…) { } @CompensationTx() public void cancelReservation(…) { }
  26. @arafkarsh arafkarsh Choreography: Use Case: Travel Booking 45 Travel :

    Hotel Booking / Car Booking / Flight Booking Example Room Reserved T1 Room Payment T2 Car Reserved T3 Car Payment T4 Cancelled Room Reservation C1 Cancelled Room Payment C2 Cancelled Car Reservation C3 Seat Reserved T5 Ticket Payment T6 Cancelled Seat Reservation C4 Cancelled Ticket Payment C5 Distributed Tx Done Hotel Microservice Rental Microservice Travel Microservice
  27. @arafkarsh arafkarsh Orchestration: SAGA Execution Container (SEC) 46 1. SEC

    is a separate Process 2. Stateless in nature and Saga state is stored in a messaging system (Kafka is a Good choice). 3. SEC process failure MUST not affect Saga Execution as the restart of the SEC must start from where the Saga left. 4. SEC – No Single Point of Failure (Master Slave Model). 5. Distributed SAGA Rules are defined using a DSL.
  28. @arafkarsh arafkarsh Orchestrator: Use Case: Travel Booking – Distributed Saga

    (SEC) 47 Hotel Booking Car Booking Flight Booking Saga Execution Container Start Saga {Booking Request} Payment End Saga Start Saga Start Hotel End Hotel Start Car End Car Start Flight End Flight Start Payment End Payment Saga Log End Saga {Booking Confirmed} SEC knows the structure of the distributed Saga and for each of the Request Which Service needs to be called and what kind of Recovery mechanism it needs to be followed. SEC can parallelize the calls to multiple services to improve the performance. The Rollback or Roll forward will be dependent on the business case. Source: Distributed Sagas By Catitie McCaffrey, June 6, 2017
  29. @arafkarsh arafkarsh Orchestrator: Use Case : Travel Booking – Rollback

    48 Hotel Booking Car Booking Flight Booking Saga Execution Container Start Saga {Booking Request} Payment Start Comp Saga End Comp Saga Start Hotel End Hotel Start Car Abort Car Cancel Hotel Cancel Flight Saga Log End Saga {Booking Cancelled} Kafka is a good choice to implement the SEC log. SEC is completely STATELESS in nature. Master Slave model can be implemented to avoid the Single Point of Failure. Source: Distributed Sagas By Catitie McCaffrey, June 6, 2017
  30. @arafkarsh arafkarsh Performance: Choreography Vs. Orchestration 49 175 ms :

    Choreography 8500 ms : Orchestration Comparison of Event Choreography and Orchestration Techniques in Microservices Architecture Chaitanya K Rudrabhatla, Executive Director, Solution Architect, Media & Entertainment Domain, LA USA
  31. @arafkarsh arafkarsh Choreography Service: Use Case: Travel Booking 50 Travel

    : Hotel Booking / Car Booking / Flight Booking Example Room Reserved T2 Room Payment T3 Car Reserved T4 Car Payment T5 Cancelled Room Reservation C1 Cancelled Room Payment C2 Cancelled Car Reservation C3 Seat Reserved T6 Ticket Payment T7 Cancelled Seat Reservation C4 Cancelled Ticket Payment C5 Distributed Tx Done Hotel Microservice Rental Microservice Travel Microservice Reservation Microservice Reservation Completed T8 Reservation Incomplete T8 Reservation Request T1 S
  32. @arafkarsh arafkarsh Scalability Requirement in Cloud 51 1. Availability and

    Partition Tolerance is more important than immediate Consistency. 2. Eventual Consistency is more suitable in a highly scalable Cloud Environment 3. Two Phase Commit has its limitations from Scalability perspective and it’s a Single Point of Failure. 4. Scalability examples from eBay, Amazon, Netflix, Uber, Airbnb etc.
  33. @arafkarsh arafkarsh Libraries to Support Saga Design Pattern 52 1.

    Eventuate Tram and Eventuate Local (Both): Eventuate Tram and Eventuate Local are Java libraries that help handle distributed transactions using both choreography and orchestration. Eventuate Tram supports traditional JDBC/JPA- based persistence, while Eventuate Local supports event sourcing. 2. Axon Framework (Both): Axon is a Java-based framework that can handle both the orchestration and choreography of Saga design patterns. This framework is designed for building scalable and extensible microservices. 3. Spring State Machine (Orchestration): While not explicitly a Saga pattern library, the Spring State Machine can be adapted to orchestrate sagas. It's designed to manage stateful entities in Spring-based Java applications, which can be used to coordinate transactions across multiple services. 4. Camunda (Orchestration): Camunda is an open-source platform for workflow and decision automation. While not a dedicated Saga pattern library, it can handle the orchestration of sagas using BPMN workflows. It supports Java, but it also has REST APIs, so it's accessible from other languages. 5. Netflix Conductor (Orchestration): Conductor is a microservices orchestration engine developed by Netflix. It's primarily used for orchestrating complex distributed systems and can be used to manage sagas. It supports Java. 6. Zeebe (Orchestration): Zeebe is a workflow engine designed to meet the scalability requirements of high- performance applications running on cloud-native and event-driven architectures. It can handle the orchestration of sagas and supports Java, among other languages. 7. Cadence Workflow (Both): Cadence is a distributed, scalable, durable, and highly available orchestration engine developed by Uber. It supports long-running, durable workflows, and it can manage sagas. The Cadence Java client provides a framework for orchestrating sagas.
  34. @arafkarsh arafkarsh Summary: 53 1. 2 Phase Commit Doesn’t scale

    well in cloud environment 2. SAGA Design Pattern Raise compensating events when the local transaction fails. 3. SAGA Supports Rollbacks & Roll Forwards Critical pattern to address distributed transactions.
  35. @arafkarsh arafkarsh 55 Enterprise Integration Patterns – Messaging System Page

    Number from Enterprise Integration Patterns Pattern Description Page 1 Message Channel The Message Channel is an internal implementation detail of the Endpoint interface and all interactions with the Message Channel are via the Endpoint Interfaces. 60 2 Message To support various message exchange patterns like one-way Event Messages and Request Reply messages, Camel uses an Exchange interface that has a pattern property that can be set to In- Only for an Event Message which has a single inbound Message or In-Out for a Request-Reply where there is an inbound and outbound message. 53 3 Message Endpoint Supports 4 endpoints • Web Services: REST & SOAP • Database : JDBC • File : Reads File 95 4 Message Translator We have 3 built-in Processors 1. SQL Processor (Take DTO and pass value to query) 2. SOAP Processor (Generate SOAP Message) 3. REST Processor(Take DTO convert into respected format 85
  36. @arafkarsh arafkarsh 56 Enterprise Integration Patterns – Messaging Systems Pattern

    Description Page 5 Pipes and Filters Supports the Pipes and Filters from the EIP patterns in various ways. You can split your processing across multiple independent Endpoint instances which can then be chained together. 70 6 Message Router The Message Router from the EIP patterns allows you to consume from an input destination, evaluate some predicate then choose the right output destination. 78 Page Number from Enterprise Integration Patterns
  37. @arafkarsh arafkarsh 57 Enterprise Integration Patterns – Messaging Channels Pattern

    Description Page 7 Publish Subscribe System Supports the Publish Subscribe Channel from the EIP patterns using for example the following components: • JMS for working with JMS Topics for high performance, clustering and load balancing • XMPP when using rooms for group communication • SEDA for working with SEDA in the same Camel Context which can work in pub- sub, but allowing multiple consumers. • VM as SEDA but for intra-JVM. 106 Page Number from Enterprise Integration Patterns
  38. @arafkarsh arafkarsh 58 Enterprise Integration Patterns – Messaging Channels Pattern

    Description Page 8 Point to Point Channel Supports the Point to Point Channel from the EIP patterns using the following components 1. SEDA for in-VM seda based messaging 2. JMS for working with JMS Queues for high performance, clustering and load balancing 3. JPA for using a database as a simple message queue 4. XMPP for point-to-point communication over XMPP (Jabber) 5. and others 103 9 Event Driven Consumer Supports the Event Driven Consumer from the EIP patterns. The default consumer model is event based (i.e. asynchronous) as this means that the Camel container can then manage pooling, threading and concurrency for you in a declarative manner. The Event Driven Consumer is implemented by consumers implementing the Processor interface which is invoked by the Message Endpoint when a Message is available for processing. 498 Page Number from Enterprise Integration Patterns
  39. @arafkarsh arafkarsh 59 Enterprise Integration Patterns – Message Routing Pattern

    Description Page 10 Aggregator The Aggregator from the EIP patterns allows you to combine a number of messages together into a single message. 268 11 Content Based Router The Content Based Router from the EIP patterns allows you to route messages to the correct destination based on the contents of the message exchanges. 230 12 Message Filter The Message Filter from the EIP patterns allows you to filter messages. 237 Page Number from Enterprise Integration Patterns
  40. @arafkarsh arafkarsh 60 Enterprise Integration Patterns – Message Routing Pattern

    Description Page 13 Routing Slip The Routing Slip from the EIP patterns allows you to route a message consecutively through a series of processing steps where the sequence of steps is not known at design time and can vary for each message. 301 14 Load Balancer The Load Balancer Pattern allows you to delegate to one of a number of endpoints using a variety of different load balancing policies. 15 Multicast The Multicast allows to route the same message to a number of endpoints and process them in a different way. The main difference between the Multicast and Splitter is that Splitter will split the message into several pieces but the Multicast will not modify the request message. Page Number from Enterprise Integration Patterns
  41. @arafkarsh arafkarsh 61 Enterprise Integration Patterns – Message Transformation Pattern

    Description Page 16 Content Enricher Supports the Content Enricher from the EIP patterns using a Message Translator, an arbitrary Processor in the routing logic, or using the enrich DSL element to enrich the message. 336 17 Content Filter Supports the Content Filter from the EIP patterns using one of the following mechanisms in the routing logic to transform content from the inbound message. • Message Translator • invoking a Java bean • Processor object 342 Page Number from Enterprise Integration Patterns
  42. @arafkarsh arafkarsh 62 Enterprise Integration Patterns – Others Pattern Description

    Page 18 Request Reply Supports the Request Reply from the EIP patterns by supporting the Exchange Pattern on a Message which can be set to In Out to indicate a request/reply. Camel Components then implement this pattern using the underlying transport or protocols. 154 19 Wire tap Wire Tap (from the EIP patterns) allows you to route messages to a separate location while they are being forwarded to the ultimate destination. 547 Page Number from Enterprise Integration Patterns
  43. @arafkarsh arafkarsh 63 Design Patterns – Service Design Pattern Patterns

    Description Page 1 Request Mapper How can a service process data from requests that are structurally different yet semantically equivalent? 109 2 Response Mapper How can the logic required to construct a response be re-used by multiple services? 122 3 Data Source Adapter How can web service provide access to internal resources like database tables, domain objects, or files with a minimum amount of custom code? 137 4 Asynchronous Response Handler How can a client avoid blocking when sending a request? 184 5 Service Connector How can clients avoid duplicating the code required to use a specific service and also be insulated from the intricacies of communication logic? 168 Page Number from service Design Patterns
  44. @arafkarsh arafkarsh 64 Design Patterns are solutions to general problems

    that software developers faced during software development. Design Patterns
  45. @arafkarsh arafkarsh 65 Thank you DREAM | AUTOMATE | EMPOWER

    Araf Karsh Hamid : India: +91.999.545.8627 http://www.slideshare.net/arafkarsh https://speakerdeck.com/arafkarsh https://www.linkedin.com/in/arafkarsh/ https://www.youtube.com/user/arafkarsh/playlists http://www.arafkarsh.com/ @arafkarsh arafkarsh
  46. @arafkarsh arafkarsh References 68 1. July 15, 2015 – Agile

    is Dead : GoTo 2015 By Dave Thomas 2. Apr 7, 2016 - Agile Project Management with Kanban | Eric Brechner | Talks at Google 3. Sep 27, 2017 - Scrum vs Kanban - Two Agile Teams Go Head-to-Head 4. Feb 17, 2019 - Lean vs Agile vs Design Thinking 5. Dec 17, 2020 - Scrum vs Kanban | Differences & Similarities Between Scrum & Kanban 6. Feb 24, 2021 - Agile Methodology Tutorial for Beginners | Jira Tutorial | Agile Methodology Explained. Agile Methodologies
  47. @arafkarsh arafkarsh References 69 1. Vmware: What is Cloud Architecture?

    2. Redhat: What is Cloud Architecture? 3. Cloud Computing Architecture 4. Cloud Adoption Essentials: 5. Google: Hybrid and Multi Cloud 6. IBM: Hybrid Cloud Architecture Intro 7. IBM: Hybrid Cloud Architecture: Part 1 8. IBM: Hybrid Cloud Architecture: Part 2 9. Cloud Computing Basics: IaaS, PaaS, SaaS 1. IBM: IaaS Explained 2. IBM: PaaS Explained 3. IBM: SaaS Explained 4. IBM: FaaS Explained 5. IBM: What is Hypervisor? Cloud Architecture
  48. @arafkarsh arafkarsh References 70 Microservices 1. Microservices Definition by Martin

    Fowler 2. When to use Microservices By Martin Fowler 3. GoTo: Sep 3, 2020: When to use Microservices By Martin Fowler 4. GoTo: Feb 26, 2020: Monolith Decomposition Pattern 5. Thought Works: Microservices in a Nutshell 6. Microservices Prerequisites 7. What do you mean by Event Driven? 8. Understanding Event Driven Design Patterns for Microservices
  49. @arafkarsh arafkarsh References – Microservices – Videos 71 1. Martin

    Fowler – Micro Services : https://www.youtube.com/watch?v=2yko4TbC8cI&feature=youtu.be&t=15m53s 2. GOTO 2016 – Microservices at NetFlix Scale: Principles, Tradeoffs & Lessons Learned. By R Meshenberg 3. Mastering Chaos – A NetFlix Guide to Microservices. By Josh Evans 4. GOTO 2015 – Challenges Implementing Micro Services By Fred George 5. GOTO 2016 – From Monolith to Microservices at Zalando. By Rodrigue Scaefer 6. GOTO 2015 – Microservices @ Spotify. By Kevin Goldsmith 7. Modelling Microservices @ Spotify : https://www.youtube.com/watch?v=7XDA044tl8k 8. GOTO 2015 – DDD & Microservices: At last, Some Boundaries By Eric Evans 9. GOTO 2016 – What I wish I had known before Scaling Uber to 1000 Services. By Matt Ranney 10. DDD Europe – Tackling Complexity in the Heart of Software By Eric Evans, April 11, 2016 11. AWS re:Invent 2016 – From Monolithic to Microservices: Evolving Architecture Patterns. By Emerson L, Gilt D. Chiles 12. AWS 2017 – An overview of designing Microservices based Applications on AWS. By Peter Dalbhanjan 13. GOTO Jun, 2017 – Effective Microservices in a Data Centric World. By Randy Shoup. 14. GOTO July, 2017 – The Seven (more) Deadly Sins of Microservices. By Daniel Bryant 15. Sept, 2017 – Airbnb, From Monolith to Microservices: How to scale your Architecture. By Melanie Cubula 16. GOTO Sept, 2017 – Rethinking Microservices with Stateful Streams. By Ben Stopford. 17. GOTO 2017 – Microservices without Servers. By Glynn Bird.
  50. @arafkarsh arafkarsh References 72 Domain Driven Design 1. Oct 27,

    2012 What I have learned about DDD Since the book. By Eric Evans 2. Mar 19, 2013 Domain Driven Design By Eric Evans 3. Jun 02, 2015 Applied DDD in Java EE 7 and Open Source World 4. Aug 23, 2016 Domain Driven Design the Good Parts By Jimmy Bogard 5. Sep 22, 2016 GOTO 2015 – DDD & REST Domain Driven API’s for the Web. By Oliver Gierke 6. Jan 24, 2017 Spring Developer – Developing Micro Services with Aggregates. By Chris Richardson 7. May 17. 2017 DEVOXX – The Art of Discovering Bounded Contexts. By Nick Tune 8. Dec 21, 2019 What is DDD - Eric Evans - DDD Europe 2019. By Eric Evans 9. Oct 2, 2020 - Bounded Contexts - Eric Evans - DDD Europe 2020. By. Eric Evans 10. Oct 2, 2020 - DDD By Example - Paul Rayner - DDD Europe 2020. By Paul Rayner
  51. @arafkarsh arafkarsh References 73 Event Sourcing and CQRS 1. IBM:

    Event Driven Architecture – Mar 21, 2021 2. Martin Fowler: Event Driven Architecture – GOTO 2017 3. Greg Young: A Decade of DDD, Event Sourcing & CQRS – April 11, 2016 4. Nov 13, 2014 GOTO 2014 – Event Sourcing. By Greg Young 5. Mar 22, 2016 Building Micro Services with Event Sourcing and CQRS 6. Apr 15, 2016 YOW! Nights – Event Sourcing. By Martin Fowler 7. May 08, 2017 When Micro Services Meet Event Sourcing. By Vinicius Gomes
  52. @arafkarsh arafkarsh References 74 Kafka 1. Understanding Kafka 2. Understanding

    RabbitMQ 3. IBM: Apache Kafka – Sept 18, 2020 4. Confluent: Apache Kafka Fundamentals – April 25, 2020 5. Confluent: How Kafka Works – Aug 25, 2020 6. Confluent: How to integrate Kafka into your environment – Aug 25, 2020 7. Kafka Streams – Sept 4, 2021 8. Kafka: Processing Streaming Data with KSQL – Jul 16, 2018 9. Kafka: Processing Streaming Data with KSQL – Nov 28, 2019
  53. @arafkarsh arafkarsh References 75 Databases: Big Data / Cloud Databases

    1. Google: How to Choose the right database? 2. AWS: Choosing the right Database 3. IBM: NoSQL Vs. SQL 4. A Guide to NoSQL Databases 5. How does NoSQL Databases Work? 6. What is Better? SQL or NoSQL? 7. What is DBaaS? 8. NoSQL Concepts 9. Key Value Databases 10. Document Databases 11. Jun 29, 2012 – Google I/O 2012 - SQL vs NoSQL: Battle of the Backends 12. Feb 19, 2013 - Introduction to NoSQL • Martin Fowler • GOTO 2012 13. Jul 25, 2018 - SQL vs NoSQL or MySQL vs MongoDB 14. Oct 30, 2020 - Column vs Row Oriented Databases Explained 15. Dec 9, 2020 - How do NoSQL databases work? Simply Explained! 1. Graph Databases 2. Column Databases 3. Row Vs. Column Oriented Databases 4. Database Indexing Explained 5. MongoDB Indexing 6. AWS: DynamoDB Global Indexing 7. AWS: DynamoDB Local Indexing 8. Google Cloud Spanner 9. AWS: DynamoDB Design Patterns 10. Cloud Provider Database Comparisons 11. CockroachDB: When to use a Cloud DB?
  54. @arafkarsh arafkarsh References 76 Docker / Kubernetes / Istio 1.

    IBM: Virtual Machines and Containers 2. IBM: What is a Hypervisor? 3. IBM: Docker Vs. Kubernetes 4. IBM: Containerization Explained 5. IBM: Kubernetes Explained 6. IBM: Kubernetes Ingress in 5 Minutes 7. Microsoft: How Service Mesh works in Kubernetes 8. IBM: Istio Service Mesh Explained 9. IBM: Kubernetes and OpenShift 10. IBM: Kubernetes Operators 11. 10 Consideration for Kubernetes Deployments Istio – Metrics 1. Istio – Metrics 2. Monitoring Istio Mesh with Grafana 3. Visualize your Istio Service Mesh 4. Security and Monitoring with Istio 5. Observing Services using Prometheus, Grafana, Kiali 6. Istio Cookbook: Kiali Recipe 7. Kubernetes: Open Telemetry 8. Open Telemetry 9. How Prometheus works 10. IBM: Observability vs. Monitoring
  55. @arafkarsh arafkarsh References 77 1. Feb 6, 2020 – An

    introduction to TDD 2. Aug 14, 2019 – Component Software Testing 3. May 30, 2020 – What is Component Testing? 4. Apr 23, 2013 – Component Test By Martin Fowler 5. Jan 12, 2011 – Contract Testing By Martin Fowler 6. Jan 16, 2018 – Integration Testing By Martin Fowler 7. Testing Strategies in Microservices Architecture 8. Practical Test Pyramid By Ham Vocke Testing – TDD / BDD
  56. @arafkarsh arafkarsh 78 1. Simoorg : LinkedIn’s own failure inducer

    framework. It was designed to be easy to extend and most of the important components are plug‐ gable. 2. Pumba : A chaos testing and network emulation tool for Docker. 3. Chaos Lemur : Self-hostable application to randomly destroy virtual machines in a BOSH- managed environment, as an aid to resilience testing of high-availability systems. 4. Chaos Lambda : Randomly terminate AWS ASG instances during business hours. 5. Blockade : Docker-based utility for testing network failures and partitions in distributed applications. 6. Chaos-http-proxy : Introduces failures into HTTP requests via a proxy server. 7. Monkey-ops : Monkey-Ops is a simple service implemented in Go, which is deployed into an OpenShift V3.X and generates some chaos within it. Monkey-Ops seeks some OpenShift components like Pods or Deployment Configs and randomly terminates them. 8. Chaos Dingo : Chaos Dingo currently supports performing operations on Azure VMs and VMSS deployed to an Azure Resource Manager-based resource group. 9. Tugbot : Testing in Production (TiP) framework for Docker. Testing tools
  57. @arafkarsh arafkarsh References 79 CI / CD 1. What is

    Continuous Integration? 2. What is Continuous Delivery? 3. CI / CD Pipeline 4. What is CI / CD Pipeline? 5. CI / CD Explained 6. CI / CD Pipeline using Java Example Part 1 7. CI / CD Pipeline using Ansible Part 2 8. Declarative Pipeline vs Scripted Pipeline 9. Complete Jenkins Pipeline Tutorial 10. Common Pipeline Mistakes 11. CI / CD for a Docker Application
  58. @arafkarsh arafkarsh References 80 DevOps 1. IBM: What is DevOps?

    2. IBM: Cloud Native DevOps Explained 3. IBM: Application Transformation 4. IBM: Virtualization Explained 5. What is DevOps? Easy Way 6. DevOps?! How to become a DevOps Engineer??? 7. Amazon: https://www.youtube.com/watch?v=mBU3AJ3j1rg 8. NetFlix: https://www.youtube.com/watch?v=UTKIT6STSVM 9. DevOps and SRE: https://www.youtube.com/watch?v=uTEL8Ff1Zvk 10. SLI, SLO, SLA : https://www.youtube.com/watch?v=tEylFyxbDLE 11. DevOps and SRE : Risks and Budgets : https://www.youtube.com/watch?v=y2ILKr8kCJU 12. SRE @ Google: https://www.youtube.com/watch?v=d2wn_E1jxn4
  59. @arafkarsh arafkarsh References 81 1. Lewis, James, and Martin Fowler.

    “Microservices: A Definition of This New Architectural Term”, March 25, 2014. 2. Miller, Matt. “Innovate or Die: The Rise of Microservices”. e Wall Street Journal, October 5, 2015. 3. Newman, Sam. Building Microservices. O’Reilly Media, 2015. 4. Alagarasan, Vijay. “Seven Microservices Anti-patterns”, August 24, 2015. 5. Cockcroft, Adrian. “State of the Art in Microservices”, December 4, 2014. 6. Fowler, Martin. “Microservice Prerequisites”, August 28, 2014. 7. Fowler, Martin. “Microservice Tradeoffs”, July 1, 2015. 8. Humble, Jez. “Four Principles of Low-Risk Software Release”, February 16, 2012. 9. Zuul Edge Server, Ketan Gote, May 22, 2017 10. Ribbon, Hysterix using Spring Feign, Ketan Gote, May 22, 2017 11. Eureka Server with Spring Cloud, Ketan Gote, May 22, 2017 12. Apache Kafka, A Distributed Streaming Platform, Ketan Gote, May 20, 2017 13. Functional Reactive Programming, Araf Karsh Hamid, August 7, 2016 14. Enterprise Software Architectures, Araf Karsh Hamid, July 30, 2016 15. Docker and Linux Containers, Araf Karsh Hamid, April 28, 2015
  60. @arafkarsh arafkarsh References 82 16. MSDN – Microsoft https://msdn.microsoft.com/en-us/library/dn568103.aspx 17.

    Martin Fowler : CQRS – http://martinfowler.com/bliki/CQRS.html 18. Udi Dahan : CQRS – http://www.udidahan.com/2009/12/09/clarified-cqrs/ 19. Greg Young : CQRS - https://www.youtube.com/watch?v=JHGkaShoyNs 20. Bertrand Meyer – CQS - http://en.wikipedia.org/wiki/Bertrand_Meyer 21. CQS : http://en.wikipedia.org/wiki/Command–query_separation 22. CAP Theorem : http://en.wikipedia.org/wiki/CAP_theorem 23. CAP Theorem : http://www.julianbrowne.com/article/viewer/brewers-cap-theorem 24. CAP 12 years how the rules have changed 25. EBay Scalability Best Practices : http://www.infoq.com/articles/ebay-scalability-best-practices 26. Pat Helland (Amazon) : Life beyond distributed transactions 27. Stanford University: Rx https://www.youtube.com/watch?v=y9xudo3C1Cw 28. Princeton University: SAGAS (1987) Hector Garcia Molina / Kenneth Salem 29. Rx Observable : https://dzone.com/articles/using-rx-java-observable