Building Cloud-Native App Series - Part 2 of 12
Microservices Architecture Series
Event Sourcing & CQRS,
Kafka, Rabbit MQ
Case Studies
- E-Commerce App,
- Movie Streaming,
- Ticket Booking,
- Restaurant,
- Hospital Management
@arafkarsh arafkarsh 8 Years Network & Security 6+ Years Microservices Blockchain 8 Years Cloud Computing 8 Years Distributed Computing Architecting & Building Apps a tech presentorial Combination of presentation & tutorial ARAF KARSH HAMID Co-Founder / CTO MetaMagic Global Inc., NJ, USA @arafkarsh arafkarsh 1 Microservice Architecture Series Building Cloud Native Apps Kafka, Replication Event Storming Distributed Tx Case Studies Part 2 of 12
@arafkarsh arafkarsh 2 Slides are color coded based on the topic colors. Event Streaming Kafka Replication 1 Event Storming Event Sourcing & CQRS 2 Distributed Transactions 3 Case Studies 4
@arafkarsh arafkarsh Problem Statement – Async Calls : Queue Based 6 Check Out Order Inventory Notification Service eMail SMS Cart • Scalability Issues • Multiple Sub Scribers are not allowed (to the same topic) Issues
@arafkarsh arafkarsh Async Calls : Fanout Exchange 8 Check Out Order Inventory Notification Service eMail SMS Cart 1. Loosely Coupled Systems 2. Scalable Exchange Duplicates the message & sends it to respective Queues Binding key
@arafkarsh arafkarsh Async Calls : Direct Exchange 9 Check Out Order Inventory Notification Service eMail SMS Cart 1. Loosely Coupled Systems 2. Scalable Exchange Duplicates the message & sends it to respective Queues Binding key Message Contains Routing Key which needs to match with Binding Key
@arafkarsh arafkarsh Async Calls : Topic Exchange 10 Check Out Order Inventory Notification Service eMail SMS Cart 1. Loosely Coupled Systems 2. Scalable Exchange Duplicates the message & sends it to respective Queues order.any Message Contains Routing Key which says order.phone then it can do a partial match with order.any
@arafkarsh arafkarsh Async Calls : Header Exchange 11 Check Out Order Inventory Notification Service eMail SMS Cart 1. Loosely Coupled Systems 2. Scalable Exchange Duplicates the message & sends it to respective Queues order.any Message Routing happens based on the Header
@arafkarsh arafkarsh Async Calls : Default Exchange 12 Check Out Order Inventory Notification Service eMail SMS Cart 1. Loosely Coupled Systems 2. Scalable Exchange Duplicates the message & sends it to respective Queues Binding key Message is moved forward if the Routing Key matches the Queue Name que-inv que-inv que-ord que-notification
@arafkarsh arafkarsh Discovering Microservices Principles…. 13 Components via Services Organized around Business Capabilities Products NOT Projects Smart Endpoints & Dumb Pipes Decentralized Governance & Data Management Infrastructure Automation Design for Failure Evolutionary Design How does the routing rules defy Microservices Principles?
@arafkarsh arafkarsh Discovering Microservices Principles…. 14 Components via Services Organized around Business Capabilities Products NOT Projects Smart Endpoints & Dumb Pipes Decentralized Governance & Data Management Infrastructure Automation Design for Failure Evolutionary Design How does the routing rules defy Microservices Principles?
@arafkarsh arafkarsh Async Calls : Kafka Solution 16 Check Out Cart 1. Highly Scalable 2. Multi Subscriber 3. Loosely Coupled Systems 4. Data Durability (Replication) 5. Ordering Guarantee (Per Partition) Use Partition Key Kafka Producer API Kafka Consumer API eMail SMS 1 2 3 4 1 2 3 4 1 2 Service Instances Order Topic (Total Partitions 6) Kafka Storage Replicated Logs Kafka Cluster 5 6 7 8 7 8 What will happen to Inventory Instance 7 and 8? Order Consumer Group Inv Consumer Group Notification Consumer Multiple Subscriber As there are only 6 Partitions Kafka can serve ONLY 6 consumers within a partition
@arafkarsh arafkarsh Async Calls : Kafka Solution 17 Check Out Cart 4. Data Durability (Replication) 5. Ordering Guarantee (Per Partition) Use Partition Key Kafka Producer API Kafka Consumer API 1 2 3 4 1 2 3 4 3 4 Service Instances Order Topic (Total Partitions 6) Kafka Storage Replicated Logs Kafka Cluster 5 6 7 8 7 8 What will happen to Inventory Instance 7 and 8? Order Consumer Group Inv Consumer Group Multiple Subscriber As there are only 6 Partitions Kafka can serve ONLY 6 consumers within a partition 2 5 1 Broadcast Orders to following Consumers All the above Consumers will get same orders available in the Order Topic 1. Highly Scalable 2. Multi Subscriber 3. Loosely Coupled Systems
@arafkarsh arafkarsh Kafka Replication, HA & Load Balancing 18 Order Consumer Group 1 2 3 3 Instances of Order Service Server 2 P1 P3 P6 Server 1 P2 P4 P5 Order Topic Partitions 6 – Split into 2 Servers P1 P3 P6 P2 P4 P5 Replication 1. Partitions are replicated in both Server 1 and Server 2 2. P1, P3, P6 are Leaders in Server 1 and followers in Server 2 3. P2, P4, P5 are Leaders in Server 2 and followers in Server 1 High Availability 1. If Server 1 goes down then followers in Server 2 will become the leader and vice versa. Load Balancing, Performance and Scalability 1. Horizontal Scalability is achieved by adding more servers. 2. Partitions from both servers are assigned to various consumers of Order Service. Order Consumer • C1 = P1, P2 • C2 = P3, P4 • C3 = P6, P5 2 Partitions each Pn Leader in Server 1 Follower in Server 2 Pn Leader in Server 2 Follower in Server 1
@arafkarsh arafkarsh Kafka – Ordering Guarantees 19 1. Messages that require relative ordering needs to be sent to the same Partition. Kafka takes care of this part. 2. Supply the same key for all the messages that require Relative Ordering. 3. For example if the Customer Order requires relative ordering then Order ID will be the Key. All the messages for the same Order ID will be sent to the same partition. 4. To maintain a Global Ordering without a key then use a Single Partition Topic. This will limit the scalability,
@arafkarsh arafkarsh Traditional Queue / Pub-Sub Vs. Kafka 20 0 1 2 3 4 5 6 7 8 9 8 7 9 Consumer 1 Consumer 2 Consumer 3 Queues Data Data can be partitioned for scalability for parallel processing by same type of consumers Pros: Cons: 1. Queues are NOT multi subscribers compare to Pub Sub. 2. Once a Consumer reads the data, it’s gone from the queue. 3. Ordering of records will be lost in asynchronous parallel processing. 0 1 2 3 4 5 6 7 8 9 9 9 9 Consumer 1 Consumer 2 Consumer 3 Pub – Sub Data Multiple subscribers can get the same data. Pros: Scaling is difficult as every message goes to every subscriber. Cons:
@arafkarsh arafkarsh Traditional Queue, Pub Sub Vs. Kafka 21 Order Consumer Group Inv Consumer Group 1 2 3 4 1 2 3 Service Instances Order Topic Total Partitions 6 – Split into 2 Servers Server 1 Server 2 P1 P3 P6 P2 P4 P5 Queue Implementation 1. Partition replaces the Queues & Consumer (within a Group) will retrieve the message from 1 or more Partition. 2. Each Consumer from within a Group will be assigned different partitions. 3. Load Balancing (Assigning Partitions to Consumer) happens based on the number of Consumers in the Group. 4. If a Consumer drops out the partitions will re- assigned to another consumer within the group. 5. No. of Partitions must be greater than the Consumers within a Group. Pub Sub Implementation 1. With Multiple Consumer Group (Ex., Order & Inventory) the Same Message (Event) is available to all the groups subscribed to the same Topic. • Order Consumer • C1 = P1, P3 • C2 = P6, P2 • C3 = P4, P5 2 Partitions each • Inventory Consumer • I1 = P1, P4 • I2 = P3, P5 • I3= P6 • I4 = P2 2 Partitions 1Partition each
@arafkarsh arafkarsh Kafka Core Concepts 23 Publish & Subscribe Read and write streams of data like a messaging system Process Write scalable stream processing apps that react to events in real- time. Store Store streams of data safely in a distributed, replicated, fault tolerant cluster.
@arafkarsh arafkarsh Kafka APIs 24 Source : https://kafka.apache.org/documentation/#gettingStarted • The Producer API allows an application to publish a stream of records to one or more Kafka topics. • The Consumer API allows an application to subscribe to one or more topics and process the stream of records produced to them. • The Streams API allows an application to act as a stream processor, consuming an input stream from one or more topics and producing an output stream to one or more output topics, effectively transforming the input streams to output streams. • The Connector API allows building and running reusable producers or consumers that connect Kafka topics to existing applications or data systems. For example, a connector to a relational database might capture every change to a table.
@arafkarsh arafkarsh Kafka Cluster – Topics & Partitions • The partitions of the log are distributed over the servers in the Kafka cluster with each server handling data and requests for a share of the partitions. Source : https://kafka.apache.org/intro m1, m2 Broker 1 Leader (A) Broker 2 Follower (B) m1,m2 Broker 3 Follower C p1 Broker 4 Follower (B,C) m1 p1,p2 Broker 5 Leader A p1,p2 Partition 1 Partition 0 Topic ABC • Each server acts as a leader for some of its partitions and a follower for others so load is well balanced within the cluster. • Each partition has one server which acts as the "leader" and zero or more servers which act as "followers". 27
@arafkarsh arafkarsh Record Commit Process Broker 1 Leader Topic 1 Broker 2 Follower Producer Consumer 3 3 Commit 2 ack • Each partition is replicated across a configurable number of servers for fault tolerance. • The leader handles all read and write requests for the partition while the followers passively replicate the leader. • If the leader fails, one of the followers will automatically become the new leader. 1 Message with Offset 4 777743 Broker 3 Follower Data Durability From Kafka v0.8.0 onwards acks Acknowledgement Description 0 If set to zero then the producer will NOT wait for any acknowledgment from the server at all. The record will be immediately added to the socket buffer and considered sent. No guarantee can be made that the server has received the record in this case, and the retries configuration will not take effect (as the client won't generally know of any failures). The offset given back for each record will always be set to -1. 1 This will mean the leader will write the record to its local log but will respond without awaiting full acknowledgement from all followers. In this case should the leader fail immediately after acknowledging the record but before the followers have replicated it then the record will be lost. All / -1 This means the leader will wait for the full set of in-sync replicas to acknowledge the record. This guarantees that the record will not be lost as long as at least one in-sync replica remains alive. This is the strongest available guarantee. This is equivalent to the acks=-1 setting. Source: https://kafka.apache.org/documentation/#topicconfigs acks Steps 0 1 1 1,2 -1 1,2,3 Producer Configuration 28
@arafkarsh arafkarsh Message Acknowledgements 30 m1 Follower (B) m2 m3 m4 m5 m1 Follower (C) m2 m3 m4 m1 Leader (A) m2 m3 m4 m5 Producer acks=all m5 m1 Follower (B) m2 m3 m4 m5 m1 Follower (C) m2 m3 m4 m1 Leader (A) m2 m3 m4 m5 Producer acks=all min.insync.replicas=2 m5 Ack Producer get Ack after the available ISR = min in sync replicas = X Producer won’t get Ack as all the ISR(In Sync Replica) are not available. Because all the 3 ISR (In Sync Replicas) are Alive. Kafka Broker will send the Ack back ONLY after receiving the ack from all the three ISRs. Why is the Ack Not Coming – even after the min in sync replicas = 2? m1 Follower (B) m2 m3 m4 m5 m1 Follower (C) m2 m3 m4 m1 Leader (A) m2 m3 m4 m5 Producer acks=all m5 min.insync.replicas=2
@arafkarsh arafkarsh Replication m1 m2 m3 L(A) m1 m2 F(B) m1 F(C) ISR = (A, B, C) Leader A commits Message m1. Message m2 & m3 not yet committed. 1 m1 m2 F(C) m1 m2 L(B) m1 m2 m3 L(A) ISR = (B,C) A fails and B is the new Leader. B commits m2 2 m1 m2 m3 L(A) m1 m2 L(B) m4 m5 m1 m2 F(C) m4 m5 ISR = (B,C) B commits new messages m4 and m5 3 m1 m2 L(B) m4 m5 m1 m2 F(C) m4 m5 m1 F(A) ISR = (A, B,C) A comes back, restores to last commit and catches up to latest messages. 4 m1 m2 L(B) m4 m5 m1 m2 F(C) m4 m5 m1 m2 F(A) m4 m5 ISR – In-sync Replica • Instead of majority vote, Kafka dynamically maintains a set of in-sync replicas (ISR) that are caught-up to the leader. • Only members of this set are eligible for election as leader. • A write to a Kafka partition is not considered committed until all in-sync replicas have received the write. • This ISR set is persisted to ZooKeeper whenever it changes. Because of this, any replica in the ISR is eligible to be elected leader. 31
@arafkarsh arafkarsh Anatomy of a Topic 33 Source : https://kafka.apache.org/intro • A Topic is a category or feed name to which records are published. • Topics in Kafka are always multi subscriber. • Each Partition is an ordered, immutable sequence of records that is continually appended to—a structured commit log. • A Partition is nothing but a directory of Log Files • The records in the partitions are each assigned a sequential id number called the offset that uniquely identifies each record within the partition.
@arafkarsh arafkarsh 34 Partition Log Segment • Partition (Kafka’s Storage unit) is Directory of Log Files. • A partition cannot be split across multiple brokers or even multiple disks • Partitions are split into Segments • Segments are two files: 000.log & 000.index • Segments are named by their base offset. The base offset of a segment is an offset greater than offsets in previous segments and less than or equal to offsets in that segment. 0 1 2 3 4 5 6 7 8 9 Partition Data 6 3 0 Segment 0 Segment 3 Segment 6 9 Segment 9 - Active $ tree kafka-logs | head -n 6 kafka-logs |──── SigmaHawk-2 | |──── 00000000006109871597.index | |──── 00000000006109871597.log | |──── 00000000007306321253.index | |──── 00000000007306321253.log Topic / Partition Segment 1 Segment 2 4 Bytes 4 Bytes
@arafkarsh arafkarsh 35 Partition Log Segment • Indexes store offsets relative to its segments base offset • Indexes map each offset to their message position in the log and they are used to look up messages. • Purging of data is based on oldest segment and one segment at a time. Rel.Offset, Position Offset, Position, Size, Payload 0000.index 0000.log 0 0 0 0 7 ABCDE67 1 7 1 7 4 ABC4 2 11 2 11 9 ABCDEF89 4 Bytes 4 Bytes $ tree kafka-logs | head -n 6 kafka-logs |──── SigmaHawk-2 | |──── 00000000006109871597.index | |──── 00000000006109871597.log | |──── 00000000007306321253.index | |──── 00000000007306321253.log Topic / Partition Segment 1 Segment 2 3 20 3 20 3 AB3
@arafkarsh arafkarsh Consumer Access & Data Retention 36 Source : https://kafka.apache.org/intro • For example, if the retention policy is set to 2 days, then for the two days after a record is published, it is available for consumption, after which it will be discarded to free up space. • The Kafka cluster retains all published records—whether or not they have been consumed—using a configurable retention period • Kafka's performance is effectively constant with respect to data size so storing data for a long time is not a problem. • Only metadata retained on a per-consumer basis is the offset or position of that consumer in the log. This offset is controlled by the consumer: normally a consumer will advance its offset linearly as it reads records, but, in fact, since the position is controlled by the consumer it can consume records in any order it likes. 777743 777742 777741 777740 777739 777738 777737 777736 Producer Consumer Consumer Consumer • Producers Push Data • Consumers Poll Data Writes Reads Offset=37 Offset=38 Offset=41
@arafkarsh arafkarsh Kafka Record / Message Structure 38 Magic Attr CRC int64 int32 int8 Timestamp Header Key (Variable Length) Value (Variable Length) Payload v1 (Supported since 0.10.0) Field Description CRC The CRC is the CRC32 of the remainder of the message bytes. This is used to check the integrity of the message on the broker and consumer. Magic Byte This is a version id used to allow backwards compatible evolution of the message binary format. The current value is 2. Attributes Bit 0-2 Compression Codec 0 No Compression 1 Gzip Compression 2 Snappy Compression Bit 3 Timestamp Type: 0 for Create Time Stamp, 1 for Log Append Time Stamp Bit. 4 is Transactional (0 means Transactional) Bit 5 is Control Batch (0 means Control Batch) Bit >5. Un used Timestamp This is the timestamp of the message. The timestamp type is indicated in the attributes. Unit is milliseconds since beginning of the epoch (midnight Jan 1, 1970 (UTC)). Key The key is an optional message key that was used for partition assignment. The key can be null. Value The value is the actual message contents as an opaque byte array. Kafka supports recursive messages in which case this may itself contain a message set. The message can be null. int8 Source: https://kafka.apache.org/documentation/#messages
@arafkarsh arafkarsh Kafka Record Structure 39 v2 (Supported since 0.11.0) Length (varint) Attr int8 Timestamp Delta (varint) Offset Delta (varint) Key Length (varint) Key (varint) Value Length (varint) Value (varint) Headers (Header Array) Header Key Length (varint) Header Key (varint) Header Value Length (varint) Header Value (varint) Header Record • In Kafka 0.11, the structure of the 'Message Set' and 'Message' were significantly changed. • A 'Message Set' is now called a 'Record Batch', which contains one or more 'Records' (and not 'Messages'). • The recursive nature of the previous versions of the message format was eliminated in favor of a flat structure. • When compression is enabled, the Record Batch header remains uncompressed, but the Records are compressed together. • Multiple fields in the 'Record' are varint encoded, which leads to significant space savings for larger batches.
@arafkarsh arafkarsh Kafka Record Batch Structure 40 v2 (Supported since 0.11.0) Field Description First Offset Denotes the first offset in the Record Batch. The 'offset Delta' of each Record in the batch would be be computed relative to this First Offset. Partition Leader Epoch this is set by the broker upon receipt of a produce request and is used to ensure no loss of data when there are leader changes with log truncation. Attributes The fifth lowest bit indicates whether the Record Batch is part of a transaction or not. 0 indicates that the Record Batch is not transactional, while 1 indicates that it is. (since 0.11.0.0) Last Offset Delta The offset of the last message in the Record Batch. This is used by the broker to ensure correct behavior even when Records within a batch are compacted out. First Timestamp The timestamp of the first Record in the batch. The timestamp of each Record in the Record Batch is its 'Timestamp Delta' + 'First Timestamp'. Max Timestamp The timestamp of the last Record in the batch. This is used by the broker to ensure the correct behavior even when Records within the batch are compacted out. Producer ID This is the broker assigned producer Id received by the 'Init Producer Id' request. Producer Epoch This is the broker assigned producer Epoch received by the 'Init Producer Id' request. First Sequence This is the producer assigned sequence number which is used by the broker to de-duplicate messages. The sequence number for each Record in the Record Batch is its Offset Delta + First Sequence. First Offset int64 Length int32 Partition Leader Epoch int32 Magic int8 CRC int32 Attr int16 Last offset Delta int32 First Timestamp int64 Max Timestamp int64 Producer Epoch int16 Producer ID int64 First Sequence int32 Records (Record Array)
@arafkarsh arafkarsh Kafka Quick Setup & Demo 42 1. install the most recent version from Kafka download page 2. Extract the binaries into a /…./Softwares/kafka folder. For the current version it's kafka_2.11-1.0.0.0. 3. Change your current directory to point to the new folder. 4. Start the Zookeeper server by executing the command: bin/zookeeper-server-start.sh config/zookeeper.properties. 5. Start the Kafka server by executing the command: bin/kafka-server-start.sh config/server.properties. 6. Create a Test topic that you can use for testing: bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic KafkaSigmaTest 7. Start a simple console Consumer that can consume messages published to a given topic, such as KafkaSigmaTest: bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic KafkaSigmaTest --from-beginning. 8. Start up a simple Producer console that can publish messages to the test topic: bin/kafka-console-producer.sh --broker-list localhost:9092 --topic KafkaSigmaTest 9. Try typing one or two messages into the producer console. Your messages should show in the consumer console.
@arafkarsh arafkarsh Kafka Use Cases – High Volume Events 49 1. Social Media 2. E-Commerce – especially on a Single Day Sale 3. Location Sharing – Ride Sharing Apps 4. Data Gathering 1. Music Streaming Service 2. Web Site Analytics
@arafkarsh arafkarsh LinkedIn Kafka Cluster 51 Brokers 60 Partitions 50K Messages / Second 800K MB / Second inbound 300 MB / Second Outbound 1024 The tuning looks fairly aggressive, but all of the brokers in that cluster have a 90% GC pause time of about 21ms, and they’re doing less than 1 young GC per second.
@arafkarsh arafkarsh Event Storming • Event sourcing / cqrs • Case study: Shopping portal • Case study: Restaurant app • Case study: movie booking • Case study: movie streaming • Case Study: Patient Health Care 54 2
@arafkarsh arafkarsh Mind Shift : From Object Modeling to Process Modeling 55 Developers with Strong Object Modeling experience will have trouble making Events a first-class citizen. • How do I start Event Sourcing? • Where do I Start on Event Sourcing / CQRS? The Key is: 1. App User’s Journey 2. Business Process 3. Ubiquitous Language – DDD 4. Capability Centric Design 5. Outcome Oriented The Best tool to define your process and its tasks. How do you define your End User’s Journey & Business Process? • Think It • Build It • Run IT
@arafkarsh arafkarsh 56 Process • Define your Business Processes. Eg. Various aspects of Order Processing in an E-Commerce Site, Movie Ticket Booking, Patient visit in Hospital. 1 Commands • Define the Commands (End-User interaction with your App) to execute the Process. Eg. Add Item to Cart is a Command. 2 Event Sourced Aggregate • Current state of the Aggregate is always derived from the Event Store. Eg. Shopping Cart, Order etc. This will be part of the Rich Domain Model (Bounded Context) of the Micro Service. 4 Projections • Projections focuses on the View perspective of the Application. As the Read & Write are different Models, you can have different Projections based on your View perspective. 5 Write Data Read Data Events • Commands generates the Events to be stored in Event Store. Eg. Item Added Event (in the Shopping Cart). 3 Event Storming – Concept
@arafkarsh arafkarsh Event Sourcing Intro 57 Standard CRUD Operations – Customer Profile – Aggregate Root Profile Address Title Profile Created Profile Address New Title Title Updated Profile New Address New Title New Address added Derived Profile Address Notes Notes Removed Time T1 T2 T4 T3 Event Sourcing and Derived Aggregate Root Commands 1. Create Profile 2. Update Title 3. Add Address 4. Delete Notes 2 Events 1. Profile Created Event 2. Title Updated Event 3. Address Added Event 4. Notes Deleted Event 3 Profile Address New Title Current State of the Customer Profile 4 Event store Single Source of Truth Greg Young
@arafkarsh arafkarsh Event Sourcing & CQRS (Command and Query Responsibility Segregation) • In traditional data management systems, both commands (updates to the data) and queries (requests for data) are executed against the same set of entities in a single data repository. • CQRS is a pattern that segregates the operations that read data (Queries) from the operations that update data (Commands) by using separate interfaces. • CQRS should only be used on specific portions of a system in Bounded Context (in DDD). • CQRS should be used along with Event Sourcing. 58 MSDN – Microsoft https://msdn.microsoft.com/en-us/library/dn568103.aspx | Martin Fowler : CQRS – http://martinfowler.com/bliki/CQRS.html CQS : Bertrand Meyer Axon Framework For Java Java Axon Framework Resource : http://www.axonframework.org Greg Young
@arafkarsh arafkarsh Differences between Commands, Events & Queries 59 Behavior / Stage Change Includes a Response Command Requested to Happen Maybe Event Just Happened Never Query None Always 1. Events are Facts and Notification 2. Event wear 2 hats: Data Hats (Fact) and Notification Hats
@arafkarsh arafkarsh Case Study: Shopping Site – Event Sourcing / CQRS 60 Catalogue Shopping Cart Order Payment • Search Products • Add Products • Update Products Commands • Add to Cart • Remove Item • Update Quantity Customer • Select Address • Select Delivery Mode • Process Order Events • Product Added • Product Updated • Product Discontinued • Item Added • Item Removed / Discontinued • Item Updated • Order Initiated • Address Selected • Delivery Mode Selected • Order Created • Confirm Order for Payment • Proceed for Payment • Cancel Order • Payment Initiated • Order Cancelled • Order Confirmed • OTP Send • Payment Approved • Payment Declined Microservices • Customer • Shopping Cart • Order Customer Journey thru Shopping Process 2 Processes 1 Customers will browse through the Product catalogue to find the products, its ratings and reviews. Once the product is narrowed down the customer will add the product to shopping cart. Once the customer is ready for the purchase, he/she will start the order processing by selecting the Delivery address, delivery method, payment option. Once the payment is done customer will get the order tracking details. ES Aggregate 4 Core Domain Supporting Domain Supporting Domain Supporting Domain Generic Domain 3
@arafkarsh arafkarsh Distributed Tx: SAGA Design Pattern instead of 2PC 62 Long Lived Transactions (LLTs) hold on to DB resources for relatively long periods of time, significantly delaying the termination of shorter and more common transactions. Source: SAGAS (1987) Hector Garcia Molina / Kenneth Salem, Dept. of Computer Science, Princeton University, NJ, USA T1 T2 Tn Local Transactions C1 C2 Cn-1 Compensating Transaction Divide long–lived, distributed transactions into quick local ones with compensating actions for recovery. Travel : Flight Ticket & Hotel Booking Example BASE (Basic Availability, Soft State, Eventual Consistency) Room Reserved T1 Room Payment T2 Seat Reserved T3 Ticket Payment T4 Cancelled Room Reservation C1 Cancelled Room Payment C2 Cancelled Ticket Reservation C3
@arafkarsh arafkarsh SAGA Design Pattern Features 63 1. Backward Recovery (Rollback) T1 T2 T3 T4 C3 C2 C1 Order Processing, Banking Transactions, Ticket Booking Examples Updating individual scores in a Team Game. 2. Forward Recovery with Save Points T1 (sp) T2 (sp) T3 (sp) • To recover from Hardware Failures, SAGA needs to be persistent. • Save Points are available for both Forward and Backward Recovery. Type Source: SAGAS (1987) Hector Garcia Molina / Kenneth Salem, Dept. of Computer Science, Princeton University, NJ, USA
@arafkarsh arafkarsh Handling Invariants – Monolithic to Micro Services 64 In a typical Monolithic App Customer Credit Limit info and the order processing is part of the same App. Following is a typical pseudo code. Order Created T1 Order Microservice Credit Reserved T2 Customer Microservice In Micro Services world with Event Sourcing, it’s a distributed environment. The order is cancelled if the Credit is NOT available. If the Payment Processing is failed then the Credit Reserved is cancelled. Payment Microservice Payment Processed T3 Order Cancelled C1 Credit Cancelled due to payment failure C2 Begin Transaction If Order Value <= Available Credit Process Order Process Payments End Transaction Monolithic 2 Phase Commit https://en.wikipedia.org/wiki/Invariant_(computer_science)
@arafkarsh arafkarsh 65 Use Case : Restaurant – Forward Recovery Domain The example focus on a concept of a Restaurant which tracks the visit of an individual or group to the Restaurant. When people arrive at the Restaurant and take a table, a table is opened. They may then order drinks and food. Drinks are served immediately by the table staff, however food must be cooked by a chef. Once the chef prepared the food it can then be served. Payment Billing Dining Source: http://cqrs.nu/tutorial/cs/01-design Soda Cancelled Table Opened Juice Ordered Soda Ordered Appetizer Ordered Soup Ordered Food Ordered Juice Served Food Prepared Food Served Appetizer Served Table Closed Aggregate Root : Dinning Order Billed Order T1 Payment CC T2 Payment Cash T3 T1 (sp) T2 (sp) T3 (sp) Event Stream Aggregate Root : Food Bill Transaction doesn't rollback if one payment method is failed. It moves forward to the NEXT one. sp Network Error C1 sp
@arafkarsh arafkarsh Local SAGA Features 66 1. Part of the Micro Services 2. Local Transactions and Compensation Transactions 3. SAGA State is persisted 4. All the Local transactions are based on Single Phase Commit (1 PC) 5. Developers need to ensure that appropriate compensating transactions are Raised in the event of a failure. API Examples @StartSaga(name=“HotelBooking”) public void reserveRoom(…) { } @EndSaga(name=“HotelBooking”) public void payForTickets(…) { } @AbortSaga(name=“HotelBooking”) public void cancelBooking(…) { } @CompensationTx() public void cancelReservation(…) { }
@arafkarsh arafkarsh SAGA Execution Container 67 1. SEC is a separate Process 2. Stateless in nature and Saga state is stored in a messaging system (Kafka is a Good choice). 3. SEC process failure MUST not affect Saga Execution as the restart of the SEC must start from where the Saga left. 4. SEC – No Single Point of Failure (Master Slave Model). 5. Distributed SAGA Rules are defined using a DSL.
@arafkarsh arafkarsh Use Case : Travel Booking – Distributed Saga (SEC) 68 Hotel Booking Car Booking Flight Booking Saga Execution Container Start Saga {Booking Request} Payment End Saga Start Saga Start Hotel End Hotel Start Car End Car Start Flight End Flight Start Payment End Payment Saga Log End Saga {Booking Confirmed} SEC knows the structure of the distributed Saga and for each of the Request Which Service needs to be called and what kind of Recovery mechanism it needs to be followed. SEC can parallelize the calls to multiple services to improve the performance. The Rollback or Roll forward will be dependent on the business case. Source: Distributed Sagas By Catitie McCaffrey, June 6, 2017
@arafkarsh arafkarsh Use Case : Travel Booking – Rollback 69 Hotel Booking Car Booking Flight Booking Saga Execution Container Start Saga {Booking Request} Payment Start Comp Saga End Comp Saga Start Hotel End Hotel Start Car Abort Car Cancel Hotel Cancel Flight Saga Log End Saga {Booking Cancelled} Kafka is a good choice to implement the SEC log. SEC is completely STATELESS in nature. Master Slave model can be implemented to avoid the Single Point of Failure. Source: Distributed Sagas By Catitie McCaffrey, June 6, 2017
@arafkarsh arafkarsh Scalability Requirement in Cloud 70 1. Availability and Partition Tolerance is more important than immediate Consistency. 2. Eventual Consistency is more suitable in a highly scalable Cloud Environment 3. Two Phase Commit has its limitations from Scalability perspective and it’s a Single Point of Failure. 4. Scalability examples from eBay, Amazon, Netflix, Uber, Airbnb etc.
@arafkarsh arafkarsh Case Studies • Case Study: Shopping Portal • Case Study: Movie Streaming • Case Study: Patient Care • Case Study: Restaurant Dinning • Case Study: Movie Ticket Booking 72 4
@arafkarsh arafkarsh Process • Define your Business Processes. Eg. Various aspects of Order Processing in an E-Commerce Site, Movie Ticket Booking, Patient visit in Hospital. 1 Commands • Define the Commands (End-User interaction with your App) to execute the Process. Eg. Add Item to Cart is a Command. 2 Event Sourced Aggregate • Current state of the Aggregate is always derived from the Event Store. Eg. Shopping Cart, Order etc. This will be part of the Rich Domain Model (Bounded Context) of the Micro Service. 4 Projections • Projections focuses on the View perspective of the Application. As the Read & Write are different Models, you can have different Projections based on your View perspective. 5 Write Data Read Data Events • Commands generates the Events to be stored in Event Store. Eg. Item Added Event (in the Shopping Cart). 3 Event Storming – Concept 73
@arafkarsh arafkarsh Case Study: Shopping Site – Event Sourcing / CQRS 75 Catalogue Shopping Cart Order Payment • Search Products • Add Products • Update Products Commands • Add to Cart • Remove Item • Update Quantity Customer • Select Address • Select Delivery Mode • Process Order Events • Product Added • Product Updated • Product Discontinued • Item Added • Item Removed / Discontinued • Item Updated • Order Initiated • Address Selected • Delivery Mode Selected • Order Created • Confirm Order for Payment • Proceed for Payment • Cancel Order • Payment Initiated • Order Cancelled • Order Confirmed • OTP Send • Payment Approved • Payment Declined Microservices • Customer • Shopping Cart • Order Customer Journey thru Shopping Process 2 Processes 1 Customers will browse through the Product catalogue to find the products, its ratings and reviews. Once the product is narrowed down the customer will add the product to shopping cart. Once the customer is ready for the purchase, he/she will start the order processing by selecting the Delivery address, delivery method, payment option. Once the payment is done customer will get the order tracking details. ES Aggregate 4 Core Domain Sub Domain Sub Domain Sub Domain Generic Domain 3
@arafkarsh arafkarsh DDD: Use Case 76 Order Service Models Value Object • Currency • Item Value • Order Status • Payment Type • Record State • Audit Log Entity • Order (Aggregate Root) • Order Item • Shipping Address • Payment DTO • Order • Order Item • Shipping Address • Payment Domain Layer Adapters • Order Repository • Order Service • Order Web Service • Order Query Web Service • Shipping Address Web Service • Payment Web Service Adapters Consists of Actual Implementation of the Ports like Database Access, Web Services API etc. Converters are used to convert an Enum value to a proper Integer value in the Database. For Example Order Status Complete is mapped to integer value 100 in the database. Services / Ports • Order Repository • Order Service • Order Web Service Utils • Order Factory • Order Status Converter • Record State Converter • Order Query Web Service • Shipping Address Web Service • Payment Web Service Shopping Portal
@arafkarsh arafkarsh Shopping Portal Design based on Hexagonal Architecture 77 Monolithic App Design using DDD Domain Driven Design helps you to migrate your monolithic App to Microservices based Apps
@arafkarsh arafkarsh Case Study: Movie Streaming – Event Sourcing / CQRS 79 Subscription Payment • Search Movies • Add Movies • Update Movies Commands • Request Streaming • Start Movie Streaming • Pause Movie Streaming • Validate Streaming License • Validate Download License Events • Movie Added • Movie Updated • Movie Discontinued • Streaming Requested • Streaming Started • Streaming Paused • Streaming Done • Streaming Request Accepted • Streaming Request Denied • Subscribe Monthly • Subscribe Annually • Monthly Subscription Added • Yearly Subscription Added • Payment Approved • Payment Declined Discovery Microservices Customer will search for specific movie or pick up a new episode from a TV Series from the watch list. Once the streaming request is authorized by the license service, video streaming will start. Customer can pause, fast forward and restart the movie streaming. Movie streaming will be based on Customer subscription to the service. • Stream List • Favorite List Customer Journey thru Streaming Movie / TV Show The purpose of this example is to demonstrate the concept of ES / CQRS thru Event Storming principles. License Streaming Processes 1 2 ES Aggregate 4 Core Domain Sub Domain Sub Domain Sub Domain Generic Domain 3
@arafkarsh arafkarsh DDD: Use Case 80 Subscription Service Models Value Object • Currency • Subscription Value • Subscription Type • Subscription Status • Payment Type • Record State • Audit Log Entity • Subscription (Aggregate Root) • Customer • Payment DTO • Subscription • Payment Domain Layer Adapters • Order Repository • Order Service • Order Web Service • Order Query Web Service • Payment Web Service Adapters Consists of Actual Implementation of the Ports like Database Access, Web Services API etc. Converters are used to convert an Enum value to a proper Integer value in the Database. For Example Order Status Complete is mapped to integer value 100 in the database. Services / Ports • Order Repository • Order Service • Order Web Service Utils • Order Factory • Order Status Converter • Record State Converter • Order Query Web Service • Streaming Web Service • Payment Web Service Movie Streaming
@arafkarsh arafkarsh Case Study: Patient Diagnosis and Treatment 81 Payment • Register Patient • Search Doctor Commands • Add Patient Info • Add Details • Add BP • Add Diagnosis • Add Prescription Events • Doctor Scheduled • Patient Added • Patient Info Added • Details Added • BP Added • Diagnosis Added • Prescription Added • Add Medicine • Add Bill • Medicine Added • Bill Prepared • Payment Approved • Payment Declined • Cash Paid Patient registers and takes an appointment with the doctor. Patient details and history is recorded. Doctor does the diagnosis and creates the prescription. Patient buys the medicine from the Pharmacy. If patient needs to be admitted, then ward appointment is scheduled and admitted to the ward. Once the treatment is over patient is discharged from the Hospital. Microservices • Diagnosis • Prescription • Hospital Bill • Discharge Summary Patient Journey thru Treatment Process Registration • Add Doctor • Add Appointment • Add Patient File • Doctor Added • Appointment Added • Patient File Added ES Aggregate 2 4 Processes 1 Doctors Diagnosis Pharmacy Ward Patient • Add Checkup • Add Treatment • Add Food • Add Discharge • Checkup Added • Treatment Added • Food Added • Discharge Added Core Domain Sub Domain Sub Domain Sub Domain Sub Domain Generic Domain Sub Domain 3
@arafkarsh arafkarsh Case Study: Movie Booking – Event Sourcing / CQRS 82 Order Payment • Search Movies • Add Movies • Update Movies Commands • Select Movie • Select Theatre / Show • Select Seats • Process Order • Select Food • Food Removed • Skip Food • Process Order Events • Movie Added • Movie Updated • Movie Discontinued • Movie Added • Theatre / Show Added • Seats Added • Order Initiated • Popcorn Added • Drinks Added • Popcorn Removed • Order Finalized • Proceed for Payment • Confirm Order for Payment • Cancel Order • Payment Initiated • Order Cancelled • Order Confirmed • OTP Send • Payment Approved • Payment Declined Movies Theatres Food Microservices Customer's will Search for the Movies after selecting the City. Once the movie is selected then they will identify a theatre and check for the show Times and then select the seats. Once the seats are selected then a choice is given to add Snacks after that the Customer will proceed to payments. Once the payment is done then the tickets are confirmed. • Theatre • Show • Order Customer Journey thru booking Movie Ticket The purpose of this example is to demonstrate the concept of ES / CQRS thru Event Storming principles. Processes 1 2 ES Aggregate 4 Core Domain Sub Domain Sub Domain Sub Domain Generic Domain 3
@arafkarsh arafkarsh Case Study: Restaurant Dining – Event Sourcing and CQRS 83 Order Payment • Add Drinks • Add Food • Update Food Commands • Open Table • Add Juice • Add Soda • Add Appetizer 1 • Add Appetizer 2 • Serve Drinks • Prepare Food • Serve Food Events • Drinks Added • Food Added • Food Updated • Food Discontinued • Table Opened • Juice Added • Soda Added • Appetizer 1 Added • Appetizer 2 Added • Juice Served • Soda Served • Appetizer Served • Food Prepared • Food Served • Prepare Bill • Process Payment • Bill Prepared • Payment Processed • Payment Approved • Payment Declined • Cash Paid When people arrive at the Restaurant and take a table, a Table is opened. They may then order drinks and food. Drinks are served immediately by the table staff; however, food must be cooked by a chef. Once the chef prepared the food it can then be served. The Bill is prepared when the Table is closed. Microservices • Dinning Order • Billable Order Customer Journey thru Dinning Processes Food Menu Kitchen Dining • Remove Soda • Add Food 1 • Add Food 2 • Place Order • Close Table • Remove Soda • Food 1 Added • Food 2 Added • Order Placed • Table Closed ES Aggregate 2 4 Processes 1 Core Domain Sub Domain Sub Domain Sub Domain Generic Domain 3
@arafkarsh arafkarsh Summary: User Journey / CCD / DDD / Event Sourcing & CQRS 84 User Journey Bounded Context 1 Bounded Context 2 Bounded Context 3 1. Bounded Contexts 2. Entity 3. Value Objects 4. Aggregate Roots 5. Domain Events 6. Repository 7. Service 8. Factory Process 1 Commands 2 Projections 5 ES Aggregate 4 Events 3 Event Sourcing & CQRS Domain Expert Analyst Architect QA Design Docs Test Cases Code Developers Domain Driven Design Ubiquitous Language Core Domain Sub Domain Generic Domain Vertically sliced Product Team FE BE DB Business Capability 1 QA Team PO FE BE DB Business Capability 2 QA Team PO FE BE DB Business Capability n QA Team PO
@arafkarsh arafkarsh 85 100s Microservices 1,000s Releases / Day 10,000s Virtual Machines 100K+ User actions / Second 81 M Customers Globally 1 B Time series Metrics 10 B Hours of video streaming every quarter Source: NetFlix: : https://www.youtube.com/watch?v=UTKIT6STSVM 10s OPs Engineers 0 NOC 0 Data Centers So what do NetFlix think about DevOps? No DevOps Don’t do lot of Process / Procedures Freedom for Developers & be Accountable Trust people you Hire No Controls / Silos / Walls / Fences Ownership – You Build it, You Run it.
@arafkarsh arafkarsh References 90 1. July 15, 2015 – Agile is Dead : GoTo 2015 By Dave Thomas 2. Apr 7, 2016 - Agile Project Management with Kanban | Eric Brechner | Talks at Google 3. Sep 27, 2017 - Scrum vs Kanban - Two Agile Teams Go Head-to-Head 4. Feb 17, 2019 - Lean vs Agile vs Design Thinking 5. Dec 17, 2020 - Scrum vs Kanban | Differences & Similarities Between Scrum & Kanban 6. Feb 24, 2021 - Agile Methodology Tutorial for Beginners | Jira Tutorial | Agile Methodology Explained. Agile Methodologies
@arafkarsh arafkarsh References 92 Microservices 1. Microservices Definition by Martin Fowler 2. When to use Microservices By Martin Fowler 3. GoTo: Sep 3, 2020: When to use Microservices By Martin Fowler 4. GoTo: Feb 26, 2020: Monolith Decomposition Pattern 5. Thought Works: Microservices in a Nutshell 6. Microservices Prerequisites 7. What do you mean by Event Driven? 8. Understanding Event Driven Design Patterns for Microservices
@arafkarsh arafkarsh References – Microservices – Videos 93 1. Martin Fowler – Micro Services : https://www.youtube.com/watch?v=2yko4TbC8cI&feature=youtu.be&t=15m53s 2. GOTO 2016 – Microservices at NetFlix Scale: Principles, Tradeoffs & Lessons Learned. By R Meshenberg 3. Mastering Chaos – A NetFlix Guide to Microservices. By Josh Evans 4. GOTO 2015 – Challenges Implementing Micro Services By Fred George 5. GOTO 2016 – From Monolith to Microservices at Zalando. By Rodrigue Scaefer 6. GOTO 2015 – Microservices @ Spotify. By Kevin Goldsmith 7. Modelling Microservices @ Spotify : https://www.youtube.com/watch?v=7XDA044tl8k 8. GOTO 2015 – DDD & Microservices: At last, Some Boundaries By Eric Evans 9. GOTO 2016 – What I wish I had known before Scaling Uber to 1000 Services. By Matt Ranney 10. DDD Europe – Tackling Complexity in the Heart of Software By Eric Evans, April 11, 2016 11. AWS re:Invent 2016 – From Monolithic to Microservices: Evolving Architecture Patterns. By Emerson L, Gilt D. Chiles 12. AWS 2017 – An overview of designing Microservices based Applications on AWS. By Peter Dalbhanjan 13. GOTO Jun, 2017 – Effective Microservices in a Data Centric World. By Randy Shoup. 14. GOTO July, 2017 – The Seven (more) Deadly Sins of Microservices. By Daniel Bryant 15. Sept, 2017 – Airbnb, From Monolith to Microservices: How to scale your Architecture. By Melanie Cubula 16. GOTO Sept, 2017 – Rethinking Microservices with Stateful Streams. By Ben Stopford. 17. GOTO 2017 – Microservices without Servers. By Glynn Bird.
@arafkarsh arafkarsh References 94 Domain Driven Design 1. Oct 27, 2012 What I have learned about DDD Since the book. By Eric Evans 2. Mar 19, 2013 Domain Driven Design By Eric Evans 3. Jun 02, 2015 Applied DDD in Java EE 7 and Open Source World 4. Aug 23, 2016 Domain Driven Design the Good Parts By Jimmy Bogard 5. Sep 22, 2016 GOTO 2015 – DDD & REST Domain Driven API’s for the Web. By Oliver Gierke 6. Jan 24, 2017 Spring Developer – Developing Micro Services with Aggregates. By Chris Richardson 7. May 17. 2017 DEVOXX – The Art of Discovering Bounded Contexts. By Nick Tune 8. Dec 21, 2019 What is DDD - Eric Evans - DDD Europe 2019. By Eric Evans 9. Oct 2, 2020 - Bounded Contexts - Eric Evans - DDD Europe 2020. By. Eric Evans 10. Oct 2, 2020 - DDD By Example - Paul Rayner - DDD Europe 2020. By Paul Rayner
@arafkarsh arafkarsh References 95 Event Sourcing and CQRS 1. IBM: Event Driven Architecture – Mar 21, 2021 2. Martin Fowler: Event Driven Architecture – GOTO 2017 3. Greg Young: A Decade of DDD, Event Sourcing & CQRS – April 11, 2016 4. Nov 13, 2014 GOTO 2014 – Event Sourcing. By Greg Young 5. Mar 22, 2016 Building Micro Services with Event Sourcing and CQRS 6. Apr 15, 2016 YOW! Nights – Event Sourcing. By Martin Fowler 7. May 08, 2017 When Micro Services Meet Event Sourcing. By Vinicius Gomes
@arafkarsh arafkarsh References 97 Databases: Big Data / Cloud Databases 1. Google: How to Choose the right database? 2. AWS: Choosing the right Database 3. IBM: NoSQL Vs. SQL 4. A Guide to NoSQL Databases 5. How does NoSQL Databases Work? 6. What is Better? SQL or NoSQL? 7. What is DBaaS? 8. NoSQL Concepts 9. Key Value Databases 10. Document Databases 11. Jun 29, 2012 – Google I/O 2012 - SQL vs NoSQL: Battle of the Backends 12. Feb 19, 2013 - Introduction to NoSQL • Martin Fowler • GOTO 2012 13. Jul 25, 2018 - SQL vs NoSQL or MySQL vs MongoDB 14. Oct 30, 2020 - Column vs Row Oriented Databases Explained 15. Dec 9, 2020 - How do NoSQL databases work? Simply Explained! 1. Graph Databases 2. Column Databases 3. Row Vs. Column Oriented Databases 4. Database Indexing Explained 5. MongoDB Indexing 6. AWS: DynamoDB Global Indexing 7. AWS: DynamoDB Local Indexing 8. Google Cloud Spanner 9. AWS: DynamoDB Design Patterns 10. Cloud Provider Database Comparisons 11. CockroachDB: When to use a Cloud DB?
@arafkarsh arafkarsh References 99 1. Feb 6, 2020 – An introduction to TDD 2. Aug 14, 2019 – Component Software Testing 3. May 30, 2020 – What is Component Testing? 4. Apr 23, 2013 – Component Test By Martin Fowler 5. Jan 12, 2011 – Contract Testing By Martin Fowler 6. Jan 16, 2018 – Integration Testing By Martin Fowler 7. Testing Strategies in Microservices Architecture 8. Practical Test Pyramid By Ham Vocke Testing – TDD / BDD
@arafkarsh arafkarsh 100 1. Simoorg : LinkedIn’s own failure inducer framework. It was designed to be easy to extend and most of the important components are plug‐ gable. 2. Pumba : A chaos testing and network emulation tool for Docker. 3. Chaos Lemur : Self-hostable application to randomly destroy virtual machines in a BOSH- managed environment, as an aid to resilience testing of high-availability systems. 4. Chaos Lambda : Randomly terminate AWS ASG instances during business hours. 5. Blockade : Docker-based utility for testing network failures and partitions in distributed applications. 6. Chaos-http-proxy : Introduces failures into HTTP requests via a proxy server. 7. Monkey-ops : Monkey-Ops is a simple service implemented in Go, which is deployed into an OpenShift V3.X and generates some chaos within it. Monkey-Ops seeks some OpenShift components like Pods or Deployment Configs and randomly terminates them. 8. Chaos Dingo : Chaos Dingo currently supports performing operations on Azure VMs and VMSS deployed to an Azure Resource Manager-based resource group. 9. Tugbot : Testing in Production (TiP) framework for Docker. Testing tools
@arafkarsh arafkarsh References 101 CI / CD 1. What is Continuous Integration? 2. What is Continuous Delivery? 3. CI / CD Pipeline 4. What is CI / CD Pipeline? 5. CI / CD Explained 6. CI / CD Pipeline using Java Example Part 1 7. CI / CD Pipeline using Ansible Part 2 8. Declarative Pipeline vs Scripted Pipeline 9. Complete Jenkins Pipeline Tutorial 10. Common Pipeline Mistakes 11. CI / CD for a Docker Application
@arafkarsh arafkarsh References 103 1. Lewis, James, and Martin Fowler. “Microservices: A Definition of This New Architectural Term”, March 25, 2014. 2. Miller, Matt. “Innovate or Die: The Rise of Microservices”. e Wall Street Journal, October 5, 2015. 3. Newman, Sam. Building Microservices. O’Reilly Media, 2015. 4. Alagarasan, Vijay. “Seven Microservices Anti-patterns”, August 24, 2015. 5. Cockcroft, Adrian. “State of the Art in Microservices”, December 4, 2014. 6. Fowler, Martin. “Microservice Prerequisites”, August 28, 2014. 7. Fowler, Martin. “Microservice Tradeoffs”, July 1, 2015. 8. Humble, Jez. “Four Principles of Low-Risk Software Release”, February 16, 2012. 9. Zuul Edge Server, Ketan Gote, May 22, 2017 10. Ribbon, Hysterix using Spring Feign, Ketan Gote, May 22, 2017 11. Eureka Server with Spring Cloud, Ketan Gote, May 22, 2017 12. Apache Kafka, A Distributed Streaming Platform, Ketan Gote, May 20, 2017 13. Functional Reactive Programming, Araf Karsh Hamid, August 7, 2016 14. Enterprise Software Architectures, Araf Karsh Hamid, July 30, 2016 15. Docker and Linux Containers, Araf Karsh Hamid, April 28, 2015
arafkarsh
tbshiki
shimap_sampo
yuya4
mii3king
ymty
matsujirushi
sasakitomohiro
tacck
hayaosuzuki
tarappo
bun913
satoshirobatofujimoto
lynnandtonic
michaelherold
jonyablonski
zakiyama
rmw
dougneiner
lauravandoore
scottboms
sferik
cassininazir
reverentgeek
aki_iinuma