Building Microservices Architecture on AWS

Slide 1

Slide 1 text

Building Microservices Architecture on AWS 23.06.2018 | Kadir Türker Gülsoy https://www.linkedin.com/in/kadir-turker-gulsoy

Slide 2

Slide 2 text

Outline ● Introduction to Microservices ● Communication Model ● Data Flow and Messaging via AWS ● Development, Continuous Integration and Deployment ● Monitoring ● Conclusion

Slide 3

Slide 3 text

Introduction to Microservices

Slide 4

Slide 4 text

What is Monolithic Application? Wikipedia: A monolithic application describes a single-tiered software application in which the user interface and data access code are combined into a single program from a single platform. A monolithic application is self-contained, and independent from other computing applications

Slide 5

Slide 5 text

What is Monolithic Application? All of the required logic is located within one unit: A war, a jar, a single application, single repository

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

What is Microservices? Wikipedia: Microservices is a software development technique—a variant of the service-oriented architecture (SOA) architectural style that structures an application as a collection of loosely coupled services. In a microservices architecture, services are fine-grained and the protocols are lightweight

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

Microservices Principles ● Service-Oriented Architecture Services communicate with each other over the network ● Loosely Coupled Elements You can update the services independently; updating one service doesn’t require changing any other services

Slide 11

Slide 11 text

Microservices Principles ● Bounded contexts Self contained; you can update the code without knowing anything about the internals of other services ● High cohesion Keep related behavior to sit together, and unrelated behavior to sit elsewhere

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Challenges with Monolithic Software ● Difficult to Scale ● Builds, Tests, Releases ● Reliability ● Operational Difficulty ● Architectural Maintenance and Evolvement ● Lack of Agility ● Lack of Innovation

Slide 15

Slide 15 text

Challenges: OpsGenie Use Case ● Code complexity ○ High learning curve ○ Easiness to introduce bugs, performance degradation , single point of failures ● Delivery speed ○ More than 30 developers on the same code-base ○ Complex deployments blocks the entire company ● Monitoring & Reliability

Slide 16

Slide 16 text

Communication Model

Slide 17

Slide 17 text

Communication Model ● Different patterns in terms of: ○ Communication ○ Discovery ○ Security ○ Testing ○ Observability ○ Deployment ○ Decomposition

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

Two Collaboration Styles ● Request / Response ● Event-Driven

Slide 21

Slide 21 text

Request / Response Synchronous: ● Client initiates a request, and waits for response Asynchronous: ● Client initiates a request, and registers a callback ● Short/Long polling the result

Slide 22

Slide 22 text

Event-Driven ● A service publishes a completed operation and expects other parties to know what to do ● Asynchronous by nature ● Business logic is not centralized ● Highly-decoupled ● You can add new subscribers any time without a need to know them

Slide 23

Slide 23 text

Orchestration vs Choreography Orchestration: ● Rely on a central brain to guide and drive the process Choreography: ● Services observe the jobs and act on the events

Slide 24

Slide 24 text

Orchestration vs Choreography Example scenario (Creating a customer): 1. A new record is created in the loyalty points 2. Postal system sends out a welcome pack 3. We send a welcome email to the customer

Slide 25

Slide 25 text

Orchestration

Slide 26

Slide 26 text

Choreography

Slide 27

Slide 27 text

API Gateway Pattern

Slide 28

Slide 28 text

API Gateway Pattern ● Centralized Middleware for Disparate Services ○ Authentication ○ Security ○ Traffic Control ○ Monitoring & Logging ○ Operations ○ Transformations ● Different Granularity of Services

Slide 29

Slide 29 text

API Gateway Pattern ● Blue/Green Deployments

Slide 30

Slide 30 text

API Gateway Pattern ● Canary Releases

Slide 31

Slide 31 text

API Gateway Pattern ● Load Balancing

Slide 32

Slide 32 text

API Gateway Pattern ● Circuit Breaking

Slide 33

Slide 33 text

Data Flow & Messaging via AWS

Slide 34

Slide 34 text

AWS Elastic Compute Cloud (EC2) ● Secure, resizable compute capacity in the cloud ● Familiar operating systems with cloud benefits ● Different instance families & types ● Completely controlled instances

Slide 35

Slide 35 text

AWS Elastic Load Balancing (ELB) ● Distributes incoming traffic across multiple applications, microservices and containers ● Three types: ○ Application Load Balancer ○ Network Load Balancer ○ Classic Load Balancer

Slide 36

Slide 36 text

AWS Elastic Load Balancing (ELB) ● Application Load Balancer ○ Advanced request routing ● Network Load Balancer ○ TCP level load balancing ● Classic Load Balancer ○ Basic load balancing

Slide 37

Slide 37 text

AWS Elastic Load Balancing (ELB) ● Highly available via multiple availability zones ● Session management ● Health checking ● Security features ○ Not-internet-facing load balancers ● Request Logging ● Operational Monitoring

Slide 38

Slide 38 text

AWS Elastic Load Balancing

Slide 39

Slide 39 text

API Gateway: OpsGenie Use Case ● Public requests are delivered to API Gateway ELB ● All ELBs are not-internet-facing except API Gateway ELB ● Requests from public network are proxied to the related ELB according to the request domain

Slide 40

Slide 40 text

API Gateway: OpsGenie Use Case

Slide 41

Slide 41 text

API Gateway: OpsGenie Use Case ● API Gateway apps authenticate, authorize and rate-limit the request ● Requests from public network are proxied to related Service ELB according to the request domain. ● Authentication information is carried over HTTP headers

Slide 42

Slide 42 text

API Gateway: OpsGenie Use Case ● Response of the proxied service is transformed and returned to the client ● Circuit breaking: Limit requests to the unhealthy service

Slide 43

Slide 43 text

REST APIs: OpsGenie Use Case ● Custom DNS records for the ELBs of each service ● HTTP requests are forwarded to the ELB of the desired service ● Rest applications process the requests in a non-blocking manner: I/O thread vs. Worker Thread

Slide 44

Slide 44 text

Rest APIs: OpsGenie Use Case ● Rest APIs of the services with high-volume traffic are designed to be asynchronous via Amazon SQS. ● SDKs are released and maintained by each responsible team.

Slide 45

Slide 45 text

Overview: Messaging via AWS

Slide 46

Slide 46 text

AWS Simple Queue Service (SQS) ● Three components: ○ Applications ○ Queues ○ Messages ● Two queue types: ○ Standard queue ○ FIFO queue

Slide 47

Slide 47 text

AWS Simple Queue Service (SQS) ● Message queue service that store messages waiting to be processed ● Batch and burst processing with no message loss ● Ensured message delivery based on message polling ● Eliminated operational overhead ● Based on Push/Pull Model

Slide 48

Slide 48 text

Flashback: Two Collaboration Styles ● Request / Response ● Event-Driven

Slide 49

Slide 49 text

Flashback: Event-Driven ● A service publishes a completed operation and expects other parties to know what to do ● Asynchronous by nature ● Business logic is not centralized ● Highly-decoupled ● You can add new subscribers any time without a need to know them ● Event-driven architecture is handled via Amazon SNS

Slide 50

Slide 50 text

AWS Simple Notification Service (SNS) ● Fully managed messaging service based on Publish/Subscribe model ● You can add/remove subscribers any time ● Supported subscribers: ○ HTTP(S) ○ SQS ○ Lambda ○ Email, SMS, Mobile Push

Slide 51

Slide 51 text

AWS Simple Notification Service (SNS)

Slide 52

Slide 52 text

Product Use Case: Some background ● Integrations ○ Software interfaces with 3rd party monitoring tools ○ Processes the tool-specific data and creates/manages alerts ○ Some integrations also have the capability to update the other tool for alert actions

Slide 53

Slide 53 text

Product Use Case: Creating Alerts ● Requests are received via integrations with 3rd party services to create alerts ● When an alert is created, on-call users are notified via E-Mail, SMS, Voice and Mobile Push notifications. ● When an alert is created, outgoing integrations updates the mapped event/incident/etc. on the 3rd party tool ● Access rights for integrations, alerts, etc. rely on the subscription plan of the customers

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

Cloudwatch to API Gateway

Slide 56

Slide 56 text

An API Gateway Receives Request

Slide 57

Slide 57 text

API Gateway Intercepts the Request

Slide 58

Slide 58 text

Integration API Receives & Responds Request

Slide 59

Slide 59 text

Message Sent to Integration Engine

Slide 60

Slide 60 text

Create Alert Request to Alert API

Slide 61

Slide 61 text

Alert API Sends Message to Processor App

Slide 62

Slide 62 text

Alert is Created

Slide 63

Slide 63 text

Integration Engine Receives Event

Slide 64

Slide 64 text

Integration Sender Receives Message

Slide 65

Slide 65 text

Integration Sender Receives Message

Slide 66

Slide 66 text

Additional Choreography for Creating an Alert ● Notification Service ○ Bi-Directional Relation ● Incident Service ○ Bi-Directional Relation ● Reporting Service ● Pricing Service

Slide 67

Slide 67 text

AWS Messaging Alternatives ● Apache Kafka ● RabbitMQ ● Google Pub/Sub ● IBM MQ ● Oracle WebLogic JMS

Slide 68

Slide 68 text

Development, Continuous Integration And Deployment

Slide 69

Slide 69 text

Development Model ● Development teams are separated according to the product domains or cross-cutting technical domains ● Product development teams can be responsible for multiple product domains

Slide 70

Slide 70 text

Development Model ● In general, each service represents a single product domain ● Each team can be responsible for multiple services ● Each service is maintained on a separate code repository

Slide 71

Slide 71 text

Development Model ● Cross-cutting core frameworks: ○ Tier 0 Service Adapters (AWS, Redis, ElasticSearch, etc) ○ Spring MVC, Core, Boot Implementations ○ Monitoring & Logging Utilities ● Cross-cutting core frameworks are extracted to separate code repositories ● Cross-cutting core frameworks are released and deployed to a private Nexus repository

Slide 72

Slide 72 text

Continuous Integration ● Unit, integration and external tests to ensure internal functionalities ● Functional tests to ensure inter-service product functionalities ○ Common for all code repositories ○ Implement with the point of view of customers ● Each merge requests trigger entire set of tests: All of them are required to merge

Slide 73

Slide 73 text

Continuous Integration & Deployment ● All services among all code repositories are kept up and running continuously for the pre-release environment ● After merging a remote branch, master branch of the code repository is deployed to the dockerized pre-release environment via Jenkins tasks ● All services can be tested and deployed in parallel

Slide 74

Slide 74 text

Blue Green Deployment ● New EC2 instances are launched and updated with the required software ● New applications are started ● When boot is completed, new applications start receiving messages from queues and streams ● If new applications have REST endpoints, they are attached to the ELB and start receiving HTTP traffic

Slide 75

Slide 75 text

Blue Green Deployment ● New and old applications are running at the same time. Traffic distribution: Canary vs. Uniform ● If everything is functional, old applications are detached from ELB and there is no HTTP traffic to them ● Old applications stop receiving messages from queues and streams ● Old applications are stopped ● Old instances are terminated

Slide 76

Slide 76 text

Monitoring

Slide 77

Slide 77 text

AWS Cloudwatch ● Cloud & Network Monitoring Service ● Almost all AWS services have native integration with Cloudwatch along with service-specific metrics ● Custom Metric & Log Support ● Alarms -> Custom thresholds and conditions ● Metrics, Statistics, Periods

Slide 78

Slide 78 text

AWS Cloudwatch

Slide 79

Slide 79 text

AWS Cloudwatch (ELB)

Slide 80

Slide 80 text

AWS Cloudwatch (EC2)

Slide 81

Slide 81 text

Infrastructure Monitoring ● AWS Cloudwatch ○ Native AWS Services ○ Custom Cloudwatch Logs & Events ● Health Monitors that run continuously ○ Cloudwatch-Free AWS Service monitor tasks ○ Service-specific health check monitor tasks ○ Infrastructural functionality monitor tasks ○ Network monitor tasks

Slide 82

Slide 82 text

Functionality Monitoring ● Synthetic monitor tasks that continuously tests core product functionalities on pre-production and production ● Anomaly detection

Slide 83

Slide 83 text

Transaction Monitoring ● Transaction: The task that is executed by a single thread within a single application ○ Each Rest API Request ○ Each processed queue message ○ Each stream record ○ Each scheduled task ○ Each in-app async task ● In general, a REST API request or received message results in multiple transactions among multiple services

Slide 84

Slide 84 text

Transaction Monitoring ● Transactions metrics are collected from both inside and outside of the application ● Transaction monitor tasks continuously evaluate those metrics ● Any unexpected error or case creates an alert directly from the applications along with the necessary context information ● Transaction events & metrics are sent to ElasticSearch & NewRelic

Slide 85

Slide 85 text

No content

Slide 86

Slide 86 text

No content

Slide 87

Slide 87 text

Transaction Events ● Each transaction submits a separate transaction event by default that is created and submitted by all applications among all services ● Each transaction event has multiple metrics: ○ Duration, external call count, build info, node info ○ Metrics of the requests to all external services (AWS Services, Redis, ElasticSearch, etc) ○ Metrics of the requests to all internal services (other microservice APIs, queues, etc) ○ Custom metrics ● Custom transaction events

Slide 88

Slide 88 text

Inter-Transaction Monitoring ● Native support via X-Amzn-Trace-Id ELB header and CloudFront headers ● Multi-Transaction-Stack information is built by all inter-service communication channels: ○ Service SDKs ○ AWS Messaging Services ● Example case: DynamoDB Auto Scaling, EC2 Auto Scaling ● Example case: A long-running transaction

Slide 89

Slide 89 text

Conclusion

Slide 90

Slide 90 text

Requirements of the Microservices ● Backward & Forward Compatibility ● Technology Agnostic APIs ● Integration with your services should be easy ● Hide internal implementation details

Slide 91

Slide 91 text

Challenges of the Microservices ● Refactoring from monolithic throughout the separation ● Continuous cross-team needs ● Backward & Forward Compatibility ● Monitoring

Slide 92

Slide 92 text

Thank you...