Building Microservices Architecture on AWS

Microservices architectures make applications easier to scale and faster to develop, enabling innovation and accelerating time-to-market for new features. Microservices architectures is a service-oriented architecture that provides loose coupling, high cohesion and bounded contexts unlike the monolithic architectures. In this talk, we will focus on the AWS implementations of the microservices architecture in the sense of the communication model, continuous integration, deployment and monitoring.

Transcript

1. Building Microservices Architecture on AWS 23.06.2018 | Kadir Türker Gülsoy https://www.linkedin.com/in/kadir-turker-gulsoy
2. Outline • Introduction to Microservices • Communication Model • Data Flow and Messaging via AWS • Development, Continuous Integration and Deployment • Monitoring • Conclusion
3. Introduction to Microservices
4. What is Monolithic Application? Wikipedia: A monolithic application describes a single-tiered software application in which the user interface and data access code are combined into a single program from a single platform. A monolithic application is self-contained, and independent from other computing applications
5. What is Monolithic Application? All of the required logic is located within one unit: A war, a jar, a single application, single repository
7. What is Microservices? Wikipedia: Microservices is a software development technique—a variant of the service-oriented architecture (SOA) architectural style that structures an application as a collection of loosely coupled services. In a microservices architecture, services are fine-grained and the protocols are lightweight
10. Microservices Principles • Service-Oriented Architecture Services communicate with each other over the network • Loosely Coupled Elements You can update the services independently; updating one service doesn’t require changing any other services
11. Microservices Principles • Bounded contexts Self contained; you can update the code without knowing anything about the internals of other services • High cohesion Keep related behavior to sit together, and unrelated behavior to sit elsewhere
14. Challenges with Monolithic Software • Difficult to Scale • Builds, Tests, Releases • Reliability • Operational Difficulty • Architectural Maintenance and Evolvement • Lack of Agility • Lack of Innovation
15. Challenges: OpsGenie Use Case • Code complexity ◦ High learning curve ◦ Easiness to introduce bugs, performance degradation , single point of failures • Delivery speed ◦ More than 30 developers on the same code-base ◦ Complex deployments blocks the entire company • Monitoring & Reliability
16. Communication Model
17. Communication Model • Different patterns in terms of: ◦ Communication ◦ Discovery ◦ Security ◦ Testing ◦ Observability ◦ Deployment ◦ Decomposition
20. Two Collaboration Styles • Request / Response • Event-Driven
21. Request / Response Synchronous: • Client initiates a request, and waits for response Asynchronous: • Client initiates a request, and registers a callback • Short/Long polling the result
22. Event-Driven • A service publishes a completed operation and expects other parties to know what to do • Asynchronous by nature • Business logic is not centralized • Highly-decoupled • You can add new subscribers any time without a need to know them
23. Orchestration vs Choreography Orchestration: • Rely on a central brain to guide and drive the process Choreography: • Services observe the jobs and act on the events
24. Orchestration vs Choreography Example scenario (Creating a customer): 1. A new record is created in the loyalty points 2. Postal system sends out a welcome pack 3. We send a welcome email to the customer
25. Orchestration
26. Choreography
27. API Gateway Pattern
28. API Gateway Pattern • Centralized Middleware for Disparate Services ◦ Authentication ◦ Security ◦ Traffic Control ◦ Monitoring & Logging ◦ Operations ◦ Transformations • Different Granularity of Services
29. API Gateway Pattern • Blue/Green Deployments
30. API Gateway Pattern • Canary Releases
31. API Gateway Pattern • Load Balancing
32. API Gateway Pattern • Circuit Breaking
33. Data Flow & Messaging via AWS
34. AWS Elastic Compute Cloud (EC2) • Secure, resizable compute capacity in the cloud • Familiar operating systems with cloud benefits • Different instance families & types • Completely controlled instances
35. AWS Elastic Load Balancing (ELB) • Distributes incoming traffic across multiple applications, microservices and containers • Three types: ◦ Application Load Balancer ◦ Network Load Balancer ◦ Classic Load Balancer
36. AWS Elastic Load Balancing (ELB) • Application Load Balancer ◦ Advanced request routing • Network Load Balancer ◦ TCP level load balancing • Classic Load Balancer ◦ Basic load balancing
37. AWS Elastic Load Balancing (ELB) • Highly available via multiple availability zones • Session management • Health checking • Security features ◦ Not-internet-facing load balancers • Request Logging • Operational Monitoring
38. AWS Elastic Load Balancing
39. API Gateway: OpsGenie Use Case • Public requests are delivered to API Gateway ELB • All ELBs are not-internet-facing except API Gateway ELB • Requests from public network are proxied to the related ELB according to the request domain
40. API Gateway: OpsGenie Use Case
41. API Gateway: OpsGenie Use Case • API Gateway apps authenticate, authorize and rate-limit the request • Requests from public network are proxied to related Service ELB according to the request domain. • Authentication information is carried over HTTP headers
42. API Gateway: OpsGenie Use Case • Response of the proxied service is transformed and returned to the client • Circuit breaking: Limit requests to the unhealthy service
43. REST APIs: OpsGenie Use Case • Custom DNS records for the ELBs of each service • HTTP requests are forwarded to the ELB of the desired service • Rest applications process the requests in a non-blocking manner: I/O thread vs. Worker Thread
44. Rest APIs: OpsGenie Use Case • Rest APIs of the services with high-volume traffic are designed to be asynchronous via Amazon SQS. • SDKs are released and maintained by each responsible team.
45. Overview: Messaging via AWS
46. AWS Simple Queue Service (SQS) • Three components: ◦ Applications ◦ Queues ◦ Messages • Two queue types: ◦ Standard queue ◦ FIFO queue
47. AWS Simple Queue Service (SQS) • Message queue service that store messages waiting to be processed • Batch and burst processing with no message loss • Ensured message delivery based on message polling • Eliminated operational overhead • Based on Push/Pull Model
48. Flashback: Two Collaboration Styles • Request / Response • Event-Driven
49. Flashback: Event-Driven • A service publishes a completed operation and expects other parties to know what to do • Asynchronous by nature • Business logic is not centralized • Highly-decoupled • You can add new subscribers any time without a need to know them • Event-driven architecture is handled via Amazon SNS
50. AWS Simple Notification Service (SNS) • Fully managed messaging service based on Publish/Subscribe model • You can add/remove subscribers any time • Supported subscribers: ◦ HTTP(S) ◦ SQS ◦ Lambda ◦ Email, SMS, Mobile Push
51. AWS Simple Notification Service (SNS)
52. Product Use Case: Some background • Integrations ◦ Software interfaces with 3rd party monitoring tools ◦ Processes the tool-specific data and creates/manages alerts ◦ Some integrations also have the capability to update the other tool for alert actions
53. Product Use Case: Creating Alerts • Requests are received via integrations with 3rd party services to create alerts • When an alert is created, on-call users are notified via E-Mail, SMS, Voice and Mobile Push notifications. • When an alert is created, outgoing integrations updates the mapped event/incident/etc. on the 3rd party tool • Access rights for integrations, alerts, etc. rely on the subscription plan of the customers
55. Cloudwatch to API Gateway
56. An API Gateway Receives Request
57. API Gateway Intercepts the Request
58. Integration API Receives & Responds Request
59. Message Sent to Integration Engine
60. Create Alert Request to Alert API
61. Alert API Sends Message to Processor App
62. Alert is Created
63. Integration Engine Receives Event
64. Integration Sender Receives Message
65. Integration Sender Receives Message
66. Additional Choreography for Creating an Alert • Notification Service ◦ Bi-Directional Relation • Incident Service ◦ Bi-Directional Relation • Reporting Service • Pricing Service
67. AWS Messaging Alternatives • Apache Kafka • RabbitMQ • Google Pub/Sub • IBM MQ • Oracle WebLogic JMS
68. Development, Continuous Integration And Deployment
69. Development Model • Development teams are separated according to the product domains or cross-cutting technical domains • Product development teams can be responsible for multiple product domains
70. Development Model • In general, each service represents a single product domain • Each team can be responsible for multiple services • Each service is maintained on a separate code repository
71. Development Model • Cross-cutting core frameworks: ◦ Tier 0 Service Adapters (AWS, Redis, ElasticSearch, etc) ◦ Spring MVC, Core, Boot Implementations ◦ Monitoring & Logging Utilities • Cross-cutting core frameworks are extracted to separate code repositories • Cross-cutting core frameworks are released and deployed to a private Nexus repository
72. Continuous Integration • Unit, integration and external tests to ensure internal functionalities • Functional tests to ensure inter-service product functionalities ◦ Common for all code repositories ◦ Implement with the point of view of customers • Each merge requests trigger entire set of tests: All of them are required to merge
73. Continuous Integration & Deployment • All services among all code repositories are kept up and running continuously for the pre-release environment • After merging a remote branch, master branch of the code repository is deployed to the dockerized pre-release environment via Jenkins tasks • All services can be tested and deployed in parallel
74. Blue Green Deployment • New EC2 instances are launched and updated with the required software • New applications are started • When boot is completed, new applications start receiving messages from queues and streams • If new applications have REST endpoints, they are attached to the ELB and start receiving HTTP traffic
75. Blue Green Deployment • New and old applications are running at the same time. Traffic distribution: Canary vs. Uniform • If everything is functional, old applications are detached from ELB and there is no HTTP traffic to them • Old applications stop receiving messages from queues and streams • Old applications are stopped • Old instances are terminated
76. Monitoring
77. AWS Cloudwatch • Cloud & Network Monitoring Service • Almost all AWS services have native integration with Cloudwatch along with service-specific metrics • Custom Metric & Log Support • Alarms -> Custom thresholds and conditions • Metrics, Statistics, Periods
78. AWS Cloudwatch
79. AWS Cloudwatch (ELB)
80. AWS Cloudwatch (EC2)
81. Infrastructure Monitoring • AWS Cloudwatch ◦ Native AWS Services ◦ Custom Cloudwatch Logs & Events • Health Monitors that run continuously ◦ Cloudwatch-Free AWS Service monitor tasks ◦ Service-specific health check monitor tasks ◦ Infrastructural functionality monitor tasks ◦ Network monitor tasks
82. Functionality Monitoring • Synthetic monitor tasks that continuously tests core product functionalities on pre-production and production • Anomaly detection
83. Transaction Monitoring • Transaction: The task that is executed by a single thread within a single application ◦ Each Rest API Request ◦ Each processed queue message ◦ Each stream record ◦ Each scheduled task ◦ Each in-app async task • In general, a REST API request or received message results in multiple transactions among multiple services
84. Transaction Monitoring • Transactions metrics are collected from both inside and outside of the application • Transaction monitor tasks continuously evaluate those metrics • Any unexpected error or case creates an alert directly from the applications along with the necessary context information • Transaction events & metrics are sent to ElasticSearch & NewRelic
87. Transaction Events • Each transaction submits a separate transaction event by default that is created and submitted by all applications among all services • Each transaction event has multiple metrics: ◦ Duration, external call count, build info, node info ◦ Metrics of the requests to all external services (AWS Services, Redis, ElasticSearch, etc) ◦ Metrics of the requests to all internal services (other microservice APIs, queues, etc) ◦ Custom metrics • Custom transaction events
88. Inter-Transaction Monitoring • Native support via X-Amzn-Trace-Id ELB header and CloudFront headers • Multi-Transaction-Stack information is built by all inter-service communication channels: ◦ Service SDKs ◦ AWS Messaging Services • Example case: DynamoDB Auto Scaling, EC2 Auto Scaling • Example case: A long-running transaction
89. Conclusion
90. Requirements of the Microservices • Backward & Forward Compatibility • Technology Agnostic APIs • Integration with your services should be easy • Hide internal implementation details
91. Challenges of the Microservices • Refactoring from monolithic throughout the separation • Continuous cross-team needs • Backward & Forward Compatibility • Monitoring
92. Thank you...