Operating Microservices at Hyperscale

Most developers today are adopting a microservices based application design. Microservices can provide higher system reliability, fine-grained scalability and faster development cycles. At hyperscale (thousands to millions of requests per second), however, additional thought, careful design, and greater operational rigor is required. In this session, learn from AWS experts who have extracted four fundamental design principles and best practices for hyperscale applications from the experiences of our customer globally. Aided by live demos, presenters will show how event driven architectures, asynchronous communication, service discover and service orchestration are the pillars of hyperscale systems.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Operating Microservices at Hyperscale Donnie Prakoso, MSc Technical Evangelist, ASEAN Amazon Web Services @donnieprakoso donnieprakoso
2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Monolith webserver .package Order UI Order service Inventory service Shipping service • Processes are tightly coupled • Run as a single service • More complexities as the code base grows • Limiting experimentation • Entire architecture must be scaled • High risk for application availability
3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T What do we need?
4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Introducing Microservices Microservices are an architectural and organizational approach to software development in which software is composed of small, independent services that communicate over well-defined APIs. Whitepaper: http://bit.ly/2A0qGdt - Running Containerized Microservices on AWS
5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. webserver .package Order UI Order service Inventory service Shipping service Order UI(1) Order UI(n) Order Service(1) Order Service(n) Inventory Service(1) Inventory Service(n) Shipping Service(1) Shipping Service(n) Monolithic Microservices
6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Characteristics of Microservices Do one thing well Independent Decentralized Black box Polyglot You build it, you run it
7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon S3 Object storage built to store and retrieve any amount of data from anywhere
8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Challenges of Microservices Organizational Structure Migration Distributed Systems Versions
9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Architectural Complexity Operational Complexity Complexities Challenges of Microservices
10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Journey to Hyperscale Microservices 1. A Container-first approach and orchestration tool 2. Automating software release process - CI/CD 3. Utilizing Asynchronous Communication and Lightweight Messaging 4. Know your resources with Service Discovery
11. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Containers & Orchestration CONTAINER IMAGE
13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Containers & Orchestration CONTAINER IMAGE Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS AZ 1 AZ 2 AZ 3
14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS Container Services Landscape Management Deployment, scheduling, scaling, and management of containerized applications Hosting Where the containers run Image Registry Container image repository Amazon Elastic Container Service Amazon Elastic Container Service for Kubernetes Amazon EC2 AWS Fargate Amazon Elastic Container Registry
15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Customers
16. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T CI/CD – Don’t Only Run It, Own it CONTAINER IMAGE Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS AZ 1 AZ 2 AZ 3
18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T MyApp CodeCommit Source Build CodeBuild Build DeployToInteg CodeDeploy Integration IntegTest End2EndTester DeployToProd CodeDeploy Production Source Build Deploy to integration stack Integration tests Deploy to production Model the release process in CodePipeline AWS CodeBuild AWS CodeCommit AWS CodeDeploy AWS CodeDeploy AWS CodePipeline
19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Approval action-based pipeline safety check Source MyAppSource CodeCommit Build MyAppBuild Build Deploy safetyCheck Approval ProductionDeploy CodeDeploy Lambda Function safetyCheck New update! SNS Topic
20. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T NASA Image and Video Library Amazon SNS Amazon SQS
22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Asynchronous Communication
23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Complexity on Interactions of Individual Microservices A D B E C H F G I
24. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS Step Functions Productivity Agility Resilience Build distributed applications using visual workflows Order Service Inventory Service Invoice Service Logistic Service Notification Service
26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS Step Functions Build distributed applications using visual workflows Breaking down the monolith with AWS Step Functions
27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Asynchronous Communication with Service Workflow
28. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon API Gateway Web Socket support for API Gateway Real-time two way communication applications backed by Lambda functions or other API Gateway integrations.
30. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Complexity of modern architectures • Wide variety of resources • Complexity grows exponentially • Multiple versions and stages coexist • Infrastructure scales dynamically • Unhealthy resources are replaced
32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Service Discovery Finding the location of a service provider myapp: {10.24.34.5:8080, 10.24.34.6:8080} mylogs: {S3bucket1, S3bucket2} How to find resources to connect to?
33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Registry for all cloud resources
34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Build the dynamic map of your cloud Define convenient names for all cloud resources Discover resources with specific attributes Ensure only healthy resources are discovered Use highly available DNS and regional API AWS Cloud Map
35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Registry for all cloud resources
36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T App Mesh with Amazon ECS and Amazon EKS to better run containerized microservices at scale Microservice A Microservice B Microservice C Microservice C Microservice C Application-level networking for all your services
37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Summary • Microservices architecture is a distributed approach designed to overcome the limitations of traditional monolithic architectures. • Understand distributed system components and how AWS can help you • More granular control also means more moving parts which will increase complexity use managed services to help you • Don’t let software pipeline holds your back – enhance your software release process by extending CI/CD • Optimize microservices by combining asynchronous & synchronous communications with lightweight messaging • Manage your cloud resources better
38. Go Build! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Donnie Prakoso Technical Evangelist, ASEAN Amazon Web Services @donnieprakoso donnieprakoso