Architecting your applications for millions of users.

Cloud computing gives you a number of advantages, such as the ability to scale your web application or website on demand. If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from one to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.

Presented at the AWS Summit in Paris.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Adrian Hornsby Cloud Architecture Evangelist – Amazon Web Services Architecting your Applications for Millions of Users @adhorn Romain Cornu SRE / Devops engineer - Ouicar @ romain_cornu #awssummit
3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. FOCUS ON BUSINESS LOGIC LEVEL OF ABSTRACTION Physical Machines • Requires “guess” planning • Lives for years on-prem • Heavy investments (CAPEX) • Low innovation factor • Deploy in months Computing evolution
4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. FOCUS ON BUSINESS LOGIC LEVEL OF ABSTRACTION Physical Machines Computing evolution • Hardware independence • Faster provisioning speed (minutes/hours) • Trade CAPEX for OPEX • More scale • Elastic resources • Faster speed and agility • Reduced maintenance Virtual Machines Virtual Servers in Datacenters
5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. FOCUS ON BUSINESS LOGIC LEVEL OF ABSTRACTION Physical Servers in Datacenters Physical Machines Computing evolution Virtual Servers in Datacenters Virtual Machines Virtual Servers in Datacenters Containerization • Platform independence • Consistent Runtime Environment • Higher resource utilization • Easier and faster deployments • Isolation and sandboxing • Start speed (deploy in seconds)
6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. FOCUS ON BUSINESS LOGIC LEVEL OF ABSTRACTION Physical Servers in Datacenters Physical Machines Computing evolution Virtual Servers in Datacenters Virtual Machines Virtual Servers in Datacenters Containerization Serverless • Continuous scaling • Fault tolerance built-in • Event-driven • Pay per usage • Zero maintenance
7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Happier Customers Better Products More Innovations Faster Release Cycles Decouple Software Systems (micro-services) Focus on Business Logic We are witnessing a paradigm shift
8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The micro-services architecture
9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. This is how architecting something feels like…
10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. From Monolith To Microservices Romain Cornu SRE / Devops engineer - Ouicar @ romain_cornu
11. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. about me Romain CORNU Paris, France rcornu romain_cornu (passive reader, do not follow me) romain-cornu (you might want to get in touch, someday)
12. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. ouicar Pioneer in collaborative consumption, Ouicar is an internet platform that links car owners with their future tenants since 2012. more than 1 000 000 users, 30 000 cars registered, and thousands rentals each month
13. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. ouicar One responsive website and one application on each mobile plateforme. Three feature teams and more than 20 people (tech, quality, data, …).
14. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. migration
15. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. initial state Application (MVC) handle both presentation and business layers. Business logic is organized in several modules in order to be then split into several services. Database is placed on dedicated servers with read replicas.
16. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. intermediary state Presentation is now a frontend service, a stand alone React application wrapped into a Docker container stack. Zend legacy application exposes business logic through an API, in order to be used by the frontend services.
17. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. final state Our API is now splitted in several services written in Symfony, NodeJS or Python, wrapped into a Docker container stacks. Services mainly talk to each other through asynchronus events. Each service has and manages its own database. Each service can now be updated independently or scaled regarding load.
18. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. infrastructure
19. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. frontend We take advantage of S3 static website hosting to forward trafic from alternative domains to ouicar.fr without writing a single line of code. Cloudfront and WAF helps us to optimize caching and performances without adding complexity to our application.
20. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. gateway Even if there is no cache on our API, we use a Cloudfront instance to protect ourself from several attack types. Cloudfront also improves our network performances.
21. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. microservices Compute layer services, such as Ouicar microservices or Lambda are stateless. Every data layer services, such as databases, elasticsearch or redis, is a AWS managed service.
22. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. benefits
23. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. managed data services From RDS to Elasticache - Redis, including Elasticsearch and SQS, every statefull component is managed by AWS. Helping us to save time by providing reliable services based on a solid configuration with a great setup / startup time.
24. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. two dimensions scaling Horizontal scaling, at infrastructure level, adds or removes EC2 instances. Vertical scaling, at application level, adds or removes ECS service instances (tasks). Two dimensions scaling helps us to quickly respond to any load.
25. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. blue green deployment ECS always keeps a healthy version of each service. So when a new version is pushed, ECS waits for the new version instances to be healthy before shutting down previous ones. Using this deployment strategy we no longer have any downtime.
26. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. logs ECS service can send logs to Cloudwatch using awslogs connector. Then using a Lambda function, logs are tagged and sent to Logmatic API. This allow us to easily analyse logs on an environment, service, container, …
27. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. next steps
28. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. next steps X-Ray is of course a part of our technical roadmap in order to easily track where latency is coming from. Rekognition is also something that we want to dig in, image moderation and text in image could help us to analyse photo and IDs.
29. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. merci ! Romain Cornu SRE / Devops engineer - Ouicar @ romain_cornu
30. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The micro-services architecture
31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Asynchronous patterns
32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. CAP Theorem Consistency Availability Partition Tolerance Data is consistent. All nodes see the same state. Every request is non-failing. Service still responds as expected if some nodes crash. Distributed System In the presence of a network partition, you must choose between consistency and availability!
33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Eventual Consistency … if no new updates are made to a given data item, eventually all accesses to that item will return the last updated value. Availability An eventually consistent system can return any value before it converges!! https://en.wikipedia.org/wiki/Eventual_consistency Distributed System Every request is non-failing.
34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Process A Process B Process A Process B Synchronous Asynchronous Waiting Working Continues get or fetch result Get result
35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Non-blocking UI https://medium.com/@sophie_paxtonUX/stop-getting-in-my-way-non-blocking-ux-5cbbfe0f0158
36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Message passing for async. patterns A Queue B A Queue B Listener Pub-Sub
37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Web Instances Worker Instance Worker Instance Queue API Instance API Instance API Instance API: {DO foo} PUT JOB: {JobID: 0001, Task: DO foo} API: {JobID: 0001} GET JOB: {JobID: 0001, Task: DO foo} Cache Result: { JobID: 0001, Result: bar }
38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Worker Instance Worker Instance Queue API Instance API Instance API Instance Cache Amazon SNS Push Notification User
39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. RDS DB Instance Active (Multi-AZ) Availability Zone Elastic Load Balancer Web Instance Web Instance Amazon Route 53 User Amazon S3 Amazon Cloudfront ElastiCache Worker Instance Worker Instance Full Decoupling Queue Amazon SNS
40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Databases
41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Read / Write Sharding RDS DB Instance Read Replica App Instance App Instance App Instance RDS DB Instance Master (Multi-AZ) RDS DB Instance Read Replica RDS DB Instance Read Replica
42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Database Federation Users DB Products DB App Instance App Instance App Instance
43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Database Sharding User ShardID 002345 A 002346 B 002347 C 002348 B 002349 A C B A App Instance App Instance App Instance
44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cross-Region Read Replicas for Amazon RDS ** For Aurora, MySQL, MariaDB and PostgreSQL engines.
45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cross-Region Usage Pattern • Regional Reads • All critical writes traffic to a single master
46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. James Hamilton – 2016 re:Invent Vice President & Distinguished Engineer
47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aurora multi-master - scale out reads & writes First MySQL compatible DB service with scale-out across multiple data centers Zero application downtime from ANY instance failure Zero application downtime from ANY AZ failure Faster write performance and higher scale Sign up for single-region multi-master preview today; Multi-Region Multi-Master coming in 2018 Availability Zone 1 Scale out both reads and writes Availability Zone 2 Availability Zone 3 Application Read/Write Master 1 Shared distributed storage volume Read/Write Master 2 Read/Write Master 3
48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon DynamoDB Global Tables (GA) First fully managed, multi-master, multi-region database Build high performance, globally distributed applications Low latency reads & writes to locally available tables Disaster proof with multi-region redundancy Easy to set up and no application rewrites required Globally dispersed users Replica (N. America) Replica (Europe) Replica (Asia) Global App Global Table
49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Transient state does not belong in the database BUT is perfect in the cache!
50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Increasing Resiliency
51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Circuit Breaker • Wrap a protected function call in a circuit breaker object, which monitors for failures. • If failures reach a certain threshold, the circuit breaker trips. https://martinfowler.com/bliki/CircuitBreaker.html
52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Degradation & Fallbacks
53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Monitoring & Steady State
54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Monitoring, metrics, and logging • What are customers saying? • Try to squeeze as much performance out of each service/component Make it ridiculously easy to start monitoring a new metric! 1 line of code rule
55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Steady State • ”normal” behavior of your system https://www.elastic.co/blog/timelion-tutorial-from-zero-to-hero
56. Steady State • ”normal” behavior of your system • Business Metric https://medium.com/netflix-techblog/sps-the-pulse-of-netflix-streaming-ae4db0e05f8a
57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. New ways of testing
58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fire Drills Game Days
59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Jesse Robbins GameDay: Creating Resiliency Through Destruction https://www.youtube.com/watch?v=zoz0ZjfrQ9s
60. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Testing to the next level: Failure Injection • Start small & build confidence • Application level • Host failure • Resource attacks (CPU, memory, …) • Network attacks (dependencies, latency, …) • Region attacks!
61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Chaos Engineering
63. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.