Scaling Up to Your First 10 Million Users

Re-delivered from re:Invent 2019 session Scaling Up to Your First 10 Million Users. This session I shared about how do you start your application using AWS, how to combine AWS services, best practices in choosing the right architecture.

Original AWS re:Invent Presentation can be watched here.
https://www.youtube.com/watch?v=kKjm4ehYiMs

Transcript

1. © 2020, Amazon Web Services, Inc. or its Affiliates. Scaling Up to Your First 10 Million Users Petra Novandi Barus Senior Developer Advocate, Indonesia
2. © 2020, Amazon Web Services, Inc. or its Affiliates.
3. © 2020, Amazon Web Services, Inc. or its Affiliates. Scaling application?
4. © 2020, Amazon Web Services, Inc. or its Affiliates. It’snot the single thing that fixes everything
5. What do we need first? © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
6. Amazon global edgenetwork © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 24 region 76 availability zones 210+ points of presence
7. Robust, fully featured technology infrastructure © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
8. AWS building blocks Amazon CloudFront Amazon Route 53 Amazon S3 Amazon DynamoDB AWS Lambda Amazon SQS Amazon SNS Amazon SES Elastic Load Balancing AWS Step Functions Amazon EFS . . . Highly scalable, available with the right architecture Amazon EC2 Amazon EBS Amazon RDS Amazon VPC © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
9. Considerations? © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
10. “Many decisions are reversible, two-way doors.” Jeff Bezos Founder & CEO Amazon
11. Build Measure Learn © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
12. Identify and avoid undifferentiated heavy lifting Guiding Tenets © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Serverless vs managed vs run it yourself Security first
13. So let’s start fromday . . © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
14. You User © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
15. One user VPC User Elastic IP address © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Instance
16. • Choose multiple plans that include bundled compute, storage, and networking • Benefit from low and predictable price • Spin up fully configured server in seconds • Manage from intuitive Lightsail console • Scale with access to AWS services • Automate with Lightsail API & AWS Command Line Amazon Lightsail: Easiest way to get started on AWS © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
17. Users >1 © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
18. Database instance VPC Users >1 User Elastic IP address © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Amazon EC2 instance
19. Database options Fully managed Self-managed Amazon RDS © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Amazon Redshift Amazon DynamoDB Amazon EC2
20. Amazon Aurora © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • MySQL or PostgreSQL compatible • Automatic storage scaling (up to 64 TB) • Up to 15 read replicas • Continuous (incremental) backups to Amazon S3 • Six-way replications across three zones • Serverless options Amazon Aurora
21. Start with SQL databases © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
22. Why start withSQL? © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Established and well-known technology • Lots of existing code, communities, books, and tools • You aren’t going to break SQL DBs in your first millions of users. No, really, you won’t* • Clear patterns to scalability *Unless you are doing something SUPER peculiar with the data or you have MASSIVE amounts of it. . . . but even then SQL will have a place in yourstack
23. > 5 TB in year one? © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Incredibly data intensiveworkload? OK! You might needNoSQL
24. Why else might you need NoSQL? © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Super low-latency applications • Metadata-driven datasets • Highly nonrelational data • Need schema-less data constructs* • Rapid ingest of data (thousands of records/sec) • Massive amounts of data (again, in the TB range) • *Need!= “It’s easier to do dev without schemas”
25. Users >100 © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
26. VPC Amazon RDS instance Users >100 User © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Elastic IP address Amazon EC2 instance
27. Users >1000 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
28. horizontally © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. vertically
29. • Simplest approach • Can now leverage PIOPS • High I/O instances • High memory instances • High CPU instances • High storage instances • Easy to change instance sizes • Will hit an endpoint eventually t3.nano m5.2xlarge c5.9xlarge “We’re gonna need a bigger box” © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
30. User VPC Users >1000 Availability zone Web instance Amazon RDS DB instance standby (Multi-AZ) Availability zone Web instance Amazon RDS DB instance active (Multi- AZ) © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
31. Sharing the load © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
32. Application LoadBalancer © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Highly available • 1 - 65535 • Health checks • Session stickiness • Monitoring / logging • Content-based routing • Container-based apps • WebSockets • HTTP/2 Recommended
33. Users >10,000 © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
34. VPC Availability zone User Users >10,000 Amazon RDS DB instance active (Multi-AZ) Web instance Amazon RDS DB instance standby (Multi-AZ) Availability zone Web instance Web instance Web instance Web instance Web instance © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Web instance Web instance
35. VPC Availability zone User Users >10,000 Amazon RDS DB instance active (Multi-AZ) Web instance Amazon RDS DB instance standby (Multi-AZ) Availability zone Web instance Web instance Web instance Web instance Web instance © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Web instance Web instance DB instance read replica DB instance read replica DB instance read replica DB instance read replica
36. Shift some loadaround User VPC Availability zone Web instances Amazon RDS DB instance active (Multi-AZ) Availability zone Web instances Amazon RDS DB instance standby (Multi-AZ) © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
37. Amazon Simple Storage Service (Amazon S3) © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Object-based storage • Highly durable • Great for static assets • “Infinitely scalable” • Objects up to 5 TB in size • Encryption at rest and in transit
38. Amazon CloudFront © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Cache content for faster delivery • Lower load on origin • Dynamic and static content • Streaming video • Custom SSL certificates • Low TTLs (as short as 0 seconds) • Optimized for AWS
39. Amazon CloudFront Response Time Server Load Response Time Server Load Response Time Server Load No CDN CDN for static content CDN for static & dynamic content 80 70 60 50 40 30 20 10 0 Volume of Data Delivered (Gbps) © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
40. Shift some more loadaround Amazon RDS DB instance active (Multi-AZ) User VPC Availability zones Web instances © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
41. Amazon ElastiCache © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Managed Memcached or Redis • Scale from one to many nodes • Self-healing (replaces dead instance) • Single-digit ms speeds (usually) • Local to a single AZ for Memcached • Multi-AZ possible with Redis
42. Shift even more load around Amazon RDS DB instance active (Multi-AZ) User VPC Availability zones Web instances © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
43. Amazon DynamoDB © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Managed NoSQL database • Provisioned throughput • Fast, predictable performance • Fully distributed, fault tolerant • JSON support • Items up to 400 KB • Time-to-live (TTL) • Streams and triggers • AWS Application Auto Scaling • Global tables AWS Database Migration Service (AWS DMS)!
44. Auto Scaling! © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
45. Sunday Monday Tuesday Wednesday Thursday Friday Saturday Typical weekly traffic to Amazon.com © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
46. Sunday Monday Tuesday Wednesday Thursday Friday Saturday © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Provisioned capacity Typical weekly traffic to Amazon.com
47. November © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. November traffic to Amazon.com
48. Provisioned capacity © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. November November traffic to Amazon.com
49. 76% 24% November Provisioned capacity © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. November traffic to Amazon.com
50. © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. November November traffic to Amazon.com
51. Auto Scaling lets you dothis! © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
52. Auto Scaling © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Automatic resizing of compute clusters • Define min/max pool sizes • Amazon CloudWatch metrics drive scaling • On-Demand or Spot Instances
53. Amazon RDS DB instance read replica Amazon RDS DB instance read replica Amazon RDS DB instance active (Multi- AZ) Auto Scaling group Users >500,000 VPC User Web instance Amazon RDS DB instance standby (Multi-AZ) Availability zone Web instance Web instance Amazon ElastiCache Auto Scaling group Web instance Availability zone Web instance © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Web instance Amazon ElastiCache
54. Use automation © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
55. AWS Systems Manager: Automate Operational Tasks © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. AWS Systems Manager • In the cloud and on premises • Managed remote access • Automate common tasks • Basic and advanced parameter store
56. AWS infrastructure automation Convenience Control Do it yourself Higher-level services © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. AWS Cloud Development Kit
57. AWS Code Services Convenience Control AWS Code Pipeline © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Develop Source Test Build Deploy Software Release Steps AWS Cloud9 AWS CodeCommit AWS CodeBuild AWS DeviceFarm AWS CodeDeploy
58. Users >500,000 • Monitoring, metrics, and logging • If you can’t build it internally, outsource it! (third-party SaaS) • What are customers saying? • Try to squeeze as much performance out of each service/component © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
59. Aggregat e level metrics Log analysis External site performance Host level metrics
60. Amazon CloudWatch © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
61. There are further improvements to be made in breaking apart our web/app layer © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
62. The monolithic architecture User interface Business logic Data access © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
63. The service-oriented architecture Presentation tier © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Logic tier Data tier
64. SOAing Move services into their own tiers • Treat them separately • Scale them independently It offers flexibility and greater understanding of each component © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
65. Serverless = winning Don’t reinvent the wheel • API • Queuing • Transcoding • Search • Databases • Monitoring • Logging • Compute • Machine learning © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
66. Loose coupling – Amazon SQS and Amazon SNS • Reliable (multi-AZ) • Scalable (unlimited messages) • Secure (queue authentication) • Simple (simple APIs) • FIFO now supported T opic Queue subscribed to topic Message Get message Put message Publish notification © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Instance Instance
67. • Functions triggered by events • Node.js (JavaScript), Java, Python, and C# • Serverless • Implicit scaling Push: event notification Pull: DynamoDB stream Pull: Amazon Kinesis stream © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Event-driven compute – AWSLambda
68. Serverless Web application © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
69. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
70. The microservicesarchitecture © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
71. The microservicesarchitecture © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
72. AWS X-Ray © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Identify performance bottlenecks and errors • Pinpoint issues to specific service(s) in your application • Identify impact of issues on users of the application • Visualize the service call graph of your application
73. Visualize service call graph © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
74. Visualize service callgraph © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
75. Users >1,000,000 © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
76. Users >1million © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Reaching a million and above is going to require some bit of all the previous things • Multi-AZ • Elastic Load Balancing between tiers • Auto Scaling • Service oriented architecture (SOA) • Serving content smartly (Amazon S3/CloudFront) • Caching off DB • Moving state off tiers that auto scale
77. Users >1million VPC User Web Auto Scalinggroup Web Instance Web Instance Web Instance Web Instance Amazon RDS DB Instance Read Replica Amazon RDS DB Instance Active (Multi- AZ) Amazon RDS DB Instance Read Replica Amazon ElastiCache Worker Auto Scalinggroup Worker Instance Worker Instance App Scaling group App Instance © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. App Instance
78. The next big steps © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
79. Users >5 million–10million © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Database issues? Solutions • Federation—Splitting into multiple DBs based on function • Sharding—Splitting one dataset up across multiple hosts • Moving some functionality to other types of DBs (NoSQL, Graph)
80. Database federation • Split up databases by function/purpose • Harder to do cross-function queries • Essentially delays sharding/NoSQL • Won’t help with single huge functions/tables Products DB Forums DB Users DB © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
81. Sharded horizontal scaling User ShardID 002345 A 002346 B 002347 C 002348 B 002349 A C B A • More complex at the application layer • No practical limit on scalability • Operation complexity/sophistication • Shard by function or key space • RDBMS or NoSQL © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
82. Shifting functionality toNoSQL • Similar in a sense to federation • NoSQL versus SQL • Leverage managed services like DynamoDB Some use cases • Leaderboards/scoring • Rapid ingest of clickstream/log data • Temporary data needs (cart data) • “Hot” tables • Metadata/lookup tables © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
83. A quickreview © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
84. A quickreview © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Multi-AZ your infrastructure • Make use of self-scaling services—Application Load Balancer, Amazon S3,AWS Lambda, Amazon SNS, Amazon SQS, AWS Step Functions, and others • Build in redundancy at every level • Start with SQL. Seriously • Cache data both inside and outside your infrastructure • Use automation tools in your infrastructure
85. Aquick review continued © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • Make sure you have good metrics/monitoring/logging • Split tiers into individual services (SOA) • Use Auto Scaling once you’re ready for it • Don’t reinvent the wheel • Move to NoSQL if and when it makes sense
86. 10+ million users! © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
87. To infinity . .. © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved.
88. User >10million © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. • More fine-tuning of your application • More SOA of features/functionality • Build serverless whenever possible • Going from multi-AZ to multi-region • Deep analysis of your entire stack
89. Next steps? © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. Read! aws.amazon.com/documentation aws.amazon.com/architecture aws.amazon.com/well-architected aws.amazon.com/solutions aws.amazon.com/quickstart Start using AWS aws.amazon.com/free
90. Join Our User Group! © 2019, Amazon Web Services, Inc. or its affiliates. All rightsreserved. https://t.me/AWSUserGroupID
91. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Thank You! Petra Novandi Barus Senior Developer Advocate, Indonesia