Building for A Million Transactions Per Hour

Where do you begin if you want to create a fail-safe, fault tolerant, high-traffic-ready architecture to handle large amounts of traffic or load? This talk is about that. Gave this talk at the 2017 http://cakephp.org http://cakefest.org event in New York

Transcript

1. Building for a Million Transactions Per Hour By Femi TAIWO | @dftaiwo
4. 2015 Presidential Elections
5. ❖ 70,000,000 Registered Voters ❖ Most visible participation expected ❖ 120,000+ Locations ❖ 32,000,000 Transactions in ~=24 Hours ❖ 24 Days - Start to Finish ❖ This was the Pilot :) Key Facts About The 2015 Elections
6. ❖ CakePHP 2.x ❖ Google App Engine ❖ Google Compute Engine ❖ Cloud SQL - MySQL ➢ 70 million Base Records ❖ BigQuery ❖ Data Store Stack Used in The 2015 Elections
7. ❖ Google Cloud Storage ❖ Memcached ❖ Push Queues ❖ Stack Driver ❖ jMeter && Python Stack Used in The 2015 Elections - Cont’d
8. Some Fundamentals
9. Building & Deploying Applications - Now easier than ever before - Ready-made systems - Done in 10 minutes or less Image courtesy DigitalOcean
10. Building & Deploying Applications Or maybe this one Image courtesy DigitalOcean
11. Building & Deploying Applications Growth/High Demand - the good and horrible kind of problem - happens - Quality of Service - Availability - Consistency - Speed
12. Building with Millions in Mind Things to think about - Performance - Scalability - High Availability - Pain-free DevOps (+/- y) - Costs
13. Performance
14. Scalability
15. Scalability : What Indicates ability to handle growing amounts of work in a graceful manner or to be readily enlarged as demands increase. David Mitzenmacher
16. Scalability : Modes Scaling Vertically Usually, as when we experience increasing loads, we keep increasing the server capacity - CPU, Memory and Bandwidth too.
17. Scalability : Challenges Challenges with Vertical Scaling - Physical Limit - Availability Loss - Cost versus Benefit Analysis - DevOps
18. Scalability : Modes Horizontal Scalability | Scale Out Increasing the capacity of application by adding more nodes to the cluster AND reducing nodes based on load volume (Scale In) “Add more nodes”
19. Scalability : Challenges Challenges with Horizontal Scaling - Data Replication - Consistency CAP Theorem
20. Scalability Freshman Year Scaling Method - Separate Application Server - Separate Database Server - Add More Application Servers as Load Increases - Load Balancer - Increase Database Server Capacity - Introduce Replication (MS, MM) Manual & mostly after the problems have been experienced
21. Scalability Veteran Scaling Method - Built-In (Application Design) - Parallelization - Asynchronous - Idempotency | Same Results - Shared-Nothing - Partitioned-Data - Fault Tolerance - Autonomous
22. Availability
23. High Availability What is it? - Availability is the ability of your system to be operational when needed. - High availability is defined as the percentage of time that the system is functioning, operational and accessible.
24. Poor Availability: Signs - Unexpected Behaviors - Errors! Errors Everywhere! - Delayed Responses - Timeouts - Server Not Responding. Necessary Restarts
25. Poor Availability: Usual Suspects - Resource Exhaustion - Unplanned load-based changes - Increased number of moving parts - Outside dependencies - Technical Debt - Unprepared Upgrades
26. High Availability Design & Implementation that ensures the application is always UP. Provides a certain degree of operation continuity even when traffic/volume spikes
27. Journey Towards High Availability - Design and Develop Deliberately - Expect things to fail - Catch, Retry and Traps - Dependency Problems - Recovery from Third-Party Failures - Identify Risks. Deal with them - Application Crash - Database Corruption - Storage Problems - Connectivity Problems
28. Journey Towards High Availability - Monitor Everything - Server Health - Application Performance - Available vs Used - Trends - Communications - StackDriver - New Relic - Datadog - PagerDuty
29. Journey Towards High Availability - Provide Predefined Steps - Alerts - Diagnosis - Issues Management - Automate Operations - Tests - Rollouts & Deployments - Configuration Changes - Rollbacks Setup
30. Design & Develop Deliberately
31. Monolithic Systems Functionally distinguishable aspects are all interwoven, rather than containing architecturally separate components. Built and Deployed as a single unit https://en.wikipedia.org/wiki/Monolithic_system https://www.mulesoft.com/resources/api/microservices-vs-monolithic Common Style
32. Some Challenges - High Coupling - Continuous Deployment is Hard - Long Term Technology Commitment - Issues Scaling Development Common Style : Monolithic Systems
33. Styling for Scalability Service-Oriented Architectures: Microservices Flexible, distinct systems that provide specific business functionality as a subset of a larger system. Incredibly easier to scale | test | deploy | maintain | improve
34. Styling for Scalability Key Criteria for Microservices - Independent Code Base - Independent Data (Matched with Event Driven Architecture) - Strong Contractual Role - Consumer of Other Services - Owned - Requirement Driven - Reasonable Size & Role
35. Microservice: Strategies Decompose by - Business capability - Subdomain - Decompose by verb or use case - e.g. a Shipping Service - Decompose by nouns or resources e.g an Account Service for managing user accounts. http://microservices.io/patterns/microservices.html
36. Microservice: Core Benefits - Independently Scalable - Logical Isolation - Flexible Choice of Technology - Better Machine Utilization - Encourages Continuous Deployment
37. Microservice: Key Challenges - Requires strong interfaces - More network calls - Machine specialization - Ops overhead - Logs - Health Checks - Deployment - Networking
38. Microservice: Case Study - Elections - Configuration Service - Transmission Service - Verification Service - Reporting Service - User Account Service - Update Service - Storage Service
39. Microservice: eCommerce example - Product Catalog Service - Shopping Cart Service - Order Service - Shipping Service - Inventory Service - Recommendation Service - Review Service https://speakerdeck.com/cyberomin/microservices-a-love-story
40. Delivered By Message Queues
41. Practically everything was pushed to a Queue
42. Message Queues: What A message queue is a queue of messages sent between applications. It includes a sequence of work objects that are waiting to be processed. Usually applies a FIFO Principle. https://www.cloudamqp.com/blog/2014-12-03-what-is-message-queuing.html
43. Message Queues: What
44. Message Queues: Why • Redundancy via Persistence • Traffic Spikes • Asynchronous Messaging • Decoupling • Transaction Ordering and Concurrency Challenges • Improve Scalability • Break Larger Tasks into Many Smaller Ones • Monitoring
45. Caching: Knowing What To ❖ What changes? ❖ What changes often? ❖ What is requested the most?
46. !Application Things To Think About Failure Domains - Server Rack - Router - Power Source - Datacenter | Zones - Country
47. Know Thy Self Test, Test and Test More
48. ❖ JMeter ❖ Apache bench ❖ Bees with Machine Guns! ❖ Selenium ❖ tourbus ❖ httperf ❖ weighttp ❖ httpress ❖ siege Load/Performance Testing
49. ❖ CakePHP 2.x ❖ Google App Engine ❖ Google Compute Engine ❖ Cloud SQL - MySQL ❖ BigQuery ❖ Data Store ❖ Google Cloud Storage ❖ Push Queues ❖ Stack Driver ❖ jMeter && Python Stack Used in The 2015 Elections
50. Google Cloud Platform
51. Scaling on Google Infrastructure Google Cloud Platform - Cloud computing service by Google - Provides developer products - Offers hosting on the same supporting infrastructure
52. Key Products on GCP Google Container Engine Powerful cluster manager and orchestration system. Automatically scale up and down your application based on your resource utilization (CPU, memory). https://cloud.google.com/container-engine/ Google Cloud Spanner World’s first fully managed relational database service to offer both strong consistency and horizontal scalability for mission-critical online transaction processing (OLTP) applications. https://cloud.google.com/spanner/
53. Key Products on GCP Google App Engine Platform As A Service. Automatic Scaling at its best. https://cloud.google.com/appengine/ Google Compute Engine Run VMs on Google Infrastructure. 1000. Sustained Discounts (~40%). Committed Use Discounts (57%) https://cloud.google.com/compute/
54. Google I/O 2017 #io17 https://events.google.com/io/ Key Links $300 Credit to Try Out GCP https://cloud.google.com/free/ Everything about GCP https://cloud.google.com Free Labs, Courses & Training on GCP https://google.qwiklabs.com Free hands-on practice
55. Amazon
56. AWS Elastic Beanstalk Easy-to-use service for deploying and scaling web applications and services developed with Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker Automatic capacity provisioning, load balancing, auto-scaling to application health monitoring. https://aws.amazon.com/elasticbeanstalk/ Scaling on AWS Infrastructure
57. Scaling on AWS Infrastructure AWS Lambda AWS Lambda lets you run code without provisioning or managing servers. Unlimited Scaling based on Events. https://aws.amazon.com/lambda/
58. More to Think About - Be Stateless. Split Code & Data - Managing State - HTTP/2 Support - CDNs - Graceful Backoffs - Alternate Services
59. Things to Remember - Design and Develop Deliberately - Expect things to Fail - Identify Risks. Deal with them - Monitor Everything - Provide Predefined Steps - Automate Operations
60. Further Reading ❖ 12 Reasons to Use Message Queuing https://stackify.com/message-queues-12-reasons/ ❖ HTTP/2 Server Push - https://www.smashingmagazine.com/2017/04/guide-http2-server-push/ ❖ CAP Theorem - https://en.wikipedia.org/wiki/CAP_theorem ❖ Cloud Spanner - https://cloud.google.com/spanner/ ❖ Plenty about Microservices - http://microservices.io/ ❖ Bees with Machine Guns - https://github.com/newsapps/beeswithmachineguns ❖ Curl-Loader - http://curl-loader.sourceforge.net/
61. Thank You @dftaiwo