Building for A Million Transactions Per Hour

Building for A Million Transactions Per Hour

Where do you begin if you want to create a fail-safe, fault tolerant, high-traffic-ready architecture to handle large amounts of traffic or load? This talk is about that. Gave this talk at the 2017 http://cakephp.org http://cakefest.org event in New York

D2a21b64d2da715a0ee5f89a9dd65d94?s=128

Femi TAIWO

June 11, 2017
Tweet

Transcript

  1. Building for a Million Transactions Per Hour By Femi TAIWO

    | @dftaiwo
  2. None
  3. None
  4. 2015 Presidential Elections

  5. ❖ 70,000,000 Registered Voters ❖ Most visible participation expected ❖

    120,000+ Locations ❖ 32,000,000 Transactions in ~=24 Hours ❖ 24 Days - Start to Finish ❖ This was the Pilot :) Key Facts About The 2015 Elections
  6. ❖ CakePHP 2.x ❖ Google App Engine ❖ Google Compute

    Engine ❖ Cloud SQL - MySQL ➢ 70 million Base Records ❖ BigQuery ❖ Data Store Stack Used in The 2015 Elections
  7. ❖ Google Cloud Storage ❖ Memcached ❖ Push Queues ❖

    Stack Driver ❖ jMeter && Python Stack Used in The 2015 Elections - Cont’d
  8. Some Fundamentals

  9. Building & Deploying Applications - Now easier than ever before

    - Ready-made systems - Done in 10 minutes or less Image courtesy DigitalOcean
  10. Building & Deploying Applications Or maybe this one Image courtesy

    DigitalOcean
  11. Building & Deploying Applications Growth/High Demand - the good and

    horrible kind of problem - happens - Quality of Service - Availability - Consistency - Speed
  12. Building with Millions in Mind Things to think about -

    Performance - Scalability - High Availability - Pain-free DevOps (+/- y) - Costs
  13. Performance

  14. Scalability

  15. Scalability : What Indicates ability to handle growing amounts of

    work in a graceful manner or to be readily enlarged as demands increase. David Mitzenmacher
  16. Scalability : Modes Scaling Vertically Usually, as when we experience

    increasing loads, we keep increasing the server capacity - CPU, Memory and Bandwidth too.
  17. Scalability : Challenges Challenges with Vertical Scaling - Physical Limit

    - Availability Loss - Cost versus Benefit Analysis - DevOps
  18. Scalability : Modes Horizontal Scalability | Scale Out Increasing the

    capacity of application by adding more nodes to the cluster AND reducing nodes based on load volume (Scale In) “Add more nodes”
  19. Scalability : Challenges Challenges with Horizontal Scaling - Data Replication

    - Consistency CAP Theorem
  20. Scalability Freshman Year Scaling Method - Separate Application Server -

    Separate Database Server - Add More Application Servers as Load Increases - Load Balancer - Increase Database Server Capacity - Introduce Replication (MS, MM) Manual & mostly after the problems have been experienced
  21. Scalability Veteran Scaling Method - Built-In (Application Design) - Parallelization

    - Asynchronous - Idempotency | Same Results - Shared-Nothing - Partitioned-Data - Fault Tolerance - Autonomous
  22. Availability

  23. High Availability What is it? - Availability is the ability

    of your system to be operational when needed. - High availability is defined as the percentage of time that the system is functioning, operational and accessible.
  24. Poor Availability: Signs - Unexpected Behaviors - Errors! Errors Everywhere!

    - Delayed Responses - Timeouts - Server Not Responding. Necessary Restarts
  25. Poor Availability: Usual Suspects - Resource Exhaustion - Unplanned load-based

    changes - Increased number of moving parts - Outside dependencies - Technical Debt - Unprepared Upgrades
  26. High Availability Design & Implementation that ensures the application is

    always UP. Provides a certain degree of operation continuity even when traffic/volume spikes
  27. Journey Towards High Availability - Design and Develop Deliberately -

    Expect things to fail - Catch, Retry and Traps - Dependency Problems - Recovery from Third-Party Failures - Identify Risks. Deal with them - Application Crash - Database Corruption - Storage Problems - Connectivity Problems
  28. Journey Towards High Availability - Monitor Everything - Server Health

    - Application Performance - Available vs Used - Trends - Communications - StackDriver - New Relic - Datadog - PagerDuty
  29. Journey Towards High Availability - Provide Predefined Steps - Alerts

    - Diagnosis - Issues Management - Automate Operations - Tests - Rollouts & Deployments - Configuration Changes - Rollbacks Setup
  30. Design & Develop Deliberately

  31. Monolithic Systems Functionally distinguishable aspects are all interwoven, rather than

    containing architecturally separate components. Built and Deployed as a single unit https://en.wikipedia.org/wiki/Monolithic_system https://www.mulesoft.com/resources/api/microservices-vs-monolithic Common Style
  32. Some Challenges - High Coupling - Continuous Deployment is Hard

    - Long Term Technology Commitment - Issues Scaling Development Common Style : Monolithic Systems
  33. Styling for Scalability Service-Oriented Architectures: Microservices Flexible, distinct systems that

    provide specific business functionality as a subset of a larger system. Incredibly easier to scale | test | deploy | maintain | improve
  34. Styling for Scalability Key Criteria for Microservices - Independent Code

    Base - Independent Data (Matched with Event Driven Architecture) - Strong Contractual Role - Consumer of Other Services - Owned - Requirement Driven - Reasonable Size & Role
  35. Microservice: Strategies Decompose by - Business capability - Subdomain -

    Decompose by verb or use case - e.g. a Shipping Service - Decompose by nouns or resources e.g an Account Service for managing user accounts. http://microservices.io/patterns/microservices.html
  36. Microservice: Core Benefits - Independently Scalable - Logical Isolation -

    Flexible Choice of Technology - Better Machine Utilization - Encourages Continuous Deployment
  37. Microservice: Key Challenges - Requires strong interfaces - More network

    calls - Machine specialization - Ops overhead - Logs - Health Checks - Deployment - Networking
  38. Microservice: Case Study - Elections - Configuration Service - Transmission

    Service - Verification Service - Reporting Service - User Account Service - Update Service - Storage Service
  39. Microservice: eCommerce example - Product Catalog Service - Shopping Cart

    Service - Order Service - Shipping Service - Inventory Service - Recommendation Service - Review Service https://speakerdeck.com/cyberomin/microservices-a-love-story
  40. Delivered By Message Queues

  41. Practically everything was pushed to a Queue

  42. Message Queues: What A message queue is a queue of

    messages sent between applications. It includes a sequence of work objects that are waiting to be processed. Usually applies a FIFO Principle. https://www.cloudamqp.com/blog/2014-12-03-what-is-message-queuing.html
  43. Message Queues: What

  44. Message Queues: Why • Redundancy via Persistence • Traffic Spikes

    • Asynchronous Messaging • Decoupling • Transaction Ordering and Concurrency Challenges • Improve Scalability • Break Larger Tasks into Many Smaller Ones • Monitoring
  45. Caching: Knowing What To ❖ What changes? ❖ What changes

    often? ❖ What is requested the most?
  46. !Application Things To Think About Failure Domains - Server Rack

    - Router - Power Source - Datacenter | Zones - Country
  47. Know Thy Self Test, Test and Test More

  48. ❖ JMeter ❖ Apache bench ❖ Bees with Machine Guns!

    ❖ Selenium ❖ tourbus ❖ httperf ❖ weighttp ❖ httpress ❖ siege Load/Performance Testing
  49. ❖ CakePHP 2.x ❖ Google App Engine ❖ Google Compute

    Engine ❖ Cloud SQL - MySQL ❖ BigQuery ❖ Data Store ❖ Google Cloud Storage ❖ Push Queues ❖ Stack Driver ❖ jMeter && Python Stack Used in The 2015 Elections
  50. Google Cloud Platform

  51. Scaling on Google Infrastructure Google Cloud Platform - Cloud computing

    service by Google - Provides developer products - Offers hosting on the same supporting infrastructure
  52. Key Products on GCP Google Container Engine Powerful cluster manager

    and orchestration system. Automatically scale up and down your application based on your resource utilization (CPU, memory). https://cloud.google.com/container-engine/ Google Cloud Spanner World’s first fully managed relational database service to offer both strong consistency and horizontal scalability for mission-critical online transaction processing (OLTP) applications. https://cloud.google.com/spanner/
  53. Key Products on GCP Google App Engine Platform As A

    Service. Automatic Scaling at its best. https://cloud.google.com/appengine/ Google Compute Engine Run VMs on Google Infrastructure. 1000. Sustained Discounts (~40%). Committed Use Discounts (57%) https://cloud.google.com/compute/
  54. Google I/O 2017 #io17 https://events.google.com/io/ Key Links $300 Credit to

    Try Out GCP https://cloud.google.com/free/ Everything about GCP https://cloud.google.com Free Labs, Courses & Training on GCP https://google.qwiklabs.com Free hands-on practice
  55. Amazon

  56. AWS Elastic Beanstalk Easy-to-use service for deploying and scaling web

    applications and services developed with Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker Automatic capacity provisioning, load balancing, auto-scaling to application health monitoring. https://aws.amazon.com/elasticbeanstalk/ Scaling on AWS Infrastructure
  57. Scaling on AWS Infrastructure AWS Lambda AWS Lambda lets you

    run code without provisioning or managing servers. Unlimited Scaling based on Events. https://aws.amazon.com/lambda/
  58. More to Think About - Be Stateless. Split Code &

    Data - Managing State - HTTP/2 Support - CDNs - Graceful Backoffs - Alternate Services
  59. Things to Remember - Design and Develop Deliberately - Expect

    things to Fail - Identify Risks. Deal with them - Monitor Everything - Provide Predefined Steps - Automate Operations
  60. Further Reading ❖ 12 Reasons to Use Message Queuing https://stackify.com/message-queues-12-reasons/

    ❖ HTTP/2 Server Push - https://www.smashingmagazine.com/2017/04/guide-http2-server-push/ ❖ CAP Theorem - https://en.wikipedia.org/wiki/CAP_theorem ❖ Cloud Spanner - https://cloud.google.com/spanner/ ❖ Plenty about Microservices - http://microservices.io/ ❖ Bees with Machine Guns - https://github.com/newsapps/beeswithmachineguns ❖ Curl-Loader - http://curl-loader.sourceforge.net/
  61. Thank You @dftaiwo