Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building for A Million Transactions Per Hour

Building for A Million Transactions Per Hour

Where do you begin if you want to create a fail-safe, fault tolerant, high-traffic-ready architecture to handle large amounts of traffic or load? This talk is about that. Gave this talk at the 2017 http://cakephp.org http://cakefest.org event in New York

Femi TAIWO

June 11, 2017
Tweet

More Decks by Femi TAIWO

Other Decks in Technology

Transcript

  1. ❖ 70,000,000 Registered Voters ❖ Most visible participation expected ❖

    120,000+ Locations ❖ 32,000,000 Transactions in ~=24 Hours ❖ 24 Days - Start to Finish ❖ This was the Pilot :) Key Facts About The 2015 Elections
  2. ❖ CakePHP 2.x ❖ Google App Engine ❖ Google Compute

    Engine ❖ Cloud SQL - MySQL ➢ 70 million Base Records ❖ BigQuery ❖ Data Store Stack Used in The 2015 Elections
  3. ❖ Google Cloud Storage ❖ Memcached ❖ Push Queues ❖

    Stack Driver ❖ jMeter && Python Stack Used in The 2015 Elections - Cont’d
  4. Building & Deploying Applications - Now easier than ever before

    - Ready-made systems - Done in 10 minutes or less Image courtesy DigitalOcean
  5. Building & Deploying Applications Growth/High Demand - the good and

    horrible kind of problem - happens - Quality of Service - Availability - Consistency - Speed
  6. Building with Millions in Mind Things to think about -

    Performance - Scalability - High Availability - Pain-free DevOps (+/- y) - Costs
  7. Scalability : What Indicates ability to handle growing amounts of

    work in a graceful manner or to be readily enlarged as demands increase. David Mitzenmacher
  8. Scalability : Modes Scaling Vertically Usually, as when we experience

    increasing loads, we keep increasing the server capacity - CPU, Memory and Bandwidth too.
  9. Scalability : Challenges Challenges with Vertical Scaling - Physical Limit

    - Availability Loss - Cost versus Benefit Analysis - DevOps
  10. Scalability : Modes Horizontal Scalability | Scale Out Increasing the

    capacity of application by adding more nodes to the cluster AND reducing nodes based on load volume (Scale In) “Add more nodes”
  11. Scalability Freshman Year Scaling Method - Separate Application Server -

    Separate Database Server - Add More Application Servers as Load Increases - Load Balancer - Increase Database Server Capacity - Introduce Replication (MS, MM) Manual & mostly after the problems have been experienced
  12. Scalability Veteran Scaling Method - Built-In (Application Design) - Parallelization

    - Asynchronous - Idempotency | Same Results - Shared-Nothing - Partitioned-Data - Fault Tolerance - Autonomous
  13. High Availability What is it? - Availability is the ability

    of your system to be operational when needed. - High availability is defined as the percentage of time that the system is functioning, operational and accessible.
  14. Poor Availability: Signs - Unexpected Behaviors - Errors! Errors Everywhere!

    - Delayed Responses - Timeouts - Server Not Responding. Necessary Restarts
  15. Poor Availability: Usual Suspects - Resource Exhaustion - Unplanned load-based

    changes - Increased number of moving parts - Outside dependencies - Technical Debt - Unprepared Upgrades
  16. High Availability Design & Implementation that ensures the application is

    always UP. Provides a certain degree of operation continuity even when traffic/volume spikes
  17. Journey Towards High Availability - Design and Develop Deliberately -

    Expect things to fail - Catch, Retry and Traps - Dependency Problems - Recovery from Third-Party Failures - Identify Risks. Deal with them - Application Crash - Database Corruption - Storage Problems - Connectivity Problems
  18. Journey Towards High Availability - Monitor Everything - Server Health

    - Application Performance - Available vs Used - Trends - Communications - StackDriver - New Relic - Datadog - PagerDuty
  19. Journey Towards High Availability - Provide Predefined Steps - Alerts

    - Diagnosis - Issues Management - Automate Operations - Tests - Rollouts & Deployments - Configuration Changes - Rollbacks Setup
  20. Monolithic Systems Functionally distinguishable aspects are all interwoven, rather than

    containing architecturally separate components. Built and Deployed as a single unit https://en.wikipedia.org/wiki/Monolithic_system https://www.mulesoft.com/resources/api/microservices-vs-monolithic Common Style
  21. Some Challenges - High Coupling - Continuous Deployment is Hard

    - Long Term Technology Commitment - Issues Scaling Development Common Style : Monolithic Systems
  22. Styling for Scalability Service-Oriented Architectures: Microservices Flexible, distinct systems that

    provide specific business functionality as a subset of a larger system. Incredibly easier to scale | test | deploy | maintain | improve
  23. Styling for Scalability Key Criteria for Microservices - Independent Code

    Base - Independent Data (Matched with Event Driven Architecture) - Strong Contractual Role - Consumer of Other Services - Owned - Requirement Driven - Reasonable Size & Role
  24. Microservice: Strategies Decompose by - Business capability - Subdomain -

    Decompose by verb or use case - e.g. a Shipping Service - Decompose by nouns or resources e.g an Account Service for managing user accounts. http://microservices.io/patterns/microservices.html
  25. Microservice: Core Benefits - Independently Scalable - Logical Isolation -

    Flexible Choice of Technology - Better Machine Utilization - Encourages Continuous Deployment
  26. Microservice: Key Challenges - Requires strong interfaces - More network

    calls - Machine specialization - Ops overhead - Logs - Health Checks - Deployment - Networking
  27. Microservice: Case Study - Elections - Configuration Service - Transmission

    Service - Verification Service - Reporting Service - User Account Service - Update Service - Storage Service
  28. Microservice: eCommerce example - Product Catalog Service - Shopping Cart

    Service - Order Service - Shipping Service - Inventory Service - Recommendation Service - Review Service https://speakerdeck.com/cyberomin/microservices-a-love-story
  29. Message Queues: What A message queue is a queue of

    messages sent between applications. It includes a sequence of work objects that are waiting to be processed. Usually applies a FIFO Principle. https://www.cloudamqp.com/blog/2014-12-03-what-is-message-queuing.html
  30. Message Queues: Why • Redundancy via Persistence • Traffic Spikes

    • Asynchronous Messaging • Decoupling • Transaction Ordering and Concurrency Challenges • Improve Scalability • Break Larger Tasks into Many Smaller Ones • Monitoring
  31. Caching: Knowing What To ❖ What changes? ❖ What changes

    often? ❖ What is requested the most?
  32. !Application Things To Think About Failure Domains - Server Rack

    - Router - Power Source - Datacenter | Zones - Country
  33. ❖ JMeter ❖ Apache bench ❖ Bees with Machine Guns!

    ❖ Selenium ❖ tourbus ❖ httperf ❖ weighttp ❖ httpress ❖ siege Load/Performance Testing
  34. ❖ CakePHP 2.x ❖ Google App Engine ❖ Google Compute

    Engine ❖ Cloud SQL - MySQL ❖ BigQuery ❖ Data Store ❖ Google Cloud Storage ❖ Push Queues ❖ Stack Driver ❖ jMeter && Python Stack Used in The 2015 Elections
  35. Scaling on Google Infrastructure Google Cloud Platform - Cloud computing

    service by Google - Provides developer products - Offers hosting on the same supporting infrastructure
  36. Key Products on GCP Google Container Engine Powerful cluster manager

    and orchestration system. Automatically scale up and down your application based on your resource utilization (CPU, memory). https://cloud.google.com/container-engine/ Google Cloud Spanner World’s first fully managed relational database service to offer both strong consistency and horizontal scalability for mission-critical online transaction processing (OLTP) applications. https://cloud.google.com/spanner/
  37. Key Products on GCP Google App Engine Platform As A

    Service. Automatic Scaling at its best. https://cloud.google.com/appengine/ Google Compute Engine Run VMs on Google Infrastructure. 1000. Sustained Discounts (~40%). Committed Use Discounts (57%) https://cloud.google.com/compute/
  38. Google I/O 2017 #io17 https://events.google.com/io/ Key Links $300 Credit to

    Try Out GCP https://cloud.google.com/free/ Everything about GCP https://cloud.google.com Free Labs, Courses & Training on GCP https://google.qwiklabs.com Free hands-on practice
  39. AWS Elastic Beanstalk Easy-to-use service for deploying and scaling web

    applications and services developed with Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker Automatic capacity provisioning, load balancing, auto-scaling to application health monitoring. https://aws.amazon.com/elasticbeanstalk/ Scaling on AWS Infrastructure
  40. Scaling on AWS Infrastructure AWS Lambda AWS Lambda lets you

    run code without provisioning or managing servers. Unlimited Scaling based on Events. https://aws.amazon.com/lambda/
  41. More to Think About - Be Stateless. Split Code &

    Data - Managing State - HTTP/2 Support - CDNs - Graceful Backoffs - Alternate Services
  42. Things to Remember - Design and Develop Deliberately - Expect

    things to Fail - Identify Risks. Deal with them - Monitor Everything - Provide Predefined Steps - Automate Operations
  43. Further Reading ❖ 12 Reasons to Use Message Queuing https://stackify.com/message-queues-12-reasons/

    ❖ HTTP/2 Server Push - https://www.smashingmagazine.com/2017/04/guide-http2-server-push/ ❖ CAP Theorem - https://en.wikipedia.org/wiki/CAP_theorem ❖ Cloud Spanner - https://cloud.google.com/spanner/ ❖ Plenty about Microservices - http://microservices.io/ ❖ Bees with Machine Guns - https://github.com/newsapps/beeswithmachineguns ❖ Curl-Loader - http://curl-loader.sourceforge.net/