Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Latency's Worst Nightmare: Performance Tuning Tips and Tricks

Latency's Worst Nightmare: Performance Tuning Tips and Tricks

Delivering high performance web applications, using Amazon Web Services.

Matt Wood

April 18, 2013
Tweet

More Decks by Matt Wood

Other Decks in Technology

Transcript

  1. Latency’s Worst Nightmare:
    Performance Tuning Tips and Tricks
    Matt Wood
    Principal Data Scientist
    @mza

    View Slide

  2. Hello.

    View Slide

  3. Let’s talk about performance...

    View Slide

  4. Figure 3 Interactive user productivity versus computer response time for human-intensive
    interactions for system A
    E 600
    3
    -
    T
    7
    w
    z
    E 500
    -
    U
    E
    w
    E
    -
    >
    >
    -
    -
    400
    -
    3
    n
    F
    2
    300
    -
    200
    -
    100
    -
    0 -
    0
    -" INTERACTIVE USER PRODUCTIVITY (IUP)
    - HUMAN-INTENSIVE COMPONENT OF IUP
    A
    MEASURED DATA (HUMAN-INTENSIVE
    " COMPONENT)
    0
    0
    0
    0
    I 1 I I I
    1 2 3 4 5
    COMPUTER RESPONSE TIME (SI
    A. J. Thadhani, IBM Systems Journal 20 (4), 1981
    Productivity and response time

    View Slide

  5. -0.70%
    -0.60%
    -0.50%
    -0.40%
    -0.30%
    -0.20%
    -0.10%
    0.00%
    50ms pre-
    header
    100ms pre-
    header
    200ms post-
    header
    200ms post-
    ads
    400ms post-
    header
    Page load time and average daily searches per user
    http://www.webperformancetoday.com/2013/04/10/cloud-connect-2013-web-acceleration-and-front-end-optimization-slides/

    View Slide

  6. -5.00%
    -4.50%
    -4.00%
    -3.50%
    -3.00%
    -2.50%
    -2.00%
    -1.50%
    -1.00%
    -0.50%
    0.00%
    50 200 500 1000 2000
    Percent change
    Added delay
    Queries per visitor Query refinement Revenue per visitor
    Any clicks Satisfaction
    Page load delay and business metrics
    http://www.webperformancetoday.com/2013/04/10/cloud-connect-2013-web-acceleration-and-front-end-optimization-slides/

    View Slide

  7. 2.2s 15.4%
    reduction in page load time increase in conversion rate
    https://blog.mozilla.org/metrics/2010/04/05/firefox-page-load-speed-%E2%80%93-part-ii/

    View Slide

  8. “Speed is more than a feature.”
    @fredwilson

    View Slide

  9. Let’s talk about a web request...

    View Slide

  10. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps

    View Slide

  11. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    Remote

    View Slide

  12. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    Remote Browser

    View Slide

  13. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    Remote Browser

    View Slide

  14. http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/

    View Slide

  15. View Slide

  16. View Slide

  17. Let’s talk about a web request...

    View Slide

  18. Web server
    Application logic
    Database

    View Slide

  19. Web server
    Application logic
    Database

    View Slide

  20. Web server
    Application logic
    Database

    View Slide

  21. Web server
    Application logic
    Database

    View Slide

  22. Web server
    Application logic
    Database

    View Slide

  23. Web server
    Application logic
    Database

    View Slide

  24. Web server
    Application logic
    Database

    View Slide

  25. Web server
    Application logic
    Database

    View Slide

  26. Web server
    Application logic
    Database
    3
    4
    6
    5
    7
    2
    1

    View Slide

  27. Web server
    Application logic
    Database
    3
    4
    6
    5
    7
    2
    1

    View Slide

  28. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    TTFB

    View Slide

  29. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    Network latency Download + negotiation time

    View Slide

  30. View Slide

  31. Variable TTFB based on geographic location.
    NYC London Sydney

    View Slide

  32. Reduce internet induced latency.
    CloudFront content delivery network.
    Lower latency. Faster downloads.

    View Slide

  33. Reduce internet induced latency.
    CloudFront content delivery network.
    Lower latency. Faster downloads.

    View Slide

  34. Europe
    Amsterdam (2)
    Dublin
    Frankfurt (2)
    London (2)
    Madrid
    Milan
    Paris (2)
    Stockholm
    South America
    Sao Paulo
    North America
    Ashburn, VA (2)
    Dallas, TX (2)
    Hayward, CA
    Jacksonville, FL
    Los Angeles, CA (2)
    Miami, FL
    Newark, NJ
    New York, NY (3)
    Palo Alto, CA
    Seattle, WA
    San Jose, CA
    South Bend, IN
    St. Louis, MO
    CloudFront Edge Locations

    View Slide

  35. Static and dynamic content
    Cache dynamic pages (search results).
    Use query strings or cookie for cache keys.
    Network and Path optimizations accelerate
    even unique content.

    View Slide

  36. Web server
    Application logic
    Database
    3
    4
    6
    5
    7
    2
    1 Content delivery

    View Slide

  37. 7
    Web server
    Application logic
    Database
    3
    4
    6
    5
    1
    2
    Content delivery

    View Slide

  38. Web server
    Application server
    Database

    View Slide

  39. Web server
    Application server
    Database

    View Slide

  40. Application server
    Database

    View Slide

  41. Application server
    Thread safe?
    Application state?
    Database

    View Slide

  42. Web server
    Application server
    Database

    View Slide

  43. Web server
    Application server
    Database
    Unit of scale

    View Slide

  44. Web server
    Application server
    Database
    Load balancer

    View Slide

  45. Web server
    Application server
    Database
    Web server
    Application server
    Web server
    Application server
    Load balancer

    View Slide

  46. Load balancer
    Web server
    Application server
    Database
    Web server
    Application server
    Web server
    Application server

    View Slide

  47. Load balancer
    Web server
    Application server
    Database
    Web server
    Application server
    Web server
    Application server

    View Slide

  48. Decouple your service tiers
    Separation of concerns.
    Easier to manage.
    Drives higher availability.

    View Slide

  49. Build for horizontal scale
    Decrease request contention.
    Reduce capacity planning headaches.
    Requires a stateless application architecture.

    View Slide

  50. Small things, loosely coupled.
    Do one thing, and do it well. The Unix Way.
    Take a look at the Unicorn and Rainbows approach.
    Asynchronous be default (where possible).

    View Slide

  51. Reduce response time.
    Concurrency limits can reappear quickly.
    Limit impact on performance through rapid scaling.

    View Slide

  52. Load balancer

    View Slide

  53. Load balancer

    View Slide

  54. Load balancer

    View Slide

  55. Fast booting with EBS-backed instances.
    Linux is faster to boot than Windows.
    EBS-backed instances are faster than S3 backed.

    View Slide

  56. Pre-baked EBS-backed AMIs.
    Each code deployment creates a new AMI.
    AMI is the unit of deployment.

    View Slide

  57. Automate response with Auto Scaling.
    Set operational thresholds.
    Faster than manual response (especially at 3am).

    View Slide

  58. Time-based response with OpsWorks.
    Set operational times.
    ‘Follow the sun’ response.

    View Slide

  59. Pre-emptive scaling.
    Contact your account manager, or
    get in touch via a Premium Support ticket.

    View Slide

  60. Reserve capacity.
    Lower costs.
    Guaranteed availability.

    View Slide

  61. 7
    Database
    3
    4
    6
    5
    1
    2
    Content delivery
    Horizontal scale

    View Slide

  62. 7
    Database
    4
    6
    5
    1
    2
    Content delivery
    Horizontal scale
    3

    View Slide

  63. A range of instance types
    Full spectrum of price/performance options.

    View Slide

  64. Choosing the right instance type.
    Application specific.
    Benchmark.
    CloudWatch metrics.

    View Slide

  65. Benchmark on business metrics.
    Relate application metrics to business metrics.
    Customers supported/instance.
    Photos processed/dollar.

    View Slide

  66. The Canary in the Coal Mine
    Standardize on 64-bit AMIs.
    Deploy across instance types.
    Evaluate new instance types with real traffic.

    View Slide

  67. View Slide

  68. View Slide

  69. View Slide

  70. 7
    Database
    4
    6
    5
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection

    View Slide

  71. 7
    Database
    6
    5
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4

    View Slide

  72. Interface with the data store.
    Faster if you don’t have to go to disk.
    Increased concurrency.

    View Slide

  73. Caching.
    Store query results in memory.
    Writes go to disk.

    View Slide

  74. Amazon ElastiCache.
    Deploy, operate and scale in-memory caches.

    View Slide

  75. Managing state.
    Transient data only.
    Web server state, high score tables, etc.
    Time consuming task results
    (many to many query results).

    View Slide

  76. Best practices.
    Assume cold cache latency in application architecture.
    Set appropriate time-to-live.
    Batch requests rather than sequential single.
    Architect for cache failure.

    View Slide

  77. 7
    Database
    6
    5
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4 Caching

    View Slide

  78. 7
    Database
    6
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4 Caching
    5

    View Slide

  79. Accelerating reads.
    Vertical scale vs horizontal scale.

    View Slide

  80. Horizontal scale.
    Add additional DB resources for scale.
    Scale out for reads.
    Shard for reads and writes.

    View Slide

  81. Vertical scale.
    More resources for a single DB engine.
    Add memory for DB caches.
    Add CPU for more intensive queries.

    View Slide

  82. Scaling IO.
    Provision throughput for your app.
    Reserved. Available. Elastic.
    Available on EBS, Amazon RDS and DynamoDB.

    View Slide

  83. Provisioned throughput is consistent.
    Consistent, predictable performance.
    Relational databases with RDS.
    NoSQL stores with DynamoDB.
    Relational & NoSQL with EC2 and EBS.

    View Slide

  84. Provisioned throughput with RDS.
    12.5k IOPS on MySQL.
    25k IOPS on Oracle.
    10k IOPS on SQL Server.
    Provision up to 30k for reduced latency.

    View Slide

  85. Provisioned throughput and instance types.
    Optimized for provisioned IO storage:
    m1.large: 500 Mbps
    m1.xlarge, m2.xlarge, m2.4xlarge: 1000 Mbps

    View Slide

  86. Provisioned throughput and DynamoDB.
    Consistent performance with
    unlimited throughput and storage.
    Single digital latency.

    View Slide

  87. Build for a uniform workload.
    Evenly distribute query patterns by key.
    Use a wide range of key values.

    View Slide

  88. 7
    6
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4 Caching
    5 Read capacity

    View Slide

  89. 7
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4 Caching
    5 Read capacity
    6

    View Slide

  90. Standard EBS volumes.
    Moderate or bursty workloads.
    100 IOPS, bursting to hundreds of IOPS.
    Bursting is good for boot volumes.

    View Slide

  91. Provisioned IOPS with EBS volumes.
    Predictable, high performance for IO intensive workloads.
    2000 IOPS per volume.
    Stripe volumes for additional IO.
    Deliver within 10% of the performance, 99.9% of the time.

    View Slide

  92. EBS-Optimized instances.
    m1.large, m2.2xlarge, m3.xlarge: 500 Mbps
    m1.xlarge, m2.4xlarge, m3.2xlarge, c1.xlarge: 1000 Mbps

    View Slide

  93. High bandwidth networking
    cg1.4xlarge, cc2.8xlarge, hi1.4xlarge, hs1.8xlarge and
    cr1.8xlarge run on non-blocking, 10 gigabit networking.
    Not EBS-Optimized, but can be used with
    provisioned IOPS volumes.

    View Slide

  94. High I/O instances
    Designed for for high throughput database workloads.
    2 x 1TB SSDs
    2 GB/s for reads
    1.1 GB/s for writes

    View Slide

  95. Para-virtual instances
    4kb random reads: 120k IOPS
    4kb random writes: 10k - 80k IOPS

    View Slide

  96. HVM instances (including Windows)
    4kb random reads: 90k IOPS
    4kb random writes: 9k - 75k IOPS

    View Slide

  97. High Storage instances
    High sequential IO.
    24 x 2TB drives.
    2.4 GB/s of 2MiB sequential reads.
    2.6 GB/s for sequential writes.

    View Slide

  98. 7
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4 Caching
    5 Read capacity
    6
    Block store

    View Slide

  99. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    Back end Front end

    View Slide

  100. http://amzn.to/170S1VV http://amzn.to/Zod4PY

    View Slide

  101. 1. Make fewer HTTP requests.
    2. Use a content delivery network.
    3. Add an expires header.
    4. GZIP components.
    5. Put style sheets at the top.
    6. Put scripts at the bottom.
    7. Avoid CSS expressions.
    8. Make JavaScript and CSS External.
    9. Reduce DNS lookups.
    10. Minify Javascript.
    11. Avoid redirects.
    12. Remove duplicate scripts.
    13. Configure ETags.
    14. Make AJAX cacheable.
    http://stevesouders.com/hpws/rules.php
    14 rules for faster loading web sites

    View Slide

  102. 7
    1
    3
    4
    6
    5
    Content delivery
    Instance selection
    Caching
    Read capacity
    Front end optimization
    2 Horizontal scale
    Block store

    View Slide

  103. One more thing...

    View Slide

  104. US East

    View Slide

  105. US East
    Sydney Dublin
    Route 53

    View Slide

  106. Thank you!
    [email protected]
    @mza

    View Slide