Latency's Worst Nightmare: Performance Tuning Tips and Tricks

Delivering high performance web applications, using Amazon Web Services.

Transcript

1. Latency’s Worst Nightmare:
   Performance Tuning Tips and Tricks
   Matt Wood
   Principal Data Scientist
   @mza
   

   View Slide

2. Hello.
   

   View Slide

3. Let’s talk about performance...
   

   View Slide

4. Figure 3 Interactive user productivity versus computer response time for human-intensive
   interactions for system A
   E 600
   3
   -
   T
   7
   w
   z
   E 500
   -
   U
   E
   w
   E
   -
   >
   >
   -
   -
   400
   -
   3
   n
   F
   2
   300
   -
   200
   -
   100
   -
   0 -
   0
   -" INTERACTIVE USER PRODUCTIVITY (IUP)
   - HUMAN-INTENSIVE COMPONENT OF IUP
   A
   MEASURED DATA (HUMAN-INTENSIVE
   " COMPONENT)
   0
   0
   0
   0
   I 1 I I I
   1 2 3 4 5
   COMPUTER RESPONSE TIME (SI
   A. J. Thadhani, IBM Systems Journal 20 (4), 1981
   Productivity and response time
   

   View Slide

5. -0.70%
   -0.60%
   -0.50%
   -0.40%
   -0.30%
   -0.20%
   -0.10%
   0.00%
   50ms pre-
   header
   100ms pre-
   header
   200ms post-
   header
   200ms post-
   ads
   400ms post-
   header
   Page load time and average daily searches per user
   http://www.webperformancetoday.com/2013/04/10/cloud-connect-2013-web-acceleration-and-front-end-optimization-slides/
   

   View Slide

6. -5.00%
   -4.50%
   -4.00%
   -3.50%
   -3.00%
   -2.50%
   -2.00%
   -1.50%
   -1.00%
   -0.50%
   0.00%
   50 200 500 1000 2000
   Percent change
   Added delay
   Queries per visitor Query refinement Revenue per visitor
   Any clicks Satisfaction
   Page load delay and business metrics
   http://www.webperformancetoday.com/2013/04/10/cloud-connect-2013-web-acceleration-and-front-end-optimization-slides/
   

   View Slide

7. 2.2s 15.4%
   reduction in page load time increase in conversion rate
   https://blog.mozilla.org/metrics/2010/04/05/firefox-page-load-speed-%E2%80%93-part-ii/
   

   View Slide

8. “Speed is more than a feature.”
   @fredwilson
   

   View Slide

9. Let’s talk about a web request...
   

   View Slide

10. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    

    View Slide

11. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    Remote
    

    View Slide

12. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    Remote Browser
    

    View Slide

13. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    Remote Browser
    

    View Slide

14. http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/
    

    View Slide

15. View Slide

16. View Slide

17. Let’s talk about a web request...
    

    View Slide

18. Web server
    Application logic
    Database
    

    View Slide

19. Web server
    Application logic
    Database
    

    View Slide

20. Web server
    Application logic
    Database
    

    View Slide

21. Web server
    Application logic
    Database
    

    View Slide

22. Web server
    Application logic
    Database
    

    View Slide

23. Web server
    Application logic
    Database
    

    View Slide

24. Web server
    Application logic
    Database
    

    View Slide

25. Web server
    Application logic
    Database
    

    View Slide

26. Web server
    Application logic
    Database
    3
    4
    6
    5
    7
    2
    1
    

    View Slide

27. Web server
    Application logic
    Database
    3
    4
    6
    5
    7
    2
    1
    

    View Slide

28. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    TTFB
    

    View Slide

29. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    Network latency Download + negotiation time
    

    View Slide

30. View Slide

31. Variable TTFB based on geographic location.
    NYC London Sydney
    

    View Slide

32. Reduce internet induced latency.
    CloudFront content delivery network.
    Lower latency. Faster downloads.
    

    View Slide

33. Reduce internet induced latency.
    CloudFront content delivery network.
    Lower latency. Faster downloads.
    

    View Slide

34. Europe
    Amsterdam (2)
    Dublin
    Frankfurt (2)
    London (2)
    Madrid
    Milan
    Paris (2)
    Stockholm
    South America
    Sao Paulo
    North America
    Ashburn, VA (2)
    Dallas, TX (2)
    Hayward, CA
    Jacksonville, FL
    Los Angeles, CA (2)
    Miami, FL
    Newark, NJ
    New York, NY (3)
    Palo Alto, CA
    Seattle, WA
    San Jose, CA
    South Bend, IN
    St. Louis, MO
    CloudFront Edge Locations
    

    View Slide

35. Static and dynamic content
    Cache dynamic pages (search results).
    Use query strings or cookie for cache keys.
    Network and Path optimizations accelerate
    even unique content.
    

    View Slide

36. Web server
    Application logic
    Database
    3
    4
    6
    5
    7
    2
    1 Content delivery
    

    View Slide

37. 7
    Web server
    Application logic
    Database
    3
    4
    6
    5
    1
    2
    Content delivery
    

    View Slide

38. Web server
    Application server
    Database
    

    View Slide

39. Web server
    Application server
    Database
    

    View Slide

40. Application server
    Database
    

    View Slide

41. Application server
    Thread safe?
    Application state?
    Database
    

    View Slide

42. Web server
    Application server
    Database
    

    View Slide

43. Web server
    Application server
    Database
    Unit of scale
    

    View Slide

44. Web server
    Application server
    Database
    Load balancer
    

    View Slide

45. Web server
    Application server
    Database
    Web server
    Application server
    Web server
    Application server
    Load balancer
    

    View Slide

46. Load balancer
    Web server
    Application server
    Database
    Web server
    Application server
    Web server
    Application server
    

    View Slide

47. Load balancer
    Web server
    Application server
    Database
    Web server
    Application server
    Web server
    Application server
    

    View Slide

48. Decouple your service tiers
    Separation of concerns.
    Easier to manage.
    Drives higher availability.
    

    View Slide

49. Build for horizontal scale
    Decrease request contention.
    Reduce capacity planning headaches.
    Requires a stateless application architecture.
    

    View Slide

50. Small things, loosely coupled.
    Do one thing, and do it well. The Unix Way.
    Take a look at the Unicorn and Rainbows approach.
    Asynchronous be default (where possible).
    

    View Slide

51. Reduce response time.
    Concurrency limits can reappear quickly.
    Limit impact on performance through rapid scaling.
    

    View Slide

52. Load balancer
    

    View Slide

53. Load balancer
    

    View Slide

54. Load balancer
    

    View Slide

55. Fast booting with EBS-backed instances.
    Linux is faster to boot than Windows.
    EBS-backed instances are faster than S3 backed.
    

    View Slide

56. Pre-baked EBS-backed AMIs.
    Each code deployment creates a new AMI.
    AMI is the unit of deployment.
    

    View Slide

57. Automate response with Auto Scaling.
    Set operational thresholds.
    Faster than manual response (especially at 3am).
    

    View Slide

58. Time-based response with OpsWorks.
    Set operational times.
    ‘Follow the sun’ response.
    

    View Slide

59. Pre-emptive scaling.
    Contact your account manager, or
    get in touch via a Premium Support ticket.
    

    View Slide

60. Reserve capacity.
    Lower costs.
    Guaranteed availability.
    

    View Slide

61. 7
    Database
    3
    4
    6
    5
    1
    2
    Content delivery
    Horizontal scale
    

    View Slide

62. 7
    Database
    4
    6
    5
    1
    2
    Content delivery
    Horizontal scale
    3
    

    View Slide

63. A range of instance types
    Full spectrum of price/performance options.
    

    View Slide

64. Choosing the right instance type.
    Application specific.
    Benchmark.
    CloudWatch metrics.
    

    View Slide

65. Benchmark on business metrics.
    Relate application metrics to business metrics.
    Customers supported/instance.
    Photos processed/dollar.
    

    View Slide

66. The Canary in the Coal Mine
    Standardize on 64-bit AMIs.
    Deploy across instance types.
    Evaluate new instance types with real traffic.
    

    View Slide

67. View Slide

68. View Slide

69. View Slide

70. 7
    Database
    4
    6
    5
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    

    View Slide

71. 7
    Database
    6
    5
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4
    

    View Slide

72. Interface with the data store.
    Faster if you don’t have to go to disk.
    Increased concurrency.
    

    View Slide

73. Caching.
    Store query results in memory.
    Writes go to disk.
    

    View Slide

74. Amazon ElastiCache.
    Deploy, operate and scale in-memory caches.
    

    View Slide

75. Managing state.
    Transient data only.
    Web server state, high score tables, etc.
    Time consuming task results
    (many to many query results).
    

    View Slide

76. Best practices.
    Assume cold cache latency in application architecture.
    Set appropriate time-to-live.
    Batch requests rather than sequential single.
    Architect for cache failure.
    

    View Slide

77. 7
    Database
    6
    5
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4 Caching
    

    View Slide

78. 7
    Database
    6
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4 Caching
    5
    

    View Slide

79. Accelerating reads.
    Vertical scale vs horizontal scale.
    

    View Slide

80. Horizontal scale.
    Add additional DB resources for scale.
    Scale out for reads.
    Shard for reads and writes.
    

    View Slide

81. Vertical scale.
    More resources for a single DB engine.
    Add memory for DB caches.
    Add CPU for more intensive queries.
    

    View Slide

82. Scaling IO.
    Provision throughput for your app.
    Reserved. Available. Elastic.
    Available on EBS, Amazon RDS and DynamoDB.
    

    View Slide

83. Provisioned throughput is consistent.
    Consistent, predictable performance.
    Relational databases with RDS.
    NoSQL stores with DynamoDB.
    Relational & NoSQL with EC2 and EBS.
    

    View Slide

84. Provisioned throughput with RDS.
    12.5k IOPS on MySQL.
    25k IOPS on Oracle.
    10k IOPS on SQL Server.
    Provision up to 30k for reduced latency.
    

    View Slide

85. Provisioned throughput and instance types.
    Optimized for provisioned IO storage:
    m1.large: 500 Mbps
    m1.xlarge, m2.xlarge, m2.4xlarge: 1000 Mbps
    

    View Slide

86. Provisioned throughput and DynamoDB.
    Consistent performance with
    unlimited throughput and storage.
    Single digital latency.
    

    View Slide

87. Build for a uniform workload.
    Evenly distribute query patterns by key.
    Use a wide range of key values.
    

    View Slide

88. 7
    6
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4 Caching
    5 Read capacity
    

    View Slide

89. 7
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4 Caching
    5 Read capacity
    6
    

    View Slide

90. Standard EBS volumes.
    Moderate or bursty workloads.
    100 IOPS, bursting to hundreds of IOPS.
    Bursting is good for boot volumes.
    

    View Slide

91. Provisioned IOPS with EBS volumes.
    Predictable, high performance for IO intensive workloads.
    2000 IOPS per volume.
    Stripe volumes for additional IO.
    Deliver within 10% of the performance, 99.9% of the time.
    

    View Slide

92. EBS-Optimized instances.
    m1.large, m2.2xlarge, m3.xlarge: 500 Mbps
    m1.xlarge, m2.4xlarge, m3.2xlarge, c1.xlarge: 1000 Mbps
    

    View Slide

93. High bandwidth networking
    cg1.4xlarge, cc2.8xlarge, hi1.4xlarge, hs1.8xlarge and
    cr1.8xlarge run on non-blocking, 10 gigabit networking.
    Not EBS-Optimized, but can be used with
    provisioned IOPS volumes.
    

    View Slide

94. High I/O instances
    Designed for for high throughput database workloads.
    2 x 1TB SSDs
    2 GB/s for reads
    1.1 GB/s for writes
    

    View Slide

95. Para-virtual instances
    4kb random reads: 120k IOPS
    4kb random writes: 10k - 80k IOPS
    

    View Slide

96. HVM instances (including Windows)
    4kb random reads: 90k IOPS
    4kb random writes: 9k - 75k IOPS
    

    View Slide

97. High Storage instances
    High sequential IO.
    24 x 2TB drives.
    2.4 GB/s of 2MiB sequential reads.
    2.6 GB/s for sequential writes.
    

    View Slide

98. 7
    1
    2
    Content delivery
    Horizontal scale
    3 Instance selection
    4 Caching
    5 Read capacity
    6
    Block store
    

    View Slide

99. Initial connection SSL negotiation Time to first byte Content download
    %CPU
    kbps
    Back end Front end
    

    View Slide

100. http://amzn.to/170S1VV http://amzn.to/Zod4PY
     

     View Slide

101. 1. Make fewer HTTP requests.
     2. Use a content delivery network.
     3. Add an expires header.
     4. GZIP components.
     5. Put style sheets at the top.
     6. Put scripts at the bottom.
     7. Avoid CSS expressions.
     8. Make JavaScript and CSS External.
     9. Reduce DNS lookups.
     10. Minify Javascript.
     11. Avoid redirects.
     12. Remove duplicate scripts.
     13. Configure ETags.
     14. Make AJAX cacheable.
     http://stevesouders.com/hpws/rules.php
     14 rules for faster loading web sites
     

     View Slide

102. 7
     1
     3
     4
     6
     5
     Content delivery
     Instance selection
     Caching
     Read capacity
     Front end optimization
     2 Horizontal scale
     Block store
     

     View Slide

103. One more thing...
     

     View Slide

104. US East
     

     View Slide

105. US East
     Sydney Dublin
     Route 53
     

     View Slide

106. Thank you!
     matthew@amazon.com
     @mza
     

     View Slide