Save 37% off PRO during our Black Friday Sale! »

Performance Testing Modern Apps

6d9b2c900628962b16a5cb5e4e73990e?s=47 Dustin Whittle
September 09, 2015

Performance Testing Modern Apps

Performance Testing Modern Apps presented at DjangoCon 2015

6d9b2c900628962b16a5cb5e4e73990e?s=128

Dustin Whittle

September 09, 2015
Tweet

Transcript

  1. Performance Tes-ng Modern Apps Dus$n Whi*le

  2. None
  3. Dus-n Whi8le • dus-nwhi8le.com • @dus-nwhi8le • San Francisco, California,

    USA • Technologist, Traveler, Pilot, Skier, Diver, Sailor, Golfer
  4. Why does performance ma8er?

  5. MicrosoI found that Bing searches that were 2 seconds slower

    resulted in a 4.3% drop in revenue per user
  6. When Mozilla shaved 2.2 seconds off the landing page, Firefox

    downloads increased 15.4%
  7. Making Barack Obama’s website 60% faster increased dona-on conversions by

    14%
  8. None
  9. Performance directly impacts the bo8om line

  10. None
  11. How fast is fast enough? § Performance is key to

    a great user experience - Under 100ms is perceived as reac$ng instantaneously - A 100ms to 300ms delay is percep$ble - 1 second is about the limit for the user's flow of thought to stay uninterrupted - Users expect a site to load in 2 seconds - AIer 3 seconds, 40% will abandon your site. - 10 seconds is about the limit for keeping the user's a*en$on § Modern applica-ons spend more -me in the browser than on the server-side
  12. Login Flight Status Search Flight Purchase Mobile Big data SOA

    NOSQL Cloud Agile Web Application complexity is exploding
  13. Up-me is cri-cal for enterprises and consumers

  14. Treat performance as a feature!

  15. Tools of the trade for performance tes-ng

  16. Understand your baseline performance

  17. Sta-c vs Hello World vs Applica-ons

  18. Apache Bench

  19. ab -c 1 -t 10 -k http://acmedemoapp.com/

  20. Benchmarking acmedemoapp.com (be patient) Finished 187 requests Server Software: Apache-Coyote/1.1

    Server Hostname: acmedemoapp.com Server Port: 80 Document Path: / Document Length: 5217 bytes Concurrency Level: 1 Time taken for tests: 10.039 seconds Complete requests: 187 Failed requests: 0 Keep-Alive requests: 187 Total transferred: 1021768 bytes HTML transferred: 975579 bytes Requests per second: 18.63 [#/sec] (mean) Time per request: 53.687 [ms] (mean)
  21. ab -c 10 -t 10 -k http://acmedemoapp.com/

  22. Benchmarking acmedemoapp.com (be patient) Finished 659 requests Server Software: Apache-Coyote/1.1

    Server Hostname: acmedemoapp.com Server Port: 80 Document Path: / Document Length: 5217 bytes Concurrency Level: 10 Time taken for tests: 10.015 seconds Complete requests: 659 Failed requests: 0 Keep-Alive requests: 659 Total transferred: 3600776 bytes HTML transferred: 3438003 bytes Requests per second: 65.80 [#/sec] (mean) Time per request: 151.970 [ms] (mean) Time per request: 15.197 [ms] (mean, across all
  23. Siege

  24. siege -c 10 -b -t 10S http://acmedemoapp.com/

  25. ** SIEGE 3.0.6 ** Preparing 10 concurrent users for battle.

    The server is now under siege... Lifting the server siege... done. Transactions: 623 hits Availability: 100.00 % Elapsed time: 9.57 secs Data transferred: 3.10 MB Response time: 0.15 secs Transaction rate: 65.10 trans/sec Throughput: 0.32 MB/sec Concurrency: 9.91 Successful transactions: 623 Failed transactions: 0 Longest transaction: 0.30 Shortest transaction: 0.10
  26. Crawl the entire app to discover all urls

  27. sproxy -o ./urls.txt

  28. SPROXY v1.02 listening on port 9001 ...appending HTTP requests to:

    ./urls.txt ...default connection timeout: 120 seconds
  29. wget -r -l 0 -t 1 --spider -w 1 -e

    "http_proxy = http://127.0.0.1:9001" "http://acmedemoapp.com/" sort -u -o urls.txt urls.txt
  30. http://acmedemoapp.com/ http://acmedemoapp.com/about http://acmedemoapp.com/cart http://acmedemoapp.com/currency/change/EUR http://acmedemoapp.com/currency/change/GBP http://acmedemoapp.com/currency/change/USD http://acmedemoapp.com/login http://acmedemoapp.com/register/ http://acmedemoapp.com/t/brand/bookmania http://acmedemoapp.com/t/category/books

    http://acmedemoapp.com/t/category/mugs http://acmedemoapp.com/t/category/stickers http://acmedemoapp.com/terms-of-service
  31. Benchmark traffic across all unique urls with siege

  32. siege -v -c 50 -i -t 3M -f urls.txt -d

    10
  33. ** SIEGE 3.0.6 ** Preparing 10 concurrent users for battle.

    The server is now under siege... Lifting the server siege... done. Transactions: 623 hits Availability: 100.00 % Elapsed time: 9.57 secs Data transferred: 3.10 MB Response time: 0.15 secs Transaction rate: 65.10 trans/sec Throughput: 0.32 MB/sec Concurrency: 9.91 Successful transactions: 623 Failed transactions: 0 Longest transaction: 0.30 Shortest transaction: 0.10
  34. None
  35. Multi-Mechanize is an open source framework for performance and load

    testing
  36. pip install multi-mechanize

  37. multimech-newproject demo

  38. import requests class Transaction(object): def run(self): r = requests.get(‘http://acmedemoapp.com/) r.raw.read()

  39. import mechanize import time class Transaction(object): def run(self): br =

    mechanize.Browser() br.set_handle_robots(False) start_timer = time.time() resp = br.open(‘http://acmedemoapp.com/) resp.read() latency = time.time() - start_timer self.custom_timers['homepage'] = latency start_timer = time.time() resp = br.open(‘http://acmedemoapp.com/cart') resp.read() latency = time.time() - start_timer self.custom_timers['cart'] = latency assert (resp.code == 200)
  40. multimech-run demo

  41. None
  42. What about when you need more than one machine?

  43. Who lives in the cloud?

  44. Bees with Machine Guns

  45. A utility for arming (creating)  many bees (micro EC2

    instances) to attack (load test) targets (web applications)
  46. pip install beeswithmachineguns

  47. # ~/.boto [Credentials] aws_access_key_id=xxx aws_secret_access_key=xxx [Boto] ec2_region_name = us-west-2 ec2_region_endpoint

    = ec2.us-west-2.amazonaws.com
  48. bees up -s 2 -g default -z us-west-2b -i ami-bc05898c

    -k appdynamics- dustinwhittle-aws-us-west-2 -l ec2-user
  49. Connecting to the hive. Attempting to call up 2 bees.

    Waiting for bees to load their machine guns... . . . . Bee i-3828400c is ready for the attack. Bee i-3928400d is ready for the attack. The swarm has assembled 2 bees.
  50. bees report

  51. Read 2 bees from the roster. Bee i-3828400c: running @

    54.212.22.176 Bee i-3928400d: running @ 50.112.6.191
  52. bees attack -n 1000 -c 50 -u http://acmedemoapp.com/

  53. Read 2 bees from the roster. Connecting to the hive.

    Assembling bees. Each of 2 bees will fire 500 rounds, 25 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. … Offensive complete. Complete requests: 1000 Requests per second: 306.540000 [#/sec] (mean) Time per request: 163.112000 [ms] (mean) 50% response time: 151.000000 [ms] (mean) 90% response time: 192.000000 [ms] (mean) Mission Assessment: Target crushed bee offensive. The swarm is awaiting new orders.
  54. bees attack -n 100000 -c 1000 -u http://acmedemoapp.com/

  55. Read 2 bees from the roster. Connecting to the hive.

    Assembling bees. Each of 2 bees will fire 50000 rounds, 500 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. … Offensive complete. Complete requests: 100000 Requests per second: 502.420000 [#/sec] (mean) Time per request: 360.114000 [ms] (mean) 50% response time: 451.000000 [ms] (mean) 90% response time: 402.000000 [ms] (mean) Mission Assessment: Target crushed bee offensive. The swarm is awaiting new orders.
  56. Read 2 bees from the roster. Connecting to the hive.

    Assembling bees. Each of 2 bees will fire 50000 rounds, 500 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. Bee 0 is joining the swarm. Bee 1 is joining the swarm. Bee 0 is firing his machine gun. Bang bang! Bee 0 lost sight of the target (connection timed out). Bee 1 lost sight of the target (connection timed out). Offensive complete. Target timed out without fully responding to 2 bees. No bees completed the mission. Apparently your bees are peace-loving hippies. The swarm is awaiting new orders.
  57. bees down

  58. Read 2 bees from the roster. Connecting to the hive.

    Calling off the swarm. Stood down 2 bees.
  59. locust.io https://github.com/locustio/locust

  60. pip install locustio

  61. None
  62. None
  63. None
  64. There are many tools… •Gatling.io •Wrk •Vegeta •Tsung •…

  65. What about the client side?

  66. In modern web applications more latency comes from the client-side

    than the server-side.
  67. None
  68. None
  69. None
  70. None
  71. npm install psi

  72. None
  73. curl "https://www.googleapis.com/ pagespeedonline/v1/runPagespeed? url=http://dustinwhittle.com/&key=xxx"

  74. WBench

  75. gem install wbench

  76. wbench http://dustinwhittle.com/

  77. None
  78. Automate client-side performance tes-ng with Grunt/Gulp

  79. Use Bower (for dependencies), Grunt/Gulp (for automa-on), and Yeoman (for

    bootstrapping)
  80. None
  81. How many people understand exactly how fast their site runs

    in produc-on?
  82. Track performance in development and produc-on

  83. Instrument everything = code, databases, caches, queues, third party services,

    and infrastructure.
  84. None
  85. Chef / Sensu

  86. Statsd + Graphite + Grafana

  87. None
  88. Track performance of end users

  89. None
  90. webpagetest.org

  91. None
  92. None
  93. None
  94. SiteSpeed.io

  95. None
  96. None
  97. None
  98. None
  99. None
  100. None
  101. None
  102. None
  103. Load tes-ng services from the cloud

  104. None
  105. None
  106. None
  107. Test for failures • NetFlix Simian Army + Chaos Monkey


    • What happens if you lose a caching layer? • What happens if dependencies slow down?
  108. Performance Ma*ers • Treat performance as a feature • Using

    the 14kb Rule for instant loading • Markup management • Elimina$ng excess AJAX calls • Working with and around applica$on cache • Developing a responsive design + image strategy • Implemen$ng a good touch-first strategy • Code management for good produc$on and development experiences • Using task runners to build and deploy produc$on code
  109. Best Prac$ces • Treat performance as a feature • Capacity

    plan and load test the server-side • Op-mize and performance test the client-side • Understand your star-ng point • Instrument everything • Monitor performance in development and produc-on • Measure the difference of every change • Automate performance tes-ng in your build and deployment process • Understand how failures impact performance
  110. The protocols are evolving • The limita$ons of HTTP/1.X forced

    us to develop various applica$on workarounds (sharding, concatena$on, spri$ng, inlining, etc.) to op$mize performance. However, in the process we’ve also introduced numerous regressions: poor caching, unnecessary downloads, delayed execu$on, and more. • HTTP/2 eliminates the need for these hacks and allows us to both simplify our applica$ons and deliver improved performance. • You should unshard, unconcat, and unsprite your assets • You should switch from inlining to server push • Read Ilya Grigorik awesome book on browser performance - h*p://hpbn.co/ h*p2
  111. Integrate automated performance tes-ng into con-nuous integra-on for server-side and

    client-side
  112. Understand the performance implica-ons of every deployment and package upgrade

  113. Monitor end user experience from end to end in produc-on

  114. Ques-ons?

  115. Find these slides on SpeakerDeck h8ps://speakerdeck.com/dus-nwhi8le

  116. http://www.appdynamics.com/