Siege and Beyond: An Intro to Benchmarking and Stress Tests

Benchmarking is an important practice for high-traffic sites, or even low traffic sites that may have high-traffic spikes.

What will happen to your site if it hits the Reddit front page? Without benchmarking and stress testing, there’s no good way to know how your site will perform under true load.

In this session, you’ll learn some of the basics of benchmarking, some good tools to get started, and when stress tests are the right tool for your site’s performance concerns. Finally, you’ll learn some common performance problems found through stress tests, and their solutions.

Presented at WordCamp Vancouver 2015 by Michael Schroder
http://www.getsource.net/2015/08/stress-testing-wcyvr

Transcript

1. Siege and Beyond:

   An Intro to Benchmarking and Stress Tests
   WordCamp Vancouver 2015
   Mike Schroder (DH-Shredder)
   WordPress Platform Lead at DreamHost
   @GetSource - http://www.getsource.net
   

   View Slide

2. Who Am I?
   • Mike Schroder, a.k.a DH-Shredder,

   a.k.a. @GetSource
   • Third Culture Kid, enjoy Coffee & Sailing
   • WordPress 3.9 Co-Lead and Core Contributor
   • Happy DreamHost Employee
   

   View Slide

3. What will happen if my site
   hits the reddit front page?
   

   View Slide

4. What if reddit doesn’t matter to me?
   

   View Slide

5. Prove how many users can access
   your site at one time.
   

   View Slide

6. Load Testing or Stress Testing?
   

   View Slide

7. Load Tests: Will my site run well with
   the expected amount of users?
   

   View Slide

8. Stress Tests: How many users before
   the site/infrastructure cracks?
   

   View Slide

9. Are we testing dynamic or cached requests?
   

   View Slide

10. INTERNET
    PHP
    FRONT-END CACHE
    

    View Slide

11. Varnish/Static: ~50ms
    

    View Slide

12. PHP: ~100ms
    (without stress)
    

    View Slide

13. PHP: ~1500ms
    (with stress)
    

    View Slide

14. All methods are not equal.
    

    View Slide

15. “Users/minute” does not necessarily
    mean “real users during a minute”.
    

    View Slide

16. There are many utilities for stress testing.
    

    View Slide

17. Let’s start with Siege.
    

    View Slide

18. You can run cached or dynamic tests.
    

    View Slide

19. It’s used only from the command line.
    

    View Slide

20. It only runs tests from one server.
    

    View Slide

21. Linux: Packages.

    Mac: Homebrew.
    https://www.joedog.org/siege-­‐home/
    

    View Slide

22. Primary configuration (.siegerc)
    

    View Slide

23. To run dynamic tests
    login-url = http://example.com/wp-login.php POST
    log=[user]&pwd=[password]&wp-submit=Log+In&
    redirect_to=http%3A%2F%2example.com%2Fwp-admin%2F&testcookie=1
    

    View Slide

24. A list of URLs.
    

    View Slide

25. List format
    HOST=http://example.com
    ${HOST}/
    ${HOST}/?feed=rss2
    ${HOST}/?p=41
    ${HOST}/?feed=rss2
    ${HOST}/?p=44
    ${HOST}/
    ...
    

    View Slide

26. Results
    Transactions: 712 hits
    Availability: 100.00 %
    Elapsed time: 59.12 secs
    Data transferred: 3.70 MB
    Response time: 1.23 secs
    Transaction rate: 12.04 trans/sec
    Throughput: 0.06 MB/sec
    Concurrency: 14.75
    Successful transactions: 702
    Failed transactions: 0
    Longest transaction: 8.49
    Shortest transaction: 0.20
    

    View Slide

27. Where can I find test content?
    

    View Slide

28. HHVM’s oss-performance is one option.
    https://github.com/hhvm/oss-­‐performance/tree/master/targets/wordpress
    

    View Slide

29. To seed from your own content, take a
    look at your logs, or try:
    http://example.com/  
    http://example.com/?feed=rss2  
    http://example.com/?p=%{*:1-­‐52}  
    

    View Slide

30. You can also use sproxy,
    not to be confused with stunnel.
    

    View Slide

31. Loader.io is a service to run stress tests.
    

    View Slide

32. However, it has a friendly GUI to run
    tests and see the results.
    

    View Slide

33. View Slide

34. Single data center, but from a
    cluster of AWS servers.
    

    View Slide

35. More complex user behavior?
    Try a service like LoadStorm.
    

    View Slide

36. What kinds of issues will I find?
    

    View Slide

37. How do I debug problems once
    they’re found?
    

    View Slide

38. Bonus: Your WordPress site runs slowly,
    and even more slowly as soon as you
    enable a persistent object-cache.
    Why?
    

    View Slide

39. alloptions.
    

    View Slide

40. Memcached slab size.
    

    View Slide

41. wp_options cleanup.
    

    View Slide

42. Questions?
    -­‐    getsource.net/tag/siege/  
    -­‐    joedog.org/siege-­‐manual/  
    -­‐    joedog.org/sproxy-­‐home/

    -­‐    github.com/hhvm/oss-­‐performance

    Mike Schroder (DH-Shredder)
    WordPress Platform Lead at DreamHost
    @GetSource - http://www.getsource.net
    

    View Slide