Agile performance testing

Agile performance testing with Amazon AWS, tmux and siege

Transcript

1. Understanding your system Andreas Bjärlestam! 2016-03-30! _____ .__.__ / _ \ ____ |__| | ____ / /_\ \ / ___\| | | _/ __ \ / | \/ /_/ > | |_\ ___/ \____|__ /\___ /|__|____/\___ > \//_____/ \/ _____ ______ ____________/ ____\___________ _____ _____ ____ ____ ____ \____ \_/ __ \_ __ \ __\/ _ \_ __ \/ \\__ \ / \_/ ___\/ __ \ | |_> > ___/| | \/| | ( <_> ) | \/ Y Y \/ __ \| | \ \__\ ___/ | __/ \___ >__| |__| \____/|__| |__|_| (____ /___| /\___ >___ > |__| \/ \/ \/ \/ \/ \/ __ __ .__ _/ |_ ____ _______/ |_|__| ____ ____ \ __\/ __ \ / ___/\ __\ |/ \ / ___\ | | \ ___/ \___ \ | | | | | \/ /_/ > |__| \___ >____ > |__| |__|___| /\___ / \/ \/ \//_____/
2. Do you know on top of your head: •  How many req/s your system can handle? •  What response >me it has? •  How it scales? •  What the bo@lenecks are? •  How stable it is over >me?
3. Many s>ll do big bang performance tes>ng
4. Stop seeing performance tests as verifica>on It should be an integrated part of your development cycle Con>nuously analyze your system
5. You should do it all the >me!
6. I’m lazy, so performance tes>ng must be quick and simple
7. isola>on
8. Your test client should do nothing but loadtes>ng, no interference
9. Amazon AWS + tmux = win!
10. You can leave it on and come back to check every now and then
11. Combine with monitoring: Newrelic Kibana Graphite
12. > sudo yum install siege
13. > siege -c10 -t30s -d1 -i -f urls.txt
14. siege •  Quick and simple •  Instant visual feedback •  Good summary of most important metrics •  Good enough for most scenarios = You can work in quick itera>ons
15. Create a Traffic Model
16. Your best guess of how the system will be used
17. When replacing a system Get access logs from the old system Analyze which parts of the system that generate the most load Plot them over >me to get a feeling for peaks and average load
18. If you replay access logs against your system with siege you can gain a lot of insights and find problems like unhandled urls, unnecessary redirects etc
19. Build a urls.txt file Based on traffic model Fill a file with urls that represent your expected traffic, one url per line Think about the propor>ons of different types of urls
20. Build a urls.txt file Expec>ng broad traffic -> many urls Expec>ng narrow traffic -> not so many
21. Build a urls.txt file h@p://example.com h@p://example.com/user/s>na h@p://example.com/user/olof h@p://example.com/user/sven h@p://example.com/user/siv h@p://example.com/user/ellen h@p://example.com/country/sweden h@p://example.com/country/norway
22. Build a urls.txt file curl cut jq grep etc are your friends
23. Build a urls.txt file You can send POST as well h@p://example.com/user/s>na POST age=23 h@p://example.com/user/s>na POST a=1&b=2 h@p://example.com/user/s>na POST <./s>na.txt
24. Finding the system limits
25. 10 req/s OK 100 req/s OK 500 req/s Slooow 300 req/s OK
26. Scale up with more CPU or processors and try again
27. Does it scale linearly?
28. This is a good >me to think about the bo@lenecks of your system I/O CPU Sync / Async
29. Adjust and try again
30. Keep an eye on system metrics Response >me CPU load Memory usage Error rates etc
31. Stability tes>ng
32. Start a linux machine on AWS Set up a session with tmux Start siege Leave it running
33. > siege -c10 -t24h -i -d1 -f urls.txt
34. Put your system under con>nuous load from your first commit Keep an eye on system metrics every now and then
35. Good to know: siege counts error responses and will stop if it encounters more than 1024 errors
36. TLDR Stop doing performance tests like its 1999 Put your system under load from day 1 Run tests interac>vely, be crea>ve Gain understanding AWS + tmux + siege is awesome!
37. tmux cheat sheet > tmux new -s loadtest starts a new session named loadtest ctrl-b + d exits session > tmux list-sessions lists all current sessions > tmux a@ach -t loadtest a@aches to the session named loadtest
38. If siege does not fit your use case You could try - gatling - locust - wrk They are bigger (more complex) hammers