Loading time testing and results visualisation of web games @ SQA Days 22

Presentation from SQA Days conference about loading time testing and results visualisation of web games

Transcript

1. Loading time testing and results visualisation of web games Andrejs Kalnačs SQA Days St. Petersburg, 17th November, 2017
2. who am i 2 Lead Software Developer in Test @ Evolution Gaming 10 years of industry experience in QA Many different roles Wide business domains experience Last 6 years – testing automation Java, Groovy, Bash, Perl, Tcl Evolution Gaming, Nokia, Accenture, Nokia Siemens Networks Riga Testing Automation Club meet-ups co-organiser English, Russian and Latvian language proficiency @TesterFromRiga
3. Evolution Gaming 3 The World leader in live dealer gaming 3 development locations – Riga, Tallinn, Amsterdam B2B product company
4. Agenda 4 Web based loading time testing Practical guide in test data points visualisation
5. Use case of Evolution Gaming 5
6. Use case of Evolution Gaming 6 16 different games 3 test environments 2 browser cache states
7. Creation and execution of web based loading time testing
8. Browser automation using WebDriver API 8
9. Custom metric collection from client 9 Loading time Intermediate loading metrics Component/service loading metrics Proprietary API
10. Interception & archiving of HTTP transactions 10 Proxy System under test HTTP, HTTPS, WebSocket
11. OWASP ZAP proxy 11 HTTPS support WebSocket support API Open Source Cross platform (Linux, Mac OS X, Windows) HAR
12. HTTP archive format (HAR) 12 JSON Libraries Many viewers Storage solutions Diff solutions http://har.tech
13. Chrome 62 13 https://developers.google.com/web/updates/2017/08/devtools-release-notes#har-imports
14. Bandwidth throttling 14 Virtual or physical machine with two network interfaces Linux tc tcconfig tcset --device eth1 --rate 800k --direction outgoing tcset --device eth0 --rate 800k --direction incoming
15. Cloud? 15 Virtual hardware for desktop browsers Traffic throating only at Sauce Labs & AWS No HAR produced
16. Local desktop & device farm 16 Support, maintenance & update of infrastructure, hardware & software Stability of devices Complexity of iOS testing automation
17. Execution from CI server 17 Selenium Hub Appium iOS node Appium Android node Proxy Bandwidth throttler Docker host Docker host
18. Tests results 18 + Information provided by API Calculated & aggregated metrics from HAR
19. Results visualization
20. Why? 20 Show stake holders how good or bad user experience is Compare your products Visualise regression Understand how new feature is affecting performance
21. Ops & finance tools 21
22. Tests results visualisation 22
24. 24 Grafana
25. 25 Templating
26. 26 Annotations
27. 27 Alerts
28. 28 Demo https://youtu.be/SzelgA2NdEM
29. Try it out! 29 https://github.com/andrejska/simplest-test-dashboard
30. Key takeaways 30 Selenium can be used for performance tests Application it self can provide a lot of insides for testers Visualisation is excellent for displaying results in interactive, understandable and easily comparable way Mastering data visualisation tools is not a rocket science
31. Thank you! Any questions? andrejska Andrejs Kalnačs @TesterFromRiga