Performance Testing HTTP-based Systems @ LisbonJS #4

A talk about Performance Testing HTTP-based Systems at LisbonJS #4, 2017-01-26, https://www.meetup.com/de-DE/lisbonjs/events/236665601/

Transcript

1. Performance Testing HTTP-based Systems
2. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 EHLO[1] Lars Wolff 
 @larsvegas • Cologne, Germany • Development • #AWSUGCGN • #WEBMONTAG #WMCGN • #DevHouseFriday • Agile Hippie (Coach) / Certified ScrumMaster® • Co-Founder & CEO of
 stormforger.com [1] http://www.ietf.org/rfc/rfc2821.txt
3. We empower DevOps to do continuous performance testing
4. I assume you are building/running a website and/or an API.
5. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Who does NOT have performance issues? :)
6. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Who does NOT have performance issues? :) • Who does clearly understand and has insights
 to these issues? • Under which circumstances do they occur? • What is your traffic scenario?
7. Basics
8. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Basics Most of this part should be clear. But I learned that it’s helpful to recap it.
9. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 HTTP
 request - response cycle http://celineotter.azurewebsites.net/world-wide-web-http-request-response-cycle/
10. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 TCP 3-Way-Handshake https://lwn.net/Articles/508865/ https://media.ccc.de/v/froscon2015-1536-webperformance_autobahn_oder_fussgangerzone#video – bad sound :(
11. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Performance Testing • Is business critical
 (Better Conversion, More $$$) • Is orthogonal in your organisation
 (Business-, Product-, Test-, Development, Operations-People) • Results should always be available
 (Reports, Dashboards, Alerts, Insights) • Should be fully automated
 (Load Generator Setup, Metrics Analysis, Testing Environment, Testing Data)
12. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Performance Testing
 – Prerequisites • Goal? What do you want to learn? • Non-functional requirements? • Performance Budget? • Test and traffic scenario? • Environment to test against?
13. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 1. (Non-functional) requirements 2. Test definition 3. Test execution 4. Analysis of resulting data Flow
14. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Performance Testing
 – Types of Testing 1. Load Testing 2. Stress Testing 3. Scalability Testing 4. Spike Testing 5. Soak Testing 6. Configuration Testing 7. Availability & Resilience Testing https://stormforger.com/blog/2016/07/08/types-of-performance-testing/ https://docs.stormforger.com/guides/
15. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 User, Application and System Monitoring • Real User Monitoring (RUM) • How does the client perform for all users? • Application Performance Monitoring • What happens inside my application? • What kind of request takes how long? Why? • What kind of follow-up requests / queries does my application create?
 • System-Monitoring • What's going on in my internal network? • What's going on in my web server / app server machine? • What’s going on in my database cluster? • What's about storage and IOPS?
16. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Mind your organisation 1. Performance testing and analysis is team work! 2. Please involve: product people, developers, QA, operations 3. “Working software over comprehensive documentation”
 – but make your results transparent to every stage of your organisation!
17. Performance Testing Websites
18. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Website characteristics • multiple types of content
 (HTML, Images, CSS, JS, ...) • “client logic”
 (JavaScript, XHR) • complex client behavior
 (client type * user behavior * context) • “heavy”
19. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 #WebPerf Performance Testing / Load Testing ≠ #WebPerf
20. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 – https://www.w3.org/2010/webperf/ “[...] methods to measure and improve aspects of application performance of user agent features and APIs.” #WebPerf
21. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 #WebPerf 101 • less requests! (cache stuff) • less requests! (minify, concat) • less requests! (CDN for assets) • image compression / optimization ... • … a lot more … • how long does a tcp handshake take? :) • how much tcp connections does a (certain) browser open in parallel? :)
22. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Things you have to take care of • XHR & clients logic (JavaScript): A lot of client logic? Single page application (SPA)? • correct HTTP (error) response codes • Assets = bandwidth => CDN • Content extractions, e.g. CSRF token to submit form data • Double opt in • Test data in general
23. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 HTTP/1.1 vs. HTTP/2 • Some things change. • Some don’t. • What to focus on first? HTTP/2 Performance Anti-patterns by Ilya Grigorik
 https://docs.google.com/presentation/d/1_SMrVmiMxW2X1QZ1EcCnLKSosiD0PppP70Q3bw-l5Lg/present
24. Recap: Website
25. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Recap: Website 1. #WebPerf FTW 2. Seriously watch your bandwidth 3. Plenty other things to take care of 4. Iterate slowly and communicate with other people in your organization. 5. Never test external partners
 (tracking, advertisers, etc. etc.)
26. Performance Testing
 (RESTful) APIs
27. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 API characteristics • Usually XML or JSON responses • “Sequential” API flow, RESTful API flow, HATEOS / Hypermedia • Authenticated
 (authentication required or available auth tokens via test data) • No (browser) client, no client behavior – *sigh* :D • Requests from different clients (browser, mobile app, IoT, etc.) should be well-formed and ask for something really really specific • And... there is client logic (filters, paging, hypermedia)
28. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Things you have to take care of • Authentication (Basic Auth, SingleSignOn, oAuth, etc.) • HTTP headers (caching, ETags, gzip) • Header and content extractions ('id' to follow, 'auth-token' to use) • Test data in general • Correct HTTP (error) response codes • HTTP (error) codes in response body (response is HTTP200, content says 401 or “authentication required”) • RESTful (HTTP PUT vs. HTTP POST) • Auto follow of hypermedia links means you need a correct and full API specification :) • Rate limiting: Watch HTTP429 / too many requests, watch the fallback • One request per item vs. a request for a filterable batched result • Polling? Every minute? Caching fun. Don't send requests on a specific and fixed time... cron-style • No client handling / bad handling for HTTP5xx errors
29. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Advanced API performance testing
 – SOA & µService • Your environment is really distributed
 – monitoring and request tracking are must haves • Try to isolate each service and test it
 (firewall? proxy to get traffic into the data center...) • Try to combine test scenarios of services for a particular user journey and test their contracts
30. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Advanced API performance testing
 – Serverless (here: AWS Lambda) • AWS Lambda limits*
 – e.g. 100 concurrent invocations • AWS API Gateway limits*
 – e.g. request throttling 1000 requests per second (rps) http://docs.aws.amazon.com/lambda/latest/dg/limits.html https://stormforger.com/presentations/serverles-microservices-vs-monolitic-beanstalk-app-loadtest/ http://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html * ask AWS, they have knobs ;-)
31. Recap: APIs
32. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Recap: APIs 1. Authentication 2. API flow and user journey 3. There is “client logic“ 4. Iterate slowly and communicate with other people
33. Wrap-up
34. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up – How to do it? 1. Discuss user journey 2. Create minimal viable test case version 3. Test slowly 4. Iterate 5. Communicate 6. “Stale” your scenario 7. Run with more traffic for specific test type 8. Website: Use HAR to help you to get all
 the requests you need for your website
35. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Both is complex and there are plenty of things to take care of.
36. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Needs to involve people from all needed departments/professions in your organisation.
37. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Should be started early.
38. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Should be started early – really!
39. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Is nothing to get stuck in the
 “Not Invented Here Syndrome”! – There are a lot of tools out there! https://en.wikipedia.org/wiki/Not_invented_here
40. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Should be done early.
 Should be done often.
 Should be done continuously.
42. <sales>
 SaaS Product for Performance- and Load Testing
 We take care of the heavy lifting and pave the way for regular performance assessments
 </sales>
43. <sales> </sales>
44. Thank you! Questions? Sign Up and create preflight Load Tests for free! 
 https://stormforger.com
45. https://stormforger.com Get in touch! Lars Wolff @larsvegas
 lars@stormforger.com StormForger GmbH, Kyffhäuserstr. 17, 50674 Köln/Germany. Commercial Register: District Court Cologne, HRB 82582. Managing Directors: Sebastian Cohnen, Lars Wolff
46. Application Screenshots
47. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Test Case Editor
48. Do you have questions? support@stormforger.com · https://stormforger.com · +49 177 600 9780 Stacked Graphs
49. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Correlation Graph
50. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 HTTP Codes
51. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Other Errors and Events