Performance Testing Websites vs. Performance Testing (RESTful) APIs @ WebPerfHH

There are a lot of differences performance testing websites vs. (RESTful) APIs. This talk will give you an overview of approaches how to load test websites and/or APIs. It will point out common pitfalls, parts to take care of and best practises.

https://www.meetup.com/de-DE/Hamburg-Web-Performance-Group/events/234260743/

See also: https://stormforger.com

Transcript

1. Performance Testing Websites
 vs.
 Performance Testing (RESTful) APIs
2. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 EHLO Lars Wolff
 @larsvegas • Focus on large, web based applications • 10+ years consulting and development • Agile Hippie (Coach) / Certified ScrumMaster® • Co-Founder & CEO of
 stormforger.com
3. We empower developers to do continuous performance testing.
4. EXPLICIT EVOLVING PARENTAL ADVISORY
5. I assume you are building/running a website and/or an API.
6. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Who does NOT have performance issues? :) • Who does clearly understand and has insights
 to these issues? • Under which circumstances do they occur? • What is your traffic scenario?
7. Basics
8. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Basics Most of this part should be clear. But I learned that it’s helpful to recap it.
9. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 HTTP request - response cycle http://celineotter.azurewebsites.net/world-wide-web-http-request-response-cycle/
10. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Performance Testing • Is business critical • Is orthogonal in your organisation • Results should always be available • Should be fully automated
11. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Performance Testing
 – Prerequisites • Goal? What do you want to learn? • Non-functional requirements? • Performance Budget? • Test and traffic scenario? • Environment to test against?
12. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 1. (Non-functional) requirements 2. Test definition 3. Test execution 4. Analysis of resulting data Flow
13. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Performance Testing
 – Types of Testing 1. Load Testing 2. Stress Testing 3. Scalability Testing 4. Spike Testing 5. Soak Testing 6. Configuration Testing 7. Availability & Resilience Testing https://stormforger.com/blog/2016/07/08/types-of-performance-testing/ https://docs.stormforger.com/guides/
14. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 User, Application and System Monitoring • Real User Monitoring (RUM) • How does the client perform for all users? • Application Performance Monitoring • What happens inside my application? • What kind of request takes how long? Why? • What kind of follow-up requests / queries does my application create?
 • System-Monitoring • What's going on in my internal network? • What's going on in my web server / app server machine? • What’s going on in my database cluster? • What's about storage and IOPS?
15. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Mind your organisation 1. Performance testing and analysis is team work! 2. Please involve: product people, developers, QA, operations 3. “Working software over comprehensive documentation” – but make your results transparent to every stage of your organization!
16. Performance Testing Websites
17. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Website characteristics • multiple types of content
 (HTML, Images, CSS, JS, ...) • “client logic”
 (JavaScript, XHR) • complex client behavior
 (client type * user behavior * context) • “heavy”
18. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 #WebPerf Performance Testing / Load Testing ≠ #WebPerf
19. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 – https://www.w3.org/2010/webperf/ “[...] methods to measure and improve aspects of application performance of user agent features and APIs.” #WebPerf
20. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 #WebPerf 101 • less requests! (cache stuff) • less requests! (minify, concat) • less requests! (CDN for assets) • image compression / optimization ... • … a lot more … • how long does a tcp handshake take? :) • how much tcp connections does a (certain) browser open in parallel? :)
21. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 #WebPerf 101 Know your language • Naming • “loads slowly” vs. “takes long until it is displayed (= rendered)”
22. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Things you have to take care of • XHR & clients logic (JavaScript): A lot of client logic? Single page application (SPA)? • correct HTTP (error) response codes • Assets = bandwidth => CDN • Content extractions, e.g. CSRF token to submit form data • Double opt in • Test data in general
23. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Things you have to take care of • Testing external services? CDN? Tracking? Why??? • What does HTTP OPTIONS do? • Bandwidth: Shiny top of the fold header image and no CDN? :P
24. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 HTTP/1.1 vs. HTTP/2 • Some things change. • Some don’t. • What to focus on first? HTTP/2 Performance Anti-patterns by Ilya Grigorik
 https://docs.google.com/presentation/d/1_SMrVmiMxW2X1QZ1EcCnLKSosiD0PppP70Q3bw-l5Lg/present
25. Recap: Website
26. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Recap: Website 1. #WebPerf FTW 2. Seriously watch your bandwidth 3. Plenty other things to take care of 4. Iterate slowly and communicate with other people 5. Never test external partners (tracking, advertisers, etc. etc.)
27. Performance Testing
 (RESTful) APIs
28. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 API characteristics • Usually XML or JSON responses • “Sequential” API flow, RESTful API flow, HATEOS / Hypermedia • Authenticated (authentication required or available auth tokens via test data) • No (browser) client, no client behavior – *sigh* :D • Requests from different clients (browser, mobile app, IoT, etc.) should be well-formed and ask for something really really specific • And... there is client logic (filters, paging, hypermedia)
29. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Things you have to take care of • Authentication (Basic Auth, SingleSignOn, oAuth, etc.) • HTTP headers (caching, ETags, gzip) • Header and content extractions ('id' to follow, 'auth-token' to use) • Test data in general • Correct HTTP (error) response codes • HTTP (error) codes in response body (response is HTTP200, content says 401 or “authentication required”) • RESTful (HTTP PUT vs. HTTP POST) • Auto follow of hypermedia links means you need a correct and full API specification :) • Rate limiting: Watch HTTP429 / too many requests, watch the fallback • One request per item vs. a request for a filterable batched result • Polling? Every minute? Caching fun. Don't send requests on a specific and fixed time... cron-style • No client handling / bad handling for HTTP5xx errors
30. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Advanced API performance testing
 – SOA & µService • Your environment is really distributed
 – monitoring and request tracking are must haves • Try to isolate each service and test it (firewall? proxy to get traffic into the data center...) • Try to combine test scenarios of services for a particular user journey and test their contracts
31. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Advanced API performance testing
 – Serverless (here: AWS Lambda) • AWS Lambda limits*
 – e.g. 100 concurrent invocations by • AWS API Gateway limits*
 – e.g. request throttling 1000 requests per second (rps) http://docs.aws.amazon.com/lambda/latest/dg/limits.html https://stormforger.com/presentations/serverles-microservices-vs-monolitic-beanstalk-app-loadtest/ http://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html * ask AWS, they have knobs ;-)
32. Recap: APIs
33. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Recap: APIs 1. Authentication 2. API flow and user journey 3. There is “client logic“ 4. Iterate slowly and communicate with other people
34. Wrap-up
35. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up – How to do it? 1. Discuss user journey 2. Create minimal viable test case version 3. Test slowly 4. Iterate 5. Communicate 6. “Stale” your scenario 7. Run with more traffic for specific test type 8. Website: Use HAR to help you to get all
 the requests you need for your website
36. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Both is complex and there are plenty of things to take care of.
37. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Needs to involve people from all needed departments/professions in your organisation.
38. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Should be started early.
39. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Should be started early – really!
40. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Is nothing to get stuck in the
 “Not Invented Here Syndrome”! – There are a lot of tools out there! https://en.wikipedia.org/wiki/Not_invented_here
41. Do you have questions? support@stormforger.com · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Should be done early.
 Should be done often.
 Should be done continuously.
42. https://stormforger.com Thank you! :) Questions? Lars Wolff @larsvegas
 lars@stormforger.com StormForger GmbH, Kyffhäuserstr. 17, 50674 Köln/Germany. Commercial Register: District Court Cologne, HRB 82582. Managing Directors: Sebastian Cohnen, Lars Wolff