Performance Testing Websites
 vs.
 Performance Testing (RESTful) APIs 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 EHLO[1] Lars Wolff
 @larsvegas • [email protected] • Co-founder & CEO of
 stormforger.com • Software Development of web based systems • Agile Hippie (Coach)
 Certified ScrumMaster®, Kanban Management Professional • Amazon Web Services User Group Cologne
 #AWSUGCGN • Web Performance User Group Cologne • Web Montag Cologne
 #WEBMONTAG #WMCGN • DevHouseFriday Cologne
 #DevHouseFriday [1] http://www.ietf.org/rfc/rfc2821.txt 

We empower organizations to do continuous performance testing. 

EXPLICIT EVOLVING PARENTAL ADVISORY 

I assume you are building/running a website and/or an API. 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Who does NOT have performance issues? :) • Who does clearly understand and has insights
 to these issues? • Under which circumstances do they occur? • What is your traffic scenario? 

Basics 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Basics Most of this part should be clear. But I learned that it’s helpful to recap it. 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 HTTP request - response cycle http://celineotter.azurewebsites.net/world-wide-web-http-request-response-cycle/ 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Performance Testing • Is business critical • Is orthogonal in your organisation • Results should always be available • Should be fully automated 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Performance Testing
 – Prerequisites • Goal? What do you want to learn? • Non-functional requirements? • Performance Budget? • Test and traffic scenario? • Environment to test against?
 (SUT = System Under Test) 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 1. (Non-functional) requirements 2. Test definition 3. Test execution 4. Analysis of resulting data Flow 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Performance Testing
 – Types of Testing 1. Load Testing 2. Stress Testing 3. Scalability Testing 4. Spike Testing 5. Soak Testing 6. Configuration Testing 7. Availability & Resilience Testing https://stormforger.com/blog/2016/07/08/types-of-performance-testing/ https://docs.stormforger.com/guides/ 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 User, Application and System Monitoring • Real User Monitoring (RUM) • How does the client perform for all users? • Application Performance Monitoring • What happens inside my application? • What kind of request takes how long? Why? • What kind of follow-up requests / queries does my application create?
 • System-Monitoring • What's going on in my internal network? • What's going on in my web server / app server machine? • What’s going on in my database cluster? • What's about storage and IOPS? 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Mind your organisation 1. Performance testing and analysis is team work! 2. Please involve: product people, developers, QA, operations 3. “Working software over comprehensive documentation” – but make your results transparent to every stage of your organization! 

Performance Testing Websites 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Website characteristics • multiple types of content
 (HTML, Images, CSS, JS, ...) • “client logic”
 (JavaScript, XHR) • complex client behavior
 (client type * user behavior * context) • “heavy” 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Performance Testing / Load Testing ≠ #WebPerf 

#WebPerf 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 – https://www.w3.org/2010/webperf/ “[...] methods to measure and improve aspects of application performance of user agent features and APIs.” #WebPerf 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 #WebPerf 101 • less requests! (cache stuff) • less requests! (minify, concat) • less requests! (CDN for assets) • image compression / optimization ... • … a lot more … • how long does a tcp handshake take? :) • how much tcp connections does a (certain) browser open in parallel? :) 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 #WebPerf 101 Know your language • Naming in general – be precise • “loads slowly” vs. “takes long until it is displayed (= rendered)” 

#PerfTest 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 –Wikipedia „In software engineering, performance testing is in general, a testing practice performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.“ 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Things you have to take care of • XHR & clients logic (JavaScript): A lot of client logic? Single page application (SPA)? • correct HTTP (error) response codes • Assets = bandwidth => CDN • Content extractions, e.g. CSRF token to submit form data • Double opt in • Test data in general 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Things you have to take care of • Testing external services? CDN? Tracking? Why??? • What does HTTP OPTIONS do? • Bandwidth: Shiny top of the fold header image and no CDN? :P 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 HTTP/1.1 vs. HTTP/2 • Some things change. • Some don’t. • What to focus on first? HTTP/2 Performance Anti-patterns by Ilya Grigorik
 https://docs.google.com/presentation/d/1_SMrVmiMxW2X1QZ1EcCnLKSosiD0PppP70Q3bw-l5Lg/present 

Recap: Website 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Recap: Website 1. #WebPerf FTW 2. Seriously watch your bandwidth 3. Plenty other things to take care of: Test Data, Double Opt-in, CSRF, Auth-Tokens 4. Iterate slowly and communicate with other people 5. Never test external partners $$$
 (tracking, advertisers, etc.) 

Performance Testing
 (RESTful) APIs 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 API characteristics • Usually XML or JSON responses • “Sequential” API flow, RESTful API flow, HATEOS / Hypermedia • Authenticated (authentication required or available auth tokens via test data) • No (browser) client, no client behavior – *sigh* :D • Requests from different clients (browser, mobile app, IoT, etc.) should be well-formed and ask for something really really specific • And... there is client logic (filters, paging, hypermedia) 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Things you have to take care of • Authentication (Basic Auth, SingleSignOn, oAuth, etc.) • HTTP headers (caching, ETags, gzip) • Header and content extractions ('id' to follow, 'auth-token' to use) • Test data in general • Correct HTTP (error) response codes • HTTP (error) codes in response body (response is HTTP200, content says 401 or “authentication required”) • RESTful (HTTP PUT vs. HTTP POST) • Auto follow of hypermedia links means you need a correct and full API specification :) • Rate limiting: Watch HTTP429 / too many requests, watch the fallback • One request per item vs. a request for a filterable batched result • Polling? Every minute? Caching fun. Don't send requests on a specific and fixed time... cron-style • No client handling / bad handling for HTTP5xx errors 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Advanced API performance testing
 – SOA & µService • Your environment is really distributed
 – monitoring and request tracing are must haves • Try to isolate each service and test it
 – firewall? proxy to get traffic into the data center... • Try to combine test scenarios of services for a particular user journey and test their contracts 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Advanced API performance testing
 – Serverless (here: AWS Lambda) • AWS Lambda limits*
 – e.g. 100 concurrent invocations • AWS API Gateway limits*
 – e.g. request throttling 1000 requests per second (rps) • What are your Service Contracts / SLAs? What happens in cascades? * ask AWS, they have knobs ;-) http://docs.aws.amazon.com/lambda/latest/dg/limits.html http://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html https://stormforger.com/presentations/serverles-microservices-vs-monolitic-beanstalk-app-loadtest/ https://stormforger.com/blog/2017/03/22/advices-on-performance-from-aws-well-architected-framework/ 

Recap: APIs 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Recap: APIs 1. Authentication 2. API flow and user journey 3. There actually is “client logic“ 4. Iterate slowly and communicate with other people 

Wrap-up 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Wrap-up – How to do it? 1. Set a goal, set non-functional-requirements 2.Discuss user journey 3. Create minimal viable test case version
 (use HAR to scaffold for websites) 4. Test slowly 5. Iterate 6. Heads up with your team – communicate! 7. “Stale” your scenario 8. Run with more traffic for specific test type 9.Identify Problems, Bottlenecks 10.Fix stuff 11.Repeat 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Both is complex and there are plenty of things to take care of. 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Needs to involve people from all needed departments/professions in your organization. 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Should be started early. 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Should be started early – really! 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs Is nothing to get stuck in the
 “Not Invented Here Syndrome”! – There are a lot of tools out there! https://en.wikipedia.org/wiki/Not_invented_here 

Do you have questions? [email protected] · https://stormforger.com · +49 221 64 30 51 28 Wrap-up Performance testing website and performance testing APIs #PerfTest early.
 #PerfTest often.
 #PerfTest continuously. 

https://stormforger.com Thank you! :) Questions? Lars Wolff – @larsvegas – [email protected]