T3CM24: Load Testing with K6

Transcript

1. MARTIN HELMICH @mittwald LOAD TESTING WITH K6 September 14th, 2024
2. MARTIN HELMICH Head of Architecture & Developer Relations Lecturer, Software Engineering & Cloud Computing Sci-Fi-Nerd, Metalhead, Amateur Woodworker
3. RELEASING YOUR PROJECT TO PROD REAL WORLD TRAFFIC
4. Image Source
5. Image Source
6. CHOOSE YOUR TOOLS
7. martin @ local $ ab -k -c 100 -t 60 https://my-loadtest.example/ This is ApacheBench, Version 2.3 <$Revision: 1903618 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking my-loadtest.example (be patient) Finished 50000 requests Server Software: Apache/2.4.59 Server Hostname: my-loadtest.example Server Port: 80 Document Path: / Document Length: 17620 bytes Concurrency Level: 100 Time taken for tests: 6.045 seconds Complete requests: 50000 Failed requests: 14 (Connect: 0, Receive: 0, Length: 14, Exceptions: 0) Keep-Alive requests: 49554 Total transferred: 894980321 bytes HTML transferred: 880753320 bytes Requests per second: 8270.99 [#/sec] (mean) Time per request: 12.090 [ms] (mean) Time per request: 0.121 [ms] (mean, across all concurrent requests) Transfer rate: 144577.66 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 3 Processing: 0 12 5.2 11 59 Waiting: 0 12 5.2 11 59 Total: 0 12 5.3 11 62 Percentage of the requests served within a certain time (ms) 50% 11 66% 13 75% 14 80% 15 90% 18 95% 21 98% 25 99% 29 100% 62 (longest request)
8. MORE TOOLS SIEGE https://www.joedog.org/siege-home/ WRK https://github.com/wg/wrk JMETER https://github.com/apache/jmeter
9. LOAD TEST REQUIREMENTS - Mimic user behaviour realistically - Verify if service level objectives ( SLOs) are met - CI/CD integration
10. https://k6.io/
11. KINDS OF LOAD TESTS
12. DURATION THROUGHPUT FEW AVERAGE ABOVE AVERAGE SMOKE TEST - Assert general availability - Get baseline performance under minimal load KINDS OF LOAD TESTS
13. import http from 'k6/http'; import { check } from 'k6'; export const options = { vus: 1, duration: '20s', }; export default function() { const res = http.get('https://my-loadtest.example'); check(res, { 'is status 200': r => r.status === 200, }); } The DEFAULT EXPORT determines how a virtual user should behave Configure how many VIRTUAL USERS k6 should test with (and for how long)
15. DURATION THROUGHPUT FEW AVERAGE ABOVE AVERAGE AVERAGE - LOAD TEST ( DAY - IN - LIFE TEST ) - Get performance under typical everyday load - Identify (new) bottlenecks early WHAT IS YOUR TYPICAL LOAD? RAMP - UP PERIOD
16. import http from 'k6/http'; import { check, sleep } from 'k6'; export const options = { stages: [ { duration: "30s", target: 100 }, { duration: "2m", target: 100 }, { duration: "30s", target: 0 }, ], thresholds: { http_req_failed: ['rate<0.01'], http_req_duration: ['p(95)<200'], }, }; export default function() { const res = http.get('http://localhost'); sleep(2); check(res, { 'is status 200': r => r.status === 200, }); } Define THRESHOLDS to automatically verify target metrics Use STAGES for ramp-up phases TIME your virtual users to mimic realistic user behaviour
17. DURATION THROUGHPUT FEW AVERAGE ABOVE AVERAGE STRESS TEST ( SURGE TEST, SCALE TEST ) - Learn how (and how much) performance degrades on above-average load - Assert that performance remains constant under constant (high) load RAMP - UP PERIOD
18. DURATION THROUGHPUT FEW AVERAGE ABOVE AVERAGE SPIKE TEST - Learn how the system behaves under unexpected immediate loads ( DOS attack, sudden user influx, ...) - Learn about failure & recovery behaviour NO/SHORT RAMP - UP PERIOD! 💥
19. DURATION THROUGHPUT FEW AVERAGE ABOVE AVERAGE BREAKPOINT TEST ( CAPACITY TEST, LIMIT TEST ) - Identify system limits - Learn about resource consumption with growing load 💥
20. HTTP LOAD TESTING vs BROWSER TESTING
21. import { browser } from 'k6/browser'; import { check } from 'k6'; export const options = { scenarios: { ui: { executor: 'shared-iterations', iterations: 10, options: { browser: { type: 'chromium', }, }, }, }, thresholds: { checks: ['rate==1.0'], browser_web_vital_ttfb: ['p(95)<50'], browser_web_vital_fcp: ['p(95)<100'], }, }; export default async function () { const context = await browser.newContext(); const page = await context.newPage(); try { await page.goto('http://localhost'); await Promise.all([page.waitForNavigation(), page.locator('a[title="Features"]').click()]); await page.screenshot({ path: 'screenshots/screenshot.png' }); const header = await page.locator('h1').textContent(); check(header, { "expected heading is found": (h) => h == 'With a rich core feature set out-of-the-box, TYPO3 is an ideal choice for building ambitious digital experiences.', }); } finally { await page.close(); } } CORE WEB VITALS are exposed as metrics The API is designed to be compatible with PLAYWRIGHT
22. import http from 'k6/http'; import { browser } from 'k6/browser'; import { check, sleep } from 'k6'; export const options = { scenarios: { ui: { executor: 'shared-iterations', exec: 'browserTest', iterations: 10, startTime: "20s", options: { browser: { type: 'chromium', }, }, }, background: { executor: 'ramping-vus', exec: 'backgroundLoad', stages: [ { duration: "20s", target: 20 }, { duration: "1m", target: 20 }, ], } }, thresholds: { checks: ['rate==1.0'], browser_web_vital_ttfb: ['p(95)<50'], browser_web_vital_fcp: ['p(95)<100'], }, }; export async function browserTest() { const context = await browser.newContext(); const page = await context.newPage(); try { await page.goto("http://localhost"); Use MULTIPLE SCENARIOS to keep up an constant average load in the background while observing browser metrics
23. IMPORTANT SERVICE ANNOUNCEMENT Some kinds of load tests (stress tests, or breakpoint tests) are indistinguishable from DoS attacks. If in doubt, get a PtA ( Permission to Attack). When in the public cloud, mind your (or your client's) traffic bill. (your hosting provider will thank you 😉)
24. https://github.com/mittwald https://github.com/martin-helmich https://www.mittwald.de https://www.linkedin.com/company/mittwald https://www.linkedin.com/in/martinhelmich