Site Availability is for Everybody

Transcript

1. Stella Cotton | @practice_cactus S I T E AVA I L A B I L I T Y I S F O R E V E RY B O D Y —
2. Stella Cotton | @practice_cactus P O P Q U I Z
3. Stella Cotton | @practice_cactus
4. Stella Cotton | @practice_cactus €1,600,000,000
5. Stella Cotton | @practice_cactus
6. Stella Cotton | @practice_cactus ☺
7. Stella Cotton | @practice_cactus 1 . G E T S TA RT E D 2 . T U R N U P T H E V O L U M E 3 . E X P L O R E T H E R E S U LT S
8. Stella Cotton | @practice_cactus G E T T I N G S TA RT E D
9. Stella Cotton | @practice_cactus Getting Started P R E PA R E
10. Getting Started: Stella Cotton | @practice_cactus Prepare http://example.com/ -c 1 -n 1000 ab
11. Getting Started: Stella Cotton | @practice_cactus Prepare http://example.com/ -c 1 -n 1000 ab
12. Getting Started: Stella Cotton | @practice_cactus Don’t DDoS Yourself Prepare
13. Getting Started: Stella Cotton | @practice_cactus (Yes, technically you are only DoS-ing yourself by running Apache Bench on a single machine but you can DDoS with other tools and it sounded funnier. ¯\_(π)_/¯ ) Prepare
14. Stella Cotton | @practice_cactus Getting Started C O N F I G U R E L O A D
15. Getting Started: Stella Cotton | @practice_cactus Configure Load http://example.com/ -c 1 -n 1000 ab
16. Getting Started: Stella Cotton | @practice_cactus “Request” Configure Load
17. Getting Started: Stella Cotton | @practice_cactus No HTML No JS Configure Load
18. Stella Cotton | @practice_cactus Getting Started A N A LY Z E
19. Getting Started: Stella Cotton | @practice_cactus Analyze Results
20. Getting Started: Stella Cotton | @practice_cactus Analyze Results
21. Getting Started: Stella Cotton | @practice_cactus Analyze Results
22. Getting Started: Stella Cotton | @practice_cactus 3xx/4xx/5xx Analyze Results
23. Getting Started: Stella Cotton | @practice_cactus Analyze Results http://example.com/ -c 5 -n 1000 ab -A user:password
24. Getting Started: Stella Cotton | @practice_cactus Analyze Results
25. Getting Started: Stella Cotton | @practice_cactus “Failed Requests” Analyze Results
26. Getting Started: Stella Cotton | @practice_cactus Analyze Results
27. Stella Cotton | @practice_cactus T U R N I N G U P T H E V O L U M E
28. Stella Cotton | @practice_cactus Turning Up the Volume Y O U R A P P + L O A D
29. Turning Up the Volume: Stella Cotton | @practice_cactus Little’s Law: L = λW Your App + Load
30. Turning Up the Volume: Stella Cotton | @practice_cactus Total Customers in Line Your App + Load Arrival
 rate How long it takes to check out = x
31. Turning Up the Volume: Stella Cotton | @practice_cactus Mean response time Your App + Load Mean # in system Mean throughput =
32. Turning Up the Volume: Stella Cotton | @practice_cactus 1. Proxy Server 2. Application Server 3. Database Your App + Load
33. Turning Up the Volume: Stella Cotton | @practice_cactus Your App + Load Proxy Server User Visits Your Site (Yay!) Firewall
34. Turning Up the Volume: Stella Cotton | @practice_cactus Your App + Load Application Server Proxy Server User Visits Your Site (Yay!) Firewall
35. Turning Up the Volume: Stella Cotton | @practice_cactus Your App + Load Proxy Server Application Server (Master Process) Worker (Child Process) Worker (Child Process) Worker (Child Process)
36. Turning Up the Volume: Stella Cotton | @practice_cactus Your App + Load Proxy Server Application Server (Master Process) Database Worker (Child Process) Worker (Child Process) Worker (Child Process)
37. Turning Up the Volume: Stella Cotton | @practice_cactus Your App + Load “504 Gateway Time-out”
38. Turning Up the Volume: Stella Cotton | @practice_cactus Multiple Queues are Less Efficient Your App + Load
39. Turning Up the Volume: Stella Cotton | @practice_cactus Your App + Load PROXY SERVER APP WORKERS
40. Turning Up the Volume: Stella Cotton | @practice_cactus Your App + Load PROXY SERVER APP WORKERS
41. Turning Up the Volume: Stella Cotton | @practice_cactus Increasing Timeout Threshold Your App + Load
42. Stella Cotton | @practice_cactus Turning Up the Volume Y O U R O S + L O A D
43. Turning Up the Volume: Stella Cotton | @practice_cactus “Too Many Open Files” Your OS + Load
44. Turning Up the Volume: Stella Cotton | @practice_cactus User file descriptor limits: ulimit -Hn ulimit -Sn Your OS + Load
45. Turning Up the Volume: Stella Cotton | @practice_cactus Estimating user file limit numbers: • Each file is ~ 1 k of memory • Don’t allocate more than 10% of memory to files • So, approximately 100 file descriptors for every MB of RAM Your OS + Load
46. Turning Up the Volume: Stella Cotton | @practice_cactus Adjust the System Limit: vi /etc/sysctl.conf with fs.file-max = 70000 Your OS + Load
47. Turning Up the Volume: Stella Cotton | @practice_cactus Adjust the User Limit # vi /etc/security/limits.conf with yourusername soft no file 10000 yourusername hard no file 30000 Your OS + Load H T T P : / / W W W. C Y B E R C I T I . B I Z / FA Q / L I N U X - U N I X - N G I N X - T O O - M A N Y- O P E N - F I L E S /
48. Turning Up the Volume: Stella Cotton | @practice_cactus Reload your changes # sysctl -p Your OS + Load
49. Turning Up the Volume: Stella Cotton | @practice_cactus Adjust the Proxy Server e.g. NGINX worker_rlimit_nofile 30000; Your OS + Load
50. Turning Up the Volume: Stella Cotton | @practice_cactus TCP/IP Port Exhaustion Your OS + Load
51. Turning Up the Volume: Stella Cotton | @practice_cactus Ephemeral Ports Your OS + Load
52. Turning Up the Volume: Stella Cotton | @practice_cactus Decrease TIME WAIT Your OS + Load
53. Turning Up the Volume: Stella Cotton | @practice_cactus Increase Port Range Your OS + Load
54. Turning Up the Volume: Stella Cotton | @practice_cactus Server recommendations 
 for OS Tuning 
 H T T P : / / U N I C O R N . B O G O M I P S . O R G / T U N I N G . H T M L Your OS + Load
55. Stella Cotton | @practice_cactus Turning Up the Volume ❄ ❄ ❄
56. Turning Up the Volume: Stella Cotton | @practice_cactus Cache Busting ❄❄❄
57. Turning Up the Volume: Stella Cotton | @practice_cactus Blocking External Requests ❄❄❄
58. Stella Cotton | @practice_cactus E X P L O R I N G Y O U R R E S U LT S
59. Stella Cotton | @practice_cactus Exploring Your Results L I M I TAT I O N S O N T H E H O S T M A C H I N E
60. Exploring Your Results: Stella Cotton | @practice_cactus top Investigating the Limits
61. Exploring Your Results: Stella Cotton | @practice_cactus Investigating the Limits
62. Exploring Your Results: Stella Cotton | @practice_cactus htop Investigating the Limits
63. Exploring Your Results: Stella Cotton | @practice_cactus Investigating the Limits
64. Exploring Your Results: Stella Cotton | @practice_cactus Zero Sum Game Investigating the Limits
65. Exploring Your Results: Stella Cotton | @practice_cactus External database Investigating the Limits
66. Exploring Your Results: Stella Cotton | @practice_cactus Don’t DDoS Yourself Investigating the Limits
67. Stella Cotton | @practice_cactus Exploring Your Results T R A N S A C T I O N T R A C I N G
68. Exploring Your Results: Stella Cotton | @practice_cactus Transaction Tracing H T T P S : / / D O C S . N E W R E L I C . C O M / D O C S / A P M / T R A N S A C T I O N S / T R A N S A C T I O N - T R A C E S / T R A N S A C T I O N - T R A C E S
69. Exploring Your Results: Stella Cotton | @practice_cactus rack-mini-profiler Transaction Tracing
70. Stella Cotton | @practice_cactus Exploring Your Results D ATA B A S E W O E S
71. Exploring Your Results: Stella Cotton | @practice_cactus Slow Query Log H T T P : / / D E V. M Y S Q L . C O M / D O C / R E F M A N / 5 . 7 / E N / S L O W - Q U E RY- L O G . H T M L Database Woes
72. Exploring Your Results: Stella Cotton | @practice_cactus SHOW PROCESSLIST; Database Woes
73. Stella Cotton | @practice_cactus O T H E R L O A D T E S T I N G T O O L S
74. Other Load Testing Tools Stella Cotton | @practice_cactus • Siege + Bombard • Bees with Machine Guns • JMeter + Ruby JMeter • ..and many more!
75. Stella Cotton | @practice_cactus F R A M E W O R K F O R C U R I O S I T Y
76. Stella Cotton | @practice_cactus T H A N K Y O U —