Scaling CANDDi

for http://www.supermondays.org/

Transcript

1. Scaling CANDDi oli wood @coldclimate
2. history
3. measure things that matter
4. back of a fag packet if good enough
5. have somewhere to test which is not live
6. use realistic data
7. ab/siege + ec2 instances
8. there are no silver bullets
9. aim for a silver shotgun cartridge
10. define the scenarios
11. build a pipeline view
12. identify symptoms but solve problems
13. symptoms are what you can see
14. problems are what you can solve
15. bad new: the problem is your code
16. good news: thus you can fix it
17. do less big things less often
18. do frequent things much faster
19. avoid waiting
20. pull less data
21. cache the painful things
22. cache: in memory
23. cache: redis/memcache
24. cache: url/browser cache
25. hunt collisions
26. use the tools
27. iostat
28. sar -P
29. strace -p
30. iftop
31. xdebug + webgrind
32. statsd
33. mongotop
34. mongosniff
35. tail -100000 mongodb. log | awk -F' ' '{print $NF}' | grep ms$ | sed 's/ms//g' |sort -n | tail -20 | xargs -I {} grep {} ms$ mongodb.log
36. and the big gestures
37. nginx
38. nodejs
39. redis
40. aim: loosely coupled components which are horizontally scalable both on box and by box
41. BULLSHIT
42. aim: fast enough for medium term business needs
43. aim: not to waste time in a geek circle jerk
44. result: 10x the throughput
45. result: 2000 non- trivial inbound events a second
46. good luck