Scaling CANDDi

Transcript

1. Scaling CANDDi oli wood @coldclimate

2. history

3. measure things that matter

4. back of a fag packet if good enough

5. have somewhere to test which is not live

6. use realistic data

7. ab/siege + ec2 instances

8. there are no silver bullets

9. aim for a silver shotgun cartridge

10. define the scenarios

11. build a pipeline view

12. identify symptoms but solve problems

13. symptoms are what you can see

14. problems are what you can solve

15. bad new: the problem is your code

16. good news: thus you can fix it

17. do less big things less often

18. do frequent things much faster

19. avoid waiting

20. pull less data

21. cache the painful things

22. cache: in memory

23. cache: redis/memcache

24. cache: url/browser cache

25. hunt collisions

26. use the tools

27. iostat

28. sar -P

29. strace -p

30. iftop

31. xdebug + webgrind

32. statsd

33. mongotop

34. mongosniff

35. tail -100000 mongodb. log | awk -F' ' '{print $NF}'

    | grep ms$ | sed 's/ms//g' |sort -n | tail -20 | xargs -I {} grep {} ms$ mongodb.log

36. and the big gestures

37. nginx

38. nodejs

39. redis

40. aim: loosely coupled components which are horizontally scalable both on

    box and by box

41. BULLSHIT

42. aim: fast enough for medium term business needs

43. aim: not to waste time in a geek circle jerk

44. result: 10x the throughput

45. result: 2000 non- trivial inbound events a second

46. good luck