Scaling CANDDi

Scaling CANDDi

6f44c2cc7923c18417bb5a4126152fb2?s=128

coldclimate

July 29, 2013
Tweet

Transcript

  1. Scaling CANDDi oli wood @coldclimate

  2. history

  3. measure things that matter

  4. back of a fag packet if good enough

  5. have somewhere to test which is not live

  6. use realistic data

  7. ab/siege + ec2 instances

  8. there are no silver bullets

  9. aim for a silver shotgun cartridge

  10. define the scenarios

  11. build a pipeline view

  12. identify symptoms but solve problems

  13. symptoms are what you can see

  14. problems are what you can solve

  15. bad new: the problem is your code

  16. good news: thus you can fix it

  17. do less big things less often

  18. do frequent things much faster

  19. avoid waiting

  20. pull less data

  21. cache the painful things

  22. cache: in memory

  23. cache: redis/memcache

  24. cache: url/browser cache

  25. hunt collisions

  26. use the tools

  27. iostat

  28. sar -P

  29. strace -p

  30. iftop

  31. xdebug + webgrind

  32. statsd

  33. mongotop

  34. mongosniff

  35. tail -100000 mongodb. log | awk -F' ' '{print $NF}'

    | grep ms$ | sed 's/ms//g' |sort -n | tail -20 | xargs -I {} grep {} ms$ mongodb.log
  36. and the big gestures

  37. nginx

  38. nodejs

  39. redis

  40. aim: loosely coupled components which are horizontally scalable both on

    box and by box
  41. BULLSHIT

  42. aim: fast enough for medium term business needs

  43. aim: not to waste time in a geek circle jerk

  44. result: 10x the throughput

  45. result: 2000 non- trivial inbound events a second

  46. good luck