Deathmatch: Bundler vs. Rubygems (GoRuCo 2013)

Deathmatch: Bundler vs. Rubygems (GoRuCo 2013)

The story of the quest to make bundle install faster; in which Rubyists around the world inadvertently DDoS rubygems.org, witness its ignominious death, and vow to rebuild it from the ashes stronger than it was before. Then, a tour of the changes; why is Redis so much slower than Postgres? Marvel at the gorgeous metrics and graphs used to measure and optimize; gasp in delight as we track, live, exactly how many Heroku dynos are needed. Finally, a happy ending: today, the server responds to requests TWO ORDERS OF MAGNITUDE faster than it did before.

4c3ed917e59156a36212d48155831482?s=128

André Arko

June 08, 2013
Tweet

Transcript

  1. Deathmatch vs

  2. André Arko @indirect    

  3. None
  4. None
  5. vs

  6. !

  7. Bundler 1.0

  8. Bundler 1.0 30 Aug 2010

  9. Bundler 1.0 30 Aug 2010 !

  10. bundle install

  11. bundle install ...and install

  12. bundle install ...and install ...and install

  13. bundle install ...and install ...and install ...and install ...and install

  14. !

  15. (what seemed like) 10 minutes later…

  16. None
  17. bundler API

  18. just the gems you need bundler API

  19. so. much. faster. bundler API

  20. so. much. faster. 32s → 8s

  21. Bundler 1.1

  22. 07 March 2012 Bundler 1.1

  23. ! 07 March 2012 Bundler 1.1

  24. are lots of work API requests

  25. etc, etc… pride cometh

  26. reenactment dramatic

  27. ! !

  28. ! ! !

  29. ! ! ! ! !

  30. ! ! ! ! ! ! !

  31. no more API

  32. None
  33. None
  34. None
  35. crap.

  36. replacement a new bundler api

  37. heroku sinatra postgres

  38. questions

  39. questions will it work, this time?

  40. questions how is it bad?

  41. questions how can it be good?

  42. answers need measurements

  43. we measure a lot of things

  44. librato tracks all the metrics

  45. None
  46. papertrail logs all the lines

  47. None
  48. knowledge measuring leads to

  49. discovery 1 postgres is fast

  50. discovery 1 big postgres is faster

  51. discovery 2 redis caching is slow

  52. discovery 2 !!! !!!

  53. discovery 2 totally unused, sadly

  54. discovery 3 threading is great!

  55. threading is great! ...but deadlocks aren’t discovery 3

  56. discovery 4 usage is tricky

  57. discovery 4 metrics are tricky

  58. discovery 4 capacity is tricky

  59. discovery 4 monitor all of them

  60. response time is all lies discovery 5

  61. discovery 5 check it from outside

  62. so it’s all good now, right?

  63. well… mostly…

  64. well… mostly… !!!

  65. time for the weird bug reports

  66. is really, really hard synchronizing

  67. is really, really slow background sync !!!

  68. are pretty unreliable webooks

  69. is pretty decent both

  70. bundling is pretty fast after all that ! ! !

  71. app time is two orders of magnitude faster 2s →

    20ms
  72. it’s okay, I guess

  73. we can do way better

  74. we have a plan

  75. ruby central grant

  76. you can help

  77. operators are standing by

  78. thank you

  79. questions?