Deathmatch: Bundler vs. Rubygems (GoRuCo 2013)

The story of the quest to make bundle install faster; in which Rubyists around the world inadvertently DDoS rubygems.org, witness its ignominious death, and vow to rebuild it from the ashes stronger than it was before. Then, a tour of the changes; why is Redis so much slower than Postgres? Marvel at the gorgeous metrics and graphs used to measure and optimize; gasp in delight as we track, live, exactly how many Heroku dynos are needed. Finally, a happy ending: today, the server responds to requests TWO ORDERS OF MAGNITUDE faster than it did before.

Transcript

1. Deathmatch vs
2. André Arko @indirect    
5. vs
6. !
7. Bundler 1.0
8. Bundler 1.0 30 Aug 2010
9. Bundler 1.0 30 Aug 2010 !
10. bundle install
11. bundle install ...and install
12. bundle install ...and install ...and install
13. bundle install ...and install ...and install ...and install ...and install
14. !
15. (what seemed like) 10 minutes later…
17. bundler API
18. just the gems you need bundler API
19. so. much. faster. bundler API
20. so. much. faster. 32s → 8s
21. Bundler 1.1
22. 07 March 2012 Bundler 1.1
23. ! 07 March 2012 Bundler 1.1
24. are lots of work API requests
25. etc, etc… pride cometh
26. reenactment dramatic
27. ! !
28. ! ! !
29. ! ! ! ! !
30. ! ! ! ! ! ! !
31. no more API
35. crap.
36. replacement a new bundler api
37. heroku sinatra postgres
38. questions
39. questions will it work, this time?
40. questions how is it bad?
41. questions how can it be good?
42. answers need measurements
43. we measure a lot of things
44. librato tracks all the metrics
46. papertrail logs all the lines
48. knowledge measuring leads to
49. discovery 1 postgres is fast
50. discovery 1 big postgres is faster
51. discovery 2 redis caching is slow
52. discovery 2 !!! !!!
53. discovery 2 totally unused, sadly
54. discovery 3 threading is great!
55. threading is great! ...but deadlocks aren’t discovery 3
56. discovery 4 usage is tricky
57. discovery 4 metrics are tricky
58. discovery 4 capacity is tricky
59. discovery 4 monitor all of them
60. response time is all lies discovery 5
61. discovery 5 check it from outside
62. so it’s all good now, right?
63. well… mostly…
64. well… mostly… !!!
65. time for the weird bug reports
66. is really, really hard synchronizing
67. is really, really slow background sync !!!
68. are pretty unreliable webooks
69. is pretty decent both
70. bundling is pretty fast after all that ! ! !
71. app time is two orders of magnitude faster 2s → 20ms
72. it’s okay, I guess
73. we can do way better
74. we have a plan
75. ruby central grant
76. you can help
77. operators are standing by
78. thank you
79. questions?