$30 off During Our Annual Pro Sale. View Details »

Deathmatch: Bundler vs. Rubygems (GoRuCo 2013)

Deathmatch: Bundler vs. Rubygems (GoRuCo 2013)

The story of the quest to make bundle install faster; in which Rubyists around the world inadvertently DDoS rubygems.org, witness its ignominious death, and vow to rebuild it from the ashes stronger than it was before. Then, a tour of the changes; why is Redis so much slower than Postgres? Marvel at the gorgeous metrics and graphs used to measure and optimize; gasp in delight as we track, live, exactly how many Heroku dynos are needed. Finally, a happy ending: today, the server responds to requests TWO ORDERS OF MAGNITUDE faster than it did before.

André Arko

June 08, 2013
Tweet

More Decks by André Arko

Other Decks in Technology

Transcript

  1. Deathmatch
    vs

    View Slide

  2. André Arko
    @indirect
     
     

    View Slide

  3. View Slide

  4. View Slide

  5. vs

    View Slide

  6. !

    View Slide

  7. Bundler 1.0

    View Slide

  8. Bundler 1.0
    30 Aug 2010

    View Slide

  9. Bundler 1.0
    30 Aug 2010
    !

    View Slide

  10. bundle install

    View Slide

  11. bundle install
    ...and install

    View Slide

  12. bundle install
    ...and install
    ...and install

    View Slide

  13. bundle install
    ...and install
    ...and install
    ...and install
    ...and install

    View Slide

  14. !

    View Slide

  15. (what seemed like)
    10 minutes later…

    View Slide

  16. View Slide

  17. bundler API

    View Slide

  18. just the gems
    you need
    bundler API

    View Slide

  19. so. much. faster.
    bundler API

    View Slide

  20. so. much. faster.
    32s → 8s

    View Slide

  21. Bundler 1.1

    View Slide

  22. 07 March 2012
    Bundler 1.1

    View Slide

  23. !
    07 March 2012
    Bundler 1.1

    View Slide

  24. are lots of work
    API requests

    View Slide

  25. etc, etc…
    pride cometh

    View Slide

  26. reenactment
    dramatic

    View Slide

  27. !
    !

    View Slide

  28. !
    !
    !

    View Slide

  29. !
    !
    !
    ! !

    View Slide

  30. !
    !
    !
    ! !
    ! !

    View Slide

  31. no more API

    View Slide

  32. View Slide

  33. View Slide

  34. View Slide

  35. crap.

    View Slide

  36. replacement
    a new bundler api

    View Slide

  37. heroku
    sinatra
    postgres

    View Slide

  38. questions

    View Slide

  39. questions
    will it work,
    this time?

    View Slide

  40. questions
    how is it bad?

    View Slide

  41. questions
    how can it be good?

    View Slide

  42. answers
    need measurements

    View Slide

  43. we measure
    a lot of things

    View Slide

  44. librato
    tracks
    all the metrics

    View Slide

  45. View Slide

  46. papertrail
    logs
    all the lines

    View Slide

  47. View Slide

  48. knowledge
    measuring leads to

    View Slide

  49. discovery 1
    postgres is fast

    View Slide

  50. discovery 1
    big postgres is faster

    View Slide

  51. discovery 2
    redis caching is slow

    View Slide

  52. discovery 2
    !!!
    !!!

    View Slide

  53. discovery 2
    totally unused, sadly

    View Slide

  54. discovery 3
    threading is great!

    View Slide

  55. threading is great!
    ...but deadlocks aren’t
    discovery 3

    View Slide

  56. discovery 4
    usage is tricky

    View Slide

  57. discovery 4
    metrics are tricky

    View Slide

  58. discovery 4
    capacity is tricky

    View Slide

  59. discovery 4
    monitor all of them

    View Slide

  60. response time
    is all lies
    discovery 5

    View Slide

  61. discovery 5
    check it from
    outside

    View Slide

  62. so it’s all good now,
    right?

    View Slide

  63. well… mostly…

    View Slide

  64. well… mostly…
    !!!

    View Slide

  65. time for the weird
    bug reports

    View Slide

  66. is really, really hard
    synchronizing

    View Slide

  67. is really, really slow
    background sync
    !!!

    View Slide

  68. are pretty unreliable
    webooks

    View Slide

  69. is pretty decent
    both

    View Slide

  70. bundling is pretty fast
    after all that
    !
    !
    !

    View Slide

  71. app time is two orders
    of magnitude faster
    2s → 20ms

    View Slide

  72. it’s okay,
    I guess

    View Slide

  73. we can do
    way better

    View Slide

  74. we have a
    plan

    View Slide

  75. ruby central
    grant

    View Slide

  76. you
    can help

    View Slide

  77. operators
    are standing by

    View Slide

  78. thank you

    View Slide

  79. questions?

    View Slide