$30 off During Our Annual Pro Sale. View Details »

Extreme Makeover: Rubygems Edition

Extreme Makeover: Rubygems Edition

Rubygems.org provides every Rubyist with an amazing service: all the libraries in the Ruby world. As amazing as that is, installing gems can be a time-consuming and even error-prone process. (Just ask the Travis guys.) In this talk, you'll learn about the recent dramatic changes in Rubygems and Bundler to improve speed and reliability by rewriting the Rubygems client/server architecture. I'll show how the new system caches more information, makes fewer requests, and takes less time to install gems. Finally, I'll cover how the changes allow worldwide mirrors of rubygems.org, improving things for Rubyists around the globe.

André Arko

March 28, 2014

More Decks by André Arko

Other Decks in Technology


  1. Extreme Makeover Rubygems Edition

  2. André Arko @indirect

  3. None
  4. State of the Rubygems Year in Review

  5. Bundler DDoS ! ! ! ! !

  6. Bundler DDoS

  7. Security breach

  8. Rebuilt rubygems.org

  9. Travis network issues ☁️

  10. Travis DNS issues ☁️

  11. SSL failures ⚠️

  12. SSL fixes

  13. How it works today

  14. How it works today with a fast connection, at least

  15. /specs/rails-4.0.1.gemspec /specs/sinatra-1.4.2.gemspec ! Gemspecs

  16. /specs/rails-4.0.1.gemspec /specs/sinatra-1.4.2.gemspec /Marshal.4.8.gz Marshalled Gemspecs

  17. /specs/rails-4.0.1.gemspec /specs/sinatra-1.4.2.gemspec /Marshal.4.8.gz /specs.4.8.gz List of Gemspecs

  18. /api/v1/dependencies?gems=rack /specs/gemname-version.gemspec ! The Bundler API

  19. Could be better

  20. We Have A Plan

  21. We Have A Plan simple plaintext indexes

  22. We Have A Plan clients cache the indexes

  23. We Have A Plan put a CDN on it

  24. We Have A Plan easy local mirrors

  25. The New Index /versions /deps/gemname

  26. /versions rack 1.0.0,1.0.1,1.0.2 sinatra 0.9.6,1.0,1.1.4 rack 2.0.0

  27. /deps/sinatra 0.9.6 rack:>= 0.9.1 1.0 rack:>= 1.0 1.1.4 rack:~> 1.1,tilt:<

    2.0&>= 1.2.2
  28. ! +

  29. Where we’re at designed the new index new index from

    bundler-api new index in bundler
  30. What’s left to do new index from rubygems.org ! new

    index in rubygems !
  31. What’s left to do new index from rubygems.org new index

    in rubygems fastly-hosted index files
  32. The Present gem install bundler

  33. bundle install -j4 Parallel gem installs export BUNDLE_JOBS=4

  34. Gem mirrors bundle config mirror.https://rubygems.org https://tokyo-m.rubygems.org

  35. New resolver! better errors faster installs many bugfixes

  36. The Future

  37. New index super fast installs! !

  38. you can help! andre@arko.net @indirect