Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A History of Bundles: 2010 to 2017

André Arko
November 16, 2017

A History of Bundles: 2010 to 2017

When Bundler 1.0 came out in 2010, it did something really great: installed all of your gems and let you use them in your app. Today, Bundler does something really great: it installs all your gems and lets you use them. So, given that, why has Bundler needed thousands upon thousands of hours of development work? What exactly has changed since then? Prepare to find out. We’ll cover performance improvements, server response optimizations, adapting to new versions of Ruby, and adding features to support new usecases. Learn the tricks of Bundler power users, and find out how to optimize your gem workfl

André Arko

November 16, 2017
Tweet

More Decks by André Arko

Other Decks in Technology

Transcript

  1. A History of Bundles
    the story of Ruby’s dependency manager

    View Slide

  2. André Arko
    @indirect

    View Slide

  3. View Slide

  4. therubyway.io

    View Slide

  5. View Slide

  6. View Slide

  7. The Road to 1.0
    (2008-2010)

    View Slide

  8. installing gems
    was easy

    View Slide

  9. upgrading gems
    was hard

    View Slide

  10. How Does Bundler
    Work, Anyway?
    see also
    André Arko, RubyConf 2015

    View Slide

  11. Bundler was made
    for a very specific problem

    View Slide

  12. two insights
    the design was driven by

    View Slide

  13. dependency resolver
    okay, what is a

    View Slide

  14. install-time
    sounds good, what about

    View Slide

  15. lockfile
    neat, how about the

    View Slide

  16. Bundler
    together, these things are

    View Slide

  17. git gems
    were new and very cool

    View Slide

  18. bundle gem
    another new thing was

    View Slide

  19. Bundler: Painless
    Dependency Management
    see also
    André Arko, RailsConf 2010

    View Slide

  20. Now It’s Too Slow
    (2010-2012)

    View Slide

  21. at first
    we just wanted it to work

    View Slide

  22. no one had giant apps
    when it was new,

    View Slide

  23. small apps were also slow
    sadly, for different reasons,

    View Slide

  24. @qrush wrote a new API
    a hero rises

    View Slide

  25. `bundle install` Y U SO SLOW
    see also
    terence lee & andré arko, ruby on ales 2012

    View Slide

  26. lots less data
    lots more requests, but

    View Slide

  27. lots more requests
    lots less data, but

    View Slide

  28. cool new stuff!

    View Slide

  29. bundle clean
    cool new stuff!

    View Slide

  30. bundle clean
    bundle outdated
    cool new stuff!

    View Slide

  31. bundle clean
    bundle outdated
    bundle cache --all
    cool new stuff!

    View Slide

  32. git clone https://github.com/foo/foo
    gem “foo”, github: “foo/foo”
    bundle config local.foo ~/path/to/foo
    git local development

    View Slide

  33. ruby “1.9.3”
    application ruby versions

    View Slide

  34. Victims of Our Own Success
    (2012-2014)

    View Slide

  35. we DDoSed RubyGems.org
    so many new Bundler users

    View Slide

  36. Bundler API
    a completely new

    View Slide

  37. (that’s very nearly almost what
    Jamie Zawinski said, anyway)
    Some people, when confronted
    with a problem, think “I know,
    I'll write a webapp and throw it
    up on Heroku.” Now they have
    two problems.

    View Slide

  38. sets of problems
    completely new

    View Slide

  39. less contributors
    less popular technology

    View Slide

  40. Deathmatch: Bundler vs. RubyGems
    see also
    `bundle install` Y U SO SLOW: Server Edition
    André Arko, Scottish RubyConf 2013
    Terence Lee, Ruby on Ales 2013

    View Slide

  41. still, some cool new stuff!

    View Slide

  42. multithreaded installs
    still, some cool new stuff!

    View Slide

  43. multithreaded installs
    non-recursive resolver
    still, some cool new stuff!

    View Slide

  44. multithreaded installs
    non-recursive resolver
    https support for git
    still, some cool new stuff!

    View Slide

  45. bundler’s first CVE
    also some uncool stuff:

    View Slide

  46. multiple gem sources
    mean your app can be attacked

    View Slide

  47. after the first source,
    use source blocks to stay safe
    source “https://rubygems.org"
    gem “some_gem”
    source “https://other-server.com" do
    gem “other_gem”
    end

    View Slide

  48. A New Hope
    (2015-2017)

    View Slide

  49. as devs burned out,
    community funding appeared

    View Slide

  50. Ruby Central
    project grants

    View Slide

  51. Stripe
    open source grants

    View Slide

  52. Stripe and Engine Yard
    Bundler project funding

    View Slide

  53. Ruby Together
    a non-profit trade association

    View Slide

  54. funded by
    viewers like you
    rubytogether.org/join

    View Slide

  55. funded work
    yielded serious progress

    View Slide

  56. now using Fastly
    for all requests to RubyGems.org

    View Slide

  57. Bundler API merger
    back into RubyGems.org

    View Slide

  58. the compact index
    gem metadata format

    View Slide

  59. Extreme Makeover:
    RubyGems Edition
    André Arko, RubyConf 2013
    see also

    View Slide

  60. bundle install
    now, finally, sometimes fast!

    View Slide

  61. other notable features

    View Slide

  62. Gemfile → gems.rb (optional)
    other notable features

    View Slide

  63. Gemfile → gems.rb (optional)
    Ruby version locked, upgradable
    other notable features

    View Slide

  64. Gemfile → gems.rb (optional)
    Ruby version locked, upgradable
    bundle lock + --add-platform
    other notable features

    View Slide

  65. Gemfile → gems.rb (optional)
    Ruby version locked, upgradable
    bundle lock + --add-platform
    bundle doctor
    other notable features

    View Slide

  66. Gemfile → gems.rb (optional)
    Ruby version locked, upgradable
    bundle lock + --add-platform
    bundle doctor
    bundle pristine
    other notable features

    View Slide

  67. bundle update --major
    bundle update --minor
    bundle update --patch
    other notable features

    View Slide

  68. bundle update --major
    bundle update --minor
    bundle update --patch
    bundle config mirror
    other notable features

    View Slide

  69. bundle update --major
    bundle update --minor
    bundle update --patch
    bundle config mirror
    checksum validation on install
    other notable features

    View Slide

  70. a plugin system! (beta)
    • command plugins:
    bundle ack
    • lifecycle plugins:
    before/after install/update, etc
    • source plugins:
    gem “foo”, mercurial: “https://example.com”
    other notable features

    View Slide

  71. The Future
    (2017-????)

    View Slide

  72. Bundler 2
    see also
    Colby Swandale, Ruby Kaigi 2017

    View Slide

  73. best practices
    bundle config only_update_to_newer_versions true

    View Slide

  74. best practices
    bundle config disable_multisource true
    bundle config only_update_to_newer_versions true

    View Slide

  75. best practices
    bundle config specific_platform true
    bundle config disable_multisource true
    bundle config only_update_to_newer_versions true

    View Slide

  76. best practices
    bundle config global_gem_cache true
    bundle config specific_platform true
    bundle config disable_multisource true
    bundle config only_update_to_newer_versions true

    View Slide

  77. best practices
    bundle config default_install_uses_path true
    bundle config global_gem_cache true
    bundle config specific_platform true
    bundle config disable_multisource true
    bundle config only_update_to_newer_versions true

    View Slide

  78. best practices
    override GitHub gems to use HTTPS
    with this line at the top of your Gemfile
    git_source(:github) {|r| “https://github.com/#{r}" }
    (or use bundle init to generate a Gemfile with this)

    View Slide

  79. power user tools

    View Slide

  80. power user tools
    instead of
    $ bundle exec rspec
    you can run
    $ bundle binstubs rspec-core
    once, then commit (and use) the stub
    $ bin/rspec

    View Slide

  81. power user tools
    bundle viz

    View Slide

  82. power user tools
    $ bundle lock --add-platform java
    $ git add Gemfile.lock
    $ git commit -am “Locked for JRuby”

    View Slide

  83. power user tools
    $ git clone github.com/foo/foo ~/src/foo
    $ bundle config local.foo ~/src/foo
    [ make changes in ~/src/foo ]
    $ bundle exec rails s
    [ commit changes in ~/src/foo ]
    $ bundle exec rails s
    $ git add Gemfile.lock
    $ git commit -m “Use the latest foo”

    View Slide

  84. power user tools
    bundler inline

    View Slide

  85. power user tools
    $ cat http.rb
    require 'bundler/inline'
    gemfile do
    source 'https://rubygems.org'
    gem 'http'
    end
    puts HTTP.get('http://example.com')

    View Slide

  86. $ gem uninstall http
    Successfully uninstalled http-3.0.0
    $ ruby http.rb


    […]
    power user tools
    needed gems installed here

    View Slide

  87. power user tools
    search your gems with
    $ grep -R STRING $(bundle show --paths)
    then, open the gem you found in $EDITOR with
    $ bundle open GEMNAME
    make changes as needed for debugging, then
    $ bundle pristine GEMNAME

    View Slide

  88. power user tools
    silence gem install messages with
    $ bundle config --system ignore_messages true

    View Slide

  89. power user tools
    silence gem install messages with
    $ bundle config --system ignore_messages true
    and you will never be told to HTTParty hard, ever again

    View Slide

  90. the end… so far
    read this talk at arko.net
    questions? tweet at @indirect!

    View Slide