A History of Bundles: 2010 to 2017

When Bundler 1.0 came out in 2010, it did something really great: installed all of your gems and let you use them in your app. Today, Bundler does something really great: it installs all your gems and lets you use them. So, given that, why has Bundler needed thousands upon thousands of hours of development work? What exactly has changed since then? Prepare to find out. We’ll cover performance improvements, server response optimizations, adapting to new versions of Ruby, and adding features to support new usecases. Learn the tricks of Bundler power users, and find out how to optimize your gem workfl

Transcript

1. A History of Bundles
   the story of Ruby’s dependency manager
   

   View Slide

2. André Arko
   @indirect
   

   View Slide

3. View Slide

4. therubyway.io
   

   View Slide

5. View Slide

6. View Slide

7. The Road to 1.0
   (2008-2010)
   

   View Slide

8. installing gems
   was easy
   

   View Slide

9. upgrading gems
   was hard
   

   View Slide

10. How Does Bundler
    Work, Anyway?
    see also
    André Arko, RubyConf 2015
    

    View Slide

11. Bundler was made
    for a very specific problem
    

    View Slide

12. two insights
    the design was driven by
    

    View Slide

13. dependency resolver
    okay, what is a
    

    View Slide

14. install-time
    sounds good, what about
    

    View Slide

15. lockfile
    neat, how about the
    

    View Slide

16. Bundler
    together, these things are
    

    View Slide

17. git gems
    were new and very cool
    

    View Slide

18. bundle gem
    another new thing was
    

    View Slide

19. Bundler: Painless
    Dependency Management
    see also
    André Arko, RailsConf 2010
    

    View Slide

20. Now It’s Too Slow
    (2010-2012)
    

    View Slide

21. at first
    we just wanted it to work
    

    View Slide

22. no one had giant apps
    when it was new,
    

    View Slide

23. small apps were also slow
    sadly, for different reasons,
    

    View Slide

24. @qrush wrote a new API
    a hero rises
    

    View Slide

25. `bundle install` Y U SO SLOW
    see also
    terence lee & andré arko, ruby on ales 2012
    

    View Slide

26. lots less data
    lots more requests, but
    

    View Slide

27. lots more requests
    lots less data, but
    

    View Slide

28. cool new stuff!
    

    View Slide

29. bundle clean
    cool new stuff!
    

    View Slide

30. bundle clean
    bundle outdated
    cool new stuff!
    

    View Slide

31. bundle clean
    bundle outdated
    bundle cache --all
    cool new stuff!
    

    View Slide

32. git clone https://github.com/foo/foo
    gem “foo”, github: “foo/foo”
    bundle config local.foo ~/path/to/foo
    git local development
    

    View Slide

33. ruby “1.9.3”
    application ruby versions
    

    View Slide

34. Victims of Our Own Success
    (2012-2014)
    

    View Slide

35. we DDoSed RubyGems.org
    so many new Bundler users
    

    View Slide

36. Bundler API
    a completely new
    

    View Slide

37. (that’s very nearly almost what
    Jamie Zawinski said, anyway)
    Some people, when confronted
    with a problem, think “I know,
    I'll write a webapp and throw it
    up on Heroku.” Now they have
    two problems.
    

    View Slide

38. sets of problems
    completely new
    

    View Slide

39. less contributors
    less popular technology
    

    View Slide

40. Deathmatch: Bundler vs. RubyGems
    see also
    `bundle install` Y U SO SLOW: Server Edition
    André Arko, Scottish RubyConf 2013
    Terence Lee, Ruby on Ales 2013
    

    View Slide

41. still, some cool new stuff!
    

    View Slide

42. multithreaded installs
    still, some cool new stuff!
    

    View Slide

43. multithreaded installs
    non-recursive resolver
    still, some cool new stuff!
    

    View Slide

44. multithreaded installs
    non-recursive resolver
    https support for git
    still, some cool new stuff!
    

    View Slide

45. bundler’s first CVE
    also some uncool stuff:
    

    View Slide

46. multiple gem sources
    mean your app can be attacked
    

    View Slide

47. after the first source,
    use source blocks to stay safe
    source “https://rubygems.org"
    gem “some_gem”
    source “https://other-server.com" do
    gem “other_gem”
    end
    

    View Slide

48. A New Hope
    (2015-2017)
    

    View Slide

49. as devs burned out,
    community funding appeared
    

    View Slide

50. Ruby Central
    project grants
    

    View Slide

51. Stripe
    open source grants
    

    View Slide

52. Stripe and Engine Yard
    Bundler project funding
    

    View Slide

53. Ruby Together
    a non-profit trade association
    

    View Slide

54. funded by
    viewers like you
    rubytogether.org/join
    

    View Slide

55. funded work
    yielded serious progress
    

    View Slide

56. now using Fastly
    for all requests to RubyGems.org
    

    View Slide

57. Bundler API merger
    back into RubyGems.org
    

    View Slide

58. the compact index
    gem metadata format
    

    View Slide

59. Extreme Makeover:
    RubyGems Edition
    André Arko, RubyConf 2013
    see also
    

    View Slide

60. bundle install
    now, finally, sometimes fast!
    

    View Slide

61. other notable features
    

    View Slide

62. Gemfile → gems.rb (optional)
    other notable features
    

    View Slide

63. Gemfile → gems.rb (optional)
    Ruby version locked, upgradable
    other notable features
    

    View Slide

64. Gemfile → gems.rb (optional)
    Ruby version locked, upgradable
    bundle lock + --add-platform
    other notable features
    

    View Slide

65. Gemfile → gems.rb (optional)
    Ruby version locked, upgradable
    bundle lock + --add-platform
    bundle doctor
    other notable features
    

    View Slide

66. Gemfile → gems.rb (optional)
    Ruby version locked, upgradable
    bundle lock + --add-platform
    bundle doctor
    bundle pristine
    other notable features
    

    View Slide

67. bundle update --major
    bundle update --minor
    bundle update --patch
    other notable features
    

    View Slide

68. bundle update --major
    bundle update --minor
    bundle update --patch
    bundle config mirror
    other notable features
    

    View Slide

69. bundle update --major
    bundle update --minor
    bundle update --patch
    bundle config mirror
    checksum validation on install
    other notable features
    

    View Slide

70. a plugin system! (beta)
    • command plugins:
    bundle ack
    • lifecycle plugins:
    before/after install/update, etc
    • source plugins:
    gem “foo”, mercurial: “https://example.com”
    other notable features
    

    View Slide

71. The Future
    (2017-????)
    

    View Slide

72. Bundler 2
    see also
    Colby Swandale, Ruby Kaigi 2017
    

    View Slide

73. best practices
    bundle config only_update_to_newer_versions true
    

    View Slide

74. best practices
    bundle config disable_multisource true
    bundle config only_update_to_newer_versions true
    

    View Slide

75. best practices
    bundle config specific_platform true
    bundle config disable_multisource true
    bundle config only_update_to_newer_versions true
    

    View Slide

76. best practices
    bundle config global_gem_cache true
    bundle config specific_platform true
    bundle config disable_multisource true
    bundle config only_update_to_newer_versions true
    

    View Slide

77. best practices
    bundle config default_install_uses_path true
    bundle config global_gem_cache true
    bundle config specific_platform true
    bundle config disable_multisource true
    bundle config only_update_to_newer_versions true
    

    View Slide

78. best practices
    override GitHub gems to use HTTPS
    with this line at the top of your Gemfile
    git_source(:github) {|r| “https://github.com/#{r}" }
    (or use bundle init to generate a Gemfile with this)
    

    View Slide

79. power user tools
    

    View Slide

80. power user tools
    instead of
    $ bundle exec rspec
    you can run
    $ bundle binstubs rspec-core
    once, then commit (and use) the stub
    $ bin/rspec
    

    View Slide

81. power user tools
    bundle viz
    

    View Slide

82. power user tools
    $ bundle lock --add-platform java
    $ git add Gemfile.lock
    $ git commit -am “Locked for JRuby”
    

    View Slide

83. power user tools
    $ git clone github.com/foo/foo ~/src/foo
    $ bundle config local.foo ~/src/foo
    [ make changes in ~/src/foo ]
    $ bundle exec rails s
    [ commit changes in ~/src/foo ]
    $ bundle exec rails s
    $ git add Gemfile.lock
    $ git commit -m “Use the latest foo”
    

    View Slide

84. power user tools
    bundler inline
    

    View Slide

85. power user tools
    $ cat http.rb
    require 'bundler/inline'
    gemfile do
    source 'https://rubygems.org'
    gem 'http'
    end
    puts HTTP.get('http://example.com')
    

    View Slide

86. $ gem uninstall http
    Successfully uninstalled http-3.0.0
    $ ruby http.rb
    
    
    […]
    power user tools
    needed gems installed here
    

    View Slide

87. power user tools
    search your gems with
    $ grep -R STRING $(bundle show --paths)
    then, open the gem you found in $EDITOR with
    $ bundle open GEMNAME
    make changes as needed for debugging, then
    $ bundle pristine GEMNAME
    

    View Slide

88. power user tools
    silence gem install messages with
    $ bundle config --system ignore_messages true
    

    View Slide

89. power user tools
    silence gem install messages with
    $ bundle config --system ignore_messages true
    and you will never be told to HTTParty hard, ever again
    

    View Slide

90. the end… so far
    read this talk at arko.net
    questions? tweet at @indirect!
    

    View Slide