Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Data driven dependency management

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Grey Baker
September 13, 2018

Data driven dependency management

Analysis of 10 years of RubySec data and 50,000 dependency upgrade PRs

Grey Baker

September 13, 2018

Transcript

11 slides
  1. • Grey Baker (@greysteil on GitHub, @greybaker on Twitter) •

    6 years programming Ruby • Bundler core team member (since August!) • 52,000 gem update PRs merged in the last 12 months
  2. • Dependabot automatically creates dependency update PRs • I built

    it, and work on it full-time • GoCardless, GOV.UK and thoughtbot use it. You should too • Creates interesting data since it updates one gem at a time
  3. Always up-to-date Late adopter Reactive STRATEGY 1 • Use the

    latest version at all times • For analysis assume they’re always 1 day behind STRATEGY 2 • Wait a month after new major or minor versions • Always on the latest patch release STRATEGY 3 • Use the latest version when you add, then forget • Equally likely to have added or updated the gem at any time in the last year Three common gem strategies
  4. • 10 years of vulnerability advisories • Includes the date

    of disclosure • Allows us to determine whether a given version is secure The data to test them Ruby Advisory Database • History of gem releases over time • Includes date of each release • Tells us which version each strategy would have been using at any time Rubygems API • Combining, we can tell what version each strategy would have been on at the time a vulnerability was disclosed, and whether that version is secure • Full analysis at https://github.com/dependabot/gem-vulnerability-analysis
  5. Always keeping your gems up-to- date is the most secure

    strategy (just) Required response to a new vulnerability disclosure Always up-to-date Late adopter Reactive 47% 47% 47% 3% 4% 4% 42% 35% 35% 8% 15% 16% Not affected Fix by upgrading Fix by downgrading No fix on day zero • In addition, remember, the late adopter has a harder job with “fix by upgrading” • Full analysis at https://github.com/dependabot/gem-vulnerability-analysis
  6. Major update a.b.c -> x.0.0 Minor update x.b.c -> x.y.0

    Patch x.y.c -> x.y.z • Based on CI info for 1,750 updates • Many major updates are just dropping old Ruby support, etc. Minor and patch versions rarely have any incompatibilities or new bugs • Based on CI info for 12,000 updates • Pre-1.0.0 updates excluded • Based on CI info for 17,000 updates • Pre-1.0.0 updates excluded Updates with passing CI Notes • SemVer might not work in theory, but it does work in practice • Minor updates are nearly as easy to upgrade as patch releases

Other Decks in Technology