Share
Embed
Data driven dependency management
Analysis of 10 years of RubySec data and 50,000 dependency upgrade PRs
Grey Baker
September 13, 2018
Transcript
11 slides
-
Data driven dependency management Analysing 10 years of RubySec data,
and 50,000 upgrade PRs -
• Grey Baker (@greysteil on GitHub, @greybaker on Twitter) •
6 years programming Ruby • Bundler core team member (since August!) • 52,000 gem update PRs merged in the last 12 months -
• Dependabot automatically creates dependency update PRs • I built
it, and work on it full-time • GoCardless, GOV.UK and thoughtbot use it. You should too • Creates interesting data since it updates one gem at a time -
Always up-to-date Late adopter Reactive STRATEGY 1 • Use the
latest version at all times • For analysis assume they’re always 1 day behind STRATEGY 2 • Wait a month after new major or minor versions • Always on the latest patch release STRATEGY 3 • Use the latest version when you add, then forget • Equally likely to have added or updated the gem at any time in the last year Three common gem strategies -
• 10 years of vulnerability advisories • Includes the date
of disclosure • Allows us to determine whether a given version is secure The data to test them Ruby Advisory Database • History of gem releases over time • Includes date of each release • Tells us which version each strategy would have been using at any time Rubygems API • Combining, we can tell what version each strategy would have been on at the time a vulnerability was disclosed, and whether that version is secure • Full analysis at https://github.com/dependabot/gem-vulnerability-analysis -
Always keeping your gems up-to- date is the most secure
strategy (just) Required response to a new vulnerability disclosure Always up-to-date Late adopter Reactive 47% 47% 47% 3% 4% 4% 42% 35% 35% 8% 15% 16% Not affected Fix by upgrading Fix by downgrading No fix on day zero • In addition, remember, the late adopter has a harder job with “fix by upgrading” • Full analysis at https://github.com/dependabot/gem-vulnerability-analysis -
Major update a.b.c -> x.0.0 Minor update x.b.c -> x.y.0
Patch x.y.c -> x.y.z • Based on CI info for 1,750 updates • Many major updates are just dropping old Ruby support, etc. Minor and patch versions rarely have any incompatibilities or new bugs • Based on CI info for 12,000 updates • Pre-1.0.0 updates excluded • Based on CI info for 17,000 updates • Pre-1.0.0 updates excluded Updates with passing CI Notes • SemVer might not work in theory, but it does work in practice • Minor updates are nearly as easy to upgrade as patch releases -
• Thanks for listening! • Security analysis available at: https://github.com/dependabot/gem-vulnerability-analysis
• Check out Dependabot at: https://dependabot.com