Structure and Evolution of Package Dependency Networks

Presentation given to MSR 2017 about the network characteristics of package managers such as NPM, RubyGems and Cargo

Transcript

1. Structure and Evolution of Package Dependency Networks Riivo Kikas, Georgios Gousios, Marlon Dumas, Dietmar Pfahl
2. Software reuse • System reuse • Application reuse • Component reuse • Libraries • COTS components • OSS packages through dependency management • Object/Function reuse
3. Dependency management library dependencies
4. Package dependency networks
 csv-parser lists dependencies, among other ndjson ndjson has its own list of dependencies
5. A network of dependency relationships Subset of Rust packages mid-2016
6. How do dependency networks look like? How do dependency networks evolve? How resilient are dependency networks to attacks?
7. How do dependency networks look like? How do dependency networks evolve? How resilient are dependency networks to attacks?
8. How do dependency networks look like? How do dependency networks evolve? How resilient are dependency networks to attacks?
9. Ecosystems studied 254,466 84,987 122,786 62,133 11,037 388,289 147,120 JavaScript Ruby Rust
10. Dependency network construction No versions A C B D E
 A
 0.1 B
 0.3 C
 0.5 E
 0.1 Latest versions A
 0.1 C
 0.4 B
 0.3 C
 0.5 D
 0.2 E
 0.1 All versions
11. A
 0.1 C
 0.4 B
 0.3 D
 0.2 E
 0.1 H
 0.6 G
 0.1 F
 0.1 L
 0.1 J
 0.2.6 K
 0.4 F
 0.2
12. A
 0.1 C
 0.4 B
 0.3 D
 0.2 E
 0.1 H
 0.6 G
 0.1 F
 0.1 L
 0.1 J
 0.2.6 K
 0.4 dependents F
 0.2
13. A
 0.1 C
 0.4 B
 0.3 D
 0.2 E
 0.1 H
 0.6 G
 0.1 F
 0.1 L
 0.1 J
 0.2.6 K
 0.4 dependents transitive dependents F
 0.2
14. A
 0.1 C
 0.4 B
 0.3 D
 0.2 E
 0.1 H
 0.6 G
 0.1 F
 0.1 L
 0.1 J
 0.2.6 K
 0.4 dependents transitive dependents dependencies F
 0.2
15. A
 0.1 C
 0.4 B
 0.3 D
 0.2 E
 0.1 H
 0.6 G
 0.1 F
 0.1 L
 0.1 J
 0.2.6 K
 0.4 dependents transitive dependents dependencies transitive dependencies F
 0.2
16. Static properties
 Mean number of dependents Direct Transitive Ratio JavaScript 1.3 15.5 11.9x Ruby 1.2 6.4 5.3x Rust 1.6 7.4 4.0x
17. Ecosystem growth - Dependents JavaScript/NPM dependents is growing at a tremendous speed Ruby is slowing down
18. Static properties
 Mean number of dependencies Direct Transitive Ratio JavaScript 5.5 54.6 9.9x Ruby 8.7 34.1 3.9x Rust 3 9.3 3.1x
19. Ecosystem growth - Dependencies JavaScript/NPM dependencies is growing at a tremendous speed Ruby is slowing down
20. Vulnerability Fraction of nodes affected by the removal of single package / version 2005 2007 2009 2011 2013 2015 0.000 0.002 0.004 0.006 0.008 0.010 0.012 0.014 9ulnerability rate ASSlicatiRn Pean J6 3ackaJe Pean J6 ASSlicatiRn Pean 5uby 3ackaJe Pean 5uby
21. Targeted attacks Number of transitive dependents for the 5 most “vulnerable” packages > 450k > 300k inherits.js, erubis, rack string_decoder, sigmund, is_array
22. Targeted attacks Number of transitive dependents for the 5 most “vulnerable” packages > 450k > 300k string_decoder, sigmund, is_array 30 LOCs! inherits.js, erubis, rack
23. Targeted attacks Number of transitive dependents for the 5 most “vulnerable” packages > 450k > 300k string_decoder, sigmund, is_array Last Commit: 2012 inherits.js, erubis, rack
24. Ecosystem response to a CVE CVE-2015-3225: DOS via request with large parameter depth
25. What can developers do?
26. What can developers do? • Understand the effect of including a dependency • For small, inherits.js-like packages just re-implement it • Actively monitor vulnerabilities in the transitive closure • More intelligent, integrated tools • Better governance of dependency management practices
27. What can researchers do?
28. What can researchers do? • Better tools: • increase visibility of transitive includes • connect ecosystems to security advisories • dependency health and ecosystem stability ratings • Better analysis: understand which parts of the dependency code are actually used • A semantic versioning system that everybody agrees upon • Qualitative work: How developers approach dependency management? • Replicate in other ecosystems
29. riivo/package-dependency-networks @riivo @gousiosg