Structure and Evolution of Package Dependency Networks

Presentation given to MSR 2017 about the network characteristics of package managers such as NPM, RubyGems and Cargo

Transcript

1. Structure and Evolution of
   Package Dependency
   Networks
   Riivo Kikas, Georgios Gousios,
   Marlon Dumas, Dietmar Pfahl
   

   View Slide

2. Software reuse
   • System reuse
   • Application reuse
   • Component reuse
   • Libraries
   • COTS components
   • OSS packages through dependency management
   • Object/Function reuse
   

   View Slide

3. Dependency management
   library
   dependencies
   

   View Slide

4. Package dependency networks

   csv-parser lists dependencies, among other ndjson
   ndjson has its own list of dependencies
   

   View Slide

5. A network of dependency
   relationships
   Subset of Rust packages mid-2016
   

   View Slide

6. How do dependency networks look like?
   How do dependency networks evolve?
   How resilient are dependency networks to attacks?
   

   View Slide

7. How do dependency networks look like?
   How do dependency networks evolve?
   How resilient are dependency networks to attacks?
   

   View Slide

8. How do dependency networks look like?
   How do dependency networks evolve?
   How resilient are dependency networks to attacks?
   

   View Slide

9. Ecosystems studied
   254,466 84,987
   122,786 62,133
   11,037
   388,289 147,120
   JavaScript
   Ruby
   Rust
   

   View Slide

10. Dependency network construction
    No versions
    A
    C
    B
    D
    E

    A

    0.1
    B

    0.3
    C

    0.5
    E

    0.1
    Latest versions
    A

    0.1
    C

    0.4
    B

    0.3
    C

    0.5
    D

    0.2
    E

    0.1
    All versions
    

    View Slide

11. A

    0.1
    C

    0.4
    B

    0.3
    D

    0.2
    E

    0.1
    H

    0.6
    G

    0.1
    F

    0.1
    L

    0.1
    J

    0.2.6
    K

    0.4
    F

    0.2
    

    View Slide

12. A

    0.1
    C

    0.4
    B

    0.3
    D

    0.2
    E

    0.1
    H

    0.6
    G

    0.1
    F

    0.1
    L

    0.1
    J

    0.2.6
    K

    0.4
    dependents
    F

    0.2
    

    View Slide

13. A

    0.1
    C

    0.4
    B

    0.3
    D

    0.2
    E

    0.1
    H

    0.6
    G

    0.1
    F

    0.1
    L

    0.1
    J

    0.2.6
    K

    0.4
    dependents
    transitive
    dependents
    F

    0.2
    

    View Slide

14. A

    0.1
    C

    0.4
    B

    0.3
    D

    0.2
    E

    0.1
    H

    0.6
    G

    0.1
    F

    0.1
    L

    0.1
    J

    0.2.6
    K

    0.4
    dependents
    transitive
    dependents
    dependencies
    F

    0.2
    

    View Slide

15. A

    0.1
    C

    0.4
    B

    0.3
    D

    0.2
    E

    0.1
    H

    0.6
    G

    0.1
    F

    0.1
    L

    0.1
    J

    0.2.6
    K

    0.4
    dependents
    transitive
    dependents
    dependencies transitive
    dependencies
    F

    0.2
    

    View Slide

16. Static properties

    Mean number of dependents
    Direct Transitive Ratio
    JavaScript 1.3 15.5 11.9x
    Ruby 1.2 6.4 5.3x
    Rust 1.6 7.4 4.0x
    

    View Slide

17. Ecosystem growth - Dependents
    JavaScript/NPM dependents is growing at a tremendous speed
    Ruby is slowing down
    

    View Slide

18. Static properties

    Mean number of dependencies
    Direct Transitive Ratio
    JavaScript 5.5 54.6 9.9x
    Ruby 8.7 34.1 3.9x
    Rust 3 9.3 3.1x
    

    View Slide

19. Ecosystem growth - Dependencies
    JavaScript/NPM dependencies is growing at a tremendous speed
    Ruby is slowing down
    

    View Slide

20. Vulnerability
    Fraction of nodes affected by the removal of single package / version
    2005 2007 2009 2011 2013 2015
    0.000
    0.002
    0.004
    0.006
    0.008
    0.010
    0.012
    0.014
    9ulnerability rate
    ASSlicatiRn Pean J6
    3ackaJe Pean J6
    ASSlicatiRn Pean 5uby
    3ackaJe Pean 5uby
    

    View Slide

21. Targeted attacks
    Number of transitive dependents for the 5 most
    “vulnerable” packages
    > 450k
    > 300k
    inherits.js, erubis, rack
    string_decoder, sigmund,
    is_array
    

    View Slide

22. Targeted attacks
    Number of transitive dependents for the 5 most
    “vulnerable” packages
    > 450k
    > 300k string_decoder, sigmund,
    is_array
    30 LOCs!
    inherits.js, erubis, rack
    

    View Slide

23. Targeted attacks
    Number of transitive dependents for the 5 most
    “vulnerable” packages
    > 450k
    > 300k string_decoder, sigmund,
    is_array
    Last Commit: 2012
    inherits.js, erubis, rack
    

    View Slide

24. Ecosystem response to a CVE
    CVE-2015-3225: DOS via request with large parameter depth
    

    View Slide

25. What can developers do?
    

    View Slide

26. What can developers do?
    • Understand the effect of including a dependency
    • For small, inherits.js-like packages just re-implement it
    • Actively monitor vulnerabilities in the transitive
    closure
    • More intelligent, integrated tools
    • Better governance of dependency management
    practices
    

    View Slide

27. What can researchers do?
    

    View Slide

28. What can researchers do?
    • Better tools:
    • increase visibility of transitive includes
    • connect ecosystems to security advisories
    • dependency health and ecosystem stability ratings
    • Better analysis: understand which parts of the
    dependency code are actually used
    • A semantic versioning system that everybody agrees upon
    • Qualitative work: How developers approach dependency
    management?
    • Replicate in other ecosystems
    

    View Slide

29. riivo/package-dependency-networks
    @riivo
    @gousiosg
    

    View Slide