Can my friends come too?

Presented at Brighton Ruby Conf 2017

Transcript

1. Can my friends come too?
   Brighton Ruby 2017
   

   View Slide

2. Andrew Nesbitt
   Hello!
   @teabass
   

   View Slide

3. Libraries.io Open Data
   This talk is powered by
   https://libraries.io
   

   View Slide

4. View Slide

5. Libraries.io indexes
   2.3 million libraries from
   33 package managers
   

   View Slide

6. Libraries.io indexes
   25 million repos from
   GitHub, GitLab and Bitbucket
   

   View Slide

7. 96 million links between
   repositories and libraries
   Libraries.io indexes
   It’s like Google PageRank for Software
   

   View Slide

8. 25GB Open Source Metadata
   Last month we released
   https://libraries.io/data
   

   View Slide

9. Open Source is huge!
   2017
   

   View Slide

10. Avg 10,000 versions every day
    Libraries.io statistics
    Avg 2,000 brand new projects every day
    

    View Slide

11. 78% of companies say their
    customer facing software is
    built on Open Source
    According to a 2015 survey
    https://www.blackducksoftware.com/2015-future-of-open-source
    

    View Slide

12. Stop reinventing the wheel
    Open Source helps us
    

    View Slide

13. Share knowledge
    Open Source helps us
    

    View Slide

14. Focus on the New and Unique
    Open Source helps us
    

    View Slide

15. The Shoulders of Giants
    We’re standing on
    

    View Slide

16. Free as in Freedom
    Open Source is
    Free to run, copy, distribute, study, change and improve
    

    View Slide

17. There is a problem…
    but
    

    View Slide

18. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
    KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
    WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
    PURPOSE AND NON INFRINGEMENT. IN NO EVENT SHALL THE AUTHORS
    OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
    OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
    OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
    SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
    Excerpt from the MIT License
    

    View Slide

19. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
    KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
    WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
    PURPOSE AND NON INFRINGEMENT. IN NO EVENT SHALL THE AUTHORS
    OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
    OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
    OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
    SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
    Excerpt from the MIT License
    

    View Slide

20. You are responsible for the
    open source software that you
    choose to use.
    That means
    

    View Slide

21. You are responsible for the
    open source software that you
    choose to use.
    That means
    

    View Slide

22. - Maintenance
    - Security
    - Licensing
    - Sustainability
    Evaluating Open Source
    

    View Slide

23. What are the hidden costs
    in using this software?
    Evaluating Open Source
    

    View Slide

24. Software doesn’t
    exist in a vacuum
    

    View Slide

25. View Slide

26. Software doesn’t
    exist in a vacuum
    

    View Slide

27. https://twitter.com/ourfounder/status/770075137332932608
    

    View Slide

28. - Testing against 3rd party API changes
    - Security reviews
    - Checking against new language features
    - Triaging issues and support requests
    - Review performance for regressions
    - Updating dependencies
    - and more…
    Software regularly needs
    

    View Slide

29. https://twitter.com/davecheney/status/616931340466786304
    

    View Slide

30. Free as in Puppy
    Open Source is
    

    View Slide

31. View Slide

32. Blossom Mabel Felix
    

    View Slide

33. Blossom Mabel Felix
    Poppy*
    

    View Slide

34. - Feeding
    - Walking
    - Grooming
    - Worming
    - Flea treatment
    - and more…
    Puppies regularly need
    

    View Slide

35. https://twitter.com/teabass/status/882895864586547201
    

    View Slide

36. “Refactored”
    

    View Slide

37. View Slide

38. “Legacy code”
    

    View Slide

39. “I get lost in the carpet”
    

    View Slide

40. Dependencies
    Let’s talk about
    

    View Slide

41. Software that your software
    needs build, test or run
    Dependencies:
    

    View Slide

42. - Libraries
    - Frameworks
    - Languages
    - Databases
    - APIs
    - Operating systems
    Dependencies:
    

    View Slide

43. gems that our Ruby code
    requires to run correctly
    For this talk lets focus on
    

    View Slide

44. 133,797 gems
    Rubygems.org statistics
    Collectively downloaded 14,749,546,331 times
    

    View Slide

45. 658,499 Gemfiles
    From Libraries.io
    Found across open source repos on Github, GitLab and Bitbucket
    

    View Slide

46. Avg 11 gems per Gemfile
    From Libraries.io
    7,413,699 total across all open source repositories
    

    View Slide

47. Transitive dependencies
    We have to go deeper
    

    View Slide

48. Puppy on Rails
    

    View Slide

49. Puppy on Rails
    

    View Slide

50. Puppy on Rails
    PuppyRecord
    

    View Slide

51. Puppy on Rails
    PuppyRecord ActionPupper
    

    View Slide

52. Puppy on Rails
    PuppyRecord Pupogiri
    ActionPupper
    

    View Slide

53. Puppy on Rails
    

    View Slide

54. Puppy on Rails
    

    View Slide

55. Puppy on Rails
    

    View Slide

56. Puppy on Rails
    

    View Slide

57. Puppy on Rails
    *head size not an indicator of lines of code
    

    View Slide

58. 481,483 Gemfile.lock
    From Libraries.io
    Found in open source repos on Github, GitLab and Bitbucket
    

    View Slide

59. Avg 52 gems per Gemfile.lock
    From Libraries.io
    24,805,249 total across all open source repositories
    

    View Slide

60. View Slide

61. - Maintenance
    - Security
    - Licensing
    - Sustainability
    Evaluating Open Source
    

    View Slide

62. Maintenance
    Evaluating Dependencies
    Looking after your new pet
    

    View Slide

63. 46% gems haven’t been
    updated in over 3 years
    Maintenance
    

    View Slide

64. Time since last release
    over 3 years
    3 years
    2 years
    1 year
    6 months
    1 month
    Gems
    0 17500 35000 52500 70000
    

    View Slide

65. Bus Factor
    Maintenance
    Higher is better
    

    View Slide

66. How many people can publish a bugfix?
    Maintenance
    Who’s going to walk the dog?
    

    View Slide

67. Gems
    0
    30000
    60000
    90000
    120000
    Owners
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 24 28 30 31 32
    Owners per Gem
    

    View Slide

68. 90% gems have a bus factor 1
    Maintenance
    Average gem has 1.2 owners
    

    View Slide

69. Security
    Evaluating Dependencies
    Dogs must be kept on a lead at all times
    

    View Slide

70. ruby-advisory-db
    https://rubysec.com
    Security
    287 advisories across 147 gems
    

    View Slide

71. $ gem install bundler-audit
    There’s a gem for that
    Automatically check your Gemfile.lock for security issues
    

    View Slide

72. $ gem install
    arbitrary code execution
    via extconf.rb
    Security
    http://incolumitas.com/2016/06/08/typosquatting-package-managers/
    

    View Slide

73. Every gem is a potential
    attack vector
    Security
    It’s not a bug, it’s a feature!
    

    View Slide

74. View Slide

75. Licensing
    Evaluating Dependencies
    I am not a lawyer
    

    View Slide

76. Open Source License compatibility
    https://timreview.ca/article/416
    

    View Slide

77. Unlicensed code
    Copyright by default
    Like walking someone else’s dog without permission
    

    View Slide

78. 28% have no license declared
    Rubygems.org statistic
    37,547 gems
    

    View Slide

79. $ gem install license_finder
    There’s a gem for that
    Automatically check your dependencies for license issues
    

    View Slide

80. Sustainability
    Evaluating Dependencies
    How are you going to pay for the dog food?
    

    View Slide

81. https://twitter.com/mperham/status/880835731874168832
    

    View Slide

82. - Volunteer time
    - Consulting
    - Sponsorship
    - Dual licensing
    - Advertising
    - Training
    - Bounties
    - SaaS
    - Venture Capital
    How is support funded?
    

    View Slide

83. Roads and Bridges:
    The Unseen Labor Behind Our
    Digital Infrastructure
    https://www.fordfoundation.org/library/reports-and-studies/roads-and-bridges-the-unseen-labor-behind-
    our-digital-infrastructure/
    Nadia Eghbal
    

    View Slide

84. Javascript
    The elephant in the room
    It’s puppies all the way down
    

    View Slide

85. 478,953 modules
    NPM statistics
    Downloads over 1 billion times per week
    

    View Slide

86. 1,527,361 package.json
    NPM usage statistics
    Found in 2,556,333 open source repos on Github, GitLab and Bitbucket
    

    View Slide

87. Avg 12 modules per package.json
    NPM usage statistics
    18,890,641 total across all open source repositories
    

    View Slide

88. 53,720 lockfiles
    NPM usage statistics
    package-lock.json, npm-shrinkwrap.json and yarn.lock
    

    View Slide

89. Avg 307 modules per lockfile
    NPM usage statistics
    Javascript dependency trees are 6 times bigger than ruby
    

    View Slide

90. View Slide

91. $ npm install
    Arbitrary code execution
    NPM Security
    https://www.infoq.com/news/2016/03/npm-infection
    

    View Slide

92. 6 times more
    chance of attack
    NPM Security
    https://www.infoq.com/news/2016/03/npm-infection
    “It’s not a bug, it’s a feature”
    

    View Slide

93. View Slide

94. Final thoughts
    Wrapping up
    A puppy is for life, not just for christmas
    

    View Slide

95. Vet your dependencies
    Wrapping up
    Over using a metaphor much?
    

    View Slide

96. Review your dependencies
    Wrapping up
    Check for updates and issues on a regular basis
    

    View Slide

97. Prune your dependencies
    Wrapping up
    Remove unused dependencies
    

    View Slide

98. Support your dependencies
    Wrapping up
    Help keep Ruby sustainable
    

    View Slide

99. Libraries.io Open Data
    This talk was powered by
    https://libraries.io/data
    

    View Slide

100. Small Title Text
     Subtitle text
     Thanks!
     

     View Slide