DevOps Days Columbus - State of the Software Supply Chain

Transcript

1. Derek E. Weeks VP and DevOps Advocate, Sonatype Co-founder, All Day DevOps @weekstweets Exemplars, Laggards, and Hoarders A Data-Driven Look at Open Source Software Supply Chains
2. …once it ceases to sacrifice quality for speed C R E D I T : N E I L B E Y E R S D O R F
3. 2019: Nicole Forsgren
4. 2013: Gene Kim
5. Everyone has a software supply chain. (including open source projects)
6. OSS download volumes are a proxy for build automation. @weekstweets
7. 85% of your code is sourced from external suppliers @weekstweets
8. 313,000 java component downloads annually 2,778 Component suppliers 8,200 Component release 27,704 8.8% with known vulnerabilities @weekstweets
9. …faster is better in the enterprise
10. …faster is better for open source?
11. How do you pick the best suppliers?
12. Two Different Worlds Enterprise Open Source Multiple deploys per day Versioned releases Consistent development team Fluid group of developers Predictable, well-resourced Variable resource availability @weekstweets
13. With Similar Metrics Enterprise Open Source Deployment Frequency Release Frequency Organizational Performance Popularity Mean Time to Respond Time to Remediate Vulnerabilities @weekstweets
14. With Similar Metrics Enterprise Open Source Deployment Frequency Release Frequency Organizational Performance Popularity Mean Time to Restore Time to Remediate Vulnerabilities @weekstweets
15. @weekstweets
16. Attributes Measure Popularity Avg. daily Central Repository downloads Size of Team Avg. unique monthly contributors Development Speed Avg. commits per month Release Speed Avg. period between releases Presence of CI Presence of popular cloud CI systems Foundation Support Associated with an open source foundation Security More complicated Update Speed More complicated @weekstweets
17. @weekstweets Hypothesis 1 Projects that release frequently have better outcomes.
18. @weekstweets Hypothesis 1 Projects that release frequently have better outcomes. (VALIDATED)
19. Projects that release frequently: are 5x more popular. attract 79% more developers. have 12% greater foundation support rates. @weekstweets
20. With Similar Metrics Enterprise Open Source Deployment Frequency Release Frequency Organizational Performance Popularity Mean Time to Restore Time to Remediate Vulnerabilities @weekstweets
21. 1945: W. Edwards Deming
22. @weekstweets The Key Metrics: Time to Remediate Time to Update Stale Dependencies
23. Security: Time to Remediate (TTR) @weekstweets
24. Security: Time to Remediate (TTR) @weekstweets B Vulnerable Time
25. Security: Time to Remediate (TTR) @weekstweets C Vulnerable Time
26. Security: Time to Remediate (TTR) @weekstweets C Remediation Time
27. Security: Time to Update (TTU) @weekstweets C Update Time (for B)
28. Security: Time to Update (TTU) @weekstweets C Update Time (for A)
29. Security: Stale Dependencies @weekstweets Stale Dependency
30. Time to Remediate Vulnerabilities
31. Time to Remediate Vulnerabilities
32. Time to Remediate Vulnerabilities Do these update quickly in general?
33. Time to Remediate (TRR) vs. Time to Update (TTU) @weekstweets Most projects stay secure by staying up to date.
34. Hypothesis 2 Projects that update dependencies more frequently are generally more secure. @weekstweets
35. Hypothesis 2 Projects that update dependencies more frequently are generally more secure. (VALIDATED) @weekstweets
36. Hypothesis 3 Projects with fewer dependencies will stay more up to date. @weekstweets
37. Hypothesis 3 Projects with fewer dependencies will stay more up to date. (REJECTED) Components with more dependencies actually have better MTTU. @weekstweets
38. More dependencies correlate with larger development teams. @weekstweets Larger development teams have 50% faster MTTU and release 2.6x more frequently.
39. More dependencies correlate with larger development teams. @weekstweets Larger development teams have 50% faster MTTU and release 2.6x more frequently.
40. @weekstweets Hypothesis 4 More popular projects will be better about staying up to date.
41. 1999: Eric S. Raymond
42. @weekstweets Hypothesis 4 More popular projects will be better about staying up to date. (REJECTED) There are plenty of popular components with poor MTTU. Popularity does not correlate with MTTU.
43. 5 Behavioral Clusters @weekstweets Small Exemplar (606) Large Exemplar (595) Laggards (521) Features First (280) Cautious (429) Small development teams (1.6 devs), exemplary MTTU. Large development teams (8.9 devs), exemplary MTTU, very likely to be foundation supported, 11x more popular. Poor MTTU, high stale dependency count, more likely to be commercially supported. Frequent releases, but poor TTU. Still reasonably popular. Good TTU, but seldom completely up to date. Rest of the population: 8,142
44. Exemplars release fast and tend to be more popular. Pick suppliers from here.
45. Not all popular projects are exemplary and release fast Don’t pick suppliers from here.
46. We schedule updating dependencies as part of our daily work We strive to use the latest version (or latest-N) of all our dependencies We use some process to add a new dependency (e.g., evaluate, approve, standardize, etc.) We have a process to proactively remove problematic or unused dependencies We have automated tools to track, manage, and/or ensure policy compliance of our dependencies 46% YES 50% YES 30% YES 37% YES Enterprise Devs Manage Dependencies @weekstweets n = 658 38% YES
47. When Devs climb the mountain every day, it’s easier. @weekstweets
48. @weekstweets
49. @weekstweets
50. 2010: Jez Humble
51. How are you informed of InfoSec and AppSec issues? Automating security enables faster DevOps feedback loops
52. Automation continues to prove difficult to ignore. Do you have an open source policy and do you follow it?
53. For organizations who tamed their supply chains, the rewards were impressive. @weekstweets
54. imagine security at the speed of development @weekstweets
55. imagine automated ratings of OSS projects @weekstweets
56. imagine automated ratings of OSS projects @weekstweets
57. imagine automated ratings of OSS projects @weekstweets
58. @weekstweets imagine machines making software paving and maintaining the road with zero trust open source
59. …once we cease to sacrifice quality for speed C R E D I T : N E I L B E Y E R S D O R F
60. weeks@sonatype.com