How I Learned to Stop Worrying and Love the Bom

Simon Vestin (LINE / Security Center / Security Engineer)
Manabu Niseki (LINE / Security Center / Security Engineer)

https://tech-verse.me/ja/sessions/236
https://tech-verse.me/en/sessions/236
https://tech-verse.me/ko/sessions/236

2. Self Introduction Manabu Niseki • JSAC, HITCON, Botconf, OBTS speaker • spdx/spdx-core contributor • Forever V3 climber Simon Vestin • First time speaker for public audience • Competitive swimmer until Uni
3. Agenda • What is SBOM? • Challenges Towards Using SBOM • How LINE Uses SBOM
4. What is SBOM? • What is BOM? • What is SBOM? • SBOM Standards
5. What is BOM? A bill of materials or product structure (sometimes bill of material, BOM or associated list) is a list of the raw materials, sub- assemblies, intermediate assemblies, sub-components, parts, and the quantities of each needed to manufacture an end product. A BOM may be used for communication between manufacturing partners or confined to a single manufacturing plant. --- https://en.wikipedia.org/wiki/Bill_of_materials
6. What is BOM? (Source: https://www.fda.gov/food/food-labeling-nutrition/food-allergies)
7. What is SBOM? A “software bill of materials” (SBOM) has emerged as a key building block in software security and software supply chain risk management. A SBOM is a nested inventory, a list of ingredients that make up software components. --- https://www.cisa.gov/sbom
8. SBOM Use Cases Asset management License management Vulnerability management
9. SBOM Standards • SPDX (Software Package Data Exchange) • https://spdx.dev/ • CycloneDX • https://cyclonedx.org/ • SWID Tags (Software Identification Tags) • https://csrc.nist.gov/projects/Software-Identification-SWID
10. Challenges Towards Using SBOM • Common Pitfalls • How to Avoid Pitfalls
11. Common Pitfalls (Source: https://commons.wikimedia.org/wiki/File:Cu_Chi_tunnels_001.jpg by Jean-Marie Hullot)
12. Case Study: Python (Source: https://commons.wikimedia.org/wiki/File:Britannica_Python,_Python_reticulatus.jpg)
13. requirements.txt (pip) poetry.lock (Poetry) Pipfile.lock (Pipenv) anchore/syft ✅ ✅ ✅ aquasecurity/trivy ✅ ✅ ✅ microsoft/sbom-tool ✅ ✅ How Common SBOM Tools Work Case Study: Python • anchore/syft: v0.60.3 • aquasecurity/trivy: v0.34.0 • micorosoft/sbom-tool: v0.2.7
14. requirements.txt Case Study: Python Requirements files serve as a list of items to be installed by pip, when using pip install. Files that use this format are often called “pip requirements.txt files”, since requirements.txt is usually what these files are named (although, that is not a requirement) --- https://pip.pypa.io/en/stable/reference/requ irements-file-format/#requirements-file- format
15. poetry.lock Case Study: Python When Poetry has finished installing, it writes all the packages and their exact versions that it downloaded to the poetry.lock file, locking the project to those specific versions. You should commit the poetry.lock file to your project repo so that all people working on the project are locked to the same versions of dependencies (more below). --- https://python-poetry.org/docs/basic- usage/
16. Pipfile.lock Case Study: Python Pipfile.lock takes advantage of some great new security improvements in pip. By default, the Pipfile.lock will be generated with the sha256 hashes of each downloaded package. This will allow pip to guarantee you’re installing what you intend to when on a compromised network, or downloading dependencies from an untrusted PyPI endpoint. --- https://pipenv.pypa.io/en/latest/basics/#exampl e-pipfile-lock
17. Common Pitfalls in Python Case Study: Python • Where is a version? • Package without version specifier is allowed (requirements.txt) • Dev dependencies mix up • Very easy to come off the rail • It’s impossible to track if you do “pip install foo”
18. Case Study: Java (Source: https://commons.wikimedia.org/wiki/File:KITLV_-_78530_-_Kurkdjian_-_Sourabaia-Java_- _Coffee_plant,_presumably_in_Java_-_circa_1915.tif)
19. pom.xml (Maven) gradle.lockfile (Gradle) anchore/syft ✅ aquasecurity/trivy ✅ ✅ microsoft/sbom-tool ✅ ✅ How Common SBOM Tools Work Case Study: Java • anchore/syft: v0.60.3 • aquasecurity/trivy: v0.34.0 • micorosoft/sbom-tool: v0.2.7
20. pom.xml Case Study: Java A Project Object Model or POM is the fundamental unit of work in Maven. It is an XML file that contains information about the project and configuration details used by Maven to build the project. It contains default values for most projects. --- https://maven.apache.org/guides/introducti on/introduction-to-the-pom
21. gradle.lockfile Case Study: Java To achieve reproducible builds, it is necessary to lock versions of dependencies and transitive dependencies such that a build with the same inputs will always resolve the same module versions. This is called dependency locking. --- https://docs.gradle.org/current/userguide/d ependency_locking.html
22. Common Java Pitfalls Case Study: Java • pom.xml is not always included in a built executable (JAR, WAR, etc.) • It’s possible to exclude pom.xml by setting addMavenDescriptor as false • gradle.lockfile is not included in a built executable by default
23. How to Avoid Pitfalls (Source:https://commons.wikimedia.org/wiki/File:A_Catch_22_Situation_(Unsplash).jpg / CC0 1.0)
24. Runtime SBOM is the answer
25. What is Runtime SBOM? • GOOD – Static SBOM (source) - This works fine, but you’ll miss runtime libraries from appservers and runtime platforms. You’ll also include libraries that don’t matter like test frameworks. You’ll also have no idea which libraries are actually active in the running application. • BETTER – Static SBOM (binary) - You’ll still miss parts, because code can be located in a variety of different places. And you’ll also probably include libraries that don’t matter but happen to be on the filesystem. • BEST – Runtime SBOM - This is what ‘jbom’ is all about. Runtime SBOM is the most accurate approach as it captures the exact libraries used by the application, even if they are in the platform, appserver, plugins, or anywhere else. This approach can also include details of services invoked and which libraries are active. --- https://github.com/eclipse/jbom
26. What is Runtime SBOM? (TL;DR) • Static SBOM: based on a lock file • Runtime SBOM: based on a running process
27. Why Runtime SBOM Is Better • Static SBOM is an unreliable narrator • It may have false positives / negatives • False positives comes from dev dependencies • False negatives comes from outside of a lock file • Runtime SBOM is more reliable because it comes from a “running” process
28. Case Study: Python (Source: https://commons.wikimedia.org/wiki/File:Britannica_Python,_Python_reticulatus.jpg)
29. Demystifying Python Module Search Path Case Study: Python (Source: https://xkcd.com/1987/ by Randall Munroe)
30. Demystifying Python Module Search Path Case Study: Python Standard libraries Standard C libraries Third party libraries
31. Demystifying Python Module Search Path Case Study: Python
32. A Few Drawbacks • Dev dependencies problem is still there • It’s difficult to deal with dynamic “sys.path” configuration • e.g. ”import sys; sys.path.insert(“/foo/bar”)”
33. Case Study: Java (Source: https://commons.wikimedia.org/wiki/File:KITLV_-_78530_-_Kurkdjian_-_Sourabaia-Java_- _Coffee_plant,_presumably_in_Java_-_circa_1915.tif)
34. Looking Into File Descriptors Case Study: Java
35. Fat/Uber JAR Case Study: Java Fat/Uber - Contains the bit you literally write yourself PLUS the direct dependencies of your app PLUS the bits needed to run your app “on its own”. --- https://developers.redhat.com/blog/2017 /08/24/the-skinny-on-fat-thin-hollow-and- uber
36. JBOM Case Study: Java jbom generates Runtime and Static SBOMs for local and remote Java apps --- https://github.com/eclipse/jbom (Source: https://www.slideshare.net/RafaelWinterhalter/understanding-java-byte-code-and-the-class-file-format)
37. Great we know how to collect SBOMs! – What now?
38. Shisa Great we know how to collect SBOMs! – What now? A cultural artifact that can be seen at rooftops or entrances of Japanese buildings, warding, keeping evil out and good spirits in. (Source: https://commons.wikimedia.org/wiki/File:Ishigaki_Shiisa.JPG by Jpatokal)
39. What is “the other” Shisa Scanning our infrastructure daily for security concerns, reporting its findings and displaying it on a dashboard. Shisa helps us in reducing vulnerabilities (evil out) and fortify (good in) LINE infrastructure. This is where we utilize the SBOM concept.
40. Report DB OSV DB Shisa Data Pipeline Collecting data Script Enhancing data Pipeline Displaying data Dashboard
41. Report DB OSV DB Shisa Data Pipeline Collecting data Script Enhancing data Pipeline Displaying data Dashboard
42. Report DB OSV DB Shisa Data Pipeline Collecting data Script Enhancing data Pipeline Displaying data Dashboard
43. Report DB OSV DB Shisa Data Pipeline Collecting data Script Enhancing data Pipeline Displaying data Dashboard
44. Report DB OSV DB Shisa Data Pipeline Collecting data Script Enhancing data Pipeline Displaying data Dashboard
45. Shisa Script Every day! Total Servers 150,000+ Server native bin Python >= 2.6 Not only SBOM Interfaces Netstats Processes Application Detection Vulnerability Checks Hardening Checks and more… source. https://commons.wikimedia.org/wiki/File:Python.svg
46. Shisa Pipeline Is information available at the servers enough? • Who is the owner of the server, what service does it belong to? • Is the server is exposed to the Internet, and if so on what ports? • Can we use SBOM to find security vulnerabilities?
47. Report API Enhance Report Index Report Owner info Exposure info …. OSV info DB
48. Report API Enhance Report Index Report Owner info Exposure info …. OSV info DB
49. Report API Enhance Report Index Report Owner info Exposure info …. OSV info DB
50. Report API Enhance Report Index Report Owner info Exposure info …. OSV info DB
51. OSV Open Source Vulnerability • Authored by two employees at Google. • A standardized format for open source vulnerabilities. • Freely available OSV DB https://osv.dev/ • https://github.com/google/osv.dev
52. Shisa OSV osv.dev Shisa OSV API DB Shisa Pipeline
53. Schemas OSV matched with SBOM "affected": [ { "package": { "ecosystem": string, "name": string, "purl": string }, "ranges": [ { "type": string, "repo": string, "events": [ { "introduced": string, "fixed": string, "last_affected": string, "limit": string } ], “database_specific”: {} } ], "versions": [ string ], "ecosystem_specific": {}, "database_specific": {} } ], "components": [ { "metadata": {}, "type": "string", "name": "string", "version": "string", "id": "string", "pid": 0, "exe": "string" } ] * Code snippets are redacted to fit the page. Full OSV schema can be found at https://ossf.github.io/osv-schema/
54. • Making parts of the Shisa report openly available • Search, drill down on each item • Gamification - Task assignment, completion and tracking history Dashboard
55. Use Case #1 Security Risk Assessment before release of a service. • Before: • Semi-manually • One time before release. • Shared by excel documents • After: • Fully automatic • Every day + on demand. • Direct access 365/24/7
56. Use Case #2 Need to provide information to financial institutions on what vulnerabilities are affecting servers with highly classified information. Before • Look though vulnerability feeds. • Develop hardcoded vulnerability check. • Manual execution. After • Vulnerability coverage increase from around 30 to 8811 (OSV) • Automatic and on demand. • Hardcoded checks for more intricate conditions.
57. Wrap Up • SBOMs helps us with transparency of what a software or process contains, but it have pitfalls. • Lock files without versions. • Components installed outside of lock files. • Lock files not mandatory • Runtime SBOM - Mitigate lock file issue, getting SBOM from process
58. Wrap Up • Boost vulnerability detection combining SBOM with OSV • Openly providing data, not keeping it in security departments only, helps in addressing security concerns more quickly.
59. Thank You