Securing Kubernetes Software Supply Chain Using SLSA Framework Edgaras Apšega Site Reliability Engineer, Vinted CNCF Ambassador 

2 ● Site Reliability Engineer at Vinted ● 6+ years of experience with running Kubernetes on bare metal ● Cloud Native Lithuania meetup co-organizer ● CNCF Ambassador Edgaras Apšega Site Reliability Engineer $ whoami 

AGENDA ● Intro to software Supply Chain and its threats ● Understanding the SLSA framework ● Best practices for implementing SLSA framework for Kubernetes software supply chain ● Future trends and innovations 3 

Intro Brief overview of software Supply Chain, its threats and some real world examples 4 

5 THE NEED FOR SECURING SUPPLY CHAIN 

6 SUPPLY CHAIN THREATS 

7 MODIFIED SOURCE CODE: CODECOV EXAMPLE ● Attacker added a malicious code in the Bash script (beautifully executed and hidden on line 525 of a 1800+ line script) ● 23 000 customers affected (incl. Twilio, Hashicorp, Rapid7, Confluent) ● The system breach remained undetected for over two months, customer alerted the company of a discrepancy between the shasums 

8 BREACHED BUILD SYSTEM: SOLARWINDS EXAMPLE ● Attackers breached Orion platform supply chain build system ● 18 000 to 30 000 customers of SolarWinds were affected ● FireEye, a cybersecurity company, detected the malware spreading to their customers. 14 months passed between first unauthorized access and malware discovery 

9 VULNERABLE DEPENDENCY: LOG4J EXAMPLE ● Security researchers have identified approximately 10 million Log4Shell exploitations attempts every hour ● Discovered on December 2021, but as of April 2022, the cybersecurity firm Rezilion claimed there were still over 68,000 publicly accessible systems at risk 

Understanding SLSA framework Definition and purpose of the SLSA framework 10 

11 SLSA FRAMEWORK SLSA is a set of incrementally adoptable guidelines for supply chain security, established by industry consensus. The specification set by SLSA is useful for both software producers and consumers: producers can follow SLSA’s guidelines to make their software supply chain more secure, and consumers can use SLSA to make decisions about whether to trust a software package. SLSA is a set of incrementally adoptable guidelines for supply chain security, established by industry consensus. The specification set by SLSA is useful for both software producers and consumers: producers can follow SLSA’s guidelines to make their software supply chain more secure, and consumers can use SLSA to make decisions about whether to trust a software package. What’s SLSA? 

12 SLSA SECURITY LEVELS (v0.1) Scripted build process; Automated provenance generation L1: Documented build process L2: Build service resistance The build runs on a hosted build platform that generates and signs the provenance itself L3: Resistance to specific threats Source and build platforms guarantee the auditability of the source and the integrity of the provenance L4: Highest level of confidence & trust Requires two-person review of all changes; Hermetic, reproducible build process 

13 SLSA SECURITY LEVELS (v1.0) Software producer follows a consistent build process Build platform automatically generates provenance Software producer distributes provenance to consumers Build L1: Provenance exists Build L2: Hosted build platform The build runs on a hosted build platform that generates and signs the provenance itself Downstream verification of provenance Build L3: Hardened builds Build platform implements strong controls to prevent runs from influencing one another Prevent secret material used to sign the provenance from being accessible to the user-defined build steps 

14 SLSA PROVENANCE It’s the verifiable information about software artifacts describing where, when and how something was produced. Has its own SLSA requirements such as: ● It exists (L1) ● It’s authentic (L2) ● It’s unforgeable (L3) Attestations include: build timestamps, build parameters and environment, version control metadata, source code details and materials (files, scripts) consumed during the build. Provenance example of SLSA Level 3 What’s provenance? 

15 SOFTWARE ATTESTATION ● Metadata (provenance) + signing + identity ● Identity + signing (who) ● Metadata (what, where, when, how) ● Verification 

16 SLSA VS. SOFTWARE BILL OF MATERIALS (SBOM) SBOMs problems: ● Author Name ● Vendor Name ● Component Name ● Version String ● Component Hash ● Unique Identifier ● Relationship SBOM includes: Don’t include or require enough information to help users respond to build tampering There’s no well-established ecosystem to easily distribute and verify SBOM documents The most common method of generating SBOMs using only audit tools after the software’s creation can result in less accurate SBOMs Code modification Uploaded artifacts that were not built by a CI/CD system Threats against the build system SLSA mitigates risks of: 

17 CONTAINER IMAGE PROVENANCE Build with provenance Inspect layers 

18 BUILD PROVENANCE Official solutions Provenance schema example ● Github Actions https://github.com/slsa-framework/slsa-gith ub-generator (Go, JS, containers) ● Google Cloud Build https://cloud.google.com/build/docs/securin g-builds/view-build-provenance ● Jenkins https://github.com/slsa-framework/slsa-jenki ns-generator (PoC) ● SLSA verifier https://github.com/slsa-framework/slsa-verifi er 

19 SLSA LEVELS RECAP 

Real world SLSA assessments ArgoCD and Prometheus did SLSA assessment openly 20 

21 ArgoCD Current practices ● 2 FA for maintainers ● 1 person review for PRs ● GitHub as VCS, maintainers are only ones allowed to merge PRs ● GitHub actions scripted builds ● Builds do not share cache and are isolated from each other ● GitHub actions generates and signs all of the artifacts, generates provenance and publishes all artifacts with signatures and build provenance ● Provenance generated with slsa-github-generator Github Action 

22 ArgoCD Current practices SLSA Assessment ● 2 FA for maintainers ● 1 person review for PRs ● GitHub as VCS, maintainers are only ones allowed to merge PRs ● GitHub actions scripted builds ● Builds do not share cache and are isolated from each other ● GitHub actions generates and signs all of the artifacts, generates provenance and publishes all artifacts with signatures and build provenance ● Provenance generated with slsa-github-generator Github Action SLSA level 3 (v0.1) 

23 PROMETHEUS Current practices ● 2 FA for maintainers ● 1 person review for PRs ● GitHub as VCS, maintainers are only ones allowed to merge PRs ● GitHub actions scripted builds ● Builds use cache but only for CI testing and does not affect release process ● Uses prometheus/promu tool to generate and publish final artifacts, which expands dependencies ● No provenance 

24 PROMETHEUS Current practices SLSA Assessment ● 2 FA for maintainers ● 1 person review for PRs ● GitHub as VCS, maintainers are only ones allowed to merge PRs ● GitHub actions scripted builds ● Builds use cache but only for CI testing and does not affect release process ● Uses prometheus/promu tool to generate and publish final artifacts, which expands dependencies ● No provenance SLSA level 0 (v0.1) 

Securing Kubernetes software supply chain Best practices for implementing SLSA framework for Kubernetes software supply chain 25 

26 SECURING KUBERNETES SUPPLY CHAIN SOFTWARE Source Code Management Continuous Integration Container build Container Registry Runtime Continuous Delivery ● 2 peer reviews ● Container scanning ● Provenance generation ● Provenance signing ● Provenance verification ● Container signing ● Container scanning ● Container signature verification ● Allow only signed containers ● Runtime security 

27 SIGSTORE Components Architecture ● Cosign - used to sign and verify OCI container images and other blobs ● Policy controller - admission controller for policy enforcement ● Fulcio - code-signing CA issuing short-lived certificates ● Rekor - append-only auditable transparency log service. Records signed metadata that can only be queried 

SIGSTORE WORKFLOW 28 

Future trends and innovations Of course it’s AI! 29 

30 SECURITY SCANNERS NOW: METADATA Scanning tools results Base Dockerfile 

31 SECURITY SCANNERS NOW: METADATA (2) Scanning tools results Modified Dockerfile 

32 SECURITY SCANNERS FUTURE: ACTUAL CODE 

Thank you References: ● slsa.dev ● sigstore.dev ● Sandworm - A New Era of Cyberwar by Andy Greenberg ● https://github.com/bgeesaman/malicious-compliance ● https://www.cncf.io/blog/2023/04/19/building-secure-software-supply-chains-in-cncf-with-slsa-assessments/ ● https://blog.virustotal.com/2023/04/introducing-virustotal-code-insight.html