SPB Jenkins Meetup #9. Managing security in Jenkins (ENG)

Managing security in Jenkins. Best practices and Ownership-Based Security Oleg
Nenashev CloudBees, Inc. St. Petersburg Jenkins Meetup November 28, 2017

@oleg_nenashev, @jenkins_spb © 2017 CloudBees, Inc. All Rights Reserved. 2
About me @oleg_nenashev oleg-nenashev LibreCores project St. Petersburg Polytechnic University Jenkins meetups

Oleg’s “Hall of Shame”(c) • Plugins • Jenkins Core • Windows Service Wrapper • Remoting • Security Team

https://jenkins.io/security/ Fixes in the core and plugins Jenkins Security Team Board Core Team Security LTS Events INFRA Website

About you Do you administer Jenkins instances?

About you Do you administer Jenkins instances? Do you have more than 20 users?

Agenda 1. Introduction to Jenkins Security 2. Protecting Jenkins. Best practices 3. Ownership-based security Disclaimer: • Presentation represent the author’s personal opinion • Author’s personal opinion may differ from official positions of CloudBees and/or Jenkins community • Many Jenkins instances were harmed, use carefully

1. Most popular CI/CD tool in the world 2. Generic automation server 3. Flexible and extensible 4. It’s open source, big community 5. Commercial support vendors 6. … Who is Mr. Jenkins? https://jenkins.io

Jenkins… is a remote execution engine (by design)

Jenkins… is a remote execution engine (by design) • One can run code and system commands • Access to master system • Access to agents • Access to private/public clouds

Jenkins… has access to sensitive data (by design)

Jenkins… has access to sensitive data (by design) • Credentials • Private repositories • Artifacts, including release ones

Jenkins… is a shared service (by design)

Jenkins… is a shared service (by design) • Multiple users • Different expertise • Users may misuse permissions

What does security mean? Jenkins security Intrusion and data theft protection Restrictions within organization

Jenkins security Intrusion and data theft protection Restrictions within organization • Must-have in internet-facing instances • Paranoid mode is fine What does security mean?

Jenkins security Intrusion and data theft protection Restrictions within organization • Better user experience • Protection from unintentional actions • Protection from lack of expertise What does security mean?

Protecting Jenkins. Best practices

Limited number of admins Permissions Security audit Rule #0. Use security!

Rule #1. Keep Updating Frequent security releases • Weekly • Current LTS baseline Info sources • https://jenkins.io/security/advisories/ • jenkinsci-advisories mailing list (including announcements) • RSS feed 2.46.2 Exploits are in the wild, update ASAP

LTS is only 12 weeks… Not enough?

Build your own core (custom fork) •“mvn clean package” in the root •HINT: Join the security team to get info about changes in advance LTS is only 12 weeks… Not enough?

Build your own core (custom fork) JEP it, help to maintain! • People is interested in longer LTS LTS is only 12 weeks… Not enough?

Build your own core (custom fork) JEP it, help to maintain! Use custom versions from vendors: • https://wiki.jenkins-ci.org/display/JENKINS/Commercial+Support • CloudBees Jenkins Enterprise LTS is only 12 weeks… Not enough?

Do you pull latest images from DockerHub?

What’s inside? Who can change them? What if there is a malicious code? Do you pull latest images from DockerHub?

What’s inside? Who can change them? What if there is a malicious code? How is it different from other package sources? Do you pull latest images from DockerHub?

Plugins may contain defects Rule #2. Know what you use

Rule #2. Know what you use Monitor plugin versions and release notes • Beware of transient dependencies (!) • Also monitor JIRA

Rule #2. Know what you use Monitor plugin versions and release notes Consider using locally managed sources • Internal Maven • Docker Registry • Custom Jenkins Update Center: Juseppe • https://github.com/yandex-qatools/juseppe

Rule #2. Know what you use Monitor plugin versions and release notes Consider using locally managed sources Use static configurations • Configuration-as-Code • One cannot simply break it

Previous Jenkins Meetup… http://bit.ly/jenkins_msk_3_groovy_hooks

System Configuration… as Code External tools Jenkins CLI and REST API python- jenkins jenkins-client (java) Configuration Management Ansible, Chef, … Docker, Docker Compose ... Solutions in Jenkins Groovy Boot Hooks Config as Code Plugin (alpha) SCM Sync Configuration Just examples…

Disabling setup wizard… turns off security, SURPRISE! Configuration-as-code. Keep in mind!

Disabling setup wizard… turns off security, SURPRISE! You are responsible for configuring security Configuration-as-code. Keep in mind!

You are responsible for configuring security Examples: • Authentication/Authorization • CSRF • Slave2Master security • Remoting protocols Configuration-as-code. Keep in mind!

https://hub.docker.com/r/onenashev/ demo-jenkins-config-as-code/ Groovy Hooks

Is it enough?

Rule #3. Keep Jenkins in a sandbox

Rule #3. Keep Jenkins in a sandbox Do not run masters/agents under system accounts • BAD - Local Administrator in Windows • BAD - Root in Unix • NOT BAD? - Root in Docker

Rule #3. Keep Jenkins in a sandbox Do not run masters/agents under system accounts Restrict access to non- required resources • Generic accounts • Read-only repositories

Rule #3. Keep Jenkins in a sandbox Do not run masters/agents under system accounts Restrict access to non- required resources Sandbox your scripts as well

Scryptocalypse https://jenkins.io/security/advisory/2017-04-10/ • Unrestricted scripting • More than 30 plugins affected •Groovy Plugin •JobDSL Plugin •Grails Plugin •Scriptler Plugin • Some of them are blocked, even now

It is not only about Groovy… Tcl Plugin •Blacklisted in April 2017 Jenkins Core 2.73.3 •Command Computer Launcher •Runs command on a master •Any user with a mode edit permission… was able to run commands on the master

DIY

Jenkins Script Security https://plugins.jenkins.io/ script-security DIY

Script Security Plugin Used in [almost] all Groovy plugins https://plugins.jenkins.io/script-security

Is it enough to become secure?

Rule #4. Do not Run Jobs on master

Builds have access to the master filesystem Examples: • Read data from other builds/artifacts • Read secret hashes • Modify Jenkins system configuration Rule #4. Do not Run Jobs on master

Rule #4. Do not Run Jobs on master • Solution 1: • Set “0” executors on master • Another node running under different account • BUT: Does not protect from fly-weight tasks

Rule #4. Do not Run Jobs on master • Solution 1: • 0 executors on master • Another node running under different account • BUT: Does not protect from fly-weight tasks • Solution 2: • Job Restrictions Plugin • Details: later

Ø By default builds run with the System account Ø Users may trigger wrong builds Ø Users can extract data Rule #5. Do not trust your builds

Authorize Project Plugin Authorize builds • Global default • Whitelist of user- configurable strategies • Job properties https://plugins.jenkins.io/authorize-project

Audit Trail – logging of actions •https://plugins.jenkins.io/audit-trail Security Inspector – permission checks •https://plugins.jenkins.io/security-inspector … Rule #6. Audit your security

Security Inspector Plugin https://plugins.jenkins.io/security-inspector Reports for jobs, agents and users

1. Assign leads to jobs and agents 2. Share the maintenance effort with them 3. Make the ownership explicit Rule #7. Make the responsibilities explicit

Common strategies do not “just work” Project Matrix Authorization Strategy • Can be managed on Job/Folder level • Hard to manage every item • WAS: No support of Node permissions

Common strategies do not “just work” Project Matrix Authorization Strategy • Hard to manage • No support of Node permissions Role-Based Strategy • Regular expression for each role • Performance: Hundreds of Regex checks every request • Web UI easily hangs

Role Strategy, it’s FUN!

Ownership-Based Security

Ownership-based Security Role- Strategy Ownership Job Restrictions • Assign owners of jobs/nodes • Fancy UI • Auth strategy • Macro engine • Restrict runs for jobs and nodes http://bit.ly/ownership-based-security + Authorize Project

Ownership Plugin • Primary and Secondary Owners • Summary Boxes, View filters, etc. • Environment variables • Integration with Security plugins Customizable layout https://plugins.jenkins.io/ownership

Ownership Info. Definition and Inheritance Folders Jobs Nodes Runs Sub- Projects Inherits

Demo. What’s inside? Ownership 0.10.0 Job Restrictions 0.6 Security Inspector 0.4 Jenkins core 2.73.3 (minimal – 1.625) Authorize Project 1.3.0 Dynamic Search View 0.2.2 Role Strategy 2.6.1

https://hub.docker.com/r/onenashev/ demo-jenkins-config-as-code/

Setting ownership info

Ownership Info. What Do you get? • Ownership Summary Boxes • Ownership View Columns • View Filters • Also: @Me macro Customizable layout

Example: Quick administration contacts Customizable template

Ownership-Based Security. Role-Based Strategy Settings Roles [1/2]

Ownership-Based Security. Role-Based Strategy Settings Roles [2/2]

Ownership-Based Security. Role-Based Strategy Settings Assignments

Jobs. Securing access Untrusted secondary owners!

Jobs. Authorize Project Jobs get authenticated as owners => • Permissions • Node access (Computer.BUILD)

Using Data in Jobs. Freestyle

Using Data in Jobs. Pipeline

Jenkins nodes • Similar Ownership Management • Special permission • Node Ownership Monitor • => info in the table

Securing Nodes

Job Restrictions. Protecting the Master node • NEVER let users run jobs on master • Only use it for system jobs owned by admins

Ownership-Based Security: Links Plugins: • https://plugins.jenkins.io/ownership • https://plugins.jenkins.io/role-strategy • https://plugins.jenkins.io/job-restrictions • https://plugins.jenkins.io/authorize-project Ownership-based security: • http://bit.ly/ownership-based-security Demo • https://github.com/oleg-nenashev/demo-jenkins-config-as-code

Ø Item-specific security Ø Ownership-based restrictions for triggering jobs Ø Ownership assignment policy on create/copy Ø “sudo” mode for admins Ownership-Based Security: Out of the scope http://bit.ly/ownership-based-security

1. Subscribe to security advisories 2. Use Security plugins 3. Keep your Jenkins up to date Takeaways

1. There are existing solutions for large-scale 2. They are not documented sometimes… 3. Google ’em all Rule #-1. Explore

Security page • https://jenkins.io/security/ Advisories • https://jenkins.io/security/advisories/ Ownership-based security • http://bit.ly/ownership-based-security Demo • https://hub.docker.com/r/onenashev/demo-jenkins-config-as-code/ Links

Thank you! Contacts: E-mail: [email protected] GitHub: oleg-nenashev Twitter: @oleg_nenashev

SPB Jenkins Meetup #9. Managing security in Jenkins (ENG)

SPB Jenkins Meetup #9. Managing security in Jenkins (ENG)

More Decks by Oleg Nenashev

Other Decks in Programming

Featured

Transcript