Tweet
Tweet

Slide 1

Slide 1 text

© 2019 CloudBees, Inc. All Rights Reserved. Docker and Jenkins [as code] Oleg Nenashev, CloudBees Docker Geneva meetup, Mar 21, 2019 1

Slide 2

Slide 2 text

© 2019 CloudBees, Inc. All Rights Reserved. Today… 2

Slide 3

Slide 3 text

© 2019 CloudBees, Inc. All Rights Reserved. > whoami @oleg_nenashev oleg-nenashev • Based in Neuchatel, Switzerland • Principal SW Engineer @ CloudBees • Jenkins Core Maintainer & Ambassador • Swiss Jenkins Meetup organizer 3

Slide 4

Slide 4 text

© 2019 CloudBees, Inc. All Rights Reserved. • Building local community • Sharing experiences • Looking for speakers Swiss Jenkins Area Meetup 4 https://www.meetup.com/Swiss-Jenkins-Area-Meetup https://twitter.com/JenkinsMeetupCH

Slide 5

Slide 5 text

© 2019 CloudBees, Inc. All Rights Reserved. > whoami -docker -casc ● Using Configuration-as-code since 2011 ● Using Docker since 2015 (for development) ● Jenkins Docker Packaging maintainer ● JCasC plugin contributor ● Custom WAR Packager maintainer 5

Slide 6

Slide 6 text

© 2019 CloudBees, Inc. All Rights Reserved. We’re Building the World’s First End-to-End Continuous Software Delivery System About CloudBees 6 https://www.cloudbees.com

Slide 7

Slide 7 text

© 2019 CloudBees, Inc. All Rights Reserved. > whoami -cloudbees 7 ● CloudBees Jenkins Distribution ● CloudBees Jenkins Support ● Community (e.g. Java 11 support, Jenkinsfile Runner) https://www.cloudbees.com

Slide 8

Slide 8 text

© 2019 CloudBees, Inc. All Rights Reserved. About you 8

Slide 9

Slide 9 text

© 2019 CloudBees, Inc. All Rights Reserved. Agenda • Docker & Jenkins • Extending official images • Build your own Jenkins images! 9 These slides: http://bit.ly/docker-genev a-jenkins-as-code

Slide 10

Slide 10 text

© 2019 CloudBees, Inc. All Rights Reserved. CI/CD. Why containers? Config Management Fast provisioning Clean environments Disposability Clustering 10

Slide 11

Slide 11 text

© 2019 CloudBees, Inc. All Rights Reserved. 11

Slide 12

Slide 12 text

© 2019 CloudBees, Inc. All Rights Reserved. Configuration as Code in Jenkins Jobs System configuration 12 12

Slide 13

Slide 13 text

© 2019 CloudBees, Inc. All Rights Reserved. Configuration as Code in Jenkins Jobs System configuration 13 13

Slide 14

Slide 14 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins and Docker 14

Slide 15

Slide 15 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins & Docker. Packaging https://hub.docker.com/r/jenkins/ 15

Slide 16

Slide 16 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins and Docker • Build agents and steps 16

Slide 17

Slide 17 text

© 2019 CloudBees, Inc. All Rights Reserved. Official agent images • jenkins/slave – base image with Remoting • jenkins/jnlp-slave – JNLP agents • jenkins/ssh-slave – SSH agents • jenkins/remoting-kafka-agent – Remoting Kafka Plugin 17 https://hub.docker.com/r/jenkins/

Slide 18

Slide 18 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins & Docker. Plugins Docker Plugin Docker Pipeline Plugin Docker Custom Build Environment Plugin … Yet Another Docker Plugin Kubernetes Plugin Kubernetes Pipeline Plugin … 18

Slide 19

Slide 19 text

© 2019 CloudBees, Inc. All Rights Reserved. Example: Docker Pipeline Plugin https://plugins.jenkins.io/docker-workflow docker.image('onenashev/gcc-riscv:6.4-rc').inside { checkout scm sh "make clean test" } 19

Slide 20

Slide 20 text

© 2019 CloudBees, Inc. All Rights Reserved. Example: Kubernetes Plugin def label = "gcc-riscv-${UUID.randomUUID().toString()}" podTemplate(name: 'test', label: label, containers: [ containerTemplate(name: 'gcc-riscv', image: 'onenashev/gcc-riscv:6.4-rc’, resourceRequestCpu: ‘2’, resourceLimitMemory: ‘8Gi' ttyEnabled: true, command: 'cat')]) { node(label) { checkout scm sh "make clean test" step([$class: 'TapPublisher', testResults: 'output/test/report.tap', ...]) } } https://plugins.jenkins.io/kubernetes 20

Slide 21

Slide 21 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins and Docker • Build agents and steps • Master images 21

Slide 22

Slide 22 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins masters in Docker • jenkins/jenkins • jenkinsci/blueocean (being deprecated) • experimental images 22

Slide 23

Slide 23 text

© 2019 CloudBees, Inc. All Rights Reserved. Official image: jenkins/jenkins docker run --rm \ -p 8080:8080 \ -p 50000:50000 \ -v jenkins_home:/var/jenkins_home \ jenkins/jenkins:lts 23 See also: https://batmat.net/2018/09/07/how-to-run-and-upgrade- jenkins-using-the-official-docker-image/

Slide 24

Slide 24 text

© 2019 CloudBees, Inc. All Rights Reserved. Official image: jenkins/jenkins • Standard image • Weekly and LTS releases • No bundled plugins, only detached ones • Debian, Alpine, Slim • Java 8, Java 11 24 https://github.com/jenkinsci/docker

Slide 25

Slide 25 text

© 2019 CloudBees, Inc. All Rights Reserved. Extending images Official images are designed to be extended Managed as code 25

Slide 26

Slide 26 text

© 2019 CloudBees, Inc. All Rights Reserved. Extending images Image source: http://container.farm/special-extended-container/ YOUR STUFF 26

Slide 27

Slide 27 text

© 2019 CloudBees, Inc. All Rights Reserved. Extending Docker images Dockerfile Plugin bundling Groovy Hooks Environment and Java Options Docs: https://github.com/jenkinsci/docker 27

Slide 28

Slide 28 text

© 2019 CloudBees, Inc. All Rights Reserved. Dockerfile 28

Slide 29

Slide 29 text

© 2019 CloudBees, Inc. All Rights Reserved. Plugin bundling 29

Slide 30

Slide 30 text

© 2019 CloudBees, Inc. All Rights Reserved. Plugin bundling 30 # Pinned version blueocean:1.8.2 # Latest antisamy-markup-formatter matrix-auth:latest # Latest from the Experimental update center git-client:experimental # Incrementals (JEP-305) workflow-support:incrementals;org.jenkins-ci.plugins.workflow;2.21-rc591.43d37d4d080a plugins.txt

Slide 31

Slide 31 text

© 2019 CloudBees, Inc. All Rights Reserved. Groovy Hooks [1/2] • Standard Groovy engine • Triggered by Jenkins • Direct access to the runtime • Extensible hook engine • 2 hook types in the core: init, boot-failure https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script 31

Slide 32

Slide 32 text

© 2019 CloudBees, Inc. All Rights Reserved. Groovy Hooks [2/2] Script locations: • WEB-INF/HOOK.groovy in jenkins.war • WEB-INF/HOOK.groovy.d/*.groovy in jenkins.war • $JENKINS_HOME/HOOK.groovy • $JENKINS_HOME/HOOK.groovy.d/*.groovy Execution in alphabetical order 32 https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script

Slide 33

Slide 33 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins Startup Flow JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() PLUGINS_STARTED . . . 33

Slide 34

Slide 34 text

© 2019 CloudBees, Inc. All Rights Reserved. Example https://github.com/oleg-nenashev/demo-jenkins-config-as-code 34

Slide 35

Slide 35 text

© 2019 CloudBees, Inc. All Rights Reserved. Adding Groovy hooks to the standard image Just put Groovy scripts in the right place! https://github.com/oleg-nenashev/demo-jenkins-config-as-code 35

Slide 36

Slide 36 text

© 2019 CloudBees, Inc. All Rights Reserved. More examples oleg-nenashev/demo-jenkins-config-as-code Praqma/JenkinsAsCodeReference librecores/librecores-ci 36

Slide 37

Slide 37 text

© 2019 CloudBees, Inc. All Rights Reserved. Add JCasC Plugin! Configuration-as-Code Plugin https://plugins.jenkins.io/configuration-as-code 37

Slide 38

Slide 38 text

© 2019 CloudBees, Inc. All Rights Reserved. 38 JCasC Plugin ● Sub-project in Jenkins ● Feb 2018 - first alpha ● Sep 2018 - 1.0 ● Now - 1.8 ● Configuration via YAML jenkinsci/configuration-as-code-plugin

Slide 39

Slide 39 text

© 2019 CloudBees, Inc. All Rights Reserved. jenkins: systemMessage: "JCasC Demo" numExecutors: 1 scmCheckoutRetryCount: 4 mode: NORMAL securityRealm: local: allowsSignup: false users: - id: demoAdmin password: ${adminpw} jenkins.yaml 39

Slide 40

Slide 40 text

© 2019 CloudBees, Inc. All Rights Reserved. • YAML specification • Configuration export from Jenkins • CLI and REST API for configuration management • Pre-flight validations and dry-run • Support of many plugins out-of-the-box Features 40

Slide 41

Slide 41 text

© 2019 CloudBees, Inc. All Rights Reserved. Adding JCasC to the Docker image YAML auto-discovery in 1.0+ Needs JCasC Plugin [and JCasC Support?] … configuration-as-code:1.1 configuration-as-code-support:1.1 … 41

Slide 42

Slide 42 text

© 2019 CloudBees, Inc. All Rights Reserved. Demos https://github.com/jenkinsci/configuration-as-code-plugi n/tree/master/demos 42

Slide 43

Slide 43 text

© 2019 CloudBees, Inc. All Rights Reserved. https://github.com/oleg-nenashev/demo- jenkins-config-as-code/tree/casc-plugin 43

Slide 44

Slide 44 text

© 2019 CloudBees, Inc. All Rights Reserved. Demo. Local Pipeline Development Env Intellij IDEA Filesystem SCM Plugin • Documentation, Syntax • Static analysis • Debug (only for hooks) Local Jenkins instance Source Code (local .git repos) • Configuration-as-Code • Same as production • Repos with libs • Jenkinsfile JENKINS_HOME (volume) 44

Slide 45

Slide 45 text

© 2019 CloudBees, Inc. All Rights Reserved. JCasC Limitations (temporary?) • Not all plugins are supported • YAML - No dynamic logic / scripting • Startup Chicken&Egg for plugins & jobs • Handling of removed sections • Export from UI – many open issues • No read-only admin access 45

Slide 46

Slide 46 text

© 2019 CloudBees, Inc. All Rights Reserved. Combining JCasC and Groovy hooks 46 JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() JCasC PLUGINS_STARTED . . . Groovy hooks run after JCasC Hooks can fine-tune the configuration: • Missing integrations • Dynamic scripting • Job configurations and migrations Use JCasC where possible!

Slide 47

Slide 47 text

© 2019 CloudBees, Inc. All Rights Reserved. Comparison JCasC Plugin Groovy Hooks YAML – standard declarative definition Low barrier to entry Limited integrations Forward compatibility risks External rollback Apply changes w/o restart No debugging, YAGNI? Dry run mode + validation Export from UI CLI and REST API for CM Plugin management Groovy – general-purpose OOP language High barrier to entry Can manage everything Compatible if API is stable External rollback Restart is required Debugging support Plugin Management 47

Slide 48

Slide 48 text

© 2019 CloudBees, Inc. All Rights Reserved. Too many options? 48 https://www.flickr.com/photos/ silgeland/2645819709

Slide 49

Slide 49 text

© 2019 CloudBees, Inc. All Rights Reserved. Custom WAR Packager https://jenkins.io/blog/2018/10/16/custom-war-packager/ ● Managed by a config YAML ● Produces ready-to fly Jenkinsfile bundles 49 https://github.com/jenkinsci/custom-war-packager

Slide 50

Slide 50 text

© 2019 CloudBees, Inc. All Rights Reserved. Single-shot masters. Build Flow Custom WAR Packager Core and Plugins Configuration-as-code Packaging Jenkins WARs, Docker images, or Jenkinsfile Runner 50

Slide 51

Slide 51 text

© 2019 CloudBees, Inc. All Rights Reserved. Custom WAR Packager Inputs • Base Docker image • Jenkins WAR • Plugins and Custom libraries • System properties • Self-configuration: JCasC & System Groovy Hooks 51

Slide 52

Slide 52 text

© 2019 CloudBees, Inc. All Rights Reserved. Quick Demo (packaging) https://github.com/jenkinsci/custom-war-packager/ tree/master/demo/casc 52

Slide 53

Slide 53 text

© 2019 CloudBees, Inc. All Rights Reserved. Custom WAR Packager YAML [1/2] Bundle settings: 53

Slide 54

Slide 54 text

© 2019 CloudBees, Inc. All Rights Reserved. Custom WAR Packager YAML [2/2] Inputs: 54

Slide 55

Slide 55 text

© 2019 CloudBees, Inc. All Rights Reserved. Custom WAR Packager. Usages Jenkins X – Serverless masters ● CWP + JCasC + Jenkinsfile Runner ● https://jenkins.io/blog/2018/09/12/speaker-blog-a-cloud-native-jenkins/ ci.jenkins.io-runner ● https://github.com/jenkinsci/ci.jenkins.io-runner Demo packages 55

Slide 56

Slide 56 text

© 2019 CloudBees, Inc. All Rights Reserved. Takeaways 1. Dockerize Jenkins 2. Use & extend official images 3. Use static configurations 4. Use Configuration-as-Code 5. Use Custom WAR/Docker Packager 56

Slide 57

Slide 57 text

© 2019 CloudBees, Inc. All Rights Reserved. Useful links Docker image • https://github.com/jenkinsci/docker Docker Package demo (with JCasC) • https://github.com/oleg-nenashev/demo-jenkins-config-as -code/tree/casc-plugin Custom WAR Packager • https://github.com/jenkinsci/custom-war-packager 57

Slide 58

Slide 58 text

© 2019 CloudBees, Inc. All Rights Reserved. Want to know more? Want to present? Swiss Jenkins Meetup ● https://www.meetup.com/Swiss-Jenkins-Area-Meetup CloudBees Days Zurich, Apr 10 ● https://www.cloudbees.com/cloudbees-days#zurich ● Discount code: CBM50 DevOps World | Jenkins World ● https://www.cloudbees.com/devops-world ● San Francisco: Aug 12-15 (CFP - Mar 24) ● Lisbon: Dec 02-05 (CFP - May) ● Discount code: JAM20 58

Slide 59

Slide 59 text

© 2019 CloudBees, Inc. All Rights Reserved. Contacts: E-mail: [email protected] GitHub: oleg-nenashev Twitter: @oleg_nenashev QUESTIONS? go.cloudbees.com 59