DW|JW 2019 - Docker and Jenkins [as code]

Docker is everywhere in the CI/CD pipelines and in development tools. Jenkins, one of the most popular automation servers, is actively evolving to improve its support of container technologies and static configurations in images. The Configuration as Code plugin is a new milestone which enables managing Jenkins configurations via YAML. Together with Docker, this plugin offers many ways to produce ready-to-fly Jenkins images for any environments. In our talk, we will describe official master and agent images offered by the Jenkins project. What's inside them? How do you configure images with JCasC and Groovy hooks? How do you use these approaches together? How do CloudBees Jenkins Distribution and CloudBees Assurance program help with plugin management? And what's next for Jenkins Docker packaging and JCasC?

Presentation by Mark Waite and me at DevOps World | Jenkins World 2019. https://sched.co/UauW

Transcript

1. Docker and Jenkins [as code]
2. © 2019 All Rights Reserved. 2 About us @oleg_nenashev oleg-nenashev @MarkEWaite MarkEWaite Long-time contributors Platform SIG https://jenkins.io/sigs/platform Docker image maintainers CloudBees
3. © 2019 All Rights Reserved. 3 About you
4. © 2019 All Rights Reserved. 4 Today • Docker packaging for Jenkins • Configuration as Code for Jenkins and Docker • What about CloudBees? • What’s next for Jenkins and Docker? HINT: Join us at the community booth for deep dive!
5. © 2019 All Rights Reserved. 5 CI/CD. Why containers? Config Management Fast provisioning Clean environments Disposability Clustering 5
6. © 2019 All Rights Reserved. 6
7. © 2019 All Rights Reserved. 7 Docker Docker Pipeline Docker Build Steps Yet Another Docker ... Kubernetes Kubernetes Pipeline Jenkins & Docker - Plugins
8. © 2019 All Rights Reserved. 8 Example: Docker Pipeline Plugin https://plugins.jenkins.io/docker-workflow docker.image('onenashev/gcc-riscv:6.4').inside { checkout scm sh "make clean test" }
9. © 2019 All Rights Reserved. 9 Jenkins & Docker integrations keep evolving https://plugins.jenkins.io/docker-workflow
10. © 2019 All Rights Reserved. 10 Jenkins runs in Docker
11. © 2019 All Rights Reserved. 11 Jenkins & Docker. Packaging https://hub.docker.com/r/jenkins/
12. © 2019 All Rights Reserved. 12 • jenkins/slave – base image with Remoting • jenkins/jnlp-slave – JNLP agents • jenkins/ssh-slave – SSH agents • jenkins/remoting-kafka-agent - Kafka Agents Official Agent Images https://hub.docker.com/r/jenkins/
13. © 2019 All Rights Reserved. 13 Remoting over Apache Kafka GSoC 2018 / 2019 Jenkins networking over Apache Kafka Docker images Helm charts for agents and Masters https://github.com/jenkinsci/remoting-kafka-plugin
14. © 2019 All Rights Reserved. 14 Jenkins Masters images • Production - jenkins/jenkins • Weekly and LTS releases • OpenJDK 8 and 11 • Debian and CentOS • Alpine and Debian Slim for OpenJDK 8
15. © 2019 All Rights Reserved. 15 Jenkins Masters images • Production - jenkins/jenkins • Weekly and LTS releases • OpenJDK 8 and 11 • Debian and CentOS • Alpine and Debian Slim for OpenJDK 8 • Experimental - jenkins4eval/jenkins • ARM, ARM 64, IBM s/390, PowerPC 64 LE • Continuous delivery for Jenkins master branches • More platforms are coming soon
16. © 2019 All Rights Reserved. 16 Official image: jenkins/jenkins docker run --rm \ -p 8080:8080 \ -p 50000:50000 \ -v jenkins_home:/var/jenkins_home \ jenkins/jenkins:lts https://github.com/jenkinsci/docker
17. © 2019 All Rights Reserved. 17 Jenkins Image is not enough • No plugins • No configuration • No jobs
18. © 2019 All Rights Reserved. 18
19. © 2019 All Rights Reserved. 19 Configuration as Code in Jenkins Jobs System configuration
20. © 2019 All Rights Reserved. 20 Docker images One FROM to rule the world
21. © 2019 All Rights Reserved. 21 Extending Docker images Image source: http://container.farm/special-extended-container/ YOUR STUFF
22. © 2019 All Rights Reserved. 22 Extending Docker images Tools Environment Configuration Plugins Docs: https://github.com/jenkinsci/docker
23. © 2019 All Rights Reserved. 23 Dockerfile
24. © 2019 All Rights Reserved. 24 Plugin management
25. © 2019 All Rights Reserved. 25 Plugin bundling jenkins/jenkins:2.190.3
26. © 2019 All Rights Reserved. 26 Plugin bundling jenkins/jenkins:2.190.3 # Pinned version blueocean:1.8.2 # Latest antisamy-markup-formatter matrix-auth:latest # Latest from the Experimental update center git-client:experimental # Incrementals (JEP-305) workflow-support:incrementals;org.jenkins-ci.plugins.workflow;2.21-rc591.43d37d4d080a plugins.txt
27. © 2019 All Rights Reserved. 27 Limitations • Latest versions of non-specified dependencies are installed • No “Check for updates” feature • Plaintext Format • No YAML • No Dependabot integration, etc.
28. © 2019 All Rights Reserved. 28 Plugin installation manager tool https://github.com/jenkinsci/plugin-installation-manager-tool • CLI tool • Java, not bash script • YAML support • Updates listing • Transient dependency management plugins: artifactId: git source: version: latest artifactId: job-import-plugin source: version: 3.1 artifactId: docker artifactId: docker-commons source: version: 2.4.4 artifactId: matrix-project
29. © 2019 All Rights Reserved. 29 Configurations
30. © 2019 All Rights Reserved. 30 Groovy Hooks [1/2] • Standard Groovy engine • Triggered by Jenkins • Direct access to the runtime • Extensible hook engine • 2 hook types in the core: init, boot-failure https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script
31. © 2019 All Rights Reserved. 31 Groovy Hooks [2/2] Script locations: • WEB-INF/HOOK.groovy in jenkins.war • WEB-INF/HOOK.groovy.d/*.groovy in jenkins.war • $JENKINS_HOME/HOOK.groovy • $JENKINS_HOME/HOOK.groovy.d/*.groovy Execution in alphabetical order https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script
32. © 2019 All Rights Reserved. 32 Jenkins Startup Flow JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() PLUGINS_STARTED . . .
33. © 2019 All Rights Reserved. 33 Example https://github.com/oleg-nenashev/demo-jenkins-config-as-code
34. © 2019 All Rights Reserved. 34 Adding Groovy hooks to the standard image Just put Groovy scripts in the right place! https://github.com/oleg-nenashev/demo-jenkins-config-as-code
35. © 2019 All Rights Reserved. 35 https://github.com/oleg-nenashev/demo-jenkins-config-as-code
36. © 2019 All Rights Reserved. 36 More examples oleg-nenashev/demo-jenkins-config-as-code Praqma/JenkinsAsCodeReference librecores/librecores-ci
37. © 2019 All Rights Reserved. 37 Add JCasC Plugin! Configuration-as-Code Plugin https://plugins.jenkins.io/configuration-as-code
38. © 2019 All Rights Reserved. 38 JCasC Plugin • Sub-project in Jenkins • Feb 2018 - first alpha • Sep 2018 - 1.0 • Now - 1.35 • Configuration via YAML jenkinsci/configuration-as-code-plugin gitter.im/jenkinsci/configuration-as-code-plugin
39. © 2019 All Rights Reserved. 39 jenkins: systemMessage: "JCasC Demo" numExecutors: 1 scmCheckoutRetryCount: 4 mode: NORMAL securityRealm: local: allowsSignup: false users: - id: demoAdmin password: ${adminpw} jenkins.yaml YAML specification
40. © 2019 All Rights Reserved. 40 • Configuration export from Jenkins • CLI and REST API for configuration management • Pre-flight validations and dry-run • Support for many plugins out-of-the-box • External credentials • Job/Folder management via JobDSL Features
41. © 2019 All Rights Reserved. 41 Adding JCasC to the Docker image Configuration file Needs JCasC Plugin … configuration-as-code:1.35 …
42. © 2019 All Rights Reserved. 42 Combining JCasC and Groovy hooks JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() JCasC PLUGINS_STARTED . . . Groovy hooks run after JCasC Hooks can fine-tune the configuration: • Missing integrations • Dynamic scripting • Job configurations and migrations Use JCasC where possible!
43. © 2019 All Rights Reserved. 43 Comparison JCasC Plugin Groovy Hooks YAML – declarative definition Low barrier to entry Limited integrations Forward compatibility risks External rollback Apply changes w/o restart No debugging, YAGNI? Dry run mode + validation Export from UI CLI and REST API for CM Plugin management Groovy – general-purpose language High barrier to entry Can manage everything Compatible if API is stable External rollback Restart is required Debugging support Plugin Management
44. © 2019 All Rights Reserved. 44 Comparison JCasC Plugin Groovy Hooks YAML – declarative definition Low barrier to entry Limited integrations Forward compatibility risks External rollback Apply changes w/o restart No debugging, YAGNI? Dry run mode + validation Export from UI CLI and REST API for CM Plugin management Groovy – general-purpose language High barrier to entry Can manage everything Compatible if API is stable External rollback Restart is required Debugging support Plugin Management Use JCasC when possible, combine with Groovy hooks when needed
45. Mark’s success story Fast setup, better interactive testing
46. © 2019 All Rights Reserved. 46 Git Plugin Development - Don’t Break the Users! • Automated tests are great, but… • Most git plugin bugs are not detected by automated tests • Git plugin users don’t like being broken by plugin upgrades • Choices • Automate everything, don’t release until it is all automated • Interactive tests of interesting and useful cases in addition to automation • Interactive test configuration is COMPLICATED • Platforms - Alpine, CentOS, Debian, Fedora, FreeBSD, OpenSUSE, Windows • Git versions - 1.8, 1.9, 2.11, 2.17, 2.23, 2.24, … • ANSWER: Simplify test setup by managing it in a repository (several repos)
47. © 2019 All Rights Reserved. 47 Faster Interactive Test Setup - Docker & Jenkins Public Repo • LTS • Debian • Debian Slim • Alpine • JDK 11 • LTS, plugins • Debian • Debian Slim • Alpine • JDK 11 Private Repo • LTS, plugins, & credentials • Debian • Debian Slim • Alpine • JDK 11 • LTS, plugins, credentials & agents • Debian • Debian Slim • Alpine • JDK 11 https://github.com/MarkEWaite/docker-lfs $ docker_build.py $ docker_run.py --clean
48. © 2019 All Rights Reserved. 48 Fully Configured Jenkins • 30+ agents • 10+ git versions • Linux - Debian, CentOS, Ubuntu • Two versions of each • FreeBSD & Windows agents • Raspberry Pi 4 agent • Cloud and local agents • 1000+ test jobs • Specific bug check jobs • Easy to start, easy to stop • Any Linux machine in network https://github.com/MarkEWaite/docker-lfs
49. © 2019 All Rights Reserved. 49 © 2019 CloudBees, Inc. All Rights Reserved. What about CloudBees?
50. © 2019 All Rights Reserved. 50 Products • All products are available as Docker images ◦ https://hub.docker.com/u/cloudbees • Helm charts for CloudBees Jenkins Distribution • CloudBees Jenkins Distribution and CloudBees Core run on-premise or in the cloud ◦ Docker ◦ Kubernetes - Helm Charts ◦ Virtual machines ◦ Bare metal
51. © 2019 All Rights Reserved. 51 Plugins • All Jenkins Docker plugins are available to customers • Extended SLA for several plugins https://docs.cloudbees.com/plugins
52. © 2019 All Rights Reserved. 52 JCasC in CloudBees products • Tier 2 (Compatible) plugin ◦ SLA: Bugfixes and Security fixes ◦ CloudBees Jenkins Support ◦ CloudBees Jenkins Distribution • Technical preview for CloudBees Core ◦ Standalone and Managed masters ◦ Configuration Bundles feature http://bit.ly/jcasc-cloudbees
53. © 2019 All Rights Reserved. 53 Plugin Management • Added value features for Docker • CloudBees Assurance Program and Beekeper ◦ https://docs.cloudbees.com/docs/admin-resources/latest/ assurance-program/ • Plugin catalog for CloudBees Core ◦ https://docs.cloudbees.com/docs/admin-resources/latest/ plugin-management/configuring-plugin-catalogs
54. © 2019 All Rights Reserved. 54 CloudBees Assurance Program • Verified plugin sets • Upgrade assistant • Administrative monitors • Opt-in automatic upgrade
55. © 2019 All Rights Reserved. 55 Lists the set of plugins, plugin versions and plugin dependencies that CloudBees has tested for compatibility and security. Administrative dashboard provides a centralized view of the monitored Jenkins plugins, recommended actions, and configuration options available.
56. © 2019 All Rights Reserved. 56 JCasC in CloudBees Jenkins Distribution • Combines with CloudBees Assurance Program • Documentation and Quick Start ◦ http://bit.ly/cjd-jcasc • Demo ◦ https://github.com/cloudbees-oss/cjd-jcasc-demo
57. © 2019 All Rights Reserved. 57 Back to Jenkins and Docker
58. © 2019 All Rights Reserved. 58 Upcoming changes in JCasC • Hardening the implementation • New features • Plugin compatibility is a focus ◦ Fixes in ~50 plugins • Jenkins core architectural changes ◦ Read-only configuration UI ◦ Initialization milestones • Developer tools (VS Code Plugin, etc.)
59. © 2019 All Rights Reserved. 59 Upcoming changes in Docker • New master images / platforms ◦ Windows ◦ Ubuntu ◦ UBI ◦ AdoptOpenJDK with HotSpot ◦ AdoptOpenJDK with OpenJ9 • Multi-platform Docker image • New agent images
60. © 2019 All Rights Reserved. 60 Takeaways 1. Dockerize Jenkins 2. Use & extend official images 3. Use static configurations, pin all versions 4. Use Configuration-as-Code
61. © 2019 All Rights Reserved. 61 It is a great time to participate! Join Platform SIG • https://jenkins.io/sigs/platform/ • https://gitter.im/jenkinsci/platform-sig Contribute • https://jenkins.io/participate/ • https://github.com/jenkinsci/docker • https://github.com/jenkinsci/configuration-as-code-plugin
62. © 2019 All Rights Reserved. 62 Join us at the community booth! • Any questions • More demos • Custom image packager • JCasC experts • How to contribute? • ...
64. © 2019 All Rights Reserved. 64 DevOptics Software Delivery Visibility & Insights Core Unified Software Delivery & Governance CodeShip CI/CD as a Service Flow Adaptive Release Orchestration DevOps Excellence Jenkins CloudBees Jenkins Distribution CloudBees Jenkins X Distribution 24x7 Technical Support Assisted Updates Support Accelerator Training Customer Success Managers DevOps Consultants Rollout Feature Flag Management Continuous Delivery Products and Services