CPH Jenkins Meetup: Developing Jenkins Pipeline

If you have ever tried developing a Jenkins Pipeline script or a library, you may have noticed how long it takes to deploy a new version to server to discover just another syntax error. I will show how to edit and test Pipeline libraries locally before committing to the repository, with Configuration-as-Code and Docker.

Transcript

1. © 2018 CloudBees, Inc. All Rights Reserved. Developing Jenkins Pipelines Oleg Nenashev, CloudBees Copenhagen, Oct 18, 2018
2. © 2018 CloudBees, Inc. All Rights Reserved. > whoami @oleg_nenashev oleg-nenashev • Based in Neuchatel, Switzerland • Engineer @ CloudBees • CloudBees Core & CloudBees Jenkins Support • User since 2008, contributor since 2012 • Jenkins Core Maintainer • Jenkins Ambassador: meetups, GSoC, etc. 2
3. © 2018 CloudBees, Inc. All Rights Reserved. Configuration as Code in Jenkins Job Configuration System Configuration 3
4. © 2018 CloudBees, Inc. All Rights Reserved. What is Jenkins Pipeline? Configuration as Code for jobs “Pipeline as Code” 4
5. © 2018 CloudBees, Inc. All Rights Reserved. Jenkins Pipeline is a… 1. Groovy DSL for automatic flow description - Defined in SCM or in Job configuration - 2 syntaxes: Scripted and Declarative // Jenkinsfile node("linux") { git url:"http://github.com/myorg/myproject.git" sh "make all" } http://bit.ly/pipeline-tutorial 5
6. © 2018 CloudBees, Inc. All Rights Reserved. Jenkins Pipeline is a… 1. Groovy DSL for automatic flow description 2. Item type Pipeline Job Multi-Branch Pipeline Organization Folder • GitHub • BitBucket* • … 6
7. © 2018 CloudBees, Inc. All Rights Reserved. Jenkins Pipeline is a… 1. Groovy DSL for automatic flow description 2. Item type 3. Pipeline as Code Ecosystem 7
8. © 2018 CloudBees, Inc. All Rights Reserved. Branch- Source, BlueOce an, ... 89 - explicit support 42 - Pipeline - specific Documentation Samples Development Tools Jenkins Pipeline Ecosystem Plugins: 8
9. © 2018 CloudBees, Inc. All Rights Reserved. Example. CD for Jenkins IRC Bot def imageName = 'jenkinsciinfra/ircbot’ node('docker') { checkout scm def imageTag = findTag() stage('Build ircbot’) { withMavenEnv (["BUILD_NUMBER=${env.BUILD_NUMBER}:${commit}"]) { sh 'make bot' // Make invokes Maven }} stage ('Build & Deploy container’) { def img = docker.build("${imageName}:${imageTag}”) img.push() } } https://github.com/jenkins-infra/ircbot Docker Registry Server Jenkins SCM Puppet 9
10. © 2018 CloudBees, Inc. All Rights Reserved. More about Jenkins Pipeline http://bit.ly/pipeline-docs http://bit.ly/pipeline-compatibility 10
11. © 2018 CloudBees, Inc. All Rights Reserved. My Jenkins Pipelines 11 Scripted Pipeline - Minimal Jenkinsfile Pipeline Libraries
12. © 2018 CloudBees, Inc. All Rights Reserved. 12 Pipeline Shared Libraries Libs are located in SCM Versioning Encapsulation Visibility scopes
13. © 2018 CloudBees, Inc. All Rights Reserved. Library usage 1. Take library components 13
14. © 2018 CloudBees, Inc. All Rights Reserved. Library usage 1. Take library components 2. Build your own bicycle Pipeline Add some glue! 14
15. © 2018 CloudBees, Inc. All Rights Reserved. Pipeline Development Tools 15
16. © 2018 CloudBees, Inc. All Rights Reserved. IDE Integration Unit Test Framework Static Analysis Library manager Debug tools Documen- tation Pipeline deployment Freestyle to Pipeline Converter Good Missing 16 Partial
17. © 2018 CloudBees, Inc. All Rights Reserved. IDE Integration https://st-g.de/2016/08/jenkins- pipeline-autocompletion-in-intellij 17
18. © 2018 CloudBees, Inc. All Rights Reserved. Documentation (root) +- README.md +- LICENSE.txt +- … +- src +- org +- oleg +- MailExtSender.groovy +- vars +- cobertura.groovy +- covertura.txt +- runTests.groovy +- runTests.txt Documentation as Code? Asciidoc / Markdown 18
19. © 2018 CloudBees, Inc. All Rights Reserved. Documentation (root) +- README.md +- LICENSE.txt +- … +- src +- org +- oleg +- MailExtSender.groovy +- vars +- cobertura.groovy +- covertura.txt +- runTests.groovy +- runTests.txt Asciidoc / Markdown Javadoc/Groovydoc Standard tools work for Classes … but not for Global Variables https://github.com/jenkinsci/gmaven/pull/1 https://github.com/jenkins-infra/pipeline- library/pull/22 19
20. © 2018 CloudBees, Inc. All Rights Reserved. Pipeline Testing https://github.com/jenkinsci/JenkinsPipelineUnit “mvn verify” has been invoked… And the execution passed… For the source Jenkinsfile… 20
21. © 2018 CloudBees, Inc. All Rights Reserved. Pipeline Testing. Libraries String clonePath = 'path/to/clone' def library = library() .name('commons') .retriever(gitSource('git@orgithub/devteam/jenkins-shared.git')) .targetPath(clonePath) .defaultVersion("master") .allowOverride(true) .build() helper.registerSharedLibrary(library) // Same as before… loadScript("job/library/exampleJob.jenkins") printCallStack() … 21
22. © 2018 CloudBees, Inc. All Rights Reserved. NEW: Pipeline Spock https://github.com/homeaway/jenkins-spock Checks: • Steps are callable • Global variables are accessible • Shared Lib Global vars are accessible + Mocking of Pipeline objects for unit tests 22
23. © 2018 CloudBees, Inc. All Rights Reserved. So, how to develop Pipeline? … without common tools? 23
24. © 2018 CloudBees, Inc. All Rights Reserved. Patch => Commit => Launch 24
25. © 2018 CloudBees, Inc. All Rights Reserved. Patch => Commit => Launch => Patch => Commit => Launch => Patch => Commit => Launch => ….? 25
26. © 2018 CloudBees, Inc. All Rights Reserved. Patch => Commit => Launch => Patch => Commit => Launch => Patch => Commit => Launch => ….? 26
27. © 2018 CloudBees, Inc. All Rights Reserved. 27
28. © 2018 CloudBees, Inc. All Rights Reserved. How do I work? Intellij IDEA • Syntax file • Built-in documentation Jenkins Test Instance Source Code local .git repo(s) • Production System Snapshot • Custom Pipeline Library Settings • Filesystem SCM plugin • Repositories for libraries • Jenkinsfile repos volume 28
29. © 2018 CloudBees, Inc. All Rights Reserved. Jenkins Master Docker + Configuration as Code (Groovy hooks and JCasC) + Init script tweaks 29
30. © 2018 CloudBees, Inc. All Rights Reserved. Mapping Pipeline Filesystem SCM Plugin 30
31. © 2018 CloudBees, Inc. All Rights Reserved. Test Jobs • Jobs can generated with Configuration-as-Code • Demo scripts come from the Pipeline library • Groovy scripts for dynamic loading 31
32. © 2018 CloudBees, Inc. All Rights Reserved. Live demo https://github.com/oleg-nenashev/demo-jenkins- config-as-code 32
33. © 2018 CloudBees, Inc. All Rights Reserved. How could I make the demo better? • Jenkins Configuration as Code Plugin • Custom WAR Packager • // to be presented tomorrow 33
34. © 2018 CloudBees, Inc. All Rights Reserved. What do I need? Dev Tools for Pipeline • IDE, Debugger, Test Tools Community Pipeline Libraries • Support common use-cases without plugins • Pipeline-only plugins work too 34
35. © 2018 CloudBees, Inc. All Rights Reserved. Takeaways There are development tools Pipeline development is possible Hacks are required 35
36. © 2018 CloudBees, Inc. All Rights Reserved. Useful links These slides: • https://speakerdeck.com/onenashev (to be published) Demo: • https://github.com/oleg-nenashev/demo-jenkins- config-as-code 36
37. © 2018 CloudBees, Inc. All Rights Reserved. Thank You! www.cloudbees.com Company Confidential