Moscow Jenkins Meetup #3: Groovy Hooks and Configuration-as-Code plugin

This is an english version of my talk about managing Jenkins configurations with Groovy Boot Hooks and about the new Configuration-as-Code Plugin Initiative (JEP-201).

Demo: https://github.com/oleg-nenashev/demo-jenkins-config-as-code
Meetup: https://www.meetup.com/Moscow-Jenkins-Meetup/events/244507783/

Transcript

1. Managing Jenkins with Groovy Hooks + Config-as-Code plugin Oleg Nenashev CloudBees, Inc. Moscow Jenkins Meetup Nov 24, 2017
2. © 2017 CloudBees, Inc. All Rights Reserved. 2 @oleg_nenashev, @jenkins_ru, #jenkinsMsk About me @oleg_nenashev oleg-nenashev librecores.org Jenkins SPb SPbSPU
3. © 2017 CloudBees, Inc. All Rights Reserved. 3 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Oleg’s “Hall of Shame”(c) • Jenkins Core • Windows Service Wrapper • Plugins • Remoting
4. © 2017 CloudBees, Inc. All Rights Reserved. 4 @oleg_nenashev, @jenkins_ru, #jenkinsMsk üGROOVY INIT SCRIPTS üLIVE DEMO üCONFIGURATION AS CODE PLUGIN Outline Disclaimer: • The presentation represents author’s personal opinion • It may differ from official positions of Jenkins project and/or CloudBees
5. © 2017 CloudBees, Inc. All Rights Reserved. 5 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Configuration as Code in Jenkins Jobs System configuration
6. © 2017 CloudBees, Inc. All Rights Reserved. 6 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Configuration as Code in Jenkins Jobs System configuration
7. © 2017 CloudBees, Inc. All Rights Reserved. 7 @oleg_nenashev, @jenkins_ru, #jenkinsMsk “Infrastructure as Code” in Jenkins API Clients Jenkins CLI and REST API python- jenkins jenkins-client (java) Configuration Management Ansible, Chef, … Docker, Docker Compose ... Jenkins- internal solutions Groovy Boot Hooks Scriptler Plugin SCM Sync Configuration
8. Groovy Hooks • Direct access to Jenkins API • Extensible hook engine • init, boot-failure – in the core
9. © 2017 CloudBees, Inc. All Rights Reserved. 9 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Groovy Hooks • Script locations: • WEB-INF/HOOK.groovy in jenkins.war • WEB-INF/HOOK.groovy.d/*.groovy in jenkins.war • $JENKINS_HOME/HOOK.groovy • $JENKINS_HOME/HOOK.groovy.d/*.groovy • Execution in alphabetical order https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script *HOOK = [init, boot]
10. © 2017 CloudBees, Inc. All Rights Reserved. 10 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Example https://github.com/oleg-nenashev/demo-jenkins-config-as-code
11. © 2017 CloudBees, Inc. All Rights Reserved. 11 @oleg_nenashev, @jenkins_ru, #jenkinsMsk •Read-only configs •No Jenkins/Administer •But… no read-only admin UI L Static configuration? It’s possible!
12. © 2017 CloudBees, Inc. All Rights Reserved. 12 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Static configuration in Jenkins • Base: https://github.com/jenkinsci/docker • plugins.txt Docker • Jenkins configuration • Setup of folders and jobs Groovy Init Hooks • Jenkinsfile in SCM • Organization Folder Pipeline
13. © 2017 CloudBees, Inc. All Rights Reserved. 13 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Example Intellij IDEA Filesystem SCM Plugin • Documentation, Syntax • Static analysis • Debug (only for hooks) Local Jenkins instance Source Code (local .git repos) • Configuration-as-Code • Same as production • Repos with libs • Jenkinsfile JENKINS_HOME (volume)
14. © 2017 CloudBees, Inc. All Rights Reserved. 14 @oleg_nenashev, @jenkins_ru, #jenkinsMsk https://github.com/oleg-nenashev/demo- jenkins-config-as-code
15. © 2017 CloudBees, Inc. All Rights Reserved. 15 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Groovy Hooks. What’s missing? • API Jenkins •Javadoc is not user-friendly •Public API is not enough • Support of classes • Error handling •Errors in Groovy hooks are ignored in Jenkins • Debugging in IDE
16. © 2017 CloudBees, Inc. All Rights Reserved. 16 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Groovy!
17. © 2017 CloudBees, Inc. All Rights Reserved. 17 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Groovy Hooks. Solving issues • API Jenkins study it? L • Support of classes • Error handling • Debugging in IDE GroovyBootstrap .groovy Let’s run Groovy engine from… Groovy script
18. © 2017 CloudBees, Inc. All Rights Reserved. 18 @oleg_nenashev, @jenkins_ru, #jenkinsMsk https://github.com/oleg-nenashev/jenkins-config-as-code-demo/ blob/master/init_scripts/src/main/groovy/GroovyBootstrap.groovy More class examples: https://github.com/librecores/librecores-ci
19. © 2017 CloudBees, Inc. All Rights Reserved. 19 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Summary – Groovy Hooks • Works well for static configurations • High entry barrier – API and docs • Migration/rollback… L • Update configs – by code. Incremental version update scripts? • Rollback – only via backups • Plugin installation – reload may be needed ▸OR: external tools (e.g. plugins.txt in Docker)
20. Configuration as Code. What else?
21. © 2017 CloudBees, Inc. All Rights Reserved. 21 @oleg_nenashev, @jenkins_ru, #jenkinsMsk “Infrastructure as Code” in Jenkins API for external tools Jenkins CLI and REST API python- jenkins jenkins-client (java) Configuration Management Ansible, Chef, … Docker, Docker Compose ... Jenkins- internal solutions Groovy Boot Hooks Scriptler Plugin SCM Sync Configuration
22. © 2017 CloudBees, Inc. All Rights Reserved. 22 @oleg_nenashev, @jenkins_ru, #jenkinsMsk State in 2016 - Moscow Meetup #1 • No default solution recommended by the Jenkins Project • Configuration Management – outside Jenkins Project • Time lag in Features • Security fun • Compatibility (e.g. Jenkins 2) • No active work on this front
23. © 2017 CloudBees, Inc. All Rights Reserved. 23 @oleg_nenashev, @jenkins_ru, #jenkinsMsk System Config DSL Plugin • Groovy DSL • Started by Kohsuke Kawaguchi in 2015 • Status: Abandoned • Forked/updated by Praqma in 2016/17 • Not production-ready https://github.com/jenkinsci/system-config-dsl-plugin
24. © 2017 CloudBees, Inc. All Rights Reserved. 24 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Configuration-as-Code Plugin • Contributors: Praqma and CloudBees • JEP-201 – two prototypes • Status: DRAFT • https://github.com/jenkinsci/jep/tree/master/jep/201 • Location: jenkinsci/configuration-as-code-plugin
25. © 2017 CloudBees, Inc. All Rights Reserved. 25 @oleg_nenashev, @jenkins_ru, #jenkinsMsk JEP-201. Prototype 1 • Prehistory: • https://github.com/Praqma/JenkinsAsCodeReference • Docker + Docker Compose • Groovy hooks • Not enough flexibility • Hard to customize and extend • => Praqma started from forking System Config DSL Plugin
26. © 2017 CloudBees, Inc. All Rights Reserved. 26 @oleg_nenashev, @jenkins_ru, #jenkinsMsk JEP-201. Prototype 1 Approach Format: JSON Needs Configurator classes for config elements Requires plugin patches or external plugins L https://github.com/jenkinsci/configuration-as-code- plugin/tree/praqma-prototype Developer: Ewelina Wilkosz (ewelinawilkosz2)
27. © 2017 CloudBees, Inc. All Rights Reserved. 27 @oleg_nenashev, @jenkins_ru, #jenkinsMsk JEP-201. Prototype 1 BUT: • Patches are needed • JSON schemas and class generators (jsonschema2pojo) https://github.com/jenkinsci/configuration -as-code-plugin/tree/praqma-prototype
28. © 2017 CloudBees, Inc. All Rights Reserved. 28 @oleg_nenashev, @jenkins_ru, #jenkinsMsk JEP-201. Prototype 2 Approach Format: YAML Automatic generation of parsers for the most of plugins (needs patching the Core) (?) Extension points for complex cases and legacy plugins (?) Snippet generators (like in Pipeline) https://github.com/jenkinsci/configuration-as-code-plugin Разработчик: Nicolas de Loof (ndeloof)
29. © 2017 CloudBees, Inc. All Rights Reserved. 29 @oleg_nenashev, @jenkins_ru, #jenkinsMsk JEP-201. Prototype 2 LDAP Settings
30. © 2017 CloudBees, Inc. All Rights Reserved. 30 @oleg_nenashev, @jenkins_ru, #jenkinsMsk JEP-201. Prototype 2 Mailer Settings
31. © 2017 CloudBees, Inc. All Rights Reserved. 31 @oleg_nenashev, @jenkins_ru, #jenkinsMsk JEP-201. Summary • Early stage • Implementation/design - TBD • Can be used together with Groovy hooks • Development tools - TBD…
32. © 2017 CloudBees, Inc. All Rights Reserved. 32 @oleg_nenashev, @jenkins_ru, #jenkinsMsk JEP-201. Summary • Early stage • Implementation/design - TBD • Can be used together with Groovy hooks • Development tools - TBD… • It’s a good time to contribute ;) • https://groups.google.com/forum/#!forum/jenkinsci-dev • https://github.com/jenkinsci/configuration-as-code-plugin
33. © 2017 CloudBees, Inc. All Rights Reserved. 33 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Takeaways • Groovy hooks are useful for static configs ▸Direct access to Java API ▸Development tools
34. © 2017 CloudBees, Inc. All Rights Reserved. 34 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Takeaways • Groovy hooks are useful for static configs • Weaponize it: GroovyBootstrap ▸Support of classes ▸Debugging ▸Error handling
35. © 2017 CloudBees, Inc. All Rights Reserved. 35 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Takeaways • Groovy hooks are useful for static configs • Weaponize it: GroovyBootstrap • Groovy hooks are not a silver bullet ▸Entry barrier / API complexity ▸Update/downgrade of configs ▸Plugin installation
36. © 2017 CloudBees, Inc. All Rights Reserved. 36 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Takeaways • Groovy hooks are useful for static configs • Weaponize it: GroovyBootstrap • Groovy hooks are not a silver bullet • Follow JEP-201 “Config-as-Code Plugin” • https://github.com/jenkinsci/configuration- as-code-plugin
37. © 2017 CloudBees, Inc. All Rights Reserved. 37 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Контакты: E-mail: onenashev@cloudbees.com GitHub: oleg-nenashev Twitter: @oleg_nenashev QUESTIONS? go.cloudbees.com
38. © 2017 CloudBees, Inc. All Rights Reserved. 38 @oleg_nenashev, @jenkins_ru, #jenkinsMsk Links Demo on DockerHub • https://hub.docker.com/r/onenashev/demo-jenkins-config-as-code/ Groovy Hook examples • https://github.com/oleg-nenashev/demo-jenkins-config-as-code • https://github.com/librecores/librecores-ci • https://github.com/Praqma/JenkinsAsCodeReference Configuration-as-Code Plugin • https://github.com/jenkinsci/configuration-as-code-plugin • https://github.com/jenkinsci/jep/tree/master/jep/201
39. Software at the speed of ideas THANK YOU! www.cloudbees.com