GBG JAM - Managing Jenkins with Groovy Hooks and JCasC

Can we consider the emerging Configuration as Code Plugin as a silver-bullet for every use-case? Of course not: it does not support each Jenkins plugin and does not offer scripting support for complex cases. But you can use other tools to achieve that! We will talk about the Groovy Hooks engine in Jenkins, which allows working with Jenkins on a low-level using a universal programming language. How to use it efficiently? How to use Groovy Hooks together with Configuration as Code plugin?

Transcript

1. Managing Jenkins with Groovy Hooks And JCasC plugin Oleg Nenashev CloudBees, Inc. Gothenburg Jenkins Meetup May 28, 2018
2. 2 @oleg_nenashev, #codeconf > whoami librecores.org Meetups SPbSPU @oleg_nenashev oleg-nenashev
3. 3 @oleg_nenashev, #codeconf > whoami -jenkins Hudson/Jenkins user since 2008 Jenkins contributor since 2012 Now: • Maintainer: Core, Remoting, plugins • Security Team member • Meetup organizer • Google Summer of Code org admin @oleg_nenashev oleg-nenashev
4. 4 @oleg_nenashev, #codeconf About you
5. 5 @oleg_nenashev, #codeconf • Scripting languages are fine • YAML/JSON/XML are fine as well • You can use them together This talk is NOT about holywars
6. 6 @oleg_nenashev, #codeconf üGROOVY HOOKS üGROOVY HOOKS & CASC PLUGIN üTIPS & TRICKS üSOME LIVE DEMOS Outline Slides:
7. 7 @oleg_nenashev, #codeconf Configuration as Code in Jenkins Jobs System configuration
8. 8 @oleg_nenashev, #codeconf Configuration as Code in Jenkins Jobs System configuration
9. 9 @oleg_nenashev, #codeconf “Infrastructure as Code” in Jenkins API Clients Jenkins CLI and REST API python- jenkins jenkins-client (java) Configuration Management Ansible, Chef, … Docker, Docker Compose ... Jenkins- internal solutions Groovy Boot Hooks Scriptler Plugin Configuration -as-Code Plugin
10. 10 @oleg_nenashev, #codeconf State in 2016 – Stockholm JAM • No default solution recommended by the Jenkins Project • No active work on this front • System Config DSL, was abandoned • Configuration Management – outside Jenkins Project • Time lag in Features • Security fun • Compatibility (e.g. Jenkins 2)
11. 11 @oleg_nenashev, #codeconf Configuration-as-Code Plugin http://bit.ly/casc-plugin-slides (older version) • Presented by Ewelina • The plugin is in alpha • Final version may differ
12. 12 @oleg_nenashev, #codeconf Limitations (temporary?). Features • Not all plugins are supported • No Folders support – cannot set project structure • No debugging support • Export from UI is not working well
13. 13 @oleg_nenashev, #codeconf Limitations (temporary?). Architecture • Forward/backward compatibility is not guaranteed • YAML - No dynamic logic / scripting • Startup Chicken&Egg for plugins • Race conditions for jobs • Handling of removed sections
14. 14 @oleg_nenashev, #codeconf Limitations. Forward/backward compatibility • Config many break… • … even if a core/plugin change in a compatible way • Example: Work directory support in 2.73.x • New fields were added to the configuration in the core • Class @DataBoundConstructor interface changed, but old one is in place • workDir settings must be always set in CasC now • Ticket: JCasC Issue #167
15. 15 @oleg_nenashev, #codeconf Limitations. Jobs loading JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED CasC Initializer PLUGINS_STARTED . . . • CasC happens in parallel with Jobs loading! • CasC may happen in parallel with initializers • Race conditions if job loading depends on plugin settings • Extra milestones are needed in the core
16. 16 @oleg_nenashev, #codeconf
17. 17 @oleg_nenashev, #codeconf Groovy!
18. 18 @oleg_nenashev, #codeconf
19. 19 @oleg_nenashev, #codeconf Groovy Hooks [1/2] • Standard Groovy engine • Triggered by Jenkins • Direct access to the runtime • Extensible hook engine • init, boot-failure – in the core https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script
20. 20 @oleg_nenashev, #codeconf Groovy Hooks [2/2] • Script locations: •WEB-INF/HOOK.groovy in jenkins.war •WEB-INF/HOOK.groovy.d/*.groovy in jenkins.war •$JENKINS_HOME/HOOK.groovy •$JENKINS_HOME/HOOK.groovy.d/*.groovy • Execution in alphabetical order https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script
21. 21 @oleg_nenashev, #codeconf Groovy Init Hooks JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() [CasC if installed] PLUGINS_STARTED . . .
22. 22 @oleg_nenashev, #codeconf Example https://github.com/oleg-nenashev/demo-jenkins-config-as-code
23. 23 @oleg_nenashev, #codeconf https://github.com/oleg-nenashev/demo- jenkins-config-as-code
24. 24 @oleg_nenashev, #codeconf Demo. Local Pipeline Development Env Intellij IDEA Filesystem SCM Plugin • Documentation, Syntax • Static analysis • Debug (only for hooks) Local Jenkins instance Source Code (local .git repos) • Configuration-as-Code • Same as production • Repos with libs • Jenkinsfile JENKINS_HOME (volume)
25. 25 @oleg_nenashev, #codeconf More examples • https://github.com/oleg-nenashev/demo-jenkins-config-as-code • https://github.com/Praqma/JenkinsAsCodeReference • https://github.com/librecores/librecores-ci
26. 26 @oleg_nenashev, #codeconf Groovy! #ornot
27. 27 @oleg_nenashev, #codeconf Groovy Hook Limitations. General • Jenkins API is scattered and complex • Javadoc is not user-friendly • Public API is not enough sometimes • No export from UI Solution: • Configuration as Code Plugin
28. 28 @oleg_nenashev, #codeconf Groovy Hook Limitations. Missing Features • No class support • No shared libraries support • Error propagation: Jenkins does not stop on script error • Restart is required to apply changes Solution: • See below
29. Groovy Hooks and CasC - make them work together
30. 30 @oleg_nenashev, #codeconf CasC and Groovy hooks JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() CasC PLUGINS_STARTED . . . • Groovy hooks run after CasC • Hooks can fine-tune the configuration: • Missing integrations • Dynamic scripting • Job configurations and migrations
31. 31 @oleg_nenashev, #codeconf https://github.com/oleg-nenashev/demo- jenkins-config-as-code/tree/casc-plugin
32. Groovy Hooks – Tips & Tricks • Class support & Error handling • Development and debugging • Static configurations
33. 33 @oleg_nenashev, #codeconf Class support & Error handling • They say: •Hard to share/reuse, no class support •No error propagation
34. 34 @oleg_nenashev, #codeconf
35. 35 @oleg_nenashev, #codeconf Let’s run Groovy engine from… a Groovy script Groovy Bootstrap
36. 36 @oleg_nenashev, #codeconf https://github.com/oleg-nenashev/jenkins-config-as-code-demo/ blob/master/init_scripts/src/main/groovy/GroovyBootstrap.groovy More class examples: https://github.com/librecores/librecores-ci
37. 37 @oleg_nenashev, #codeconf What can be done? Almost anything… • Class support (in demo) • Error propagation (in demo) • Library loading… ▸@Grab or other library management ▸Library bundling into WAR / Docker image • Partial CasC plugin invocation • Apply config on API call • …
38. 38 @oleg_nenashev, #codeconf Groovy Hook Development?
39. 39 @oleg_nenashev, #codeconf 1. Use your favorite IDE with Groovy support 2. Create Maven project 3. Use virtual Jenkins “plugin” with deps • Static analysis • Test automation Groovy Hook Development?
40. 40 @oleg_nenashev, #codeconf Groovy Hook Debugging?
41. 41 @oleg_nenashev, #codeconf 1. Start Jenkins with Debug flags 2. Just attach your IDE Groovy Hook Debugging? https://github.com/oleg-nenashev/demo- jenkins-config-as-code/blob/master/jenkins2.sh
42. 42 @oleg_nenashev, #codeconf •Read-only configurations •No Jenkins/Administer •But… no read-only admin UI L Static configuration? It’s possible!
43. 43 @oleg_nenashev, #codeconf Static configuration in Jenkins • Base: https://github.com/jenkinsci/docker • plugins.txt Docker • Jenkins configuration • Setup of folders and jobs Groovy Init Hooks • Jenkinsfile in SCM • Organization Folder Pipeline
44. What does not “just work”?
45. 45 @oleg_nenashev, #codeconf Comparison JCasC Plugin Groovy Hooks YAML – standard declarative definition Low barrier to entry Limited integrations Compatibility issues External rollback Apply changes w/o restart No debugging, YAGNI? Future: Export from UI Groovy – general-purpose OOP language High barrier to entry Can manage everything Compatible if API is stable External rollback Restart is required Debugging support
46. 46 @oleg_nenashev, #codeconf What is missing in BOTH engines? • Rollback support •Job/Build entries may be migrated with any update •Update is possible, but there may be breaking changes •Rollback – only via backups
47. 47 @oleg_nenashev, #codeconf What is missing in BOTH engines? • Rollback support •Job/Build entries may be migrated with any update •Update is possible, but there may be breaking changes •Rollback – only via backups Solution: • External config management logic (change config, test initialization, rollback) • Backup plugin with CLI / REST API for Restore (e.g. Periodic Backup Plugin)
48. 48 @oleg_nenashev, #codeconf What is missing in BOTH engines? • Rollback support • Plugin installation Chicken & Egg •New plugin installation is possible •Other cases: Reload may be needed ▸upgrade, downgrade, uninstall ▸CasC plugin self-upgrade? JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() CasC PLUGINS_STARTED . . .
49. 49 @oleg_nenashev, #codeconf What is missing in BOTH engines? • Rollback support • Plugin installation Chicken & Egg •New plugin installation is possible •Other cases: Reload may be needed JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() CasC PLUGINS_STARTED . . . Solutions: A. CasC – work in progress (by Nicolas de Loof) B. External pre-startup tools • plugins.txt in stock Docker images • Custom WAR Packager (WiP by Oleg)
50. 50 @oleg_nenashev, #codeconf What is missing in BOTH engines? • Rollback support • Plugin installation Chicken & Egg • UX: Lack of support in the Jenkins core •Admin UI is must-have for diagnostics •But: ▸No read-only admin UI ▸No read-only job configurations for admins ▸No read-only support in API
51. 51 @oleg_nenashev, #codeconf Read-only UI. Partial solution • “sudo” mode in Matrix Auth or Role Strategy plugin •Task for Role Strategy plugin (JENKINS-51567) • Admin permissions are granted only when required • Job with System Groovy script to grant/revoke permissions • OR: REST API for Admin
52. So…
53. 53 @oleg_nenashev, #codeconf Takeaways • Two engines, no silver bullet JCasC Plugin Groovy Hooks YAML – standard declarative definition Low barrier to entry Limited integrations Compatibility issues External rollback Apply changes w/o restart No debugging, YAGNI? Future: Export from UI Groovy – general-purpose OOP language High barrier to entry Can manage everything Compatible if API is stable External rollback Restart is required Debugging support
54. 54 @oleg_nenashev, #codeconf Takeaways • Select an engine depending on your needs and expertise • New to Jenkins? Use JCasC
55. 55 @oleg_nenashev, #codeconf Takeaways • Select an engine depending on your needs and expertise • New to Jenkins? Use JCasC • Combine engines if needed
56. 56 @oleg_nenashev, #codeconf Takeaways • It’s a great time to contribute ;) •https://jenkins.io/participate
57. 57 @oleg_nenashev, #codeconf Links Groovy hooks: • https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script • These slides: https://speakerdeck.com/onenashev/gbg-jam-managing- jenkins-with-groovy-hooks-and-jcasc Demo and examples: • https://hub.docker.com/r/onenashev/demo-jenkins-config-as-code/ • https://github.com/oleg-nenashev/demo-jenkins-config-as-code • https://github.com/Praqma/JenkinsAsCodeReference • https://github.com/librecores/librecores-ci Configuration-as-Code Plugin • https://github.com/jenkinsci/configuration-as-code-plugin
58. 58 @oleg_nenashev, #codeconf Contacts: E-mail: onenashev@cloudbees.com GitHub: oleg-nenashev Twitter: @oleg_nenashev QUESTIONS? go.cloudbees.com
59. Software at the speed of ideas THANK YOU! www.cloudbees.com