Upgrade to Pro — share decks privately, control downloads, hide ads and more …

GBG JAM - Managing Jenkins with Groovy Hooks and JCasC

GBG JAM - Managing Jenkins with Groovy Hooks and JCasC

Can we consider the emerging Configuration as Code Plugin as a silver-bullet for every use-case? Of course not: it does not support each Jenkins plugin and does not offer scripting support for complex cases. But you can use other tools to achieve that! We will talk about the Groovy Hooks engine in Jenkins, which allows working with Jenkins on a low-level using a universal programming language. How to use it efficiently? How to use Groovy Hooks together with Configuration as Code plugin?

Oleg Nenashev

May 28, 2018
Tweet

More Decks by Oleg Nenashev

Other Decks in Programming

Transcript

  1. Managing Jenkins with Groovy Hooks And JCasC plugin Oleg Nenashev

    CloudBees, Inc. Gothenburg Jenkins Meetup May 28, 2018
  2. 3 @oleg_nenashev, #codeconf > whoami -jenkins Hudson/Jenkins user since 2008

    Jenkins contributor since 2012 Now: • Maintainer: Core, Remoting, plugins • Security Team member • Meetup organizer • Google Summer of Code org admin @oleg_nenashev oleg-nenashev
  3. 5 @oleg_nenashev, #codeconf • Scripting languages are fine • YAML/JSON/XML

    are fine as well • You can use them together This talk is NOT about holywars
  4. 6 @oleg_nenashev, #codeconf üGROOVY HOOKS üGROOVY HOOKS & CASC PLUGIN

    üTIPS & TRICKS üSOME LIVE DEMOS Outline Slides:
  5. 9 @oleg_nenashev, #codeconf “Infrastructure as Code” in Jenkins API Clients

    Jenkins CLI and REST API python- jenkins jenkins-client (java) Configuration Management Ansible, Chef, … Docker, Docker Compose ... Jenkins- internal solutions Groovy Boot Hooks Scriptler Plugin Configuration -as-Code Plugin
  6. 10 @oleg_nenashev, #codeconf State in 2016 – Stockholm JAM •

    No default solution recommended by the Jenkins Project • No active work on this front • System Config DSL, was abandoned • Configuration Management – outside Jenkins Project • Time lag in Features • Security fun • Compatibility (e.g. Jenkins 2)
  7. 12 @oleg_nenashev, #codeconf Limitations (temporary?). Features • Not all plugins

    are supported • No Folders support – cannot set project structure • No debugging support • Export from UI is not working well
  8. 13 @oleg_nenashev, #codeconf Limitations (temporary?). Architecture • Forward/backward compatibility is

    not guaranteed • YAML - No dynamic logic / scripting • Startup Chicken&Egg for plugins • Race conditions for jobs • Handling of removed sections
  9. 14 @oleg_nenashev, #codeconf Limitations. Forward/backward compatibility • Config many break…

    • … even if a core/plugin change in a compatible way • Example: Work directory support in 2.73.x • New fields were added to the configuration in the core • Class @DataBoundConstructor interface changed, but old one is in place • workDir settings must be always set in CasC now • Ticket: JCasC Issue #167
  10. 15 @oleg_nenashev, #codeconf Limitations. Jobs loading JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED CasC

    Initializer PLUGINS_STARTED . . . • CasC happens in parallel with Jobs loading! • CasC may happen in parallel with initializers • Race conditions if job loading depends on plugin settings • Extra milestones are needed in the core
  11. 19 @oleg_nenashev, #codeconf Groovy Hooks [1/2] • Standard Groovy engine

    • Triggered by Jenkins • Direct access to the runtime • Extensible hook engine • init, boot-failure – in the core https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script
  12. 20 @oleg_nenashev, #codeconf Groovy Hooks [2/2] • Script locations: •WEB-INF/HOOK.groovy

    in jenkins.war •WEB-INF/HOOK.groovy.d/*.groovy in jenkins.war •$JENKINS_HOME/HOOK.groovy •$JENKINS_HOME/HOOK.groovy.d/*.groovy • Execution in alphabetical order https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script
  13. 24 @oleg_nenashev, #codeconf Demo. Local Pipeline Development Env Intellij IDEA

    Filesystem SCM Plugin • Documentation, Syntax • Static analysis • Debug (only for hooks) Local Jenkins instance Source Code (local .git repos) • Configuration-as-Code • Same as production • Repos with libs • Jenkinsfile JENKINS_HOME (volume)
  14. 27 @oleg_nenashev, #codeconf Groovy Hook Limitations. General • Jenkins API

    is scattered and complex • Javadoc is not user-friendly • Public API is not enough sometimes • No export from UI Solution: • Configuration as Code Plugin
  15. 28 @oleg_nenashev, #codeconf Groovy Hook Limitations. Missing Features • No

    class support • No shared libraries support • Error propagation: Jenkins does not stop on script error • Restart is required to apply changes Solution: • See below
  16. 30 @oleg_nenashev, #codeconf CasC and Groovy hooks JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED

    Groovy Hooks: init() CasC PLUGINS_STARTED . . . • Groovy hooks run after CasC • Hooks can fine-tune the configuration: • Missing integrations • Dynamic scripting • Job configurations and migrations
  17. Groovy Hooks – Tips & Tricks • Class support &

    Error handling • Development and debugging • Static configurations
  18. 33 @oleg_nenashev, #codeconf Class support & Error handling • They

    say: •Hard to share/reuse, no class support •No error propagation
  19. 37 @oleg_nenashev, #codeconf What can be done? Almost anything… •

    Class support (in demo) • Error propagation (in demo) • Library loading… ▸@Grab or other library management ▸Library bundling into WAR / Docker image • Partial CasC plugin invocation • Apply config on API call • …
  20. 39 @oleg_nenashev, #codeconf 1. Use your favorite IDE with Groovy

    support 2. Create Maven project 3. Use virtual Jenkins “plugin” with deps • Static analysis • Test automation Groovy Hook Development?
  21. 41 @oleg_nenashev, #codeconf 1. Start Jenkins with Debug flags 2.

    Just attach your IDE Groovy Hook Debugging? https://github.com/oleg-nenashev/demo- jenkins-config-as-code/blob/master/jenkins2.sh
  22. 43 @oleg_nenashev, #codeconf Static configuration in Jenkins • Base: https://github.com/jenkinsci/docker

    • plugins.txt Docker • Jenkins configuration • Setup of folders and jobs Groovy Init Hooks • Jenkinsfile in SCM • Organization Folder Pipeline
  23. 45 @oleg_nenashev, #codeconf Comparison JCasC Plugin Groovy Hooks YAML –

    standard declarative definition Low barrier to entry Limited integrations Compatibility issues External rollback Apply changes w/o restart No debugging, YAGNI? Future: Export from UI Groovy – general-purpose OOP language High barrier to entry Can manage everything Compatible if API is stable External rollback Restart is required Debugging support
  24. 46 @oleg_nenashev, #codeconf What is missing in BOTH engines? •

    Rollback support •Job/Build entries may be migrated with any update •Update is possible, but there may be breaking changes •Rollback – only via backups
  25. 47 @oleg_nenashev, #codeconf What is missing in BOTH engines? •

    Rollback support •Job/Build entries may be migrated with any update •Update is possible, but there may be breaking changes •Rollback – only via backups Solution: • External config management logic (change config, test initialization, rollback) • Backup plugin with CLI / REST API for Restore (e.g. Periodic Backup Plugin)
  26. 48 @oleg_nenashev, #codeconf What is missing in BOTH engines? •

    Rollback support • Plugin installation Chicken & Egg •New plugin installation is possible •Other cases: Reload may be needed ▸upgrade, downgrade, uninstall ▸CasC plugin self-upgrade? JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() CasC PLUGINS_STARTED . . .
  27. 49 @oleg_nenashev, #codeconf What is missing in BOTH engines? •

    Rollback support • Plugin installation Chicken & Egg •New plugin installation is possible •Other cases: Reload may be needed JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() CasC PLUGINS_STARTED . . . Solutions: A. CasC – work in progress (by Nicolas de Loof) B. External pre-startup tools • plugins.txt in stock Docker images • Custom WAR Packager (WiP by Oleg)
  28. 50 @oleg_nenashev, #codeconf What is missing in BOTH engines? •

    Rollback support • Plugin installation Chicken & Egg • UX: Lack of support in the Jenkins core •Admin UI is must-have for diagnostics •But: ▸No read-only admin UI ▸No read-only job configurations for admins ▸No read-only support in API
  29. 51 @oleg_nenashev, #codeconf Read-only UI. Partial solution • “sudo” mode

    in Matrix Auth or Role Strategy plugin •Task for Role Strategy plugin (JENKINS-51567) • Admin permissions are granted only when required • Job with System Groovy script to grant/revoke permissions • OR: REST API for Admin
  30. 53 @oleg_nenashev, #codeconf Takeaways • Two engines, no silver bullet

    JCasC Plugin Groovy Hooks YAML – standard declarative definition Low barrier to entry Limited integrations Compatibility issues External rollback Apply changes w/o restart No debugging, YAGNI? Future: Export from UI Groovy – general-purpose OOP language High barrier to entry Can manage everything Compatible if API is stable External rollback Restart is required Debugging support
  31. 54 @oleg_nenashev, #codeconf Takeaways • Select an engine depending on

    your needs and expertise • New to Jenkins? Use JCasC
  32. 55 @oleg_nenashev, #codeconf Takeaways • Select an engine depending on

    your needs and expertise • New to Jenkins? Use JCasC • Combine engines if needed
  33. 56 @oleg_nenashev, #codeconf Takeaways • It’s a great time to

    contribute ;) •https://jenkins.io/participate
  34. 57 @oleg_nenashev, #codeconf Links Groovy hooks: • https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script • These

    slides: https://speakerdeck.com/onenashev/gbg-jam-managing- jenkins-with-groovy-hooks-and-jcasc Demo and examples: • https://hub.docker.com/r/onenashev/demo-jenkins-config-as-code/ • https://github.com/oleg-nenashev/demo-jenkins-config-as-code • https://github.com/Praqma/JenkinsAsCodeReference • https://github.com/librecores/librecores-ci Configuration-as-Code Plugin • https://github.com/jenkinsci/configuration-as-code-plugin