Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Andreas Caternberg on Jenkins Pipelines

Andreas Caternberg on Jenkins Pipelines

Transcript

  1. Let’s Build a Jenkins Pipeline Andreas Caternberg

  2. Download Slides Now https://gist.github.com/cccaternberg/3984e4a3d1a84ec6faffbd728ce6c247

  3. About Your Guide Andreas Caternberg Senior Consultant at CloudBees @acaternberg

  4. Plan for today

  5. CloudBees Jenkins Enterprise © 2017 CloudBees, Inc. All Rights Reserved

    • Community Innovation • Jenkins for the Enterprise
  6. What we are going to cover • What is a

    pipeline • Compare scripted pipeline to declarative pipeline • Create a declarative pipeline • Learn about Blue Ocean • Share pipeline resources • Extra scripted pipeline information © 2017 CloudBees, Inc. All Rights Reserved
  7. What is a Jenkins pipeline • Formerly known as Jenkins

    Workflow, Jenkins Pipeline was introduced in 2014 • New job type – a single Groovy script using an embedded DSL - no need to jump between multiple job configuration screens to see what is going on • Durable - keeps running even if Jenkins master is not • Distributable – a Pipeline job may be run across an unlimited number of nodes, including in parallel • Pausable – allows waiting for input from users before continuing to the next step • Visualized - Pipeline Stage View provides status at-a-glance dashboards to include trending © 2017 CloudBees, Inc. All Rights Reserved
  8. What is a declarative pipeline? • A syntax for Jenkins

    Pipelines designed to… –Have smarter default behavior for most use cases –Make Pipelines more structured –Provide better error reporting and handling © 2017 CloudBees, Inc. All Rights Reserved
  9. Have a smarter behavior • Default SCM checkout • Built

    with Docker and containers in mind from day one • No more need for try/catch for notifications, etc. • More intuitive ways to specify job properties, triggers and parameters • Streamlined syntax for conditional execution of stages © 2017 CloudBees, Inc. All Rights Reserved
  10. More structure in Pipelines • Separate configuration from actual steps

    to be executed • Everything happens in a stage • Easier to look at any Declarative Pipeline and understand what it’s doing - more consistency! • Enables round-tripping with the visual editor (more on this later!) • Built with and for Blue Ocean visualization © 2017 CloudBees, Inc. All Rights Reserved
  11. Better error reporting • Syntax and validation errors reported at

    the very beginning of the build • No more waiting until a build has gone an hour to discover that you misspelled “timeout”! • Errors report as compilation errors, pointing to the specific code or text that’s causing problems • Eliminates many potentially confusing stack traces (though not all!) • Validates things like syntax, required configuration, step parameter types, and more © 2017 CloudBees, Inc. All Rights Reserved
  12. What happens to Scripted Pipeline? • In short? Nothing. •

    We’re now calling “traditional” Pipeline “Scripted Pipeline” to distinguish it from Declarative Pipeline • All one execution engine behind the scenes ◦Declarative is built on top of Scripted Pipeline, not replacing it ◦Blue Ocean and Stage View don’t distinguish between a Scripted Pipeline or a Declarative Pipeline • Scripted Pipeline still used *inside* Declarative Pipeline in steps blocks, post blocks ◦You can copy the contents of a Declarative steps block into a Scripted Pipeline (inside a node block, of course!) and it’ll Just Work. © 2017 CloudBees, Inc. All Rights Reserved
  13. Benefits of Declarative for new users • Lower barrier of

    entry than Scripted Pipelines • Human readable • Does not require Groovy-specific expertise • Create and edit Declarative Pipelines via the upcoming UI Editor © 2017 CloudBees, Inc. All Rights Reserved
  14. Benefits of Declarative for existing users • Uses same engine

    as Scripted, so existing investments still pay off • Expand usage and usability ◦Less burden on “experts” ◦Easier empowerment of other users • Easier collaboration and code review • Lintable! • Separation of Jenkins infrastructure-related configuration from steps to execute © 2017 CloudBees, Inc. All Rights Reserved
  15. Syntax Overview © 2017 CloudBees, Inc. All Rights Reserved

  16. Pipeline Comparison © 2017 CloudBees, Inc. All Rights Reserved Scripted

    Pipeline node { try { stage('Build') { checkout scm docker.image('ubuntu').inside { sh 'mvn clean install' } } } catch(exc) { mail to:'me@ex.com', subject:'FAILURE:' } finally { deleteDir() } } Declarative Pipeline pipeline { agent { docker 'ubuntu' } stages { stage('Build') { steps { sh 'mvn clean install' } } } post { always { deleteDir() } failure { mail to:'me@ex.com', subject:'FAILURE:' } } }
  17. First example © 2017 CloudBees, Inc. All Rights Reserved https://gist.github.com/cccaternberg/1d2169d445d04d5d6264a241fbc53e8e

  18. © 2017 CloudBees, Inc. All Rights Reserved

  19. © 2017 CloudBees, Inc. All Rights Reserved

  20. © 2017 CloudBees, Inc. All Rights Reserved

  21. © 2017 CloudBees, Inc. All Rights Reserved

  22. © 2017 CloudBees, Inc. All Rights Reserved

  23. © 2017 CloudBees, Inc. All Rights Reserved

  24. Second Example © 2017 CloudBees, Inc. All Rights Reserved https://gist.github.com/cccaternberg/b45e58163725c3ca81f76a55e1328009

  25. © 2017 CloudBees, Inc. All Rights Reserved

  26. © 2017 CloudBees, Inc. All Rights Reserved

  27. © 2017 CloudBees, Inc. All Rights Reserved

  28. © 2017 CloudBees, Inc. All Rights Reserved

  29. Time to build a pipeline © 2017 CloudBees, Inc. All

    Rights Reserved
  30. Get Ready • Log into our CloudBees Jenkins Operations Center

    http://dpa.beedemo.net/pipeline- workshop/ • Create an account © 2017 CloudBees, Inc. All Rights Reserved
  31. Get Ready • Log in • Go to the LBAJP

    server © 2017 CloudBees, Inc. All Rights Reserved
  32. Create a Pipeline Job © 2017 CloudBees, Inc. All Rights

    Reserved • Select ”New Item” • Name your job
  33. A basic pipeline pipeline { agent any stages { stage('Build')

    { steps { echo 'Hello from Build' } } stage('Test') { steps { echo 'Testing Testing 123' } } stage('Deploy') { steps { echo 'Deploy some things' } } } } © 2017 CloudBees, Inc. All Rights Reserved
  34. Add an agent © 2017 CloudBees, Inc. All Rights Reserved

    pipeline { agent any stages { pipeline { agent { label ‘docker’ } stages {
  35. Stash Files © 2017 CloudBees, Inc. All Rights Reserved stages

    { stage('Build') { steps { echo 'Hello from Build’ writeFile(file: 'config', text: 'version=1', encoding: 'UTF-8') stash(name: 'pom-config', includes: 'config') } } stage('Test') { environment { configValue = readFile(encoding: 'UTF-8', file: 'config') } steps { echo 'Testing Testing 123’ unstash 'pom-config' sh 'echo "$configValue"' } }
  36. Create an input step stage('Deploy') { steps { echo 'Deploy

    some things' input 'Do you want to deploy?' echo 'deployed' } } © 2017 CloudBees, Inc. All Rights Reserved
  37. Blue Ocean © 2017 CloudBees, Inc. All Rights Reserved

  38. Blue Ocean Visualization • Some special smarts for optimized visualization

    • Blue Ocean keeps built-in stages for things like checkout, Docker image prep, post-build actions out of your way unless there’s a problem • Special marking of stages that have been skipped due to either an unsatisfied when condition or an earlier failed stage © 2017 CloudBees, Inc. All Rights Reserved
  39. Blue Ocean Editor © 2017 CloudBees, Inc. All Rights Reserved

  40. © 2017 CloudBees, Inc. All Rights Reserved

  41. © 2017 CloudBees, Inc. All Rights Reserved

  42. © 2017 CloudBees, Inc. All Rights Reserved

  43. Reference Card https://www.cloudbees.com/sites/default/files/declarative-pipeline-refcard.pdf © 2017 CloudBees, Inc. All Rights Reserved

  44. Resources on Declarative • Documentation https://jenkins.io/doc/ • Examples https://github.com/jenkinsci/pipeline-examples •

    Plugin source https://github.com/jenkinsci/pipeline-model-definition-plugin © 2017 CloudBees, Inc. All Rights Reserved
  45. Scripted Pipeline © 2017 CloudBees, Inc. All Rights Reserved

  46. Pipeline: a new job type -<YourName>

  47. Key Benefits ✓ Long-running ✓ Durable ✓ Scriptable ✓ One-place

    for Everything Pipeline: a new job type ✓ Makes building Pipelines Simple
  48. Domain Specific Language Simple, Groovy-based DSL (“Domain Specific Language”) to

    manage build step orchestration Can be defined as DSL in Jenkins or as Jenkinsfile in SCM Groovy is widely used in Jenkins ecosystem. You can leverage the large Jenkins Groovy experience If you don’t like/care about Groovy, just consider the DSL as a dedicated simple syntax
  49. Snippet Generator 49

  50. Create a Pipeline - core steps 50 stage - group

    your steps into its component parts and control concurrency node - schedules the steps to run by adding them to Jenkins' build queue and creates a workspace specific to this pipeline sh (or bat) - execute shell (*nix) or batch scripts (Windows) just like freestyle jobs, calling any tools in the agent
  51. Create a Pipeline stage('build') { echo 'hello from jenkins master'

    node { sh 'echo hello from jenkins agent' } } stage('test') { echo 'test some things' } stage('deploy') { echo 'deploy some things' } 51
  52. Lab Exercise: Checkout from SCM

  53. Files stash - store some files for later use unstash

    - retrieve previously stashed files (even across nodes) writeFile - write a file to the workspace readFile - read a file from the workspace 53
  54. Checkout from SCM and stash Files stage('build') { node {

    git 'https://github.com/beedemo/mobile-deposit-api.git' writeFile encoding: 'UTF-8', file: 'config', text: 'version=1' stash includes: 'pom.xml,config', name: 'pom-config' } } stage('test'){ node { unstash 'pom-config' configValue = readFile encoding: 'UTF-8', file: 'config' echo configValue } } 54
  55. try up to N times retry(5) { // some block

    } wait for a condition waitUntil { // some block } Flow Control wait for a set time sleep time: 1000, unit:'NANOSECONDS' timeout timeout(time: 30, unit: 'SECONDS'){ // some block }
  56. You can also rely on Groovy control flow syntax! while(something)

    { // do something if (something_else) { // do something else } } Flow Control 56 try{ //some things }catch(e){ // }
  57. Advanced Flow Control input - pause for manual or automated

    approval parallel - allows simultaneous execution of build steps on the current node or across nodes, thus increasing build speed parallel 'quality scan': { node {sh 'mvn sonar:sonar'} }, 'integration test': { node {sh 'mvn verify'} } checkpoint - capture the workspace state so it can be reused as a starting point for subsequent runs
  58. Lab Exercise: Input and Checkpoints

  59. Input Approval & Checkpoints 59 stage('deploy') { input message: 'Do

    you want to deploy?' node { echo 'deployed' } }
  60. Input Approval & Checkpoints 60 checkpoint 'testing-complete' stage('approve') { mail

    body: "Approval needed for '${env.JOB_NAME}' at '${env.BUILD_URL}'", subject: "${env.JOB_NAME} Approval", to: "ops@yourcompanyhere.com" timeout(time: 7, unit: 'DAYS') { input message: 'Do you want to deploy?', parameters: [string(defaultValue: '', description: 'Provide any comments regarding decision.', name: 'Comments')], submitter: 'ops' } }
  61. Lab Exercise: Tool Management

  62. tool Step •Binds a tool installation to a variable •The

    tool home directory is returned •Only tools already configured are available def mvnHome = tool 'M3' sh "${mvnHome}/bin/mvn -B verify" 62
  63. Use Docker Containers •The CloudBees Docker Pipeline plugin allows running

    steps inside a container •You can even build the container as part of the same Pipeline docker.image('maven:3.3.3-jdk-8').inside() { sh 'mvn -B verify' } 63
  64. Lab Exercise: Pipeline as Code

  65. Pipeline script in SCM 65

  66. Pipeline Multibranch 66

  67. Pipeline Shared Libraries

  68. What Next?

  69. More Advanced Steps Send email mail body: 'Uh oh.', subject:

    'Build Failed!', to: 'dev@cloudbees.com' step([$class: 'Mailer', notifyEveryUnstableBuild: true, recipients: 'info@cloudbees.com']) Deploy to Amazon Elastic Beanstalk wrap([$class: 'AmazonAwsCliBuildWrapper', credentialsId: 'aws-beanstalk- credentials', defaultRegion: 'us-east-1']) { sh 'aws elasticbeanstalk create-application-version' } Integrate with Jira jiraComment(issueKey: "EX-111", body: "Job '${env.JOB_NAME}' ($ {env.BUILD_NUMBER}) built. Please go to ${env.BUILD_URL}.")
  70. None
  71. Gartner: “Using Docker to Run Build Nodes is Ideal.” Agent

    Master z Agent Agent
  72. Accelerating CD with Containers Workflow CD Pipeline Triggers: ✓ New

    application code (feature, bug fix, etc.) ✓ Updated certified stack (security fix in Linux, etc.) ✓ Will lead to a new gold image being built and available for … TESTING … STAGING … PRODUCTION ✓ All taking place in a standardized/similar/consistent OS environment + Jenkins Pipeline TEST STAG E PRODUCTIO N App <code> (git, etc.) Gold Docker Image (~per app) <OS config> Certified Docker Images (Ubuntu, etc.) <OS config>
  73. Jenkins Pipeline Docker Commands • withRegistry Specifies which Docker Registry

    to use for pushing/pulling • withServer Specifies which Server to issue Docker commands against • Build Builds container from Dockerfile • Image.run Runs a container • Image.inside Runs container and allows you to execute command inside the container • Image.push Pushed image to Docker Registry with specified credentials
  74. Pipeline - Docker Example stage 'Build' node('docker-cloud') { checkout scm

    docker.image('java:jdk-8').inside('-v /data:/data') { sh "mvn clean package" }} stage 'Quality Analysis' node('docker-cloud') { unstash 'pom' parallel( JRE8Test: { docker.image('java:jre-8').inside('-v /data:/data') { sh 'mvn -Dmaven.repo.local=/data/mvn/repo verify' } },JRE7Test: { docker.image('java:jre-7').inside('-v /data:/data') { sh 'mvn -Dmaven.repo.local=/data/mvn/repo verify' } }, failFast: true )}
  75. Pipeline - Docker Example node('docker-cloud') { stage 'Build Docker Image'

    dir('target') { mobileDepositApiImage = docker.build "beedemo/mobile-deposit-api:$ {dockerTag}" } stage 'Publish Docker Image' withDockerRegistry(registry: [credentialsId: 'docker-hub-beedemo']) { mobileDepositApiImage.push() } }
  76. None