Tweet
Tweet

Slide 1

Slide 1 text

JENKINS AUTOMATION TONI VAN DE VOORDE

Slide 2

Slide 2 text

ABOUT ME Toni Van de Voorde tonivdv [email protected] @tonivdv ▸ CTO @ Adsdaq ▸ +10 years experience building web applications ▸ Love clean stuff and automations ▸ “Relax” with family

Slide 3

Slide 3 text

BACKGROUND ▸ Written in Java ▸ First released in 2005 ▸ Formerly known as the “Hudson Project” ▸ The name “Jenkins” was born in 2011

Slide 4

Slide 4 text

WHY ▸ Free ▸ OSS ▸ Customisation ▸ Plugin System ▸ Full Control

Slide 5

Slide 5 text

BUT “ JENKINS & JOBS CAN BE TIME- CONSUMING”

Slide 6

Slide 6 text

USE CASES

Slide 7

Slide 7 text

USE CASES ▸ Traditional: Freestyle Job


Slide 8

Slide 8 text

USE CASES ▸ Traditional: Freestyle Job
 ▸ Jenkins Pipeline


Slide 9

Slide 9 text

USE CASES ▸ Traditional: Freestyle Job
 ▸ Jenkins Pipeline
 ▸ Intro to JCasC


Slide 10

Slide 10 text

USE CASES ▸ Traditional: Freestyle Job
 ▸ Jenkins Pipeline
 ▸ Intro to JCasC
 ▸ Extra: Demo

Slide 11

Slide 11 text

USE CASES Freestyle Job

Slide 12

Slide 12 text

FREESTYLE JOB

Slide 13

Slide 13 text

FREESTYLE JOB Job DSL Job DSL plugin

Slide 14

Slide 14 text

Groovy-based Domain Specific Language FREESTYLE JOB Job DSL

Slide 15

Slide 15 text

FREESTYLE JOB Job DSL job(‘jobs-name’) {
 
 scm {
 git('[email protected]:adlogix/zf-symfony-container.git')
 } triggers {
 scm('H/15 * * * *')
 } steps {
 shell('composer install && vendor/bin/phpunit')
 }
 
 }

Slide 16

Slide 16 text

FREESTYLE JOB Job DSL job(‘jobs-name’) {
 
 scm {
 git('[email protected]:adlogix/zf-symfony-container.git')
 } triggers {
 scm('H/15 * * * *')
 } steps {
 shell('composer install && vendor/bin/phpunit')
 }
 
 }

Slide 17

Slide 17 text

FREESTYLE JOB Job DSL Installation Dockerfile FROM jenkins/jenkins:lts-alpine
 RUN /usr/local/bin/install-plugins.sh job-dsl

Slide 18

Slide 18 text

FREESTYLE JOB Job DSL Seed Job

Slide 19

Slide 19 text

FREESTYLE JOB Job DSL Seed Job

Slide 20

Slide 20 text

FREESTYLE JOB Job DSL Seed Job

Slide 21

Slide 21 text

FREESTYLE JOB Job DSL Seed Job

Slide 22

Slide 22 text

FREESTYLE JOB Job DSL Seed Job

Slide 23

Slide 23 text

FREESTYLE JOB Job DSL Oops

Slide 24

Slide 24 text

FREESTYLE JOB Job DSL Security 1. Script Approval
 2. Sandboxing
 3. Disable

Slide 25

Slide 25 text

FREESTYLE JOB Job DSL Example: Folders folder(‘components’)
 
 job(‘components/component1’) {
 ...
 } Plugin ID: cloudbees-folder

Slide 26

Slide 26 text

FREESTYLE JOB Job DSL def gitUrl = 'git://github.com/jenkinsci/job-dsl-plugin.git’ def branches = ['master','dev','hotfix'] branches.each { branch -> 
 job(“demo-${branch}") { 
 scm { 
 git(gitUrl, branch) 
 } 
 }
 } Example: Static Branches

Slide 27

Slide 27 text

FREESTYLE JOB Job DSL def project = 'Netflix/asgard’
 def branchApi = new URL("https://api.github.com/repos/${project}/branches")
 def branches = new groovy.json.JsonSlurper().parse(branchApi.newReader()) branches.each { def branchName = it.name 
 def jobName = "${project}-${branchName}".replaceAll('/','-') job(jobName) { 
 scm { 
 git("https://github.com/${project}.git", branchName) 
 } 
 }
 } Example: Dynamic Branches

Slide 28

Slide 28 text

FREESTYLE JOB Job DSL def project = 'automate-jenkins-job-creation’
 def repo = "https://[email protected]/tonivdv/${project}.git" Process proc1 = "git ls-remote -h ${repo}".execute()
 Process proc2 = 'sed s/.*refs\\/heads\\///g'.execute()
 Process all = proc1 | proc2
 all.waitFor() def branches = "${all.in.text}".split('\\n') branches.each {
 …
 } Example: Dynamic Branches

Slide 29

Slide 29 text

FREESTYLE JOB Job DSL # ./scripts/utilities/GitUtils.groovy
 
 package utilities
 
 public class GitUtils { 
 public static String[] getRemoteBranches(def repoUrl, def out) { 
 
 Process proc1 = "git ls-remote -h ${repoUrl}".execute() 
 Process proc2 = 'sed s/.*refs\\/heads\\///g'.execute() 
 Process all = proc1 | proc2 all.waitFor() 
 
 def branches = "${all.in.text}".split('\\n') 
 return branches }
 } Example: Functions

Slide 30

Slide 30 text

FREESTYLE JOB Job DSL # ./scripts/demo.groovy import utilities.GitUtils
 
 def project = 'automate-jenkins-job-creation’
 def repo = https://[email protected]/tonivdv/${project}.git
 
 def branches = GitUtils.getRemoteBranches(repo, out) Example: Functions

Slide 31

Slide 31 text

FREESTYLE JOB Job DSL ‣ Don’t store passwords in your DSL scripts! DON’T ! 
 ‣ Use the Credentials 
 Plugin Example: Credentials Plugin ID: credentials

Slide 32

Slide 32 text

FREESTYLE JOB Job DSL job(‘jobs-name') {
 scm {
 git {
 remote {
 github('account/repo', 'ssh')
 credentials('github-ci-key')
 }
 }
 }
 } Example: Credentials

Slide 33

Slide 33 text

FREESTYLE JOB Job DSL // assign the my-userpassword credentials to the PASSWORD build variable
 
 job(‘jobs-name') {
 wrappers {
 credentialsBinding { 
 usernamePassword('PASSWORD', ‘my-userpassword’)
 }
 }
 } Example: Credentials

Slide 34

Slide 34 text

FREESTYLE JOB Job DSL Api Viewer https://jenkinsci.github.io/job-dsl-plugin/

Slide 35

Slide 35 text

Help !!! 
 My Plugin is not Supported FREESTYLE JOB Job DSL

Slide 36

Slide 36 text

FREESTYLE JOB Job DSL 1. Dynamic DSL
 2. Configure Block

Slide 37

Slide 37 text

FREESTYLE JOB Job DSL ‣ Powerful & Easy ‣ Looks like a generic DSL ‣ Available in API Viewer ‣ But plugin must use following annotations: ‣ @DataBoundConstructor ‣ @DataBoundSetter
 Dynamic DSL

Slide 38

Slide 38 text

FREESTYLE JOB Job DSL Dynamic DSL https://{base_url}/plugin/job-dsl/api-viewer/index.html

Slide 39

Slide 39 text

FREESTYLE JOB Job DSL Dynamic DSL job(‘jobs-name’) {
 
 scm {
 git(‘[email protected]:...’)
 bitbucketWeb {
 repoUrl(‘https://...’)
 }
 }
 
 }

Slide 40

Slide 40 text

FREESTYLE JOB Job DSL ‣ Gives access to underlying XML configuration ‣ Harder & Ugly ‣ Not fun ‣ BUT it works Configure Block

Slide 41

Slide 41 text

FREESTYLE JOB Job DSL Configure Block job('example-1') {
 
 configure { node -> // node represents }
 
 }

Slide 42

Slide 42 text

FREESTYLE JOB Job DSL Configure Block https://{base_url}/job/demo2/config.xml

Slide 43

Slide 43 text

USE CASES Jenkins Pipeline

Slide 44

Slide 44 text

PIPELINE Jenkins Pipeline (or simply "Pipeline" with a capital "P") is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins. © https://jenkins.io/doc/book/pipeline/

Slide 45

Slide 45 text

Why? PIPELINE ‣ “as-code” paradigm (DSL) ‣ Durable ‣ Pausable ‣ Parallel executions ‣ …

Slide 46

Slide 46 text

PIPELINE © https://jenkins.io/doc/book/pipeline/

Slide 47

Slide 47 text

How? PIPELINE Jenkinsfile

Slide 48

Slide 48 text

Example PIPELINE Jenkinsfile pipeline {
 agent any 
 stages {
 stage('Checkout') { 
 steps {
 Checkout scm
 }
 }
 stage('Build') { 
 steps {
 sh 'make it-build’
 }
 }
 stage('Test') { 
 steps {
 sh 'make test’
 }
 }
 
 }
 }

Slide 49

Slide 49 text

2 Flavours … PIPELINE Jenkinsfile ‣ Declarative Pipeline ‣ Easier to use ‣ Limited ‣ Targets easy CD ‣ Scripted Pipeline ‣ Power Users ‣ Groovy is the limit

Slide 50

Slide 50 text

Declarative Pipeline PIPELINE Jenkinsfile pipeline {
 agent any 
 stages {
 ...
 stage('Deploy') { 
 when {
 branch 'master'
 }
 steps {
 make deploy
 }
 }
 }
 }

Slide 51

Slide 51 text

PIPELINE Jenkinsfile node { 
 stage('Deploy') { 
 if (env.BRANCH_NAME == 'master') { 
 make deploy 
 }
 }
 } Scripted Pipeline

Slide 52

Slide 52 text

PIPELINE Jenkinsfile pipeline {
 agent none
 stages {
 stage('php 7.3') {
 agent {
 docker { image 'php:7.3-alpine' }
 }
 steps {
 sh 'php --version'
 }
 } stage('php 7.2') {
 agent {
 docker { image 'php:7.2-alpine' }
 }
 steps {
 sh 'php --version'
 }
 }
 }
 } Using Docker Blue Ocean Plugin

Slide 53

Slide 53 text

PIPELINE Jenkinsfile pipeline {
 agent none
 stages {
 stage(‘php') {
 parallel {
 stage('php 7.3') {
 agent {
 docker { image 'php:7.3-alpine' }
 }
 steps {
 sh 'php --version'
 }
 } stage('php 7.2') {
 agent {
 docker { image 'php:7.2-alpine' }
 }
 steps {
 sh 'php --version'
 }
 }
 }
 }
 }
 } Parallel Blue Ocean Plugin

Slide 54

Slide 54 text

PIPELINE Jenkinsfile Discovering ‣ Pipeline Job ‣ Multi-Branch Pipeline ‣ Organisation Folder ‣ Github ‣ Bitbucket

Slide 55

Slide 55 text

PIPELINE Jenkinsfile Discovering

Slide 56

Slide 56 text

PIPELINE Jenkinsfile Multi-Branch Pipeline

Slide 57

Slide 57 text

PIPELINE Jenkinsfile Bitbucket Organisation

Slide 58

Slide 58 text

PIPELINE Job DSL plugin

Slide 59

Slide 59 text

PIPELINE multibranchPipelineJob('mypipelinejob') {
 branchSources {
 branchSource {
 source {
 bitbucket {
 id('bitbucket')
 serverUrl('https://bitbucket.org')
 repoOwner(‘my-user')
 repository('my-repository')
 credentialsId(‘bitbucket-ssh’)
 }
 }
 }
 }
 
 ...
 }
 
 Job DSL Multi-Branch Pipeline

Slide 60

Slide 60 text

PIPELINE multibranchPipelineJob('mypipelinejob') {
 ...
 
 // discover Branches (workaround due to JENKINS-46202)
 configure { def traits = it / sources / data / 'jenkins.branch.BranchSource' / source / traits // Exclude branches that are also filed as PRs 
 traits << 'com.cloudbees.jenkins.plugins.bitbucket.BranchDiscoveryTrait' {
 strategyId(1)
 }
 
 // Discover each pull request once ... without merging
 traits << 'com.cloudbees.jenkins.plugins.bitbucket.OriginPullRequestDiscoveryTrait' {
 strategyId(2)
 }
 }
 }
 
 Job DSL Multi-Branch Pipeline

Slide 61

Slide 61 text

PIPELINE organizationFolder('example') {
 
 description(‘Some description')
 displayName(‘Other name')
 organizations {
 bitbucket { 
 serverUrl('https://bitbucket.org') 
 repoOwner('adqphpbenl19')
 credentialsId('bitbucket-tonivdv') 
 }
 }
 
 ...
 } Job DSL Organisation Folder

Slide 62

Slide 62 text

PIPELINE organizationFolder('example') {
 
 ...
 
 configure {
 def traits = it / navigators / 'com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator' / traits
 traits << 'com.cloudbees.jenkins.plugins.bitbucket.BranchDiscoveryTrait' {
 strategyId(1)
 }
 traits << 'com.cloudbees.jenkins.plugins.bitbucket.OriginPullRequestDiscoveryTrait' {
 strategyId(2)
 }
 }
 
 } Job DSL Organisation Folder

Slide 63

Slide 63 text

USE CASES Intro to JCasC

Slide 64

Slide 64 text

JCASC JCasC or Jenkins Configuration as Code allows to define the “configuration” of plugins and parameters as a simple, human friendly, plain text Yaml syntax.
 
 © https://jenkins.io/projects/jcasc/

Slide 65

Slide 65 text

JCASC Installation

Slide 66

Slide 66 text

JCASC Installation ‣ Define location of the casc configurations ‣ Path to a folder containing a set of configs
 e.g. /var/jenkins_home/casc_configs ‣ Full path to single file
 e.g. /var/jenkins_home/casc_configs/jenkins.yaml ‣ A URL pointing to a file served on the web
 e.g. https://acme.org/jenkins.yaml ‣ Default: $JENKINS_ROOT/jenkins.yaml

Slide 67

Slide 67 text

JCASC # casc_configs/core.yaml
 
 jenkins: systemMessage: | Hello JCasC,
 
 This is exciting! numExecutors: 1 Examples

Slide 68

Slide 68 text

JCASC # casc_configs/realm.yaml
 
 jenkins:
 securityRealm:
 bitbucket:
 clientID: ${b_id}
 clientSecret: ${b_secret} Examples # casc_configs/authorization.yaml
 
 jenkins:
 authorizationStrategy:
 globalMatrix:
 grantedPermissions:
 - "Overall/Read:anonymous"
 - "Overall/Administer:authenticated"

Slide 69

Slide 69 text

JCASC Api Viewer

Slide 70

Slide 70 text

JCASC Api Viewer

Slide 71

Slide 71 text

JCASC Docker

Slide 72

Slide 72 text

DEMO

Slide 73

Slide 73 text

QUESTIONS

Slide 74

Slide 74 text

THANK YOU joind.in: 
 https://joind.in/talk/446a8

Slide 75

Slide 75 text

REFERENCES ‣ Doc ‣ https://jenkins.io/ ‣ https://github.com/jenkinsci/job-dsl-plugin/wiki ‣ https://jenkins.io/doc/book/pipeline/ ‣ https://www.phparch.com/article/jenkins-automation/ ‣ Code Examples ‣ https://github.com/tonivdv/rnd-jenkins-as-code ‣ https://github.com/tonivdv/automate-jenkins-job-creation ‣ Various ‣ https://jenkins.io/artwork/ ‣ https://www.cloudbees.com/