Jenkins

1. Jenkins
2. Eloy Coto Software craftsman Ex Quobis, IPGlobal, Foehn Cilium & Kamailio contributor @eloycoto eloycoto@acalustra.com
3. Why do I need a CI system?
5. What is Jenkins?
6. Jenkins has a XX century interface! And it is true!
8. Jenkins Blue Ocean - Like all the Jenkins features, it is a plugin. - Can be used for job view, the admin part is not ready. - A lot of plugins do not have integration yet - Another themes around: - github.com/afonsof/jenkins-material-theme - github.com/djonsson/jenkins-atlassian-theme $ docker run -p 8080:8080 jenkins/blueocean
9. Jenkins pipelines
10. Jenkins declarative pipelines
11. Usual problems with Jenkins scripts - Jenkins has a lot of disrepute - On early days all were scripts, too many bash&sh scripts - Groovy pipelines are nice, but the learning curve is hard. - Declarative pipelines - Easy to understand. - Still has the power of using Groovy scripts - You should use declarative pipelines
12. Declarative Pipelines pipeline { agent any options { timeout(time: 300, unit: 'MINUTES') } stages { stage("First"){ steps { sh "echo 'Hello World!'" } } } } Jenkinsfile
13. Stages pipeline { agent any stages { stage("First"){ steps { sh "echo 'Hello World!'"} } stage("Second"){ steps { sh "echo 'Hello World!'"} } stage("Third"){ steps { sh "echo 'Hello World!'"} } } } Jenkinsfile
14. Agent pipeline { agent { label "fedora" } stages { stage("OsX"){ agent { label "osX" } steps { sh "echo 'Hello World!'" } } } } Jenkinsfile
15. Options pipeline { options { timeout(time: 500, unit: 'MINUTES') timestamps() ansiColor('xterm') } stages { stage("OsX"){ options { timeout(time: 10, unit: 'MINUTES') } steps { sh "echo 'Hello World!'" } } } } Jenkinsfile
16. Timeouts Job global timeout (20min) Stage 1 (10m) Stage 2(10m) Stage 3 (20m) Waiting in the queue Job global timeout (2 hours) Stage 1 (10m) Stage 2(10m) Stage 3(20m) Waiting in the queue On autoscaling slaves or busy systems, use a large global timeout and adjust stages timeouts. Global Timeout
17. Retry pipeline{ stages{ stage("deploy"){ steps{ retry(3){ sh './apply.sh' } sh './post.sh' } } } } Jenkinsfile
18. Environment variables pipeline { environment { CFG="TEST" DISTRIBUTION="fedora" } stages { stage("OsX"){ environment { DISTRIBUTION="osx" } steps { sh 'echo "${DISTRIBUTION}"' } } } } Jenkinsfile
19. Job parameters pipeline { parameters { string( defaultValue: 'master', name: 'FRONTEND_BRANCH') choice( choices: 'eu-central-1\neu-west-1', description: 'What AWS region?', name: 'region') } stages { stage("Unit Testing"){ steps {sh "echo '${FRONTEND_BRANCH}'"} } } } Jenkinsfile
20. Post actions pipeline { stages { stage("Unit Testing"){ steps {sh "go test"} post { always { junit '*.xml' } } } } post { always { cleanWs(); } } } Jenkinsfile
21. Post actions pipeline { stages { ...} post { always { cleanWs(); } success { echo "It works!" } failure { echo "It fails!" } changed { echo "I'm different" } } } Jenkinsfile
22. Parallel pipeline{ stages{ stage("Browser testing"){ parallel( "Firefox": { sh "echo testing FFX" sh "echo more steps" }, "Chrome": { sh "echo testing Chrome" sh "echo more steps" } ) } } } Jenkinsfile
23. When pipeline { agent any stages { stage('Deploy') { when { allOf { branch 'master'; environment name: 'DEPLOY_TO', value: 'production' } } steps { sh 'docker push ..' } } } } Jenkinsfile
24. Human interaction pipeline { agent any stages { stage('Deploy') { steps { input ( id : "Deploy", message: "Do you want to deploy the application?" ) } } } } Jenkinsfile
25. Human interaction
26. Credentials
27. Credentials types - Username with password - Secret text - Secret file - SSH Username with private key - Certificate
28. Credentials pipeline { stages { stage("deploy"){ steps { withCredentials([string( credentialsId: 'mytoken', variable: 'TOKEN')]) { sh 'echo "Token->${TOKEN}"' } } } } } Jenkinsfile
29. Credentials pipeline { stages { stage("deploy"){ environment { TOKEN=credentials('MY_TOKEN') } steps { sh 'echo ${TOKEN}' } } } } Jenkinsfile
30. Credentials pipeline { stages { stage("deploy"){ environment { AWS=credentials('AWS') } steps { sh 'echo "${AWS_USR}-${AWS_PWD}"' } } } } Jenkinsfile
31. Jenkins shared libraries
32. Jenkins Shared libraries ./vars/wrappedNode.groovy ./vars/withVpn.groovy ./vars/withTool.groovy ./vars/withChownWorkspace.groovy ./vars/s3Fetch.groovy ./vars/s3Archive.groovy ./vars/nodeExists.groovy ./vars/golangTester.groovy ./vars/gitCommit.groovy ./vars/getOutput.groovy ./vars/documentationChecker.groovy ./vars/dockerImageTagToDigest.groovy ./vars/codecov.groovy ./src/com/docker/utilities/AWSSync.groovy ./src/com/docker/projects/EngineApi.groovy ./src/com/docker/projects/Docker.groovy github.com/docker/jenkins-pipeline-scripts
33. Jenkins Shared Libraries String call() { if (env.GIT_COMMIT == null) { env.GIT_COMMIT = sh( script: "git rev-parse HEAD", returnStdout: true).trim() } env.GIT_COMMIT } vars/gitCommit.groovy
34. Jenkins Shared libraries @Library('utils') _ pipeline { stages { stage("Checkout"){ steps { gitCommit() } } } } Jenkinsfile
35. Job DSL
38. Job DSL example pipelineJob('DSL_Pipeline') { def repo = 'https://github.com/eloycoto/jenkins_test.git' description("Pipeline for $repo") triggers { scm('H/5 * * * *') } definition { cpsScm { scm { git { remote { url(repo) } branches('master', '**/feature*') scriptPath('Jenkinsfile') extensions { } } } } } } Jenkinsfile
39. Job DSL loops def jobsSpecs = [ "eloycoto/jenkins_test.git": "jenkinsfile" ] jobsSpecs.each{ JobRepo, JobScriptName -> pipelineJob("${JobRepo}") { description("Pipeline for ${JobRepo}") ... ... scriptPath("${JobScriptName}") } } Jenkinsfile https://job-dsl.herokuapp.com/
40. Job DSL configuration
41. Scaling Jenkins
42. Add Slave http://localhost:8080/computer/createItem
43. Autoscaling slaves - Amazon EC2 plugin - Terraform Jenkins slaves - github.com/dragoonis/terraform-jenkins - Kubernetes Plugin - github.com/jenkinsci/kubernetes-plugin
44. Controlling Slaves curl -s -u "${USER}:${TOKEN}" \ "https://localhost:8080/computer/api/json" | \ jq . Terminal
45. Must Plugins
46. Test Result Analyzer
47. JUnit Attachments post { always{ junit testDataPublishers: [ [$class: 'AttachmentPublisher']], testResults: 'test/*.xml' } } Jenkinsfile
48. Tips
49. Validating Jenkinsfiles $ export JENKINS=http://localhost:8080 $ curl -X POST \ -F "jenkinsfile=<jenkinsfile" \ $JENKINS/pipeline-model-converter/validate Terminal
50. Clean Workspaces pipeline { ... ... post { always { cleanWs() } } } Jenkinsfile
51. API calls $ export URL='http://localhost:8080' # reference $ curl "$URL/api/json" # Queue status $ curl "$URL/queue/api/json?pretty=true" # Run groovy scripts ❤ $ CRUMB=$(curl -s "${URL}/crumbIssuer/api/json" \ | jq -r '.crumbRequestField +": "+ .crumb') $ JEscript=$(cat script.groovy) $ curl -s -X POST -H "${CRUMB}" "${URL}/scriptText" \ -d "script=${JEscript}" Terminal
52. Tricks for success - Implement CI workflow, it pays off! - Skip any bash script, use declarative pipelines. - Keep Jenkinsfiles simple. Logic should be avoid. - Export data to Junit, it makes things easier. - Clean the workspace at the end.
53. Graciñas! Email: eloycoto@acalustra.com Blog: http://acalustra.com Twitter: @eloycoto