Jenkins

 Jenkins

66dc269635b1dbaaf2751ca5a5728525?s=128

Eloy Coto

June 29, 2018
Tweet

Transcript

  1. Jenkins

  2. Eloy Coto Software craftsman Ex Quobis, IPGlobal, Foehn Cilium &

    Kamailio contributor @eloycoto eloycoto@acalustra.com
  3. Why do I need a CI system?

  4. None
  5. What is Jenkins?

  6. Jenkins has a XX century interface! And it is true!

  7. None
  8. Jenkins Blue Ocean - Like all the Jenkins features, it

    is a plugin. - Can be used for job view, the admin part is not ready. - A lot of plugins do not have integration yet - Another themes around: - github.com/afonsof/jenkins-material-theme - github.com/djonsson/jenkins-atlassian-theme $ docker run -p 8080:8080 jenkins/blueocean
  9. Jenkins pipelines

  10. Jenkins declarative pipelines

  11. Usual problems with Jenkins scripts - Jenkins has a lot

    of disrepute - On early days all were scripts, too many bash&sh scripts - Groovy pipelines are nice, but the learning curve is hard. - Declarative pipelines - Easy to understand. - Still has the power of using Groovy scripts - You should use declarative pipelines
  12. Declarative Pipelines pipeline { agent any options { timeout(time: 300,

    unit: 'MINUTES') } stages { stage("First"){ steps { sh "echo 'Hello World!'" } } } } Jenkinsfile
  13. Stages pipeline { agent any stages { stage("First"){ steps {

    sh "echo 'Hello World!'"} } stage("Second"){ steps { sh "echo 'Hello World!'"} } stage("Third"){ steps { sh "echo 'Hello World!'"} } } } Jenkinsfile
  14. Agent pipeline { agent { label "fedora" } stages {

    stage("OsX"){ agent { label "osX" } steps { sh "echo 'Hello World!'" } } } } Jenkinsfile
  15. Options pipeline { options { timeout(time: 500, unit: 'MINUTES') timestamps()

    ansiColor('xterm') } stages { stage("OsX"){ options { timeout(time: 10, unit: 'MINUTES') } steps { sh "echo 'Hello World!'" } } } } Jenkinsfile
  16. Timeouts Job global timeout (20min) Stage 1 (10m) Stage 2(10m)

    Stage 3 (20m) Waiting in the queue Job global timeout (2 hours) Stage 1 (10m) Stage 2(10m) Stage 3(20m) Waiting in the queue On autoscaling slaves or busy systems, use a large global timeout and adjust stages timeouts. Global Timeout
  17. Retry pipeline{ stages{ stage("deploy"){ steps{ retry(3){ sh './apply.sh' } sh

    './post.sh' } } } } Jenkinsfile
  18. Environment variables pipeline { environment { CFG="TEST" DISTRIBUTION="fedora" } stages

    { stage("OsX"){ environment { DISTRIBUTION="osx" } steps { sh 'echo "${DISTRIBUTION}"' } } } } Jenkinsfile
  19. Job parameters pipeline { parameters { string( defaultValue: 'master', name:

    'FRONTEND_BRANCH') choice( choices: 'eu-central-1\neu-west-1', description: 'What AWS region?', name: 'region') } stages { stage("Unit Testing"){ steps {sh "echo '${FRONTEND_BRANCH}'"} } } } Jenkinsfile
  20. Post actions pipeline { stages { stage("Unit Testing"){ steps {sh

    "go test"} post { always { junit '*.xml' } } } } post { always { cleanWs(); } } } Jenkinsfile
  21. Post actions pipeline { stages { ...} post { always

    { cleanWs(); } success { echo "It works!" } failure { echo "It fails!" } changed { echo "I'm different" } } } Jenkinsfile
  22. Parallel pipeline{ stages{ stage("Browser testing"){ parallel( "Firefox": { sh "echo

    testing FFX" sh "echo more steps" }, "Chrome": { sh "echo testing Chrome" sh "echo more steps" } ) } } } Jenkinsfile
  23. When pipeline { agent any stages { stage('Deploy') { when

    { allOf { branch 'master'; environment name: 'DEPLOY_TO', value: 'production' } } steps { sh 'docker push ..' } } } } Jenkinsfile
  24. Human interaction pipeline { agent any stages { stage('Deploy') {

    steps { input ( id : "Deploy", message: "Do you want to deploy the application?" ) } } } } Jenkinsfile
  25. Human interaction

  26. Credentials

  27. Credentials types - Username with password - Secret text -

    Secret file - SSH Username with private key - Certificate
  28. Credentials pipeline { stages { stage("deploy"){ steps { withCredentials([string( credentialsId:

    'mytoken', variable: 'TOKEN')]) { sh 'echo "Token->${TOKEN}"' } } } } } Jenkinsfile
  29. Credentials pipeline { stages { stage("deploy"){ environment { TOKEN=credentials('MY_TOKEN') }

    steps { sh 'echo ${TOKEN}' } } } } Jenkinsfile
  30. Credentials pipeline { stages { stage("deploy"){ environment { AWS=credentials('AWS') }

    steps { sh 'echo "${AWS_USR}-${AWS_PWD}"' } } } } Jenkinsfile
  31. Jenkins shared libraries

  32. Jenkins Shared libraries ./vars/wrappedNode.groovy ./vars/withVpn.groovy ./vars/withTool.groovy ./vars/withChownWorkspace.groovy ./vars/s3Fetch.groovy ./vars/s3Archive.groovy ./vars/nodeExists.groovy

    ./vars/golangTester.groovy ./vars/gitCommit.groovy ./vars/getOutput.groovy ./vars/documentationChecker.groovy ./vars/dockerImageTagToDigest.groovy ./vars/codecov.groovy ./src/com/docker/utilities/AWSSync.groovy ./src/com/docker/projects/EngineApi.groovy ./src/com/docker/projects/Docker.groovy github.com/docker/jenkins-pipeline-scripts
  33. Jenkins Shared Libraries String call() { if (env.GIT_COMMIT == null)

    { env.GIT_COMMIT = sh( script: "git rev-parse HEAD", returnStdout: true).trim() } env.GIT_COMMIT } vars/gitCommit.groovy
  34. Jenkins Shared libraries @Library('utils') _ pipeline { stages { stage("Checkout"){

    steps { gitCommit() } } } } Jenkinsfile
  35. Job DSL

  36. None
  37. None
  38. Job DSL example pipelineJob('DSL_Pipeline') { def repo = 'https://github.com/eloycoto/jenkins_test.git' description("Pipeline

    for $repo") triggers { scm('H/5 * * * *') } definition { cpsScm { scm { git { remote { url(repo) } branches('master', '**/feature*') scriptPath('Jenkinsfile') extensions { } } } } } } Jenkinsfile
  39. Job DSL loops def jobsSpecs = [ "eloycoto/jenkins_test.git": "jenkinsfile" ]

    jobsSpecs.each{ JobRepo, JobScriptName -> pipelineJob("${JobRepo}") { description("Pipeline for ${JobRepo}") ... ... scriptPath("${JobScriptName}") } } Jenkinsfile https://job-dsl.herokuapp.com/
  40. Job DSL configuration

  41. Scaling Jenkins

  42. Add Slave http://localhost:8080/computer/createItem

  43. Autoscaling slaves - Amazon EC2 plugin - Terraform Jenkins slaves

    - github.com/dragoonis/terraform-jenkins - Kubernetes Plugin - github.com/jenkinsci/kubernetes-plugin
  44. Controlling Slaves curl -s -u "${USER}:${TOKEN}" \ "https://localhost:8080/computer/api/json" | \

    jq . Terminal
  45. Must Plugins

  46. Test Result Analyzer

  47. JUnit Attachments post { always{ junit testDataPublishers: [ [$class: 'AttachmentPublisher']],

    testResults: 'test/*.xml' } } Jenkinsfile
  48. Tips

  49. Validating Jenkinsfiles $ export JENKINS=http://localhost:8080 $ curl -X POST \

    -F "jenkinsfile=<jenkinsfile" \ $JENKINS/pipeline-model-converter/validate Terminal
  50. Clean Workspaces pipeline { ... ... post { always {

    cleanWs() } } } Jenkinsfile
  51. API calls $ export URL='http://localhost:8080' # reference $ curl "$URL/api/json"

    # Queue status $ curl "$URL/queue/api/json?pretty=true" # Run groovy scripts ❤ $ CRUMB=$(curl -s "${URL}/crumbIssuer/api/json" \ | jq -r '.crumbRequestField +": "+ .crumb') $ JEscript=$(cat script.groovy) $ curl -s -X POST -H "${CRUMB}" "${URL}/scriptText" \ -d "script=${JEscript}" Terminal
  52. Tricks for success - Implement CI workflow, it pays off!

    - Skip any bash script, use declarative pipelines. - Keep Jenkinsfiles simple. Logic should be avoid. - Export data to Junit, it makes things easier. - Clean the workspace at the end.
  53. Graciñas! Email: eloycoto@acalustra.com Blog: http://acalustra.com Twitter: @eloycoto