Leading in IT Education Continuous Integration using Jenkins CI

Leading in IT Education Evgeny Zislis www.devops.co.il

Leading in IT Education Continuous Integration ... is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

Leading in IT Education “Continuous Integration doesn’t get rid of bugs, but it does make them dramatically easier to find and remove.” - Martin Fowler

Leading in IT Education Continuous Integration Practices ● Maintain a single source repository ● Automate the build ● Make your build self-testing ● Every commit should build on an integration machine ● Keep the build fast ● Test in a clone of the production environment ● Make it easy for anyone to get the latest executable ● Everyone can see what’s happening ● Automate deployment

Leading in IT Education Continuous Integration Checklist ● Developers check out code into their private workspaces. ● When done, commit the changes to the repository. ● The CI server monitors the repository and checks out changes as they occur. ● The CI server builds the system and runs unit and integration tests. ● The CI server releases deployable artefacts for testing. ● The CI server assigns a build label to the version of the code it just built. ● The CI server informs the team of the successful build. ● If the build or tests fail, the CI server alerts the team. ● The team fix the issue at the earliest opportunity. ● Continue to continually integrate and test throughout the project.

Leading in IT Education Responsable Teams ● Check in frequently ● Don’t check in broken code ● Don’t check in untested code ● Don’t check in when the build is broken ● Don’t go home after checking in until the system builds

Leading in IT Education Continuous Integration Tools continuous feedback! Jenkins CI / Bamboo / TeamCity Travis CI / Circle CI magnum-ci.com / semaphoreapp.com / codeship.io / drone.io / solanolabs.com / shiningpanda-ci.com hosted-ci.com / fazend.com / appharbor.com / cloudbees.com / clinkerhq.com

Leading in IT Education Jenkins

Leading in IT Education Jenkins is an open source CI server written in Java. It has an enormous community that contributes plugins. Initially, Jenkins was called Hudson, and was developed in side Sun Microsystems' offices. In early 2011, tensions between Oracle and the community lead to a project form, and Jenkins was born. Homepage: http://jenkins-ci.org GitHub: https://github.com/jenkinsci/jenkins

Leading in IT Education Running Jenkins 1. Download Jenkins from the homepage. 2. Run the WAR file standalone version java -jar jenkins.war Jenkins will store its files in the directory configured in the environment variable JENKINS_HOME, default is $HOME/.jenkins Detailed options available using the help flag java -jar jenkins.war --help

Leading in IT Education Jenkins Setup Wizard Bypass the setup wizard JENKINS_JAVA_OPTIONS=-Djenkins.install.runSetupWizard=false

Leading in IT Education

Leading in IT Education

Leading in IT Education

Leading in IT Education

Leading in IT Education

Leading in IT Education

Leading in IT Education snowflake jenkins-es martinfowler.com/bliki/SnowflakeServer.html

Leading in IT Education click me

Leading in IT Education oops

Leading in IT Education phoenix jenkins martinfowler.com/bliki/PhoenixServer.html

Leading in IT Education Jenkins CLI Jenkins comes with a command line interface. To access it, point your browser to http://ip:port/cli More information about using the CLI is on the Jenkins Wiki http://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI Getting help java -jar jenkins-cli.jar -s http://ip:port help

Leading in IT Education 1. Create private/public SSH keypair ssh-keygen -f jenkins_key.pem 2. Configure Jenkins user with public key http://ip:port/me/configure 3. Enable TCP port for JNLP in Jenkins global security 4. Use private key when executing CLI commands java -jar jenkins-cli.jar -s http://ip:port \ -i jenkins_key.pem list-plugins Jenkins CLI: Authenticating

Leading in IT Education A new Jenkins job

Leading in IT Education Jenkins Job

Leading in IT Education A new Jenkins Job cat job-config.xml \ | java -jar jenkins-cli.jar \ -s http://ip:port \ -i jenkins_key.pem \ create-job \ job-name

Leading in IT Education wiki.jenkins-ci.org/display/JENKINS/Terminology ○ Job / Project ○ Build ○ Artifact ○ Node / Slave ○ Up/Down-stream project Jenkins Terminology

Leading in IT Education Install Jenkins configuration **management**

Leading in IT Education github.com/jenkinsci/puppet-jenkins aka forge.puppet.com/rtyler/jenkins puppet jenkins module

Leading in IT Education github.com/chef-cookbooks/jenkins/ aka supermarket.chef.io/cookbooks/jenkins chef jenkins cookbook

Leading in IT Education recipe: jenkins::master node.default['jenkins']['master']['version'] = '1.651' include_recipe 'jenkins::master' # need java too? no problem! include_recipe 'jenkins::java'

Leading in IT Education # Create password credentials jenkins_password_credentials 'wcoyote' do id 'f2361e6b-b8e0-4b2b-890b-82e85bc1a59f' description 'Wile E Coyote' password 'beepbeep' end # Create private key credentials jenkins_private_key_credentials 'wcoyote' do id 'fa3aab48-4edc-446d-b1e2-1d89d86f4458' description 'Wile E Coyote' private_key "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQ..." end jenkins_credentials

Leading in IT Education # :delete credentials example jenkins_credentials 'wcoyote' do action :delete end jenkins_private_key_credentials 'wcoyote' do action :delete end jenkins_credentials

Leading in IT Education jenkins_job # Generate using `template` or download using `cookbook_file` template xml_filename do source 'jobdsl-config.xml.erb' end # Create a jenkins job jenkins_job 'bacon' do config File.join(Chef::Config[:file_cache_path], 'jobdsl-config.xml') action :create end

Leading in IT Education # Install version 1.15 of the greenballs plugin jenkins_plugin 'greenballs' do version '1.15' end # Less typing, a whole bunch of plugins at once %w{ greenballs=1.15 credentials=1.28 }.each do |addon| name, ver = plugin.split('=') jenkins_plugin name do version ver end end jenkins_plugin

Leading in IT Education # Create a slave launched via SSH jenkins_ssh_slave 'executor' do description 'Run test suites' remote_fs '/share/executor' labels ['executor', 'freebsd', 'jail'] # SSH specific attributes user 'jenkins' credentials 'wcoyote' host '172.11.12.53' # or 'slave.example.org' # or node['env][ node['env'] ]['slave_host'] end jenkins_slave

Leading in IT Education jenkins_user # Create a Jenkins user with specific attributes jenkins_user 'grumpy' do full_name 'Grumpy Dwarf' email 'grumpy@example.com' public_keys ['ssh-rsa AAAAB3NzaC1y...'] end data_bag('admins').each do |login| user = data_bag_item('admins', login) jenkins_user user.name end

Leading in IT Education jenkins job dsl github.com/jenkinsci/job-dsl-plugin

Leading in IT Education job dsl plugin - basic job("PROJ-unit-tests") { using 'TMPL-test' scm { git('https://github.com/spring-projects/spring-petclinic.git') } triggers { scm('*/15 * * * *') } steps { // build step maven('-e clean test') } }

Leading in IT Education job dsl plugin - advanced def project = 'quidryan/aws-sdk-test' def branchApi = new URL("https://api.github.com/repos/${project}/branches") def branches = new groovy.json.JsonSlurper().parse(branchApi.newReader()) branches.each { def branchName = it.name job { name "${project}-${branchName}".replaceAll('/','-') scm { git("git://github.com/${project}.git", branchName) } steps { maven("test -Dproject.name=${project}/${branchName}") } } }

Leading in IT Education /* file name "common.groovy" */ class common { // // Use the GitLab API to quest all projects that have a tag in their tag_list // static List projectsTagged( String tagName, String token, String gitlabURL = 'http://my-gitlab.example.com/api/v3' ) { def projectSearch = new URL("${gitlabApiURL}/projects/all?private_token=${token}") def projectResults = new groovy.json.JsonSlurper().parse(projectSearch.newReader()) return projectResults.findAll { project -> tagName in project.tag_list } } } // class common

Leading in IT Education import common use(common) { common.projectsWithTag("jenkins", "${GITLAB_JENKINS_TOKEN}").each { project -> def project_name_canonical = project.path_with_namespace.replaceAll('/', '-') multibranchPipelineJob(project_name_canonical) { displayName project.name_with_namespace description "Build job for ${project.path_with_namespace}: ${project. description}" triggers { periodic 1 // every minute, check for new branches } branchSources { git { credentialsId 'jenkins-gitlab-id' remote project.ssh_url_to_repo } } } } }

Leading in IT Education job dsl playground job-dsl.herokuapp.com

Leading in IT Education Jenkins Pipeline jenkins.io/doc/pipeline/ jenkins.io/solutions/pipeline/

Leading in IT Education

Leading in IT Education // Only keep the 10 most recent builds properties([[$class: 'BuildDiscarderProperty', strategy: [$class: 'LogRotator', numToKeepStr: '10']]]) // https://jenkins.io/doc/pipeline/steps/ node { stage 'clean' // start with an empty workspace deleteDir() stage 'checkout' checkout scm stage 'compile' wrap([$class: 'TimestamperBuildWrapper']) { sh './build.sh compile' } step([$class: 'WarningsPublisher', consoleParsers: [[parserName: 'Java Compiler (javac)']]]) gitlabCommitStatus { } // ... Jenkinsfile

Leading in IT Education // ... stage 'test' wrap([$class: 'TimestamperBuildWrapper']) { sh './build.sh test' } step([$class: 'JUnitResultArchiver', testResults: 'build/test-results/TEST-*.xml']) // trigger sonarqube zip archive: true, dir: 'build', glob: 'reports/**,test-results/**,classes/**', zipFile: 'test- results.zip' build job: "${env.JOB_NAME}-sonarqube", wait: false }

Leading in IT Education

Leading in IT Education

Leading in IT Education Jenkins: Blueocean youtu.be/3dITffteCD4 jenkins.io/projects/blueocean/

Leading in IT Education

Leading in IT Education

Leading in IT Education _Pipeline visualisation_

Leading in IT Education _Branch & pull requests view_

Leading in IT Education _Personalised view_

Leading in IT Education DEMO & QA

Leading in IT Education Jenkins User Conference July 3 at Daniel Hotel eventbrite.com/e/jenkins-user-conference-2016-israel-daniel-hotel-herzliya-tickets-23233603333

Leading in IT Education Thank you! We invite you to join Operations Israel Facebook group on on.fb.me/Ops-IL we are hiring at jobs@devops.co.il www.devops.co.il