Slide 1
Slide 1 text
Jenkins
Slide 2
Slide 2 text
Eloy Coto
Software craftsman
Ex Quobis, IPGlobal, Foehn
Cilium & Kamailio contributor
@eloycoto
[email protected]
Slide 3
Slide 3 text
Why do I need a CI system?
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
What is Jenkins?
Slide 6
Slide 6 text
Jenkins has a XX century interface!
And it is true!
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
Jenkins Blue Ocean
- Like all the Jenkins features, it is a plugin.
- Can be used for job view, the admin part is not ready.
- A lot of plugins do not have integration yet
- Another themes around:
- github.com/afonsof/jenkins-material-theme
- github.com/djonsson/jenkins-atlassian-theme
$ docker run -p 8080:8080 jenkins/blueocean
Slide 9
Slide 9 text
Jenkins pipelines
Slide 10
Slide 10 text
Jenkins declarative pipelines
Slide 11
Slide 11 text
Usual problems with Jenkins scripts
- Jenkins has a lot of disrepute
- On early days all were scripts, too many bash&sh scripts
- Groovy pipelines are nice, but the learning curve is hard.
- Declarative pipelines
- Easy to understand.
- Still has the power of using Groovy scripts
- You should use declarative pipelines
Slide 12
Slide 12 text
Declarative Pipelines
pipeline {
agent any
options {
timeout(time: 300, unit: 'MINUTES')
}
stages {
stage("First"){
steps {
sh "echo 'Hello World!'"
}
}
}
}
Jenkinsfile
Slide 13
Slide 13 text
Stages
pipeline {
agent any
stages {
stage("First"){
steps { sh "echo 'Hello World!'"}
}
stage("Second"){
steps { sh "echo 'Hello World!'"}
}
stage("Third"){
steps { sh "echo 'Hello World!'"}
}
}
}
Jenkinsfile
Slide 14
Slide 14 text
Agent
pipeline {
agent {
label "fedora"
}
stages {
stage("OsX"){
agent {
label "osX"
}
steps {
sh "echo 'Hello World!'"
}
}
}
}
Jenkinsfile
Slide 15
Slide 15 text
Options
pipeline {
options {
timeout(time: 500, unit: 'MINUTES')
timestamps()
ansiColor('xterm')
}
stages {
stage("OsX"){
options {
timeout(time: 10, unit: 'MINUTES')
}
steps {
sh "echo 'Hello World!'"
}
}
}
}
Jenkinsfile
Slide 16
Slide 16 text
Timeouts
Job global timeout (20min)
Stage 1 (10m) Stage 2(10m) Stage 3 (20m)
Waiting in the queue
Job global timeout (2 hours)
Stage 1 (10m) Stage 2(10m) Stage 3(20m)
Waiting in the queue
On autoscaling slaves or busy systems, use a large global timeout and adjust stages timeouts.
Global Timeout
Slide 17
Slide 17 text
Retry
pipeline{
stages{
stage("deploy"){
steps{
retry(3){
sh './apply.sh'
}
sh './post.sh'
}
}
}
}
Jenkinsfile
Slide 18
Slide 18 text
Environment variables
pipeline {
environment {
CFG="TEST"
DISTRIBUTION="fedora"
}
stages {
stage("OsX"){
environment {
DISTRIBUTION="osx"
}
steps {
sh 'echo "${DISTRIBUTION}"'
}
}
}
}
Jenkinsfile
Slide 19
Slide 19 text
Job parameters
pipeline {
parameters {
string(
defaultValue: 'master',
name: 'FRONTEND_BRANCH')
choice(
choices: 'eu-central-1\neu-west-1',
description: 'What AWS region?',
name: 'region')
}
stages {
stage("Unit Testing"){
steps {sh "echo '${FRONTEND_BRANCH}'"}
}
}
}
Jenkinsfile
Slide 20
Slide 20 text
Post actions
pipeline {
stages {
stage("Unit Testing"){
steps {sh "go test"}
post {
always {
junit '*.xml'
}
}
}
}
post {
always {
cleanWs();
}
}
}
Jenkinsfile
Slide 21
Slide 21 text
Post actions
pipeline {
stages { ...}
post {
always {
cleanWs();
}
success {
echo "It works!"
}
failure {
echo "It fails!"
}
changed {
echo "I'm different"
}
}
}
Jenkinsfile
Slide 22
Slide 22 text
Parallel
pipeline{
stages{
stage("Browser testing"){
parallel(
"Firefox": {
sh "echo testing FFX"
sh "echo more steps"
},
"Chrome": {
sh "echo testing Chrome"
sh "echo more steps"
}
)
}
}
}
Jenkinsfile
Slide 23
Slide 23 text
When
pipeline {
agent any
stages {
stage('Deploy') {
when {
allOf {
branch 'master';
environment name: 'DEPLOY_TO',
value: 'production'
}
}
steps {
sh 'docker push ..'
}
}
}
}
Jenkinsfile
Slide 24
Slide 24 text
Human interaction
pipeline {
agent any
stages {
stage('Deploy') {
steps {
input (
id : "Deploy",
message: "Do you want to deploy the
application?"
)
}
}
}
}
Jenkinsfile
Slide 25
Slide 25 text
Human interaction
Slide 26
Slide 26 text
Credentials
Slide 27
Slide 27 text
Credentials types
- Username with password
- Secret text
- Secret file
- SSH Username with private key
- Certificate
Slide 28
Slide 28 text
Credentials
pipeline {
stages {
stage("deploy"){
steps {
withCredentials([string(
credentialsId: 'mytoken',
variable: 'TOKEN')])
{
sh 'echo "Token->${TOKEN}"'
}
}
}
}
}
Jenkinsfile
Slide 29
Slide 29 text
Credentials
pipeline {
stages {
stage("deploy"){
environment {
TOKEN=credentials('MY_TOKEN')
}
steps {
sh 'echo ${TOKEN}'
}
}
}
}
Jenkinsfile
Slide 30
Slide 30 text
Credentials
pipeline {
stages {
stage("deploy"){
environment {
AWS=credentials('AWS')
}
steps {
sh 'echo "${AWS_USR}-${AWS_PWD}"'
}
}
}
}
Jenkinsfile
Slide 31
Slide 31 text
Jenkins shared libraries
Slide 32
Slide 32 text
Jenkins Shared libraries
./vars/wrappedNode.groovy
./vars/withVpn.groovy ./vars/withTool.groovy
./vars/withChownWorkspace.groovy
./vars/s3Fetch.groovy ./vars/s3Archive.groovy
./vars/nodeExists.groovy
./vars/golangTester.groovy
./vars/gitCommit.groovy
./vars/getOutput.groovy
./vars/documentationChecker.groovy
./vars/dockerImageTagToDigest.groovy
./vars/codecov.groovy
./src/com/docker/utilities/AWSSync.groovy
./src/com/docker/projects/EngineApi.groovy
./src/com/docker/projects/Docker.groovy
github.com/docker/jenkins-pipeline-scripts
Slide 33
Slide 33 text
Jenkins Shared Libraries
String call() {
if (env.GIT_COMMIT == null) {
env.GIT_COMMIT = sh(
script: "git rev-parse HEAD",
returnStdout: true).trim()
}
env.GIT_COMMIT
}
vars/gitCommit.groovy
Slide 34
Slide 34 text
Jenkins Shared libraries
@Library('utils') _
pipeline {
stages {
stage("Checkout"){
steps {
gitCommit()
}
}
}
}
Jenkinsfile
Slide 35
Slide 35 text
Job DSL
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
No content
Slide 38
Slide 38 text
Job DSL example
pipelineJob('DSL_Pipeline') {
def repo = 'https://github.com/eloycoto/jenkins_test.git'
description("Pipeline for $repo")
triggers { scm('H/5 * * * *') }
definition {
cpsScm {
scm {
git {
remote { url(repo) }
branches('master', '**/feature*')
scriptPath('Jenkinsfile')
extensions { }
}
}
}
}
}
Jenkinsfile
Slide 39
Slide 39 text
Job DSL loops
def jobsSpecs = [
"eloycoto/jenkins_test.git": "jenkinsfile"
]
jobsSpecs.each{ JobRepo, JobScriptName ->
pipelineJob("${JobRepo}") {
description("Pipeline for ${JobRepo}")
...
...
scriptPath("${JobScriptName}")
}
}
Jenkinsfile
https://job-dsl.herokuapp.com/
Slide 40
Slide 40 text
Job DSL configuration
Slide 41
Slide 41 text
Scaling Jenkins
Slide 42
Slide 42 text
Add Slave
http://localhost:8080/computer/createItem
Slide 43
Slide 43 text
Autoscaling slaves
- Amazon EC2 plugin
- Terraform Jenkins slaves
- github.com/dragoonis/terraform-jenkins
- Kubernetes Plugin
- github.com/jenkinsci/kubernetes-plugin
Slide 44
Slide 44 text
Controlling Slaves
curl -s -u "${USER}:${TOKEN}" \
"https://localhost:8080/computer/api/json" | \
jq .
Terminal
Slide 45
Slide 45 text
Must Plugins
Slide 46
Slide 46 text
Test Result Analyzer
Slide 47
Slide 47 text
JUnit Attachments
post {
always{
junit testDataPublishers: [
[$class: 'AttachmentPublisher']],
testResults: 'test/*.xml'
}
}
Jenkinsfile
Slide 48
Slide 48 text
Tips
Slide 49
Slide 49 text
Validating Jenkinsfiles
$ export JENKINS=http://localhost:8080
$ curl -X POST \
-F "jenkinsfile=
Slide 50
Slide 50 text
Clean Workspaces
pipeline {
...
...
post {
always {
cleanWs()
}
}
}
Jenkinsfile
Slide 51
Slide 51 text
API calls
$ export URL='http://localhost:8080'
# reference
$ curl "$URL/api/json"
# Queue status
$ curl "$URL/queue/api/json?pretty=true"
# Run groovy scripts ❤
$ CRUMB=$(curl -s "${URL}/crumbIssuer/api/json" \
| jq -r '.crumbRequestField +": "+ .crumb')
$ JEscript=$(cat script.groovy)
$ curl -s -X POST -H "${CRUMB}" "${URL}/scriptText" \
-d "script=${JEscript}"
Terminal
Slide 52
Slide 52 text
Tricks for success
- Implement CI workflow, it pays off!
- Skip any bash script, use declarative pipelines.
- Keep Jenkinsfiles simple. Logic should be avoid.
- Export data to Junit, it makes things easier.
- Clean the workspace at the end.
Slide 53
Slide 53 text
Graciñas!
Email: [email protected]
Blog: http://acalustra.com
Twitter: @eloycoto