Tweet
Tweet

Slide 1

Slide 1 text

Jenkins as a code Leveraging Groovy for infrastructure management Łukasz Szczęsny & Marcin Zajączkowski Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog GR8Day Warsaw, 19th March 2016

Slide 2

Slide 2 text

About Łukasz Software engineer @ Uber FOSS and Open Hardware lover Co-organizer of the Warsaw Linux User Group Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 3

Slide 3 text

About Marcin Areas of expertise Automatic Testing / TDD Software Craftsmanship / Code Quality Java8 / Groovy Concurrency / Parallel Computing / Reactive Systems Deployment Automation / Continuous Delivery FOSS projects author and contributor, blogger Leads a small software house - Codearte targeted at clients who care about the quality Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 4

Slide 4 text

Agenda Manual Jenkins maintenance Job configuration as code Jenkins Job DSL Infrastructure as code Case study - Continuous Delivery in Jenkins Live demo Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 5

Slide 5 text

Manual Jenkins maintenance Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 6

Slide 6 text

Manual Jenkins maintenance configuration via GUI does not scale slow, error prone, and boring Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 7

Slide 7 text

Manual Jenkins maintenance configuration via GUI does not scale slow, error prone, and boring problematic with dozens of jobs and plugins Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 8

Slide 8 text

Manual Jenkins maintenance configuration via GUI does not scale slow, error prone, and boring problematic with dozens of jobs and plugins mission impossible with several microservices deployed in several countries for multiple products using deployment pipeline with several steps each Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 9

Slide 9 text

Automation to the rescue! Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 10

Slide 10 text

Jenkins Job DSL Job configuration in code Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 11

Slide 11 text

(Jenkins) Job DSL - 2 parts Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 12

Slide 12 text

(Jenkins) Job DSL - 2 parts Domain Specific Language to specify job configuration Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 13

Slide 13 text

(Jenkins) Job DSL - 2 parts Domain Specific Language to specify job configuration Jenkins plugin to transform configuration DSL into real jobs in Jenkins Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 14

Slide 14 text

Job DSL - part 1 - configuration Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 15

Slide 15 text

Job DSL - part 1 - configuration Groovy based DSL (Domain Specific Language) job/view/dashboard configuration developed as "normal" code in IDE with auto-completion type check Groovy magic if needed outside Jenkins instance Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 16

Slide 16 text

Job DSL - simple example j o b ( ' g r 8 d a y - w e b s i t e - p u b l i s h ' ) { s c m { g i t h u b ( ' g r 8 d a y / w e b s i t e ' ) } t r i g g e r s { s c m ( ' * / 1 5 * * * * ' ) } s t e p s { g r a d l e ( ' p u b l i s h ' ) } } Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 17

Slide 17 text

Job DSL - dynamic example S t r i n g r e p o = ' g r 8 d a y / m o b i l e - a p p ' U R L b r a n c h U r l = " h t t p s : / / a p i . g i t h u b . c o m / r e p o s / $ r e p o / b r a n c h e s " . t o U R L ( ) L i s t b r a n c h e s = n e w J s o n S l u r p e r ( ) . p a r s e T e x t ( b r a n c h U r l . t e x t ) b r a n c h e s . e a c h { b r a n c h - > S t r i n g s a f e B r a n c h N a m e = b r a n c h . n a m e . r e p l a c e A l l ( ' / ' , ' - ' ) j o b ( " $ r e p o - $ s a f e B r a n c h N a m e - b u i l d " ) { s c m { g i t h u b r e p o , b r a n c h . n a m e } t r i g g e r s { s c m ' H / 1 0 * * * * ' } s t e p s { g r a d l e ' c h e c k ' } } } Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 18

Slide 18 text

Job DSL - features Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 19

Slide 19 text

Job DSL - features comprehensive support for Jenkins Core stuff Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 20

Slide 20 text

Job DSL - features comprehensive support for Jenkins Core stuff extensive support for additional plugins over 180 plugins as of version 1.44 active community - continuous flow of new pull requests Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 21

Slide 21 text

Job DSL - features comprehensive support for Jenkins Core stuff extensive support for additional plugins over 180 plugins as of version 1.44 active community - continuous flow of new pull requests powerful c o n f i g u r a t i o n block for not yet supported features custom stuff virtually everything possible in XML should be achievable Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 22

Slide 22 text

Job DSL - part 2 - Jenkins plugin Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 23

Slide 23 text

Job DSL - part 2 - Jenkins plugin installed on Jenkins instance used in seed jobs on Jenkins leverages DSL configuration updates jobs & views in Jenkins to bring them to desired state by XML configuration files modification Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 24

Slide 24 text

Job DSL - benefits Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 25

Slide 25 text

Job DSL - benefits source code instead of XML or GUI single source of truth manageable jobs and views backed by SCM reviewable - possibly with pull requests Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 26

Slide 26 text

Job DSL - benefits source code instead of XML or GUI single source of truth manageable jobs and views backed by SCM reviewable - possibly with pull requests testable automatic "unit" testing pre-production environment Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 27

Slide 27 text

Job DSL - benefits source code instead of XML or GUI single source of truth manageable jobs and views backed by SCM reviewable - possibly with pull requests testable automatic "unit" testing pre-production environment scalable hundreds of jobs created/modified in seconds* Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 28

Slide 28 text

Job DSL - drawbacks/limitations Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 29

Slide 29 text

Job DSL - drawbacks/limitations quite steep learning curve can become hard to understand for complex configurations Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 30

Slide 30 text

Job DSL - drawbacks/limitations quite steep learning curve can become hard to understand for complex configurations small error in DSL can remove some/all jobs can be easily recreated, but without execution history Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 31

Slide 31 text

Job DSL - drawbacks/limitations quite steep learning curve can become hard to understand for complex configurations small error in DSL can remove some/all jobs can be easily recreated, but without execution history not suitable for global Jenkins configuration management credentials, machine provisioning, Jenkins and plugin update, ... Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 32

Slide 32 text

Infrastructure Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 33

Slide 33 text

Infrastructure challenges Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 34

Slide 34 text

Infrastructure challenges install and configure Jenkins master install and configure all required dependencies install and configure plugins create and connect slaves add JDK installation configure authentication create credentials ... Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 35

Slide 35 text

Infrastructure toolbelt configuration management tools Ansible Puppet Chef Salt etc. Groovy console Jenkins CLI Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 36

Slide 36 text

Infrastructure toolbelt Slave management Swarm plugin Docker plugin SSH slaves Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 37

Slide 37 text

Continuous Delivery in Jenkins Case study Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 38

Slide 38 text

Continuous Delivery Clearly defined way how to transform source code into project deployed to production a set of steps arranged into pipeline unified way for various projects/variants/realms Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 39

Slide 39 text

Continuous Delivery in Jenkins not a first class citizen in Jenkins 1.x bunch of jobs triggering each other can be emulated with various plugins Delivery Pipeline Plugin, Build Flow Plugin, Pipeline Plugin, ... Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 40

Slide 40 text

Continuous Delivery in Jenkins not a first class citizen in Jenkins 1.x bunch of jobs triggering each other can be emulated with various plugins Delivery Pipeline Plugin, Build Flow Plugin, Pipeline Plugin, ... no easy (and unified) way to setup usually even harder to maintain Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 41

Slide 41 text

Continuous Delivery - case study custom Continuous Delivery framework on top of Jenkins Job DSL one standardized way for Continuous Delivery reused in all projects in the company Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 42

Slide 42 text

Continuous Delivery - case study custom Continuous Delivery framework on top of Jenkins Job DSL one standardized way for Continuous Delivery reused in all projects in the company Ansible for infrastructure management Rundeck for deployment Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 43

Slide 43 text

Continuous Delivery - case study custom Continuous Delivery framework on top of Jenkins Job DSL one standardized way for Continuous Delivery reused in all projects in the company Ansible for infrastructure management Rundeck for deployment open sourced to make live easier to others jenkins-pipeline-dsl - core library sample-jenkins-microservice-pipeline - sample pipeline Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 44

Slide 44 text

Live demo The whole delivery pipeline with one click! Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 45

Slide 45 text

Summary Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 46

Slide 46 text

Summary automation is good for you Jenkins Job DSL is great together with infrastructure tools it is a match made in heaven looking forward to Jenkins 2.0 release to simplify Continuous Delivery pipelines creation Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 47

Slide 47 text

Questions? Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

Slide 48

Slide 48 text

Marcin Zajączkowski @SolidSoftBlog http://blog.solidsoft.info [email protected] IRC: szpak@freenode Łukasz Szczęsny @wybczu https://wybcz.pl luk@{uber.com,wybcz.pl} IRC: wybczu@freenode Thank you Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog