Old Jenkins, New Jenkins (ENG)

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

© 2019 CloudBees, Inc. All Rights Reserved. DevOptics Software Delivery Visibility & Insights Core Unified Software Delivery & Governance CodeShip CI/CD as a Service Flow Adaptive Release Orchestration DevOps Excellence Jenkins CloudBees Jenkins Distribution CloudBees Jenkins X Distribution 24x7 Technical Support Assisted Updates Support Accelerator Training Customer Success Managers DevOps Consultants Rollout Feature Flag Management Continuous Delivery Products and Services

Slide 5

Slide 5 text

© 2019 CloudBees, Inc. All Rights Reserved. > whoami -cloudbees 5 ● CloudBees Jenkins Distribution ● CloudBees Jenkins Support ● Community (e.g. Java 11 support, Jenkinsfile Runner, JCasC) https://www.cloudbees.com

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

© 2019 CloudBees, Inc. All Rights Reserved. 1. It’s a framework => flexible 2. It’s open source => customizable 3. More than 1600 plugins 4. Big community 5. Commercial support is available Who is Mr. Jenkins?

Slide 10

Slide 10 text

Slide 11

Slide 11 text

© 2019 CloudBees, Inc. All Rights Reserved. Integrations 11 >1600 plugins Integrations with hundreds of tools Automation management Automation for any kind of tasks Code & Commit Build & Config Scan & Test Release Deploy

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

© 2019 CloudBees, Inc. All Rights Reserved. >10 years of compatibility 27 Hudson 2 plugins still work! BUT: • Architecture issues • Old libraries • Obsolete UI • Plugin compatibility issues • Regular Security-updates

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

© 2019 CloudBees, Inc. All Rights Reserved. Approach: Using “classic” Job types: Freestyle, Matrix, JobDSL, … Classic Approaches Keywords: • Script steps to run the logic • Custom Tools Plugin to manage tools • Throttle Concurrent Builds and Lockable Resources to dispatch access to hardware • Naginator Plugin to restart builds on-demand • Parameterized Trigger / Copy artifacts to parallelize tests and to collect results 30

Slide 31

Slide 31 text

© 2019 CloudBees, Inc. All Rights Reserved. Classic Approaches Spaghetti Automation • Many job dependencies • Difficult build flow tracking • Complex configurations • Duplication Expensive maintenance 31

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

© 2019 CloudBees, Inc. All Rights Reserved. Система Docs Plugins Configs Infra- structure Samples Demos Automation Framework … as code Infra- structure Infra- structure SCM Infrastructure as Code Configuration as Code Documentation as Code Pipeline Libraries Pipelines 39

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

© 2019 CloudBees, Inc. All Rights Reserved. Automation as Code 43 Job DSL Pipeline (бывш. Workflow) По-умолчанию предлагается в Jenkins 2.0 Jenkins Job Builder Groovy Groovy YAML Ещё несколько десятков плагинов…

Slide 44

Slide 44 text

Slide 45

Slide 45 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins Pipeline is a… Groovy DSL for automatic flow description ➢ Defined in SCM (Jenkinsfile) or in Job configuration ➢ Scripted or Declarative // Run on a node having the “my_fpga” label node("linux && ml509") { git url:"http://github.com/myorg/myproject.git" sh "make all" } http://bit.ly/pipeline-tutorial 45

Slide 46

Slide 46 text

© 2019 CloudBees, Inc. All Rights Reserved. Syntax sugar on the top of Pipeline • “pipeline { }” closure Common configuration sections • Easy to set up • More static checks • Less flexibility 46 Declarative Pipeline https://github.com/jenkinsci/pipeline-model- definition-plugin/wiki

Slide 47

Slide 47 text

© 2019 CloudBees, Inc. All Rights Reserved. 47 Example. Declarative Pipeline pipeline { agent label:"generic-linux" tools { maven "Maven 3.3.9" jdk "Oracle JDK 8u40" } stages { stage("build") { steps { sh 'mvn clean install -Dmaven.test.failure.ignore=true' } } } post { always { junit "path/to/xml" } failure { mail to:"[email protected]", subject:"FAILURE:${currentBuild.name}", …} } }

Slide 48

Slide 48 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins Pipeline is a… Groovy DSL for automatic flow description Item type in Jenkins Pipeline Job Multi-Branch Pipeline Organization Folder • GitHub org • BitBucket* • … 48

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Slide 56

Slide 56 text

Slide 57

Slide 57 text

Slide 58

Slide 58 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins Configuration… as Code External Tools Jenkins CLI and REST API python- jenkins jenkins- client (java) Configuration Management systems Puppet, Ansible, Chef Docker Solutions within Jenkins project Groovy Hooks Scriptler Plugin JCasC Plugin Just examples… 58

Slide 59

Slide 59 text

Slide 60

Slide 60 text

© 2019 CloudBees, Inc. All Rights Reserved. 60 JCasC Plugin ● Sub-project in Jenkins ● Feb 2018 - first alpha ● Sep 2018 - 1.0 ● Now - 1.8 ● Configuration via YAML jenkinsci/configuration-as-code-plugin

Slide 61

Slide 61 text

© 2019 CloudBees, Inc. All Rights Reserved. jenkins: systemMessage: "JCasC Demo" numExecutors: 1 scmCheckoutRetryCount: 4 mode: NORMAL securityRealm: local: allowsSignup: false users: - id: demoAdmin password: ${adminpw} jenkins.yaml 61

Slide 62

Slide 62 text

© 2019 CloudBees, Inc. All Rights Reserved. • YAML specification • Configuration export from Jenkins • CLI and REST API for configuration management • Pre-flight validations and dry-run • Support of many plugins out-of-the-box • [SOON] – JSON Schema for instances Features 62

Slide 63

Slide 63 text

Slide 64

Slide 64 text

© 2019 CloudBees, Inc. All Rights Reserved. Demo. Local Pipeline Development Env Intellij IDEA Filesystem SCM Plugin • Documentation, Syntax • Static analysis • Debug (only for hooks) Local Jenkins instance Source Code (local .git repos) • Configuration-as-Code • Same as production • Repos with libs • Jenkinsfile JENKINS_HOME (volume) 64

Slide 65

Slide 65 text

© 2019 CloudBees, Inc. All Rights Reserved. JCasC Limitations (temporary?) • Not all plugins are supported • YAML - No dynamic logic / scripting • Startup Chicken&Egg for plugins & jobs • Handling of removed sections • Export from UI – many open issues • No read-only admin access 65

Slide 66

Slide 66 text

© 2019 CloudBees, Inc. All Rights Reserved. Combining JCasC and Groovy hooks 66 JOBS_LOADED EXTENSIONS_AUGMENTED COMPLETED Groovy Hooks: init() JCasC PLUGINS_STARTED . . . Groovy hooks run after JCasC Hooks can fine-tune the configuration: • Missing integrations • Dynamic scripting • Job configurations and migrations Use JCasC where possible!

Slide 67

Slide 67 text

© 2019 CloudBees, Inc. All Rights Reserved. Comparison JCasC Plugin Groovy Hooks YAML – standard declarative definition Low barrier to entry Limited integrations Forward compatibility risks External rollback Apply changes w/o restart No debugging, YAGNI? Dry run mode + validation Export from UI CLI and REST API for CM Plugin management Groovy – general-purpose OOP language High barrier to entry Can manage everything Compatible if API is stable External rollback Restart is required Debugging support Plugin Management 67

Slide 68

Slide 68 text

Slide 69

Slide 69 text

Slide 70

Slide 70 text

Slide 71

Slide 71 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins & Docker. Plugins Docker Plugin Docker Pipeline Plugin Docker Custom Build Environment Plugin … Yet Another Docker Plugin Kubernetes Plugin Kubernetes Pipeline Plugin … 71

Slide 72

Slide 72 text

© 2019 CloudBees, Inc. All Rights Reserved. Example: Docker Pipeline Plugin https://plugins.jenkins.io/docker-workflow docker.image('onenashev/gcc-riscv:6.4-rc').inside { checkout scm sh "make clean test" } 72

Slide 73

Slide 73 text

Slide 74

Slide 74 text

© 2019 CloudBees, Inc. All Rights Reserved. Example: Kubernetes Plugin def label = "gcc-riscv-${UUID.randomUUID().toString()}" podTemplate(name: 'test', label: label, containers: [ containerTemplate(name: 'gcc-riscv', image: 'onenashev/gcc-riscv:6.4-rc’, resourceRequestCpu: ‘2’, resourceLimitMemory: ‘8Gi' ttyEnabled: true, command: 'cat')]) { node(label) { checkout scm sh "make clean test" step([$class: 'TapPublisher', testResults: 'output/test/report.tap', ...]) } } https://plugins.jenkins.io/kubernetes 74

Slide 75

Slide 75 text

Slide 76

Slide 76 text

Slide 77

Slide 77 text

Slide 78

Slide 78 text

Slide 79

Slide 79 text

Slide 80

Slide 80 text

Slide 81

Slide 81 text

Slide 82

Slide 82 text

© 2019 CloudBees, Inc. All Rights Reserved. Aug 31, 2018 – Kohsuke Kawaguchi, “Jenkins: Shifting Gears” https://jenkins.io/blog/2018/08/31/shifting-gears/ Slides: https://drive.google.com/file/d/160LjRKw42XXuXT2n4j8BLCzsSL5z2m2_/view 82

Slide 83

Slide 83 text

Slide 84

Slide 84 text

Slide 85

Slide 85 text

© 2019 CloudBees, Inc. All Rights Reserved. Cloud Native Jenkins “General purpose CI/CD engine that runs on Kubernetes and embraces fundamentally different architecture & extensibility mechanisms” 85 https://flic.kr/p/75ncBF Kohsuke Kawaguchi, 2018

Slide 86

Slide 86 text

© 2019 CloudBees, Inc. All Rights Reserved. ● Best service for each need ● Pay per use ● “Infinite” scaling ● Easy to use ● Easy to maintain ● Fast to develop 86 https://flic.kr/p/75ncBF Cloud Native Jenkins

Slide 87

Slide 87 text

© 2019 CloudBees, Inc. All Rights Reserved. Cloud Native Special Interest Group ● Founded in July 2018 ● https://jenkins.io/sigs/cloud-native ● https://gitter.im/jenkinsci/cloud-native-sig Goal: Improve Jenkins to run on Cloud environments as a "Cloud Native" application 87 ?

Slide 88

Slide 88 text

Slide 89

Slide 89 text

Slide 90

Slide 90 text

Slide 91

Slide 91 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkins X. OOTB experience • Build packs • GitOps • Nexus, chartmuseum, monocular • Environments: Local, staging, production • IDE Integration • Jenkins X Pipelines 91

Slide 92

Slide 92 text

Slide 93

Slide 93 text

Slide 94

Slide 94 text

Slide 95

Slide 95 text

Slide 96

Slide 96 text

Slide 97

Slide 97 text

© 2019 CloudBees, Inc. All Rights Reserved. 97 ● Cloud native ● K8s native ● Scalable ● AWESOME (c) Universal ● GitHub-only ● GitOps-only ● Kubernetes-only ● … Jenkins X 2.x is… Jenkins X 2.x is NOT… Try out Jenkins X if it fits your environment OR if you plan to go all-in Kubernetes. Jenkins is still around for other cases

Slide 98

Slide 98 text

Slide 99

Slide 99 text

Slide 100

Slide 100 text

Slide 101

Slide 101 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkinsfile Runner https://github.com/jenkinsci/jenkinsfile-runner Started by Kohsuke in Mar 2018 3 maintainers, 19 contributors Status: Beta (1.0-beta-8) 101

Slide 102

Slide 102 text

Slide 103

Slide 103 text

© 2019 CloudBees, Inc. All Rights Reserved. Usage ● Pass workspace with project/Jenkinsfile ● Pass cache for local data (if needed) docker run --rm \ -v $(shell pwd)/Jenkinsfile:/workspace/Jenkinsfile \ jenkins4eval/jenkinsfile-runner 103

Slide 104

Slide 104 text

Slide 105

Slide 105 text

Slide 106

Slide 106 text

Slide 107

Slide 107 text

Slide 108

Slide 108 text

Slide 109

Slide 109 text

Slide 110

Slide 110 text

© 2019 CloudBees, Inc. All Rights Reserved. Limitations Jenkinsfile Runner is not fully compatible ● No Web UI, no way to extract Jenkins reports ● No native support of external storage ● No build throttling, cross-master communication, etc. (?) Single container, single agent ● Docker and Kubernetes plugins need tweaks to produce sidecar containers ● Remoting 3.34 allows connection to headless masters ● https://github.com/jenkinsci/kubernetes-plugin/pull/602 110

Slide 111

Slide 111 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkinsfile Runner. Usage examples Jenkins X, Serverless mode (before Jenkins X 2.0 with Tekton) ● https://github.com/jenkins-x/jenkins-x-serverless ci.jenkins.io-runner ● https://github.com/jenkinsci/ci.jenkins.io-runner “FaaS” packaging ● GitHub Actions: https://github.com/jonico/jenkinsfile-runner-github-actions ● Codeship: https://github.com/oleg-nenashev/codeship-jenkinsfile ● AWS Lambda: https://github.com/carlossg/jenkinsfile-runner-lambda ● Project Fn: https://github.com/carlossg/jenkinsfile-runner-fn 111

Slide 112

Slide 112 text

Slide 113

Slide 113 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkinsfile Runner flavors 113 Vanilla: Docker/Binary in the official repo ● https://github.com/jenkinsci/jenkinsfile-runner ● https://hub.docker.com/r/jenkins4eval/jenkinsfile-runner

Slide 114

Slide 114 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkinsfile Runner flavors Vanilla: Docker/Binary in the official repo ● https://github.com/jenkinsci/jenkinsfile-runner ● https://hub.docker.com/r/jenkins4eval/jenkinsfile-runner Custom JFR packages ● https://jenkins.io/blog/2018/10/16/custom-war-packager/ ● Ready-to-fly Docker images with plugins and configs ● Way to go in 95% of cases 114

Slide 115

Slide 115 text

© 2019 CloudBees, Inc. All Rights Reserved. Building custom Jenkinsfile Runner images Custom WAR Packager Jenkinsfile Runner Core and Plugins Configuration-as-code Packaging Ready-to-fly Docker image Base image https://jenkins.io/blog/2018/10/16/custom-war-packager/ 115

Slide 116

Slide 116 text

© 2019 CloudBees, Inc. All Rights Reserved. Custom WAR Packager and Self-configuration 1. Custom WAR Packager is managed by a config YAML 2. Docker image: plugins.txt, Java flags, etc. 3. Groovy Init Scripts 4. AND: Configuration-as-Code Plugin https://plugins.jenkins.io/configuration-as-code Allows configuring Jenkins from YAML 116 https://github.com/jenkinsci/custom-war-packager

Slide 117

Slide 117 text

Slide 118

Slide 118 text

Slide 119

Slide 119 text

Slide 120

Slide 120 text

Slide 121

Slide 121 text

© 2019 CloudBees, Inc. All Rights Reserved. Jenkinsfile Runner on CloudBees CodeShip ● Jenkins has no good SaaS ● CodeShip Pro - CI SaaS with native Docker support ● Free tiers for open-source projects ● What if we run Jenkins there? 121 https://codeship.com/features/pro

Slide 122

Slide 122 text

Slide 123

Slide 123 text

Slide 124

Slide 124 text

© 2019 CloudBees, Inc. All Rights Reserved. Example 2. Building a Jenkins plugin in CodeShip https://github.com/oleg-nenashev/mailer-plugin/tree/codeship-demo ● Using ci.jenkins.io-runner ○ https://github.com/jenkinsci/ci.jenkins.io-runner ● Using the standard Jenkins Pipeline library ○ https://github.com/jenkins-infra/pipeline-library ● Using standard Jenkinsfile ○ buildPlugin() 124

Slide 125

Slide 125 text

Slide 126

Slide 126 text

Slide 127

Slide 127 text

Slide 128

Slide 128 text

Slide 129

Slide 129 text

Slide 130

Slide 130 text

Slide 131

Slide 131 text

Slide 132

Slide 132 text

© 2019 CloudBees, Inc. All Rights Reserved. My wishlist Better tools for plugin maintainers and contributors Cloud Native Jenkins ➢ Pluggable storage ➢ Jenkinsfile Runner ➢ Jenkinsfile Runner Jenkins Plugin ➢ Everything-as-code Modern UX/UI ➢ New framework for developers (JS-based) ➢ REST API / GraphQL 132

Slide 133

Slide 133 text

Slide 134

Slide 134 text

Slide 135

Slide 135 text

A neutral home for the next generation of continuous delivery collaboration

Slide 136

Slide 136 text

Slide 137

Slide 137 text

Slide 138

Slide 138 text

© 2019 CloudBees, Inc. All Rights Reserved. CDF Mission ● CDF believes in the power of Continuous Delivery to empower developers and teams and to produce high quality software more rapidly ● CDF believes in the open-source solutions collectively addressing the whole Software Delivery LifeCycle ● CDF fosters and sustains the ecosystem of open-source, vendor neutral projects through collaborations and interoperability ● CDF advocates this idea and encourages collaborations among practitioners to share and improve their practices 138

Slide 139

Slide 139 text

© 2019 CloudBees, Inc. All Rights Reserved. Projects in CDF ● Jenkins: https://github.com/jenkinsci ● Jenkins-X: https://github.com/jenkins-x ● Spinnaker: https://github.com/spinnaker ● Tekton: https://github.com/tektoncd 139

Slide 140

Slide 140 text

© 2019 CloudBees, Inc. All Rights Reserved. 2019 Jenkins Board and Officer elections ● 3 => 5 board members ● All officers to be reelected: Release, Security, Infrastructure, Events, Documentation (NEW) ● Oct 4 – Nominations close ● Oct 14-27 - Voting ● Nov 4 – Results 140 https://groups.google.com/forum/#!msg/jenkinsci- dev/vKi9JpxTQxY/2KgDsKUeAQAJ

Slide 141

Slide 141 text

Slide 142

Slide 142 text

© 2019 CloudBees, Inc. All Rights Reserved. Join special interest groups (SIGs) Let us know about your pain-points Share your stories and experiences ➢ jenkins.io/blog and other blogs ➢ Meetups ➢ Jenkins DEV list Contribution is not just Code Jenkins needs you! 142

Slide 143

Slide 143 text

Slide 144

Slide 144 text