Share
Embed
SPb Jenkins Meetup #1 - Jenkins 2.0 и Pipeline-As-Code (Eng)
In the previous presentation I was talking about Jenkins and its weaknesses, which justify the development of Jenkins 2.0. Presentation: https://speakerdeck.com/onenashev/who-is-mr-jenkins-english-version
In this presentation I've concentrated on the major features like incoming UX improvements, new website and Pipeline-as-Code.
Oleg Nenashev PRO
March 10, 2016
Transcript
57 slides
-
St. Petersburg Jenkins Meetup @jenkins_spb Jenkins 2.0 and Pipeline-‐as-‐Code.
What’s going on? St. Petersburg, March 10th,2016 Oleg Nenashev (@oleg_nenashev) CloudBees, Inc. -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 > whoami – Oleg
Nenashev •Jenkins OSS contributor •Core & Security teams member •Plugin Developer and Maintainer •Developer at CloudBees •CloudBees Jenkins Platform •Background: •R&D, HW/Embedded SW •Intel, Sitronics, Synopsys •Large-‐scale automation infrastructure lead @oleg_nenashev oleg-‐nenashev 2 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Hudson/Jenkins – 8
years •Open-‐source contribution – 4 years • My plugins: Ownership, Job Restrictions, … • Legacy: Role Strategy, Custom Tools, Promoted Builds • Core & Security teams in Jenkins •Meetup organizer: • http://www.meetup.com/St-‐Petersburg-‐Jenkins-‐Meetup/ •Jenkins Org Admin in Google Summer of Code 2016 > whoami –v jenkins –m oss 3 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Disclaimer • This presentation
is not a training material • The talk represents the speaker’s personal opinion • This opinion may differ from the official position of CloudBees or Jenkins community 4 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Automation Systems Evolvement Trends
•SW area shifts focus from CI: •Continuous Delivery •DevOps •Many integrations => communities and OSS •Centralization of automation systems •High requirements and strict SLAs •Commercial solutions 6 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 User experience issues Where
are the docs? How to find a plugin? Is this plugin alive? How to configure THIS? Why do they ignore my issues? Where is the maintainer? Why the UI is so complicated? How to implement it? 7 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Jenkins Learning Curve Basic
scripts, CRON with UI Automation flows based on plugins Advanced features, Custom Plugins 8 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Jenkins weaknesses. Limited scalability
Node 1 Node 2 Node N . . . • Master handles the most of the build logic • Jobs on nodes ALWAYS load the master • Input/Output on master is also a bottleneck Multi-‐ tenancy ??? Gearman Plugin (OSS) CloudBees Jenkins Platform 9 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Jenkins weaknesses. Compatibility costs
• More than 5 years of backward compatibility • Hudson 2 plugins still work on Jenkins! • BUT: • Architecture design issues, complex APIs • Outdated libs • Outdated UI • Plugin integration issues 10 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 1 April, 2010 –
Hudson 2.0 Announcement Hudson 2.0 is expected to follow in the intrepid footsteps of other major revamps such as Netscape 5 and Perl 6 in its ambitiousness and innovation. Implementation of core in portable C for greater speed and code readability Revamped storage architecture around the Oracle 11g Database to provide more optimal synergy over flat XML files. https://jenkins-‐ci.org/blog/2010/04/01/announcing-‐the-‐hudson-‐2-‐0-‐roadmap/ Perl 6, 2016 Hudson 2.0, ? 12 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •… don’t want to
spend 4 years on 2.0 •… want to keep the backward compatibility •… want to support major architecture changes in the nearest future •… focus on UX and CD In Jenkins 2.0 we… 13 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •2015 – Discussions at
Jenkins Office Hours •October 2015 -‐ Jenkins 2.0 scope proposals •https://wiki.jenkins-‐ ci.org/display/JENKINS/Jenkins+2.0 Jenkins 2.0. Key Events 14 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •SCALEx16 and FOSDEM2016 (Jan
30-‐31) •First 2.0 Demos •Face-‐to-‐face discussions with Jenkins users •Jenkins 2.0 Contributor Summit (1 февраля) • >20 Jenkins Core developers •Meetup: http://bit.ly/jenkins2-‐summit •Notes: http://bit.ly/jenkins2-‐summit-‐notes •New Plans •Review of Jenkins 2.x improvements Jenkins 2.0. Key Events 15 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Jenkins is accepted to
Google Summer of Code •http://bit.ly/jenkins-‐gsoc-‐wiki •Main Focus – Jenkins 2.0 and UX •9 project ideas, 15 mentors Key Events @Students: Student applications should be submitted till March 24th! Contact the community: http://bit.ly/jenkins-‐gsoc-‐apply 16 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Initial plans, 2015 Jenkins
1.x “Jenkins N.x” Jenkins 2.x Middle of 2016 End of 2016 (?) • Website • Focus on UX and project infra • Minimal compatibility changes • Major architectural changes • At least 1 LTS line in 2016 17 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •In Jenkins 2.0 we
deliver the minimal required set of changes •We do not break the binary compatibility in Jenkins 2.0 •All Jenkins 1.x plugins should work •We continue working on features in Jenkins 2.x New plan, Feb 2016 18 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •UX improvements: •“Plugin gallery”
•New installation wizard •WebUI improvements (config pages, etc.) •WebUI library updates •Security will be enabled by default •Pipeline-‐as-‐Code •API deprecation engine (postponed a bit) Jenkins 2.0. What’s inside the core? http://bit.ly/jenkins20 19 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •February 29: alpha version
=> DONE •March 16: •"feature complete” •beta version •wide testing •March 23: •Release candidate •jenkins.io launch •March 30: (or later) – Release Current plans 20 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Branches are being integrated
now, stay tuned! •Regular alpha version releases •https://jenkins-‐ci.org/2.0/ Jenkins 2.0. Alpha version Overview and download page 21 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •New domain: http://jenkins.io •Static
web-‐site, sources are on GitHub ⇒Easy content management (CD!) ⇒Everybody can contribute a pull-‐request •Content rework •Structured info •MOAR information •Website goes beyond Continuous Integration Jenkins 2.0. Website http://bit.ly/jenkins20 23 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Static website – DONE
•Domain and new layout – DONE •Localization -‐ TODO •Content rework – Work-‐in-‐progress Status 24 -
Editing in GitHub: Markdown, Asciidoc, HAML Various sidebar
panels for better navigation -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Wishlist •More content •Solution
pages •CloudBees moves Jenkins Handbook to OSS •Centralized documentation for plugins •«Plugin gallery» 28 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Focus -‐ Out of
the Box Experience •Installation wizard, plugin management, configuration pages Jenkins 2.0 UI 29 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Plugin selection in alpha-‐2
On hover – popup with the dependency list 31 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Job configuration pages. Tabs
Tabs Restore the old view 33 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Storage along with the
project •Configuration change tracking Why would we benefit from it? 35 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Automation as Code Job
DSL DotCI Pipeline (fka Workflow) Pipeline will be offered by default in Jenkins 2,0 36 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Groovy DSL for build
flow description •Job type in Jenkins (Pipeline Job) •Ecosystem for automation-‐as-‐code •~15 dedicated plugins •Integration in many other plugins What is Pipeline? 37 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Pipeline. Example #1 •
Syntax: Groovy-‐based DSL • Job descriptions can be stored in SCM • E.g. Jenkinsfile in the repo root • http://bit.ly/pipeline-‐tutorial 38 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Pipeline. Available steps •
Build steps and publishers from plugins • Core steps (file operations, etc.) • Extra utility steps 40 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Snippet generator UI •User
sets up settings in UI •Jenkins generates Pipeline command for copy&paste Pipeline features 41 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Parallelization •parallel() – running
of subscripts in parallel •No Freestyle job triggering spaghetti anymore •Multiple node() operations in a job •Parallel runs on different nodes are supported •Multiple scm() usages in a single job •Interactive steps (user input via for) •You can involve manual QA into your CD! Pipeline features 42 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Robustness against master restarts
•Pipeline stores context and stores jobs after the restart •Robustness against network issues •We use Durable Task Plugin •sh() call won’t fail if network fails •Security: •Groovy Sandbox (Script Security Plugin) •Every call from Pipeline is validated (whitelists) Pipeline features 43 -
Pipeline. Example #2. Jenkins IRC bot CD #!groovy def imageName
= 'jenkinsciinfra/ircbot’ node('docker') { checkout scm // Some magic for getting Docker image tag sh 'git rev-‐parse HEAD > GIT_COMMIT' shortCommit = readFile('GIT_COMMIT').take(6) def imageTag = "build${shortCommit}" stage 'Build ircbot' withMavenEnv (["BUILD_NUMBER=${env.BUILD_NUMBER}:${shortCommit}"]) { sh 'make bot' // Make calls Maven, which calls Ant } stage 'Build container' def whale = docker.build("${imageName}:${imageTag}”) stage 'Deploy container' whale.push() } https://github.com/jenkins-‐infra/ircbot Temporary solution -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Multi-‐Branch Pipeline Plugin •Job
configuration only contains reference to the SCM (Git/GitHub, BitBucket, …) •Jenkins takes Pipeline from Jenkinsfile in SCM •Jenkins creates separate subtasks for branches, builds and reports results separately •[COMING SOON] – Pull-‐request builds •(+) Same thing for GitHub and BitBucket orgs •1 single spec in Jenkins for the many projects Pipeline Features. Multi-‐Branch 46 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •load() – loading of
Groovy code from a file •CPS Global library •Comes as a part of Pipeline plugin set •Creates Git repo within Jenkins •Defines global variables and classes •Pipeline Remote Loader • https://github.com/jenkinsci/workflow-‐remote-‐loader-‐plugin •Loads Pipeline from Git/GitHub •The plugin logic is implemented as Pipeline Code sharing between Pipelines 48 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Pipeline Remote Loader. Example
stage 'Load files from GitHub' def environment, helloworld fileLoader.withGit ('https://github.com/jenkinsci/workflow-‐remote-‐loader-‐plugin.git', 'master', null, '') { helloworld = fileLoader.load('examples/fileLoader/helloworld'); environment = fileLoader.load('examples/fileLoader/environment'); } stage 'Run methods from the loaded content' helloworld.printHello() environment.dumpEnvVars() 49 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •More plugin integrations •Pipeline
Stage view •Pipeline syntax highlighting and docs in IDE •Graphical editor of Jenkins pipelines Coming soon… 50 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •«Pretty» integration for many
plugins •Compatibility list: •http://bit.ly/pipeline-‐compatibility •If you need other steps… Compatibility Snippet generator is your friend 51 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 • CloudBees moves the
project to OSS • Beta version is available in the experimental update center Pipeline Stage View • Stage() • Separate logs for stages • Good UI for interactive steps 52 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Syntax highlighting and code
completion •Intellij IDEA – GDSL -‐ works •Eclipse DSLD – PoC – almost works •Built-‐in documentation -‐ TODO •Context help – TODO IDE Integration 53 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 •Jenkins website: https://jenkins-‐ci.org •New
website prototype: http://jenkins.io •Jenkins 2.0: https://jenkins-‐ci.org/2.0/ •Russian-‐speaking community resources: •Gitter: https://gitter.im/jenkinsci-‐ru/public •Twitter – @jenkins_ru •Pipeline-‐as-‐Code tutorial: •https://github.com/jenkinsci/workflow-‐ plugin/blob/master/TUTORIAL.md Links 56 -
@oleg_nenashev, @jenkins_spb 2.0 March 10, 2016 Q & A Contacts:
• E-‐mail: [email protected] • Gitter: /oleg-‐nenashev • Twitter: @oleg_nenashev