Tweet
Tweet

Slide 1

Slide 1 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb ©  2015  CloudBees,   Inc.    All  Rights  Reserved Who is Mr. Jenkins? Current state, common usage issues and trends in the community Oleg Nenashev, CloudBees, Inc. St.  Petersburg  Jenkins  Meetup  #0 @jenkins_spb December  9th,  2015

Slide 2

Slide 2 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb > Whoami – Oleg Nenashev • Background: R&D, HW/Embedded SW • Jenkins OSS contributor – Core & Security teams member – Plugin Developer and Maintainer • Developer at CloudBees – CloudBees Jenkins Platform https://ru.linkedin.com/in/onenashev @oleg_nenashev Twitter: LinkedIn: GitHub:    https://github.com/oleg-­‐nenashev

Slide 3

Slide 3 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb ©  2015  CloudBees,   Inc.    All  Rights  Reserved Outline 3 • Introduction.  What  is  Jenkins? • Common  Jenkins  usage  issues • Why  does  it  happen?  How  to  overcome  the  issues? • Jenkins  trends  (community  changes,  Jenkins  2.0)

Slide 4

Slide 4 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Disclaimer • This presentation is not a training material • The talk represents the speaker’s personal opinion • This opinion may differ from the official position of CloudBees or Jenkins community 4

Slide 5

Slide 5 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Jenkins 101 5 • 2005  – Hudson  (Sun) • 2008  – It  becomes  popular,  Duke's  Choice  Award • 2011  – Conflict  with  Oracle,  renaming  to  Jenkins • License:  MIT  (core) • https://github.com/jenkinsci/

Slide 6

Slide 6 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Jenkins is popular 6 http://stats.jenkins-­‐ci.org/jenkins-­‐stats/ • 1000k   installations • ~300  active   committers  (6   months) • ~3  new  plugins   every  week • ~30  plugin   releases  every   week

Slide 7

Slide 7 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Who is Mr. Jenkins? 7 Jenkins – Continuous  Integration System Automation Framework

Slide 8

Slide 8 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Why “automation”? 8 Continuous  Integration Continuous   Integration Continuous   Delivery DevOps Accounting   automation Home   Automation CRON  + WebUI

Slide 9

Slide 9 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Why “framework”? 9 System Flexible  and  Extensible Minimal  OOTB  functionality Solves  basic  use-­‐cases  only Requires  customization Jenkins  is…

Slide 10

Slide 10 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb 10 Система Infra-­‐ structure Docs Infra-­‐ structure Plugins Configs Infra-­‐ structure Templates   Demos Jenkins-based Automation System

Slide 11

Slide 11 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb How does Jenkins grow? After setup 11

Slide 12

Slide 12 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb How does Jenkins grow? Small instance 12 Dozens  of  jobs Several  nodes Small  number  of  plugins Several  users

Slide 13

Slide 13 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb How does Jenkins grow? It becomes popular 13 Hundreds  of  jobs Dozens  of  nodes Many  plugins Dozens  of  users Complex  configurations

Slide 14

Slide 14 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb 14 By  NASA,  ESA,  J.  Hester  and  A.  Loll  (Arizona  State  University)  (HubbleSite:  gallery,  release.)  [Public  domain],  via  Wikimedia   Commons.  https://upload.wikimedia.org/wikipedia/commons/thumb/0/00/Crab_Nebula.jpg/512px-­‐Crab_Nebula.jpg” There  was  Jenkins At  some  point… Ongoing  projects   have  been  affected   as  well

Slide 15

Slide 15 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Why does Jenkins blow up? •Jenkins is scalable •Default Jenkins setups are not scalable •Scalability requires proper system design 15

Slide 16

Slide 16 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Rule for admins #0. Design automation systems as an internal products 16 But  there  are  issues  in  Jenkins  as  well…

Slide 17

Slide 17 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb User experience issues 17 Where  are  the  docs? How  to  find  a  plugin? Is  this  plugin  alive? How  to  configure  THIS? Why  do  they  ignore  my  issues? Where  is  the  maintainer? Why  the  UI  is  so  complicated? How  to  implement  it?

Slide 18

Slide 18 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Jenkins Learning Curve 18 Basic  scripts,   CRON  with  UI Automation   flows  based   on  plugins Advanced  features, Custom  Plugins

Slide 19

Slide 19 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Jenkins weaknesses. Limited scalability 19 Node  1 Node  2 Node  N .  .  . • Master  handles  the  most  of  the  build  logic • Jobs  on  nodes  ALWAYS load  the  master • Input/Output on  master  is  also  a  bottleneck Multi-­‐ tenancy ??? Gearman Plugin  (OSS) CloudBees  Jenkins  Platform

Slide 20

Slide 20 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Jenkins weaknesses. Growth Problems 20 Jenkins  used  to  grow  faster  than  its  Dev  community

Slide 21

Slide 21 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Jenkins weaknesses. Compatibility costs 21 • More  than  5  years  of  backward  compatibility • Hudson  2  plugins  still  work  on  Jenkins! • BUT: • Architecture  design  issues,  complex  APIs • Outdated  libs • Outdated  UI • Plugin  integration  issues

Slide 22

Slide 22 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb The problems can be partially solved! 22

Slide 23

Slide 23 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Rule for admins #1. Do not overload Masters! 23

Slide 24

Slide 24 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Rule #1. Do not overload Masters! •Use labels to specify target nodes •or Job Restrictions Plugin to secure the master 24

Slide 25

Slide 25 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Rule for admins #2. Test Jenkins changes before the deployment! 25

Slide 26

Slide 26 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Test servers •Must-have for large-scale instances •Target: best possible coverage 26 New   feature/plugin Bugfix Test  server Production Showstoppers  only Production   mirror?

Slide 27

Slide 27 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Rule for admins #3. Use LTS versions on production 27

Slide 28

Slide 28 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Jenkins LTS releases • LTS are much more stable • Bugs backporting, 3 monthly patch releases 28

Slide 29

Slide 29 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb LTS Specifics •LTS releases still require testing – Potential regressions across major versions – SECURITY fixes cause “regressions” •3 months is not enough, what to do? – Custom-fork of Jenkins core [and plugins] – Enterprise-solutions based on Jenkins – Help community to increase coverage and to extend the support 29

Slide 30

Slide 30 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Rule for admins #4 Memento Mori 30

Slide 31

Slide 31 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Your Jenkins instance will die •Major risk: System and Infra failures •Updates may corrupt the system as well ⇒ALWAYS perform backups – Periodic Backup Plugin, Backup Plugin, … •Config History Plugin – configs backup •Naginator Plugin – failover of jobs 31

Slide 32

Slide 32 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Job Config History Plugin – configuration backup 32 •Jenkins System Configs •Jobs •Nodes

Slide 33

Slide 33 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Monitoring •Monitoring Plugin (Java Melody) •Reporting to external systems –Nagios, Zenoss, Zabbix, … •Centralized log collection & processing 33 Don’t  forget  about  the   infrastructure  and  backend -­‐ they  frequently  fail !

Slide 34

Slide 34 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Summary • Jenkins has real weaknesses • Jenkins is effective if you know how to solve them • Useful materials: – http://jenkins-ci.org - Blogs, Wiki, Bugtracker – Materials from Jenkins User Conferences: o https://www.cloudbees.com/company/events/juc – Google Search – Need help? o IRC: #jenkinsci o https://groups.google.com/forum/#!forum/jenkinsci-users 34

Slide 35

Slide 35 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Automation Systems Evolvement Trends •SW area shifts focus from CI: – Continuous Delivery – DevOps •Many integrations => communities and OSS •Centralization of automation systems – High requirements and strict SLAs – Commercial solutions 35

Slide 36

Slide 36 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Community reorganization •New teams focusing on hot areas •Community manager •More contributors (mostly plugin developers) •Jenkins Board re-elections 36 Board Core  Team Security LTS Community INFRA Website

Slide 37

Slide 37 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb “Adopt a plugin!” 37 http://bit.ly/adoptplugin

Slide 38

Slide 38 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Automation as Code 38 Job  DSL DotCI* Workflow Jenkinsfile +  Multi-­‐branch *not  available  in  the  main  UC

Slide 39

Slide 39 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb 39

Slide 40

Slide 40 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Plans under discussion (not a final roadmap) 40 Jenkins  1.x “Jenkins  3.x” Jenkins  2.x Middle  of  2016 End  of  2016  (?) • Website • Focus  on  UX  and  project  infra • Minimal  compatibility  changes • Major  architectural   changes • At  least  1  LTS  line  in  2016

Slide 41

Slide 41 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Jenkins 2.0. Website •Static web-site with sources on GitHub ⇒Improves contributions •Content rework – MORE structured info – Going beyond Continuous Integration 41 http://bit.ly/jenkins20

Slide 42

Slide 42 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Jenkins 2.0. What’s in the core? •UX: – “Plugin gallery” – Web UI improvements (configs, etc.) •Update of WebUI libraries •API deprecation and deletion engine 42 http://bit.ly/jenkins20

Slide 43

Slide 43 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Update Manager 2.0 43

Slide 44

Slide 44 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb «Jenkins 3.0» •Aim– architecture changes •Features list– TBD – Database storage for configs and build data (Andrew Bayer) – Remoting 3.0 (Oleg Nenashev) – … – 44

Slide 45

Slide 45 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb How to contribute? 45 http://bit.ly/jenkinscidev 1.  GOTO  Jenkins  mailing  list: 2.  Search  for  Jenkins  2.0  tracks:

Slide 46

Slide 46 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb ©  2015  CloudBees,   Inc.    All  Rights  Reserved 46

Slide 47

Slide 47 text

sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved @jenkins_spb Useful links • Jenkins project – Website: http://jenkins-ci.org – GitHub: http://github.com/jenkinsci/ • CloudBees – Website: https://www.cloudbees.com – CloudBees Jenkins Platform o https://www.cloudbees.com/products/cloudbees-jenkins-platform 47