Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Who is Mr. Jenkins? (English Version)

Who is Mr. Jenkins? (English Version)

English version of https://speakerdeck.com/onenashev/spb-jenkins-meetup-number-0-who-is-mr-jenkins , which has been presented at St. Petersburg Jenkins meetup.

The presentation provides a brief introduction to the current Jenkins project state, it's common weaknesses and ongoing activities within the community.

Oleg Nenashev

December 09, 2015

More Decks by Oleg Nenashev

Other Decks in Programming


  1. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb ©  2015  CloudBees,   Inc.    All  Rights  Reserved Who is Mr. Jenkins? Current state, common usage issues and trends in the community Oleg Nenashev, CloudBees, Inc. St.  Petersburg  Jenkins  Meetup  #0 @jenkins_spb December  9th,  2015
  2. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb > Whoami – Oleg Nenashev • Background: R&D, HW/Embedded SW • Jenkins OSS contributor – Core & Security teams member – Plugin Developer and Maintainer • Developer at CloudBees – CloudBees Jenkins Platform https://ru.linkedin.com/in/onenashev @oleg_nenashev Twitter: LinkedIn: GitHub:    https://github.com/oleg-­‐nenashev
  3. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb ©  2015  CloudBees,   Inc.    All  Rights  Reserved Outline 3 • Introduction.  What  is  Jenkins? • Common  Jenkins  usage  issues • Why  does  it  happen?  How  to  overcome  the  issues? • Jenkins  trends  (community  changes,  Jenkins  2.0)
  4. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Disclaimer • This presentation is not a training material • The talk represents the speaker’s personal opinion • This opinion may differ from the official position of CloudBees or Jenkins community 4
  5. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins 101 5 • 2005  – Hudson  (Sun) • 2008  – It  becomes  popular,  Duke's  Choice  Award • 2011  – Conflict  with  Oracle,  renaming  to  Jenkins • License:  MIT  (core) • https://github.com/jenkinsci/
  6. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins is popular 6 http://stats.jenkins-­‐ci.org/jenkins-­‐stats/ • 1000k   installations • ~300  active   committers  (6   months) • ~3  new  plugins   every  week • ~30  plugin   releases  every   week
  7. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Who is Mr. Jenkins? 7 Jenkins – Continuous  Integration System Automation Framework
  8. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Why “automation”? 8 Continuous  Integration Continuous   Integration Continuous   Delivery DevOps Accounting   automation Home   Automation CRON  + WebUI
  9. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Why “framework”? 9 System Flexible  and  Extensible Minimal  OOTB  functionality Solves  basic  use-­‐cases  only Requires  customization Jenkins  is…
  10. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb 10 Система Infra-­‐ structure Docs Infra-­‐ structure Plugins Configs Infra-­‐ structure Templates   Demos Jenkins-based Automation System
  11. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb How does Jenkins grow? After setup 11
  12. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb How does Jenkins grow? Small instance 12 Dozens  of  jobs Several  nodes Small  number  of  plugins Several  users
  13. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb How does Jenkins grow? It becomes popular 13 Hundreds  of  jobs Dozens  of  nodes Many  plugins Dozens  of  users Complex  configurations
  14. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb 14 By  NASA,  ESA,  J.  Hester  and  A.  Loll  (Arizona  State  University)  (HubbleSite:  gallery,  release.)  [Public  domain],  via  Wikimedia   Commons.  https://upload.wikimedia.org/wikipedia/commons/thumb/0/00/Crab_Nebula.jpg/512px-­‐Crab_Nebula.jpg” There  was  Jenkins At  some  point… Ongoing  projects   have  been  affected   as  well
  15. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Why does Jenkins blow up? •Jenkins is scalable •Default Jenkins setups are not scalable •Scalability requires proper system design 15
  16. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule for admins #0. Design automation systems as an internal products 16 But  there  are  issues  in  Jenkins  as  well…
  17. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb User experience issues 17 Where  are  the  docs? How  to  find  a  plugin? Is  this  plugin  alive? How  to  configure  THIS? Why  do  they  ignore  my  issues? Where  is  the  maintainer? Why  the  UI  is  so  complicated? How  to  implement  it?
  18. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins Learning Curve 18 Basic  scripts,   CRON  with  UI Automation   flows  based   on  plugins Advanced  features, Custom  Plugins
  19. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins weaknesses. Limited scalability 19 Node  1 Node  2 Node  N .  .  . • Master  handles  the  most  of  the  build  logic • Jobs  on  nodes  ALWAYS load  the  master • Input/Output on  master  is  also  a  bottleneck Multi-­‐ tenancy ??? Gearman Plugin  (OSS) CloudBees  Jenkins  Platform
  20. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins weaknesses. Growth Problems 20 Jenkins  used  to  grow  faster  than  its  Dev  community
  21. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins weaknesses. Compatibility costs 21 • More  than  5  years  of  backward  compatibility • Hudson  2  plugins  still  work  on  Jenkins! • BUT: • Architecture  design  issues,  complex  APIs • Outdated  libs • Outdated  UI • Plugin  integration  issues
  22. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb The problems can be partially solved! 22
  23. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule for admins #1. Do not overload Masters! 23
  24. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule #1. Do not overload Masters! •Use labels to specify target nodes •or Job Restrictions Plugin to secure the master 24
  25. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule for admins #2. Test Jenkins changes before the deployment! 25
  26. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Test servers •Must-have for large-scale instances •Target: best possible coverage 26 New   feature/plugin Bugfix Test  server Production Showstoppers  only Production   mirror?
  27. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule for admins #3. Use LTS versions on production 27
  28. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins LTS releases • LTS are much more stable • Bugs backporting, 3 monthly patch releases 28
  29. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb LTS Specifics •LTS releases still require testing – Potential regressions across major versions – SECURITY fixes cause “regressions” •3 months is not enough, what to do? – Custom-fork of Jenkins core [and plugins] – Enterprise-solutions based on Jenkins – Help community to increase coverage and to extend the support 29
  30. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule for admins #4 Memento Mori 30
  31. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Your Jenkins instance will die •Major risk: System and Infra failures •Updates may corrupt the system as well ⇒ALWAYS perform backups – Periodic Backup Plugin, Backup Plugin, … •Config History Plugin – configs backup •Naginator Plugin – failover of jobs 31
  32. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Job Config History Plugin – configuration backup 32 •Jenkins System Configs •Jobs •Nodes
  33. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Monitoring •Monitoring Plugin (Java Melody) •Reporting to external systems –Nagios, Zenoss, Zabbix, … •Centralized log collection & processing 33 Don’t  forget  about  the   infrastructure  and  backend -­‐ they  frequently  fail !
  34. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Summary • Jenkins has real weaknesses • Jenkins is effective if you know how to solve them • Useful materials: – http://jenkins-ci.org - Blogs, Wiki, Bugtracker – Materials from Jenkins User Conferences: o https://www.cloudbees.com/company/events/juc – Google Search – Need help? o IRC: #jenkinsci o https://groups.google.com/forum/#!forum/jenkinsci-users 34
  35. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Automation Systems Evolvement Trends •SW area shifts focus from CI: – Continuous Delivery – DevOps •Many integrations => communities and OSS •Centralization of automation systems – High requirements and strict SLAs – Commercial solutions 35
  36. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Community reorganization •New teams focusing on hot areas •Community manager •More contributors (mostly plugin developers) •Jenkins Board re-elections 36 Board Core  Team Security LTS Community INFRA Website
  37. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb “Adopt a plugin!” 37 http://bit.ly/adoptplugin
  38. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Automation as Code 38 Job  DSL DotCI* Workflow Jenkinsfile +  Multi-­‐branch *not  available  in  the  main  UC
  39. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Plans under discussion (not a final roadmap) 40 Jenkins  1.x “Jenkins  3.x” Jenkins  2.x Middle  of  2016 End  of  2016  (?) • Website • Focus  on  UX  and  project  infra • Minimal  compatibility  changes • Major  architectural   changes • At  least  1  LTS  line  in  2016
  40. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins 2.0. Website •Static web-site with sources on GitHub ⇒Improves contributions •Content rework – MORE structured info – Going beyond Continuous Integration 41 http://bit.ly/jenkins20
  41. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins 2.0. What’s in the core? •UX: – “Plugin gallery” – Web UI improvements (configs, etc.) •Update of WebUI libraries •API deprecation and deletion engine 42 http://bit.ly/jenkins20
  42. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb «Jenkins 3.0» •Aim– architecture changes •Features list– TBD – Database storage for configs and build data (Andrew Bayer) – Remoting 3.0 (Oleg Nenashev) – … – <feel free to propose changes> 44
  43. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb How to contribute? 45 http://bit.ly/jenkinscidev 1.  GOTO  Jenkins  mailing  list: 2.  Search  for  Jenkins  2.0  tracks:
  44. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb ©  2015  CloudBees,   Inc.    All  Rights  Reserved 46
  45. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Useful links • Jenkins project – Website: http://jenkins-ci.org – GitHub: http://github.com/jenkinsci/ • CloudBees – Website: https://www.cloudbees.com – CloudBees Jenkins Platform o https://www.cloudbees.com/products/cloudbees-jenkins-platform 47