Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Who is Mr. Jenkins? (English Version)

Who is Mr. Jenkins? (English Version)

English version of https://speakerdeck.com/onenashev/spb-jenkins-meetup-number-0-who-is-mr-jenkins , which has been presented at St. Petersburg Jenkins meetup.

The presentation provides a brief introduction to the current Jenkins project state, it's common weaknesses and ongoing activities within the community.

568e3391c8b528f2b255443e4cca27ca?s=128

Oleg Nenashev

December 09, 2015
Tweet

Transcript

  1. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb ©  2015  CloudBees,   Inc.    All  Rights  Reserved Who is Mr. Jenkins? Current state, common usage issues and trends in the community Oleg Nenashev, CloudBees, Inc. St.  Petersburg  Jenkins  Meetup  #0 @jenkins_spb December  9th,  2015
  2. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb > Whoami – Oleg Nenashev • Background: R&D, HW/Embedded SW • Jenkins OSS contributor – Core & Security teams member – Plugin Developer and Maintainer • Developer at CloudBees – CloudBees Jenkins Platform https://ru.linkedin.com/in/onenashev @oleg_nenashev Twitter: LinkedIn: GitHub:    https://github.com/oleg-­‐nenashev
  3. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb ©  2015  CloudBees,   Inc.    All  Rights  Reserved Outline 3 • Introduction.  What  is  Jenkins? • Common  Jenkins  usage  issues • Why  does  it  happen?  How  to  overcome  the  issues? • Jenkins  trends  (community  changes,  Jenkins  2.0)
  4. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Disclaimer • This presentation is not a training material • The talk represents the speaker’s personal opinion • This opinion may differ from the official position of CloudBees or Jenkins community 4
  5. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins 101 5 • 2005  – Hudson  (Sun) • 2008  – It  becomes  popular,  Duke's  Choice  Award • 2011  – Conflict  with  Oracle,  renaming  to  Jenkins • License:  MIT  (core) • https://github.com/jenkinsci/
  6. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins is popular 6 http://stats.jenkins-­‐ci.org/jenkins-­‐stats/ • 1000k   installations • ~300  active   committers  (6   months) • ~3  new  plugins   every  week • ~30  plugin   releases  every   week
  7. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Who is Mr. Jenkins? 7 Jenkins – Continuous  Integration System Automation Framework
  8. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Why “automation”? 8 Continuous  Integration Continuous   Integration Continuous   Delivery DevOps Accounting   automation Home   Automation CRON  + WebUI
  9. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Why “framework”? 9 System Flexible  and  Extensible Minimal  OOTB  functionality Solves  basic  use-­‐cases  only Requires  customization Jenkins  is…
  10. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb 10 Система Infra-­‐ structure Docs Infra-­‐ structure Plugins Configs Infra-­‐ structure Templates   Demos Jenkins-based Automation System
  11. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb How does Jenkins grow? After setup 11
  12. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb How does Jenkins grow? Small instance 12 Dozens  of  jobs Several  nodes Small  number  of  plugins Several  users
  13. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb How does Jenkins grow? It becomes popular 13 Hundreds  of  jobs Dozens  of  nodes Many  plugins Dozens  of  users Complex  configurations
  14. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb 14 By  NASA,  ESA,  J.  Hester  and  A.  Loll  (Arizona  State  University)  (HubbleSite:  gallery,  release.)  [Public  domain],  via  Wikimedia   Commons.  https://upload.wikimedia.org/wikipedia/commons/thumb/0/00/Crab_Nebula.jpg/512px-­‐Crab_Nebula.jpg” There  was  Jenkins At  some  point… Ongoing  projects   have  been  affected   as  well
  15. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Why does Jenkins blow up? •Jenkins is scalable •Default Jenkins setups are not scalable •Scalability requires proper system design 15
  16. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule for admins #0. Design automation systems as an internal products 16 But  there  are  issues  in  Jenkins  as  well…
  17. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb User experience issues 17 Where  are  the  docs? How  to  find  a  plugin? Is  this  plugin  alive? How  to  configure  THIS? Why  do  they  ignore  my  issues? Where  is  the  maintainer? Why  the  UI  is  so  complicated? How  to  implement  it?
  18. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins Learning Curve 18 Basic  scripts,   CRON  with  UI Automation   flows  based   on  plugins Advanced  features, Custom  Plugins
  19. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins weaknesses. Limited scalability 19 Node  1 Node  2 Node  N .  .  . • Master  handles  the  most  of  the  build  logic • Jobs  on  nodes  ALWAYS load  the  master • Input/Output on  master  is  also  a  bottleneck Multi-­‐ tenancy ??? Gearman Plugin  (OSS) CloudBees  Jenkins  Platform
  20. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins weaknesses. Growth Problems 20 Jenkins  used  to  grow  faster  than  its  Dev  community
  21. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins weaknesses. Compatibility costs 21 • More  than  5  years  of  backward  compatibility • Hudson  2  plugins  still  work  on  Jenkins! • BUT: • Architecture  design  issues,  complex  APIs • Outdated  libs • Outdated  UI • Plugin  integration  issues
  22. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb The problems can be partially solved! 22
  23. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule for admins #1. Do not overload Masters! 23
  24. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule #1. Do not overload Masters! •Use labels to specify target nodes •or Job Restrictions Plugin to secure the master 24
  25. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule for admins #2. Test Jenkins changes before the deployment! 25
  26. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Test servers •Must-have for large-scale instances •Target: best possible coverage 26 New   feature/plugin Bugfix Test  server Production Showstoppers  only Production   mirror?
  27. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule for admins #3. Use LTS versions on production 27
  28. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins LTS releases • LTS are much more stable • Bugs backporting, 3 monthly patch releases 28
  29. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb LTS Specifics •LTS releases still require testing – Potential regressions across major versions – SECURITY fixes cause “regressions” •3 months is not enough, what to do? – Custom-fork of Jenkins core [and plugins] – Enterprise-solutions based on Jenkins – Help community to increase coverage and to extend the support 29
  30. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Rule for admins #4 Memento Mori 30
  31. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Your Jenkins instance will die •Major risk: System and Infra failures •Updates may corrupt the system as well ⇒ALWAYS perform backups – Periodic Backup Plugin, Backup Plugin, … •Config History Plugin – configs backup •Naginator Plugin – failover of jobs 31
  32. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Job Config History Plugin – configuration backup 32 •Jenkins System Configs •Jobs •Nodes
  33. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Monitoring •Monitoring Plugin (Java Melody) •Reporting to external systems –Nagios, Zenoss, Zabbix, … •Centralized log collection & processing 33 Don’t  forget  about  the   infrastructure  and  backend -­‐ they  frequently  fail !
  34. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Summary • Jenkins has real weaknesses • Jenkins is effective if you know how to solve them • Useful materials: – http://jenkins-ci.org - Blogs, Wiki, Bugtracker – Materials from Jenkins User Conferences: o https://www.cloudbees.com/company/events/juc – Google Search – Need help? o IRC: #jenkinsci o https://groups.google.com/forum/#!forum/jenkinsci-users 34
  35. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Automation Systems Evolvement Trends •SW area shifts focus from CI: – Continuous Delivery – DevOps •Many integrations => communities and OSS •Centralization of automation systems – High requirements and strict SLAs – Commercial solutions 35
  36. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Community reorganization •New teams focusing on hot areas •Community manager •More contributors (mostly plugin developers) •Jenkins Board re-elections 36 Board Core  Team Security LTS Community INFRA Website
  37. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb “Adopt a plugin!” 37 http://bit.ly/adoptplugin
  38. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Automation as Code 38 Job  DSL DotCI* Workflow Jenkinsfile +  Multi-­‐branch *not  available  in  the  main  UC
  39. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb 39
  40. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Plans under discussion (not a final roadmap) 40 Jenkins  1.x “Jenkins  3.x” Jenkins  2.x Middle  of  2016 End  of  2016  (?) • Website • Focus  on  UX  and  project  infra • Minimal  compatibility  changes • Major  architectural   changes • At  least  1  LTS  line  in  2016
  41. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins 2.0. Website •Static web-site with sources on GitHub ⇒Improves contributions •Content rework – MORE structured info – Going beyond Continuous Integration 41 http://bit.ly/jenkins20
  42. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Jenkins 2.0. What’s in the core? •UX: – “Plugin gallery” – Web UI improvements (configs, etc.) •Update of WebUI libraries •API deprecation and deletion engine 42 http://bit.ly/jenkins20
  43. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Update Manager 2.0 43
  44. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb «Jenkins 3.0» •Aim– architecture changes •Features list– TBD – Database storage for configs and build data (Andrew Bayer) – Remoting 3.0 (Oleg Nenashev) – … – <feel free to propose changes> 44
  45. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb How to contribute? 45 http://bit.ly/jenkinscidev 1.  GOTO  Jenkins  mailing  list: 2.  Search  for  Jenkins  2.0  tracks:
  46. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb ©  2015  CloudBees,   Inc.    All  Rights  Reserved 46
  47. sd ©  2015  CloudBees,   Inc.    All  Rights  Reserved

    @jenkins_spb Useful links • Jenkins project – Website: http://jenkins-ci.org – GitHub: http://github.com/jenkinsci/ • CloudBees – Website: https://www.cloudbees.com – CloudBees Jenkins Platform o https://www.cloudbees.com/products/cloudbees-jenkins-platform 47