Who is Mr. Jenkins? (English Version)

English version of https://speakerdeck.com/onenashev/spb-jenkins-meetup-number-0-who-is-mr-jenkins , which has been presented at St. Petersburg Jenkins meetup.

The presentation provides a brief introduction to the current Jenkins project state, it's common weaknesses and ongoing activities within the community.

Transcript

1. sd
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   @jenkins_spb
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   Who is Mr. Jenkins?
   Current state, common usage issues and
   trends in the community
   Oleg Nenashev,
   CloudBees, Inc.
   St.  Petersburg  Jenkins  Meetup  #0
   @jenkins_spb
   December  9th,  2015
   

   View Slide

2. sd
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   @jenkins_spb
   > Whoami – Oleg Nenashev
   • Background: R&D, HW/Embedded SW
   • Jenkins OSS contributor
   – Core & Security teams member
   – Plugin Developer and Maintainer
   • Developer at CloudBees
   – CloudBees Jenkins Platform
   https://ru.linkedin.com/in/onenashev
   @oleg_nenashev
   Twitter:
   LinkedIn:
   GitHub:    https://github.com/oleg-­‐nenashev
   

   View Slide

3. sd
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   @jenkins_spb
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   Outline
   3
   • Introduction.  What  is  Jenkins?
   • Common  Jenkins  usage  issues
   • Why  does  it  happen?  How  to  overcome  the  issues?
   • Jenkins  trends  (community  changes,  Jenkins  2.0)
   

   View Slide

4. sd
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   @jenkins_spb
   Disclaimer
   • This presentation is not a training material
   • The talk represents the speaker’s personal opinion
   • This opinion may differ from the official position of
   CloudBees or Jenkins community
   4
   

   View Slide

5. sd
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   @jenkins_spb
   Jenkins 101
   5
   • 2005  – Hudson  (Sun)
   • 2008  – It  becomes  popular,  Duke's  Choice  Award
   • 2011  – Conflict  with  Oracle,  renaming  to  Jenkins
   • License:  MIT  (core)
   • https://github.com/jenkinsci/
   

   View Slide

6. sd
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   @jenkins_spb
   Jenkins is popular
   6
   http://stats.jenkins-­‐ci.org/jenkins-­‐stats/
   • 1000k  
   installations
   • ~300  active  
   committers  (6  
   months)
   • ~3  new  plugins  
   every  week
   • ~30  plugin  
   releases  every  
   week
   

   View Slide

7. sd
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   @jenkins_spb
   Who is Mr. Jenkins?
   7
   Jenkins –
   Continuous  Integration
   System
   Automation
   Framework
   

   View Slide

8. sd
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   @jenkins_spb
   Why “automation”?
   8
   Continuous  Integration
   Continuous  
   Integration
   Continuous  
   Delivery
   DevOps
   Accounting  
   automation
   Home  
   Automation
   CRON  + WebUI
   

   View Slide

9. sd
   ©  2015  CloudBees,   Inc.    All  Rights  Reserved
   @jenkins_spb
   Why “framework”?
   9
   System
   Flexible  and  Extensible
   Minimal  OOTB  functionality
   Solves  basic  use-­‐cases  only
   Requires  customization
   Jenkins  is…
   

   View Slide

10. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    10
    Система
    Infra-­‐
    structure
    Docs
    Infra-­‐
    structure
    Plugins Configs
    Infra-­‐
    structure
    Templates  
    Demos
    Jenkins-based
    Automation
    System
    

    View Slide

11. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    How does Jenkins grow? After setup
    11
    

    View Slide

12. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    How does Jenkins grow? Small instance
    12
    Dozens  of  jobs
    Several  nodes
    Small  number  of  plugins
    Several  users
    

    View Slide

13. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    How does Jenkins grow? It becomes popular
    13
    Hundreds  of  jobs
    Dozens  of  nodes
    Many  plugins
    Dozens  of  users
    Complex  configurations
    

    View Slide

14. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    14
    By  NASA,  ESA,  J.  Hester  and  A.  Loll  (Arizona  State  University)  (HubbleSite:  gallery,  release.)  [Public  domain],  via  Wikimedia  
    Commons.  https://upload.wikimedia.org/wikipedia/commons/thumb/0/00/Crab_Nebula.jpg/512px-­‐Crab_Nebula.jpg”
    There  was  Jenkins
    At  some  point…
    Ongoing  projects  
    have  been  affected  
    as  well
    

    View Slide

15. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Why does Jenkins blow up?
    •Jenkins is scalable
    •Default Jenkins setups are not scalable
    •Scalability requires proper system design
    15
    

    View Slide

16. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Rule for admins #0.
    Design automation systems as
    an internal products
    16
    But  there  are  issues  in  Jenkins  as  well…
    

    View Slide

17. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    User experience issues
    17
    Where  are  the  docs?
    How  to  find  a  plugin?
    Is  this  plugin  alive?
    How  to  configure  THIS?
    Why  do  they  ignore  my  issues?
    Where  is  the  maintainer?
    Why  the  UI  is  so  complicated?
    How  to  implement  it?
    

    View Slide

18. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Jenkins Learning Curve
    18
    Basic  scripts,  
    CRON  with  UI
    Automation  
    flows  based  
    on  plugins
    Advanced  features,
    Custom  Plugins
    

    View Slide

19. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Jenkins weaknesses. Limited scalability
    19
    Node  1
    Node  2
    Node  N
    .  .  .
    • Master  handles  the  most  of  the  build  logic
    • Jobs  on  nodes  ALWAYS load  the  master
    • Input/Output on  master  is  also  a  bottleneck
    Multi-­‐
    tenancy
    ??? Gearman Plugin  (OSS)
    CloudBees  Jenkins  Platform
    

    View Slide

20. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Jenkins weaknesses. Growth Problems
    20
    Jenkins  used  to  grow  faster  than  its  Dev  community
    

    View Slide

21. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Jenkins weaknesses. Compatibility costs
    21
    • More  than  5  years  of  backward  compatibility
    • Hudson  2  plugins  still  work  on  Jenkins!
    • BUT:
    • Architecture  design  issues,  complex  APIs
    • Outdated  libs
    • Outdated  UI
    • Plugin  integration  issues
    

    View Slide

22. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    The problems can be partially solved!
    22
    

    View Slide

23. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Rule for admins #1.
    Do not overload Masters!
    23
    

    View Slide

24. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Rule #1. Do not overload Masters!
    •Use labels to specify target nodes
    •or Job Restrictions Plugin to secure the master
    24
    

    View Slide

25. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Rule for admins #2.
    Test Jenkins changes before the
    deployment!
    25
    

    View Slide

26. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Test servers
    •Must-have for large-scale instances
    •Target: best possible coverage
    26
    New  
    feature/plugin
    Bugfix
    Test  server Production
    Showstoppers  only
    Production  
    mirror?
    

    View Slide

27. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Rule for admins #3.
    Use LTS versions on production
    27
    

    View Slide

28. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Jenkins LTS releases
    • LTS are much more stable
    • Bugs backporting, 3 monthly patch releases
    28
    

    View Slide

29. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    LTS Specifics
    •LTS releases still require testing
    – Potential regressions across major versions
    – SECURITY fixes cause “regressions”
    •3 months is not enough, what to do?
    – Custom-fork of Jenkins core [and plugins]
    – Enterprise-solutions based on Jenkins
    – Help community to increase coverage and to
    extend the support
    29
    

    View Slide

30. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Rule for admins #4
    Memento Mori
    30
    

    View Slide

31. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Your Jenkins instance will die
    •Major risk: System and Infra failures
    •Updates may corrupt the system as well
    ⇒ALWAYS perform backups
    – Periodic Backup Plugin, Backup Plugin, …
    •Config History Plugin – configs backup
    •Naginator Plugin – failover of jobs
    31
    

    View Slide

32. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Job Config History Plugin – configuration backup
    32
    •Jenkins System Configs
    •Jobs
    •Nodes
    

    View Slide

33. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Monitoring
    •Monitoring Plugin (Java Melody)
    •Reporting to external systems
    –Nagios, Zenoss, Zabbix, …
    •Centralized log collection & processing
    33
    Don’t  forget  about  the  
    infrastructure  and  backend
    -­‐ they  frequently  fail
    !
    

    View Slide

34. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Summary
    • Jenkins has real weaknesses
    • Jenkins is effective if you know how to solve them
    • Useful materials:
    – http://jenkins-ci.org - Blogs, Wiki, Bugtracker
    – Materials from Jenkins User Conferences:
    o https://www.cloudbees.com/company/events/juc
    – Google Search
    – Need help?
    o IRC: #jenkinsci
    o https://groups.google.com/forum/#!forum/jenkinsci-users
    34
    

    View Slide

35. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Automation Systems Evolvement Trends
    •SW area shifts focus from CI:
    – Continuous Delivery
    – DevOps
    •Many integrations => communities and OSS
    •Centralization of automation systems
    – High requirements and strict SLAs
    – Commercial solutions
    35
    

    View Slide

36. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Community reorganization
    •New teams focusing on
    hot areas
    •Community manager
    •More contributors (mostly
    plugin developers)
    •Jenkins Board re-elections
    36
    Board
    Core  Team
    Security
    LTS
    Community
    INFRA
    Website
    

    View Slide

37. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    “Adopt a plugin!”
    37
    http://bit.ly/adoptplugin
    

    View Slide

38. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Automation as Code
    38
    Job  DSL
    DotCI*
    Workflow
    Jenkinsfile +  Multi-­‐branch
    *not  available  in  the  main  UC
    

    View Slide

39. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    39
    

    View Slide

40. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Plans under discussion (not a final roadmap)
    40
    Jenkins  1.x
    “Jenkins  3.x”
    Jenkins  2.x
    Middle  of  2016
    End  of  2016  (?)
    • Website
    • Focus  on  UX  and  project  infra
    • Minimal  compatibility  changes
    • Major  architectural  
    changes
    • At  least  1  LTS  line  in  2016
    

    View Slide

41. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Jenkins 2.0. Website
    •Static web-site with sources on GitHub
    ⇒Improves contributions
    •Content rework
    – MORE structured info
    – Going beyond Continuous Integration
    41
    http://bit.ly/jenkins20
    

    View Slide

42. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Jenkins 2.0. What’s in the core?
    •UX:
    – “Plugin gallery”
    – Web UI improvements (configs, etc.)
    •Update of WebUI libraries
    •API deprecation and deletion engine
    42
    http://bit.ly/jenkins20
    

    View Slide

43. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Update Manager 2.0
    43
    

    View Slide

44. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    «Jenkins 3.0»
    •Aim– architecture changes
    •Features list– TBD
    – Database storage for configs and build data
    (Andrew Bayer)
    – Remoting 3.0 (Oleg Nenashev)
    – …
    –
    44
    

    View Slide

45. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    How to contribute?
    45
    http://bit.ly/jenkinscidev
    1.  GOTO  Jenkins  mailing  list:
    2.  Search  for  Jenkins  2.0  tracks:
    

    View Slide

46. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    46
    

    View Slide

47. sd
    ©  2015  CloudBees,   Inc.    All  Rights  Reserved
    @jenkins_spb
    Useful links
    • Jenkins project
    – Website: http://jenkins-ci.org
    – GitHub: http://github.com/jenkinsci/
    • CloudBees
    – Website: https://www.cloudbees.com
    – CloudBees Jenkins Platform
    o https://www.cloudbees.com/products/cloudbees-jenkins-platform
    47
    

    View Slide