Who is Mr. Jenkins? (English Version)

sd © 2015 CloudBees, Inc. All Rights Reserved
@jenkins_spb © 2015 CloudBees, Inc. All Rights Reserved Who is Mr. Jenkins? Current state, common usage issues and trends in the community Oleg Nenashev, CloudBees, Inc. St. Petersburg Jenkins Meetup #0 @jenkins_spb December 9th, 2015

@jenkins_spb > Whoami – Oleg Nenashev • Background: R&D, HW/Embedded SW • Jenkins OSS contributor – Core & Security teams member – Plugin Developer and Maintainer • Developer at CloudBees – CloudBees Jenkins Platform https://ru.linkedin.com/in/onenashev @oleg_nenashev Twitter: LinkedIn: GitHub: https://github.com/oleg-‐nenashev

@jenkins_spb © 2015 CloudBees, Inc. All Rights Reserved Outline 3 • Introduction. What is Jenkins? • Common Jenkins usage issues • Why does it happen? How to overcome the issues? • Jenkins trends (community changes, Jenkins 2.0)

@jenkins_spb Disclaimer • This presentation is not a training material • The talk represents the speaker’s personal opinion • This opinion may differ from the official position of CloudBees or Jenkins community 4

@jenkins_spb Jenkins 101 5 • 2005 – Hudson (Sun) • 2008 – It becomes popular, Duke's Choice Award • 2011 – Conflict with Oracle, renaming to Jenkins • License: MIT (core) • https://github.com/jenkinsci/

@jenkins_spb Jenkins is popular 6 http://stats.jenkins-‐ci.org/jenkins-‐stats/ • 1000k installations • ~300 active committers (6 months) • ~3 new plugins every week • ~30 plugin releases every week

@jenkins_spb Who is Mr. Jenkins? 7 Jenkins – Continuous Integration System Automation Framework

@jenkins_spb Why “automation”? 8 Continuous Integration Continuous Integration Continuous Delivery DevOps Accounting automation Home Automation CRON + WebUI

@jenkins_spb Why “framework”? 9 System Flexible and Extensible Minimal OOTB functionality Solves basic use-‐cases only Requires customization Jenkins is…

@jenkins_spb 10 Система Infra-‐ structure Docs Infra-‐ structure Plugins Configs Infra-‐ structure Templates Demos Jenkins-based Automation System

@jenkins_spb How does Jenkins grow? After setup 11

@jenkins_spb How does Jenkins grow? Small instance 12 Dozens of jobs Several nodes Small number of plugins Several users

@jenkins_spb How does Jenkins grow? It becomes popular 13 Hundreds of jobs Dozens of nodes Many plugins Dozens of users Complex configurations

@jenkins_spb 14 By NASA, ESA, J. Hester and A. Loll (Arizona State University) (HubbleSite: gallery, release.) [Public domain], via Wikimedia Commons. https://upload.wikimedia.org/wikipedia/commons/thumb/0/00/Crab_Nebula.jpg/512px-‐Crab_Nebula.jpg” There was Jenkins At some point… Ongoing projects have been affected as well

@jenkins_spb Why does Jenkins blow up? •Jenkins is scalable •Default Jenkins setups are not scalable •Scalability requires proper system design 15

@jenkins_spb Rule for admins #0. Design automation systems as an internal products 16 But there are issues in Jenkins as well…

@jenkins_spb User experience issues 17 Where are the docs? How to find a plugin? Is this plugin alive? How to configure THIS? Why do they ignore my issues? Where is the maintainer? Why the UI is so complicated? How to implement it?

@jenkins_spb Jenkins Learning Curve 18 Basic scripts, CRON with UI Automation flows based on plugins Advanced features, Custom Plugins

@jenkins_spb Jenkins weaknesses. Limited scalability 19 Node 1 Node 2 Node N . . . • Master handles the most of the build logic • Jobs on nodes ALWAYS load the master • Input/Output on master is also a bottleneck Multi-‐ tenancy ??? Gearman Plugin (OSS) CloudBees Jenkins Platform

@jenkins_spb Jenkins weaknesses. Growth Problems 20 Jenkins used to grow faster than its Dev community

@jenkins_spb Jenkins weaknesses. Compatibility costs 21 • More than 5 years of backward compatibility • Hudson 2 plugins still work on Jenkins! • BUT: • Architecture design issues, complex APIs • Outdated libs • Outdated UI • Plugin integration issues

@jenkins_spb The problems can be partially solved! 22

@jenkins_spb Rule for admins #1. Do not overload Masters! 23

@jenkins_spb Rule #1. Do not overload Masters! •Use labels to specify target nodes •or Job Restrictions Plugin to secure the master 24

@jenkins_spb Rule for admins #2. Test Jenkins changes before the deployment! 25

@jenkins_spb Test servers •Must-have for large-scale instances •Target: best possible coverage 26 New feature/plugin Bugfix Test server Production Showstoppers only Production mirror?

@jenkins_spb Rule for admins #3. Use LTS versions on production 27

@jenkins_spb Jenkins LTS releases • LTS are much more stable • Bugs backporting, 3 monthly patch releases 28

@jenkins_spb LTS Specifics •LTS releases still require testing – Potential regressions across major versions – SECURITY fixes cause “regressions” •3 months is not enough, what to do? – Custom-fork of Jenkins core [and plugins] – Enterprise-solutions based on Jenkins – Help community to increase coverage and to extend the support 29

@jenkins_spb Rule for admins #4 Memento Mori 30

@jenkins_spb Your Jenkins instance will die •Major risk: System and Infra failures •Updates may corrupt the system as well ⇒ALWAYS perform backups – Periodic Backup Plugin, Backup Plugin, … •Config History Plugin – configs backup •Naginator Plugin – failover of jobs 31

@jenkins_spb Job Config History Plugin – configuration backup 32 •Jenkins System Configs •Jobs •Nodes

@jenkins_spb Monitoring •Monitoring Plugin (Java Melody) •Reporting to external systems –Nagios, Zenoss, Zabbix, … •Centralized log collection & processing 33 Don’t forget about the infrastructure and backend -‐ they frequently fail !

@jenkins_spb Summary • Jenkins has real weaknesses • Jenkins is effective if you know how to solve them • Useful materials: – http://jenkins-ci.org - Blogs, Wiki, Bugtracker – Materials from Jenkins User Conferences: o https://www.cloudbees.com/company/events/juc – Google Search – Need help? o IRC: #jenkinsci o https://groups.google.com/forum/#!forum/jenkinsci-users 34

@jenkins_spb Automation Systems Evolvement Trends •SW area shifts focus from CI: – Continuous Delivery – DevOps •Many integrations => communities and OSS •Centralization of automation systems – High requirements and strict SLAs – Commercial solutions 35

@jenkins_spb Community reorganization •New teams focusing on hot areas •Community manager •More contributors (mostly plugin developers) •Jenkins Board re-elections 36 Board Core Team Security LTS Community INFRA Website

@jenkins_spb “Adopt a plugin!” 37 http://bit.ly/adoptplugin

@jenkins_spb Automation as Code 38 Job DSL DotCI* Workflow Jenkinsfile + Multi-‐branch *not available in the main UC

@jenkins_spb 39

@jenkins_spb Plans under discussion (not a final roadmap) 40 Jenkins 1.x “Jenkins 3.x” Jenkins 2.x Middle of 2016 End of 2016 (?) • Website • Focus on UX and project infra • Minimal compatibility changes • Major architectural changes • At least 1 LTS line in 2016

@jenkins_spb Jenkins 2.0. Website •Static web-site with sources on GitHub ⇒Improves contributions •Content rework – MORE structured info – Going beyond Continuous Integration 41 http://bit.ly/jenkins20

@jenkins_spb Jenkins 2.0. What’s in the core? •UX: – “Plugin gallery” – Web UI improvements (configs, etc.) •Update of WebUI libraries •API deprecation and deletion engine 42 http://bit.ly/jenkins20

@jenkins_spb Update Manager 2.0 43

@jenkins_spb «Jenkins 3.0» •Aim– architecture changes •Features list– TBD – Database storage for configs and build data (Andrew Bayer) – Remoting 3.0 (Oleg Nenashev) – … – <feel free to propose changes> 44

@jenkins_spb How to contribute? 45 http://bit.ly/jenkinscidev 1. GOTO Jenkins mailing list: 2. Search for Jenkins 2.0 tracks:

@jenkins_spb Useful links • Jenkins project – Website: http://jenkins-ci.org – GitHub: http://github.com/jenkinsci/ • CloudBees – Website: https://www.cloudbees.com – CloudBees Jenkins Platform o https://www.cloudbees.com/products/cloudbees-jenkins-platform 47

Who is Mr. Jenkins? (English Version)

Who is Mr. Jenkins? (English Version)

More Decks by Oleg Nenashev

Other Decks in Programming

Featured

Transcript