Using Jenkins Pipeline and Job DSL together

SPB Jenkins Meetup
https://www.meetup.com/St-Petersburg-Jenkins-Meetup/events/258988655/

Transcript

1. Using Jenkins Pipeline and Job DSL together Filippova Ekaterina ECS, Dell EMC
2. © Copyright 2019 Dell Inc. 2 Our plan for today • Intro • Job DSL & Pipeline DSL • Challenges & Requirements • Ideas & Solutions • Summary • Demo • Use cases deferred to the next time
3. © Copyright 2019 Dell Inc. 3 Intro
4. © Copyright 2019 Dell Inc. 4 Intro > whoami ECS Pipelines Team SPB Jenkins Meetup #10. Николай Румянцев. Наш опыт управления агентами Jenkins. Алексей Никитин и Фёдор Чемашкин. Инструменты разработки Jenkins Pipeline.
5. © Copyright 2019 Dell Inc. 5 Jenkins • CI/CD • Scripted pipelines • ~ 1500 jobs • +/- 220 nodes – mostly swarm agents • 8 components: 3 – 18 services • > 150 users • > 200 builds per day
6. © Copyright 2019 Dell Inc. 6 Job DSL & Pipeline DSL
7. © Copyright 2019 Dell Inc. 7 Job DSL • Job DSL plugin - https://plugins.jenkins.io/job-dsl • Jenkins Job DSL API - https://jenkinsci.github.io/job-dsl-plugin/ • Jenkins Job DSL Playground - https://job-dsl.herokuapp.com/ SPB Jenkins Meetup #6. Павел Щетько. Jenkins задачи как код.
8. © Copyright 2019 Dell Inc. 8 Pipeline DSL • Pipeline Syntax - https://jenkins.io/doc/book/pipeline/syntax/#scripted-pipeline
9. © Copyright 2019 Dell Inc. 9 Challenges & Requirements
10. © Copyright 2019 Dell Inc. 10 Once upon a time … Pipelines as code Detect and control pipelines’ limits “Feel free to contribute” (c)
11. © Copyright 2019 Dell Inc. 11 Yes, but … • tedious tasks • manual operations • testing and checking on regression • many questions: how it works? • permissions, schedule, enable/disable, retention policy • apply some new feature everywhere
12. © Copyright 2019 Dell Inc. 12 Ideas & Solutions
13. © Copyright 2019 Dell Inc. 13 How to combine Pipeline DSL and Job DSL: structure Job config parts: • Script path and entry point • Job parameters • Script flags • Job DSL configs Code parts: • Job config • Generator script • Pipeline script ⇓ • Generated Job DSL code ⇓ • Generated ‘ready to use’ Job
14. © Copyright 2019 Dell Inc. 14 How to combine Pipeline DSL and Job DSL: structure Job config parts: • Script path and entry point • Job parameters • Script flags • Job DSL configs Code parts: • Job config • Generator script • Pipeline script ⇓ • Generated Job DSL code ⇓ • Generated ‘ready to use’ Job time to see some code
15. © Copyright 2019 Dell Inc. 15 How to combine Pipeline DSL and Job DSL: rules • PIPELINE must know how to generate itself • All jobs must be defined in one place • Keep code and data separately • Different approaches for different code parts – Job config – any config file format: ini, yml, json – keep it in product repo – Generator script – Job DSL templates – Pipeline script – pure Pipeline DSL, Shared Libraries, Unit test, Linters
16. © Copyright 2019 Dell Inc. 16 Summary
17. © Copyright 2019 Dell Inc. 17 Results • Separated data from code • Facilitated pipeline development process for product developers • Got libraries which are easily tested and supported • Build some code design rules
18. © Copyright 2019 Dell Inc. 18 3 major objectives Pipelines as code - DONE Detect and control pipelines’ limits – IN PROGRESS Feel free to contribute – 80/20
19. © Copyright 2019 Dell Inc. 19 To Be Continued …