Yaml as Pipeline GSoC 218 Phase 2 evaluation

Presentation slides of GSoC Phase 2 evaluation of project Pipeline as YAML (Simple pull request job plugin).

Project page: https://jenkins.io/projects/gsoc/2018/simple-pull-request-job-plugin/

Transcript

2. Jenkins Pipeline as YAML Second Evaluation by: Abhishek Gautam 12 July 2018 . 2
3. • I am a 3rd year Computer Science student from Visvesvaraya National Institute of Technology, Nagpur, India. • I am a regular competitive programmer. • I have done two internships as a Game Programmer as well. • My interest in automation motivated my participation in the Jenkins GSOC 2018 program . 3 Introduction
4. • Use-case: Simple pull-request job definitions ◦ Easy to use by non-experts ◦ Minimal definition for common cases (Makefile, validated merge) • Jenkins Pipeline is Groovy DSL • Groovy DSL may be complex, even with Declarative Pipeline • YAML definitions can be used to simplify. 4 Problem area https://jenkins.io/projects/gsoc/2018/simple-pull-request-job-plugin/
5. 1. Travis YML Plugin: Runs “.travis.yml” as jenkins pipeline job. Does not support external pull request. Last commit for this plugin was on 14 Nov 2016. 2. CodeShip Plugin: Converts codeship "steps.yaml" and "services.yaml" to scripted pipeline code. This plugin has never been released. 3. Jenkins pipeline builder: This is a external non-Java-based tool, which cannot be easily converted to a Jenkins plugin. 5 Prior work
6. • Objective: New plugin for defining Jenkins jobs as YAML • Links ◦ GitHub: https://github.com/jenkinsci/simple-pull-request-job-plugin ◦ Gitter Chat: https://gitter.im/jenkinsci/simple-pull-request-job-plugin • Mentors: ◦ Martin d'Anjou ◦ Kristin Whetstone ◦ Jeff Knurek ◦ Oleg Nenashev (Organisation Admin) 6 My project
7. 1. Configure job for pull requests using a YAML file. 2. Interaction with Bitbucket Server, Bitbucket Cloud, Gitlab and Github. 3. Detect the presence of certain types of reports at conventional location, and publish them. Non-conventional report locations can be configured in the YAML file. 4. Publish build status. 7 Project Objectives
8. 1. Build on the top of multibranch pipeline plugin, leveraging: a. Interface to show branch and pull requests build. b. Detection of trusted revisions in a repository. c. Publishing of build status to the repository. 2. Convert the YAML configuration to declarative pipeline. a. Parse the yaml description extracting steps and configuration values. b. Declarative step lookup handled by the CasC plugin. 8 Design
9. Example: Path provided: “./scripts/hello” a. On UNIX machine “./scripts/hello.sh” will be called b. On non-UNIX machine “./scripts/hello.bat” will be called. 9 Design (Specifying scripts to run)
10. 1. Use a Jenkinsfile.yaml for build. 2. Git Push step (Users do not call git push explicitly) 3. Build and tests performed by user defined script called from Jenkinsfile.yaml 4. Agent configuration in “Jenkinsfile.yaml” 5. harvest results and reports. a. junit() b. findbugs() c. archiveArtifacts section 6. Basic interface to parse and get build specifications from YAML file. Phase 1 blog post: https://jenkins.io/blog/2018/06/15/simple-pull-request-plugin/ 10 Coding phase 1 features
11. 1. Step Configurator (Based on JCasC plugin) a. Limitations: i. Difficulty with Enums (will be resolved in next alpha release of JCasC plugin, PR 346) ii. Difficulty with “JiraTestDataPublisher.class” and may be other classes (JENKINS-52444) 2. A format for “Jenkinsfile.yaml”. (All details present in Blog Post) 3. Tools property in agent section of Jenkinsfile.yaml. 4. Some tests for the plugin. (PR 14) Code improvements: • Manual indentation removed from PipelineSnippetGenerator class. 11 Current features
12. 12 Step Code Generation junit: testResults: "target/**.xml" allowEmptyResults: true testDataPublishers: - AutomateTestDataPublisher - JunitResultPublisher: urlOverride: "urlOverride" junit allowEmptyResults: true, testDataPublishers: [[$class: 'AutomateTestDataPublisher'], [$class: 'JunitResultPublisher', urlOverride: 'urlOverride']], testResults: 'target/**.xml'
13. https://github.com/gautamabhishek46/dummy/blob/master/Jenkinsfile.yaml 13 Example YAML https://github.com/gautamabhishek46/dummy
14. https://jenkins.io/blog/2018/06/15/simple-pull-request-plugin/#how-to-run-the-plugin 14 Plugin Configuration
15. Demo 15
16. 1. Test Multi-Branch Pipeline features support: a. Support for webhooks (JENKINS-51941) b. Check if trusted people have approved a pull request and start build accordingly (JENKINS-52517) 2. Finalize documentation (JENKINS-52518) 3. Release 1.0 (JENKINS-52519) 4. Plugin overview blogpost 16 Plan for Phase 3 Phase 3 EPIC: https://issues.jenkins-ci.org/browse/JENKINS-52452
17. 1. Support the “when” Declarative Pipeline directive (JENKINS-52520) 2. Nice2have: Support hierarchical report types (JENKINS-52521) 3. Add unit tests, JenkinsRule tests, and ATH tests (JENKINS-52495, JENKINS-52496) 4. Automatic Workspace Cleanup when PR is closed (JENKINS-51897) 5. Refactor snippet generator to extensions (JENKINS-52491) 17 Plan for Phase 3 - after the release Phase 3 EPIC: https://issues.jenkins-ci.org/browse/JENKINS-52452
18. Gitter chat: https://gitter.im/jenkinsci/simple-pull-request-job-plugin Jenkins blog post: (Not ready till now) Github project: https://github.com/jenkinsci/simple-pull-request-job-plugin Demo project: https://github.com/gautamabhishek46/dummy Phase 1 presentation slides: https://docs.google.com/presentation/d/1R63XK4Nmd5EBmMdF2rkevV1iZxjTom1 9XFHdgjY9qeA/edit?usp=sharing Phase 2 presentation video: https://www.youtube.com/watch?v=tuTODhJOTBU&t=3229s 18 Links
19. Thank You 19