Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Yaml as Pipeline GSoC 2108 Phase 3 evaluation

Yaml as Pipeline GSoC 2108 Phase 3 evaluation

Presentation slides of GSoC Phase 3 evaluation of project Pipeline as YAML (Simple pull request job plugin).

Project page: https://jenkins.io/projects/gsoc/2018/simple-pull-request-job-plugin/

Abhishek Gautam

August 13, 2018
Tweet

More Decks by Abhishek Gautam

Other Decks in Education

Transcript

  1. 1

    View Slide

  2. Jenkins Pipeline as YAML
    Final Evaluation by: Abhishek Gautam
    13 August 2018
    .
    2

    View Slide

  3. ● I am a 3rd year Computer Science student from Visvesvaraya National
    Institute of Technology, Nagpur, India.
    ● I am a regular competitive programmer.
    ● I have done two internships as a Game Programmer as well.
    ● My interest in automation motivated my participation in the Jenkins GSOC
    2018 program
    .
    3
    Introduction

    View Slide

  4. ● Use-case: Jenkins Pipeline as YAML job definitions
    ○ Easy to use by non-experts
    ○ Minimal definition for common cases (Makefile, validated merge)
    ● Jenkins Pipeline is Groovy DSL
    ● Groovy DSL may be complex, even with Declarative Pipeline
    ● YAML definitions can be used to simplify.
    4
    Problem area
    https://jenkins.io/projects/gsoc/2018/simple-pull-request-job-plugin/

    View Slide

  5. 1. Travis YML Plugin: Runs “.travis.yml” as jenkins pipeline job. Does not
    support external pull request. Last commit for this plugin was on 14 Nov 2016.
    2. CodeShip Plugin: Converts codeship "steps.yaml" and "services.yaml" to
    scripted pipeline code. This plugin has never been released.
    3. Jenkins pipeline builder: This is a external non-Java-based tool, which
    cannot be easily converted to a Jenkins plugin.
    5
    Prior work

    View Slide

  6. ● Objective: New plugin for defining Jenkins jobs as YAML
    ● Links
    ○ GitHub: https://github.com/jenkinsci/simple-pull-request-job-plugin
    ○ Gitter Chat: https://gitter.im/jenkinsci/simple-pull-request-job-plugin
    ● Mentors:
    ○ Martin d'Anjou
    ○ Kristin Whetstone
    ○ Jeff Knurek
    ○ Oleg Nenashev (Organisation Admin)
    6
    My project

    View Slide

  7. 1. Configure job for pull requests using a YAML file.
    2. Interaction with Bitbucket Server, Bitbucket Cloud, Gitlab and Github.
    3. Detect the presence of certain types of reports at conventional location, and
    publish them. Non-conventional report locations can be configured in the
    YAML file.
    4. Publish build status.
    7
    Project Objectives

    View Slide

  8. 1. Build on the top of multibranch pipeline plugin, leveraging:
    a. Interface to show branch and pull requests build.
    b. Detection of trusted revisions in a repository.
    c. Publishing of build status to the repository.
    2. Convert the YAML configuration to declarative pipeline.
    a. Parse the yaml description extracting steps and configuration values.
    b. Declarative step lookup handled by the CasC plugin.
    8
    Design

    View Slide

  9. 1. Use a Jenkinsfile.yaml for build.
    2. Git Push step (Users do not call git push explicitly)
    3. Build and tests performed by user defined script called from Jenkinsfile.yaml
    4. Agent configuration in “Jenkinsfile.yaml”
    5. harvest results and reports.
    a. junit()
    b. findbugs()
    c. archiveArtifacts section
    6. Basic interface to parse and get build specifications from YAML file.
    Phase 1 blog post: https://jenkins.io/blog/2018/06/15/simple-pull-request-plugin/
    9
    Coding phase 1 features

    View Slide

  10. 1. Step Configurator (Based on JCasC plugin)
    a. Limitations:
    i. Difficulty with Enums (will be resolved in next alpha release of JCasC plugin, PR 346)
    ii. Difficulty with “JiraTestDataPublisher.class” and may be other classes (JENKINS-52444)
    2. A format for “Jenkinsfile.yaml”. (All details present in Blog Post)
    3. Tools property in agent section of Jenkinsfile.yaml.
    4. Some tests for the plugin. (PR 14)
    Code improvements:
    ● Manual indentation removed from PipelineSnippetGenerator class.
    10
    Coding phase 2 features

    View Slide

  11. 11
    Step Code Generation
    junit:
    testResults: "target/**.xml"
    allowEmptyResults: true
    testDataPublishers:
    - AutomateTestDataPublisher
    - JunitResultPublisher:
    urlOverride: "urlOverride"
    junit allowEmptyResults: true, testDataPublishers:
    [[$class: 'AutomateTestDataPublisher'], [$class:
    'JunitResultPublisher', urlOverride: 'urlOverride']],
    testResults: 'target/**.xml'

    View Slide

  12. 1. Declarative pipeline code generator detach to extensions.
    2. More unit tests were written.
    12
    Work done in Phase 3
    https://github.com/gautamabhishek46/dummy

    View Slide

  13. https://github.com/gautamabhishek46/dummy/blob/master/Jenkinsfile.yaml
    13
    Example YAML
    https://github.com/gautamabhishek46/dummy

    View Slide

  14. https://jenkins.io/blog/2018/06/15/simple-pull-request-plugin/#how-to-run-the-plugin
    14
    Plugin Configuration

    View Slide

  15. Demo
    15

    View Slide

  16. 1. Release 1.0 version (JENKINS-52519)
    2. Test the integration with various platforms Bitbucket, Gitlab, Github.
    3. Support for webhooks (JENKINS-51941)
    4. Check if trusted people have approved a pull request and start build
    accordingly (JENKINS-52517)
    5. Automatic Workspace Cleanup when PR is closed (JENKINS-51897)
    6. Support the “when” Declarative Pipeline directive (JENKINS-52520)
    7. Nice2have: Support hierarchical report types (JENKINS-52521)
    8. Acceptance Test Harness tests (JENKINS-52496)
    16
    Future Tasks
    Phase 3 EPIC: https://issues.jenkins-ci.org/browse/JENKINS-52452

    View Slide

  17. Gitter chat: https://gitter.im/jenkinsci/simple-pull-request-job-plugin
    Jenkins blog post: (Not ready till now)
    Github project: https://github.com/jenkinsci/simple-pull-request-job-plugin
    Demo project: https://github.com/gautamabhishek46/dummy
    Phase 1 presentation slides:
    https://docs.google.com/presentation/d/1R63XK4Nmd5EBmMdF2rkevV1iZxjTom1
    9XFHdgjY9qeA/edit?usp=sharing
    Phase 2 presentation video:
    https://www.youtube.com/watch?v=tuTODhJOTBU&t=3229s
    17
    Links

    View Slide

  18. Phase 3 presentation video: https://www.youtube.com/watch?v=GGEtN4nbtng
    18
    Links

    View Slide

  19. Thank You
    19

    View Slide