External Workspace Manager Plugin - GSoC midterm evaluation

This is the presentation about External Workspace Manager Plugin for the Google Summer of Code 2016 midterm evaluation.
You can watch it here: https://youtu.be/u4zhxfUT8P4?t=22m7s

Transcript

1. Mentors Student Oleg Nenashev Alexandru Somai Martin d'Anjou External Workspace Manager Plugin 1 Google Summer of Code 2016 @ Jenkins Project 23 June 2016
2. About myself alexsomai @alex_somai 2 • GSoC Student at Jenkins project • Following a major in Software Engineering at Babes-Bolyai University of Cluj-Napoca, Romania
3. Background • >2 years experience in Software Development • Coding in Java, Groovy, JavaScript • Interested in Spring, Hibernate, Web Services • Build tools and CI: Maven, Jenkins 3 alexsomai @alex_somai
4. The problem • Difficult to reuse the same workspace for multiple jobs • e.g. Running parallel testing across nodes 4
5. Possible solutions • Stash/unstash pre-made artifacts (pipeline only) • Copy Artifacts plugin • Advanced job setting ◦ custom workspace (freestyle) ◦ ws (pipeline) 5
6. About the project • Jenkins plugin • Focus on Pipeline jobs • Share and reuse workspaces across multiple jobs ◦ No need to copy, archive or move files 6
7. 7 Concept
8. 8 Infrastructure
9. 9 Jenkins global config Disk 1 Disk 2
10. 10 Jenkins global config Node 1 config
11. 11 Node 2 config Jenkins global config
12. 12 Disk allocation strategy & path computation • Disk allocation strategy ◦ Disk with the most usable space • Path computation ◦ /mountPoint/pathOnDisk/$JOB_NAME/$BUILD_NUMBER • Subject for extensibility JENKINS-36073
13. Example 1. Usage in a single job def extWorkspace = exwsAllocate diskPoolId: 'diskpool1' node ('linux') { exws (extWorkspace) { scm checkout sh 'mvn clean install -DskipTests' } } node ('test') { exws (extWorkspace) { sh 'mvn test' } } 13
14. 14 Example 2. Upstream/Downstream
15. def extWorkspace = exwsAllocate diskPoolId: 'diskpool1' node ('linux') { exws (extWorkspace) { scm checkout sh 'mvn clean install -DskipTests' } } 15 Example 2. Upstream job • No changes in External Workspace commands!
16. 16 Example 2. Downstream job def extWorkspace = exwsAllocate upstream: 'job-full-name' node ('test') { exws (extWorkspace) { sh 'mvn test' } } • Identify upstream job by its full name • Uses the workspace from the last stable build
17. Demo 17
18. Completed tasks • Plugin Design Document • Jenkins global configs & Node property • Implement exwsAllocate & exws steps • Write unit tests • Workspace allocation summary UI • Draft user documentation • Proof of Concept by June 20th • Alpha version release 18 Planned Achieved
19. Community interaction • Set up public meetings on Jenkins hangout ◦ https://jenkins.io/hangout ◦ Every Monday at 12 PM UTC ◦ Every Thursday at 5 PM UTC • Status updates on dev mailing list • Discussions in IRC and Gitter ◦ https://gitter.im/jenkinsci/external-workspace-manager-plugin • Project intro blog post ◦ https://jenkins.io/blog/2016/05/23/external-workspace-manager-plugin/ 19
20. Next steps • Get in touch with potential users • Alpha testing • Bug scrub • Task scrub 20 Backlog EPIC: JENKINS-35971
21. Plans. Hot Topics • Automatic workspace cleanup JENKINS-35976 • Integration with external disk providers JENKINS-36072 ◦ At least one Extension point in the plugin ◦ Candidates: Amazon EBS, Google Cloud Storage • Extensibility via Extension Points JENKINS-36073 ◦ DiskRestriction - restrict to particular jobs/nodes ◦ DiskInfoProvider - disk space calculation, etc. ◦ DiskAllocationStrategy - intelligent disk selection 21
22. Plans. Automation • More functional tests JENKINS-36077 ◦ Real-world test cases ◦ Configuration Roundtrips • Acceptance Test Harness JENKINS-36076 • Build flow - Add Jenkinsfile in plugin repo INFRA-797 22
23. Links • Repository ◦ https://github.com/jenkinsci/external-workspace-manager-plugin • Gitter ◦ https://gitter.im/jenkinsci/external-workspace-manager-plugin • Project intro blog post ◦ https://jenkins.io/blog/2016/05/23/external-workspace-manager-plugin/ • Design Document ◦ https://docs.google.com/document/d/1yiisnsR7qg3XEEvch8vocWbitSU CZcoQ-pfzEVFg1eA/edit?usp=sharing • Phase 1 & 2 EPICs JENKINS-35019 JENKINS-35971 23
24. Testing & Feedback • The plugin is ready for testing! • Usage guidelines on the plugin README page • Feedback is welcome on Jira and Gitter 24
25. Q&A 25 https://gitter.im/jenkinsci/external-workspace-manager-plugin
26. 26 Architecture
27. Upstream location Now: • Last stable build from the upstream job Future JENKINS-36152: • CauseAction in build • EXWSAllocateAction in upstream job 27