Mailing List, Meet CI

What does it take to implement continuous integration-style automated testing into a mailing list-driven software project? Not a lot, actually. In this talk, we demonstrate how a simple but easily scaled testing system can be implemented for a such a project through the combination of Patchwork, the web-based patch tracking system, and open source CI tools such as Jenkins.

At FOSDEM 2016, developers working on Patchwork, the web-based patch tracking system, demonstrated some of the ongoing work in Patchwork. This work ranged from UI improvements to new features and APIs but, collectively, it had the goal of enabling automated testing functionality for software projects developed via mailing lists. The Patchwork developers have been busy since then and the application, in widespread use since 2008, recently hit the 2.0 milestone, marking this functionality as complete.

Projects such as the Dataplane Development Kit (DPDK) have quickly adopted the features that 2.0 brings, using them to enable real time, automated testing of patches sent to the mailing list. This automated testing provides a mechanism for developers to not only sidestep the more perfunctory of tasks, such as coding standard checks, but also to test changes in environments that they may not have at their disposal, such as differing hardware or OS configurations. As seen in projects pairing open source code collaboration tools like Gerrit or Rietveld with CI systems such as Jenkins or BuildBot, this continuous testing can provide huge improvements in developer velocity.

In this talk, we demonstrate how to build a basic "checkstyle" testing system through a combination of Patchwork and an off-the-shelf, open source CI system. This system will retrieve patches and dependencies, apply and test them, and report the results back to both Patchwork and a separate mailing list. This configuration demonstrates the best of Patchwork's new features, and can be easily extended to cover far more complex testing scenarios.

These slides were presented at FOSDEM 2017 [1]

[1] https://fosdem.org/2017/schedule/event/patchwork_jenkins/

Transcript

2. stephen@that.guru github.com/stephenfin Stephen Finucane (@stephenfin) OpenStack Software Developer @ Red Hat
4. Houston, we have a problem. 1
9. Patchwork, you say? 2
14. “How It’s Made” 3
18. This is still WIP!
20. This is still WIP!
21. But how can we use it? 4
22. A Patchwork instance, a Jenkins instance, and a helpful little Script walk into a bar... 0
23. 1 Patchwork receives a new patch from the mailing list. Patchwork raises a new 'patch-complete' event*. 1 * Along with many other events
24. 1 2 Script polls '/events', sees new event and builds a job request. 2
25. 1 2 3 4 Jenkins receives new job. Jenkins submits a check with 'pending' status to Patchwork. 4 3 +
26. 1 2 3 4 5 Jenkins downloads the patch along with any dependencies required. Jenkins runs tests. 5
27. 1 2 3 4 5 6 7 Jenkins completes all builds and/or tests. Jenkins submits a check with 'success', 'fail' or 'warning' status to Patchwork. 7 6 +
28. What's next? 5
33. patchwork@lists.ozlabs.org github.com/getpatchwork/patchwork patchwork.ozlabs.org
34. stephen@that.guru github.com/stephenfin Stephen Finucane (@stephenfin) OpenStack Software Developer @ Red Hat
35. Credits • Icons made by Freepik from www.flaticon.com are licensed under CC BY 3.0 • Icons made by Dave Gandy from www.flaticon.com are licensed under CC BY 3.0