Mailing List, Meet CI

What does it take to implement continuous integration-style automated testing into a mailing list-driven software project? Not a lot, actually. In this talk, we demonstrate how a simple but easily scaled testing system can be implemented for a such a project through the combination of Patchwork, the web-based patch tracking system, and open source CI tools such as Jenkins.

At FOSDEM 2016, developers working on Patchwork, the web-based patch tracking system, demonstrated some of the ongoing work in Patchwork. This work ranged from UI improvements to new features and APIs but, collectively, it had the goal of enabling automated testing functionality for software projects developed via mailing lists. The Patchwork developers have been busy since then and the application, in widespread use since 2008, recently hit the 2.0 milestone, marking this functionality as complete.

Projects such as the Dataplane Development Kit (DPDK) have quickly adopted the features that 2.0 brings, using them to enable real time, automated testing of patches sent to the mailing list. This automated testing provides a mechanism for developers to not only sidestep the more perfunctory of tasks, such as coding standard checks, but also to test changes in environments that they may not have at their disposal, such as differing hardware or OS configurations. As seen in projects pairing open source code collaboration tools like Gerrit or Rietveld with CI systems such as Jenkins or BuildBot, this continuous testing can provide huge improvements in developer velocity.

In this talk, we demonstrate how to build a basic "checkstyle" testing system through a combination of Patchwork and an off-the-shelf, open source CI system. This system will retrieve patches and dependencies, apply and test them, and report the results back to both Patchwork and a separate mailing list. This configuration demonstrates the best of Patchwork's new features, and can be easily extended to cover far more complex testing scenarios.

These slides were presented at FOSDEM 2017 [1]

[1] https://fosdem.org/2017/schedule/event/patchwork_jenkins/

Transcript

1. View Slide

2. stephen@that.guru
   github.com/stephenfin
   Stephen Finucane (@stephenfin)
   OpenStack Software Developer @ Red Hat
   

   View Slide

3. View Slide

4. Houston, we have a problem.
   1
   

   View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. Patchwork, you say?
   2
   

   View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. “How It’s Made”
    3
    

    View Slide

15. View Slide

16. View Slide

17. View Slide

18. This is still WIP!
    

    View Slide

19. View Slide

20. This is still WIP!
    

    View Slide

21. But how can we use it?
    4
    

    View Slide

22. A Patchwork
    instance, a Jenkins
    instance, and a
    helpful little Script
    walk into a bar...
    0
    

    View Slide

23. 1
    Patchwork receives a
    new patch from the
    mailing list.
    Patchwork raises a
    new 'patch-complete'
    event*.
    1
    * Along with many other events
    

    View Slide

24. 1
    2
    Script polls '/events',
    sees new event and
    builds a job request.
    2
    

    View Slide

25. 1
    2
    3
    4
    Jenkins receives new
    job.
    Jenkins submits a
    check with 'pending'
    status to Patchwork.
    4
    3 +
    

    View Slide

26. 1
    2
    3
    4
    5
    Jenkins downloads
    the patch along with
    any dependencies
    required.
    Jenkins runs tests.
    5
    

    View Slide

27. 1
    2
    3
    4
    5 6
    7
    Jenkins completes all
    builds and/or tests.
    Jenkins submits a
    check with 'success',
    'fail' or 'warning'
    status to Patchwork.
    7
    6 +
    

    View Slide

28. What's next?
    5
    

    View Slide

29. View Slide

30. View Slide

31. View Slide

32. View Slide

33. patchwork@lists.ozlabs.org
    github.com/getpatchwork/patchwork
    patchwork.ozlabs.org
    

    View Slide

34. stephen@that.guru
    github.com/stephenfin
    Stephen Finucane (@stephenfin)
    OpenStack Software Developer @ Red Hat
    

    View Slide

35. Credits
    • Icons made by Freepik from www.flaticon.com are licensed under CC
    BY 3.0
    • Icons made by Dave Gandy from www.flaticon.com are licensed
    under CC BY 3.0
    

    View Slide