A New Patchwork: Bringing CI, patch tracking and more to the Mailing List

Mailing lists are the bedrock of many open source software projects, and have been so since the early days of the Internet. However, mailing lists can struggle to compete with code collaboration tools like Gerrit and Rietveld, many of which offer features such as integration with automated testing tools and patch tracking. How can such features be integrated into existing, mailing list-based projects like the Linux kernel or DPDK? The presenter reports on the ongoing work around the widely-deployed 'patchwork' tool to do just this.

Tracking of patches between development mailing lists and automated testing tools can be accomplished using patchwork. patchwork provides a web interface for patches submitted to mailing lists and presents them alongside any comments. patchwork also supports the tracking of the state of patches, be they Accepted, Rejected or Under Review. By design, patchwork is not intended to replace mailing lists but rather supplement them: these features are entirely optional, but where used can help ease the burden on maintainers by removing the need to perform tedious, time-consuming and low-value tasks like manual sorting. patchwork is already widely deployed for many projects, and instances can be found on kernel.org, dpdk.org and openembedded.org, to name but a few. There are also many projects, including Open vSwitch and QEMU, that can be found on ozlabs.org, which is maintained by the original author of patchwork.

This talk outlines the ongoing work to supplement development mailing lists with web-based workflows using patchwork. We demonstrate how features such as continuous integration support, automated patch and series tracking and a vastly expanded API help position the combination of the Mailing List and patchwork as a viable, feature-competitive alternative to tools such as Gerrit or Rietveld. We also detail some of the complexities of structuring the inherently unstructured data found on mailing lists. How do I identify a series of patches, for example? How do identify a reply to a patch? How do I identify a patch itself?

This talk be of particular interest to testers of open source, mailing list-based projects such as the Linux kernel, QEMU, DPDK or Open vSwitch. These users will be able take advantage of features like support for continuous integration servers and series tracking to develop automated, distributed testing infrastructures. Communities like OpenStack have shown that such infrastructures are critical for delivering software faster while reducing risk and maintenance cost. Such an infrastructure is currently being developed for the DPDK community, and it is easy to envision a similar roll-out for other projects with similar requirements.

It will also be of interest to the developers of these projects. These developers can already use patchwork as a tool to track patch backlogs, maintain personal TODO lists, or just browse submissions via a web UI/XML-RPC API. However, the new features provide these users with functionality that can help automate even more of the tedious yet necessary overhead tasks.

These slides were presented at FOSDEM 2016 [1]

[1] https://fosdem.org/2016/schedule/event/patchwork/

Transcript

1. View Slide

2. Stephen Finucane (@stephenfin)
   OpenStack Software Developer @ Intel
   stephen.finucane@intel.com
   github.com/stephenfin
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. state
   submitter
   reviews
   subject
   date delegate
   

   View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. View Slide

15. different types
    hyperlinked reports
    multiple providers
    

    View Slide

16. View Slide

17. View Slide

18. View Slide

19. View Slide

20. View Slide

21. Django 1.8 
    Python 3 
    PEP8 
    Django 1.9+ □
    Documentation □
    Code Coverage □
    

    View Slide

22. View Slide

23. View Slide

24. View Slide

25. [1/3] Do something
    [2/3] Do something else
    [3/3] Do final thing
    

    View Slide

26. View Slide

27. View Slide

28. [2/3] Do something else
    Message-Id:
    In-Reply-To:
    References: []
    

    View Slide

29. [1/3 v2] Do something
    [2/3 v2] Do something else
    [3/3 v2] Do final thing
    

    View Slide

30. [1/3] Do something
    From:
    To: ...
    

    View Slide

31. [1/3] Do something
    From:
    To: ...
    

    View Slide

32. [1/3 v2] Do sumin
    From:
    To: ...
    

    View Slide

33. View Slide

34. swagger: '2.0'
    info:
    title: Patchwork API
    description: |
    Patchwork is a web-based patch tracking system designed to
    facilitate the contribution and management of contributions
    to an open-source project.
    version: "1.0.0"
    schemes:
    - https
    basePath: /v1
    produces:
    - application/json
    paths:
    /:
    get:
    ...
    

    View Slide

35. View Slide

36. [
    {
    "days": [
    2,
    11,
    ...
    0
    ],
    "total": 8,
    "week": 1410652800
    },
    {
    ...
    }
    ]
    contributors
    additions, deletions
    patches per hour/day
    

    View Slide

37. View Slide

38. View Slide

39. View Slide

40. waffle.io/getpatchwork/patchwork
    github.com/getpatchwork/patchwork
    patchwork.ozlabs.org
    

    View Slide

41. Stephen Finucane (@stephenfin)
    OpenStack Software Developer @ Intel
    stephen.finucane@intel.com
    github.com/stephenfin
    

    View Slide

42. • Icons made by Freepik from www.flaticon.com are licensed under
    CC BY 3.0
    • Icons made by Dave Gandy from www.flaticon.com are licensed
    under CC BY 3.0
    

    View Slide