Upgrade to Pro — share decks privately, control downloads, hide ads and more …

GitHub Universe 2019 Don't Be a Robot; Automate Your Workflows at Scale

6b6e72d297aa0270654a0d4575f1287e?s=47 Mariatta
November 14, 2019

GitHub Universe 2019 Don't Be a Robot; Automate Your Workflows at Scale

Managing a large open source project like CPython is no easy task. Learn how the Python core team automated their GitHub workflow with bots, making it easier for maintainers and contributors to collaborate together. Even if you’re not managing a large project, you can still build your own bot! Hear some ideas on what you can automate and personalize on GitHub to build at your best. Don’t be a robot, build one.

6b6e72d297aa0270654a0d4575f1287e?s=128

Mariatta

November 14, 2019
Tweet

Transcript

  1. Don’t Be a Robot; Automating Workflows at Scale @mariatta

  2. Python Core Developer Backend Engineer CPython’s emoji and core-workflow expert

    @mariatta mariatta.ca #IceCreamSelfie #BotsForHumanity WHO AM I? Mariatta ! GitHub Universe 2019
 @mariatta https://zapier.com/jobs/
  3. Problems? CORE PYTHON DEVELOPMENT GitHub Universe 2019
 @mariatta

  4. Problem: We’re outnumbered Source: Stephane Wirtel, CPython loves your pull

    request https://speakerdeck.com/matrixise/ python-loves-your-contributions GitHub Universe 2019
 @mariatta
  5. Problem: Complicated Workflow Bug or feature? CLA signed? Tests passed?

    Needs an issue? Needs news entry? News entry added? Needs backport? How? Who? When? First time contributor? Commit message? GitHub Universe 2019
 @mariatta
  6. Problem: Complicated Workflow Bug or feature? CLA signed? Tests passed?

    Needs an issue? Needs news entry? News entry added? Needs backport? How? Who? When? First time contributor? Commit message? GitHub Universe 2019
 @mariatta
  7. Problem: Complicated Workflow Bug or feature? CLA signed? Tests passed?

    Needs an issue? Needs news entry? News entry added? Needs backport? How? Who? When? First time contributor? Commit message? GitHub Universe 2019
 @mariatta
  8. Backport? GitHub Universe 2019
 @mariatta

  9. Backport? Applying changes from newer version to older version GitHub

    Universe 2019
 @mariatta
  10. • master (3.9.0a1, due Nov 18, 2019) • 3.8 (3.8.1,

    due Dec 2019) • 3.7 (3.7.6, due Dec 2019) • 2.7 (EOL, January 2020) Current maintained versions GitHub Universe 2019
 @mariatta
  11. SCENARIO: BUG IN 3.6 Bug in f-strings GitHub Universe 2019


    @mariatta
  12. SCENARIO: BUG IN 3.6 Bug in f-strings 3.7 3.8 3.9

    GitHub Universe 2019
 @mariatta
  13. SCENARIO: BUG IN 3.6 Bug in f-strings 3.9 3.7 3.8

    3.9 GitHub Universe 2019
 @mariatta
  14. SCENARIO: BUG IN 3.6 Bug in f-strings 3.9 3.7 3.8

    3.9 GitHub Universe 2019
 @mariatta
  15. SCENARIO: BUG IN 3.6 Bug in f-strings 3.9 3.7 3.8

    3.9 3.7 3.8 GitHub Universe 2019
 @mariatta
  16. SCENARIO: BUG IN 3.6 Bug in f-strings 3.9 3.7 3.8

    3.9 3.7 3.8 GitHub Universe 2019
 @mariatta
  17. CHERRY-PICKING $ git fetch upstream $ git checkout -b backport-3384d38d-3.8

    upstream/3.8 $ git cherry-pick -x 3384d38d51a2c3450e742175db5d6d638fa5d2eb $ git push origin backport-3384d38d-3.8 $ git checkout master $ git branch -D backport-3384d38d-3.8 3.8 GitHub Universe 2019
 @mariatta
  18. CHERRY_PICKER.PY $ python3 -m pip install cherry_picker GitHub Universe 2019


    @mariatta
  19. CHERRY_PICKER.PY $ python3 -m pip install cherry_picker $ cherry_picker 3384d38d51a2c3450e742175db5d6d638fa5d2eb

    3.8 3.7 GitHub Universe 2019
 @mariatta
  20. Problems: GitHub Universe 2019
 @mariatta

  21. Problems: GitHub Universe 2019
 @mariatta

  22. Problems: GitHub Universe 2019
 @mariatta

  23. Problems: GitHub Universe 2019
 @mariatta

  24. Problems: GitHub Universe 2019
 @mariatta

  25. DON’T BE A ROBOT Build the Bot GitHub Universe 2019


    @mariatta
  26. GITHUB WEBHOOKS Pull request (opened, closed, labeled) Status (pending, success,

    failure) Pull request review (submitted, dismissed) … Events GitHub Webservice Docs: https://developer.github.com/webhooks GitHub Universe 2019
 @mariatta
  27. GITHUB APIS Create a pull request Merge a pull request

    Comment on a pull request Apply labels to pull request … APIs Webservice GitHub Docs: https://developer.github.com/v3 GitHub Universe 2019
 @mariatta
  28. PR merged event (webhook) Open pull request (REST API) Run

    cherry_picker.py GitHub Universe 2019
 @mariatta
  29. PR merged event (webhook) Open pull request (REST API) Run

    cherry_picker.py Clone CPython repo GitHub Universe 2019
 @mariatta
  30. PR merged event (webhook) Open pull request (REST API) Run

    cherry_picker.py Clone CPython repo 2 minutes 12-15 seconds GitHub Universe 2019
 @mariatta
  31. PR merged event (webhook) Open pull request (REST API) Run

    cherry_picker.py Clone CPython repo 2 minutes 12-15 seconds timesout after 30 seconds GitHub Universe 2019
 @mariatta
  32. PR merged event (webhook) Open pull request (REST API) Run

    cherry_picker.py Clone CPython repo 2 minutes 12-15 seconds timesout after 30 seconds GitHub Universe 2019
 @mariatta
  33. PR merged event (webhook) Open pull request (REST API) Run

    cherry_picker.py Clone CPython repo Webservice (aiohttp) Background task (celery) Start a task GitHub Universe 2019
 @mariatta
  34. PR merged event (webhook) Open pull request (REST API) Run

    cherry_picker.py Clone CPython repo Webservice (aiohttp) Background task (celery) miss-islington Start a task GitHub Universe 2019
 @mariatta
  35. miss-islington • open-source https://github.com/python/miss-islington • Python 3.6 (because f-strings) •

    aiohttp: async web server/client • gidgethub: async Python library for GitHub GitHub Universe 2019
 @mariatta
  36. miss-islington @router.register("pull_request", action="closed") async def backport_pr(event, gh, *args, **kwargs): if

    event.data["pull_request"]["merged"]: commit_hash = event.data["pull_request"]["merge_commit_sha"] ... branches = [ label["name"].split()[-1] for label in pr_labels if label["name"].startswith("needs backport to”) ] for branch in branches: ... tasks.backport_task.delay(commit_hash, branch, ...) source: https://github.com/python/miss-islington/blob/master/miss_islington/backport_pr.py GitHub Universe 2019
 @mariatta
  37. First PR https://twitter.com/ChristianHeimes/status/905218281937575936 GitHub Universe 2019
 @mariatta

  38. Problems: GitHub Universe 2019
 @mariatta

  39. Problems: ✅ ✅ ✅ ✅ GitHub Universe 2019
 @mariatta

  40. Problems: ✅ ✅ ✅ ✅ GitHub Universe 2019
 @mariatta

  41. Problem: Complicated Workflow Bug or feature? CLA signed? Tests passed?

    Needs an issue? Needs news entry? News entry added? Needs backport? How? Who? When? First time contributor? Commit message? GitHub Universe 2019
 @mariatta
  42. GitHub Universe 2019
 @mariatta

  43. GitHub Universe 2019
 @mariatta

  44. GitHub Universe 2019
 @mariatta

  45. PR status event (webhook) Get the combined status for a

    ref (REST API) Post a comment in PR (REST API) Search PR containing commit (REST API) 1 (status of a Git commit changes) 2 3 4 2. https://developer.github.com/v3/repos/statuses/#get-the-combined-status-for-a-specific-ref 1. https://developer.github.com/v3/activity/events/types/#statusevent 4. https://developer.github.com/v3/issues/comments/#create-a-comment 3. https://developer.github.com/v3/search/#search-issues GitHub Universe 2019
 @mariatta
  46. GitHub Universe 2019
 @mariatta

  47. ✅ ✅ ✅ ✅ ✅ GitHub Universe 2019
 @mariatta

  48. Problem: Complicated Workflow Bug or feature? CLA signed? Tests passed?

    Needs an issue? Needs news entry? News entry added? Needs backport? How? Who? When? First time contributor? Commit message? GitHub Universe 2019
 @mariatta
  49. https://mail.python.org/pipermail/python-committers/2018-January/005174.html GitHub Universe 2019
 @mariatta

  50. Merge the PR pr_number = 5590 await gh.put(f"/repos/python/cpython/pulls/{pr_number}/merge", data={"commit_title": "Fix

    typo in f-strings docs", "commit_message": "Shorten the comment to: 'using integer format specifier'"), "sha": sha, "merge_method": "squash" } ) GitHub Universe 2019
 @mariatta
  51. Merge the PR pr_number = 5590 await gh.put(f"/repos/python/cpython/pulls/{pr_number}/merge", data={"commit_title": "Fix

    typo in f-strings docs", "commit_message": "Shorten the comment to: 'using integer format specifier'"), "sha": sha, "merge_method": "squash" } ) GitHub Universe 2019
 @mariatta
  52. GitHub Universe 2019
 @mariatta

  53. GitHub Universe 2019
 @mariatta

  54. GitHub Universe 2019
 @mariatta

  55. Don’t Be a Robot BUILD THE BOT GitHub Universe 2019


    @mariatta
  56. black-out Bot that formats code with black GitHub Universe 2019


    @mariatta
  57. Close all PRs GitHub Universe 2019
 @mariatta

  58. Close all PRs PR opened/reopened event Close pull request GitHub

    Universe 2019
 @mariatta
  59. Close all PRs PR opened/reopened event Close pull request @router.register("pull_request",

    action="opened") @router.register("pull_request", action="reopened") async def close_pr(event, gh, *args, **kwargs): await gh.patch(event.data["pull_request"]["url"], data= {"state": "closed"} ) Source: https://github.com/Mariatta/close-all-pr/blob/master/close-all-pr/close_pr.py GitHub Universe 2019
 @mariatta
  60. OOOS bot Out of open source auto reply bot GitHub

    Universe 2019
 @mariatta
  61. RHYMES WITH HAPPIER! OOOS bot Out of open source auto

    reply bot GitHub Universe 2019
 @mariatta
  62. Greet your contributors GitHub Universe 2019
 @mariatta Source code: https://say-it-with-bots-gh-universe.readthedocs.io/

  63. GitHub Universe 2019
 @mariatta Don’t Be a Robot; Automate your

    workflow https://say-it-with-bots-gh-universe.readthedocs.io/
  64. GitHub Universe 2019
 @mariatta Don’t Be a Robot; Automate your

    workflow https://say-it-with-bots-gh-universe.readthedocs.io/
  65. Contact mariatta@python.org
 Follow on twitter @mariatta Sponsor on GitHub https://github.com/sponsors/mariatta

    GitHub Universe 2019
 @mariatta Thanks!