Oh No! The Robots Have Taken Over

Oh No! The Robots Have Taken Over

As part of his work, Chris and his team at Google are responsible for managing libraries for 150 APIs across 7 languages. This results in a crushing amount of toil, making it hard at times to both make forward progress and maintain what has been created.

When you own over 70 node repositories you have to get creative. So the team decided to make an army of sorts. An army of screaming, free roaming robots.

Bots can have a freeing effect on your engineering team. Come to hear how embracing automation has let a team of engineers do what they do best and let the toil fall to the machines.

Presented at Node+JS Interactive 2019: https://sched.co/T5HD

5d95f89b9892e12c0d7fa2f671edac62?s=128

Christopher Wilcox

December 11, 2019
Tweet

Transcript

  1. @chriswilcox47 https://chriswilcox.dev Oh No! The Robots Have Taken Over! Chris

    Wilcox Engineer, Google Cloud @chriswilcox47 https://chriswilcox.dev
  2. @chriswilcox47 https://chriswilcox.dev What’s a bot anyway?

  3. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Photo by Lenny Kuhne on Unsplash

  4. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev

  5. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev

  6. @chriswilcox47 https://chriswilcox.dev What are bots good at?

  7. @chriswilcox47 https://chriswilcox.dev

  8. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Levels of automation

  9. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev

  10. @chriswilcox47 https://chriswilcox.dev Level Description Execution Monitoring Fallback System Capabilities 1

    Tooling, Basic Automation Human System Human Humam Scoped 2 Human monitored automation. System Human Human Scoped 3 System monitored automation with human fallback System System Human Scoped 4 System handles fallback within a scope System System System Scoped 5 System handles work with nebulous scope System System System Unscoped @chriswilcox47 https://chriswilcox.dev
  11. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Level 1: Automate portions of your

    workflow.
  12. @chriswilcox47 https://chriswilcox.dev Level 1 - Automate portions of workflow -

    Problem: Releasing a package takes multiple actions and it is easy to forget a bit. - Bot: Run script that tags main branch and publishes to package manager. The Simpsons, 20th Century Fox
  13. @chriswilcox47 https://chriswilcox.dev Level 2: Automate discovery and work, but under

    supervision.
  14. @chriswilcox47 https://chriswilcox.dev Level 2: Automate, but supervise - Problem: The

    script we authored to publish releases is easy to forget to run after we have a PR to update our version. - Bot: Automatically publish when a release PR is merged. Futurama, 20th Century Fox
  15. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Level 3: Let the robot do

    work for you, with limited supervision.
  16. @chriswilcox47 https://chriswilcox.dev Level 3: Automate with limited supervision - Problem:

    We have issues go stale in our repository. - Bot: Reassign issues to other contributors when issues go stale. Boston Dynamics
  17. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Level 4: Stop Supervising, Stop Fallback

    support.
  18. @chriswilcox47 https://chriswilcox.dev Level 4: Automate with *no* supervision - Problem:

    Sometimes branches other than master lie around after merges. - Bot: Delete stale branches from the repository. Matt Unsworth / YouTube
  19. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Level 5: The robot is your

    boss now.
  20. @chriswilcox47 https://chriswilcox.dev crwilcox @chriswilcox47

  21. @chriswilcox47 https://chriswilcox.dev TriStar Pictures, 1991

  22. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Writing a bot

  23. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Writing a bot to write bots.

  24. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Writing a bot to write bots.

  25. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Writing a bot to write bots.

  26. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev

  27. @chriswilcox47 https://chriswilcox.dev Describe the scenario we want to solve “Can

    we run renovate pull request CI automatically?”
  28. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Identify the event - Initial PR

    - Every PR update - On merge to master - On issue creation
  29. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Decide if bot will alert or

    change.
  30. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Let’s build it already!

  31. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev

  32. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev

  33. @chriswilcox47 https://chriswilcox.dev Just another node package.

  34. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Minimal deps.

  35. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev src

  36. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev

  37. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Setting up your environment to run

    a probot bot.
  38. @chriswilcox47 https://chriswilcox.dev

  39. @chriswilcox47 https://chriswilcox.dev Run smee proxy

  40. @chriswilcox47 https://chriswilcox.dev npm start

  41. @chriswilcox47 https://chriswilcox.dev Creating the GitHub App

  42. @chriswilcox47 https://chriswilcox.dev

  43. @chriswilcox47 https://chriswilcox.dev

  44. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Configure Permissions

  45. @chriswilcox47 https://chriswilcox.dev

  46. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Install the bot in a repository

  47. @chriswilcox47 https://chriswilcox.dev

  48. @chriswilcox47 https://chriswilcox.dev

  49. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Running a bot locally.

  50. @chriswilcox47 https://chriswilcox.dev Set a few environment variables - APP_ID -

    WEBHOOK_SECRET - PRIVATE_KEY_PATH
  51. @chriswilcox47 https://chriswilcox.dev

  52. @chriswilcox47 https://chriswilcox.dev

  53. @chriswilcox47 https://chriswilcox.dev

  54. @chriswilcox47 https://chriswilcox.dev Starting test bot locally

  55. @chriswilcox47 https://chriswilcox.dev

  56. @chriswilcox47 https://chriswilcox.dev

  57. @chriswilcox47 https://chriswilcox.dev

  58. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Deployment.

  59. @chriswilcox47 https://chriswilcox.dev Google Cloud Functions

  60. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev

  61. @chriswilcox47 https://chriswilcox.dev Google Cloud Storage + Google Cloud KMS Key

    Management Service maintains secrets so they aren’t hard coded in Google Cloud Function.
  62. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev

  63. @chriswilcox47 https://chriswilcox.dev Google Cloud Build Builds and Deploys our bots

    once merged to HEAD.
  64. @chriswilcox47 https://chriswilcox.dev Building npm install npm run compile mkdir -p

    "target" cp -r build "target" cp package.json "target/package.json"
  65. @chriswilcox47 https://chriswilcox.dev Publishing functionname="function_name" gcloud functions deploy "$functionname" --trigger-http \

    --runtime nodejs10 \ --region "$FUNCTION_REGION" \ --set-env-vars DRIFT_PRO_BUCKET="$BUCKET",KEY_LOCATION="$KEY_LOCATION",KEY_ RING="$KEY_RING",GCF_SHORT_FUNCTION_NAME="$functionname",PRO JECT_ID="$PROJECT_ID"
  66. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Maybe we can stop playing bad

    robots?
  67. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev https://github.com/googleapis/repo-automation-bots

  68. @chriswilcox47 https://chriswilcox.dev Thanks to the Googlers that contributed! And others...

    Benjamin Coe bcoe BenjaminCoe Jeff Ching chingor13 Colin Nelson orthros orthros_ Justin Beckwith justinbeckwith JustinBeckwith Chris Wilcox crwilcox chriswilcox47 Alexander Fenster alexander-fenster Sofia Leon sofisl https://github.com/googleapis/repo-automation-bots