Oh No! The Robots Have Taken Over

As part of his work, Chris and his team at Google are responsible for managing libraries for 150 APIs across 7 languages. This results in a crushing amount of toil, making it hard at times to both make forward progress and maintain what has been created.

When you own over 70 node repositories you have to get creative. So the team decided to make an army of sorts. An army of screaming, free roaming robots.

Bots can have a freeing effect on your engineering team. Come to hear how embracing automation has let a team of engineers do what they do best and let the toil fall to the machines.

Presented at Node+JS Interactive 2019: https://sched.co/T5HD

Transcript

1. @chriswilcox47 https://chriswilcox.dev Oh No! The Robots Have Taken Over! Chris Wilcox Engineer, Google Cloud @chriswilcox47 https://chriswilcox.dev
2. @chriswilcox47 https://chriswilcox.dev What’s a bot anyway?
3. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Photo by Lenny Kuhne on Unsplash
4. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev
5. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev
6. @chriswilcox47 https://chriswilcox.dev What are bots good at?
7. @chriswilcox47 https://chriswilcox.dev
8. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Levels of automation
9. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev
10. @chriswilcox47 https://chriswilcox.dev Level Description Execution Monitoring Fallback System Capabilities 1 Tooling, Basic Automation Human System Human Humam Scoped 2 Human monitored automation. System Human Human Scoped 3 System monitored automation with human fallback System System Human Scoped 4 System handles fallback within a scope System System System Scoped 5 System handles work with nebulous scope System System System Unscoped @chriswilcox47 https://chriswilcox.dev
11. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Level 1: Automate portions of your workflow.
12. @chriswilcox47 https://chriswilcox.dev Level 1 - Automate portions of workflow - Problem: Releasing a package takes multiple actions and it is easy to forget a bit. - Bot: Run script that tags main branch and publishes to package manager. The Simpsons, 20th Century Fox
13. @chriswilcox47 https://chriswilcox.dev Level 2: Automate discovery and work, but under supervision.
14. @chriswilcox47 https://chriswilcox.dev Level 2: Automate, but supervise - Problem: The script we authored to publish releases is easy to forget to run after we have a PR to update our version. - Bot: Automatically publish when a release PR is merged. Futurama, 20th Century Fox
15. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Level 3: Let the robot do work for you, with limited supervision.
16. @chriswilcox47 https://chriswilcox.dev Level 3: Automate with limited supervision - Problem: We have issues go stale in our repository. - Bot: Reassign issues to other contributors when issues go stale. Boston Dynamics
17. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Level 4: Stop Supervising, Stop Fallback support.
18. @chriswilcox47 https://chriswilcox.dev Level 4: Automate with *no* supervision - Problem: Sometimes branches other than master lie around after merges. - Bot: Delete stale branches from the repository. Matt Unsworth / YouTube
19. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Level 5: The robot is your boss now.
20. @chriswilcox47 https://chriswilcox.dev crwilcox @chriswilcox47
21. @chriswilcox47 https://chriswilcox.dev TriStar Pictures, 1991
22. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Writing a bot
23. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Writing a bot to write bots.
24. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Writing a bot to write bots.
25. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Writing a bot to write bots.
26. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev
27. @chriswilcox47 https://chriswilcox.dev Describe the scenario we want to solve “Can we run renovate pull request CI automatically?”
28. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Identify the event - Initial PR - Every PR update - On merge to master - On issue creation
29. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Decide if bot will alert or change.
30. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Let’s build it already!
31. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev
32. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev
33. @chriswilcox47 https://chriswilcox.dev Just another node package.
34. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Minimal deps.
35. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev src
36. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev
37. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Setting up your environment to run a probot bot.
38. @chriswilcox47 https://chriswilcox.dev
39. @chriswilcox47 https://chriswilcox.dev Run smee proxy
40. @chriswilcox47 https://chriswilcox.dev npm start
41. @chriswilcox47 https://chriswilcox.dev Creating the GitHub App
42. @chriswilcox47 https://chriswilcox.dev
43. @chriswilcox47 https://chriswilcox.dev
44. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Configure Permissions
45. @chriswilcox47 https://chriswilcox.dev
46. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Install the bot in a repository
47. @chriswilcox47 https://chriswilcox.dev
48. @chriswilcox47 https://chriswilcox.dev
49. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Running a bot locally.
50. @chriswilcox47 https://chriswilcox.dev Set a few environment variables - APP_ID - WEBHOOK_SECRET - PRIVATE_KEY_PATH
51. @chriswilcox47 https://chriswilcox.dev
52. @chriswilcox47 https://chriswilcox.dev
53. @chriswilcox47 https://chriswilcox.dev
54. @chriswilcox47 https://chriswilcox.dev Starting test bot locally
55. @chriswilcox47 https://chriswilcox.dev
56. @chriswilcox47 https://chriswilcox.dev
57. @chriswilcox47 https://chriswilcox.dev
58. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Deployment.
59. @chriswilcox47 https://chriswilcox.dev Google Cloud Functions
60. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev
61. @chriswilcox47 https://chriswilcox.dev Google Cloud Storage + Google Cloud KMS Key Management Service maintains secrets so they aren’t hard coded in Google Cloud Function.
62. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev
63. @chriswilcox47 https://chriswilcox.dev Google Cloud Build Builds and Deploys our bots once merged to HEAD.
64. @chriswilcox47 https://chriswilcox.dev Building npm install npm run compile mkdir -p "target" cp -r build "target" cp package.json "target/package.json"
65. @chriswilcox47 https://chriswilcox.dev Publishing functionname="function_name" gcloud functions deploy "$functionname" --trigger-http \ --runtime nodejs10 \ --region "$FUNCTION_REGION" \ --set-env-vars DRIFT_PRO_BUCKET="$BUCKET",KEY_LOCATION="$KEY_LOCATION",KEY_ RING="$KEY_RING",GCF_SHORT_FUNCTION_NAME="$functionname",PRO JECT_ID="$PROJECT_ID"
66. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev Maybe we can stop playing bad robots?
67. @chriswilcox47 https://chriswilcox.dev @chriswilcox47 https://chriswilcox.dev https://github.com/googleapis/repo-automation-bots
68. @chriswilcox47 https://chriswilcox.dev Thanks to the Googlers that contributed! And others... Benjamin Coe bcoe BenjaminCoe Jeff Ching chingor13 Colin Nelson orthros orthros_ Justin Beckwith justinbeckwith JustinBeckwith Chris Wilcox crwilcox chriswilcox47 Alexander Fenster alexander-fenster Sofia Leon sofisl https://github.com/googleapis/repo-automation-bots