Prow! Leveraging Developer-Centric CI for Your OSS Project!

Prow is a CI system maintained by Kubernetes SIG Testing to test Kubernetes on Kubernetes. Prow is designed as a pluggable system of components and it can be used as a generic CI system. The robust architecture of Prow can lead to challenges in deploying it. In the talk, we will navigate the challenges faced when deploying and using Prow, including setting up the Prow control plane components, configuring access for GitHub repos, and enabling Prow plugins. Prow is used by large projects in the CNCF landscape like Kubernetes, Knative, cert-manager, Falco, to name a few. Even though a lot of these projects have deployed Prow successfully, it is a challenge to set up Prow. The talk will highlight the common pitfalls and gotchas that one will run into when deploying Prow. The talk would cover * A roundup of Prow Architecture * Cloud resources required for Prow and setting them up * Capabilities of Prow like running tests, using GitHub comments for interaction, auto merging pull requests.

Transcript

1. Prow! Leveraging Developer Centric CI for your OSS project! Arsh Sharma & Nabarun Pal KubeCon + CloudNativeCon Europe 2022 1
2. $ whoarewe KubeCon + CloudNativeCon Europe 2022 2
3. Arsh Sharma • Developer Experience Engineer at Okteto • Kubernetes 1.25 CI Signal Lead • New Contributor Ambassador for Kubernetes SIG Docs KubeCon + CloudNativeCon Europe 2022 3
4. Nabarun Pal • Senior Engineer at VMware • Kubernetes Release Engineering • Kubernetes Code of Conduct Committee KubeCon + CloudNativeCon Europe 2022 4
5. What is Prow? KubeCon + CloudNativeCon Europe 2022 5
6. What is Prow? • Kubernetes based CI/CD system KubeCon + CloudNativeCon Europe 2022 5
7. What is Prow? • Kubernetes based CI/CD system • GitHub automation through policies KubeCon + CloudNativeCon Europe 2022 5
8. What is Prow? • Kubernetes based CI/CD system • GitHub automation through policies • Configuration as code KubeCon + CloudNativeCon Europe 2022 5
9. What is Prow? • Kubernetes based CI/CD system • GitHub automation through policies • Configuration as code • Highly pluggable KubeCon + CloudNativeCon Europe 2022 5
10. What is Prow? • Kubernetes based CI/CD system • GitHub automation through policies • Configuration as code • Highly pluggable • ChatOps via /honk style commands KubeCon + CloudNativeCon Europe 2022 5
11. ChatOps KubeCon + CloudNativeCon Europe 2022 6
12. ChatOps • /yolo style commands which allow interacting with GitHub issues and PRs KubeCon + CloudNativeCon Europe 2022 6
13. ChatOps • /yolo style commands which allow interacting with GitHub issues and PRs • Assign issues to contributors or PRs to reviewers KubeCon + CloudNativeCon Europe 2022 6
14. ChatOps • /yolo style commands which allow interacting with GitHub issues and PRs • Assign issues to contributors or PRs to reviewers • Label issues and PRs KubeCon + CloudNativeCon Europe 2022 6
15. ChatOps • /yolo style commands which allow interacting with GitHub issues and PRs • Assign issues to contributors or PRs to reviewers • Label issues and PRs • Provide approval for PRs KubeCon + CloudNativeCon Europe 2022 6
16. ChatOps • /yolo style commands which allow interacting with GitHub issues and PRs • Assign issues to contributors or PRs to reviewers • Label issues and PRs • Provide approval for PRs • Run tests for a PR KubeCon + CloudNativeCon Europe 2022 6
17. ChatOps • /yolo style commands which allow interacting with GitHub issues and PRs • Assign issues to contributors or PRs to reviewers • Label issues and PRs • Provide approval for PRs • Run tests for a PR • A lot more stuff! KubeCon + CloudNativeCon Europe 2022 6
18. Architecture KubeCon + CloudNativeCon Europe 2022 7
19. Tide KubeCon + CloudNativeCon Europe 2022 8
20. Tide • Manages pool of GitHub PRs KubeCon + CloudNativeCon Europe 2022 8
21. Tide • Manages pool of GitHub PRs • Automatically retests PRs when they meet the criteria KubeCon + CloudNativeCon Europe 2022 8
22. Tide • Manages pool of GitHub PRs • Automatically retests PRs when they meet the criteria • Merges PRs without user intervention when tests pass KubeCon + CloudNativeCon Europe 2022 8
23. Testgrid KubeCon + CloudNativeCon Europe 2022 9
24. Testgrid • Interactive dashboard to view test results KubeCon + CloudNativeCon Europe 2022 9
25. Testgrid • Interactive dashboard to view test results • And they are shown in a grid! KubeCon + CloudNativeCon Europe 2022 9
26. KubeCon + CloudNativeCon Europe 2022 10
27. Configuration KubeCon + CloudNativeCon Europe 2022 11
28. Inrepo KubeCon + CloudNativeCon Europe 2022 12
29. Inrepo • Define jobs in the same repo where the source code is KubeCon + CloudNativeCon Europe 2022 12
30. Inrepo • Define jobs in the same repo where the source code is • In a .prow directory or .prow.yaml file at the root of the repo KubeCon + CloudNativeCon Europe 2022 12
31. Inrepo • Define jobs in the same repo where the source code is • In a .prow directory or .prow.yaml file at the root of the repo • Jobs are NOT defined centrally in this approach KubeCon + CloudNativeCon Europe 2022 12
32. Centralizing configuration KubeCon + CloudNativeCon Europe 2022 13
33. Centralizing configuration • Have a separate repo for all your prow jobs across different repos KubeCon + CloudNativeCon Europe 2022 13
34. Centralizing configuration • Have a separate repo for all your prow jobs across different repos • Jobs are defined centrally in this approach KubeCon + CloudNativeCon Europe 2022 13
35. Centralizing configuration • Have a separate repo for all your prow jobs across different repos • Jobs are defined centrally in this approach • Let's look at kubernetes/test-infra! KubeCon + CloudNativeCon Europe 2022 13
36. kubernetes/test-infra KubeCon + CloudNativeCon Europe 2022 14
37. Deploying your own instance of Prow KubeCon + CloudNativeCon Europe 2022 15
38. Creating the GitHub app KubeCon + CloudNativeCon Europe 2022 16
39. Add hook URL KubeCon + CloudNativeCon Europe 2022 17
40. Give Repository and Org Permissions KubeCon + CloudNativeCon Europe 2022 18
41. Subscribe to all events KubeCon + CloudNativeCon Europe 2022 19
42. Note App ID and Private Key KubeCon + CloudNativeCon Europe 2022 20
43. Run tackle Set up the GCP credentials before running tackle $ tackle -skip-github ... Let it run and pause where it asks you to specify location for starter.yaml. KubeCon + CloudNativeCon Europe 2022 21
44. Apply the Prow CRDs $ kubectl apply --server-side=true -f https://raw.githubusercontent.com/kubernetes/test-infra/master/config/prow/cluster/prowjob-crd/prowjob_customresourcedefinition.yaml customresourcedefinition.apiextensions.k8s.io/prowjobs.prow.k8s.io serverside-applied KubeCon + CloudNativeCon Europe 2022 22
45. Create a GCS bucket $ gcloud iam service-accounts create prow-gcs-publisher $ identifier="$(gcloud iam service-accounts list --filter 'name:prow-gcs-publisher' --format 'value(email)')" $ gsutil mb gs://master-shifus-knowledge/ $ gsutil iam ch allUsers:objectViewer gs://master-shifus-knowledge $ gsutil iam ch "serviceAccount:${identifier}:objectAdmin" gs://master-shifus-knowledge $ gcloud iam service-accounts keys create --iam-account "${identifier}" service-account.json $ kubectl -n test-pods create secret generic gcs-credentials --from-file=service-account.json $ kubectl -n prow create secret generic gcs-credentials --from-file=service-account.json KubeCon + CloudNativeCon Europe 2022 23
46. Generate a HMAC token $ openssl rand -hex 20 f9abc03ce9...cb3d92df96d2f4c0466a2cf KubeCon + CloudNativeCon Europe 2022 24
47. Fetch the starter yaml, make necessary changes and then specify to tackle https://git.k8s.io/test-infra/config/prow/cluster/starter/starter- gcs.yaml KubeCon + CloudNativeCon Europe 2022 25
48. Point your domain to the Ingress IP KubeCon + CloudNativeCon Europe 2022 26
49. Update the HMAC Token in your GitHub App KubeCon + CloudNativeCon Europe 2022 27
50. Install the app on a repo KubeCon + CloudNativeCon Europe 2022 28
51. Enjoy! KubeCon + CloudNativeCon Europe 2022 29
52. Benefits of using Prow KubeCon + CloudNativeCon Europe 2022 30
53. Benefits of using Prow • Enables Effective Collaboration KubeCon + CloudNativeCon Europe 2022 30
54. Benefits of using Prow • Enables Effective Collaboration • Accessible! KubeCon + CloudNativeCon Europe 2022 30
55. Benefits of using Prow • Enables Effective Collaboration • Accessible! • New Contributor Friendly KubeCon + CloudNativeCon Europe 2022 30
56. Contribute Come join us in Kubernetes SIG Testing to maintain Prow. • Mailing List: https://groups.google.com/forum/#!forum/ kubernetes-sig-testing • Slack: #sig-testing on Kubernetes Slack KubeCon + CloudNativeCon Europe 2022 31
57. Links • Deploying Prow • Setting up GitHub App KubeCon + CloudNativeCon Europe 2022 32
58. Thank You!