Let the Bot Review it - Reducing Toil with Fossabot

This presentation outlines the issues of manual pull request reviewing and highlights the benefits of using AI reviews in these scenarios with Fossabot and Claude.

Transcript

1. Let the Bot Review it Reducing Toil with Fossabot 17.02.2026 WhizUs GmbH by Julian Zhuang
2. Julian Zhuang DevOps Engineer @ WhizUs GmbH 🚀 Cloud Native & Kubernetes 🔧 CI/CD & GitOps practioner 🔒 Supply Chain Security enthusiast 17.02.2026 WhizUs GmbH 2
3. Pull Request Reviews Why they matter — and why they’re hard
4. What Is a PR Review? A Pull Request Review is a quality gate before code reaches production. Reviewers check for: ✅ Correctness — does it do what it should? ✅ Security — does it introduce vulnerabilities? ✅ Dependencies — are upgrades safe? ✅ Breaking changes — will it break existing behavior? ✅ Best practices — is the code maintainable? | The Basics 17.02.2026 WhizUs GmbH 4
5. The Challenge In practice, manual reviews struggle with: 📦 Dependency updates — "just a version bump", but what changed upstream? ⏱️ Time pressure — reviewers skim diffs, approve quickly 🔗 Transitive dependencies — hidden changes deep in the tree 🧩 Context gaps — no one reads every changelog for every package 🔒 Security blind spots — CVEs introduced via minor/patch bumps The more "boring" a PR looks, the more dangerous it can be. | Manual Reviewing 17.02.2026 WhizUs GmbH 5
6. A Real-World Example Let’s look at a Dependabot PR from our CNDA demo application
7. The PR PR #39: chore(deps): bump the npm_and_yarn group with 4 updates Statistics: 1 file changed ( package-lock.json ) All minor/patch bumps. Would you approve this? | CNDA Showcase App jsonpath 1.1.1 → 1.2.1 lodash 4.17.21 → 4.17.23 qs 6.13.0 → 6.14.2 webpack 5.102.1 → 5.105.2 17.02.2026 WhizUs GmbH 7
9. jsonpath 1.2.1:
10. jsonpath commit for 1.2.1:
11. What’s Actually Hiding Package CVE Severity Status jsonpath 1.2.1 CVE-2025-61140 🔴 Prototype Pollution No fix available! qs 6.13.0 CVE-2025-15284 🔴 DoS via memory exhaustion Fixed in 6.14.1+ lodash 4.17.23 CVE-2025-13465 🟡 Prototype Pollution Fixed in this version webpack User info bypass 🟡 HttpUriPlugin Fixed in this version Upgrading jsonpath actually introduces a known, unfixed CVE! | Security Issues 17.02.2026 WhizUs GmbH 11
12. The Irony This PR is supposed to fix things, but: 1. jsonpath 1.2.1 ships with CVE-2025-61140 — no patched version exists yet 2. The fix was pushed to master but not published to npm 3. Merging this PR puts a known vulnerable package into production 4. A human reviewer would never catch this from the diff alone Minor/patch bumps can introduce security vulnerabilities. | Patch Bumps ≠ Safe 17.02.2026 WhizUs GmbH 12
13. How Can AI Help? Automated analysis beyond the diff
14. AI PR Review Instead of reviewing just the diff, AI can: 🔍 Research upstream changelogs across all updated packages 🔒 Cross-reference CVE databases for every dependency version 📋 Analyze transitive dependency trees for hidden risks ⚠️ Flag breaking changes even in minor/patch bumps 🛠️ Suggest actionable fixes with specific commands 📝 Generate testing checklists tailored to the changes | The Concept 17.02.2026 WhizUs GmbH 14
15. Fossa & Fossabot Fossa is a supply chain security platform that helps teams manage: 📦 Dependencies & licenses 🔒 Security vulnerabilities (CVE scanning) 📋 Compliance with licensing requirements 🐙 Requires GitHub.com Fossabot is Fossa’s AI-powered PR review assistant: 🤖 Automatically reviews dependency update PRs 🔍 Analyzes changelogs, CVEs, and breaking changes 💬 Comments directly on GitHub/GitLab PRs ⚡ Works for npm, Maven, PyPI, Go modules, and more 🔒 Uses LLM & AI services from Anthropic through an enterprise agreement - no training or data retention 17.02.2026 WhizUs GmbH 15
16. Fossabot | AI Review for the CNDA App
17. Fossabot | AI Review for the CNDA App
18. Fossabot Fossabot automatically analyzed: ✅ CVE databases for all 4 packages ✅ Whether packages are direct or transitive dependencies ✅ Actual usage in the application source code ✅ Breaking changes in changelogs ✅ Runtime compatibility (Node 20) ✅ Fix availability and timelines All of this — in 5-10 minutes, on every PR, automatically. | What It Checked 17.02.2026 WhizUs GmbH 18
19. Fossabot | Supported Ecosystems & Tools
20. And For DevOps? Helm Charts, Pipelines, Infrastructure-as-Code
21. The DevOps Problem DevOps PRs follow the same pattern: 1 file changed, 1 line added, 1 line deleted. | Same Pattern, Higher Stakes dependencies: - name: argo-cd - version: 9.3.7 + version: 9.4.2 repository: https://argoproj.github.io/argo-helm 17.02.2026 WhizUs GmbH 21
22. Fossabot’s Limitation Fossabot works great for Javascript/Typescript ecosystems: ✅ React, Node.js, frontend dependencies ✅ CVE cross-referencing for npm packages ✅ Dependency usage analysis in JS/TS source But for DevOps artifacts, it falls short: ❌ No Helm chart changelog analysis ❌ No understanding of Kubernetes breaking changes ❌ No ArgoCD/Flux upgrade guide awareness ❌ No infrastructure-level impact assessment We need something for DevOps PRs. | DevOps Ecosystem 17.02.2026 WhizUs GmbH 22
23. AI Reviews with Claude Opus Deep analysis for DevOps Pull Requests
24. The Approach Using Claude (Opus) we can build deep PR reviews: 1. Feed the PR diff + upstream changelogs + upgrade guides 2. Ask for breaking changes, security risks, migration steps 3. Get a structured review with risk assessment Let’s see what Claude found for the ArgoCD bump… | Claude Opus 17.02.2026 WhizUs GmbH 24
25. Instructions Please review following pull request: https//www... - these instructions should be used when reviewing Pull Requests - point out the changes on a surface level - point out security risks and issues. If possible, write full CVE name, which CVE's has been fixed and which new ones a - although this is a minor update, there could still be underlying breaking changes (e.g. from underlying dependencies). - do not use emojis - you can use gh cli - dump your results in a new file (naming convention PR-<number>-REVIEW.md). If a file is already existing, create a new
26. What Claude Found What looks like a 1-line change actually contains: 📊 16 commits between chart versions 🚨 Major version jump: ArgoCD v3.2.6 → v3.3.0 💥 Breaking changes: SSA now default sync strategy 🔒 Security fixes: Helm 3.19.4, Go crypto patches, Redis ⚠️ CRD size limit: kubectl apply will fail 📋 Multiple upgrade guides to review (v2.14 → v3.3!) | The ArgoCD PR 17.02.2026 WhizUs GmbH 26
27. What Claude Found | The ArgoCD PR
28. What Claude Found | The ArgoCD PR
29. Breaking Change #1 The ApplicationSet CRD exceeds the 262144-byte annotation limit: ❌ kubectl apply will fail ✅ helm upgrade is not affected ⚠️ Self-managing ArgoCD must use SSA | CRD Size Limit The CustomResourceDefinition "applicationsets.argoproj.io" is invalid: metadata.annotations: Too long: may not be more than 262144 bytes 17.02.2026 WhizUs GmbH 29
30. Breaking Change #2 New health checks may change application status unexpectedly: ceph.rook.io/CephCluster , CephObjectStore keda.sh/ScaledJob services.cloud.sap.com/ServiceBinding *.cnrm.cloud.google.com/* (GCP Config Connector) Impact: Apps previously marked "Healthy" may now show different statuses. This can trigger alerts and break automated workflows! | New Health Checks 17.02.2026 WhizUs GmbH 30
31. Risk Matrix | Claude’s Assessment Risk Factor Level Notes CRD size limit (self-managed) 🔴 HIGH Requires SSA pre-config SSA default change � HIGH Fundamental behavioral shift Health check changes 🟡 MEDIUM Status transitions Kustomize namespace fix 🟡 MEDIUM May alter manifests Supply chain 🟡 MEDIUM Many transitive updates Stale appVersion 🟢 LOW Cosmetic but misleading
32. Claude’s Verdict DO NOT MERGE BLINDLY Despite being a one-line change: 1. ✅ Read all upgrade guides (v2.14 → v3.0 → v3.1 → v3.2 → v3.3) 2. ✅ Deploy to staging environment first 3. ✅ Enable ServerSideApply=true if self-managing 4. ✅ Update wrapper chart appVersion to v3.3.0 5. ✅ Monitor for 24-48 hours 6. ✅ Have rollback plan ready | Conditional Approval 17.02.2026 WhizUs GmbH 32
33. The Comparison Fossabot vs. Claude — different strengths
34. Fossabot | JS/TS Strengths: ✅ Automatic on every PR ✅ CVE database cross-referencing ✅ Dependency usage analysis ✅ Fix suggestions with commands ✅ Free & easy to set up Limitations: ❌ requires GitHub.com ❌ dev ecosystems only ❌ No Helm/K8s awareness ❌ No infrastructure context Claude | DevOps/Infra Strengths: ✅ Deep changelog analysis ✅ Helm/K8s breaking changes ✅ Multi-version upgrade paths ✅ Security risk assessment ✅ Migration step generation Limitations: ❌ Requires manual prompting ❌ No automatic PR integration (yet) ❌ Needs context feeding 17.02.2026 WhizUs GmbH 34
35. Key Takeaways What we learned
36. The Core Message What a human reviewer sees "Bump 4 npm packages" — minor/patch versions ✅ "Bump argo-cd from 9.3.7 to 9.4.2" — 1 line changed ✅ What AI uncovers � Known CVEs introduced by the update 🚨 Breaking changes hidden behind semver 🔒 Security implications in transitive deps ⚠️ Migration requirements spanning multiple versions | Never Trust the Diff 17.02.2026 WhizUs GmbH 36
37. Human + AI | Better Together Humans excel at: Business context Architecture decisions Risk tolerance judgment Stakeholder communication AI excels at: Reading changelogs exhaustively Tracking transitive dependencies Identifying breaking changes Generating migration steps Security vulnerability analysis 17.02.2026 WhizUs GmbH 37
38. Thank You! Let the Bot Review it — Reducing Toil with Fossabot WhizUs / ArgoCD / Slidev 17.02.2026 WhizUs GmbH 38