When Agent Helps, and why I still write the PR myself

Transcript

1. When Agent Helps and Why I Still Write the PR Myself Tommy Han, 2026
2. Before we Begin…
3. Before we begin A few things to mention… • This talk is a Self-experience sharing • This is NOT a “how you should vibe-code in Open Source” talk • This is NOT a “do not use AI Agents in Open Source” talk. • There will be some suggested, open-source agent skills from me. • Understanding the context yourself is the key.
4. Problems we are facing! You may got some of them already…
5. Are we going to solve them? Not in this talk…
6. Agenda We are going to talk about… Level 0: Usage in Open Source Level 1: Problems shipping only with Agents Level 2: Make Pull Requests Better Level 3: Why still need Human-In-The-Loop
7. Level 0 Usage in Open Source
8. How many of you use Agents in Open-Source? Agents, Vibe-Code, Harness, Loops, Work ows… fl Photo Reference: Google Photo Search
9. Level 0 Why you can ship that fast nowadays? Thanks, Gemini
10. Level 0 Why you can ship that fast nowadays? Thanks, Gemini
11. Level 0 Where do we use AI? Coding
12. Level 0 Where do we use AI? Coding https://github.com/zed-industries/zed/pull/59025 Thanks, Github.
13. Level 0 Where do we use AI in open-source Projects Prototyping
14. Level 0 Where do we use AI in open-source Projects Prototyping(Let’s continue using my example in Zed!)
15. Level 0 Where do we use AI in open-source Projects Design
16. Level 0 Where do we use AI in open-source Projects Documentation (mostly missed) Image Reference: https://www.linkedin.com/pulse/llm-agent-system-document-code-jeremy-harper-m9tyc/
17. Seems Pretty Good!
18. Level 1 Problems of Shipping ONLY with Agents
19. Demo
20. Level 1 Demo Let’s try to build a Pull Request for Zed (Sorry Zed.) Pick a random issue. I will not commit it, since it is a demo.
21. Will it become a good PR?
22. Level 1 Why shipping with AI can be a problem Ask yourself a question: Do you know what is going on inside the generated code?
23. Level 1 How it hurts our projects? • Too many issues created • Too many pull requests created • Too many feature requests created • Too many backlashes after merging AI created PRs • Too many…
24. Level 1 How it hurts our projects?
25. Level 1 How it hurts the reviewers?
26. Level 1 How it hurts the ecosystem?
27. Level 1 Wait…Slop is not something new? We mostly won’t call them “AI slop”. “Change README” Change from Language A to B Add Feature A Replace Package A with B…
28. Level 2 Make Pull Requests Better
29. Level 2 How can we make PR less shallow…? Think before you do anything. Understanding is the key. Agents can solve problem, but not framing the problem
30. Level 2 Think how are you going to use the skill • What is your goal? • How this skills help you? • Can it guide the agent?
31. Level 2 Agent Skills are useful
32. Level 2 Agent Skills Recommendation
33. Level 2 Build Your Own Skill! You have your own expertise! You should teach the skill to the agent, right?
34. Level 3 Why still need Human-in-the-loop?
35. Level 3 Human in the loop is necessary…? Personal Judgement vs Objective Statement fi Is the Solution ts your use-case the most?
36. Level 3 Software Engineering is more than just coding Agents delegates the coding and problem-solving part But not the whole software engineering
37. Summary
38. Summary What we have discussed today • How Great LLM Agent Is! • How Bad LLM Agents Are! • Why we felt pain and fatigue when handling open-source projects. • How can we do better, from a PR creator perspective.
39. How about Reviewers? That is another story…a very big one.
40. You can outsource your thinking, but you cannot outsource your understanding. Frame your problem before you start working.
41. Thank You! Andddd Q&A time.
42. References Articles that inspires/helps me to build this talk • https://cursor.com/insights • https://borretti.me/article/human-bottlenecks • https://red oatplane.lol/blog/17-why-share/ • https://ladybird.org/posts/changing-how-we-develop-ladybird/ • https://daniel.haxx.se/blog/2026/06/10/a-human-in-control/ • https://chris.eidhof.nl/post/learning-in-the-age-of-llms/ • https://www.brethorsting.com/blog/2025/07/agentic-coding-tools-not-skynet%2C-not-a-stochastic-parrot/ • https://www.brethorsting.com/blog/2026/05/domain-expertise-has-always-been-the-real-moat/ • https://annievella.com/posts/ nding-comfort-in-the-uncertainty/ • https://www.anthropic.com/research/claude-code-expertise • https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6097646 • https://arxiv.org/abs/2601.20245 fi fl • https://end-of-determinism.vercel.app/article