In the Age of AI Code Review, What Should Android Engineers Review? - Building the Foundation & Redesigning Code Review -

Slides for my talk at DroidKaigi 2026.

Transcript

1. In the Age of AI Code Review, What Should Android Engineers Review? — Building the Foundation & Redesigning Code Review — kobaken DroidKaigi 2026
2. kobaken pixiv Inc. Developing the pixiv Comic Android app
3. A quick question! Do you have AI review your code? 🙋 3
4. Opening ̶ Problem Statement The bottleneck has shifted to review • AI sped up technical design, implementation, and testing • Faster stages make review lead time stand out • Human review determines throughput, becoming the bottleneck Design Implement Test Faster with AI ──────────────────────────────▶ Review Things pile up here 4
5. Opening ̶ Problem Statement Ideal: full-speed dev with AI review! • Why not have AI do the code review too? • Humans just define the spec and the intended UX • AI generates the code, AI flags issues, AI fixes them Loop until there are no more findings! Design Implement Test Review Faster with AI ────────────────────────────▶ 5
6. Opening ̶ Problem Statement Ideal: full-speed dev with AI review! • Why not have AI do the code review too? 🤔 • Humans just define the spec and the intended UX • AI generates the code, AI flags issues, AI fixes them Loop until there are no more findings! Design Implement Test Review Faster with AI ────────────────────────────▶ 5
7. The question for this session In the age of AI code reviews, what should humans review? 6
8. Opening Agenda Section 1. Opening — Problem Statement 2. Background — pixiv Comic Android & the context of adopting AI review 3. Building the Review Foundation 4. Redesigning the Review Process 5. Summary 7
9. SECTION 02 Background 02 The pixiv Comic Android app and how we came to adopt AI review
10. Background The pixiv Comic Android app at a glance • Over 10 years of history • Migrating to a multi-module structure • Most legacy code still lives in the app module • Migrating to Compose • ViewBinding / DataBinding still remain • MVVM 9
11. Background The pixiv Comic Android app at a glance 42 202,727 2 58 GitLab Modules Lines of code Committers MRs per Version app 1 / library 1 incl. tests 45,758 kobaken and one month control core 14 feature 26 build/CI 3,396 more average / max 95 Self-hosted in- 2025/7‒2026/7 house 10
12. When we first adopted AI AI generates the code for us Feature development is now faster than ever before. 11
13. When we first adopted AI But despite faster MR creation, lead time wasn't improving… 12
14. Background The problem that hit pixiv Comic Problem MR pile-up: review wait grows, lead time doesn't improve ▲ Cause Large-diff MRs mean heavy review load AI coding increased large-diff MRs Each review is heavy; MRs pile up faster than they can be handled 13
15. We want to solve the MR pile-up Let's have AI review our code 🤖📝 14
16. Background ̶ How our AI review setup evolved Introducing AI review on GitLab • Accepts arbitrary prompts • Posts comments on GitLab 15
17. Background ̶ How our AI review setup evolved Introducing AI review on GitLab 16
18. Background ̶ How our AI review setup evolved Introducing AI review on GitLab 17
19. How our AI review setup evolved We introduced AI review, but human review didn't change much The bottleneck remained… 18
20. Neither "AI does it all" nor "AI is useless" Deciding what to delegate and who owns what How do we redesign the review process itself? 19
21. Agenda Section 1. Opening — Problem Statement 2. Background — pixiv Comic Android & the context of adopting AI review 3. Building the Review Foundation 4. Redesigning the Review Process 5. Summary 20
22. SECTION 03 03 Building the Review Foundation The groundwork that determines accuracy, and Android-specific failure stories
23. Foundation ̶ Why we need one What happened without a foundation AI Slop findings False positives Findings stop → noise being read A vicious cycle where code review breaks down 22
24. Hypothesis AI review accuracy = the quality of the context you provide 23
25. Going further Reduce the trivial fixes flagged in review → Raise the quality of AI coding itself 24
26. Building the Foundation Giving AI context Module dependencies Architecture Code style Android knowledge Testing Agent guidelines Project knowledge 25
27. Foundation ̶ Giving AI context Module structure and dependency direction • The overall multi-module picture • What each module is responsible for • Rules such as no feature → app dependencies 26
28. Foundation ̶ Giving AI context Architecture policy • Explanations of the components in each layer • Document each one's responsibilities • State management for ViewModels and Compose • UiState design policy 27
29. Foundation ̶ Giving AI context Android knowledge • Mostly unnecessary • Provide knowledge when results repeatedly miss your intent • Edge-to-edge 28
30. Foundation ̶ Giving AI context Agent behavior guidelines • "Propose an execution plan for tasks that take 3+ steps" • "Run build commands in quiet mode (-q)" 29
31. Foundation ̶ Giving AI context Unit testing rules • Testing methodology • Policy on dummy data and mocking 30
32. To provide well-refined context, write an AGENTS.md 31
33. Foundation ̶ Giving AI context What is AGENTS.md? • Where you write project-specific rules for AI agents • In Claude Code, it's CLAUDE.md • Handy to reference AGENTS.md from CLAUDE.md as well 32
34. Foundation ̶ Giving AI context Growing your AGENTS.md • About 30 lines in total • Sets out the agent's behavior guidelines • We create and reference the following md files: • architecture.md • junit5_unit_tests.md 33
35. Foundation ̶ Giving AI context: Growing AGENTS.md architecture.md • Explanations and sample implementations for each layer • Architecture and module structure • Project-specific UiState design policy • Jetpack Compose guidelines • Anti-patterns documented too, detectable in code generation & review 34
36. Foundation ̶ Giving AI context: Growing AGENTS.md junit5̲unit̲tests.md • Lists the basic rules and libraries for writing unit tests • Extra notes for tests involving coroutines • How to handle mock data and dummy data 35
37. Foundation ̶ Giving AI context: Skills Define specific workflows as Skills • adapting-edge-to-edge (our own skill) • A skill for making screens support edge-to-edge • Supports both Android View / Compose • The official skill covers Compose only 36
38. Foundation ̶ Giving AI context: Skills Define specific workflows as Skills • multi-perspective-code-review (our own skill) • Has subagents review from multiple perspectives "Behavior" or "structure" Data / Infra Design UI / Compose Testing Security 37
39. Foundation ̶ Giving AI context: Skills Designing the Output Format of multi-perspective-code-review Severity Target layer Why it's bad Alternative Must-fix or UI / domain / data Convention, bug, or Concrete fix informational — which one design principle proposal 38
40. Foundation ̶ Giving AI context: Skills Designing the Output Format of multi-perspective-code-review Example finding (sample) [High / UI layer] A ViewModel Flow is collected directly inside a LazyColumn. The subscription is re-established on every recomposition (rule: subscribe via UiState). Alternative: subscribe with collectAsStateWithLifecycle() and consolidate into UiState. Fixed output format → better decisions on adopting findings 39
41. Foundation ̶ Prompts for review Establish a review guideline • review_guideline.md • References architecture.md, junit5_unit_tests.md, etc. • "Review against the standards in ~.md" • Also check the MR description against the implementation • "Does the description make the purpose clear?" • "Does it diverge from the implementation?" 40
42. Context isn't the only thing that matters Strengthen your test suite for an even more robust setup ✅ 41
43. Foundation ̶ Tests feed into review Unit tests Overview Logic verification with JUnit / Robolectric Domain logic and UI state changes verified automatically in code Relation to AI review Makes AI findings "verifiable" Findings are fact-checked automatically; refactoring proposals accepted with confidence 42
44. Foundation ̶ Tests feed into review Visual regression testing (VRT) Overview Screenshot comparison with Roborazzi and similar tools Automatically verifies "the UI hasn't changed" via image diffs Relation to AI review Automatically proves "the appearance hasn't changed" Reduces the burden of humans visually checking screens for refactorings 43
45. Foundation ̶ Tests feed into review With tests, AI findings become "verifiable" AI findings Verified by Accept / tests reject • Unit tests raise confidence in logic and make refactoring safer • VRT automatically proves "the UI hasn't changed" 44
46. Foundation ̶ Taking it further Detect architecture violations automatically with Konsist • Write architecture rules as test code • Auto-check forbidden dependencies / layer reference directions in CI • Rules that tools can check go to Konsist; judgment criteria go to AGENTS.md Wherever possible, catch rule violations with tests, not AI findings 45
47. Failures along the way Failure stories pixiv Comic ran into while solidifying the foundation 💣💥 46
48. Foundation ̶ pixiv Comic's failure stories Failure: auto-generated code misjudged as a "build error" What happened AI couldn't reference the generated ViewBinding / DataBinding classes and wrongly flagged "this will be a build error" 47
49. Foundation ̶ pixiv Comic's failure stories Failure: auto-generated code misjudged as a "build error" Cause Code auto-generated at build time isn't in the repository 48
50. Foundation ̶ pixiv Comic's failure stories Workaround: fill in "what AI can't see" with context • Document the existence and handling of auto-generated code in the review md file • Ignore findings like "undefined" or "build error" 49
51. Foundation ̶ Recap Foundation recap ✓Raise the quality of AI coding ✓Grow your AGENTS.md ✓Agent behavior guidelines and project-specific knowledge ✓Maintain review prompts & Skills ✓Multiple perspectives finding severity ✓Adopt tests (Unit / VRT) to make findings verifiable 50
52. Agenda Section 1. Opening — Problem Statement 2. Background — pixiv Comic Android & the context of adopting AI review 3. Building the Review Foundation 4. Redesigning the Review Process 5. Summary 51
53. SECTION 04 04 Redesigning the Review Process Redefining the Responsibilities of Humans and AI in Practice
54. Review Redesign ̶ Our previous flow Review flow: Before Design & implement Create MR AI review bot (optional) Human review (required) Merge 53
55. Review Redesign ̶ Shifting responsibility Six code review perspectives Quality Maintainability Security Bugs & regressions Readability Vulnerabilities & data leaks Requirements Consensus Accountability Works to spec? Align on design Who is accountable 54
56. Review Redesign ̶ Shifting responsibility What AI handles vs. what humans own Perspective AI Human Quality ◎ ◯ AI detects rule violations & routine bugs; humans confirm Maintainability ◎ ◯ Naming & consistency: AI. Structure & design: humans Security ◯ ◎ AI detects patterns; risk judgment & responsibility: humans Requirements △ ◎ AI can assist if given the spec; humans check against it Consensus — ◎ Sharing design decisions & building buy-in is human-only Accountability — ◎ "Who approved it" can't be shifted to AI ◎ Primary ◦ Support △ Conditional Notes — Not covered 55
57. Review Redesign ̶ Review before implementation Review requirements, specs & design before coding What we did Added spec & design review before implementation Pre-code decisions aren't automated ̶ humans review them ▼ Framework we adopted Design Docs Before implementing, summarize background, goals, design & trade-offs in a short doc, and build consensus on design decisions through stakeholder review 56
58. Making code easier to review What does "easy to review" actually look like? 🤔 57
59. Review Redesign ̶ MR splits that are easy to review Split MRs into "behavior" and "structure" Behavioral changes Structural changes Spec/behavior changes, bug Refactoring that doesn't fixes, etc. affect behavior A category with high user impact No user impact Mixed MRs slow review down → split them for faster turnaround Reference: Kent Beck, "Tidy First?" ̶ structural / behavioral changes 58
60. Review Redesign ̶ MR splits that are easy to review Review operations by category Behavioral changes Structural changes AI code review × human review AI code review + VRT is enough The MR author must present the change Often fixable after the fact intent Note: for mobile apps, a "fix it later" doesn't reach users instantly Draw the line by whether you can tolerate the fix lead time 59
61. Review Redesign ̶ "Behavior" changes vs. "structure" changes How do we classify them? • multi-perspective-code-review, built during foundation work • Classifies changes as "behavior", "structure", or "mixed" • If "mixed", instruct the AI to separate them • Splitting into separate MRs reduces cognitive load 60
62. Review Redesign ̶ Risk-based review intensity Vary review intensity by change risk Risk High Example changes Review policy Billing / auth / personal data / targetSdk updates Spec link required + human review required Medium Feature additions / logic changes AI review + human review Low Minor UI tweaks / copy changes Mainly AI + self-review (human review optional) Allocate human time according to risk 61
63. With great power comes great responsibility Who's responsible — the AI? 🤖 The reviewer? The reviewee? 🧐😎 62
64. Review Redesign ̶ Shifting responsibility The MR author owns it ̶ AI self-review • The MR author owns and is responsible for the code • The reviewer functions as a supporter • Loop AI review & fixes before creating the MR • Leverage multi-perspective-code-review Per-perspective self review Style findings already fixed by MR time Reviewers focus on intent & design 63
65. Review Redesign ̶ The new review flow Review flow: After Spec & design review (human) Implement Multi-perspective review Create MR AI review bot Human review (by type) Merge 64
66. Review Redesign ̶ Flow transition Review flow: Before → After Before After Spec & design review (human) Design & implement Implement Create MR AI review bot (optional) Human review (required) → Multi-perspective review Create MR AI review bot Human review (by type) Merge Merge 65
67. As a result of the redesign Less rework from missed requirements after implementation, and higher quality in the final deliverables 🎉 66
68. As a result of the redesign Refactoring gets accepted more smoothly than before 😎 The MR pile-up problem was reduced 67
69. Failures during the review redesign Failure stories pixiv Comic ran into while redesigning the review flow 💣💥 68
70. Review Redesign ̶ pixiv Comic's failure stories Failure #1: The AI never got the spec What happened The code was valid, but AI review missed the gap between implementation and spec Superficial reviews without clear requirement context 69
71. Review Redesign ̶ pixiv Comic's failure stories Failure #1: The AI never got the spec • Fix: add spec writing to the development flow • Pass the Spec to AI agents during implementation • Consolidate spec info into Design Docs for human review 70
72. Review Redesign ̶ pixiv Comic's failure stories Failure #2: AI LGTMs weaken human review What happened When the AI says "no issues," the rigor drains out of human review MRs that look already approved get less attention (varies by person) 71
73. Review Redesign ̶ pixiv Comic's failure stories Failure #2: AI LGTMs weaken human review • Fix: make explicit what AI checks vs. what humans check • Humans look at specs, intent, and risk • Shift from pointing out issues to confirming intent and asking questions 72
74. Review Redesign ̶ pixiv Comic's failure stories Failure #3: Design Docs ≈ Plan What happened We ended up writing overly detailed designs in Design Docs Too concrete — review cost was creeping up 73
75. Review Redesign ̶ pixiv Comic's failure stories Failure #3: Design Docs ≈ Plan • Fix: adopt them for high-uncertainty, large-scale work & refactoring • Use mini Design Docs for everyday feature development • A minimal version with reduced granularity and fewer sections 74
76. SECTION 05 Summary 05
77. Summary Build the foundation, redesign review Build the review foundation AI review accuracy = "context quality" Grow AGENTS.md, Skills & tests to make findings verifiable Redesign the review process Redefine the responsibilities of AI and humans Run pre-implementation review, MR splitting & AI self-review 76
78. AI isn't a silver bullet. Scope its role, focus on risk and intent. 77
79. Closing Resources • We plan to publish as many of our prompts and skills as possible • Questions are welcome at Ask the Speaker 78