Code Migration at Scale: AI Agents and RuboCop

Learn how to automate large-scale code migrations by leveraging AI Agents to write Rubocop Cops

Transcript

1. George Ma - Rails World 2025 Code Migration at Scale AI-Powered Test Driven Development
2. Observe.log Rails.event.notify 10,000+ callsites ~3000 files
4. 3,207 file changes 30,000+ total lines changed
5. 3,207 file changes 30,000+ total lines changed
6. “Migrate pls”
7. Agenda - ❗Why agentic tooling alone was suboptimal - 💚 Why a combination of Test Driven Development, RuboCop and agentic tooling succeeded
8. 10,000+ instances/ 3000+ files Migrate all these instances to the new Rails Structured Logger ❗Why agentic tooling alone was suboptimal
9. ❗Why agentic tooling alone was suboptimal
10. Hash rockets Mixed (hash rockets + kwargs) ❗Why agentic tooling alone was suboptimal
11. Need to preserve indentation Attribute chaining (.merge) ❗Why agentic tooling alone was suboptimal
12. Base hash merged with additional parameters Kwsplat combined with explicit key-value pairs ❗Why agentic tooling alone was suboptimal
14. Give a hat
15. Transform A Transform B Transform C … …. Transform X Transform Y Transform Z
17. - Read files one at a time, manually approve changes - Frequent hallucinations - Barely faster than performing manual edits and claude-3.5
18. and claude-4.0 - Sub-tasks (better parallelization) - Great when the task at hand is simpler / smaller in scope - Imperfections when the prompt is too granular
19. - Great when the task at hand is simpler / smaller in scope and claude-4.0 - Sub-tasks (better parallelization) - Imperfections when the prompt is too granular
20. Recap
21. Recap Observe.log Rails.event.notify
22. +
23. 💚 Test Driven Development Step 0: Start with a new Cop and its corresponding test Cop Cop_test
24. Step 1: Prompt Claude to write a (failing) test 💚 Test Driven Development
25. - Provides a base-line for Cop development / correctness - Prevents regressions as you add additional cases 💚 Test Driven Development
26. Step 2: Ask Claude to create the Cop’s autocorrect logic to pass this test case 💚 Test Driven Development
27. Step 2.5: Repeat steps 1 & 2 until all cases are autocorrectable 💚 Test Driven Development
28. cops/observe_log_to_rails_event.rb
29. observe_log_to_rails_event_test.rb
30. Step 3: Run the Cop with the autocorrect flag (`-a`) 💚 Test Driven Development
31. Step 4: Reap the benefits 💚 Test Driven Development
32. - Less token use/time spent waiting for agents to run - Leverage TDD and agentic tooling’s strengths to write the Cop itself - Cop that produces deterministic results every time 🚀