TDD: Test-Driven Development vs TAB-Driven Development (Devnexus)

The rise of AI-powered coding assistants is changing the way we approach code completion, often allowing us to accept suggestions with a simple press of the TAB key, significantly speeding up our workflow. These tools can even generate the necessary tests automatically.
In this context, is Test-Driven Development (TDD) becoming obsolete? Or does the practice of using tests to drive the design of our applications still hold substantial value in the age of AI? Let’s explore the evolving landscape of software development and look at the ongoing relevance of TDD in an AI-enhanced world.

Transcript

1. TDD: Test-Driven Development vs TAB-Driven Development Marit van Dijk Developer Advocate
3. Question •Can we do (strict) TDD with AI?
4. Agenda •What is TDD? •TDD with AI: •A series of experiments •Results
5. Test-Driven Development
6. Test-Driven Development 1. Write a failing test (Red)
7. Failing test Never trust a test you haven’t seen fail https://maritvandijk.com/use-testing-to-develop-better-software-faster/
8. Test-Driven Development 2. Write just enough code pass the test (Green)
9. YAGNI You Ain't Gonna Need It
10. Test-Driven Development 3. Improve the code, without changing its behaviour (Refactor)
11. TDD is not about testing
13. Think about • Intended behavior
14. Think about • Intended behavior • How to test it
16. Code completion is here • Auto completion while typing • with syntax highlighting
17. Tab-Driven Development Cloud completion Local completion
18. Side note • You can change the key used for completion 🤫
19. Accept partial suggestion macOS Windows / Linux Accept suggestion ⇥ TAB Accept line ⌘→ END Accept word ⌥→ Ctrl+→
20. Tab-Driven Development For the purpose of this talk: Tab-Driven Development = AI assisted programming
21. Generate tests
23. Generate code • Can we generate code from tests?
24. Tools
25. JetBrains AI Assistant
 —
26. Plugin
27. JetBrains Junie
 —
29. Plugin
30. Junie A coding agent by JetBrains
31. LLM demo
32. A series of experiment
33. Experiments: "TDD" with AI Assistant • No equivalent "Generate code from tests"
34. Experiments: "TDD" with AI Assistant • No equivalent "Generate code from tests" • Find the best way to do so
35. Experiments: "TDD" with AI Assistant • Problem: FizzBuzz Tried different options: 1. Strict TDD (Red - Green - Refactor) 2. From Chat 3. Inline
36. Experiment: Strict TDD • Problem: FizzBuzz 1. Write a test 2. Ask for implementation 3. (Ask for) refactoring
37. Experiment: Strict TDD • Problem: FizzBuzz 1. Write a test 2. Ask for implementation 3. (Ask for) refactoring • Code completion kicks in for known problem.
38. Experiment: From Chat • Problem: FizzBuzz 1. Ask AI Assistant about FizzBuzz
39. Experiment: From Chat • Problem: FizzBuzz 1. Ask AI Assistant about FizzBuzz 2. Ask for list of test cases
40. Experiment: From Chat • Problem: FizzBuzz 1. Ask AI Assistant about FizzBuzz 2. Ask for list of test cases 3. Ask for tests in Java
41. Experiment: From Chat • Problem: FizzBuzz 1. Ask AI Assistant about FizzBuzz 2. Ask for list of test cases 3. Ask for tests in Java
42. Experiment: From Chat • Problem: FizzBuzz 1. Ask AI Assistant about FizzBuzz 2. Ask for list of test cases 3. Ask for tests in Java 4. Ask for implementation
43. Experiment: Inline • Problem: FizzBuzz 1. Ask AI Assistant for test cases based on description
44. Experiment: Inline
45. Experiment: Inline • Problem: FizzBuzz 1. Ask AI Assistant for test cases based on description 2. Ask to implement code that passes the tests
46. Experiment: Inline • Wrong place in file • Easy to fix
47. TDD workflow with AI Assistant 1. Create test(s) 2. Ask AI Assistant Chat for code that passes the tests 3. Apply code from chat • Works with simple problems
48. Result Can do (strict) TDD with AI Assistant Can apply code directly (no copy-paste)
49. Experiment: Strict TDD • Problem: FizzBuzz • Strict TDD with Junie (Red/Green/Refactor): 1. Write a test 2. Ask Junie for implementation
50. Experiment: Strict TDD • Provide Junie with Guidelines first
51. Experiment: Strict TDD
52. Experiment: Strict TDD
53. Experiment: Strict TDD
54. Experiment: Strict TDD
55. Experiment: Strict TDD
56. Experiment: Strict TDD • Problem: FizzBuzz • DEMO
57. Experiment: Strict TDD • Problem: FizzBuzz • Junie creates the right code in the right location • Junie runs the tests & makes sure they pass
58. TDD workflow with Junie 1. Create tests 2. Ask Junie for code that passes the tests 3. Repeat • Junie creates the right code in the right location • Junie runs the tests & makes sure they pass
59. Result Can do (strict) TDD with Junie Tips: • Provide guidelines • Take turns
60. Toy problems • One class • Limited functionality • What about multiple classes?
61. Experiment: Multiple files • Problem: Calculator • Tests require multiple classes + interface
62. Experiment: Multiple files • Problem: Calculator • Can generate multiple classes • but not always...
63. Experiment: Multiple files • Problem: Calculator • Provide set of tests • Ask for implementation
65. Experiment: Multiple files
66. Experiment: Multiple files
67. Experiment: Multiple files • DEMO
68. Experiment: Multiple files • Problem: Calculator • Junie creates the right files in the right location • Junie runs the tests & makes sure they pass
69. Toy problems • What about actual apps?
70. Experiment: Build a small app test first • Ask Junie to generate implementation based on tests
71. Experiment: Build a small app test first • We need an example • With tests • Let's reverse engineer!
72. Example: Build a small app • Create Spring Boot project • Ask Junie to implement small app
75. Example: Build a small app • Add task • Mark task done • Delete task
77. The code is tested • Junie added tests • Junie ran tests • How good are the tests?
78. Example: Build a small app Let's use Junie to check up on Junie...
79. Example: Build a small app Let's use Junie to check up on Junie... 1. Add coverage with JaCoCo
81. Example: Build a small app Let's use Junie to check up on Junie... 1. Add coverage with JaCoCo
82. Example: Build a small app Let's use Junie to check up on Junie... 1. Add coverage with JaCoCo 2. Add mutation testing with Pitest
84. Example: Build a small app Let's use Junie to check up on Junie... 1. Add coverage with JaCoCo 2. Add mutation testing with Pitest
86. Example: Build a small app Use Junie to increase test coverage
88. Example: Build a small app Use Junie to increase test coverage
89. Experiment: Rebuild the small app • Remove implementation • Ask Junie to build app based on tests • Might get it right based on project history (?)
90. Experiment: Rebuild the small app • Create empty project • Copy in the tests • Ask Junie to build app based on tests
91. Experiment: Rebuild the small app
92. Experiment: Rebuild the small app
94. Experiment: Rebuild the small app • Tests don't always pass on first try
95. Experiment: Rebuild the small app • Tests don't always pass on first try • Junie makes fixes and tries again
96. Experiment: Rebuild the small app • Working app • Not always the same
97. Experiment: Rebuild the small app • Helps to provide guidelines
98. Experiment: Rebuild the small app • Helps to provide guidelines
99. Experiment: Build small app test-first • DEMO
100. Results • Tests are not enough • Needs guidelines
101. We need more than passing tests • Does the application start up? • What about security? • Performance? • Code quality? • Etc
102. A learning curve • Which requirements • Which guidelines • Which prompts • Which models
103. Can we do (strict) TDD with AI? • While it's possible to do (strict) TDD with AI • Generating code based on tests alone does not give (me) reliable results
104. There is still value in • Defining requirements & guidelines • Fast feedback, like tests • Small(er) increments
107. https://x.com/karpathy/status/1617979122625712128
108. Slides & More