From No-Life Testing to Behavioral Testing

AI made writing tests cheap. Choosing what deserves an oracle is now the hard part. A Darkwood technical presentation on behavioral tests, meaningful oracles, runtime independence and Darkwood Flow jobs.

Canonical interactive presentation: https://slides.darkwood.com/slides/from-nolife-testing-to-behavioral-testing

Transcript

1. Darkwood · PHP · PHPUnit · agents · Flow From No-Life Testing to Behavioral Testing AI made writing tests cheap. Choosing what deserves an oracle is now the hard part. Jobs ≠ Drivers @matyo91
2. Opening Most unit tests are protecting the wrong thing. Not because PHPUnit is weak. Because the oracle chose implementation over meaning. 01 | provocation @matyo91
3. Fabien Potencier · Sebastian Bergmann Your AI writes tests. Are they worth anything? Liability Oracle Weak tests under agent velocity What decides “correct” 02 | the question @matyo91
4. Slide 4 · 1/2 · représentation statique Scope Not a tutorial. Not a bake-off. ✗ PHPUnit basics ✗ TDD sermon ✗ ReactPHP vs Amp ✓ Oracles that survive refactoring 03 | scope Version interactive : https://slides.darkwood.com/slides/from-nolife-testing-to-behavioral-testing @matyo91
5. Slide 4 · 2/2 · représentation statique Scope Not a tutorial. Not a bake-off. Jobs = behavior · Drivers = furniture 03 | scope Version interactive : https://slides.darkwood.com/slides/from-nolife-testing-to-behavioral-testing @matyo91
6. The economics flipped Writing got cheap. Thinking got scarce. BEFORE AI WITH AGENTS Cost of writing ≫ cost of choosing oracles Cost of writing ≪ cost of choosing oracles Volume without oracles is false insurance. 04 | economics @matyo91
7. Scarce skill Not more tests. Better oracles. Which behaviors deserve insurance? Which assertions survive a refactor? Which doubles hide a design smell? Would green still pass if meaning broke? 05 | judgment @matyo91
8. Vocabulary No-life testing A green suite that pins implementation, call order, and runtime plumbing — while leaving business meaning unasserted. Alive in CI. Dead as insurance. 06 | no-life @matyo91
9. Bad · 01 · ImplementationCoupledTest Mocks all the way down $parse->expects($this->once()) ->method('__invoke') ->with($raw) ->willReturn($parsed); // … $this->assertTrue(true); // Green. Never asserted meaning. Broken real ParseJob 07 | bad-mocks → suite stays green. @matyo91
10. Bad · 02 · RuntimeCoupledTest Testing the furniture public function await(array &$stream): void { ++$this->counter->awaitCalls; $this->inner->await($stream); } $this->assertSame(1, $counter->awaitCalls); Correct excerpt + driver swap 08 | bad-runtime → still red. Wrong claim. @matyo91
11. Bad · 03 · CoverageTheaterTest Shape without meaning $this->assertInstanceOf(ExcerptResult::class, $result); $this->assertNotNull($result->title); $this->assertIsString($result->excerpt); // Never asked: is the excerpt faithful? Coverage theatre. Agents love it. 09 | bad-theatre @matyo91
12. Same workflow · three lies False insurance, catalogued Call order await() Shape Mocks erase Jobs Pins the Driver Green on nonsense What should we assert instead? 10 | three-lies @matyo91
13. Slide 12 · 1/1 · représentation statique Flow · Jobs vs Drivers Behavior lives in Jobs DocumentRequest Business behavior — test this FetchJob Runtime furniture — do not pin scheduled by FiberDriver AmpDriver ParseJob ReactDriver ExcerptJob ValidateJob ExcerptResult 11 | jobs-vs-drivers Version interactive : https://slides.darkwood.com/slides/from-nolife-testing-to-behavioral-testing @matyo91
14. Workflow · not furniture Input · Steps · Output Reque st → Fetch → Parse → Excer pt → Valida te → Result yield new FetchJob($source); yield new ParseJob(); yield new ExcerptJob($maxExcerptLength); yield new ValidateJob($minExcerptLength); Same chain. Inject FiberDriver or AmpDriver. Jobs do not care. 12 | workflow @matyo91
15. Good · JobBehaviorTest Observable transforms $parsed = (new ParseJob())(new RawDocument('doc://1', $html)); $this->assertSame('Hello Flow', $parsed->title); $this->assertSame( 'Jobs transform packets. Drivers schedule them.', $parsed->body, ); No Flow. No Driver. Still business. 13 | job-oracle @matyo91
16. Good · WorkflowBehaviorTest Faithfulness, not wiring $result = FlowCollector::run($flow, new Ip( new DocumentRequest('fixture://article'), )); $this->assertSame('Darkwood Flow Notes', $result->title); $this->assertStringContainsString( 'Jobs transform information packets', $result->excerpt, ); Oracle = excerpt meaning. Not call order. 14 | workflow-oracle @matyo91
17. Slide 16 · 1/2 · représentation statique Good · MultiDriverBehaviorTest Same oracle. Two runtimes. FiberDriver Green AmpDriver Green Same behavioral oracle await / driver spies Red after swap yield 'fiber' => [new FiberDriver()]; yield 'amp' => [new AmpDriver()]; // identical assertions on title + excerpt 15 | multi-driver Version interactive : https://slides.darkwood.com/slides/from-nolife-testing-to-behavioral-testing @matyo91
18. Slide 16 · 2/2 · représentation statique Good · MultiDriverBehaviorTest Same oracle. Two runtimes. Survive the driver swap → you tested the product. 15 | multi-driver Version interactive : https://slides.darkwood.com/slides/from-nolife-testing-to-behavioral-testing @matyo91
19. Beyond Flow The runtime will change. Your architecture should not. Sequential PHP ReactPHP Call stack ≠ outcome Promises / loop furniture Amp Framework X Amp types ≠ domain packets HTTP adapter ≠ product If the suite breaks while product meaning is unchanged — you tested furniture. 16 | runtime-independence @matyo91
20. Doubles under agents Stub the port. Don’t mock the Jobs. DO DON’T InMemoryDocumentSource Unit-test App\Job\* composer test:good Mock DriverInterface Assert job call order Treat Bad as insurance $source = new InMemoryDocumentSource([ 'fixture://article' => $html, ]); 17 | stubs @matyo91
21. Guillaume Moigneu · Agent Skills Pressure-test the decision. Don’t generate the suite. Use $pressure-test-testing-decisions on tests/Bad/ImplementationCoupledTest.php — should we keep, rewrite, or delete it? Oracle Furniture Coverage theatre Driver swap 18 | pressure-test @matyo91
22. Useful grill · one at a time Defend the insurance claim Should this be tested at all? Would this fail for the right reason? Does this assertion survive a refactoring? Are these mocks hiding a design problem? Fiber → Amp: would this fail — and should it? Suite green, product wrong — which test lied? 19 | grill @matyo91
23. Slide 21 · 1/2 · représentation statique One habit change Rewrite the brief WEAK “Add unit tests. Aim for high coverage.” STRONG “Pressure-test first. Oracle on Job / workflow meaning. Stub the port. No driver spies.” 20 | brief Version interactive : https://slides.darkwood.com/slides/from-nolife-testing-to-behavioral-testing @matyo91
24. Slide 21 · 2/2 · représentation statique One habit change Rewrite the brief Agent proposes a test Pressure-test furniture / theatre Job meaning composition runtime claim Delete or park under Bad JobBehaviorTest WorkflowBehaviorTest MultiDriverBehaviorTest 20 | brief Version interactive : https://slides.darkwood.com/slides/from-nolife-testing-to-behavioral-testing @matyo91
25. Companion · content/nolife-tests Clone it. Break it. Decide. composer install composer test:good # behavioral insurance composer test:bad # educational false insurance php bin/excerpt.php fiber php bin/excerpt.php amp workflows 21 | repository tests Good/Bad skills/ fixtures @matyo91
26. Homework Break ParseJob on purpose. Bad → stays green Good → goes red Worth more than another coverage report. 22 | experiment @matyo91
27. Honesty What this does not prove 23 | limits No sync run(Ip) yet Stubs at ports still useful Driver CI ≠ app suite Not an Amp vs Fiber bench AI can write good tests Risk = volume without oracles @matyo91
28. Closing The value of a test is not that it passes. It is that it fails for the right reason. Jobs → Oracles → Insurance @matyo91
29. Slide 26 · 1/2 · représentation statique Resources Go deeper Article — From No-Life Testing to Behavioral Testing content/nolife-tests — composer test:good Darkwood Flow — darkwood-com.github.io/flow Potencier — Weak tests are now a liability Bergmann — Seeing the Truth: Test Oracles Moigneu — pressure-test-decisions / testing specialization 24 | resources Version interactive : https://slides.darkwood.com/slides/from-nolife-testing-to-behavioral-testing @matyo91
30. Slide 26 · 2/2 · représentation statique Resources Go deeper Your AI writes tests. Make sure they are worth claiming. 24 | resources Version interactive : https://slides.darkwood.com/slides/from-nolife-testing-to-behavioral-testing @matyo91