Laravel Pipeline & Darkwood Flow

Two execution models for composing PHP steps. A Darkwood comparison of closure composition, explicit next middleware, dataflow packets, drivers and concurrent waits, with clear limits on what the demonstration proves.

Canonical interactive presentation: https://slides.darkwood.com/slides/laravel-pipeline-flow

Transcript

1. Darkwood · PHP 8.5 · Laravel · Flow Laravel Pipeline & Darkwood Flow Two execution models for composing PHP steps Symfony Console demo · demo:pipeline:compare Composition ≠ Execution @matyo91
2. The overloaded word "Pipeline" means composition. Not execution. HTTP middleware Unix pipes CI/CD stages Dataflow / ETL Same word. Different execution contracts. 01 | pipeline @matyo91
3. Core idea A pipeline defines how work is composed. The runtime defines how that work executes. Ask: "what are the steps?" and "how does it execute?" 02 | thesis @matyo91
4. Laravel Pipeline The closure onion array_reverse array_reduce One passable Pipes folded outer-first Nested closures One sync call stack $pipeline = array_reduce( array_reverse($pipes), $carry, $destination ); return $pipeline($passable); 03 | laravel-onion @matyo91
5. Slide 5 · 1/1 · représentation statique Laravel execution A(B(C(destination))) through A, B, C Pipe A outermost Pipe B Pipe C destination $next = the rest of the chain, already folded. 04 | laravel-diagram Version interactive : https://slides.darkwood.com/slides/laravel-pipeline-flow @matyo91
6. Why $next matters The middleware contract BEFORE AFTER Run logic, then $next($passable) $response = $next($passable), then transform SHORT-CIRCUIT Return without calling $next stops → chain Perfect for one HTTP request · one passable · one call stack 05 | next @matyo91
7. The demo Same pipeline. Two runtimes. Fetch → Hash 0.8s 0.3s 0.2s 0.5s → Save 0.6s php bin/console demo:pipeline:compare In-memory fixture · controlled timers · not a benchmark 06 | demo @matyo91
8. Common misconception Fibers do not make blocking code async. STILL BLOCKS THE THREAD USEFUL OVERL AP NEEDS sleep() / usleep() Event-loop timers Blocking HTTP clients Non-blocking I/O integration file_get_contents() Driver that schedules waits Demo uses AmpDriver + Amp\delay() — not raw Fibers alone. 07 | fibers @matyo91
9. Darkwood Flow Nodes · packets · driver Flow Job Ip Transformation node T1 → T2 Instruction packet Driver await() Runtime scheduler Completion barrier · returns void 08 | flow-model @matyo91
10. Slide 10 · 1/1 · représentation statique Flow execution Five packets · one pipeline graph Submit Ip task 1 Ip task 2 FetchJob HashJob SaveJob Collector Ip task 5 foreach ($tasks as $task) { $flow(new Ip($task)); } $flow->await(); 09 | flow-diagram Version interactive : https://slides.darkwood.com/slides/laravel-pipeline-flow @matyo91
11. Slide 11 · 1/2 · représentation statique AmpDriver Event-loop scheduling · one thread Diagramme : notation source conservée (rendu graphique indisponible). sequenceDiagram participant T1 as Task1 participant Loop as RevoltEventLoop participant T2 as Task2 T1->>Loop: Amp delay 0.8s T2->>Loop: Amp delay 10 | amp Version interactive : https://slides.darkwood.com/slides/laravel-pipeline-flow @matyo91
12. Slide 11 · 2/2 · représentation statique AmpDriver Event-loop scheduling · one thread 0.3s Note over T1,T2: timers registered Loop->>T2: resume 0.3s Loop->>T1: resume 0.8s ✓ concurrent waits ✗ parallel CPU 10 | amp Version interactive : https://slides.darkwood.com/slides/laravel-pipeline-flow @matyo91
13. Slide 12 · 1/2 · représentation statique Measurements Scheduling behavior — not a benchmark SYNC ONION ~2.42 s delays accumulate (Σ 2.4s) FLOW + AMP ~0.81 s delays overlap (max 0.8s) 11 | timing Version interactive : https://slides.darkwood.com/slides/laravel-pipeline-flow @matyo91
14. Slide 12 · 2/2 · représentation statique Measurements Scheduling behavior — not a benchmark Fetch phase only T1 T2 T3 Sync T4 T5 T1f T2f Flow T3f T4f T5f 000s000s000s000s000s000s000s000s000s 11 | timing Version interactive : https://slides.darkwood.com/slides/laravel-pipeline-flow @matyo91
15. What the demo proves Evidence, not hype Independent timer waits can overlap on one thread Multiple Ip packets traverse the same stages Logs interleave — task #2 fetch starts before task #1 fetch ends Identical hashes and output files (PHPUnit verified) [FLOW][task #1][fetch] start [FLOW][task #2][fetch] start [FLOW][task #5][fetch] start [FLOW][task #4][fetch] end 12 | proves @matyo91
16. What it does NOT prove Stay precise Not universal speed Not slow Laravel Flow is not always faster Pipeline is not poorly designed Not parallel CPU Not async HTTP Single OS thread throughout Blocking I/O would still block 13 | limits @matyo91
17. Slide 15 · 1/3 · représentation statique Comparison Different problems · different tools Laravel Pipeline Simple onion (~30 LOC core) Sync · one passable Explicit $next Returns from then() Middleware / commands 14 | compare Version interactive : https://slides.darkwood.com/slides/laravel-pipeline-flow @matyo91
18. Slide 15 · 2/3 · représentation statique Comparison Different problems · different tools Darkwood Flow Nodes · drivers · dispatchers Async-capable · many Ips await() returns void Terminal collector required Overlapping wait-heavy work 14 | compare Version interactive : https://slides.darkwood.com/slides/laravel-pipeline-flow @matyo91
19. Slide 15 · 3/3 · représentation statique Comparison Different problems · different tools Sync demo: 5 sequential onions · Flow demo: 5 concurrent packets 14 | compare Version interactive : https://slides.darkwood.com/slides/laravel-pipeline-flow @matyo91
20. Slide 16 · 1/1 · représentation statique What Flow can learn Laravel got ergonomics right AWAIT()Returns void — easy to forget RESULTSTerminal collector, not thenReturn() IPManual wrapping per input BUNDLESymfony services config is empty IDEAS sink() · runAll() · TraceSubscriber · hrtime() delays 15 | api-lessons Version interactive : https://slides.darkwood.com/slides/laravel-pipeline-flow @matyo91
21. Choosing the model Match runtime to workload L ARAVEL PIPELINE DARK WOOD FLOW One passable per call Many independent packets Middleware / command intercept Non-blocking driver waits Short-circuit via $next Per-node error handling Sync, fast steps Runtime flexibility They coexist. Different layers, different models. 16 | choose @matyo91
22. Takeaway Both are pipelines. They are not the same machine. Always ask: How does it execute? Onion or Dataflow — composition ≠ execution @matyo91
23. Resources Go deeper darkwood/content/laravel-pipeline — runnable demo demo:pipeline:sync · demo:pipeline:flow · demo:pipeline:compare darkwood/flow — dataflow component Blog article — execution models deep dive Re-run timings on your machine before citing exact numbers in slides. 17 | resources @matyo91