NoLife Models — Exploring Local AI Runtime Infrastructure

A Darkwood technical presentation about model catalogs, local inference runtimes, evaluation, governance and portable traces. NoLife Models illustrates a thin experimental slice through this stack; the PDF links to the canonical presentation for interactive demonstrations.

Canonical interactive presentation: https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure

Transcript

1. Darkwood / local inference infrastructure NoLife Models Exploring Local AI Runtime Infrastructure @matyo91
2. Thesis We are moving from “chatbot APIs” to AI runtime infrastructure. Catalogs, local engines, evaluation harnesses, and exportable traces - the same primitives ops expects from any serious runtime. 01 | thesis @matyo91
3. Agenda Five arcs - one pipeline. Catalogs Runtimes NoLife Models models.dev · modalities · cost · context. Ollama as HTTP surface · future adapters. Compare · benchmark · export. Kronk Governance Outlook Embedded inference - not Ports, traces, “better Ollama”. reproducibility. 02 | agenda Orchestration & embedded edge. @matyo91
4. Part 1 - Catalog plane models.dev is a contract surface for the model economy. Structured metadata: modalities, limits, pricing tiers, reasoning flags. Provider 03 | models.dev → Model record → Tooling & SDKs @matyo91
5. Slide 5 · 1/1 · représentation statique Live - models.dev models.dev Public catalog in the browser. Some sites block iframes; if the frame is blank, open the URL in a new tab. Démonstration externe : models.dev (consulter la version interactive). https://models.dev/ 04 | models.dev · site Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure @matyo91
6. Comparator plane modles - local browser lab for rows from the catalog. Filter, sort by $/M, context, capabilities. 05 | modles @matyo91
7. Slide 7 · 1/1 · représentation statique Live - modles modles Local comparator lab for catalog rows (this deck expects it on port 8001). Démonstration externe : modles (consulter la version interactive). 06 | modles · site Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure @matyo91
8. ↔ Slide 8 · 1/2 · représentation statique Analogy - shipping vs inference Docker ecosystem AI runtime ecosystem Docker Hubmodels.dev ImagesModel artifacts (GGUF, weights) dockerdOllama / Kronk / vLLM… KubernetesAgent & batch orchestration 07 | analogy Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure @matyo91
9. ↔ Slide 8 · 2/2 · représentation statique Analogy - shipping vs inference Docker ecosystem AI runtime ecosystem ObservabilityTraces · exports · benchmarks containerd / OCILocalModelRuntimeInterface 07 | analogy Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure @matyo91
10. Part 2 - Process-local HTTP runtime Ollama exposes models as local HTTP services. Inventory Inference Metadata /api/tags - what’s pulled, which quant. /api/generate - nonstream MVP in NoLife. Load time, eval counts, wall clock - rudimentary SRE signals. 08 | ollama @matyo91
11. Slide 10 · 1/1 · représentation statique Live - NoLife Models Catalog explorer Merged models.dev API + JSON overrides · live filters · modalities · $/M · context windows. Démonstration externe : NoLife catalog (consulter la version interactive). 09 | demo · catalog Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure @matyo91
12. Slide 11 · 1/1 · représentation statique Live - NoLife Models Installed local models Runtime discovery against your daemon - tags, quant, what’s actually callable. Démonstration externe : NoLife Ollama inventory (consulter la version interactive). 10 | demo · ollama Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure @matyo91
13. Part 3 - Reference app NoLife Models - Symfony 8 + Symfony UX DDD / hexagonal core: catalog ports, LocalModelRuntimeInterface, exporters, session-backed results. UX Live Turbo Catalog explorer (Live Component) filters without round-trips. Fast morphing between compare / results. 11 | nolife stack @matyo91
14. Slide 13 · 1/1 · représentation statique ↔ ports ↔ adapters Hexagonal slice Application UserInterface Twig + Forms LiveComponent Application Compare handler Infrastructure Benchmark handler Ollama runtime models.dev cache Session + files Domain LocalModelRuntime 12 | architecture Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure CatalogRepository Exporter @matyo91
15. Runtime port LocalModelRuntimeInterface interface LocalModelRuntimeInterface { /** @return list<LocalModel> */ public function listLocalModels(): array; public function generate(GeneratePromptCommand $command): ModelInferenceResult; } 13 | port @matyo91
16. Slide 15 · 1/1 · représentation statique Live - NoLife Models Compare two models Same prompt · two tags · wall clock + token-ish metrics engineering comparison, not vibes. Démonstration externe : NoLife compare (consulter la version interactive). 14 | demo · compare Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure @matyo91
17. Evaluation plane Latency is cheap to measure. Quality is expensive. Metrics Reasoning Next Duration, load, eval counts - reproducible numbers. Side-by-side text for human triage. Structured rubrics · model grading pipelines. 15 | compare concepts @matyo91
18. Slide 17 · 1/1 · représentation statique Live - NoLife Models Benchmark suite Micro-cases (EN/FR) · deterministic prompts · matrix results orchestration over the same runtime port. Démonstration externe : NoLife benchmark (consulter la version interactive). 16 | demo · benchmark Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure @matyo91
19. Slide 18 · 1/1 · représentation statique Orchestration Benchmark flow Suite definition Model A / Model B Prompt cases 17 | benchmark flow Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure Runtime generate Timing + text capture Matrix + exports @matyo91
20. Slide 19 · 1/1 · représentation statique Live - NoLife Models Exports JSON · CSV · Markdown - stable schema/version fields for downstream tooling. Démonstration externe : NoLife exports (consulter la version interactive). 18 | demo · exports Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure @matyo91
21. Governance Inference without artifacts is faith, not engineering. Audit trail Runtime governance Exports + session IDs - what ran, when, with which tags. Policies on top of the same port: quotas, routing, kill switches. 19 | observability @matyo91
22. Part 4 - Embedded inference Kronk is not “better Ollama”. Ollama: models as local HTTP services. Kronk: push inference into the application process - GGUF via llama.cpp, streaming, OpenAI-compatible ergonomics where it fits. 20 | kronk @matyo91
23. Embedded stack Primitives GGUF llama.cpp Single-file weights - ship like any binary asset. CPU/GPU inference core - predictable foot-print. Streaming OpenAI-shaped APIs Token deltas - UX + back-pressure aware Interchange clients - ecosystem leverage servers. without cloud lock-in. 21 | embedded primitives @matyo91
24. Slide 23 · 1/1 · représentation statique → runtime → evaluation → export End-to-end Catalog Catalog plane Runtime plane 22 | pipeline Version interactive : https://slides.darkwood.com/slides/nolife-local-runtime-infrastructure Evaluation plane Observability plane @matyo91
25. Part 5 - Outlook Multi-runtime clusters · policyaware routing · edge bundles with embedded weights. The same governance patterns (ports, traces, exports) port from Symfony apps to agents to on-device runtimes. 23 | future @matyo91
26. Closing We are moving from chatbot APIs to AI runtime infrastructure. NoLife Models is a thin, opinionated slice through that stack catalogs, local engines, benchmarks, exports - built as a real Symfony product surface. Catalog 24 | outro + Runtime port + Evaluation + Artifacts @matyo91