AI Agent Orchestration using Docker

The slide deck used during API Days Australia. This presentation on AI Agent Orchestration using Docker was part of the Kubernetes + AI track.

https://www.apidays.global/events/australia

The presentation covered key features of Docker Model Runner to enable offline Agentic AI development and testing capabilities

Transcript

1. Nilesh Gule @nileshgule AI Agent Orchestration using Docker
2. $whoami { “name” : “Nilesh Gule”, “role” : “Senior Cloud Solutions Architect at Avanade” “website” : “https://www.HandsOnArchitect.com", “github” : “https://GitHub.com/NileshGule" “twitter” : “@nileshgule”, “linkedin” : “https://www.linkedin.com/in/nileshgule”, “YouTube” : “https://www.YouTube.com/@nilesh-gule” “likes” : “Technical Evangelism, Cricket”, }
3. Challenges in building Agentic AI Apps • Dependency on hosted models • Quotas based on service tiers • Unpredictable costs • Additional setup / infrastructure to build and test Agentic apps
4. AI Grocery Shopper Meal Planner Agent Inventory Agent Budget Agent Shopper Agent Generate meal plans and shopping list based on preferences and dietary restrictions Compare inventory items and build revise shopping list Optimize items based on budget Optimize items based on categories
5. /demo Docker Model Runner
6. Docker Model Runner Architecture Docker Model Runner Design
7. Docker Model Runner Endpoints • From Containers - http://model-runner.docker.internal/ • From host processes - http://localhost:12434/ • Create model - POST /models/create • List models - GET /models • Get a model - GET /models/{namespace}/{name} • Delete local model - DELETE /models/{namespace}/{name}
8. OpenAI Endpoints • List Models - GET /engines/llama.cpp/v1/models • Retrieve models - GET /engines/llama.cpp/v1/models/{namespace}/{name} • List chat completions - POST /engines/llama.cpp/v1/chat/completions • Create completions - POST /engines/llama.cpp/v1/completions • Create embeddings - POST /engines/llama.cpp/v1/embeddings
9. Summary - Docker Model Runner Docker Model Runner General Availability | LinkedIn
10. Resources • Docker Docks - Model Runner • Docker Docs - Use AI models in Compose • Docker Model Runner Design and What’s Next • Docker Model Runner General Availability Annoncement
11. Nilesh Gule ARCHITECT | MICROSOFT MVP “Code with Passion and Strive for Excellence” nileshgule @nileshgule Nilesh Gule NileshGule www.handsonarchitect.com https://www.youtube.com/@nilesh-gule
12. Source Code & slide deck AI Grocery Shopper https://github.com/NileshGule/AI-Grocery-Shopper https://speakerdeck.com/nileshgule/ https://www.slideshare.net/nileshgule/
13. Q&A