Elevate AI Community Tech talk

Transcript

1. 1 ? => Context Engineering => ? Rahul Krishnan R A
2. 2 Three Core Disciplines of LLM Intelligence Prompt Engineering - shaping the model’s immediate behavior Context Engineering - organizing the context before starting task Memory Engineering - solves long-term continuity problems tasks
3. 3 LLMs Are No Longer Models — They Are Like Operating Systems LLM RAM Context Window CPU Disk File System (+embeddings) https://x.com/karpathy/status/1723140519554105733?lang=en Modern LLM functions like an operating system, not just a model. • Model: think of it more like a “platform” that sits at the center of many components — retrieval, external memory, plugins, agents, tools • Disk analogy: highlights that there’s a shift from ephemeral stateless input/output toward integrating persistent storage (embeddings, external knowledge bases, memory modules) • Context window: the “working memory” (RAM)
4. 4 What is An AI Agent An AI Agent is an autonomous computational entity endowed with four core capabilities: 1. Cognitive reasoning, driven by large language models. 2. Environmental perception, enabled through multimodal inputs. 3. Actionable intelligence, achieved through seamless tool integration. 4. Persistent memory, the most critical component, which supports continuous learning, long-term adaptation, and context-aware behavior.
5. 5 AI Agent Memory Tools (Action) Reasoning Abilities Perception What is an AI Agent ?
6. 6 Prompt Engineering Instructions Few Shot Examples Chain of Thought Temperature Tuning ReAct Input( message ) User Agent Response Developer [ “Can you recommend some good holiday destinations for the summer?” ] This is Prompt
7. 7 Context Engineering
8. 8 Context Engineering https://github.com/humanlayer/12-factor-agents/blob/main/content/factor-03-own-your-context-window.md Context engineering is the next step, where you design and optimize the full context instead of relying on simple prompts.
9. 9 Instructions Knowledge Tools Types of context Context Window LLM Types of Context Instructions – prompts, memories, few‑shot examples, tool descriptions, etc Knowledge – facts, memories, etc Tools – feedback from tool calls
10. 10 Context Engineering - Tool Call Human Tool Call Feedback { "user_message": "What is the weather in London right now?" } { "tool_call": { "name": "get_weather", "arguments": { "location": "London", "unit": "celsius" } } } { "tool_response": { "name": "get_weather", "data": { "location": "London", "temperature": 12, "condition": "Cloudy", "humidity": 65 } } } Tool Call + Feedback Loop 1 User → LLM 2 LLM → Tool Call (JSON Instruction to Tool) (“I can’t answer this directly — call the weather API tool with these arguments.”) 3 TOOL → Returns Data Back to LLM 4 LLM → Final Natural Language Answer (Feedback Loop)
11. 11 [Example 1] Q: Can you recommend good destinations for winter? A: Certainly! For winter, consider Switzerland for skiing, Japan for snow festivals, or Canada for mountain adventures. [Metadata] Current Date: May 15, 2025 User Location: London, UK Budget Range: £1500–£3000 Family Members: 2 Adults, 1 Child Preferred Climate: Warm, but not extreme humidity [Constraints] - Limit to destinations with direct flights under 6 hours. - Include a mix of relaxation and cultural options. - Include one “hidden gem” or less-traveled recommendation. [Goal] Deliver 3–5 recommendations with reasoning and grouping by experience type. [User Message] Can you recommend some good holiday destinations for the summer? [Previous Interactions] User previously asked about family-friendly travel options under £2000. User mentioned preference for warm weather and short-haul flights from London. Assistant previously suggested Portugal and Greece for spring travel. [Retrieved Knowledge from Vector Database / API] - "Top Summer Destinations 2025" (Travel+Leisure) - Greece: Santorini and Crete ranked among top 5 for European summer travel. - Bali: Popular for wellness retreats and digital nomads. - Croatia: Rising popularity for affordable coastal getaways. - Canada: Banff and Vancouver for adventure and scenic travel. This is Prompt This is Context
12. 12 Short Term Memory Long Term Memory Coordination Agent Memory Working Memory Semantic Cache Procedural (Instructions) Episodic (Experiences) Semantic (Facts) Shared Memory Transforming stateless AI applications into intelligent agents capable of learning, maintaining continuity, and adapting from interactions across sessions. Key memory types in AI agents
13. 13 Architecture of AI Agent Memory
14. 14 General categories of context engineering Source : https://blog.langchain.com/context-engineering-for-agents/
15. 15 Demo
16. 16 Vibe coding Hackathon with Cursor at UoN