Building Locally, Build with AI, Build at Scale Bethany Jepchumba | @bethanyjep 

• Cloud Advocate (DevRel) at Microsoft • Works with NGOs & Comunities, currently ICT Director at TOFA • I am a hobbist, I enjoy trying out new hobbies I occasionally blog at bethany-jep.com Bethany Jepchumba Microsoft confidential 2 

3 The concept of AI and its adoption in Kenya 

and the government In the news, of late…. Images from the citizen & business daily 4 

and building locally The concept of LLMs…. 5 

How language models work Natural language input Model Tokens Probability distribution Natural language output Decoding + Post-processing Get results Pre-processing 

Foundry Local winget install Microsoft.FoundryLocal Design, customize and manage AI applications and agents on-device Preview Microsoft confidential 7 brew tap microsoft/foundrylocal brew install foundrylocal Installation Privacy and Security Latency Cost Efficient Low Bandwidth 

Showcasing Foundry Local demo… 

RAG, Agents and MCP Giving your LLMs some powers, sort of…. 12 

BYOD -> Retrieval-Augmented Generation (RAG) 

Showcasing RAG on Kenya’s constitution 

What is an AI agent? LLM Instructions Tools Agent + + An AI agent is a micro-service that takes unstructured messages, optionally invokes other APIs and returns messages/action 1 2 3 Input System events User messages Agent messages 1 Tool calls Knowledge Actions Memory 2 Output Agent messages Tool results 3 

Unlike previous tools, AI Agent can… reason, act and learn 

MCP Model Context Protocol - Easier to give context to models - Optimized communication between LLMs, external tools, data sources and applications - Uses a client-server model for interactions 

Showcasing Demo: Microsoft Learn MCP with GH Copilot 

Showcasing Demo: Spotify MCP 

Security, monitoring and evaluations. Going global…. 20 

Generative AI Development Lifecycle Hypo thesis Find LLMs Try prompts Manage Customize Design Managing PREPARE FOR APP DEPLOYMENT ADVANCE PROJECT BUSINESS NEED Deploy LLM App/UI Quo ta and cost management REVERT PROJECT C ontent Filtering M onitoring SEND FEEDBACK Prompt Engine ering or Fine-tuning Evalua tion Exceptio n Handling Retrieval Augmente d Generation Prote ct & govern 

How do you typically test if a model or application is working as intended? 

Mitigation layers Application Platform 

Evaluators in Azure AI Foundry RAG Retrieval Document Retrieval Groundedness Relevance Response Completeness General Purpose Fluency Coherence QA Safety & Security Violence Sexual Self-harm Hate and Unfairness Ungrounded Attributes Code Vulnerability Protected Materials Content Safety Agents Intent Resolution Task Adherence Tool call accuracy 

Foundry Observability Dashboard Azure AI Foundry Observability, integrated with Azure Monitor Application Insights, enables you to continuously monitor your AI applications 

Showcasing Demo: AI Evaluators 

AI Use Cases 

From english to xx language in a single click 

UlizaLlama - 7 B parameter transformer, instruction-tuned for Swahili and English dialogue and QA tasks - Fine-tuned via LoRA (Low-Rank Adaptation), then merged back into the base model so it can run stand- alone without adapters - Designed for real-world contexts in healthcare patient support, agriculture Q&A via apps, legal aid bots, educational assistants, tourism tools, public services, retail bots 29 

Fastagger - the AI runtime that extends “seeing, deciding, and acting” beyond data centers, right onto entry-level Android phones, IoT tablets, and industrial gateways. - Built as a plug-in to Safaricom’s M-PESA Business App, Auni automatically parses mobile-money PDF statements offline and provides personalized customer insights to MSMEs. Launched March 2024, it hit over 2,500 MSME users within weeks, and now powers AI insights across 1 million+ M-PESA devices. 30 

AI Agent workflows can enhance efficiency, accuracy, and customer satisfaction Key Use Cases Across Industries • Assists employees in booking business trips • Integrates with Tripadvisor, Outlook, and SharePoint • Books via Teams chat or email • Uses OCR to gather receipts • Automates expense report submission and tracking Travel Booking & Expense Management • Personalized onboarding assistant for new hires • Uses LLMs grounded in HR data from SharePoint • Provide relevant training materials • Schedule orientations and set up software accounts • Monitor task completion and ensure efficient onboarding Employee Onboarding • Diagnoses issues by referencing history and product manuals • Provides tailored solutions or escalates through automated workflows • Creates tickets and schedules follow-ups • Updates CRM records, enhancing future support Personalized Customer Support • Analytics data from data lake and data warehouse • Responds to user requests in natural language • Generates insights, visualization, and sends via Teams or email • Automates data handling for real-time, effortless decision-making Data Analytics and Reporting 

Resources - Learning resources on GenAI, agents and MCP - Getting started with Azure AI - Links to the different tools – foundry local, ai toolkit on VSCode, GitHub Models and GitHub Copilot - Links to different articles and use cases https://bethany-jep.com/events/ai- summit-kenya-2025/ 

Thank You