What is “Small AI”? - Small AI on the rise - Case studies 1. Protecting Sensitive Data with Small Language Models 2. Small Image Models for Environmental Monitoring 3. Small AI Teachers 4. Small AI for Medical Imaging - Strategic recommendations and takeaways
Nathan Kirk Large AI Small AI Trillions of parameters Millions–billions of parameters Expensive, energy intensive Cost-efficient, relatively “green” Broad knowledge Task/domain focused Training demands huge, diverse datasets Trained on limited datasets for intended usage Remotely hosted, requiring GPUs Runs locally, on your workstation (“Edge AI”) Risky: data leaves system Private: data doesn’t leave workstation Difficult to adapt to bespoke needs Easily customisable • A “small AI” model is compact, efficient and importantly, task- specific. • In contrast to LLMs (large language models) like ChatGPT and Grok, which are built to work in a very general domain • Examples include Mistral 7B (small LLM), YOLO (small image model), Voice Chat AI (small voice model) and DistilGPT-2 (tiny LLM)
| Nathan Kirk Tech Trend: Bigger isn’t always better Leading tech companies have recently released lightweight AI models, e.g., IBM has the Granite Models, Google has Gemma, and Microsoft has Phi-3(mini) Why now? - Compute costs are becoming unsustainable from huge data centres - Regulations now demand data sovregnity (EU AI Act, UK AI Bill) - Users want more reliable models, which hallucinate less often
| Nathan Kirk Tech Trend: Bigger isn’t always better Leading tech companies have recently released lightweight AI models, e.g., IBM has the Granite Models, Google has Gemma, and Microsoft has Phi-3(mini) Why now? - Compute costs are becoming unsustainable from huge data centres - Regulations now demand data sovregnity (EU AI Act, UK AI Bill) - Users want more reliable models, which hallucinate less often
Case Study 1 Protecting Sensitive Legal Data with SLMs Problem: Law firms manage highly-sensitive data Uploading documents to LLMs or cloud-based AI is not a viable option. Solution and Outcomes: - Deploy an on-premise, local SLM - For example, DistilGPT-2 (a SLM with < 100M parameters) fine-tuned on legal documents - Functional on a regular workstation - Capable of summarising contracts, extracting clauses or drafting briefs - Compliance assured: no client data leaves the firm - Time saved: Paralegals spend less time scanning long documents - Cost efficiency: Eliminates expensive reliance on external APIs - Trust: Clients assured data is secure
Case Study 2 Small Vision Models for Environmental Monitoring Problem: Monitoring forests, farms or oceans requires constant camera or sensor data. Sending collected visual data is expensive and requires continuous reliable internet connection Solution and Outcomes: - Deploy tiny image classifiers on drones, or IoT cameras - Trained on specific hazards - Detect pests, or plastic waste on device, e.g., via ‘MobilePlantViT’ - Real-time alerts: problems spotted instantly, action taken faster - Lower cost: Only useful signals are sent to the cloud - Sustainability: more data to empower conversation at scale
Case Study 3 Small AI Teachers Problem: Quality, personalized learning is hard to deliver at scale. Many rely on online Webinars. However, this requires high-quality, steady connectivity. Solution and Outcomes: - Small AI language model for content generation, feedback and assessment - Use existing infrastructure like WhatsApp on low-end phones - Learning gains at low cost: “Rori” AI Math tutor costs $5/student/year - Inclusion at scale: Personalized education reaching remote locations in local language - Educational and economic benefits
Case Study 4 Small AI for Medical Imaging Problem: Radiology and ultrasound often require AI support in image enhancement or anomaly detection Sending patient scans to the cloud raises privacy concerns, and importantly, adds latency Solution and Outcomes: - Compact CNN (convolutional NN) directly on MRI/ultrasound for on device inference and image enhancement in real-time - Recently Siemens and NVIDIA partnered successfully - Faster diagnosis: scan interpretation time cut to seconds - Reduced diagnostic errors: small anomalies detected - Cost saving and regulatory peace- of-mind: No cloud storage
Nathan Kirk Small AI works best when tackling a hyper-local, specific task, e.g., crop-disease detection. It builds on existing infrastructure, e.g., smart phones, WhatsApp. Public: Provides funding, datasets, privacy/safety regulation Private: Builds and trains lightweight models with creative solutions Designing for mobile/“edge” devices and off-line functionality is crucial in applications where connectivity is sparse and unreliable Effective small AI deployment thrives in public-private partnerships.
nuanced approach to AI deployment is underway. Strategic implementation of smaller, specialized models for appropriate case use can yield: - Comparable performance - Matching model capabilities to business requirements - Data regulation compliance - Transparency, encouraging trust Organisations should: - Hire a statistical consultant to thoughtfully identify areas for AI deployment - Dynamically allocate AI resources based on task importance
Further Reading: - ‘Databricks’ $100B valuation using speedy AI and specialized models - The Rise of Small Giants - 601 Real-World AI Use Cases - Small AI Firms Challenge Big Tech (DeepSeek) - Tiny AI - Think Big with Small AI Please approach me to chat further throughout the next few days!
nmkirk
tammyeverts
erikaheidi
bluesmoon
cromwellryan
philnash
chriscoyier
kastner
sachag
kevinliebholz
jeffersonlam
addyosmani
dougneiner
