On Device AI with Google AI Edge

Transcript

1. On-device AI with Google AI Edge Darshan Parikh Software Engineer, Autodesk
2. Agenda • On-device AI • Google AI Edge ◦ Gemini Nano ◦ ML Kit ◦ MediaPipe Solutions • Live demo ◦ MediaPipe Tasks ◦ MediaPipe Model Maker • Summary
3. On-device AI with Google AI Edge
4. On-device AI • AI processing and computations performed on a device • For ex, smartphone, wearable, IoT device etc. • Does not rely on cloud servers
5. On-device AI • Advantages ◦ Low latency ◦ Privacy ◦ Offline functionality ◦ Reduced reliance on cloud infrastructure • Challenges ◦ Limited computational power compared to cloud ◦ Model size limitations
7. On-device AI with Google AI Edge
8. Google AI Edge • Platform that enables developers to build and deploy AI models directly on devices • On-device AI for mobile, web, and embedded applications
9. Tools available for on-device intelligence • Gemini Nano • ML Kit • MediaPipe Solutions
10. • Android’s foundation model for building on-device GenAI applications • Integrated to the Android OS via AICore in Android 14+ • Google AI Edge SDK provides access to APIs • Application examples such as, ◦ Speech transcription ◦ Smart reply ◦ Text summarization • Available on Pixel 8 series and Samsung S24 series 1.0 Nano
12. Tools available for on-device intelligence • Gemini Nano • ML Kit • MediaPipe Solutions
13. • Focused on on-device machine learning tasks • Image recognition, Text recognition, Face detection etc. • Designed for real-time applications with limited computational resources • ML Kit is not designed for GenAI tasks • Android & iOS
14. // Vision APIs implementation("com.google.mlkit:image-labeling:version") implementation("com.google.mlkit:object-detection:version") implementation("com.google.mlkit:pose-detection:version") implementation("com.google.mlkit:face-mesh-detection:version") implementation("com.google.mlkit:barcode-scanning:version") implementation("com.google.mlkit:text-recognition:version") // Natural Language APIs implementation("com.google.mlkit:mlkit-language-id:version") implementation("com.google.mlkit:translate:version") implementation("com.google.mlkit:entity-extraction:version") implementation("com.google.mlkit:mlkit-smart-reply:version")
16. Tools available for on-device intelligence • Gemini Nano • ML Kit • MediaPipe Solutions
17. • MediaPipe Solutions provides a suite of libraries and tools • Apply AI and ML techniques in apps • Initially designed for real-time processing of image and video data • Expanded capabilities to include generative AI functions • Android, Python, Web, iOS (coming soon)
18. • MediaPipe Tasks • MediaPipe Studio • MediaPipe Model Maker • MediaPipe Framework
19. MediaPipe Tasks Includes set of libraries for deploying ML solutions on devices with less code - link
20. MediaPipe LLM Inference • Available on-device LLMs ◦ Gemma 2B ◦ Gemma 7B ◦ Falcon 1B ◦ StableLM 3B ◦ Phi-2 (2.7B)
21. MediaPipe LLM Inference process • Download the model from HuggingFace • Convert the model to MediaPipe-friendly format • Save the model to /data/local/tmp/llm/ directory on device • Run the app
22. Result • Download the model from HuggingFace • Convert the model to MediaPipe-friendly format • Save the model to /data/local/tmp/llm/ directory on device • Run the app
23. • MediaPipe Tasks • MediaPipe Studio • MediaPipe Model Maker • MediaPipe Framework
24. Preview on-device models and pipelines - link MediaPipe Studio
25. • MediaPipe Tasks • MediaPipe Studio • MediaPipe Model Maker • MediaPipe Framework
26. Low-code solution for customizing on-device ML models MediaPipe Model Maker
27. MediaPipe Model Maker
28. • MediaPipe Tasks • MediaPipe Studio • MediaPipe Model Maker • MediaPipe Framework
29. • Low-level component to build efficient on-device machine learning pipelines, similar to the premade MediaPipe Solutions • For ex, Augmented Reality application ◦ Face filters ◦ Hand tracking ◦ Object recognition MediaPipe Framework
30. Summary
31. Thank you! Darshan Parikh Software Engineer, Autodesk