AI for Mobile

AI for Mobile

Transcript

1. AI for Mobile Chetan Sachdeva, Uber
3. Agenda 01 What is GenAI? 02 What is Prompt Engineering? 03 RAG (Retrieval Augmented Generation) 04 Hugging Face 05 Gemini AI 06 Firebase and Project IDX 07 Multimodal using GenKit 08 Q&A
4. Prerequisites • Install python3 • Run `pip install jupyter` • Run `jupyter notebook` • Run `python3 -m venv .` • Run `source ./bin/activate`
5. What is GenAI? • AI that Generates content from data. ◦ Generative AI creates data, while Discriminative AI categorizes data. • Applications: Content Creation, Data Augmentation, Recommendations, Customer Support, Code Generation. • Key Technologies: GPT, GAN etc.
6. Why GenAI matters? • Potential to enhance human creativity and productivity across various domains. • Copilot for creative artists. • Ability to automate tasks, improve efficiency, and reduce costs. • Can provide personalized experiences and recommendations. • Potential to solve complex problems and drive innovation.
7. Why now? • Availability of large & diverse set of labelled datasets. Link • Huge number of pre-trained SOTA models (eg: Llama, GPT older versions, Falcon, T5 etc.). Link • Innovative DL methods ◦ Model architectures like Transformers, GAN ◦ Training techniques like LoRA, RLHF • Advances In Computation & Hardware ◦ Nvidia Ampere (e.g. A100) & Hopper (e.g. H100, H200) series. ◦ Groq LPUs
8. GenAI at Uber • Gen AI use cases at Uber span across multiple teams and projects. • Customer Obsession - NOVA, Agent Assist Response Generation, and Summarization. • Delivery: Menu description generation, Merchant chatbot, Ads copilot, AI Shopping Assistant, Product Tagging, Query Tagging, Search Quality.
9. Prompting Pitfalls & Considerations • Use prompts that are clear, concise, and relevant. Unoptimized prompts can lead to unintended consequences. • Use delimiters: Separate different parts of your prompt (instructions, context, examples, etc.) using clear delimiters like ###, """, —, or XML like tags to help the AI distinguish between sections. • Overall structure of the prompt: Start with role, task, instructions on how to accomplish the task, finally have styling instructions. • Provide relevant context: When possible relevant background information to help the AI understand the task better. This is typically interleaved with the task instructions. For domain-specific context/knowledge, we typically use retrieval augmented generation. • Avoid prompts that are discriminatory, offensive, or unsafe. Be mindful of the potential for bias in your prompts.
10. Most Widely Used GenAI models In The Industry Proprietary ◦ Open AI - GPT-4, GPT-4o ◦ Google - Gemini, Bison ◦ Anthropic - Claude variants Open Source ◦ Llama (Meta) ◦ Mistral, Mixtral ◦ Falcon ◦ Dolly etc.
11. What is Prompt Engineering ? • Prompting is the process of giving an AI model input or instructions to guide it in generating a specific response. • Well-structured prompts guide the AI to generate more precise and relevant outputs. • Prompting allows users to steer AI responses in specific directions, making it easier to achieve the intended tone, format, or content. • By varying prompts, you can unlock a wide range of creative and diverse responses from the AI.
12. Prompt Engineering Quick Demo Prompting AI is like ordering coffee—be specific, or you might end up with something weird and way too extra!
13. Prompt Engineering Techniques Zero-shot: The model responds to a query with only instructions (without examples). One/Few-shot: The model is given one/few examples to learn from before responding. Dynamic few-shot: Model is given a subset of relevant examples dynamically chosed based on user query. ReAct: The model asks clarifying questions to gather more information before responding, and interleaves reasoning and action specific generations in the process of solving the given task. Other variants: Least-to-Most prompting, Reflexion Chain-of-thought: The model thinks step-by-step, showing its reasoning before giving a final answer. Other variants like Tree-of-thought, Graph-of-thought etc. LLM Input text Output text LLM Input text Output text 2x LLM Input text Output text Nx Agent LLM Input text Output text LLM Expert 1 LLM Expert 2 Tool 1 Tool 2
14. Prompt Engineering Drawbacks • Factual Errors and Information Accuracy - Outdated information. • Domain Specific Knowledge Deficits. • To address these drawbacks: ◦ Pre-Train The LLM (If Possible) ◦ Fine Tune A Base Model ◦ Pass Information While Prompting (RAG Approach)
15. RAG (Retrieval Augmented Generation)
16. RAG (Retrieval Augmented Generation)
17. RAG (Retrieval Augmented Generation)
18. Prompt Engineering (Open AI Course)
19. Hugging Face • Use Colab or Jupyter Notebook https://colab.research.google.com • Getting Started With Hugging Face in 15 Minutes | Transformers, Pipeline, Tokenizer, Models • Running a Hugging Face Large Language Model (LLM) locally on my laptop | Mark Needham • https://huggingface.co/docs/transformers/en/quicktour
20. Hugging Face Quick Demo • https://colab.research.google.com • https://colab.research.google.com/drive/1Q0k6vNLiy0SlYf086wZt_c8jhVNu31 Br?usp=sharing
21. Gemini AI Gemini API is free while Vertex AI requires credits. • Add Generative AI to your Android app with the Gemini API • Gemini API: ◦ https://ai.google.dev/gemini-api/docs/quickstart?lang=android • Vertex AI: ◦ Vertex AI SDKs for #Firebase ◦ https://firebase.google.com/docs/vertex-ai/get-started?platform=android
22. Firebase and Project IDX • Vertex AI for Android Get started with the Gemini API using the Vertex AI in Firebase SDKs • Firebase Angular Building an app with Gemini in Firebase and Project IDX • AI Studio
23. Multimodal using GenKit
24. Multimodal using GenKit
25. Multimodal using GenKit