Fine-tuning LLMs in Vertex AI — DevFest Kampala 2023

Fine-tuning LLMs in Vertex AI — DevFest Kampala 2023

Transcript

1. #MachineLearning Fine-tuning LLMs in Vertex AI Olayinka Peter Oluwafemi Machine Learning GDE @olayinkapeter_ Nov 2023
2. $whoami • ML Google Developer Expert • Senior ML @ Credit Direct • ❤ TensorFlow, anime & peanut butter • Coordinates 󰗓 coffeewithpeter.com
3. What is the problem of general large language models?
4. Tuning is required when you want the model to learn something niche or specific that deviates from general language patterns
5. Vertex AI One AI platform, every ML tool you need 🔥🚀
6. Vertex AI Overview Unified development and deployment platform for data science, machine learning, and generative AI
7. Model Garden Generative AI Studio Open Source Models Task Specific AutoML and APIs Foundation Models Tuning Adaptive Layers Prompt Design Data Science Workbench (Colab Enterprise & Workbench Instances) Experiment Train Deploy MLOps ML Platform Vertex AI Platform provides data science, ML, and Gen AI practitioners with tools to customize and build
8. Vertex AI areas of investment 1 Unified Data & AI platform for all users to accelerate time to value 2 End-to-End MLOps to efficiently and responsibly manage and govern AI 3 Open and Scalable AI Infrastructure to flexibly and successfully deploy AI State-of-the-Art AI to drive innovation and improve customer outcomes 4
9. Vertex AI Pre-Trained Models Vision Language Conversation Vision Video Intelligence AutoML Vision AutoML Video Intelligence Translation Natural Language AutoML Natural Language AutoML Translation Speech-to-Text Dialogflow Text-to-Speech Best in class tools allowing customers to leverage Google’s leadership in AI to solve common problems Structured data Fleet Routing API AutoML Tables Vertex AI Forecast Time Series Insights API TabNet Speaker ID
10. Vertex AI Model Garden 01 One stop shop 02 Use Foundation Models directly 03 Tune Foundation Models 04 Customize popular open source models 05 API access to task-specific solutions One place to launch various enterprise user journeys
11. Google Cloud Generative AI Empower enterprises to innovate faster with enterprise-ready generative AI APIs | SDK | Libraries AI Platform AI Notebooks Experiment | Train | Deploy Prompt | Tune | Distill | Extend ML Ops | RAI & Eval Tooling Google Cloud Infrastructure (GPU/TPU) | Google Data Cloud Model Garden Build Your Own Gen AI powered Agents AI Solutions Doc AI Contact Center AI Translation Hub Risk AI User Interfaces: Voice | UI Widgets | Multimodal | … Generative AI Studio Search & Conversation Data Connectors Extensions Orchestrate Grounding Enterprise Readiness NTK Google Models PaLM, Codey, Imagen, Chirp, Embeddings… Open Source Models LLaMA 2, Falcon, Stable Diffusion… Partner Models Claude 2,… Broad Ecosystem of Partners Search Retrieve | Summarize | Generate Conversation Answer | Generate | Transact … Model Garden Vertex AI Data Governance Data Privacy Responsible AI Security & Compliance
12. 01 A simple UI for interacting with models 02 Tune models with your own data 03 Use models in production 04 Multiple modalities Generative AI Studio Simple UI for interfacing, experimenting, and tuning models
13. 02 Tune models with your own data
14. Imagine you wanted to use a text-bison model to generate an answer for this: Multi-choice problem: What is the topic of this text? - business - entertainment - health - sports - technology - politics Text: What is a large language model (LLM)? The answer is:
15. {"input_text": "question: How many copies of Gears of War 3 were sold ? context: Like its predecessors , the game received widespread critical acclaim from critics . Critics praised its story , voice acting , graphics and visuals , and music , but criticized its lack of innovation . Gears of War 3 sold over 3 million copies and was the second best selling game in the U . S . ", "output_text": "over 3 million copies"} {"input_text": "question: How many parishes are there in Louisiana ? context: The U . S . state of Louisiana is divided into 64 parishes ( French : paroisses ) in the same manner that 48 other states of the United States are divided into counties , and Alaska is divided into boroughs . ", "output_text": "64"} Sample Tuning Data
16. { "input_text": "Multi-choice problem: What is the topic of this text?\n- business\n- entertainment\n- health\n- sports\n- technology\n- politics\nText: Canadians spent $22.1B on alcohol last year, StatsCan says\nThe answer is:", "output_text": "business"}, Actual Tuning Data
17. Let’s dive in Dive in
18. Thank You Olayinka Peter Oluwafemi he/him @olayinkapeter_ Nov 2023