LLM in enterprise market

Transcript

1. ©2022 Databricks Inc. — All rights reserved LLMs in the enterprise market 1 Victor van den Broek Big Data Expo Utrecht - 12 september 2023
2. ©2022 Databricks Inc. — All rights reserved 2 Victor van den Broek - Databricks since 2023 - Focus on Dutch & Belgian public sector and Financial services - 15 years of ‘data experience’ as DE, DS, DA, PO… - linkedin.com/in/victorvdb Solutions Architect @ Databricks
3. ©2023 Databricks Inc. — All rights reserved $3B in investment 5000+ global employees $1B+ in revenue Inventor and pioneer of the data lakehouse Gartner-recognized Leader Database Management Systems Data Science and Machine Learning Platforms The Lakehouse Company Creator of
4. ©2022 Databricks Inc. — All rights reserved 4 LLM 101
5. ©2022 Databricks Inc. — All rights reserved Generative AI, LLMs and Foundation Models 5 Artificial Intelligence (AI) Multidisciplinary field of computer science that aims to create systems capable emulating human intelligence Machine Learning (ML) Learn from existing data and make predictions without being explicitly programmed Deep Learning (DL) Use artificial neural networks to learn from data Generative AI Subfield of AI focussing on generating new data (images, text, audio, code, ...) LLM Models trained on massive datasets to achieve advanced language processing capabilities Foundation Models (GPT-4, BARD, MPT-7B, …) LLMs which can serve as the base for a wide range or applications
6. ©2022 Databricks Inc. — All rights reserved LLMs are not that new Why should I care now? Accuracy and effectiveness has hit a tipping point • Many new use cases are unlocked! • Accessible by all. Readily available data and tooling • Large datasets. • Open-sourced model options. • Requires powerful GPUs, but are available on the cloud.
7. ©2022 Databricks Inc. — All rights reserved Machine Translation Text Summarization Chatbots & Conversational Interfaces Language Models are everywhere…
8. ©2022 Databricks Inc. — All rights reserved What is a language model? Finds the most likely next word in a sequence Avocados are … Stochastic Parrot Green Fruit Delicious Luxurious
9. ©2022 Databricks Inc. — All rights reserved I might read about 700 books in my lifetime
10. ©2022 Databricks Inc. — All rights reserved A Large LM may be trained on 10.000.000 book equivalents
11. ©2022 Databricks Inc. — All rights reserved 11 - Faster software development - More users can leverage AI - More use cases - Reduce development cost - Reduce monotonous tasks
12. ©2022 Databricks Inc. — All rights reserved 12 How do you get to an enterprise deployment?
13. Use Existing Model or Build Your Own Model Serving and Monitoring Data Collection and Preparation DATA PLATFORM UNITY CATALOG Datasets Models Applications
14. ©2022 Databricks Inc. — All rights reserved 14 LLM level 0 Plain foundational models “Everyone” has done this - go to ChatGPT and ask questions without much engineering. Typical enterprise use cases: - Text summarization - Text classification - Generic coding assistants
15. ©2022 Databricks Inc. — All rights reserved
16. ©2022 Databricks Inc. — All rights reserved Alternative to Azure OpenAI are Open Source Models 16
17. Use Existing Model or Build Your Own Model Serving and Monitoring Data Collection and Preparation DATA PLATFORM UNITY CATALOG Datasets Models Applications Curated AI Models Model Serving optimized for LLMs MLflow AI Gateway Plain LLM Lakehouse Monitoring
18. ©2022 Databricks Inc. — All rights reserved 18 LLM level 1 Prompt engineering Add contextual information in the prompt, to give the model specific information pertaining to the question. Typical enterprise use cases: - Customer service chatbots - Specific coding assistants
19. ©2022 Databricks Inc. — All rights reserved 19
20. Use Existing Model or Build Your Own Model Serving and Monitoring Data Collection and Preparation DATA PLATFORM UNITY CATALOG Datasets Models Applications Curated AI Models Model Serving optimized for LLMs Lakehouse Monitoring MLflow AI Gateway Feature Serving Mlflow Evaluation Plain LLM Simple prompt engineering
21. ©2022 Databricks Inc. — All rights reserved 21 LLM level 2 Fine tuning Using data you have available, you can fine tune LLMs to fit your use case. Depending on whether they are open-source or closed source, the methodology will differ. Regardless, it will require data specific to your use case, and engineering capabilities - humans and hardware! Typical enterprise use cases: - LLM fine tuned to answer questions in a specialist area (e.g. legal, medical)
22. Use Existing Model or Build Your Own Model Serving and Monitoring Data Collection and Preparation DATA PLATFORM UNITY CATALOG Datasets Models Applications Feature Serving Curated AI Models AutoML for LLM training Model Serving optimized for LLMs Lakehouse Monitoring MLflow AI Gateway Mlflow Evaluation Plain LLM Simple prompt engineering Fine tuning
23. ©2022 Databricks Inc. — All rights reserved 23 LLM level 3 Retrieval Augmented Generation Encode all relevant data you have with an LLM to a vector database. Then, retrieve the most relevant data and ingest them into the prompts. Basically prompt engineering on steroids, but requires you to encode all the data you have already, and keep using that LLM to encode questions as well. Typical enterprise use cases: - LLM answering about specifics in documents, such as purchase orders and contracts
24. Use Existing Model or Build Your Own Model Serving and Monitoring Data Collection and Preparation DATA PLATFORM UNITY CATALOG Datasets Models Applications Vector Search Feature Serving Curated AI Models AutoML for LLM training Model Serving optimized for LLMs Lakehouse Monitoring MLflow AI Gateway Mlflow Evaluation Plain LLM Simple prompt engineering Fine tuning Retrieval Augmented Generation
25. ©2022 Databricks Inc. — All rights reserved 25 RAG vs Fine-Tuning Generic answers with specific knowledge vs specific answers
26. ©2022 Databricks Inc. — All rights reserved 26 RAG vs Fine-Tuning Generic answers with specific knowledge vs specific answers
27. ©2022 Databricks Inc. — All rights reserved 27 RAG vs Fine-Tuning Generic answers with specific knowledge vs specific answers
28. ©2022 Databricks Inc. — All rights reserved 28 LLM level 4 Training your own model from 0 If all else fails, or you have specific governance / IP / risk requirements, then training a model from scratch becomes an option. However this is both very difficult and very expensive, and there are currently very few enterprise use cases in which this is the solution. If you are one of them, you will know ;-)
29. Use Existing Model or Build Your Own Model Serving and Monitoring Data Collection and Preparation DATA PLATFORM UNITY CATALOG Datasets Models Applications Vector Search Feature Serving Curated AI Models AutoML for LLM training Model Serving optimized for LLMs Lakehouse Monitoring MLflow AI Gateway Mlflow Evaluation Plain LLM Simple prompt engineering Fine tuning Retrieval Augmented Generation Training from scratch
30. Generative AI Fundamentals Course Earn your badge today and share your accomplishment on LinkedIn or résumé Build foundational knowledge of generative AI, including large language models (LLMs), with this free training course. ➔ Welcome and Introduction to the Course ➔ Introducing Generative AI ➔ Finding Success With Generative AI ➔ Assessing Potential Risks and Challenges Available on Databricks.com
31. ©2023 Databricks Inc. — All rights reserved 23 November | Beurs van Berlage Register now 31