Generative AI for Data Platforms - Databricks Data Intelligence Platform

Slide 1

Slide 1 text

Slide 2

Slide 2 text

©2022 Databricks Inc. — All rights reserved Hi, I am Frank! • Principal @Databricks. TMM for Data, Analytics and AI products • Large scale data & compute • Based in 🍻 ⛰ 🥨 󰎲 Munich • Formerly AWS Tech Evangelist, SW architect, data scientist, published author etc. • @frankmunz / LindedIn

Slide 3

Slide 3 text

©2023 Databricks Inc. — All rights reserved 10,000+ global customers $1.5B+ in revenue $4B in investment Inventor of the lakehouse & Pioneer of generative AI Gartner-recognized Leader Database Management Systems + Data Science and Machine Learning Platforms The data and AI company Creator of

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

©2023 Databricks Inc. — All rights reserved SIMPLE and FAST EFFICIENT RELIABLE Serverless Compute for Data Platforms Serverless Compute Hands-off auto-optimized compute No knobs Fast startup For any practitioner Fully managed and versionless Paying only what you use Strong cost governance Secure by default Stable with smart fail-overs Storage Notebooks with Spark Pipelines AI Model hosting SQL DWH "Put your vendor T-shirts down" 14 multi-cloud

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

©2023 Databricks Inc. — All rights reserved Conﬁdential and Proprietary Walk trough 17 ● Single Page App (S3) ● Kinesis Stream ○ JSON Structure ○ Kinesis Ingest with EFO ● Delta Live Tables (ETL) ● Spark Streaming Data Analytics ○ Histogram streaming data ○ Window-based aggregation ● Databricks Workﬂows ● Databricks SQL

Slide 18

Slide 18 text

©2024 Databricks Inc. — All rights reserved Databricks Data Intelligence Platform Use generative AI to understand the semantics of your data Data Intelligence Engine Open Data Lake (lake ﬁrst approach: S3, ADLS, GCS) Databricks SQL Text-to-SQL Workﬂows optimized based on past runs Delta Live Tables Automated data qualility Mosaic AI Create, tune, and serve custom LLMs Unity Catalog Securely get insights in natural language Delta Lake with Delta UniForm Data layout is automatically optimized based on usage patterns

Slide 19

Slide 19 text

©2023 Databricks Inc. — All rights reserved Streaming ETL with Delta Live Tables Pipelines Python or SQL. STs for ingestion and MVs for transformation Bronze cloud_files CREATE STREAMING TABLE Use a short retention period to avoid compliance risks and reduce costs Avoid complex transformations that could have bugs or drop important data Retain inﬁnite history Easy to perform GDPR and other compliance tasks CREATE MATERIALIZED VIEW Materialized views automatically handle complex joins / aggregations, and propagate updates and deletes. Silver/Gold Ad-hoc DML for GDPR / Corrections

Slide 20

Slide 20 text

©2023 Databricks Inc. — All rights reserved 20 Delta Live Tables ● Serverless Compute (zero compute settings) ● Streaming Ingest from Message Buses with SQL read_kafka(), read_kinesis(), … ● Incrementally computed Materialized Views Link to blog

Slide 21

Slide 21 text

©2023 Databricks Inc. — All rights reserved Building Blocks of Databricks Workflows 21 A unit of orchestration in Databricks Workflows is called a Job. Databricks Notebooks Python Scripts Python Wheels SQL Files/Queries Delta Live Tables Pipeline dbt Java JAR file Spark Submit Jobs consist of one or more Tasks Sequential Parallel Conditionals (Run If) Jobs-as-a-Task (Modular) Control flows can be established between Tasks. Jobs supports different Triggers DBSQL Dashboards Manual Trigger Scheduled (Cron) API Trigger File Arrival Triggers Table Triggers Continuous (Streaming) Preview Coming Soon

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

©2023 Databricks Inc. — All rights reserved Conﬁdential and Proprietary We’re infusing AI in our experiences AI-generated docs + semantic search in Catalog Explorer Databricks Assistant SQL to Dashboard Data Rooms (Project Genie)

Slide 25

Slide 25 text

Slide 26

Slide 26 text

©2023 Databricks Inc. — All rights reserved MosaicML Model Serving MosaicML Model Serving Vector Search MLflow AI Gateway Model Serving MLflow AI Gateway MLflow Evaluation MLflow Prompt Engg Generative AI Solutions Enable every architectural pattern Prompt Engineering and Chains Retrieval Augmented Generation (RAG) Fine-tuning Pre-training Unity Catalog | Lakehouse Monitoring Crafting specialized prompts to guide LLM behavior Combining an LLM with enterprise data Adapting a pre-trained LLM to specific data sets or domains Training an LLM from scratch Complexity / Compute-intensiveness

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Model Serving Custom Models Foundation Models APIs External Models Deploy any model as a REST API with Serverless compute, managed via MLﬂow. CPU and GPU. Integration with Feature Store and Vector Search. Govern external models and APIs. This provides the governance of MLﬂow Deployments for LLMs, plus the monitoring and payload logging of traditional Databricks Model Serving. Databricks curates top Foundation Models and provides them behind simple APIs. You can start experimentation immediately, without setting up serving yourself. Databricks Model Serving Unified UI, API & SDK for managing all types of AI Models

Slide 29

Slide 29 text

©2024 Databricks Inc. — All rights reserved Built-in governance with permissions and lineage Automatically synchronizes streaming source data with vector db. No separate data pipelines Vector DB Serverless vector database for RAG

Slide 30

Slide 30 text

©2024 Databricks Inc. — All rights reserved Finetuning Finetune your LLM on your data Serverless: no need to reserve or pick GPUs Pick the data from Unity Catalog or from Huggingface Maintain control and ownership of the model. It is your Intellectual Property.

Slide 31

Slide 31 text

©2024 Databricks Inc. — All rights reserved Mosaic AI Training Up to 7X faster and cheaper training of large AI Models Simpliﬁed, scalable, and cost-effective training of large AI models. Train or ﬁne-tune your own generative AI model with your data in your secure environment. Full control of your model and privacy of your data. Your data, your model, built in your secure environment.

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

©2024 Databricks Inc. — All rights reserved Model architecture ● Sparse Mixture-of-Experts (MoE) ● 4 of 16 experts for a given input Model training ● Pre-trained on 3072 NVIDIA H100s in 3 months. ● on Databricks Data Intelligence Platform, Notebooks, Jobs, etc. The models ● DBRX Base for ﬁne-tuning ● DBRX Instruct for RAG chains ● 132B parameters ● 32k token context length License and data ● Open-source for commercial use ● Pretrained on publicly available 12T tokens ● Designed for enterprises Introducing DBRX’s details

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

©2023 Databricks Inc. — All rights reserved Conﬁdential and Proprietary 39 Gen AI meets Data Platforms Data Intelligence Engine + Uniﬁed Governance -> Assistant, Intelligent Search, automated documentation, natural language queries and better scheduling, automated data quality