[Oracle Data Week 2023] - MySQL HeatWave Lakehouse - Modernize MySQL & non-MySQL workloads with MySQL HeatWave

MySQL HeatWave enables users to process and query hundreds of terabytes of data in the object store—in a variety of file formats, such as CSV, Parquet, and Aurora/Redshift export files.
The data remains in the object store and customer can query it with standard SQL syntax.
With this capability, MySQL HeatWave provides one service for transaction processing, analytics across data warehouses and data lakes, and machine learning—without ETL across cloud services.
There is no additional cost for this capability except the cost of storing the data in object store.

Transcript

1. MySQL HeatWave Lakehouse Olivier Dasini MySQL Cloud Principal Solutions Architect EMEA olivier.dasini@oracle.com Blogs : www.dasini.net/blog/en : www.dasini.net/blog/fr Linkedin: www.linkedin.com/in/olivier-dasini December 2023 Modernize MySQL & non-MySQL workloads with MySQL HeatWave
2. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 2 Me, Myself & I  MySQL Geek  Addicted to MySQL for 15+ years  Playing with databases for 20+ years  MySQL Writer, Blogger and Speaker  Also: DBA, Consultant, Architect, Trainer, ...  MySQL Cloud Principal Solutions Architect EMEA at Oracle  Stay up to date!  Blog: www.dasini.net/blog/en  Linkedin: www.linkedin.com/in/olivier-dasini/ Olivier DASINI
3. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 3 MySQL is the #1 Open Source Database https://db-engines.com/en/ranking
4. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 4 The MySQL universe - The view from the moon… MySQL Commercial/Enterprise MySQL Community + MySQL Enterprise Backup MySQL Enterprise Monitor MySQL Enterprise Authentication MySQL Enterprise Audit MySQL Enterprise TDE MySQL Enterprise Masking MySQL Enterprise Firewall MySQL Technical Support … MySQL Cluster CGE MySQL Cluster NDB + MySQL Enterprise + MySQL Cluster Manager MySQL Community MySQL Server MySQL Client, Workbench MySQL Shell MySQL GR plugin & InnoDB Cluster & Router MySQL Operator for Kubernetes MySQL Connector (C API, Java, Node.js, others) MySQL Support for MS VS Code (Preview) … MySQL Cluster NDB MySQL NDB Storage Engine MySQL NDB Operator for Kubernetes MySQL Cloud Services (PaaS) MySQL HeatWave Databases Services (for OLTP) MySQL HeatWave (for Analytics) MySQL HeatWave AutoML (for Machine Learning) MySQL HeatWave on AWS MySQL HeatWave on Azure MySQL HeatWave Lakehouse Community, Enterprise, Cloud Service (HeatWave) https://www.mysql.com/products
5. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 5 MySQL HeatWave One Database for OLTP, OLAP , ML & Lakehouse
6. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 6 … the MySQL Cloud services made by the MySQL Team The MySQL HeatWave “Family” “MySQL HeatWave” MySQL HeatWave for OLTP a.k.a MDS: (OLTP) General Purpose Optimized for OLTP MySQL HeatWave Lakehouse (Lakehouse) MySQL Heatwave (Analytics) OLTP + Analytics (OLAP) + Machine Learning InnoDB Lakehouse InnoDB RAPID InnoDB RAPID
7. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 7 MySQL HeatWave Optimized for Data Warehouse , Machine Learning, and OLTP
8. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 8 Already lowest cost in industry for data warehouse TPC-H 10TB price performance comparison 13x better than Redshift 28x better than Snowflake 28x better than BigQuery 62x better than Databricks 3 year reserved, paid upfront Standard Edition 1 year reserved 1 year reserved Benchmark queries are derived from the TPC-H benchmarks, but results are not comparable to published TPC-H benchmark results since these do not comply with the TPC-H specifications. https://www.oracle.com/mysql/heatwave/performance/#heatwave-on-oci
9. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 9 MySQL HeatWave Lakehouse Enables users to process & query hundreds of terabytes of data in the object store
10. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 10 MySQL HeatWave Lakehouse Process data in object store and transactional database  Determines right cluster size for processing object store data  Estimates load time  Auto schema inference Scales from 16 GB to 512 TB
11. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 11 MySQL HeatWave can process data from multiple data sources e.g. Oracle Golden Gate, ... AWS Aurora AWS Redshift Data can be in a file or other databases → No requirement to have data in MySQL https://www.mysql.com/products/mysqlheatwave/lakehouse
12. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 12 Load & Query performance comparison – best in the industry 500 TB TPC-H HeatWave Lakehouse Snowflake Redshift Databricks Google BigQuery Annual Cost $1,742,036 $2,300,160 $1,544,268 $1,822,817 $1,446,900 Pricing Term PAYG Standard Edition 1 year upfront 1 year reserved 1 year reserved Load Time (hrs) 4.43 9.04 (2x slower) 40.86 (9.2x slower) 25.42 (5.7x slower) 38.2 (8.6x slower) Query Time (sec) 2,150 39,040 (18x slower) 32,715 (15x slower) 37,729 (17x slower) 76,180 (35x slower) MySQL HeatWave is faster to load & query data and still less expensive https://www.oracle.com/mysql/heatwave/performance/#heatwave-lakehouse
13. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 13 Provides flexibility to develop applications on object store without any performance, cost impact Same price-performance when data inside MySQL or in object store HeatWave HeatWave Lakehouse Snowflake Redshift Google Big Query Databricks 0 10 20 30 40 50 60 70 80 90 100 1.5 1.5 41.9 20.2 41.4 92.5 10TB TPC-H Price-Performance Price-Performance (cents) • 10 HeatWave Nodes, X-Large cluster for Snowflake; 10 nodes of ra3.4xlarge for Redshift; 800 slots for Google BigQuery; Large cluster for Databricks • Standard edition price for Snowflake; 3 yr upfront price for Redshift; 1 year reserved price for Google BigQuery and Databricks https://www.oracle.com/mysql/heatwave/performance/#heatwave-lakehouse
14. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 14 MySQL HeatWave AutoML Build, train, deploy, & explain machine learning models within MySQL HeatWave, at no additional cost
15. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 15 In-database machine learning with MySQL HeatWave Accelerate ML initiatives, increase security, and reduce costs
16. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 16 HeatWave AutoML automates the ML lifecycle & all models can be explained Dataset Data preprocessing Algorithm selection Adaptive sampling Feature selection Hyper-parameter tuning Tuned model Model explainer Prediction explainer Regulatory compliance Fairness Repeatability Causality Trust HeatWave AutoML leverages Oracle AutoML technology to automate the process of training a ML model https://dev.mysql.com/doc/heatwave/en/heatwave-machine-learning.html
17. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 17 Classification Classify warranty claims Identify similar users Recommend movies Recommender System Loan default prediction Predict flight delay Rain fall prediction Regression Predict Advt spend ROI Demand forecasting Anomaly Detection Detect anomalous credit card spend Identify game hacker Fully automated in-database machine learning • In-database • Secure • Fully automated training • 25x faster than Redshift ML • Explainable • No additional cost Time-series forecasting Training, inference, explanation with HeatWave AutoML
18. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 18 https://blogs.oracle.com/mysql/post/introducing-vector-store-and-generative-ai-in-mysql-heatwave  Build ML models on data in object store or the database  Use the same APIs as used for data in the database  No additional cost – reuse the same HeatWave Cluster Training, inference and explanations on data in object store HeatWave AutoML also supports Lakehouse files
19. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 19 Generative AI and vector store Interact with MySQL HeatWave in natural language Private preview
20. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 20 Generative AI with MySQL HeatWave vector store  Users can query and retrieve information in natural language  Efficient searching of documents in HeatWave Lakehouse Users can interact with MySQL HeatWave in natural language https://blogs.oracle.com/mysql/post/introducing-vector-store-and-generative-ai-in-mysql-heatwave Private preview
21. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 21 Vector store provides context to LLM for more relevant results Users can interact with MySQL HeatWave in natural language https://blogs.oracle.com/mysql/post/introducing-vector-store-and-generative-ai-in-mysql-heatwave Private preview
22. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 22 Vector store provides context to LLM for more relevant results Users can interact with MySQL HeatWave in natural language https://blogs.oracle.com/mysql/post/introducing-vector-store-and-generative-ai-in-mysql-heatwave Private preview
23. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 23 Vector store provides context to LLM for more relevant results Users can interact with MySQL HeatWave in natural language https://blogs.oracle.com/mysql/post/introducing-vector-store-and-generative-ai-in-mysql-heatwave Private preview
24. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 24 MySQL HeatWave Social ECommerce FinTech SaaS InnoDB HeatWave OLTP OLAP ML Tools Machine Learning Autopilot Lakehouse Database Exports MySQL HeatWave Analytics tools Database One Database for Transactional, Data Warehouse, Machine Learning & Lakehouse
25. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 25 One service for transaction processing, analytics across data warehouses & data lakes, and ML MySQL HeatWave Lakehouse LIVE DEMO • Upload a file into the object storage • Load it to MySQL HeatWave Lakehouse • Query it with standard SQL syntax • Training, Inference, Explanation with MySQL HeatWave AutoML Watch this demo on Youtube: https://youtu.be/1A9OPZ3dEkg
26. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 26 • MySQL HeatWave is a fully managed database service that provides, in a single cloud database service: – an online transaction processing (OLTP) database, – a real-time in-memory data warehouse, – fully automated in-database automated machine learning, – lakehouse, – and Generative AI (in private preview) capabilities • Designed to process both MySQL and non-MySQL workloads • Best performance and price performance for data warehouse • MySQL Autopilot automates data management • MySQL HeatWave provides the best performance and price performance in the industry for analytics processing in both data warehouse and lakehouse environments MySQL HeatWave - Summary Available in all OCI regions
27. Get $300 in credits and try free for 30 days Get started with MySQL HeatWave oracle.com/mysql/free Learn more about MySQL HeatWave oracle.com/mysql Request a guided workshop Ask your account manager 27 Copyright © 2023, Oracle and/or its affiliates
28. Follow us on Social Media “Data is the Oxygen of Business” 28 Copyright © 2023, Oracle and/or its affiliates
29. Merci! Q&R Olivier Dasini MySQL Cloud Principal Solutions Architect EMEA olivier.dasini@oracle.com Blogs : www.dasini.net/blog/en : www.dasini.net/blog/fr Linkedin: www.linkedin.com/in/olivier-dasini
31. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 31 MySQL HeatWave customer momentum Data warehouse, machine learning and OLTP workloads https://www.oracle.com/customers/?product=mpd-cld-infra:db-services:mysql-heatwave
32. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 32 Contribution to MySQL Become part of the contributor community for the MySQL open-source project, https://forums.oracle.com/ords/apexds/post/contributing-code-to-mysql-8037 • What Contributor should have • A wish to change/fix something in MySQL or have a new feature • Downloaded MySQL source code http://dev.mysql.com/downloads/ • An account in bugs.mysql.com http://bugs.mysql.com or • Working GitHub account https://github.com • Signed Oracle Contribution Agreement (OCA) https://oca.opensource.oracle.com/ • OCA Is a short legal agreement which protects both you as a contributor and Oracle from legal attack. By signing the OCA, you agree that Oracle is legally allowed to use your code in Oracle software and that the code is, to the best of your knowledge, unencumbered by any patent issues
33. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 33 Oracle Operates MySQL at Cloud Scale to Deliver Innovations Faster High Availability Security Backup Patching AutoPilot Analytics Machine Learning Lakehouse Operate MySQL at Cloud Scale Oracle operates MySQL HeatWave across 41 public cloud regions in 22 countries, providing deep insights into product requirements. Multi-cloud operations across OCI, AWS and Azure increase intelligence. Improve MySQL Database Knowledge gained from operating MySQL HeatWave at scale results in improvements to MySQL Database including High Availability, Security, Analytics, Machine Learning, and more.
34. Copyright © 2023, Oracle and/or its affiliates. All rights reserved. 34 “Oracle announced MySQL HeatWave with Autopilot last August, which may very well have been the single greatest innovation in open source cloud databases in the last 20 years to that point. Now Oracle has gone beyond its original unifying of OLTP and OLAP in HeatWave, with MySQL HeatWave ML. Oracle is bringing all of the machine learning processing and models inside the database, so that customers not only avoid managing ML databases apart from the core database, but also eliminate the hassles of ETL, gaining speed, accuracy, and cost-effectiveness in the bargain.” “This latest announcement from Oracle is the third major release of MySQL HeatWave in just over 12 months. Oracle has delivered more cloud database innovations during that timeframe than most cloud database vendors have delivered in the last decade. Not only does the in-database HeatWave ML make Redshift ML look like yesterday’s tech in terms of engineering, performance and cost, but the latest MySQL HeatWave TPC-DS benchmarks demonstrate that Amazon Redshift, Snowflake, Azure Synapse and Google BigQuery are all slower and more expensive. It’s rather clear who’s innovating in cloud databases and who’s being complacent.” Feedback from analysts