Slide 1

Slide 1 text

Dipti Borkar Co-Founder & CPO | Ahana An introduction to Presto, an open source distributed SQL engine

Slide 2

Slide 2 text

Founder Mom Immigrant Girl data geek (DB) Engineer always Product techie Team builder Open source believer Mixologist

Slide 3

Slide 3 text

3 Agenda • What is Presto? • History of federation • Introduction to Presto • What made Presto different? • Scalable architecture • Flexible Connectors • Performance • The life of a query

Slide 4

Slide 4 text

4 Technology Cycles Rhyme: Data Federation FDBMS Challenges RDBMS FDBMS Paper by McCleod / Heimbigner (1985) FDBMS Paper by Sheth / Larson (1990) OLTP to DW Wins Data Warehouse becomes the source of truth Star schema becomes sacred Cloud & Big Data Composite Software (founded 2001) Garlic Paper by Laura Haas (2002) à DB2 Federated Google File System Paper (2003) MapReduce paper (2006) Spark Paper (2010) Too many Data Sources, No one uber schema New Cloud DW w/ Data Lakes Based on SQL Self Service Platforms which enable Self-Service Analytics SQL Federation Makes Comeback Dremel Paper (2010) à Drill paper (2012) SQL ++ paper (2014) à Couchbase SQL++ engine (2018) Presto paper (2019), PartiQL (2019) 80’s 90’s 2000’s 2010’s 2020’s

Slide 5

Slide 5 text

5 Presto: One of the Fastest Growing Open Source Projects in Data Analytics Business Needs Data-driven decision making Businesses need more data to iterate over Technology Trends Disaggregation of Storage and Compute The rise of data lakes

Slide 6

Slide 6 text

6 What is Presto? • Distributed SQL query engine • ANSI SQL on Databases, Data lakes • Designed to be interactive • Access to petabytes of data • Opensource, hosted on github • https://github.com/prestodb

Slide 7

Slide 7 text

7 Presto Overview

Slide 8

Slide 8 text

8 Common Questions? • Is presto a database? • How is it related to Hadoop? • How is it different from a data warehouse?

Slide 9

Slide 9 text

9 Sample Presto deployment stack & use cases • Ad hoc • BI tools • Dashboard • A/B testing • ETL/scheduled job • Online service

Slide 10

Slide 10 text

10 What made Presto different? • Scalable architecture • Pluggable Connectors • Performance

Slide 11

Slide 11 text

11 Scalable Architecture • Two roles - coordinator and worker • Easy scale up and scale down • Scale up to 1000 workers • Validated at web scaled companies

Slide 12

Slide 12 text

12 Scalable Architecture

Slide 13

Slide 13 text

13 Pluggable Presto Connectors

Slide 14

Slide 14 text

14 Presto Connector Data Model • Connector: Driver for a data source. • Example: HDFS, AWS S3, Cassandra, MySQL, SQL Server, Kafka • Catalog: Contains schemas from a data source specified by the connector • Schemas: Namespace to organize tables. • Tables: Set of unordered rows organized into columns with types.

Slide 15

Slide 15 text

15 Presto Hive Connector for Object stores & Files systems

Slide 16

Slide 16 text

16 Presto Hive Connector – Access Control

Slide 17

Slide 17 text

17 Presto Hive Connector – Data File Types • Supported File Types • ORC • Parquet • Avro • RCFile • SequenceFile • JSON • Text • No data ingestion needed

Slide 18

Slide 18 text

18 Presto Druid Connector for real-time analytics

Slide 19

Slide 19 text

19 Why Presto is Fast • In-Memory processing • Pull model • Columnar storage and execution

Slide 20

Slide 20 text

20 The Life of a Query – Simple Scan

Slide 21

Slide 21 text

21 The Life of a Query – Join and Aggregation SELECT orders.orderkey, SUM(tax) FROM orders LEFT JOIN lineitem ON orders.orderkey = lineitem.orderkey WHERE discount = 0 GROUP BY orders.orderkey This example is from Presto: SQL on Everything https://research.fb.com/publications/ presto-sql-on-everything/

Slide 22

Slide 22 text

22 Logical Plan - Do NOT Join Two Big Tables

Slide 23

Slide 23 text

23 Limitations • Memory Limitation • Fault Tolerance • Single Coordinator

Slide 24

Slide 24 text

24 Get started Docker Sandbox for Presto https://hub.docker.com/r/ahanaio/prestodb-sandbox AWS Sandbox AMI for Presto https://ahana.io/tutorials/aws-sandbox/

Slide 25

Slide 25 text

25 Ahana • SQL analytics company based on Presto • Team of experts in cloud, database, and Presto • Investment from Google Ventures • Named CRN Top 10 Big Data Startup of 2020 • Premier member of “[Ahana founders] have been strong supporters of the Presto Foundation since its launch in September 2019” “We are excited to welcome Ahana, as the first and only company focused on supporting Presto of the Presto Foundation”

Slide 26

Slide 26 text

https://events.linuxfoundation.org/prestocon/ PRESTO20WIBD Free for WiBD Members

Slide 27

Slide 27 text

27 Join the Presto Community • Require new feature or file a bug: github.com/prestodb/presto • Slack: prestodb.slack.com • Twitter: @prestodb Stay Up-to-Date with Ahana • URL: ahana.io • Twitter: @ahanaio

Slide 28

Slide 28 text

Q & A And yes! We are hiring!

Slide 29

Slide 29 text

8/27/20

Slide 30

Slide 30 text

30 Presto Foundation: Community Driven

Slide 31

Slide 31 text

31 Data-Driven Companies need Low Data Latency Analysts and Scientists need to answer questions: The time it takes from a user having a question to the time they can actually answer it “Data Latency” = 1. User wants to track or explore some new data 2. User meets with Data Eng team to make plan 3. Data team acquire data and check access permissions 4. Build and test the ETLs and make tables available to user 5. Notify the user so they can ask their questions ! Can be days or weeks of time