Tweet
Tweet

Slide 1

Slide 1 text

© 2020 Snowflake Inc. All Rights Reserved Snowflake July Tech Festa 2021 winter Yoshi Matsuzaki, Principal Cloud Support Engineer of Snowflake | Dec 2, 2020

Slide 2

Slide 2 text

© 2020 Snowflake Inc. All Rights Reserved ABOUT ME 2 Yoshi Matsuzaki Principal Cloud Support Engineer - Snowflake Work: Technical Support in English/Japanese (7:3) Launching Japanese language support team SQL functionality expert Escalation review Supportability readiness for new features Background: Infrastructure Engineer (MSP) Support Engineer (Cloud Vendor) Database Engineer of MySQL engines (Cloud Vendor)

Slide 3

Slide 3 text

© 2020 Snowflake Inc. All Rights Reserved SNOWFLAKE? 3

Slide 4

Slide 4 text

© 2020 Snowflake Inc. All Rights Reserved SNOWFLAKE? 4 (Micro-partitioning) / ( ) / ( ) ( × ) 90 / ...

Slide 5

Slide 5 text

© 2020 Snowflake Inc. All Rights Reserved SNOWFLAKE? 5 DATA SOURCES OLTP DATABASES ENTERPRISE APPLICATIONS THIRD-PARTY WEB/LOG DATA IoT DATA CONSUMERS DATA MONETIZATION OPERATIONAL REPORTING AD HOC ANALYSIS REAL-TIME ANALYTICS → Data Platform → Data Cloud

Slide 6

Slide 6 text

© 2020 Snowflake Inc. All Rights Reserved DATA CLOUD? 6 "The Data Cloud is a network that connects Snowflake customers, partners, data providers, and data service providers." ( ) ( )

Slide 7

Slide 7 text

© 2020 Snowflake Inc. All Rights Reserved DATA CLOUD? 7 Zero-Copy Cloning Data Sharing Data Exchange Snowflake Data Marketplace

Slide 8

Slide 8 text

© 2020 Snowflake Inc. All Rights Reserved DATA CLOUD? Zero-Copy Cloning Data Exchange Snowflake Data Marketplace Data Sharing

Slide 9

Slide 9 text

© 2020 Snowflake Inc. All Rights Reserved CHARACTERISTICS 9

Slide 10

Slide 10 text

© 2020 Snowflake Inc. All Rights Reserved ISOLATED STORAGE? 10

Slide 11

Slide 11 text

© 2020 Snowflake Inc. All Rights Reserved MICRO-PARTITIONING 11 (Micro-Partitions) Snowflake / 50-500 MB ( : S3) … Partition Pruning / NDV ( )

Slide 12

Slide 12 text

© 2020 Snowflake Inc. All Rights Reserved 12

Slide 13

Slide 13 text

© 2020 Snowflake Inc. All Rights Reserved BENEFITS OF MICRO-PARTIONING 13

Slide 14

Slide 14 text

© 2020 Snowflake Inc. All Rights Reserved ZERO-COPY CLONING 14 'a' 'b' 'c' Table A Partitions Table A'

Slide 15

Slide 15 text

© 2020 Snowflake Inc. All Rights Reserved DATA SHARING 15 'a' 'b' 'c' Table A Partitions Account A Account B Table A

Slide 16

Slide 16 text

© 2020 Snowflake Inc. All Rights Reserved BENEFITS OF MICRO-PARTIONING 16 …

Slide 17

Slide 17 text

© 2020 Snowflake Inc. All Rights Reserved BENEFITS OF MICRO-PARTITIONING 17 S3 ( )

Slide 18

Slide 18 text

© 2020 Snowflake Inc. All Rights Reserved BENEFITS OF MICRO-PARTITIONING 18 / GC = (Time Travel)

Slide 19

Slide 19 text

© 2020 Snowflake Inc. All Rights Reserved 19 TIME TRAVEL 'a' 'b' 'c' Table Partitions UPDATE tab SET col = 'd' WHERE col = 'a'; 'a' 'b' 'c' Table Partitions 'd' v1 v2

Slide 20

Slide 20 text

© 2020 Snowflake Inc. All Rights Reserved 20 TIME TRAVEL Table SELECT col FROM tab; 'a' 'b' 'c' Partitions 'd' v1 v2 SELECT col FROM tab BEFORE (STATEMENT = ...); a b c b c d

Slide 21

Slide 21 text

© 2020 Snowflake Inc. All Rights Reserved 21 TIME TRAVEL Table t_old 'a' 'b' 'c' Partitions 'd' v1 v2 CREATE TABLE t_old CLONE t BEFORE (STATEMENT = ...); Table t

Slide 22

Slide 22 text

© 2020 Snowflake Inc. All Rights Reserved 22 TIME TRAVEL & FAIL SAFE 0 1 2 90 91 97 89 Time Travel ( / ) Fail Safe ( ) 0 90 ( ) Time Travel 7

Slide 23

Slide 23 text

© 2020 Snowflake Inc. All Rights Reserved 23 TIME TRAVEL & FAIL SAFE 0 1 2 90 91 97 89 Time Travel ( / ) Fail Safe ( ) 98 0 90 ( ) Time Travel 7

Slide 24

Slide 24 text

© 2020 Snowflake Inc. All Rights Reserved 24 TIME TRAVEL & FAIL SAFE 0 1 2 90 91 97 89 Time Travel ( / ) Fail Safe ( ) 98 GC 0 90 ( ) Time Travel 7

Slide 25

Slide 25 text

© 2020 Snowflake Inc. All Rights Reserved TIME TRAVEL & FAIL SAFE 25 Time Travel 90 / CREATE TABLE ... CLONE ... ... SELECT ... FROM ... ... Fail Safe Time Travel 7 GC

Slide 26

Slide 26 text

© 2020 Snowflake Inc. All Rights Reserved BENEFITS OF STORAGE ISOLATION 26

Slide 27

Slide 27 text

© 2020 Snowflake Inc. All Rights Reserved BENEFITS OF STORAGE ISOLATION 27 (I/O ) ( ) ( )

Slide 28

Slide 28 text

© 2020 Snowflake Inc. All Rights Reserved BENEFITS OF STORAGE ISOLATION 28 / / /

Slide 29

Slide 29 text

© 2020 Snowflake Inc. All Rights Reserved BENEFITS OF STORAGE ISOLATION 29 (cont.) / ( ) …

Slide 30

Slide 30 text

© 2020 Snowflake Inc. All Rights Reserved SNOWFLAKE ARCHITECTURE 30

Slide 31

Slide 31 text

© 2020 Snowflake Inc. All Rights Reserved REAL-WORLD USE CASE 31 Continuous Loading (4TB/day) S3 <5min SLA Virtual Warehouse Medium ETL & Maintenance Virtual Warehouse Large 4 trillion rows 3+ petabyte raw data 8x compression ratio 25M micro partitions Prod DB Virtual Warehouse 2X-Large Reporting (Segmented) Interactive Dashboard 50% < 1s 85% < 2s 95% < 5s Virtual Warehouse Auto Scale – X-Large x 5

Slide 32

Slide 32 text

© 2020 Snowflake Inc. All Rights Reserved ARCHITECTURE TO X 32 → → → (Extensibility) → Data Warehouse → Data Platform → Data Cloud → ...

Slide 33

Slide 33 text

© 2020 Snowflake Inc. All Rights Reserved SNOWFLAKE? 33

Slide 34

Slide 34 text

© 2020 Snowflake Inc. All Rights Reserved