Tweet
Tweet

Slide 1

Slide 1 text

© 2020 Snowflake Inc. All Rights Reserved Snowflake Data Engineering Study #5 Yoshi Matsuzaki, Principal Cloud Support Engineer of Snowflake | Dec 2, 2020

Slide 2

Slide 2 text

© 2020 Snowflake Inc. All Rights Reserved ABOUT ME 2 Yoshi Matsuzaki Principal Cloud Support Engineer - Snowflake Work: Technical Support in English/Japanese (7:3) Launching Japanese language support team SQL functionality expert Escalation review Supportability readiness for new features Background: Infrastructure Engineer (MSP) Support Engineer (Cloud Vendor) Database Engineer of MySQL engines (Cloud Vendor)

Slide 3

Slide 3 text

© 2020 Snowflake Inc. All Rights Reserved SNOWFLAKE? 3

Slide 4

Slide 4 text

© 2020 Snowflake Inc. All Rights Reserved SNOWFLAKE? 4 DATA SOURCES OLTP DATABASES ENTERPRISE APPLICATIONS THIRD-PARTY WEB/LOG DATA IoT DATA CONSUMERS DATA MONETIZATION OPERATIONAL REPORTING AD HOC ANALYSIS REAL-TIME ANALYTICS → Data Platform → Data Cloud

Slide 5

Slide 5 text

© 2020 Snowflake Inc. All Rights Reserved SNOWFLAKE ARCHITECTURE 5

Slide 6

Slide 6 text

© 2020 Snowflake Inc. All Rights Reserved Snowflake 6

Slide 7

Slide 7 text

© 2020 Snowflake Inc. All Rights Reserved 7 Snowflake 50-500 MB ( : S3) … Partition Pruning / NDV ( )

Slide 8

Slide 8 text

© 2020 Snowflake Inc. All Rights Reserved 8

Slide 9

Slide 9 text

© 2020 Snowflake Inc. All Rights Reserved 9 ( ) “ ( : immutable) ” - Wikipedia

Slide 10

Slide 10 text

© 2020 Snowflake Inc. All Rights Reserved 10 : (UPDATE) : → I/O … : →

Slide 11

Slide 11 text

© 2020 Snowflake Inc. All Rights Reserved 11 'a' 'b' 'c' Table Partitions UPDATE tab SET col = 'd' WHERE col = 'a'; 'a' 'b' 'c' Table Partitions 'd' v1 v2

Slide 12

Slide 12 text

© 2020 Snowflake Inc. All Rights Reserved 12 I/O … OLAP OLTP OLAP →

Slide 13

Slide 13 text

© 2020 Snowflake Inc. All Rights Reserved 13 / / : UNDO (MySQL, Oracle) UNDO (PostgreSQL) :

Slide 14

Slide 14 text

© 2020 Snowflake Inc. All Rights Reserved 14 → / → Time Travel 90 / SELECT * FROM tab AT (TIMESTAMP => ...); SELECT * FROM tab BEFORE (STATEMENT => ); CREATE TABLE tab CLONE src AT (TIMESTAMP => ...); → Fail Safe 7 GC ...

Slide 15

Slide 15 text

© 2020 Snowflake Inc. All Rights Reserved 15 Table SELECT col FROM tab; 'a' 'b' 'c' Partitions 'd' v1 v2 SELECT col FROM tab BEFORE (STATEMENT = ...); a b c b c d

Slide 16

Slide 16 text

© 2020 Snowflake Inc. All Rights Reserved 16 ( ) : M:N →

Slide 17

Slide 17 text

© 2020 Snowflake Inc. All Rights Reserved 17 ... / Zero Copy Cloning Data Sharing, Data Exchange ...

Slide 18

Slide 18 text

© 2020 Snowflake Inc. All Rights Reserved 18 Zero Copy Cloning → Data Sharing ( ) → Data Exchange (1:1 ) → Snowflake Data Marketplace ( ) → Snowflake Data Service Marketplace ( ) → ...

Slide 19

Slide 19 text

© 2020 Snowflake Inc. All Rights Reserved 19 → → →… (Extensibility)

Slide 20

Slide 20 text

© 2020 Snowflake Inc. All Rights Reserved 20 Extensibility ( ) Snowflake Snowflake extensible → (MP → S/C ) → (S/C → Zero Copy Cloning) → (Cloning → Data Sharing → ...) Snowflake

Slide 21

Slide 21 text

© 2020 Snowflake Inc. All Rights Reserved 21 ... but ( ) ... but ...

Slide 22

Slide 22 text

© 2020 Snowflake Inc. All Rights Reserved 22 → → Snowflake →

Slide 23

Slide 23 text

© 2020 Snowflake Inc. All Rights Reserved Snowflake

Slide 24

Slide 24 text

2020 12 10 11:00 12:00am Live Snowflake Snowflake trocco Snowflake primeNumber

Slide 25

Slide 25 text

© 2020 Snowflake Inc. All Rights Reserved