Share
Embed
Slide 1
Slide 1 text
© 2020 Snowflake Inc. All Rights Reserved
Snowflake
Data Engineering Study #5
Yoshi Matsuzaki, Principal Cloud Support Engineer of Snowflake | Dec 2, 2020
Slide 2
Slide 2 text
© 2020 Snowflake Inc. All Rights Reserved
ABOUT ME
2
Yoshi Matsuzaki
Principal Cloud Support Engineer - Snowflake
Work:
Technical Support in English/Japanese (7:3)
Launching Japanese language support team
SQL functionality expert
Escalation review
Supportability readiness for new features
Background:
Infrastructure Engineer (MSP)
Support Engineer (Cloud Vendor)
Database Engineer of MySQL engines (Cloud Vendor)
Slide 3
Slide 3 text
© 2020 Snowflake Inc. All Rights Reserved
SNOWFLAKE?
3
Slide 4
Slide 4 text
© 2020 Snowflake Inc. All Rights Reserved
SNOWFLAKE?
4
DATA
SOURCES
OLTP DATABASES
ENTERPRISE
APPLICATIONS
THIRD-PARTY
WEB/LOG DATA
IoT
DATA
CONSUMERS
DATA MONETIZATION
OPERATIONAL
REPORTING
AD HOC ANALYSIS
REAL-TIME ANALYTICS
→ Data Platform → Data Cloud
Slide 5
Slide 5 text
© 2020 Snowflake Inc. All Rights Reserved
SNOWFLAKE ARCHITECTURE
5
Slide 6
Slide 6 text
© 2020 Snowflake Inc. All Rights Reserved
Snowflake
6
Slide 7
Slide 7 text
© 2020 Snowflake Inc. All Rights Reserved 7
Snowflake
50-500 MB
( : S3)
… Partition Pruning
/
NDV ( )
Slide 8
Slide 8 text
© 2020 Snowflake Inc. All Rights Reserved 8
Slide 9
Slide 9 text
© 2020 Snowflake Inc. All Rights Reserved 9
( )
“ ( : immutable)
”
- Wikipedia
Slide 10
Slide 10 text
© 2020 Snowflake Inc. All Rights Reserved 10
: (UPDATE)
:
→ I/O
…
:
→
Slide 11
Slide 11 text
© 2020 Snowflake Inc. All Rights Reserved 11
'a' 'b' 'c'
Table Partitions
UPDATE tab SET col = 'd'
WHERE col = 'a';
'a' 'b' 'c'
Table Partitions
'd'
v1 v2
Slide 12
Slide 12 text
© 2020 Snowflake Inc. All Rights Reserved 12
I/O
… OLAP
OLTP
OLAP
→
Slide 13
Slide 13 text
© 2020 Snowflake Inc. All Rights Reserved 13
/ /
:
UNDO (MySQL, Oracle)
UNDO
(PostgreSQL)
:
Slide 14
Slide 14 text
© 2020 Snowflake Inc. All Rights Reserved 14
→ /
→ Time Travel
90 /
SELECT * FROM tab AT (TIMESTAMP => ...);
SELECT * FROM tab BEFORE (STATEMENT => );
CREATE TABLE tab CLONE src AT (TIMESTAMP => ...);
→ Fail Safe
7
GC ...
Slide 15
Slide 15 text
© 2020 Snowflake Inc. All Rights Reserved 15
Table
SELECT col FROM tab;
'a' 'b' 'c'
Partitions
'd'
v1 v2
SELECT col FROM tab
BEFORE (STATEMENT = ...);
a
b
c
b
c
d
Slide 16
Slide 16 text
© 2020 Snowflake Inc. All Rights Reserved 16
( )
: M:N
→
Slide 17
Slide 17 text
© 2020 Snowflake Inc. All Rights Reserved 17
... /
Zero Copy Cloning
Data Sharing, Data Exchange ...
Slide 18
Slide 18 text
© 2020 Snowflake Inc. All Rights Reserved 18
Zero Copy Cloning
→ Data Sharing ( )
→ Data Exchange (1:1 )
→ Snowflake Data Marketplace ( )
→ Snowflake Data Service Marketplace ( )
→ ...
Slide 19
Slide 19 text
© 2020 Snowflake Inc. All Rights Reserved 19
→ → →…
(Extensibility)
Slide 20
Slide 20 text
© 2020 Snowflake Inc. All Rights Reserved 20
Extensibility ( )
Snowflake
Snowflake extensible
→ (MP → S/C )
→ (S/C → Zero Copy Cloning)
→ (Cloning → Data Sharing → ...)
Snowflake
Slide 21
Slide 21 text
© 2020 Snowflake Inc. All Rights Reserved 21
... but ( )
... but
...
Slide 22
Slide 22 text
© 2020 Snowflake Inc. All Rights Reserved 22
→ →
Snowflake
→
Slide 23
Slide 23 text
© 2020 Snowflake Inc. All Rights Reserved
Snowflake
Slide 24
Slide 24 text
2020 12 10
11:00 12:00am
Live
Snowflake
Snowflake trocco
Snowflake primeNumber
Slide 25
Slide 25 text
© 2020 Snowflake Inc. All Rights Reserved