Slide 1
Slide 1 text
Data Engineering Study #3
#%()!
DeNA
2020/9/16
-*&
",# +,
$'
Slide 2
Slide 2 text
(& *Ryoji Hasegawa+
● -> IT
● %)@"SaaS
● 20169 DeNA
○ !'
$#
Slide 3
Slide 3 text
54B'/
3
● %+!#"(6,: <=9C38
;B
DeNA.?)&7(6,: 9C$
(6,: 2<=((6,:<=)
1 -
B
● <=A@*>0
Slide 4
Slide 4 text
1
2
DeNA
DeNA
3
Slide 5
Slide 5 text
DeNA
5
1
Slide 6
Slide 6 text
● 9e<i
● (EgW5`J0KP
\a
7^1W5VY:?
6
$
%
"
● (Eh+ZgQF&/A
=6S,D
● UC>;H83[
!
$
%
"
A(Ec
B(Ec
C(Ec
…
(Ec.VY
*
N
f
O
b
M
G[-VY
… A(Ec
B(Ec
C(Ec
*
N
f
O
b
M
#"%VY
…
● (EcgW5`J2
4d]L
● (EcgR_,D
● ()(Ec.G
[-
● (EQF&/A=6,
D
● @BTX)dI'
L
Slide 7
Slide 7 text
*
A
Q
K
;
6
G
A
Q
DeNA
JM=?
4BQ: + CN9
"&.
7
2*1020+04BAQ
(24BAQ
'/4BAQ
!2$4BAQ
4BQ:JM CN9JM
4BQ:K;6G
54CN9JM 23OHDeNA
*0%-
07I8P
CN9JM EF>
#
,
|
)
0
.
|
A
Q
D
<
L
@
A
Q
0
A
Q
Slide 8
Slide 8 text
$
<
M
G
3
.
B
<
M
&
|
#
)
'
|
<
M
@
7
H
9
<
M
)
<
M
0=5DFI ->M2FI?J1FI
8
6
8
+$*)+)%)->M2FI ?J1FI
A4 $M0=FI
CK
0
=
:
L
M
D
P
G
Slide 9
Slide 9 text
DeNA
9
2
Slide 10
Slide 10 text
• 6F@RNO=QG6F;
•
RDBExcel6F
• !1+#6F@4#$&,5J#$&,0/"*?D
• -1 '/%1.I6F@I(1$-1)>;
!1+#HCK6F
P
6F
Slide 11
Slide 11 text
HR34/1;W.'17,C(!$
→ :MFI^ <?LB
7)JA]XGa
7,:M\>?LE_UVQZ (2010)
11
7,65 7"5-5$:MK\
:M\@UV
?L\
…
● 7,:M\>%0$?L
BS5&8=
UV
NICYO#7 ?LB
SE_UVT9
● 5*2+ Hadoop [P`D
Slide 12
Slide 12 text
2)1?TD^]IR (2011)
12
#
+
S
l
4+32 4#2,2%5USl
5UlBbd Ve@bd
● 4+5U'/%4+5U
OJ
● 68?Tf 5U9G
*-(4.2W&,2_hLn
?
T
D
^
G
?TD^QoKC;`Im bd>c
?Tl
…
?
P
D
^
k
\
?
T
[
E
a
Z
<
i
$
|
a
Z
?
T
"
|
0
m
]
● ?TD^2(Ve@bd
OJ
● Q
04)=Ve@
!4+cN
● Ve
XC
D^oKCFY
○ oj?TFY: Vertica H:
○ oA7MFY: BI"40ETL"40=g
Slide 13
Slide 13 text
• DeNA ,!4 (
• *-"%
&').
$
5 +
15 +
BI
Monthly Active
User
1,000 -
,)
#+
Slide 14
Slide 14 text
14
3
Slide 15
Slide 15 text
•
•
15
Slide 16
Slide 16 text
•
•
16
Slide 17
Slide 17 text
vM7\qt
]7\4>em6dIDL#%)/0'5%exyN7\/RpdIJA9=ZHh7\qta
=ZHh?UnX: 3-1=ZHhs|
17
PW=ZHhc`
On-Premises
7\ A
7\ B
7\ C
…
…
7\ A
7\ B
7\ C
…
• Q{jGNrf:^6o@%,5*5T
Fz
• bK7\C
dI!%(.&z
• MLAIiE
}OBlSw
• 7\4-2$"+_q8g
• URpkVB;
• 7\_dI[u!%(.&
Slide 18
Slide 18 text
O
• Google Cloud Data Platform
Day #2
Google Cloud 3
.
'*!%.);QCA
GJHF
https://cloudonair.withgoogle.c
om/events/jp-data-platform-
day?talk=transformation
• DeNA TechCon 2020
DeNA .'*!%.)
NKM6#*,
https://engineer.dena.com/post
s/2020.03/techcon_2020_live/
GCP(.4P7AE@
$&+*"J5D:L4P7A
18
BI Tools
Service A
Service Env
App Server
DB Server
Data Platform
Service A
BigQuery
Cloud
Storage
GKE
digdag batch
web
app
Argus
app logs
snapshot
Service B
Service C
Service B
Service C
---
---
I9:L7A78B<0
2>?/<01=
Slide 19
Slide 19 text
•
•
19
Slide 20
Slide 20 text
uW~@'80{}
• NsR\{tYl#Uo
• ,AE9*K.FI,07Mm
• Vr^z)I/9%D3>K.FI
e
• NsRP→ahQ
• "$Pn#|Mah
!%8G07-K;0R
[
#S] ah
!
Z
Xxc O^z ~#i!
~i: Nsi
20
+KBJ)I2K6'ICI7Nsq
0?K5Nsq
=H0*%Nsq
:47-K;0Nsq
V
r
^
z
G
^z}M
JybpkwfJvguW~
^zL`_jx<(K1TduV[<&47
Slide 21
Slide 21 text
IcsL\^0F,#D*3!no
21
IcsLVdrx ]
• IcsM0F,#D*3!no\^%F+N
• g&F;IcsOaWknoVqenoy0F,#D*3!AD$$BF
7z!2A+1cTShU'+1QfY ^b H
Z
• G_noC"=X':>3%F)?D'+1XcTtpmv
ShiLw
&F;E#D,F/"D3%F)?D'+1XR
→ !2A+10F,#D*3!
OaWk#D*3!3u[':
>3%F)?D
● cTtp
→ Jlj0F,5"7
@"D PK
m
Slide 22
Slide 22 text
.A)nt ,A<@Vwu
● BI
● .A)$?'jgX
{
.A)kCP\ & 20?'ZP
● (A%7<;'1?"VOCKf
● looker YF\
● .A)MtplSP@cP
{
mODh.A)nt^]
● .A)34:?`E@t
● 0;'/^]
{
• #A5Dh.A)?&1GKfVwG~TGyH}J
• jL}[*A5QKjLxvLJ
%'-5e
qUB}Nb
22
KfVwG
K_Vwt
#A5@?)A-?6?/Dhe
KfsW|r
H.A)|r
Kf,A<u
%'-5e Kf\
.A)?&1;?"G .A)?&1;?"G
Dh^]
ia4>&!/
(;8A%9??&1;?"G
KfVw.A)?&1;?"0=+&I
HdoDhF
^]iavR}CY
z
Slide 23
Slide 23 text
$3G!)()AF
LF?1
6>B*?
)AF "
6>4
$AF6>M/-@8
23
*?;:/)AF '9K
3GLF
/=.E3G
,5(LF
OS()&#
IE
…
DC
*?;:/)AF '9K
$HJ7
%M/W
,5(LF
3GLF
%M/W
,5(LF
Google +
6>0B
3G<2
3G<2
*?)AF
"
6>
4
Slide 24
Slide 24 text
!
24
• HmC9XGE>WFaegLQ(h&68Xs@eg Zi
?eg
o^&
• $
[n7uLQ"%N&jbjb>W
Fa_\OJVregLQ(Ydf#
• DeNAGE
• PRt7+528Xs=>WFa(c6
M
• ;b
WFa$&wKC4IlYC(q! • VU*3)0Sk(D%:'>WFa(AU&"#%8XTq4. 5-]`v]B(,1/#&
Slide 25
Slide 25 text
DeNA Tech Twitter !
DeNA !!)
&$(% '# *
Twitter " *