생성부터 활용까지, 게임데이터 Flow 따라가기

#DataLake

카카오게임즈의 모바일 게임 로그 데이터가 어떻게 생성되고 수집되는지, 이 수집된 데이터를 어떻게 처리하고 활용하는지에 대한 전체적인 과정을 소개합니다.

발표자 : kyle.l
카카오게임즈 BI팀 / DataLake TFT에서 ETL 및 데이터 엔지니어링 업무를 맡고 있는 kyle입니다

Transcript

1. $PQZSJHIU
   
   ,BLBP
   $PSQ
   "MM
   SJHIUT
   SFTFSWFE
   3FEJTUSJCVUJPO
   PS
   QVCMJD
   EJTQMBZ
   JT
   OPU
   QFSNJUUFE
   XJUIPVU
   XSJUUFO
   QFSNJTTJPO
   GSPN
   ,BLBP ࢤࢿࠗఠ
   ഝਊө૑ ѱ੐ؘ੉ఠ
   'MPX
   ٮۄоӝ JG LBLBP  ஠஠য়ѱ੐ૉ ੉೟൞
   LZMFM
2. *OUSP ؘ੉ఠ
   ࢤࢿ
   ߂
   ࣻ૘ ؘ੉ఠ
   ୊ܻ ؘ੉ఠ
   ഝਊ 0VUSP
3. *OUSP ؘ੉ఠ
   ࢤࢿ
   ߂
   ࣻ૘ ؘ੉ఠ
   ୊ܻ ؘ੉ఠ
   ഝਊ 0VUSP
4. ѱ੐
   ۽Ӓ
   ؘ੉ఠ੄
   ഝਊ
   о஖
5. $MPVE %BUBMBLF #BTFE
   PO
   "84 0O - QSFNJTF %8 'MFYJCJMJUZ "OBMZUJDT
6. *OUSP ؘ੉ఠ
   ࢤࢿ
   ߂
   ࣻ૘ ؘ੉ఠ
   ୊ܻ ؘ੉ఠ
   ഝਊ 0VUSP
7. ۽Ӓ
   ؘ੉ఠ
   ࢤࢿ
   ߂
   ࣻ૘
8. ;JOOZ
   1MBUGPSN
9. 4%, ;JOOZ
   1MBUGPSN
10. ;JOOZ
    1MBUGPSN 0QFO"1* 4FTTJPO -PHCVDLFU
11. (BNF -PH 4FSWJDF -PH 1MBUGPSN 
 -PH BDUJPO
    JUFN
    QVSDIBTFj BVUI
    MPHJO
    IFBSUCFBUj ࢎ੹৘ড
    ࢸޙઑࢎj ۽Ӓ
    ؘ੉ఠ੄
    ઙܨ
12. (BNF -PH 4FSWJDF -PH 1MBUGPSN -PH ੹୓
    ড
    5#
    ೞܖ
    ড
    (#ঀ
    ૐо BDUJPO
    JUFN
    QVSDIBTFj BVUI
    MPHJO
    IFBSUCFBUj ࢎ੹৘ড
    ࢸޙઑࢎj ۽Ӓ
    ؘ੉ఠ੄
    ઙܨ
13. -PHCVDLFU ,JOFTJT
    %BUB
    4USFBN
14. -BNCEB ,JOFTJT %BUB
    4USFBN ,JOFTJT 'JSFIPTF 4 *OHFTUJPO
15. -BNCEB ,JOFTJT %BUB
    4USFBN ,JOFTJT 'JSFIPTF 4 *OHFTUJPO j JG
    DPEF
    
    BDUJPO
    
    
    
    
    
    
    
    
    
    
    
    PVUQVU"DUJPO-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZ
    
    
    
    
    
    
    
    
    
    
    
    GBJMFE@SFDPSET
    
    DIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU"DUJPO-JTU
    EFMJWFSZ@TUSFBN@OBNFEML - SFBM - BDUJPO
    
    
    
    
    
    
    
    FMJG
    DPEF
    
    JUFN
    
    
    
    
    
    
    
    
    
    
    
    PVUQVU*UFN-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZ
    
    
    
    
    
    
    
    
    
    
    
    GBJMFE@SFDPSET
    
    DIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU*UFN-JTU
    EFMJWFSZ@TUSFBN@OBNFEML - SFBM - JUFN
    
    
    
    
    
    
    
    FMJG
    DPEF
    
    rSFTPVSDF
    
    
    
    
    
    
    
    
    
    
    
    PVUQVU*UFN-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZ
    
    
    
    
    
    
    
    
    
    
    
    GBJMFE@SFDPSET
    
    DIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU*UFN-JTU
    EFMJWFSZ@TUSFBN@OBNFEML - SFBM - SFTPVSDF j
16. पदрਵ۽
    ࣻ૘غח
    ؘ੉ఠܳ
    ߄۽
    ࠅ
    ࣻ
    ੓ਸө
17. 4USFBNJOH
    %BUB
    *OHFTUJPO $POOFDUPS PO
    &$ ,BGLB &MBTUJD4FBSDI ,JCBOB $POOFDUPS PO
    &$ "VUP
    4DBMJOH
    (SPVQ 0QFO4FBSDI -BNCEB ,JOFTJT %BUB
    4USFBN ,JOFTJT 'JSFIPTF %BTI#PBSE
18. 4USFBNJOH
    %BUB
    *OHFTUJPO $POOFDUPS PO
    &$ ,BGLB &MBTUJD4FBSDI ,JCBOB $POOFDUPS PO
    &$ "VUP
    4DBMJOH
    (SPVQ 0QFO4FBSDI -BNCEB ,JOFTJT %BUB
    4USFBN ,JOFTJT 'JSFIPTF %BTI#PBSE
19. 4USFBNJOH
    %BUB
    *OHFTUJPO $POOFDUPS PO
    &$ ,BGLB &MBTUJD4FBSDI ,JCBOB $POOFDUPS PO
    &$ "VUP
    4DBMJOH
    (SPVQ 0QFO4FBSDI %BTI#PBSE -BNCEB ,JOFTJT %BUB
    4USFBN ,JOFTJT 'JSFIPTF
20. ৻ࠗ
    ؘ੉ఠ
    ࢤࢿ
    ߂
    ࣻ૘
21. &YUFSOBM
    %BUB
    *OHFTUJPO *OHFTUJPO
    4FSWFS 1FOUBIP )BEPPQ "1* ,BGLB ,BLBP
    ;POF "1*
    ജਯ
    ҟҊؘ੉ఠ
    j
    1VCMJD
    ;POF ,BLBPHBNFT
    ;POF .8"" 5SJHHFS 4 *OHFTUJPO 
    .BOBHFE
    8PSLGMPX
    GPS
    "QBDIF
    "JSGMPX
22. Ӓۢ
    ࣻ૘ػ
    ؘ੉ఠח
    যڌѱ
    ೞաਃ
23. *OUSP ؘ੉ఠ
    ࢤࢿ
    ߂
    ࣻ૘ ؘ੉ఠ
    ୊ܻ ؘ੉ఠ
    ഝਊ 0VUSP
24. 'BDUPSZ %# "1* -PH 4/4 'JMF j *OTJHIU
25. 1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS j 4 "OBMZTJT 4 *OHFTUJPO 3FE4IJGU .BSU 0QFSBUPS 4UFQ 4BWF
    SFQSPDFTT
    EBUFUJNF $SFBUF
    UBTL
    HSPVQ 3FGSFTI
    DVCF .8"" 4 "SUJGBDUT %BTI#PBSE #*4 'BDUPSZ 8PSLGMPX &.3 $MVTUFS j (JU)VC &OUFSQSJTF +FOLJOT
26. .8"" 4 "OBMZTJT 4 "SUJGBDUT &.3 $MVTUFS (JU)VC &OUFSQSJTF 4 *OHFTUJPO 3FE4IJGU .BSU +FOLJOT %BTI#PBSE #*4 'BDUPSZ 8PSLGMPX 1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS j 0QFSBUPS 4UFQ 4BWF
    SFQSPDFTT
    EBUFUJNF $SFBUF
    UBTL
    HSPVQ 3FGSFTI
    DVCF j
27. 4 *OHFTUJPO .8"" 4 "SUJGBDUT (JU)VC &OUFSQSJTF +FOLJOT 4 "OBMZTJT &.3 $MVTUFS 3FE4IJGU .BSU %BTI#PBSE #*4 1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS j 0QFSBUPS 4UFQ 4BWF
    SFQSPDFTT
    EBUFUJNF $SFBUF
    UBTL
    HSPVQ 3FGSFTI
    DVCF 'BDUPSZ 8PSLGMPX j
28. ৘द
    DPEF
    
    rSFTPVSDFs \BQQ*E
    t
    QMBZFS*E
    ĉ 
    BNPVOU
    
    DIBSBDUFS*E
    ĉ 
    DIBSBDUFS-W
    
    DIBSBDUFS/N
    ĉ 
    DPVOUSZ
    58
    EFMUB
    
    HBNF8PSME
    ĉXPSME
    NPE5JNF
    
    NPE5ZQF
    BEE
    PT
    8JOEPXT
    QBJE
    V
    QMBZFS-W
    
    S$VSSFODZ
    (PME
    SFBTPO
    "@26&45
    SFTPVSDF"UUS
    
    TVC3FBTPO
    &MJUF%VOHFPO
    MBOH
    [I - IBOU
    TFTTJPO*E
    
    DMJFOU*Q
    u
    
    NBSLFU
    VOLOPXO
    DPEF
    SFTPVSDF
    QVCMJTI5JNF
    
    MPH*E
    ĉ     ^ \BQQ*E
    t
    QMBZFS*E
     
    BNPVOU
    
    DIBSBDUFS*E
     
    DIBSBDUFS-W
    
    DIBSBDUFS/N
     
    DPVOUSZ
    ,3
    EFMUB
    
    HBNF8PSME
    ĉXPSME
    NPE5JNF
    
    NPE5ZQF
    BEE
    PT
    8JOEPXT
    QBJE
    V
    QMBZFS-W
    
    S$VSSFODZ
    (PME
    SFBTPO
    "@1-":
    SFTPVSDF"UUS
    
    TVC3FBTPO
    'JFME
    MBOH
    LP
    TFTTJPO*E
    
    
    
    DMJFOU*Q
    
    
    
    
    NBSLFU
    VOLOPXO
    DPEF
    SFTPVSDF
    QVCMJTI5JNF
    
    MPH*E
    ĉ     ^
29. .8"" 4 "SUJGBDUT (JU)VC &OUFSQSJTF +FOLJOT 4 "OBMZTJT &.3 $MVTUFS 3FE4IJGU .BSU %BTI#PBSE #*4 4 *OHFTUJPO 1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS j 0QFSBUPS 4UFQ 4BWF
    SFQSPDFTT
    EBUFUJNF $SFBUF
    UBTL
    HSPVQ 3FGSFTI
    DVCF 'BDUPSZ 8PSLGMPX j
30. QMBZFSJE HET@DE STO@DE BNU TFSWFS  HPME "@1-":  XPSME  HPME "@26&45  XPSME  HPME "@1-":  XPSME  HPME "@26&45  XPSME  HPME "@1-":  XPSME  HPME "@1-":  XPSME  HPME "@26&45  XPSME  HPME "@26&45  XPSME  HPME "@26&45  XPSME  HPME "@1-":  XPSME %BUB
    8BSFIPVTF HET@DE STO@DE TFSWFS DOU TVN HPME "@1-": XPSME   HPME "@1-": XPSME   HPME "@1-": XPSME   HPME "@26&45 XPSME   HPME "@26&45 XPSME   4VNNBSZ
    %BUB
31. 3FE4IJGU .BSU %BTI#PBSE #*4 .8"" +FOLJOT 4 "SUJGBDUT (JU)VC &OUFSQSJTF 4 "OBMZTJT &.3 $MVTUFS 4 *OHFTUJPO 'BDUPSZ 8PSLGMPX 1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS j 0QFSBUPS 4UFQ 4BWF
    SFQSPDFTT
    EBUFUJNF $SFBUF
    UBTL
    HSPVQ 3FGSFTI
    DVCF j
32. 3FE4IJGU .BSU HET@DE STO@DE TFSWFS DOU TVN HPME "@1-": XPSME   HPME "@1-": XPSME   HPME "@1-": XPSME   HPME "@26&45 XPSME   HPME "@26&45 XPSME   3FTPVSDF
    4VNNBSZ
    %BUB *UFN "DUJPO 3PVOE 1VSDIBTF *EQ-PHJO j j 4 "OBMZTJT .8"" %BTI#PBSE #*4 4QFDUSVN
33. *OUSP ؘ੉ఠ
    ࢤࢿ
    ߂
    ࣻ૘ ؘ੉ఠ
    ୊ܻ ؘ੉ఠ
    ഝਊ 0VUSP
34. 4 "OBMZTJT 3FE4IJGU .BSU 4QFDUSVN "UIFOB %BTI#PBSE "E - IPD
    2VFSZ #VTJOFTT
    *OUFMMJHFODF
    3FQPSU
35. 4 "OBMZTJT 3FE4IJGU .BSU 4QFDUSVN "UIFOB %BTI#PBSE "E - IPD
    2VFSZ #VTJOFTT
    *OUFMMJHFODF
    3FQPSU
36. #VTJOFTT
    *OUFMMJHFODF
    3FQPSU "E - IPD
    2VFSZ 4 "OBMZTJT 3FE4IJGU .BSU 4QFDUSVN %BTI#PBSE "UIFOB
37. $POOFDUPS PO
    &$ ,BGLB $POOFDUPS PO
    &$ "VUP
    4DBMJOH
    (SPVQ 4 *OHFTUJPO ,JOFTJT 'JSFIPTF "CVTJOH %FUFDUJPO 3FBMUJNF
    "CVTJOH
    %FUFDUJPO 3.
    4ZTUFN "UIFOB
38.                য࠭૚
    ߊࢤ  ੿೧૓
    3VMFী
    ੄೧ࢲ
    ੉࢚
    ਬ੷ܳ
    ఐ૑  ઱۽
    ؀दࠁ٘ীࢲ
    য࠭૚ਸ
    ౵ঈ  ঌۈ
    ੗ز
    ࠁഐઑ஖
    *1
    ରױ 71/
    "4/
    ١ 3VMF - CBTFE
    "CVTJOH
    %FUFDUJPO
    3.
39.  ੿೧૓
    3VMF੉
    ইפۄ
    ؘ੉ఠ
    ঌҊ્ܻ
    "*ݽ؛
    ١ਸ
    ੉ਊೠ
    ੉࢚ఐ૑  ਬ੷
    ೯ز
    ӝ߈
    ੉࢚
    ੤ച
    ݽפఠ݂  ੘স੢
    ఐ૑ܳ
    ਤೠ
    Ѣېࣗ
    ݽפఠ݂ ೻؊
    ҅੿ ਕழ
    ҅੿ ׮੉ই
    ҳݒ҅੿ ਕழ
    ҅੿ ਕழ
    ҅੿ ਕழ
    ҅੿ ਕழ
    ҅੿ ਕழ
    ҅੿ .PEFM - CBTFE
    "CVTJOH
    %FUFDUJPO
40. %BUB "OBMZTJT  ѱ੐߹
    ਬ੷੄
    -57
    -JGF
    5JNF
    7BMVF
    ৘ஏ  ೒ۨ੉
    زೱ
    ߂
    ੉ఎ
    ৘ஏ  ݃ா౴
    ബਯ
    ࠙ࢳ %BUB
    "OBMZTJT
41. *OUSP ؘ੉ఠ
    ࢤࢿ
    ߂
    ࣻ૘ ؘ੉ఠ
    ୊ܻ ؘ੉ఠ
    ഝਊ 0VUSP
42. %BUB
    %JTDPWFSZ
    1MBUGPSN ୶ୌ
    दझమ
43. %BUB
    %JTDPWFSZ
    1MBUGPSN ୶ୌ
    दझమ
44. хࢎ೤פ׮
45. &0%