생성부터 활용까지, 게임데이터 Flow 따라가기

#DataLake

카카오게임즈의 모바일 게임 로그 데이터가 어떻게 생성되고 수집되는지, 이 수집된 데이터를 어떻게 처리하고 활용하는지에 대한 전체적인 과정을 소개합니다.

발표자 : kyle.l
카카오게임즈 BI팀 / DataLake TFT에서 ETL 및 데이터 엔지니어링 업무를 맡고 있는 kyle입니다

Transcript

1. $PQZSJHIU,BLBP$PSQ"MMSJHIUTSFTFSWFE3FEJTUSJCVUJPOPSQVCMJDEJTQMBZJTOPUQFSNJUUFEXJUIPVUXSJUUFOQFSNJTTJPOGSPN,BLBP
   ࢤࢿࠗఠഝਊө૑
   
   
   ѱ੐ؘ੉ఠ'MPXٮۄоӝ
   JG LBLBP
   
   ஠஠য়ѱ੐ૉ
   
   
   ੉೟൞LZMFM
   

   View Slide

2. *OUSP
   
   
   ؘ੉ఠࢤࢿ߂ࣻ૘
   
   
   ؘ੉ఠ୊ܻ
   
   
   ؘ੉ఠഝਊ
   
   
   0VUSP
   

   View Slide

3. *OUSP
   
   
   ؘ੉ఠࢤࢿ߂ࣻ૘
   
   
   ؘ੉ఠ୊ܻ
   
   
   ؘ੉ఠഝਊ
   
   
   0VUSP
   

   View Slide

4. ѱ੐۽Ӓؘ੉ఠ੄ഝਊ о஖
   

   View Slide

5. $MPVE
   
   
   %BUBMBLF
   
   
   #BTFEPO"84
   
   0O
   -
   QSFNJTF
   
   
   %8
   'MFYJCJMJUZ
   "OBMZUJDT
   

   View Slide

6. *OUSP
   
   
   ؘ੉ఠࢤࢿ߂ࣻ૘
   
   
   ؘ੉ఠ୊ܻ
   
   
   ؘ੉ఠഝਊ
   
   
   0VUSP
   

   View Slide

7. ۽Ӓؘ੉ఠࢤࢿ߂ࣻ૘
   

   View Slide

8. ;JOOZ1MBUGPSN
   

   View Slide

9. 4%,
   ;JOOZ1MBUGPSN
   

   View Slide

10. ;JOOZ1MBUGPSN
    0QFO"1*
    4FTTJPO
    -PHCVDLFU
    

    View Slide

11. (BNF
    
    
    -PH
    4FSWJDF
    
    
    -PH
    1MBUGPSN
    

    -PH
    BDUJPO JUFN QVSDIBTFj BVUI MPHJO IFBSUCFBUj ࢎ੹৘ড ࢸޙઑࢎj
    ۽Ӓؘ੉ఠ੄ઙܨ
    

    View Slide

12. (BNF
    
    
    -PH
    4FSWJDF
    
    
    -PH
    1MBUGPSN
    
    
    -PH
    ੹୓ড5# ೞܖড(#ঀૐо
    BDUJPO JUFN QVSDIBTFj BVUI MPHJO IFBSUCFBUj ࢎ੹৘ড ࢸޙઑࢎj
    ۽Ӓؘ੉ఠ੄ઙܨ
    

    View Slide

13. -PHCVDLFU ,JOFTJT%BUB4USFBN
    

    View Slide

14. -BNCEB
    ,JOFTJT
    
    
    %BUB4USFBN
    ,JOFTJT
    
    
    'JSFIPTF
    4
    
    
    *OHFTUJPO
    
    

    View Slide

15. -BNCEB
    ,JOFTJT
    
    
    %BUB4USFBN
    ,JOFTJT
    
    
    'JSFIPTF
    4
    
    
    *OHFTUJPO
    
    j
    
    
    JGDPEFBDUJPO
    
    
    PVUQVU"DUJPO-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZ
    
    
    
    
    GBJMFE@SFDPSETDIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU"DUJPO-JTU EFMJWFSZ@TUSFBN@OBNFEML
    -
    SFBM
    -
    BDUJPO
    
    
    
    FMJGDPEFJUFN
    
    
    PVUQVU*UFN-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZ
    
    
    
    
    GBJMFE@SFDPSETDIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU*UFN-JTU EFMJWFSZ@TUSFBN@OBNFEML
    -
    SFBM
    -
    JUFN
    
    
    
    FMJGDPEFrSFTPVSDF
    
    
    PVUQVU*UFN-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZ
    
    
    
    
    GBJMFE@SFDPSETDIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU*UFN-JTU EFMJWFSZ@TUSFBN@OBNFEML
    -
    SFBM
    -
    SFTPVSDF
    
    
    
    j
    

    View Slide

16. पदрਵ۽ࣻ૘غחؘ੉ఠܳ߄۽ࠅࣻ੓ਸө
    

    View Slide

17. 4USFBNJOH%BUB*OHFTUJPO
    $POOFDUPS
    
    
    PO&$
    ,BGLB &MBTUJD4FBSDI ,JCBOB
    $POOFDUPS
    
    
    PO&$
    "VUP4DBMJOH(SPVQ 0QFO4FBSDI
    -BNCEB
    ,JOFTJT
    
    
    %BUB4USFBN
    ,JOFTJT
    
    
    'JSFIPTF
    %BTI#PBSE
    

    View Slide

18. 4USFBNJOH%BUB*OHFTUJPO
    $POOFDUPS
    
    
    PO&$
    ,BGLB &MBTUJD4FBSDI ,JCBOB
    $POOFDUPS
    
    
    PO&$
    "VUP4DBMJOH(SPVQ 0QFO4FBSDI
    -BNCEB
    ,JOFTJT
    
    
    %BUB4USFBN
    ,JOFTJT
    
    
    'JSFIPTF
    %BTI#PBSE
    

    View Slide

19. 4USFBNJOH%BUB*OHFTUJPO
    $POOFDUPS
    
    
    PO&$
    ,BGLB &MBTUJD4FBSDI ,JCBOB
    $POOFDUPS
    
    
    PO&$
    "VUP4DBMJOH(SPVQ 0QFO4FBSDI
    %BTI#PBSE
    -BNCEB
    ,JOFTJT
    
    
    %BUB4USFBN
    ,JOFTJT
    
    
    'JSFIPTF
    

    View Slide

20. ৻ؘࠗ੉ఠࢤࢿ߂ࣻ૘
    

    View Slide

21. &YUFSOBM%BUB*OHFTUJPO
    *OHFTUJPO4FSWFS
    
    
    1FOUBIP
    
    )BEPPQ "1*
    ,BGLB
    ,BLBP;POF
    "1*
    
    
    ജਯ ҟҊؘ੉ఠj
    
    1VCMJD;POF
    ,BLBPHBNFT;POF
    .8""
    
    5SJHHFS
    4
    
    
    *OHFTUJPO
    
    
    .BOBHFE8PSLGMPXGPS"QBDIF"JSGMPX
    

    View Slide

22. Ӓۢࣻ૘ػؘ੉ఠחযڌѱೞաਃ
    

    View Slide

23. *OUSP
    
    
    ؘ੉ఠࢤࢿ߂ࣻ૘
    
    
    ؘ੉ఠ୊ܻ
    
    
    ؘ੉ఠഝਊ
    
    
    0VUSP
    

    View Slide

24. 'BDUPSZ
    %#
    "1*
    -PH
    4/4
    'JMF
    j
    *OTJHIU
    

    View Slide

25. 1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS
    j
    4
    
    
    "OBMZTJT
    
    4
    
    
    *OHFTUJPO
    
    3FE4IJGU
    
    
    .BSU
    
    0QFSBUPS
    4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF
    .8""
    4
    
    
    "SUJGBDUT
    
    %BTI#PBSE
    
    
    #*4
    
    'BDUPSZ
    
    
    8PSLGMPX
    
    &.3
    
    
    $MVTUFS
    j
    (JU)VC
    
    
    &OUFSQSJTF
    +FOLJOT
    

    View Slide

26. .8""
    4
    
    
    "OBMZTJT
    
    4
    
    
    "SUJGBDUT
    
    &.3
    
    
    $MVTUFS
    (JU)VC
    
    
    &OUFSQSJTF
    4
    
    
    *OHFTUJPO
    
    3FE4IJGU
    
    
    .BSU
    
    +FOLJOT
    %BTI#PBSE
    
    
    #*4
    
    'BDUPSZ
    
    
    8PSLGMPX
    
    1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS
    j
    0QFSBUPS
    4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF
    j
    

    View Slide

27. 4
    
    
    *OHFTUJPO
    
    .8""
    4
    
    
    "SUJGBDUT
    
    (JU)VC
    
    
    &OUFSQSJTF
    +FOLJOT
    4
    
    
    "OBMZTJT
    
    &.3
    
    
    $MVTUFS
    3FE4IJGU
    
    
    .BSU
    
    %BTI#PBSE
    
    
    #*4
    
    1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS
    j
    0QFSBUPS
    4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF
    'BDUPSZ
    
    
    8PSLGMPX
    
    j
    

    View Slide

28. ৘द DPEFrSFTPVSDFs
    
    
    
    \BQQ*Et QMBZFS*Eĉ BNPVOU DIBSBDUFS*Eĉ DIBSBDUFS-W
    DIBSBDUFS/Nĉ DPVOUSZ58 EFMUB HBNF8PSMEĉXPSME NPE5JNF
    NPE5ZQFBEE PT8JOEPXT QBJEV QMBZFS-W S$VSSFODZ(PME SFBTPO"@26&45
    SFTPVSDF"UUS TVC3FBTPO&MJUF%VOHFPO MBOH[I
    -
    IBOU TFTTJPO*E
    DMJFOU*Qu NBSLFUVOLOPXO DPEFSFTPVSDF QVCMJTI5JNF MPH*Eĉ
    ^
    
    
    \BQQ*Et QMBZFS*E BNPVOU DIBSBDUFS*E DIBSBDUFS-W
    DIBSBDUFS/N DPVOUSZ,3 EFMUB HBNF8PSMEĉXPSME NPE5JNF
    NPE5ZQFBEE PT8JOEPXT QBJEV QMBZFS-W S$VSSFODZ(PME SFBTPO"@1-": SFTPVSDF"UUS
    TVC3FBTPO'JFME MBOHLP TFTTJPO*E DMJFOU*Q NBSLFU
    VOLOPXO DPEFSFTPVSDF QVCMJTI5JNF MPH*Eĉ^
    

    View Slide

29. .8""
    4
    
    
    "SUJGBDUT
    
    (JU)VC
    
    
    &OUFSQSJTF
    +FOLJOT
    4
    
    
    "OBMZTJT
    
    &.3
    
    
    $MVTUFS
    3FE4IJGU
    
    
    .BSU
    
    %BTI#PBSE
    
    
    #*4
    
    4
    
    
    *OHFTUJPO
    
    1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS
    j
    0QFSBUPS
    4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF
    'BDUPSZ
    
    
    8PSLGMPX
    
    j
    

    View Slide

30. QMBZFSJE HET@DE STO@DE BNU TFSWFS
    HPME "@1-": XPSME
    HPME "@26&45 XPSME
    HPME "@1-": XPSME
    HPME "@26&45 XPSME
    HPME "@1-": XPSME
    HPME "@1-": XPSME
    HPME "@26&45 XPSME
    HPME "@26&45 XPSME
    HPME "@26&45 XPSME
    HPME "@1-": XPSME
    %BUB8BSFIPVTF
    HET@DE STO@DE TFSWFS DOU TVN
    HPME "@1-": XPSME
    HPME "@1-": XPSME
    HPME "@1-": XPSME
    HPME "@26&45 XPSME
    HPME "@26&45 XPSME
    4VNNBSZ%BUB
    

    View Slide

31. 3FE4IJGU
    
    
    .BSU
    
    %BTI#PBSE
    
    
    #*4
    
    .8""
    +FOLJOT
    4
    
    
    "SUJGBDUT
    
    (JU)VC
    
    
    &OUFSQSJTF
    4
    
    
    "OBMZTJT
    
    &.3
    
    
    $MVTUFS
    4
    
    
    *OHFTUJPO
    
    'BDUPSZ
    
    
    8PSLGMPX
    
    1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS
    j
    0QFSBUPS
    4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF
    j
    

    View Slide

32. 3FE4IJGU
    
    
    .BSU
    
    HET@DE STO@DE TFSWFS DOU TVN
    HPME "@1-": XPSME
    HPME "@1-": XPSME
    HPME "@1-": XPSME
    HPME "@26&45 XPSME
    HPME "@26&45 XPSME
    3FTPVSDF 4VNNBSZ%BUB
    
    *UFN
    
    
    "DUJPO
    
    
    3PVOE
    
    
    1VSDIBTF
    
    
    *EQ-PHJO
    
    
    j
    
    
    j
    4
    
    
    "OBMZTJT
    
    .8""
    %BTI#PBSE
    
    
    #*4
    
    4QFDUSVN
    

    View Slide

33. *OUSP
    
    
    ؘ੉ఠࢤࢿ߂ࣻ૘
    
    
    ؘ੉ఠ୊ܻ
    
    
    ؘ੉ఠഝਊ
    
    
    0VUSP
    

    View Slide

34. 4
    
    
    "OBMZTJT
    
    3FE4IJGU
    
    
    .BSU
    
    4QFDUSVN
    "UIFOB
    %BTI#PBSE
    "E
    -
    IPD
    
    
    2VFSZ
    #VTJOFTT*OUFMMJHFODF
    3FQPSU
    

    View Slide

35. 4
    
    
    "OBMZTJT
    
    3FE4IJGU
    
    
    .BSU
    
    4QFDUSVN
    "UIFOB
    %BTI#PBSE
    "E
    -
    IPD
    
    
    2VFSZ
    #VTJOFTT*OUFMMJHFODF
    3FQPSU
    

    View Slide

36. #VTJOFTT*OUFMMJHFODF
    3FQPSU
    "E
    -
    IPD
    
    
    2VFSZ
    4
    
    
    "OBMZTJT
    
    3FE4IJGU
    
    
    .BSU
    
    4QFDUSVN
    %BTI#PBSE
    "UIFOB
    

    View Slide

37. $POOFDUPS
    
    
    PO&$
    ,BGLB $POOFDUPS
    
    
    PO&$
    "VUP4DBMJOH(SPVQ
    4
    
    
    *OHFTUJPO
    
    ,JOFTJT
    
    
    'JSFIPTF
    "CVTJOH
    
    
    %FUFDUJPO
    3FBMUJNF"CVTJOH%FUFDUJPO
    3.4ZTUFN
    "UIFOB
    

    View Slide

38. 
    
    
    
    
    
    য࠭૚ߊࢤ
    ੿೧૓3VMFী੄೧ࢲ੉࢚ਬ੷ܳఐ૑
    
    
    ઱۽؀दࠁ٘ীࢲয࠭૚ਸ౵ঈ
    
    
    ঌۈ ੗زࠁഐઑ஖ *1ରױ 71/ "4/١
    
    
    
    3VMF
    -
    CBTFE"CVTJOH%FUFDUJPO 3.
    
    

    View Slide

39. ੿೧૓3VMF੉ইפۄؘ੉ఠ ঌҊ્ܻ "*ݽ؛١ਸ੉ਊೠ੉࢚ఐ૑
    
    
    ਬ੷೯زӝ߈੉࢚੤ചݽפఠ݂
    
    
    ੘স੢ఐ૑ܳਤೠѢېࣗݽפఠ݂
    
    
    ೻؊҅੿
    ਕழ҅੿
    ׮੉ইҳݒ҅੿
    ਕழ҅੿
    ਕழ҅੿
    ਕழ҅੿
    ਕழ҅੿
    ਕழ҅੿
    .PEFM
    -
    CBTFE"CVTJOH%FUFDUJPO
    

    View Slide

40. %BUB
    
    
    "OBMZTJT
    ѱ੐߹ਬ੷੄-57 -JGF5JNF7BMVF
    ৘ஏ
    
    
    ೒ۨ੉زೱ߂੉ఎ৘ஏ
    
    
    ݃ா౴ബਯ࠙ࢳ
    
    
    %BUB"OBMZTJT
    

    View Slide

41. *OUSP
    
    
    ؘ੉ఠࢤࢿ߂ࣻ૘
    
    
    ؘ੉ఠ୊ܻ
    
    
    ؘ੉ఠഝਊ
    
    
    0VUSP
    

    View Slide

42. %BUB%JTDPWFSZ1MBUGPSN ୶ୌदझమ
    

    View Slide

43. %BUB%JTDPWFSZ1MBUGPSN ୶ୌदझమ
    

    View Slide

44. хࢎ೤פ׮
    

    View Slide

45. &0%
    

    View Slide