Upgrade to Pro — share decks privately, control downloads, hide ads and more …

생성부터 활용까지, 게임데이터 Flow 따라가기

kakao
PRO
December 09, 2022

생성부터 활용까지, 게임데이터 Flow 따라가기

#DataLake

카카오게임즈의 모바일 게임 로그 데이터가 어떻게 생성되고 수집되는지, 이 수집된 데이터를 어떻게 처리하고 활용하는지에 대한 전체적인 과정을 소개합니다.

발표자 : kyle.l
카카오게임즈 BI팀 / DataLake TFT에서 ETL 및 데이터 엔지니어링 업무를 맡고 있는 kyle입니다

kakao
PRO

December 09, 2022
Tweet

More Decks by kakao

Other Decks in Programming

Transcript

  1. $PQZSJHIU,BLBP$PSQ"MMSJHIUTSFTFSWFE3FEJTUSJCVUJPOPSQVCMJDEJTQMBZJTOPUQFSNJUUFEXJUIPVUXSJUUFOQFSNJTTJPOGSPN,BLBP
    ࢤࢿࠗఠഝਊө૑


    ѱ੐ؘ੉ఠ'MPXٮۄоӝ
    JG LBLBP

    ஠஠য়ѱ੐ૉ


    ੉೟൞LZMFM

    View Slide

  2. *OUSP


    ؘ੉ఠࢤࢿ߂ࣻ૘


    ؘ੉ఠ୊ܻ


    ؘ੉ఠഝਊ


    0VUSP

    View Slide

  3. *OUSP


    ؘ੉ఠࢤࢿ߂ࣻ૘


    ؘ੉ఠ୊ܻ


    ؘ੉ఠഝਊ


    0VUSP

    View Slide

  4. ѱ੐۽Ӓؘ੉ఠ੄ഝਊ о஖

    View Slide

  5. $MPVE


    %BUBMBLF


    #BTFEPO"84

    0O
    -
    QSFNJTF


    %8
    'MFYJCJMJUZ
    "OBMZUJDT

    View Slide

  6. *OUSP


    ؘ੉ఠࢤࢿ߂ࣻ૘


    ؘ੉ఠ୊ܻ


    ؘ੉ఠഝਊ


    0VUSP

    View Slide

  7. ۽Ӓؘ੉ఠࢤࢿ߂ࣻ૘

    View Slide

  8. ;JOOZ1MBUGPSN

    View Slide

  9. 4%,
    ;JOOZ1MBUGPSN

    View Slide

  10. ;JOOZ1MBUGPSN
    0QFO"1*
    4FTTJPO
    -PHCVDLFU

    View Slide

  11. (BNF


    -PH
    4FSWJDF


    -PH
    1MBUGPSN

    -PH
    BDUJPO JUFN QVSDIBTFj BVUI MPHJO IFBSUCFBUj ࢎ੹৘ড ࢸޙઑࢎj
    ۽Ӓؘ੉ఠ੄ઙܨ

    View Slide

  12. (BNF


    -PH
    4FSWJDF


    -PH
    1MBUGPSN


    -PH
    ੹୓ড5# ೞܖড(#ঀૐо
    BDUJPO JUFN QVSDIBTFj BVUI MPHJO IFBSUCFBUj ࢎ੹৘ড ࢸޙઑࢎj
    ۽Ӓؘ੉ఠ੄ઙܨ

    View Slide

  13. -PHCVDLFU ,JOFTJT%BUB4USFBN

    View Slide

  14. -BNCEB
    ,JOFTJT


    %BUB4USFBN
    ,JOFTJT


    'JSFIPTF
    4


    *OHFTUJPO

    View Slide

  15. -BNCEB
    ,JOFTJT


    %BUB4USFBN
    ,JOFTJT


    'JSFIPTF
    4


    *OHFTUJPO

    j


    JGDPEFBDUJPO


    PVUQVU"DUJPO-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZ




    GBJMFE@SFDPSETDIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU"DUJPO-JTU EFMJWFSZ@TUSFBN@OBNFEML
    -
    SFBM
    -
    BDUJPO



    FMJGDPEFJUFN


    PVUQVU*UFN-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZ




    GBJMFE@SFDPSETDIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU*UFN-JTU EFMJWFSZ@TUSFBN@OBNFEML
    -
    SFBM
    -
    JUFN



    FMJGDPEFrSFTPVSDF


    PVUQVU*UFN-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZ




    GBJMFE@SFDPSETDIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU*UFN-JTU EFMJWFSZ@TUSFBN@OBNFEML
    -
    SFBM
    -
    SFTPVSDF



    j

    View Slide

  16. पदрਵ۽ࣻ૘غחؘ੉ఠܳ߄۽ࠅࣻ੓ਸө

    View Slide

  17. 4USFBNJOH%BUB*OHFTUJPO
    $POOFDUPS


    PO&$
    ,BGLB &MBTUJD4FBSDI ,JCBOB
    $POOFDUPS


    PO&$
    "VUP4DBMJOH(SPVQ 0QFO4FBSDI
    -BNCEB
    ,JOFTJT


    %BUB4USFBN
    ,JOFTJT


    'JSFIPTF
    %BTI#PBSE

    View Slide

  18. 4USFBNJOH%BUB*OHFTUJPO
    $POOFDUPS


    PO&$
    ,BGLB &MBTUJD4FBSDI ,JCBOB
    $POOFDUPS


    PO&$
    "VUP4DBMJOH(SPVQ 0QFO4FBSDI
    -BNCEB
    ,JOFTJT


    %BUB4USFBN
    ,JOFTJT


    'JSFIPTF
    %BTI#PBSE

    View Slide

  19. 4USFBNJOH%BUB*OHFTUJPO
    $POOFDUPS


    PO&$
    ,BGLB &MBTUJD4FBSDI ,JCBOB
    $POOFDUPS


    PO&$
    "VUP4DBMJOH(SPVQ 0QFO4FBSDI
    %BTI#PBSE
    -BNCEB
    ,JOFTJT


    %BUB4USFBN
    ,JOFTJT


    'JSFIPTF

    View Slide

  20. ৻ؘࠗ੉ఠࢤࢿ߂ࣻ૘

    View Slide

  21. &YUFSOBM%BUB*OHFTUJPO
    *OHFTUJPO4FSWFS


    1FOUBIP

    )BEPPQ "1*
    ,BGLB
    ,BLBP;POF
    "1*


    ജਯ ҟҊؘ੉ఠj

    1VCMJD;POF
    ,BLBPHBNFT;POF
    .8""

    5SJHHFS
    4


    *OHFTUJPO


    .BOBHFE8PSLGMPXGPS"QBDIF"JSGMPX

    View Slide

  22. Ӓۢࣻ૘ػؘ੉ఠחযڌѱೞաਃ

    View Slide

  23. *OUSP


    ؘ੉ఠࢤࢿ߂ࣻ૘


    ؘ੉ఠ୊ܻ


    ؘ੉ఠഝਊ


    0VUSP

    View Slide

  24. 'BDUPSZ
    %#
    "1*
    -PH
    4/4
    'JMF
    j
    *OTJHIU

    View Slide

  25. 1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS
    j
    4


    "OBMZTJT

    4


    *OHFTUJPO

    3FE4IJGU


    .BSU

    0QFSBUPS
    4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF
    .8""
    4


    "SUJGBDUT

    %BTI#PBSE


    #*4

    'BDUPSZ


    8PSLGMPX

    &.3


    $MVTUFS
    j
    (JU)VC


    &OUFSQSJTF
    +FOLJOT

    View Slide

  26. .8""
    4


    "OBMZTJT

    4


    "SUJGBDUT

    &.3


    $MVTUFS
    (JU)VC


    &OUFSQSJTF
    4


    *OHFTUJPO

    3FE4IJGU


    .BSU

    +FOLJOT
    %BTI#PBSE


    #*4

    'BDUPSZ


    8PSLGMPX

    1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS
    j
    0QFSBUPS
    4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF
    j

    View Slide

  27. 4


    *OHFTUJPO

    .8""
    4


    "SUJGBDUT

    (JU)VC


    &OUFSQSJTF
    +FOLJOT
    4


    "OBMZTJT

    &.3


    $MVTUFS
    3FE4IJGU


    .BSU

    %BTI#PBSE


    #*4

    1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS
    j
    0QFSBUPS
    4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF
    'BDUPSZ


    8PSLGMPX

    j

    View Slide

  28. ৘द DPEFrSFTPVSDFs



    \BQQ*Et QMBZFS*Eĉ BNPVOU DIBSBDUFS*Eĉ DIBSBDUFS-W
    DIBSBDUFS/Nĉ DPVOUSZ58 EFMUB HBNF8PSMEĉXPSME NPE5JNF
    NPE5ZQFBEE PT8JOEPXT QBJEV QMBZFS-W S$VSSFODZ(PME SFBTPO"@26&45
    SFTPVSDF"UUS TVC3FBTPO&MJUF%VOHFPO MBOH[I
    -
    IBOU TFTTJPO*E
    DMJFOU*Qu NBSLFUVOLOPXO DPEFSFTPVSDF QVCMJTI5JNF MPH*Eĉ
    ^


    \BQQ*Et QMBZFS*E BNPVOU DIBSBDUFS*E DIBSBDUFS-W
    DIBSBDUFS/N DPVOUSZ,3 EFMUB HBNF8PSMEĉXPSME NPE5JNF
    NPE5ZQFBEE PT8JOEPXT QBJEV QMBZFS-W S$VSSFODZ(PME SFBTPO"@1-": SFTPVSDF"UUS
    TVC3FBTPO'JFME MBOHLP TFTTJPO*E DMJFOU*Q NBSLFU
    VOLOPXO DPEFSFTPVSDF QVCMJTI5JNF MPH*Eĉ^

    View Slide

  29. .8""
    4


    "SUJGBDUT

    (JU)VC


    &OUFSQSJTF
    +FOLJOT
    4


    "OBMZTJT

    &.3


    $MVTUFS
    3FE4IJGU


    .BSU

    %BTI#PBSE


    #*4

    4


    *OHFTUJPO

    1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS
    j
    0QFSBUPS
    4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF
    'BDUPSZ


    8PSLGMPX

    j

    View Slide

  30. QMBZFSJE HET@DE STO@DE BNU TFSWFS
    HPME "@1-": XPSME
    HPME "@26&45 XPSME
    HPME "@1-": XPSME
    HPME "@26&45 XPSME
    HPME "@1-": XPSME
    HPME "@1-": XPSME
    HPME "@26&45 XPSME
    HPME "@26&45 XPSME
    HPME "@26&45 XPSME
    HPME "@1-": XPSME
    %BUB8BSFIPVTF
    HET@DE STO@DE TFSWFS DOU TVN
    HPME "@1-": XPSME
    HPME "@1-": XPSME
    HPME "@1-": XPSME
    HPME "@26&45 XPSME
    HPME "@26&45 XPSME
    4VNNBSZ%BUB

    View Slide

  31. 3FE4IJGU


    .BSU

    %BTI#PBSE


    #*4

    .8""
    +FOLJOT
    4


    "SUJGBDUT

    (JU)VC


    &OUFSQSJTF
    4


    "OBMZTJT

    &.3


    $MVTUFS
    4


    *OHFTUJPO

    'BDUPSZ


    8PSLGMPX

    1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPS
    j
    0QFSBUPS
    4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF
    j

    View Slide

  32. 3FE4IJGU


    .BSU

    HET@DE STO@DE TFSWFS DOU TVN
    HPME "@1-": XPSME
    HPME "@1-": XPSME
    HPME "@1-": XPSME
    HPME "@26&45 XPSME
    HPME "@26&45 XPSME
    3FTPVSDF 4VNNBSZ%BUB

    *UFN


    "DUJPO


    3PVOE


    1VSDIBTF


    *EQ-PHJO


    j


    j
    4


    "OBMZTJT

    .8""
    %BTI#PBSE


    #*4

    4QFDUSVN

    View Slide

  33. *OUSP


    ؘ੉ఠࢤࢿ߂ࣻ૘


    ؘ੉ఠ୊ܻ


    ؘ੉ఠഝਊ


    0VUSP

    View Slide

  34. 4


    "OBMZTJT

    3FE4IJGU


    .BSU

    4QFDUSVN
    "UIFOB
    %BTI#PBSE
    "E
    -
    IPD


    2VFSZ
    #VTJOFTT*OUFMMJHFODF
    3FQPSU

    View Slide

  35. 4


    "OBMZTJT

    3FE4IJGU


    .BSU

    4QFDUSVN
    "UIFOB
    %BTI#PBSE
    "E
    -
    IPD


    2VFSZ
    #VTJOFTT*OUFMMJHFODF
    3FQPSU

    View Slide

  36. #VTJOFTT*OUFMMJHFODF
    3FQPSU
    "E
    -
    IPD


    2VFSZ
    4


    "OBMZTJT

    3FE4IJGU


    .BSU

    4QFDUSVN
    %BTI#PBSE
    "UIFOB

    View Slide

  37. $POOFDUPS


    PO&$
    ,BGLB $POOFDUPS


    PO&$
    "VUP4DBMJOH(SPVQ
    4


    *OHFTUJPO

    ,JOFTJT


    'JSFIPTF
    "CVTJOH


    %FUFDUJPO
    3FBMUJNF"CVTJOH%FUFDUJPO
    3.4ZTUFN
    "UIFOB

    View Slide







  38. য࠭૚ߊࢤ
    ੿೧૓3VMFী੄೧ࢲ੉࢚ਬ੷ܳఐ૑


    ઱۽؀दࠁ٘ীࢲয࠭૚ਸ౵ঈ


    ঌۈ ੗زࠁഐઑ஖ *1ରױ 71/ "4/١



    3VMF
    -
    CBTFE"CVTJOH%FUFDUJPO 3.

    View Slide

  39. ੿೧૓3VMF੉ইפۄؘ੉ఠ ঌҊ્ܻ "*ݽ؛١ਸ੉ਊೠ੉࢚ఐ૑


    ਬ੷೯زӝ߈੉࢚੤ചݽפఠ݂


    ੘স੢ఐ૑ܳਤೠѢېࣗݽפఠ݂


    ೻؊҅੿
    ਕழ҅੿
    ׮੉ইҳݒ҅੿
    ਕழ҅੿
    ਕழ҅੿
    ਕழ҅੿
    ਕழ҅੿
    ਕழ҅੿
    .PEFM
    -
    CBTFE"CVTJOH%FUFDUJPO

    View Slide

  40. %BUB


    "OBMZTJT
    ѱ੐߹ਬ੷੄-57 -JGF5JNF7BMVF
    ৘ஏ


    ೒ۨ੉زೱ߂੉ఎ৘ஏ


    ݃ா౴ബਯ࠙ࢳ


    %BUB"OBMZTJT

    View Slide

  41. *OUSP


    ؘ੉ఠࢤࢿ߂ࣻ૘


    ؘ੉ఠ୊ܻ


    ؘ੉ఠഝਊ


    0VUSP

    View Slide

  42. %BUB%JTDPWFSZ1MBUGPSN ୶ୌदझమ

    View Slide

  43. %BUB%JTDPWFSZ1MBUGPSN ୶ୌदझమ

    View Slide

  44. хࢎ೤פ׮

    View Slide

  45. &0%

    View Slide