#DataLake
카카오게임즈의 모바일 게임 로그 데이터가 어떻게 생성되고 수집되는지, 이 수집된 데이터를 어떻게 처리하고 활용하는지에 대한 전체적인 과정을 소개합니다.
발표자 : kyle.l 카카오게임즈 BI팀 / DataLake TFT에서 ETL 및 데이터 엔지니어링 업무를 맡고 있는 kyle입니다
$PQZSJHIU,BLBP$PSQ"MMSJHIUTSFTFSWFE3FEJTUSJCVUJPOPSQVCMJDEJTQMBZJTOPUQFSNJUUFEXJUIPVUXSJUUFOQFSNJTTJPOGSPN,BLBPࢤࢿࠗఠഝਊө ѱؘఠ'MPXٮۄоӝJG LBLBPয়ѱૉ൞LZMFM
View Slide
*OUSPؘఠࢤࢿ߂ؘࣻఠؘܻఠഝਊ0VUSP
ѱ۽Ӓؘఠഝਊ о
$MPVE%BUBMBLF#BTFEPO"840O-QSFNJTF%8'MFYJCJMJUZ"OBMZUJDT
۽Ӓؘఠࢤࢿ߂ࣻ
;JOOZ1MBUGPSN
4%,;JOOZ1MBUGPSN
;JOOZ1MBUGPSN0QFO"1*4FTTJPO-PHCVDLFU
(BNF-PH4FSWJDF-PH1MBUGPSN -PHBDUJPO JUFN QVSDIBTFj BVUI MPHJO IFBSUCFBUj ࢎড ࢸޙઑࢎj۽Ӓؘఠઙܨ
(BNF-PH4FSWJDF-PH1MBUGPSN-PHড5# ೞܖড(#ঀૐоBDUJPO JUFN QVSDIBTFj BVUI MPHJO IFBSUCFBUj ࢎড ࢸޙઑࢎj۽Ӓؘఠઙܨ
-PHCVDLFU ,JOFTJT%BUB4USFBN
-BNCEB,JOFTJT%BUB4USFBN,JOFTJT'JSFIPTF4*OHFTUJPO
-BNCEB,JOFTJT%BUB4USFBN,JOFTJT'JSFIPTF4*OHFTUJPOjJGDPEFBDUJPOPVUQVU"DUJPO-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZGBJMFE@SFDPSETDIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU"DUJPO-JTU EFMJWFSZ@TUSFBN@OBNFEML-SFBM-BDUJPOFMJGDPEFJUFNPVUQVU*UFN-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZGBJMFE@SFDPSETDIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU*UFN-JTU EFMJWFSZ@TUSFBN@OBNFEML-SFBM-JUFNFMJGDPEFrSFTPVSDFPVUQVU*UFN-JTUBQQFOE HFU@T@EFGBVMU MPH@CPEZMPH#PEZGBJMFE@SFDPSETDIFDL@BOE@GMVTI UBSHFU@MJTUPVUQVU*UFN-JTU EFMJWFSZ@TUSFBN@OBNFEML-SFBM-SFTPVSDFj
पदрਵ۽ࣻغחؘఠܳ߄۽ࠅࣻਸө
4USFBNJOH%BUB*OHFTUJPO$POOFDUPSPO&$,BGLB &MBTUJD4FBSDI ,JCBOB$POOFDUPSPO&$"VUP4DBMJOH(SPVQ 0QFO4FBSDI-BNCEB,JOFTJT%BUB4USFBN,JOFTJT'JSFIPTF%BTI#PBSE
4USFBNJOH%BUB*OHFTUJPO$POOFDUPSPO&$,BGLB &MBTUJD4FBSDI ,JCBOB$POOFDUPSPO&$"VUP4DBMJOH(SPVQ 0QFO4FBSDI%BTI#PBSE-BNCEB,JOFTJT%BUB4USFBN,JOFTJT'JSFIPTF
৻ؘࠗఠࢤࢿ߂ࣻ
&YUFSOBM%BUB*OHFTUJPO*OHFTUJPO4FSWFS1FOUBIP)BEPPQ "1*,BGLB,BLBP;POF"1*ജਯ ҟҊؘఠj1VCMJD;POF,BLBPHBNFT;POF.8""5SJHHFS4*OHFTUJPO.BOBHFE8PSLGMPXGPS"QBDIF"JSGMPX
Ӓۢࣻػؘఠחযڌѱೞաਃ
'BDUPSZ%#"1*-PH4/4'JMFj*OTJHIU
1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPSj4"OBMZTJT4*OHFTUJPO3FE4IJGU.BSU0QFSBUPS4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF.8""4"SUJGBDUT%BTI#PBSE#*4'BDUPSZ8PSLGMPX&.3$MVTUFSj(JU)VC&OUFSQSJTF+FOLJOT
.8""4"OBMZTJT4"SUJGBDUT&.3$MVTUFS(JU)VC&OUFSQSJTF4*OHFTUJPO3FE4IJGU.BSU+FOLJOT%BTI#PBSE#*4'BDUPSZ8PSLGMPX1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPSj0QFSBUPS4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCFj
4*OHFTUJPO.8""4"SUJGBDUT(JU)VC&OUFSQSJTF+FOLJOT4"OBMZTJT&.3$MVTUFS3FE4IJGU.BSU%BTI#PBSE#*41ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPSj0QFSBUPS4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF'BDUPSZ8PSLGMPXj
द DPEFrSFTPVSDFs\BQQ*Et QMBZFS*Eĉ BNPVOU DIBSBDUFS*Eĉ DIBSBDUFS-W DIBSBDUFS/Nĉ DPVOUSZ58 EFMUB HBNF8PSMEĉXPSME NPE5JNF NPE5ZQFBEE PT8JOEPXT QBJEV QMBZFS-W S$VSSFODZ(PME SFBTPO"@26&45 SFTPVSDF"UUS TVC3FBTPO&MJUF%VOHFPO MBOH[I-IBOU TFTTJPO*E DMJFOU*Qu NBSLFUVOLOPXO DPEFSFTPVSDF QVCMJTI5JNF MPH*Eĉ^ \BQQ*Et QMBZFS*E BNPVOU DIBSBDUFS*E DIBSBDUFS-W DIBSBDUFS/N DPVOUSZ,3 EFMUB HBNF8PSMEĉXPSME NPE5JNF NPE5ZQFBEE PT8JOEPXT QBJEV QMBZFS-W S$VSSFODZ(PME SFBTPO"@1-": SFTPVSDF"UUS TVC3FBTPO'JFME MBOHLP TFTTJPO*E DMJFOU*Q NBSLFUVOLOPXO DPEFSFTPVSDF QVCMJTI5JNF MPH*Eĉ^
.8""4"SUJGBDUT(JU)VC&OUFSQSJTF+FOLJOT4"OBMZTJT&.3$MVTUFS3FE4IJGU.BSU%BTI#PBSE#*44*OHFTUJPO1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPSj0QFSBUPS4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCF'BDUPSZ8PSLGMPXj
QMBZFSJE HET@DE STO@DE BNU TFSWFS HPME "@1-": XPSME HPME "@26&45 XPSME HPME "@1-": XPSME HPME "@26&45 XPSME HPME "@1-": XPSME HPME "@1-": XPSME HPME "@26&45 XPSME HPME "@26&45 XPSME HPME "@26&45 XPSME HPME "@1-": XPSME%BUB8BSFIPVTFHET@DE STO@DE TFSWFS DOU TVNHPME "@1-": XPSME HPME "@1-": XPSME HPME "@1-": XPSME HPME "@26&45 XPSME HPME "@26&45 XPSME 4VNNBSZ%BUB
3FE4IJGU.BSU%BTI#PBSE#*4.8""+FOLJOT4"SUJGBDUT(JU)VC&OUFSQSJTF4"OBMZTJT&.3$MVTUFS4*OHFTUJPO'BDUPSZ8PSLGMPX1ZUIPO0QFSBUPS 44)0QFSBUPS #BTI0QFSBUPSj0QFSBUPS4UFQ 4BWFSFQSPDFTTEBUFUJNF $SFBUFUBTLHSPVQ 3FGSFTIDVCFj
3FE4IJGU.BSUHET@DE STO@DE TFSWFS DOU TVNHPME "@1-": XPSME HPME "@1-": XPSME HPME "@1-": XPSME HPME "@26&45 XPSME HPME "@26&45 XPSME 3FTPVSDF 4VNNBSZ%BUB*UFN"DUJPO3PVOE1VSDIBTF*EQ-PHJOjj4"OBMZTJT.8""%BTI#PBSE#*44QFDUSVN
4"OBMZTJT3FE4IJGU.BSU4QFDUSVN"UIFOB%BTI#PBSE"E-IPD2VFSZ#VTJOFTT*OUFMMJHFODF3FQPSU
#VTJOFTT*OUFMMJHFODF3FQPSU"E-IPD2VFSZ4"OBMZTJT3FE4IJGU.BSU4QFDUSVN%BTI#PBSE"UIFOB
$POOFDUPSPO&$,BGLB $POOFDUPSPO&$"VUP4DBMJOH(SPVQ4*OHFTUJPO,JOFTJT'JSFIPTF"CVTJOH%FUFDUJPO3FBMUJNF"CVTJOH%FUFDUJPO3.4ZTUFN"UIFOB
য࠭ߊࢤ ೧3VMFী೧ࢲ࢚ਬܳఐ ۽दࠁ٘ীࢲয࠭ਸঈ ঌۈ زࠁഐઑ *1ରױ 71/ "4/١3VMF-CBTFE"CVTJOH%FUFDUJPO 3.
೧3VMFইפۄؘఠ ঌҊ્ܻ "*ݽ؛١ਸਊೠ࢚ఐ ਬ೯زӝ߈࢚ചݽפఠ݂ সఐܳਤೠѢېࣗݽפఠ݂؊҅ਕழ҅ইҳݒ҅ਕழ҅ਕழ҅ਕழ҅ਕழ҅ਕழ҅.PEFM-CBTFE"CVTJOH%FUFDUJPO
%BUB"OBMZTJT ѱ߹ਬ-57 -JGF5JNF7BMVFஏ ۨزೱ߂ఎஏ ݃ாബਯ࠙ࢳ%BUB"OBMZTJT
%BUB%JTDPWFSZ1MBUGPSN ୶ୌदझమ
хࢎפ
&0%