Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
NDC 2015 Cookie Run Log System
Search
Jung-gun Lim
May 20, 2015
Programming
44
14k
NDC 2015 Cookie Run Log System
NDC 2015에서 발표한 쿠키런 로그 시스템 슬라이드를 공유합니다.
Jung-gun Lim
May 20, 2015
Tweet
Share
Other Decks in Programming
See All in Programming
フロントエンド開発のためのブラウザ組み込みAI入門
masashi
7
3.6k
バッチ処理を「状態の記録」から「事実の記録」へ
panda728
PRO
0
210
Google Opalで使える37のライブラリ
mickey_kubo
3
170
Vueのバリデーション、結局どれを選べばいい? ― 自作バリデーションの限界と、脱却までの道のり ― / Which Vue Validation Library Should We Really Use? The Limits of Self-Made Validation and How I Finally Moved On
neginasu
3
1.7k
他言語経験者が Golangci-lint を最初のコーディングメンターにした話 / How Golangci-lint Became My First Coding Mentor: A Story from a Polyglot Programmer
uma31
0
480
Using Types to Save Your Code's Future
rollbear
0
100
kiroとCodexで最高のSpec駆動開発を!!数時間で web3ネイティブなミニゲームを作ってみたよ!
mashharuki
0
990
Claude Agent SDK を使ってみよう
hyshu
0
1.4k
CSC305 Lecture 10
javiergs
PRO
0
320
Register is more than clipboard
satorunooshie
1
140
ALL CODE BASE ARE BELONG TO STUDY
uzulla
28
6.8k
Blazing Fast UI Development with Compose Hot Reload (droidcon London 2025)
zsmb
0
420
Featured
See All Featured
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
34
2.5k
GraphQLとの向き合い方2022年版
quramy
49
14k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
31
2.9k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
31
2.7k
Mobile First: as difficult as doing things right
swwweet
225
10k
Imperfection Machines: The Place of Print at Facebook
scottboms
269
13k
Building Adaptive Systems
keathley
44
2.8k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
1k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.7k
Testing 201, or: Great Expectations
jmmastey
45
7.7k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
253
22k
Documentation Writing (for coders)
carmenintech
75
5.1k
Transcript
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓदझమ ߄ࢁҊоդೠѐߊܳਤ೧ ؘ࠳दझఠૉ Ӕ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ߊࣗѐ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ѐਃ ѱ۽Ӓदझమਃࢿ ߊࣗѐ ః۠۽Ӓदझమ୍
ః۠۽Ӓदझమҳઑ ഝਊࢎ۹
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Ӕ ߊࣗѐ ߊࣗѐ ؘ࠳दझఠૉ
_അ /$40'5 _ )BEPPQӝ߈..031( ۽Ӓदझమҳ୷߂ ࠙झܻ߁42-௪ܻ ূѐߊ߂ ѱࢲߡ ѐߊ߂
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠ࢲ࠺झѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠ য়ۄੋਸాೠӖ۽ߥࢲ࠺झ %"6ୌ݅ӝ۾ ୌ݅־۽٘ӝ۾ ֙߮ܳܲࣻݽ߄ੌѱ
ః۠ࢲ࠺झѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠ ↟ੜೞחѱ ↟গীೠഛपೠࠁ࢚ ↟ೞҊݺഛೠҊё ↟ਬ৬न܉ҳ୷
ః۠ࢲ࠺झѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓदझమ ↟ః۠҃੬۱ ↟୨5#࢚۽Ӓઁহࠗࠁҙ ↟ೠਬۨӝ۾ਸୡউীৌۈ ↟)BEPPQ۞झఠ۽ࡅܲӏݽ࠙ࢳоמ ↟ୡӝై࠺ਊઁ۽
ਘࣻभ݅ਗਬ࠺ਊ ః۠ࢲ࠺झѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ җѢ۽Ӓܳ৵ೞભ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ߨޖ җѢ۽Ӓܳ৵ೞભ ইమহযޛযղۄ
ഛೠറടಁ݄ࣗܳח
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Ҋёޙ ইమহযযਃ ࠁ࢚೧ࣁਃ ޚبٮبঋҊࠁ࢚೧ݶ Ҋё݃࠺ػ
җѢ۽Ӓܳ৵ೞભ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ా҅߂ %"6 ."6 8"6 16 j
࢜۽ѐߊೠܳҗѢী೧ࢲࡳח ѐߊऔҊࡈۄ җѢ۽Ӓܳ৵ೞભ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓदझమ୍
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠ѱࢲߡ +BWB 4QSJOH.7$ 5PNDBU $PVDICBTF .Z42-
3FEJT ই݃ઓਢࢲ࠺झ ޖѨޖױࢲ࠺झܳೱ ః۠۽Ӓदझమ୍
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ޖѨࢲߡಁ ః۠۽Ӓदझమ୍ ࢲߡ Ѩ
ࢲߡ ࢲߡ ੌ߈ੋ҃ ࢲߡ ః۠҃ ࢲߡ ࢲߡ ࢲߡ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ࢲߡஹನքઑѤ • ݽٚࢲߡ۽ࣁझחਗঋਸٸલח – ࢚ޅೠदীલӝبೞ݅ –
ӟәࠁউಁоӝبೠ • ઁաݫੋఠքझ৬ಁܳоמೞѱ – ৵ջೞݶj ః۠۽Ӓदझమ୍
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ r߄ࢁs • ஶబஎѐߊೞוۄ߄ࢁ • ߊࢤೠޙઁחӒٸӒٸ೧ѾೞҊ •
ӝݫੋఠքझ୭ࣗച • ѐߊஹೊఠীࢲऔѱӔоמೞѱ ః۠۽Ӓदझమ୍
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ rоդೞs • ѱࢲߡ࠺ਊࠁ۽Ӓࢲߡ࠺ਊ࠺ऱݶҌۆ • ޖܻೠୡӝైܳೡࣻহ –
ѱࢿಁחইޖبݽܲ • ࢤпࠁ഻ঁ۴ೠਬ࠺۽ҳ୷ೡࣻ ః۠۽Ӓदझమ୍
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓदझమѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓ • ݒੌ_(#+40/ఫझഋध – ֙୨5#࢚ •
࠺ഋझః݃ • ৵߄ցܻನݘউॳաਃ ª߄ࡅਃ – ۽Ӓܳ୶оೡٸ݃दझమਸসؘೞӝफযਃ ః۠۽Ӓदझమѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓदझమҳࢿ Tomcat Kafka ࣻ ѐߊ ܻ
ࢲ࠺झ ઁ Amazon S3 (Daily Log) Real Time Log Server LevelDB Log Broker CS Server Amazon EMR (Hadoop) CS ѐߊ1$ ః۠۽Ӓदझమѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓࣻ"QBDIF,BGLB
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓࣻ • "QBDIF,BGLBࢎਊ • ࠙ݫࣁदझమ –
1VCMJTI4VCTDSJCF • ੌ߹ష աൗ Kafka ۽Ӓࣻ"QBDIF,BGLB Tomcat
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ,BGLB • ݫࣁܳࠂઁೞৈ – ֢٘গীъೣ –
ࢲߡসؘ߂࢚ࠗदоמ • ױࣽೠࣗҳઑ Ҋੜউդ • ੍֫ӝॳӝ ۽Ӓࣻ"QBDIF,BGLB
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ -PHTUBTIª,BGLB • ஹನքܳݥ୶ݶয٥оޙઁоੌযդ – ۽ࣁझоલਸٸ۽Ӓਬपߊࢤ •
3FEJTܳղ۷ৢܻݶੌաਃ – 3FEJTীӟәѨझாેѦܻݶਕࢪٸޅ֥ইਃ • -PHTUBTIח1VTIߑध ,BGLBח1VMMߑध – ۽Ӓ੍ܳח۽ࣁझܳղ۷ৢܾࣻ ۽Ӓࣻ"QBDIF,BGLB
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ,BGLBࢎਊ • ݫࣁࠂೖೡࣻহ – 5XP(FOFSBMTs1SPCMFN –
ӝ۾ࢿҕਸޅ߉ਵݶࠂߊࢤ • ࠂઁѢחഛೠా҅ܳਤ೧ࣻ ۽Ӓࣻ"QBDIF,BGLB
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓߔস • ӝపੑߔস • /"4
• .Z42- ੌ߹ߔস • )BEPPQ۞झఠ • )#BTF$BTTBOESB۞झఠ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓߔস • ӝపੑߔস ª߄ࡅਃ • /"4
ªоդ೧ਃ • .Z42- ੌ߹ߔস ª߄ࡅਃ • )BEPPQ۞झఠ ªоդ೧ਃ • )#BTF$BTTBOESB۞झఠ ª߄ࢁҊоդ೧ਃ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Kafka ই݃ઓ4 • ޖೠਊ • Ҋоਊࢿ
• ؘఠܳয֍ਸٸ ਊ࠺ਊহ • ۴ೠਬ࠺ – $16࠺ਊਸղঋ Amazon S3 (Daily Log) ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ই݃ઓ4ഝਊ • ੌदझమۄࠗܰӝূਗغחӝמݻহ – ੌೠߣٜ݅যݶࣻউػ •
ੌ੍ࠗ࠙ӝܳਗೣ – ࠶۟୷߂ੋؙयоמ • ࡅܲઑഥ৬ߓ࠙ࢳоמೠನݘਵ۽ࠁҙ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ੌ߹ߔস • ,BGLB۽ࠗఠযઁ۽Ӓషਸؒ߉ח • যઁ۽Ӓܳ۳ೠ –
ః<"DUJPO .FNCFS*E 5JNFTUBNQ )BTI> – ਗೞחؘఠܳݽৈѱೠ – ࡅܰѱࠂઁѢೡࣻ – ୷ੜػ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ࠶۟୷ • 4OBQQZ'SBNFE'PSNBUࢎਊ • .#ਗࠄ࠶۟ױਤ۽୷ –
୷റ࠶۟ӝ.#݅۽যٚ – ౠੌղۨӝ۾ਸৌۈೞחؘী.#੍݅ਵݶػ • ୷࠶۟ਸযࠢݶೞաѢೠ୷ੌ – 4ীࢲ۽٘߉ইഝਊೞӝಞೞ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ੌ߹ߔসੌҳઑ T[ buy, id:10127, 10:12 am
buy, id:10127, 10:34 am … buy, id:91729, 10:12 am login, id:10013, 01:13 am … … play, id:10013, 10:14 am play, id:10013, 10:23 am … JOEFYKTPO CVZ JE BN CVZ JE BN j QMBZ JE BN ୡ ੌ߹ߔস ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ࢠ݂ • ۽Ӓ۳दী.FNCFSJEܳࣽਵ۽ٍ • ª ª
• rs rsਵ۽աחNFNCFSJE۽Ӓоݽੋ • ࢠ݂ػ۽Ӓܳࡅܰѱ࠙ࢳ – ѐߊ1$ীࢲࡅؘܲఠযܻா࣌ѐߊ߂పझ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Ѿҗ • ౠਬೞܖۨ۽Ӓઑഥୡ – যઁ۽Ӓٚ֙۽Ӓٚэࣘب۽ •
ౠਬੌ֙ۨ۽Ӓઑഥୡ – "NB[PO&$DYMBSHF যӝળ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Ѿҗ • ࢠؘ݂ఠੌੌ – ۽ஸীࢲୡ݅ী࠙ࢳ •
ੌ֙ۨ۽Ӓ – ই݃ઓ&.3۞झఠীࢲ࠙উী࠙ࢳ • ઑഥ ࠙ࢳݽفझாੌইਓоמ – ࢲߡܳטܾࣻ۾ࢿמࡈۄ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ࠺ਊ • ୡӝై࠺ਊহ • ֙ਊ5#ղ৻ •
ਘਬ࠺݅ – ࢎޖपӝࣁࠁऱ • җѢ۽Ӓܳ(MBDJFSীӝחѪਵ۽࠺ਊхоמ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ पदр۽Ӓदझమ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ पदр۽Ӓઑഥ • $4ܳਤ೧ࢲחয়ט۽Ӓبઑഥ೧ঠೠ • যڃदझమਸॶѪੋо
– ୡ੍ӝॳӝے࣌ࣻоѪ – ߧਤ௪ܻܳਗೡѪ • ਤઑѤਸ݅ೞחदझమӒ݆ঋ पदр۽Ӓदझమ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Kafka -FWFM%# • ,FZ7BMVFझషܻ • ҊॳחҳӖઁಿ
– ҳӖ܁ীࢎਊ • ߧਤ௪ܻਗ • ୡ݅Ѥॳӝоמ Real Time Log Server LevelDB पदр۽Ӓदझమ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ पदр۽Ӓࢲߡ • -FWFM%#ূ֎ਕ"1*оহ • 3&45"1*ࢲߡܳ(P۽ࢿ –
֫زदࢿ – ݣ౭যܳഝਊೠಞೠझாੌ • ੌ߹-FWFM%# – ੌ߹షী – ࠂઁѢ पदр۽Ӓदझమ Real Time Log Server Real Time Log Server LevelDB
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓ࠳۽ழ • (P۽ࢿ • җѢ۽Ӓח4 •
୭Ӕ۽Ӓחपदр۽Ӓࢲߡ • زੌೠੋఠಕझ۽֙۽Ӓ৬ ୡ۽ӒܳݽفӔоמ Real Time Log Server LevelDB पदр۽Ӓदझమ Log Broker Amazon S3 (Daily Log)
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ࢎ۹ۨ܀ߔ • ౠఃীؘఠয়ܨߊѼ – ࠺࢚ਵ۽֫ࣻߊࢤ –
࠺࢚ۨ۽ੋधغযೞחਬߊࢤ • ః۠ӝ۾҃੬ѱ ҕೣਃೞ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۨ܀ߔ • য়טۨѨ࢝ਵ۽ೱ߉ਬഛੋ • ࣻߔݺೱ߉ਬٜীೞৈ –
ݽٚۨӝ۾ਸоઉ৬ࢲ – ޙઁחۨܳఠ݂ೞҊ – ޙઁহחۨ୭Ҋࣻ۽јन • ਬୡݶоמªࣁदр݅ী೧Ѿ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ $4݃܄ਢࢲ࠺झ • ೠੋఠಕझ۽۽Ӓܳࠁৈળ – ܻझఎੑ –
ইమੑ – ۨӝ۾ • ഛೠറ࢚ടਸ౸߹ೞҊ$4оמ – ೲਤ$4ߑ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ؘੌܻۨನಃா • 4DBMEJOHӝ߈۽Ӓ࠙ࢳۨਕ – ೣࣻഋযޙߨਸࢎਊೠрѾೠ٘ –
ࢠ݂ਸഝਊೠࡅܲ۽ஸపझ • ݒੌইஜ݃যઁܳݫੌ۽࣠ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ 4DBMEJOHਸഝਊೠ࠙ࢳ class ActiveUserCount(args : Args) extends
Job(args) { CookierunLog("member/connect", args("sampling"), args("timeStart"), args("timeEnd"), Tuple1("memberSeq")) .groupBy(‘memberSeq){ _.take(1) } .groupAll{ _.size } .write(Csv(args("output"))) } ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ؘੌܻۨನಃா • "NB[PO&.3ਤীࢲоز – ই݃ઓਃীٮۄزਵ۽ڸਕח)BEPPQ۞झఠ •
4QPU*OTUBODFࢎਊ – যࢲߡ DYMBSHF ೠदр࠽ܻח࠺ਊ݅ – ӔҮా࠺ࣻળ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ଵҊܐ • ഘࢿ ః۠֙ࢲߡѐߊ࠙ైӝ /%$ •
࢚Ҍ 1ZUIPOਵ۽ః۠ೞӝ 1ZDPO,PSFB • ӣ оߺҊਬোೞѱؘఠ࠙ࢳೞӝ ,($ • ହઁ "84ഝਊೞৈ%BJMZ3FQPSUٜ݅ӝ "84(MPCBM 4VNNJU4FPVM
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ 8FBSFIJSJOH ೣԋݧदझమਸٜ݅যࠊਃ DBSFFS!EFWTJTUFSTDPN
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ 2"
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ хࢎפ ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧