Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
NDC 2015 Cookie Run Log System
Search
Jung-gun Lim
May 20, 2015
Programming
44
14k
NDC 2015 Cookie Run Log System
NDC 2015에서 발표한 쿠키런 로그 시스템 슬라이드를 공유합니다.
Jung-gun Lim
May 20, 2015
Tweet
Share
Other Decks in Programming
See All in Programming
React Nativeならぬ"Vue Native"が実現するかも?_新世代マルチプラットフォーム開発フレームワークのLynxとLynxのVue.js対応を追ってみよう_Vue Lynx
yut0naga1_fa
2
1.4k
Developer Joy - The New Paradigm
hollycummins
1
370
CSC509 Lecture 08
javiergs
PRO
0
250
AIと人間の共創開発!OSSで試行錯誤した開発スタイル
mae616
2
810
品質ワークショップをやってみた
nealle
0
640
スキーマ駆動で、Zod OpenAPI Honoによる、API開発するために、Hono Takibiというライブラリを作っている
nakita628
0
320
Vueのバリデーション、結局どれを選べばいい? ― 自作バリデーションの限界と、脱却までの道のり ― / Which Vue Validation Library Should We Really Use? The Limits of Self-Made Validation and How I Finally Moved On
neginasu
2
1.6k
AI駆動で0→1をやって見えた光と伸びしろ
passion0102
1
850
AkarengaLT vol.38
hashimoto_kei
1
120
CSC509 Lecture 06
javiergs
PRO
0
270
Six and a half ridiculous things to do with Quarkus
hollycummins
0
210
チームの境界をブチ抜いていけ
tokai235
0
230
Featured
See All Featured
The Power of CSS Pseudo Elements
geoffreycrofte
80
6k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.5k
Making Projects Easy
brettharned
120
6.4k
Building a Scalable Design System with Sketch
lauravandoore
463
33k
Being A Developer After 40
akosma
91
590k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.7k
Statistics for Hackers
jakevdp
799
220k
BBQ
matthewcrist
89
9.9k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
10
620
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
How To Stay Up To Date on Web Technology
chriscoyier
791
250k
Making the Leap to Tech Lead
cromwellryan
135
9.6k
Transcript
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓदझమ ߄ࢁҊоդೠѐߊܳਤ೧ ؘ࠳दझఠૉ Ӕ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ߊࣗѐ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ѐਃ ѱ۽Ӓदझమਃࢿ ߊࣗѐ ః۠۽Ӓदझమ୍
ః۠۽Ӓदझమҳઑ ഝਊࢎ۹
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Ӕ ߊࣗѐ ߊࣗѐ ؘ࠳दझఠૉ
_അ /$40'5 _ )BEPPQӝ߈..031( ۽Ӓदझమҳ୷߂ ࠙झܻ߁42-௪ܻ ূѐߊ߂ ѱࢲߡ ѐߊ߂
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠ࢲ࠺झѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠ য়ۄੋਸాೠӖ۽ߥࢲ࠺झ %"6ୌ݅ӝ۾ ୌ݅־۽٘ӝ۾ ֙߮ܳܲࣻݽ߄ੌѱ
ః۠ࢲ࠺झѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠ ↟ੜೞחѱ ↟গীೠഛपೠࠁ࢚ ↟ೞҊݺഛೠҊё ↟ਬ৬न܉ҳ୷
ః۠ࢲ࠺झѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓदझమ ↟ః۠҃੬۱ ↟୨5#࢚۽Ӓઁহࠗࠁҙ ↟ೠਬۨӝ۾ਸୡউীৌۈ ↟)BEPPQ۞झఠ۽ࡅܲӏݽ࠙ࢳоמ ↟ୡӝై࠺ਊઁ۽
ਘࣻभ݅ਗਬ࠺ਊ ః۠ࢲ࠺झѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ җѢ۽Ӓܳ৵ೞભ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ߨޖ җѢ۽Ӓܳ৵ೞભ ইమহযޛযղۄ
ഛೠറടಁ݄ࣗܳח
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Ҋёޙ ইమহযযਃ ࠁ࢚೧ࣁਃ ޚبٮبঋҊࠁ࢚೧ݶ Ҋё݃࠺ػ
җѢ۽Ӓܳ৵ೞભ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ా҅߂ %"6 ."6 8"6 16 j
࢜۽ѐߊೠܳҗѢী೧ࢲࡳח ѐߊऔҊࡈۄ җѢ۽Ӓܳ৵ೞભ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓदझమ୍
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠ѱࢲߡ +BWB 4QSJOH.7$ 5PNDBU $PVDICBTF .Z42-
3FEJT ই݃ઓਢࢲ࠺झ ޖѨޖױࢲ࠺झܳೱ ః۠۽Ӓदझమ୍
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ޖѨࢲߡಁ ః۠۽Ӓदझమ୍ ࢲߡ Ѩ
ࢲߡ ࢲߡ ੌ߈ੋ҃ ࢲߡ ః۠҃ ࢲߡ ࢲߡ ࢲߡ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ࢲߡஹನքઑѤ • ݽٚࢲߡ۽ࣁझחਗঋਸٸલח – ࢚ޅೠदীલӝبೞ݅ –
ӟәࠁউಁоӝبೠ • ઁաݫੋఠքझ৬ಁܳоמೞѱ – ৵ջೞݶj ః۠۽Ӓदझమ୍
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ r߄ࢁs • ஶబஎѐߊೞוۄ߄ࢁ • ߊࢤೠޙઁחӒٸӒٸ೧ѾೞҊ •
ӝݫੋఠքझ୭ࣗച • ѐߊஹೊఠীࢲऔѱӔоמೞѱ ః۠۽Ӓदझమ୍
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ rоդೞs • ѱࢲߡ࠺ਊࠁ۽Ӓࢲߡ࠺ਊ࠺ऱݶҌۆ • ޖܻೠୡӝైܳೡࣻহ –
ѱࢿಁחইޖبݽܲ • ࢤпࠁ഻ঁ۴ೠਬ࠺۽ҳ୷ೡࣻ ః۠۽Ӓदझమ୍
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓदझమѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓ • ݒੌ_(#+40/ఫझഋध – ֙୨5#࢚ •
࠺ഋझః݃ • ৵߄ցܻನݘউॳաਃ ª߄ࡅਃ – ۽Ӓܳ୶оೡٸ݃दझమਸসؘೞӝफযਃ ః۠۽Ӓदझమѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ః۠۽Ӓदझమҳࢿ Tomcat Kafka ࣻ ѐߊ ܻ
ࢲ࠺झ ઁ Amazon S3 (Daily Log) Real Time Log Server LevelDB Log Broker CS Server Amazon EMR (Hadoop) CS ѐߊ1$ ః۠۽Ӓदझమѐਃ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓࣻ"QBDIF,BGLB
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓࣻ • "QBDIF,BGLBࢎਊ • ࠙ݫࣁदझమ –
1VCMJTI4VCTDSJCF • ੌ߹ష աൗ Kafka ۽Ӓࣻ"QBDIF,BGLB Tomcat
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ,BGLB • ݫࣁܳࠂઁೞৈ – ֢٘গীъೣ –
ࢲߡসؘ߂࢚ࠗदоמ • ױࣽೠࣗҳઑ Ҋੜউդ • ੍֫ӝॳӝ ۽Ӓࣻ"QBDIF,BGLB
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ -PHTUBTIª,BGLB • ஹನքܳݥ୶ݶয٥оޙઁоੌযդ – ۽ࣁझоલਸٸ۽Ӓਬपߊࢤ •
3FEJTܳղ۷ৢܻݶੌաਃ – 3FEJTীӟәѨझாેѦܻݶਕࢪٸޅ֥ইਃ • -PHTUBTIח1VTIߑध ,BGLBח1VMMߑध – ۽Ӓ੍ܳח۽ࣁझܳղ۷ৢܾࣻ ۽Ӓࣻ"QBDIF,BGLB
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ,BGLBࢎਊ • ݫࣁࠂೖೡࣻহ – 5XP(FOFSBMTs1SPCMFN –
ӝ۾ࢿҕਸޅ߉ਵݶࠂߊࢤ • ࠂઁѢחഛೠా҅ܳਤ೧ࣻ ۽Ӓࣻ"QBDIF,BGLB
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓߔস • ӝపੑߔস • /"4
• .Z42- ੌ߹ߔস • )BEPPQ۞झఠ • )#BTF$BTTBOESB۞झఠ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓߔস • ӝపੑߔস ª߄ࡅਃ • /"4
ªоդ೧ਃ • .Z42- ੌ߹ߔস ª߄ࡅਃ • )BEPPQ۞झఠ ªоդ೧ਃ • )#BTF$BTTBOESB۞झఠ ª߄ࢁҊоդ೧ਃ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Kafka ই݃ઓ4 • ޖೠਊ • Ҋоਊࢿ
• ؘఠܳয֍ਸٸ ਊ࠺ਊহ • ۴ೠਬ࠺ – $16࠺ਊਸղঋ Amazon S3 (Daily Log) ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ই݃ઓ4ഝਊ • ੌदझమۄࠗܰӝূਗغחӝמݻহ – ੌೠߣٜ݅যݶࣻউػ •
ੌ੍ࠗ࠙ӝܳਗೣ – ࠶۟୷߂ੋؙयоמ • ࡅܲઑഥ৬ߓ࠙ࢳоמೠನݘਵ۽ࠁҙ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ੌ߹ߔস • ,BGLB۽ࠗఠযઁ۽Ӓషਸؒ߉ח • যઁ۽Ӓܳ۳ೠ –
ః<"DUJPO .FNCFS*E 5JNFTUBNQ )BTI> – ਗೞחؘఠܳݽৈѱೠ – ࡅܰѱࠂઁѢೡࣻ – ୷ੜػ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ࠶۟୷ • 4OBQQZ'SBNFE'PSNBUࢎਊ • .#ਗࠄ࠶۟ױਤ۽୷ –
୷റ࠶۟ӝ.#݅۽যٚ – ౠੌղۨӝ۾ਸৌۈೞחؘী.#੍݅ਵݶػ • ୷࠶۟ਸযࠢݶೞաѢೠ୷ੌ – 4ীࢲ۽٘߉ইഝਊೞӝಞೞ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ੌ߹ߔসੌҳઑ T[ buy, id:10127, 10:12 am
buy, id:10127, 10:34 am … buy, id:91729, 10:12 am login, id:10013, 01:13 am … … play, id:10013, 10:14 am play, id:10013, 10:23 am … JOEFYKTPO CVZ JE BN CVZ JE BN j QMBZ JE BN ୡ ੌ߹ߔস ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ࢠ݂ • ۽Ӓ۳दী.FNCFSJEܳࣽਵ۽ٍ • ª ª
• rs rsਵ۽աחNFNCFSJE۽Ӓоݽੋ • ࢠ݂ػ۽Ӓܳࡅܰѱ࠙ࢳ – ѐߊ1$ীࢲࡅؘܲఠযܻா࣌ѐߊ߂పझ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Ѿҗ • ౠਬೞܖۨ۽Ӓઑഥୡ – যઁ۽Ӓٚ֙۽Ӓٚэࣘب۽ •
ౠਬੌ֙ۨ۽Ӓઑഥୡ – "NB[PO&$DYMBSHF যӝળ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Ѿҗ • ࢠؘ݂ఠੌੌ – ۽ஸীࢲୡ݅ী࠙ࢳ •
ੌ֙ۨ۽Ӓ – ই݃ઓ&.3۞झఠীࢲ࠙উী࠙ࢳ • ઑഥ ࠙ࢳݽفझாੌইਓоמ – ࢲߡܳטܾࣻ۾ࢿמࡈۄ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ࠺ਊ • ୡӝై࠺ਊহ • ֙ਊ5#ղ৻ •
ਘਬ࠺݅ – ࢎޖपӝࣁࠁऱ • җѢ۽Ӓܳ(MBDJFSীӝחѪਵ۽࠺ਊхоמ ۽Ӓߔস"NB[PO4
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ पदр۽Ӓदझమ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ पदр۽Ӓઑഥ • $4ܳਤ೧ࢲחয়ט۽Ӓبઑഥ೧ঠೠ • যڃदझమਸॶѪੋо
– ୡ੍ӝॳӝے࣌ࣻоѪ – ߧਤ௪ܻܳਗೡѪ • ਤઑѤਸ݅ೞחदझమӒ݆ঋ पदр۽Ӓदझమ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ Kafka -FWFM%# • ,FZ7BMVFझషܻ • ҊॳחҳӖઁಿ
– ҳӖ܁ীࢎਊ • ߧਤ௪ܻਗ • ୡ݅Ѥॳӝоמ Real Time Log Server LevelDB पदр۽Ӓदझమ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ पदр۽Ӓࢲߡ • -FWFM%#ূ֎ਕ"1*оহ • 3&45"1*ࢲߡܳ(P۽ࢿ –
֫زदࢿ – ݣ౭যܳഝਊೠಞೠझாੌ • ੌ߹-FWFM%# – ੌ߹షী – ࠂઁѢ पदр۽Ӓदझమ Real Time Log Server Real Time Log Server LevelDB
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓ࠳۽ழ • (P۽ࢿ • җѢ۽Ӓח4 •
୭Ӕ۽Ӓחपदр۽Ӓࢲߡ • زੌೠੋఠಕझ۽֙۽Ӓ৬ ୡ۽ӒܳݽفӔоמ Real Time Log Server LevelDB पदр۽Ӓदझమ Log Broker Amazon S3 (Daily Log)
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ࢎ۹ۨ܀ߔ • ౠఃীؘఠয়ܨߊѼ – ࠺࢚ਵ۽֫ࣻߊࢤ –
࠺࢚ۨ۽ੋधغযೞחਬߊࢤ • ః۠ӝ۾҃੬ѱ ҕೣਃೞ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ۨ܀ߔ • য়טۨѨ࢝ਵ۽ೱ߉ਬഛੋ • ࣻߔݺೱ߉ਬٜীೞৈ –
ݽٚۨӝ۾ਸоઉ৬ࢲ – ޙઁחۨܳఠ݂ೞҊ – ޙઁহחۨ୭Ҋࣻ۽јन • ਬୡݶоמªࣁदр݅ী೧Ѿ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ $4݃܄ਢࢲ࠺झ • ೠੋఠಕझ۽۽Ӓܳࠁৈળ – ܻझఎੑ –
ইమੑ – ۨӝ۾ • ഛೠറ࢚ടਸ౸߹ೞҊ$4оמ – ೲਤ$4ߑ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ؘੌܻۨನಃா • 4DBMEJOHӝ߈۽Ӓ࠙ࢳۨਕ – ೣࣻഋযޙߨਸࢎਊೠрѾೠ٘ –
ࢠ݂ਸഝਊೠࡅܲ۽ஸపझ • ݒੌইஜ݃যઁܳݫੌ۽࣠ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ 4DBMEJOHਸഝਊೠ࠙ࢳ class ActiveUserCount(args : Args) extends
Job(args) { CookierunLog("member/connect", args("sampling"), args("timeStart"), args("timeEnd"), Tuple1("memberSeq")) .groupBy(‘memberSeq){ _.take(1) } .groupAll{ _.size } .write(Csv(args("output"))) } ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ؘੌܻۨನಃா • "NB[PO&.3ਤীࢲоز – ই݃ઓਃীٮۄزਵ۽ڸਕח)BEPPQ۞झఠ •
4QPU*OTUBODFࢎਊ – যࢲߡ DYMBSHF ೠदр࠽ܻח࠺ਊ݅ – ӔҮా࠺ࣻળ ۽Ӓഝਊ
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ ଵҊܐ • ഘࢿ ః۠֙ࢲߡѐߊ࠙ైӝ /%$ •
࢚Ҍ 1ZUIPOਵ۽ః۠ೞӝ 1ZDPO,PSFB • ӣ оߺҊਬোೞѱؘఠ࠙ࢳೞӝ ,($ • ହઁ "84ഝਊೞৈ%BJMZ3FQPSUٜ݅ӝ "84(MPCBM 4VNNJU4FPVM
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ 8FBSFIJSJOH ೣԋݧदझమਸٜ݅যࠊਃ DBSFFS!EFWTJTUFSTDPN
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ 2"
ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧ хࢎפ ః۠ ۽Ӓ दझమ ߄ࢁҊоդೠѐߊܳਤ೧