리디북스 비콘 로그 수집 개선

Transcript

1. ܻ٣࠘झ ࠺௑ ۽Ӓ ࣻ૘ ѐࢶ ؘ੉ఠ౱ ݃ࢿള
2. Table of Contents • ؘ੉ఠ౱੄ ۽Ӓ ࣻ૘ Pipeline ߂ ࣻ૘ೞח ۽Ӓ੄ ઙܨ • Beacon Log ࣻ૘ അട ߂ ޙઁ • Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda • AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ • അ੤੄ ؘ੉ఠ ౵੉೐ۄੋ ҳࢿ
4. ؘ੉ఠ౱ীࢲ ࣻ૘ೞח ۽Ӓ੄ ઙܨ • ࢲ੼ ਢࢲߡ Access Logs => Beacon Log • ࠭য (ਢ࠭য ನೣ) Viewer Logs • Ѩ࢝ ۽Ӓ • ӝఋ ࢲߡ Access Log, Django Log, etc...
5. Beacon Log ࣻ૘ അട ߂ ޙઁ • ࢎ੹੸ ੄޷ • ੌ੿ೠ ૑੼ীࢲ ౠ੿ೠ ߑೱ੄ ੹౵ܳ ߊࢎೞৈ
 
 ೦ҕӝ, ࢶ߅ ١ী Ӓ ૑੼ী ؀ೠ ਤ஖ա ߑೱਸ ঌܻח ੢஖ • प୓(beacon.gif) : 1 * 1 px ੄ gif ੉޷૑
6. Beacon Log ࣻ૘ അട ߂ ޙઁ • ೨ब IDEA • ౠ੿ ࢲߡী 1px૞ܻ gif(beacon.gif) ౵ੌਸ publicೞѱ ࢲ࠺झ • ۽Ӧ ೙ਃद ؘ੉ఠܳ gif ౵ੌ ઱ࣗ ٍী request parameter۽ ઁҕ • ೧׼ ਃ୒਷ ਢࢲߡী Access Log۽ ӝ۾ؽ • Access Log ઺ beacon.gif ܳ GET ೠ ਃ୒݅ ୶۰ղ ౵ۄ޷ఠ ౵य
7. Beacon Log ࣻ૘ അട ߂ ޙઁ
8. Beacon Log ࣻ૘ അട ߂ ޙઁ
9. Beacon Log ࣻ૘ അട ߂ ޙઁ <a class="thumbnail_btn trackable" href="/v2/Detail?id=593000658" data-track- params="{'section':'bestseller','idx':0,'u_id':'masunghoon','type':'impression','tags': {'name':'न҃ Սӝ੄ ӝࣿ','brand':'ѓܻৡ ୹౸', 'genre':'general', 'genre_kor':'ੌ߈', 'category':'ࢿҕ/࢕੄੗ࣁ', 'price':'9000', 'quantity':1, 'rep_id':'', 'rep_title':'', 'is_checkpoint':false, 'is_last_item':false}, 'obj_id':'606001670'}" data-track- type="beacon,ga">
10. Beacon Log ࣻ૘ അട ߂ ޙઁ Request URL:https://ridibooks.com/images/log/beacon.gif? section=Pc.home.bestseller.general&obj_id=606001670&idx=0&even t=display&u_id=masunghoon&ruid=5b8dc371-c07d-487c-b554- a96676b44bb6&device=Pc
11. Beacon Log ࣻ૘ അട ߂ ޙઁ • Access Log • 1ੌ: ড 40,000,000Ѥ (ࢲ੼ ਢࢲߡ 1؀ ӝળ) • 1ѐਘ: > 1,000,000,000Ѥ (10র Ѥ ੉࢚) • Beacon Log • 1ੌ: ড 13,000,000Ѥ (ড 1/3) • 1ѐਘ: > 400,000,000Ѥ (4র Ѥ ੿ب)
12. Beacon Log ࣻ૘ അട ߂ ޙઁ
13. Beacon Log ࣻ૘ അട ߂ ޙઁ • ୶ۿ • ݫੋ ಕ੉૑ী ૘઺ ػ Beacon objects • 1PV ׼, ݆਷ ҃਋ ࣻभഥ੄ beacon tracking ਃ୒ • पઁ۽ • ಣࣗ 30%࠺઺੄ Beacon logо 50% ୡҗ (ࣽࣻೠ Access Log ৉੹) • ژೠ ਃ୒ ੺؀۝੉ ಩ૐ
14. Beacon Log ࣻ૘ അട ߂ ޙઁ • ୶ۿ • ݫੋ ಕ੉૑ী ૘઺ ػ Beacon objects • 1PV ׼, ݆਷ ҃਋ ࣻभߣ੄ beacon tracking ਃ୒ • पઁ۽ • ಣࣗ 30%࠺઺੉؍ Beacon logо 40% ੉࢚ ੼ਬ • ژೠ ਃ୒ ੺؀۝੉ ಩ૐ
15. Beacon Log ࣻ૘ ѐࢶ ߑೱ Pure Access Log
 +
 Beacon Log
16. Beacon Log ࣻ૘ ѐࢶ ߑೱ Pure Access Log Beacon Log
17. Beacon Log ࣻ૘ ѐࢶ ߑೱ
18. Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda • ҳ࢚ೞח दझమ੉ Lamdbaэ਷ ࢲ࠺झ৬ Ҿ೤੉ જ׮Ҋ ౸ױ • HTTP request ܳ Ӓ؀۽ KAFKA۽ షझ • json ١ਵ۽ ৘ࢁѱ ౵यೞৈ షझೞݶ ؊ જ਺ ! • ਃ୒੉ ੸਷ दр؀ী ஹೊ౴ ੗ਗਸ խ࠺ೡ ੌ হ਺
19. Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda Amazon API Gateway KAFKA AWS Lambda
20. Beacon Log അട • Access Log • 1ੌ: ড 40,000,000Ѥ (ࢲ੼ ਢࢲߡ 1؀ ӝળ) • 1ѐਘ: > 1,000,000,000Ѥ (10র Ѥ ੉࢚) • Beacon Log • 1ੌ: ড 13,000,000Ѥ (ড 1/3) • 1ѐਘ: > 400,000,000Ѥ (4র Ѥ ੿ب)
21. Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda • AWS Lambda: 1,000,000 ഥ ׼ $0.2 • 4রѤ: ؀ۚ $80 • API Gateway: ???
22. Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda • 1,000,000 : $3.50 = 400,000,000 : ? • 1 : $3.50 = 400 : x • x = $3.5 * 400 = $1,400 • 1,500,000 ਗ / ਘ x 400,000,000 Ѥ
23. Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda
24. AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ AWS Lambda JavaScript SDK KAFKA
25. AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ AWS Lambda JavaScript SDK
26. AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ AWS Lambda Amazon
 S3 KAFKA
27. AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ beacon.gif beacon-log bucket Lambda Function HTTP GET trigger S3 Access Log
28. AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ • 10,000 : $0.0035 = 400,000,000 : ? • 1 : $0.0035 = 40,000 : x • x = $0.0035 * 40,000 = $140 • 150,000 ਗ / ਘ
29. AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ • ױ੼ • S3੄ Access-logо ߓ஖ࢿਵ۽ ࣻ૘ - पदрࢿ ੌࠗ ࢚प • ੢੼ • $ • S3੄ Access-logо ߓ஖ࢿਵ۽ ࣻ૘ - Lambda ഐ୹ഥࣻ хࣗ AWS Lambda Amazon
 S3 KAFKA
30. AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ • ૑դ 11ਘ ೞࣽ ಕ੉ಌࢪ ࠺௑ ۽Ӓ ࣻ૘ ೐۽ࣁझী ੸ਊ • 1ѐਘ ੿ب ݽפఠ݂ റ ղ֙ 1ਘ҃ ࢲ੼ীب ੸ਊ ৘੿ AWS Lambda Amazon
 S3 KAFKA
32. Q&A
33. Thanks