리디북스 비콘 로그 수집 개선

by RIDI

Slide 1

Slide 1 text

ܻ٣࠘झ ࠺௑ ۽Ӓ ࣻ૘ ѐࢶ ؘ੉ఠ౱ ݃ࢿള

Slide 2

Slide 2 text

Table of Contents • ؘ੉ఠ౱੄ ۽Ӓ ࣻ૘ Pipeline ߂ ࣻ૘ೞח ۽Ӓ੄ ઙܨ • Beacon Log ࣻ૘ അട ߂ ޙઁ • Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda • AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ • അ੤੄ ؘ੉ఠ ౵੉೐ۄੋ ҳࢿ

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

ؘ੉ఠ౱ীࢲ ࣻ૘ೞח ۽Ӓ੄ ઙܨ • ࢲ੼ ਢࢲߡ Access Logs => Beacon Log • ࠭য (ਢ࠭য ನೣ) Viewer Logs • Ѩ࢝ ۽Ӓ • ӝఋ ࢲߡ Access Log, Django Log, etc...

Slide 5

Slide 5 text

Beacon Log ࣻ૘ അട ߂ ޙઁ • ࢎ੹੸ ੄޷ • ੌ੿ೠ ૑੼ীࢲ ౠ੿ೠ ߑೱ੄ ੹౵ܳ ߊࢎೞৈ    ೦ҕӝ, ࢶ߅ ١ী Ӓ ૑੼ী ؀ೠ ਤ஖ա ߑೱਸ ঌܻח ੢஖ • प୓(beacon.gif) : 1 * 1 px ੄ gif ੉޷૑

Slide 6

Slide 6 text

Beacon Log ࣻ૘ അട ߂ ޙઁ • ೨ब IDEA • ౠ੿ ࢲߡী 1px૞ܻ gif(beacon.gif) ౵ੌਸ publicೞѱ ࢲ࠺झ • ۽Ӧ ೙ਃद ؘ੉ఠܳ gif ౵ੌ ઱ࣗ ٍী request parameter۽ ઁҕ • ೧׼ ਃ୒਷ ਢࢲߡী Access Log۽ ӝ۾ؽ • Access Log ઺ beacon.gif ܳ GET ೠ ਃ୒݅ ୶۰ղ ౵ۄ޷ఠ ౵य

Slide 7

Slide 7 text

Beacon Log ࣻ૘ അട ߂ ޙઁ

Slide 8

Slide 8 text

Beacon Log ࣻ૘ അട ߂ ޙઁ

Slide 9

Slide 9 text

Beacon Log ࣻ૘ അട ߂ ޙઁ

Slide 10

Slide 10 text

Beacon Log ࣻ૘ അട ߂ ޙઁ Request URL:https://ridibooks.com/images/log/beacon.gif? section=Pc.home.bestseller.general&obj_id=606001670&idx=0&even t=display&u_id=masunghoon&ruid=5b8dc371-c07d-487c-b554- a96676b44bb6&device=Pc

Slide 11

Slide 11 text

Beacon Log ࣻ૘ അട ߂ ޙઁ • Access Log • 1ੌ: ড 40,000,000Ѥ (ࢲ੼ ਢࢲߡ 1؀ ӝળ) • 1ѐਘ: > 1,000,000,000Ѥ (10র Ѥ ੉࢚) • Beacon Log • 1ੌ: ড 13,000,000Ѥ (ড 1/3) • 1ѐਘ: > 400,000,000Ѥ (4র Ѥ ੿ب)

Slide 12

Slide 12 text

Beacon Log ࣻ૘ അട ߂ ޙઁ

Slide 13

Slide 13 text

Beacon Log ࣻ૘ അട ߂ ޙઁ • ୶ۿ • ݫੋ ಕ੉૑ী ૘઺ ػ Beacon objects • 1PV ׼, ݆਷ ҃਋ ࣻभഥ੄ beacon tracking ਃ୒ • पઁ۽ • ಣࣗ 30%࠺઺੄ Beacon logо 50% ୡҗ (ࣽࣻೠ Access Log ৉੹) • ژೠ ਃ୒ ੺؀۝੉ ಩ૐ

Slide 14

Slide 14 text

Beacon Log ࣻ૘ അട ߂ ޙઁ • ୶ۿ • ݫੋ ಕ੉૑ী ૘઺ ػ Beacon objects • 1PV ׼, ݆਷ ҃਋ ࣻभߣ੄ beacon tracking ਃ୒ • पઁ۽ • ಣࣗ 30%࠺઺੉؍ Beacon logо 40% ੉࢚ ੼ਬ • ژೠ ਃ୒ ੺؀۝੉ ಩ૐ

Slide 15

Slide 15 text

Beacon Log ࣻ૘ ѐࢶ ߑೱ Pure Access Log  +  Beacon Log

Slide 16

Slide 16 text

Beacon Log ࣻ૘ ѐࢶ ߑೱ Pure Access Log Beacon Log

Slide 17

Slide 17 text

Beacon Log ࣻ૘ ѐࢶ ߑೱ

Slide 18

Slide 18 text

Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda • ҳ࢚ೞח दझమ੉ Lamdbaэ਷ ࢲ࠺झ৬ Ҿ೤੉ જ׮Ҋ ౸ױ • HTTP request ܳ Ӓ؀۽ KAFKA۽ షझ • json ١ਵ۽ ৘ࢁѱ ౵यೞৈ షझೞݶ ؊ જ਺ ! • ਃ୒੉ ੸਷ दр؀ী ஹೊ౴ ੗ਗਸ խ࠺ೡ ੌ হ਺

Slide 19

Slide 19 text

Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda Amazon API Gateway KAFKA AWS Lambda

Slide 20

Slide 20 text

Beacon Log അട • Access Log • 1ੌ: ড 40,000,000Ѥ (ࢲ੼ ਢࢲߡ 1؀ ӝળ) • 1ѐਘ: > 1,000,000,000Ѥ (10র Ѥ ੉࢚) • Beacon Log • 1ੌ: ড 13,000,000Ѥ (ড 1/3) • 1ѐਘ: > 400,000,000Ѥ (4র Ѥ ੿ب)

Slide 21

Slide 21 text

Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda • AWS Lambda: 1,000,000 ഥ ׼ $0.2 • 4রѤ: ؀ۚ $80 • API Gateway: ???

Slide 22

Slide 22 text

Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda • 1,000,000 : $3.50 = 400,000,000 : ? • 1 : $3.50 = 400 : x • x = $3.5 * 400 = $1,400 • 1,500,000 ਗ / ਘ x 400,000,000 Ѥ

Slide 23

Slide 23 text

Beacon Log ࣻ૘ ѐࢶ ߑೱ - with AWS Lambda

Slide 24

Slide 24 text

AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ AWS Lambda JavaScript SDK KAFKA

Slide 25

Slide 25 text

AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ AWS Lambda JavaScript SDK

Slide 26

Slide 26 text

AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ AWS Lambda Amazon  S3 KAFKA

Slide 27

Slide 27 text

AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ beacon.gif beacon-log bucket Lambda Function HTTP GET trigger S3 Access Log

Slide 28

Slide 28 text

AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ • 10,000 : $0.0035 = 400,000,000 : ? • 1 : $0.0035 = 40,000 : x • x = $0.0035 * 40,000 = $140 • 150,000 ਗ / ਘ

Slide 29

Slide 29 text

AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ • ױ੼ • S3੄ Access-logо ߓ஖ࢿਵ۽ ࣻ૘ - पदрࢿ ੌࠗ ࢚प • ੢੼ • $ • S3੄ Access-logо ߓ஖ࢿਵ۽ ࣻ૘ - Lambda ഐ୹ഥࣻ хࣗ AWS Lambda Amazon  S3 KAFKA

Slide 30

Slide 30 text

AWS Lambda੄ Trigger ҙ۲ Ҋ޹җ ؀উ • ૑դ 11ਘ ೞࣽ ಕ੉ಌࢪ ࠺௑ ۽Ӓ ࣻ૘ ೐۽ࣁझী ੸ਊ • 1ѐਘ ੿ب ݽפఠ݂ റ ղ֙ 1ਘ҃ ࢲ੼ীب ੸ਊ ৘੿ AWS Lambda Amazon  S3 KAFKA

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text