Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How we deal with large traffic of LINE Official Account in realtime

How we deal with large traffic of LINE Official Account in realtime

Yusuke Todaka
LINE LINE OA Dev 2 Team Software Engineer
https://linedevday.linecorp.com/jp/2019/sessions/S1-14

Be4518b119b8eb017625e0ead20f8fe7?s=128

LINE DevDay 2019

November 20, 2019
Tweet

More Decks by LINE DevDay 2019

Other Decks in Technology

Transcript

  1. 2019 DevDay How We Deal With Large Traffic of LINE

    Official Account in Realtime > Yusuke Todaka > LINE LINE OA Dev 2 Team Software Engineer
  2. > Server Side Engineer > Developing LINE Official Account /

    LINE Bot SDK Ruby > Developer Center 4, OA Dev 1 team Todaka Yusuke
  3. Ƃ LINE Official Account Manager https://manager.line.biz

  4. Ƃ Event Tracking System

  5. Ƃ Retargeting With User Action impression / click

  6. Agenda > What Is Broadcast Targeting Estimation > Technical Difficulties

    > Solution - KMV Sketch > Accuracy & Benchmark Love Realtime,
 Love Approximation.
  7. Ƃ Target Estimation impression / click

  8. Large Set Operation in Sub-Second for UX > Complex set

    operations : Combination of Union, Exclude, Filtering > Everything should be done in less than 1 second for UX > Large User Group Size : ~ 100M users Technical Difficulties
  9. Kth Minimum Values Sketch Solution : KMV Sketch > Distinct

    Counting Algorithm > Sampling With Hash Function > Supports Union, Intersection, Difference Operation
 > Use More Space Than HyperLogLog (~800KB for 60K Nominal Entries)
  10. https://datasketches.github.io/ Implementation : Apache DataSketches

  11. Architecture Events Sketch
 Storage Sketch Calculator Server Target Estimator Server

  12. https://datasketches.github.io/docs/Theta/ThetaErrorTable.html 1 RSE (%) 0.01 0.1 1 10 Nominal Entries

    2^8 2^12 2^16 2^20 2^24 RSE Accuracy vs Nominal Entries
  13. Intersection Operation of 2 Sets. Read From Local File. Nominal

    Entries = 65K. Time (ms) 1 10 100 1000 10000 Entries 1K 10K 100K 1M 10M HashSet KMV Performance Benchmark https://github.com/toduq/theta-sketch-benchmark
  14. Thank You