LINE's New Year campaign - Taking Control of High Traffic

LINE's New Year campaign - Taking Control of High Traffic

Kapilevich Viacheslav (Slava)
LINE Shop Team Software Engineer
https://linedevday.linecorp.com/jp/2019/sessions/S1-09

Be4518b119b8eb017625e0ead20f8fe7?s=128

LINE DevDay 2019

November 20, 2019
Tweet

Transcript

  1. 2019 DevDay LINE's New Year Campaign - Taking Control of

    High Traffic > Kapilevich Viacheslav (Slava) > LINE Shop Team Software Engineer
  2. What it looks like to us in LINE when you

    send “Happy New Year!” to your friends at midnight? >
  3. Traffic During New Year Night LINE Services Load 23:00 23:30

    00:00 00:30 01:00 01:30
  4. LINE’s New Year Campaign

  5. Last Year LINE’s New Year Campaign > And the campaign

    starts … > Disney, Ghibli and other official partners > More than 3000 specially made stickers > More than 25 000 000 users
  6. New Year Campaign Starts … HERE LINE Services Load 23:00

    23:30 00:00 00:30 01:00 01:30
  7. How to design system that can work in such conditions?

  8. Normal Scalable Service Millions Of Users … New Year Campaign

    Servers LINE Internal APIs … … DB Cluster
  9. Normal Scalable Service Millions Of Users … New Year Campaign

    Servers LINE Internal APIs … … DB Cluster Denial of Service Timed out and failed requests Increased latency of user requests
  10. Too Much Traffic Too Little Control

  11. Service With Async Execution Millions Of Users … New Year

    Campaign Servers LINE Internal APIs Action Topic Action Topic Action Topic … … … Consumers
  12. Service With Async Execution Millions Of Users … New Year

    Campaign Servers LINE Internal APIs API Topic API Topic API Topic … … … Consumers Instant response to user Offloading app servers Async actions are persisted
  13. Control Over Async Execution Users … App Servers LINE Internal

    APIs Action Topic … … … Consumers Processing Speed Control Processing Management Tool API Rate Limit Control Retry Throttling
  14. Control Over Async Execution Users … App Servers LINE Internal

    APIs Action Topic … … … Consumers Processing Speed Control Processing Management Tool API Rate Limit Control Retry Throttling Single control panel to control all processing parameters
  15. Control Over Async Execution Users … App Servers LINE Internal

    APIs Action Topic … … … Consumers Processing Speed Control Processing Management Tool API Rate Limit Control Retry Throttling
  16. Control Over Async Execution Users … App Servers LINE Internal

    APIs Action Topic … … … Consumers Processing Speed Control Processing Management Tool API Rate Limit Control Retry Can automatically or manually adjust speed of processing Throttling
  17. Control Over Async Execution Users … App Servers LINE Internal

    APIs Action Topic … … … Consumers Processing Speed Control Processing Management Tool API Rate Limit Control Retry Throttling
  18. Control Over Async Execution Users … App Servers LINE Internal

    APIs Action Topic … … … Consumers Processing Speed Control Processing Management Tool API Rate Limit Control Retry Rate limit access to a particular API Throttling
  19. Control Over Async Execution Users … App Servers LINE Internal

    APIs Action Topic … … … Consumers Processing Speed Control Processing Management Tool API Rate Limit Control Retry Throttling
  20. Control Over Async Execution Users … App Servers LINE Internal

    APIs Action Topic … … … Consumers Processing Speed Control Processing Management Tool API Rate Limit Control Retry If API fails, reproduce to Kafka and retry Throttling
  21. Control Over Async Execution Users … App Servers LINE Internal

    APIs Action Topic … … … Consumers Processing Speed Control Processing Management Tool API Rate Limit Control Retry Throttling
  22. Control Over Async Execution Users … App Servers LINE Internal

    APIs Action Topic … … … Consumers Processing Speed Control Processing Management Tool API Rate Limit Control Retry Throttling In case of our own system overload, can throttle user requests
  23. Full Control Over Traffic No Failed Requests

  24. Thank you