High-Speed Distribution to Enormous Group Using Pub/Sub of Redis

Kazuya Horiuchi
LINE / Development Team H / Engineer

https://linedevday.linecorp.com/2021/ja/sessions/115
https://linedevday.linecorp.com/2021/en/sessions/115
https://linedevday.linecorp.com/2021/ko/sessions/115

Transcript

2. Agenda - What is LINE LIVE - LINE LIVE Chat System - Architecture Introduction - Diversion of new architecture - For even safer operation - How are the results
3. What is LINE LIVE - LIVE Streaming service - Streams from celebrities, companies and other users - Support the broadcaster with cheer items and free hearts - Communication using chat comments - Real time is important
4. - LIVE Streaming service - Streams from celebrities, companies and other users - Support the broadcaster with cheer items and free hearts - Communication using chat comments - Real time is important What is LINE LIVE
5. LINE LIVE Chat System Core Part Introduction
6. - Parallel processing using Akka - Client-to-Server communication using WebSocket - Server-to-Server communication using Pub/Sub of Redis - Temporary storage of chat data in Redis - Permanent storage of chat data in MySQL from Redis - Control comments by user or system configuration LINE LIVE Chat System Core Part Introduction
7. - Parallel processing using Akka - Client-to-Server communication using WebSocket - Server-to-Server communication using Pub/Sub of Redis - Temporary storage of chat data in Redis - Permanent storage of chat data in MySQL from Redis - Control comments by user or system configuration LINE LIVE Chat System Core Part Introduction
8. Architecture Introduction How to distribute the configuration in a LINE LIVE Chat System
9. Conventional Architecture How to distribute the configuration in a LINE LIVE Chat System API LIVE Broadcaster Block LIVE Viewer A Send comments Save the block info MySQL Send the block info WebSocket MySQL local cache or MySQL Save the block info Internal API Chat No comments will be sent Other LIVE Viewer LINE LIVE API No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer LIVE Viewer A
10. Conventional Architecture How to distribute the configuration in a LINE LIVE Chat System API LIVE Broadcaster Block LIVE Viewer A Send comments Save the block info MySQL Send the block info WebSocket MySQL local cache or MySQL Save the block info Internal API Chat No comments will be sent Other LIVE Viewer LINE LIVE API No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer LIVE Viewer A
11. Conventional Architecture How to distribute the configuration in a LINE LIVE Chat System API LIVE Broadcaster Block LIVE Viewer A Send comments Save the block info MySQL Send the block info WebSocket MySQL local cache or MySQL Save the block info Internal API Chat No comments will be sent Other LIVE Viewer LINE LIVE API No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer LIVE Viewer A
12. Conventional Architecture How to distribute the configuration in a LINE LIVE Chat System API LIVE Broadcaster Block LIVE Viewer A Send comments Save the block info MySQL Send the block info WebSocket MySQL local cache or MySQL Save the block info Internal API Chat No comments will be sent Other LIVE Viewer LINE LIVE API No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer No comments will be sent Other LIVE Viewer LIVE Viewer A
13. What’s happened Why do we need a new architecture?
14. Chat decoration
15. Chat decoration
16. New Architecture How to distribute the configuration in a LINE LIVE Chat System API LIVE Viewer Set chat decoration Send comment Update chat decoration settings Set chat decoration WebSocket Redis Subscribe & local cache Publish Internal API Chat Decorated comment Other LIVE Viewer LINE LIVE API Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer MySQL
17. New Architecture How to distribute the configuration in a LINE LIVE Chat System API LIVE Viewer Set chat decoration Send comment Update chat decoration settings MySQL Set chat decoration WebSocket Redis Subscribe & local cache Publish Internal API Chat Decorated comment Other LIVE Viewer LINE LIVE API Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer
18. New Architecture How to distribute the configuration in a LINE LIVE Chat System API LIVE Viewer Set chat decoration Send comment Update chat decoration settings MySQL Set chat decoration WebSocket Redis Subscribe & local cache Publish Internal API Chat Decorated comment Other LIVE Viewer LINE LIVE API Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer
19. New Architecture How to distribute the configuration in a LINE LIVE Chat System API LIVE Viewer Set chat decoration Send comment Update chat decoration settings MySQL Set chat decoration WebSocket Redis Subscribe & local cache Publish Internal API Chat Decorated comment Other LIVE Viewer LINE LIVE API Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer Decorated comment Other LIVE Viewer
20. Diversion of new architecture Use cases where the new architecture can be assumed to be diverted
21. Diversion of new architecture Use cases where the new architecture can be assumed to be diverted API Publish system notification Save system notification MySQL Publish system notification WebSocket Redis Subscribe Publish Internal API Chat Send system notification LIVE Viewer LINE LIVE API Send system notification LIVE Viewer Send system notification LIVE Viewer Send system notification LIVE Viewer Send system notification LIVE Viewer Internal CMS
22. Diversion of new architecture Use cases where the new architecture can be assumed to be diverted API Send system notification Save system notification MySQL Send system notification WebSocket Redis Subscribe Publish Internal API Chat Send system notification LIVE Viewer LINE LIVE API Send system notification LIVE Viewer Send system notification LIVE Viewer Send system notification LIVE Viewer Send system notification LIVE Viewer Internal CMS
23. Diversion of new architecture Use cases where the new architecture can be assumed to be diverted API Publish system notification Save system notification MySQL Publish system notification WebSocket Redis Subscribe Publish Internal API Chat Send system notification LIVE Viewer LINE LIVE API Send system notification LIVE Viewer Send system notification LIVE Viewer Send system notification LIVE Viewer Send system notification LIVE Viewer Internal CMS
24. For even safer operation Improving the fault tolerance of Redis Subscribe
25. Redis Subscribe health check Improving the fault tolerance of Redis Subscribe Redis Subscriber Subscriber Subscriber Subscriber Subscriber Listen Listen Listen Listen Listen
26. Redis Subscribe health check Improving the fault tolerance of Redis Subscribe Redis Subscriber Subscriber Subscriber Subscriber Subscriber Listen Listen failed Listen Listen Listen
27. Redis Subscribe health check Improving the fault tolerance of Redis Subscribe Redis Subscriber Subscriber Subscriber Subscriber Subscriber Listen Listen failed Listen Listen Listen Publish (ping)
28. Redis Subscribe health check Improving the fault tolerance of Redis Subscribe Redis Subscriber Subscriber Subscriber Subscriber Subscriber Subscribe (ping) Subscribe (ping) failed Subscribe (ping) Subscribe (ping) Subscribe (ping) Publish (ping)
29. Redis Subscribe health check Improving the fault tolerance of Redis Subscribe Redis Subscriber Subscriber Subscriber Subscriber Subscriber Listen Reconnect Listen Listen Listen
30. How are the results - So far, we've been able to operate without any major problems. - New features are ready to be implemented quickly using this architecture. - The new feature has been well received by users.
31. Thank you