Realtime targeting platform use cases in LINE GAME PLATFORM

Jaeho Lee
LINE Plus Game Platform Dev 2 Software Developer
https://linedevday.linecorp.com/2020/ja/sessions/4801
https://linedevday.linecorp.com/2020/en/sessions/4801

Transcript

2. Agenda › Motivation › Log processing platform › Realtime targeting platform › Use case › Technical issues 1
3. Motivation 2
4. Motivation Targeting? Behavior Profile Environment Context Push Banner Reward … Persona 3
5. Motivation Real time? < 1s User < 2ms DBA < 100ms Developer 4
6. Motivation Why? Noise Cost Performance Reduce Reduce Increase 5
7. Motivation Event processing system 6
8. Motivation LOG vs EVENT › Not structured › Not important › Audit LOG › Structured › Important › Audit, Live update, Analysis EVENT 7
9. Motivation Why event processing system? › Filtering + Validation › Windowing + Aggregation processing › Latency
10. Log processing platform 9
11. Processor Storage D/E Service Ingestion Log processing architecture Targeting Graph Query Platform Service Spark Chat Auth Promotion Kafka ElasticSearch Hive Impala HBase HDFS Spring Batch Recommen dation 10
12. Platform Service D/E Service Recommen dation Storage Ingestion Query Processor Log processing architecture Targeting Spark Chat Auth Promotion Kafka ElasticSearch Hive Impala HBase HDFS Spring Batch Targeting Graph 11
13. Ingestion Platform Service Promotion Auth Chat D/E Service Recommen dation Storage Query Processor Log processing architecture Targeting Spark Kafka ElasticSearch Hive Impala HBase HDFS Spring Batch Targeting Graph Kafka 12
14. Storage HDFS HBase ElasticSearch Platform Service Promotion Auth Chat D/E Service Recommen dation Ingestion Query Processor Log processing architecture Targeting Spark Kafka Hive Impala Spring Batch Targeting Graph 13
15. Query Impala Hive Platform Service Promotion Auth Chat D/E Service Recommen dation Storage Ingestion Processor Log processing architecture Targeting Spark Kafka ElasticSearch HBase HDFS Spring Batch Targeting Graph 14
16. Processor Spring Batch Spark Platform Service Promotion Auth Chat D/E Service Recommen dation Storage Ingestion Query Log processing architecture Targeting Kafka ElasticSearch Hive Impala HBase HDFS Targeting Graph 15
17. Platform Service Promotio n Auth Chat Storage Ingestion Query Processor Log processing architecture Spark Kafka ElasticSearch Hive Impala HBase HDFS Spring Batch D/E Service Targeting Graph Recommen dation 16
18. Realtime Targeting Platform 17
19. Other Services Batch Realtime Targeting Platform Background Event Processing System Targeting Source 18
20. Event Processing System Overview 19
21. Source Sink Event Processing System Logical flow Event Collector Event Manager Service Event Event Log Session Event Channel 20
22. Session Manager Event Collector Sink Event Processing System Overall Event Manager Platform Service Game HBase Redis HBase Game Service Redis Stream Kafka Redis Stream 21
23. Event Processing System Session Manager 22
24. Event Processing System Session Manager Session Manager MetaStore Log 23
25. MetaStore Event Processing System Session Manager Session Manager Redis HBase Log 24
26. Event Processing System Session Manager MetaStore 25
27. Event Processing System Event Collector 26
28. Event Processing System Event Collector Session Session Manager Event Collector Log MetaStore
29. Event Processing System Event Collector Session Session Manager Event Collector Event Log MetaStore 28
30. MetaStore Event Processing System Event Collector 29
31. Event Processing System Event Manager 30
32. Event Processing System Event Manager Event Event Manager MetaStore Storage HDFS HBase ES Service Targeting Channels Game A Login User A 31
33. Targeting platform Overall 32
34. Source Targeting architecture Targeting service Event Manager ES History HBase Profile Realtime targeting Batch processing Redis Service Targeting users 33
35. Use case 34
36. Use Case Overall Admin Targeting Platform Push Platform ADMIN WEB Message Queue LINE API Server Sender Server Target Users Get Audience ID Deeplink App Tracking Server Send Reward 35
37. Technical issues 36
38. Technical issues › Dynamic dimension pre-processing › Redis stream adoption 37
39. Technical issues Dynamic dimension pre-processing 38
40. Dynamic dimension pre-processing Game has different dimensions Game A Game B LEVEL STAGE User1 1 300 LVL STAGE User1 30 22 LEVEL STAGE LVL Game A User 1 300 NULL Game B User NULL 22 30 LEVEL == LVL 39
41. Dynamic dimension pre-processing Dimensions changes frequently Game A Game C Game B LEVEL STAGE User1 1 300 LP Item1 User1 1 22 LVL STAGE User1 30 22 LEVEL STAGE LVL Game A User 1 300 NULL Game B User NULL 22 30 New Game!! 40
42. Dynamic dimension pre-processing Using remote configuration + SpEL 41 Profile Updater Kafka Configuration
43. Dynamic dimension pre-processing Using remote configuration + SpEL Kafka HBase LEVEL Item1 STAGE Game A User 1 NULL 300 Game B User 30 NULL 22 Profile Updater 42
44. Dynamic dimension pre-processing Using remote configuration + SpEL 43
45. Technical issues Redis stream adoption 44
46. Redis stream adoption Key Features › Consumer group support › Create stream easily › Since 5.0.0 45
47. Redis stream adoption Case 1: Purchase history of user User purchase history 46 Biz
48. Redis stream adoption Case 2: Purchase history of users Purchase history of users 47 Biz
49. Redis stream adoption Case3: System integration - Send reward - Purchase history of user 48 Biz
50. Redis stream adoption Case4: Purchase history + Daily login 49 - Send reward - Purchase history of user - Login history of user Biz
51. Redis stream adoption Problems 1. Business requirements can be always changed. 3. Changing the table structure for one game needs to much cost. 2. It’s hard to maintain bucketing or window. 50
52. Redis stream adoption Scoped event Event Event Manager MetaStore User Scope Game Scope Topic Scope 51
53. Redis stream adoption User event Event Event Manager MetaStore User Scope 52 Field1 Field2 Field3 Event1 1 1 300 Event2 30 1 22 … … … …
54. Redis stream adoption User event - TTL settings Event Event Manager MetaStore User Scope 53 Field1 Field2 Field3 Event1 1 1 300 Event2 30 1 22 … … … … › EXPIREAT key timestamp
55. Redis stream adoption User event - Aggregation update Event Manager User Scope 54 Field1 Field2 Field3 Event1 1 1 300 Event2 30 1 22 … … … … Profile aggrField aggrFieldExpireAt Game A User 31 timestamp value
56. Wrap up › Motivation › Log processing platform › Realtime targeting platform › Use case › Technical issues 55
57. Thank you 56