post-backs from installed apps, conversions Provide more detailed and effective feedbacks Make targeting more efficient › Estimated user attributes, audience delivery, geographic information › Tools to estimate reach of contents Protect user privacy and respect user choice › Data handling with care, Opt-out
post-backs from installed apps, conversions Provide more detailed and effective feedbacks Make targeting more efficient › Estimated user attributes, audience delivery, geographic information › Tools to estimate reach of contents Protect user privacy and respect user choice › Data handling with care, Opt-out
post-backs from installed apps, conversions Provide more detailed and effective feedbacks Make targeting more efficient › Estimated user attributes, audience delivery, geographic information › Tools to estimate reach of contents Protect user privacy and respect user choice › Data handling with care, Opt-out
audience › Look Alike Audience › … Batch Accumulation Upload by Companies › Phone Number / E-mail Address Audience › IDFA/AAID Audience › LINE User ID Audience › … Real-time Accumulation › Mobile App Event Audience › LINE Tag Event Audience › Official Account Message Click Event Audience › … User A DB Advertiser X’s Web Site Audience Group 001 for X 6TFS*% 5JNFTUBNQ VTFS@B VTFS@C ʜ ʜ Visit Accumulate Example: LINE Tag Event Audience
› DSP = Demand Side Platform › Optimize ads distribution based on several KPIs › Decides which ad is shown to a user › Wants to return as fast responses to media side as possible › To distribute contents to as many users as possible! › Utilizes audience data retrieved from DMP’s DB: Redis
Our DSP ,FZ 6TFS*% 'JFME ,FZ7BMVF 'JFME 7BMVF VTFS@B BVEJFODF@HSPVQ BVEJFODF@HSPVQ ʜ ʜ VTFS@C BVEJFODF@HSPVQ BVEJFODF@HSPVQ ʜ ʜ VTFS@D ʜ Redis Data Schema: Using Redis Hashes ID for a group of Audience When a user is added to the group?
field names of a single key › According to the service expansion, we have more users and more audience groups › More audience groups which a user belongs to, slower latency of HKEYS become › Difficult to satisfy the DSP’s requirement
value = Need for GET before SET › Need to execute the entire process above atomically › because we had the multiple points to store data into Redis › The First Approach: Redis Lua scripting Redis App 5. SET 2. Deserialize 3. Put a new group 4. Serialize 1. GET Realtime Accumulation Batch Accumulation User Upload Redis (Hash) Introduced in Redis v2.6.0
local packed = redis.call('GET', key) if (packed) then unpackedAudiEenceGroupIds = cmsgpack.unpack(packed) end for i =1, #ARGV do ¥n" if (not unpackedAudienceGroupIds[ARGV[i]]) then unpackedAudienceGroupIds[ARGV[i]] = 1 end end local setDataPacked = cmsgpack.pack(unpackedAudienceGroupIds) redis.call('SET', key, setDataPacked) return 'OK' But, it was slow T_T - Redis’s Single Threaded Model - MessagePack Ser/Deser requires much time resource
Accumulation Batch Accumulation User Upload Kafka Redis Audience Indexer Our DSP › By using ”User ID” as a Kafka key, we ensure the order of GET - > SET without any workarounds on Redis servers › Side benefits: it made easier global control of rate limits to Redis. Kafka Key: “User ID”
changes › 1. Changed the data structure › Redis Hash => Redis String › Improve GET latency › 2. Changed the process › Store directly to Redis => Via Kafka › Improve SET simplicity
post-backs from installed apps, conversions Provide more detailed and effective feedbacks Make targeting more efficient › Estimated user attributes, audience delivery, geographic information › Tools to estimate reach of contents Protect user privacy and respect user choice › Data handling with care, Opt-out
post-backs from installed apps, conversions Provide more detailed and effective feedbacks Make targeting more efficient › Estimated user attributes, audience delivery, geographic information › Tools to estimate reach of contents Protect user privacy and respect user choice › Data handling with care, Opt-out
$"%&'()*"% # "# +,) $-*./ Everyone wants better CVR ▷ Need to measure exact CVR! › Conversion = A “visitor” performs an action that you wanted them to do! › ex) Install an app, purchase a product, register an account etc..
Storage › Use Redis › Stored with some different DB keys › Cookies (3rd party) made on the LINE domain › click ID: unique ID for each click issued in the redirector Redirector Click Log Click Ads Store clicks
= a cookie made on the domain in the address bar › Cookie owner = A website (which a user visits) owner › 3rd party cookie = a cookie made on a different domain › cf: iframe, image tags › Example: when a user visit https://example.com, › Cookies on ”example.com” = 1st party cookie › Cookies on any other domains = 3rd party cookie LINE’s domain Advertisers’ domain
judge if a user has already clicked the ads? › 1. Click ID: a unique ID for each click issued in the redirector › If their landing page = Conversion point, we can find a click log directly › 2. 3rd party cookie: a cookie associated with LINE’s domain › BUT: 3rd party cookie is almost dead (e.g. Safari, Chrome..) › 3. 1st party cookie: a cookie associated with advertiser’s domain › Created on JavaScript (by LINE Tag) › Oh wait, we don’t have a mapping between a click and 1st party cookie, right?
for 1st party cookie › Sub-flow 1 should be completed before Sub-flow 2 › ex) click ads ▷ visit a LP ▷ (1sec) ▷ visit a conversion page immediately LINE Tag Frond End Sub-flow 2: Conversion Measurement
for 1st party cookie › Sub-flow 1 should be completed before Sub-flow 2 › ex) click ads ▷ visit a LP ▷ (1sec) ▷ visit a conversion page immediately › Add a delay before searching click logs by 1st party cookie! › To put Sub-flow 2 behind Sub-flow 1 (Best effort) LINE Tag Frond End Conversion Condition Matcher Kafka Click Log Finder Add small delay An event matches any conditions?
post-backs from installed apps, conversions Provide more detailed and effective feedbacks Make targeting more efficient › Estimated user attributes, audience delivery, geographic information › Tools to estimate reach of contents Protect user privacy and respect user choice › Data handling with care, Opt-out
post-backs from installed apps, conversions Provide more detailed and effective feedbacks Protect user privacy and respect user choice › Data handling with care, Opt-out Make targeting more efficient › Estimated user attributes, audience delivery, geographic information › Tools to estimate reach of contents
only in the inside of LINE › Protect personal information › Provide the several opt-out functions › Respect user choice › One of DMP’s missions › How can we provide good touchpoints to companies with keeping user privacy?
Cross Targeting › Cross Report › Beyond Cookie & IDFA, AAID › Strengthen utilization of data owned by LINE › E.g. LINE Login How has LINE DMP made progress?
Manager + 3 Server-side Engineer + (1 Front-end Engineer) › Massive traffic makes developers join in discussions from the very beginning › Can enjoy the challenges as an engineer twice J › Severe business requirement › Massive traffic and complex data architecture
users › LINE DMP supports it with the power of data › LINE DMP provides various data with massive traffic to: › Make targeting more efficient › Give more detailed and sophisticated feedbacks › Protect user privacy and respect user choice › Challenging business problems & data processing › So many next challenges are waiting for us and you!
line_devday2020
kappa4
helayoty
hgsgtk
kilometer
kanaugust
hhiroshell
yuyamada
sugimomoto
kyonmm
chaspy
bells17
torisoup
geeforr
swwweet
myddelton
maltzj
michaelherold
tmm1
lara
keathley
jcasabona
shpigford
danielanewman
chrislema
![Lua Script local key = KEYS[1] local unpackedAudienceGroupIds = {}](https://files.speakerdeck.com/presentations/d47b467795834502bdf560af41f8126c/slide_18.jpg){kind=link}
