Slide 1

Slide 1 text

Client-side JS for infeed layout native ad at fluct SSP Tetsuharu OHZEKI

Slide 2

Slide 2 text

This slide is for i18N • For Japanese speaker –俺の話を聞け (Listen to me :) • For English speaker –Sorry, I’ll talk Japanese.

Slide 3

Slide 3 text

About me • Name: Tetsuharu OHZEKI – github: @saneyuki • Software Engineer of “fluct” div. in VOYAGE GROYP, Inc. – Almost coding JavaScript for advertisement system or management system. • Architecting today’s talking area. • Architecting React based internal applications. • Open Source Hacker – Mozilla committer level 3 (Servo reviewer, but inactive now) – Contributing to rxjs – or etc...

Slide 4

Slide 4 text

Today, I don’t talk about… • Adblock & future of Web • Legacy display advertisement –Or “document.write() must be died”. • You should use my proposed perfect special greatest flux architecture & framework. • Are you ok?

Slide 5

Slide 5 text

Introduction

Slide 6

Slide 6 text

Today’s online advertisement Your Device SSP (Our stance) DSP DSP DSP RTB auction Adnetwork Adnetwork Adnetwork • Manage ad request • The behavior is like a “tag manager” as warehouser

Slide 7

Slide 7 text

Infeed layout web advertisement • Construct Ad from: – Design template • html or App’s native view – Ad content • represented as json or other data format. • 2 development style – SDK style – Consulting style (our service style)

Slide 8

Slide 8 text

OpenRTB protocol • De-facto standard – Google does not use it as “Tier 1” – “At least, this is my feeling” • This protocol is defined by iAB – Definitions of response/request format. – An ad company creates super-set or sub-set spec for their system • As a platform, we should (would like to) support full features.

Slide 9

Slide 9 text

js-tracker in OpenRTB protocol • This feature enable that DSP embeds an arbitrary JavaScript. – Purpose: custom metrics by DSP (e.g. checking viewability in viewport) • Of course, this can be a vulnerable point. – DSP can inject malware script via this. – In almost case, we can fight on court. • By the protocol, DSP may use document.write()! – We need to restrict a protocol with patching! – We must kill document.write() for performance!

Slide 10

Slide 10 text

Support various ad networks • So to up filling rate, get a chance to show many ads. – So SSP’s core value is how many DSP/adnetworks a SSP connects. • What ad networks should be deliver? – Adjustment by a pre-defined data. – In almost case, RTB auction would be a top priority.

Slide 11

Slide 11 text

Current Client-Software Design

Slide 12

Slide 12 text

Data flow overview Web Page Our Script Ad network Ad network Ad network Our (SSP) ad server RTB Auction DSP

Slide 13

Slide 13 text

Data flow overview Web Page Our Script Ad network Ad network Ad network Try to request Try to request Try to request Our (SSP) ad server RTB Auction DSP If auction is failed

Slide 14

Slide 14 text

Construct JavaScript in ad server • To reduce RTT and a file size – We can response only codes that will be used “actually”. – JIT compiling in a broad sense. • Concatenate a strings is fast. – It’s slow that executing escodegen every time per request. • But simple concat string is dangerous. • Let’s imagine `var a = ${x}` • If x is ”1; alert(‘I am malware’)”…? • Expand => `var a = 1; alert(‘I am malware’);` // alert on load!

Slide 15

Slide 15 text

Construct JavaScript in ad server safely • Answer => Encode a value to JSON once. – Create the string that will be evaluate as an object literal of JavaScript. • Reimagine:`var a = ${JSON.stringify({ x })}` • Expand `var a = { “x”: “1; alert(‘I am malware’)” };` – This will be evaluated as an object on load time! • After all, we response RTB results + JS code.

Slide 16

Slide 16 text

Support full async loading • Today’s web advertisement is required “performance”. – E.g. iAB’s LEAN: https://www.iab.com/news/lean/ • We cannot use document.write() newly in 2016.

Slide 17

Slide 17 text

Support full async loading • We need to detect “where does this request comes from?” – Same script tag may be inserted into the same page. – document.currentScript is not supported by legacy Android. • Generate an unique key to a request. – The response reply it. Web Page Ad script Ad script Ad script Ad script

Slide 18

Slide 18 text

Prioritized Retry-able queue • To increase filling rate. • We’d like to fallback to the next priority if the top one does not return any ad contents on client.

Slide 19

Slide 19 text

Prioritized Retry-able queue • Two-dimentional array: Array> – 1d: an order to try sequential. – 2d: a group to try parallel. – Index=0 is max priority. – Items: Network Request Tasking to an ad network.

Slide 20

Slide 20 text

Prioritized Retry-able queue [ [RTB], // The result of RTB [A, B, C], // stage a: the group to try request adnetworks. [D, E, F], // stage b: start to try if all request fails in stage a. [G, H], // stage c: start to try if …. in stage b ] A is most high priority H is most lowest priority

Slide 21

Slide 21 text

Construct DOM safely • In native ad, we construct ad from design template + content. • If we does this in server side, we might cause XSS because there may be a difference with browser’s parser. • Use web browser’s html parser & DOM APIs is most reliable way.

Slide 22

Slide 22 text

Implementation

Slide 23

Slide 23 text

Implementation Style • Runtime (Supervisor) – Manage driver tasks & provide foundations. • Driver Task – Create for each ad networks. – The behavior is like a co-routine. • No use large try-catch – Async at all. – Convert to return an error value: tryBar()=>{ok, val, err,}

Slide 24

Slide 24 text

Limitations • We cannot large 3rd party libraries. – 1kb/res * 1billion request => 1Tb • So we cannot use Promise… – github:stefanpenner/es6-promise/dist/es6-promise.min.js requires 6.2kb. – We need to support Android ~4.x still now for “the long-tail”.

Slide 25

Slide 25 text

Implementations • We use TypeScript – target=ES3, modules=AMD • We write a classic async co-routine without Promise/ES6 generator. – Core Runtime: about 2000 lines – Each of drivers: almost 300 lines~ • Size (minified & minimum case): 10kb / response – This would be increase 6~8kb per driver.

Slide 26

Slide 26 text

Others implementation problem… • navigator.sendBeacon() might not work on a loading critical path. – From the spec, this behavior would be valid. • Unit tests fails in legacy Android because Android browser’s timer queue is overflowed. • Integration with DOM event loop & our co-routine’s event loop. • I can talk you about them in a networking time

Slide 27

Slide 27 text

Future Work

Slide 28

Slide 28 text

Telemetry Reporter (work in progress) • We cannot debug all environment in chaos – Network Conditions – Devices – Web browsers – Ad content – Embed Page – Our program state • Need to detect fatal errors. • Report only our errors, not embedders. • Catch only “expected” errors, not all.

Slide 29

Slide 29 text

Refactoring priority queue (still be in considered) • The current implementation may not be good abstraction. – We “had” many dreams when we designed it. – But now, we abandoned some of them…. • We might be able to make it an one-dimentional array.

Slide 30

Slide 30 text

Any questions in the networking time?