Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dealing With Time Travelers in Analytics

xyu
September 10, 2015

Dealing With Time Travelers in Analytics

It's not enough to just have an analytics infrastructure for real-time online actions. The proliferation of mobile and international expansions means analytics platforms must also account for offline actions while ensuring that it does not harm user experience. At WordPress.com, we are building and scaling an analytic infrastructure to satisfy the dual mandate of accepting arbitrarily ordered events while efficiently coalesce them for querying. Here's what we've learned on our journey towards a lambda architecture.

xyu

September 10, 2015
Tweet

More Decks by xyu

Other Decks in Technology

Transcript

  1. [email protected]

    @HypertextRanch 

    xyu.io 

    xyu 
    Dealing With Time Travelers in Analytics
    Xiao Yu / Automattic

    View full-size slide




  2. VaultPress
    Jetpack
    Simplenote
    Akismet
    Polldaddy
    Gravatar
    VideoPress
    IntenseDebate
    Simperium
    Code Poet
    Cloudup
    WooCommerce
    Longreads
    WordPress.com
    WordPress.com VIP

    View full-size slide

  3. –No One Ever
    “Gee... I really like watching a spinner go round
    and round.”

    View full-size slide

  4. 100ms Delay → 62 Years Wasted
    100ms Delay

    View full-size slide

  5. Better Page Speed → More Conversions

    View full-size slide

  6. Emerging Markets &
    Mobile
    More Latencies & Less Bandwidth

    View full-size slide

  7. Data
    Scientist
    Performance
    Engineer

    View full-size slide

  8. –Sage Advice
    “You can't improve what you can't measure.”

    View full-size slide

  9. Data
    Scientist
    Performance
    Engineer
    • Load pre-render
    • Add callback to all actions
    • Wait for successful response
    • Lazy load
    • Track only minimum required
    • Send via background task

    View full-size slide


  10. Don’t Block The Critical Path

    View full-size slide


  11. Don’t Block The Critical Path
    Build Optimistic UIs

    View full-size slide


  12. Don’t Block The Critical Path
    Build Optimistic UIs
    Build Native Apps

    View full-size slide

  13. Client Speed — Don’t Block

    View full-size slide

  14. <br/>window._tkq = window._tkq || [];<br/>window._tkq.push( [<br/>'recordEvent',<br/>'wpcom_post_edit',<br/>{ custom_data: '…' }<br/>] );<br/>
    <br/>window._tkq = window._tkq || [];<br/>window._tkq.push( [<br/>'recordEvent',<br/>'wpcom_post_edit',<br/>{ custom_data: '…' }<br/>] );<br/>

    View full-size slide

  15. <br/>window._tkq = window._tkq || [];<br/>window._tkq.push( [<br/>'recordEvent',<br/>'wpcom_post_edit',<br/>{ custom_data: '…' }<br/>] );<br/>
    <br/>window._tkq = window._tkq || [];<br/>window._tkq.push( [<br/>'recordEvent',<br/>'wpcom_post_edit',<br/>{ custom_data: '…' }<br/>] );<br/>

    View full-size slide

  16. <br/>window._tkq = window._tkq || [];<br/>window._tkq.push( [<br/>'recordEvent',<br/>'wpcom_post_edit',<br/>{ custom_data: '…' }<br/>] );<br/>

    src = '//stats.wp.com/w.js'
    async defer
    >
    <br/>window._tkq = window._tkq || [];<br/>window._tkq.push( [<br/>'recordEvent',<br/>'wpcom_post_edit',<br/>{ custom_data: '…' }<br/>] );<br/>

    src = '//stats.wp.com/w.js'
    async defer
    >

    View full-size slide

  17. –Einstein
    “Everything should be made as simple as
    possible, but not simpler.”

    View full-size slide

  18. Faster Analytics → 11.7% Bump in Conversions

    View full-size slide

  19. Client guarantees events are sent at-least once;
    servers cleans up data and does everything else.

    View full-size slide

  20. Clients tell us when events happened…

    View full-size slide

  21. Clients tell us when events happened… but

    client clocks are unreliable.

    View full-size slide

  22. Event A Event B
    ?

    View full-size slide

  23. Ȑ
    Ȑ
    Ȑ
    Ȑ
    Ȑ
    Ȑ
    Morphine

    View full-size slide

  24. Client Clock
    Server Clock

    10 20 21

    22 32
    10

    View full-size slide

  25. Client Clock
    Server Clock

    10 11 20 21

    22 32
    10

    View full-size slide

  26. Ȑ
    Ȑ
    Ȑ
    Ȑ
    Ȑ
    Ȑ
    Morphine

    View full-size slide

  27. https://automattic.com/work-with-us/data-wrangler/

    View full-size slide

  28. [email protected]

    @HypertextRanch 

    xyu.io 

    xyu 
    Thanks!

    View full-size slide