Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Proxy Wars: The Battle For Your Content

Proxy Wars: The Battle For Your Content

We’re seeing an important shift in the world of web publishing, and it’s driven in part by performance. As web pages become heavier and more complex, and as the public demand a broader spectrum of sources for their news, internet giants such as Facebook, Google and Apple have seen an opportunity to step in. In return for the promise of an immersive, tailor-made and lightening fast web experience, they offer to promote, curate and host your content for you. But who’s winning here, and at what cost?

First presented at Bristol Web Performance on 16/6/2016: http://www.meetup.com/bristolwebperf/events/230315192/.

George is a Senior Developer at the Financial Times, and currently engaged with the company’s drive to increase ‘reach’ into new readership markets. He’ll describe how they went about implementing offerings for Apple News, Google AMP and Facebook Instant Articles, the technical issues they presented, and how they’re performing against the early promises.

-------------------

Speaker Notes

-------------------

SLIDE 1

SLIDE 2
Hello!
I’m going to talk to you about the battle for your content, known to us in this room as the Proxy Wars

SLIDE 3

SLIDE 4
Thought experiment:
Imagine you were Facebook, and every day, you watched your billions of users doing this:

SLIDE 5

Very slow to load
not responsive
user leaves the Facebook ecosystem

From FB’s point of view, there are two big problems here:

terrible User Experience
they’re losing users to the publisher’s site

SLIDE 6

SLIDE 7
Over 2 billion people with smartphones around the world
90% of time on mobile is spent in apps
mobile apps mainly for:
social networking and messaging, plus entertainment and games
An average U.S. user uses 27 apps a month
but spends 79% of her time on just 5 apps
These statistics have a US bias, but give an idea of the problem for mobile news consumption

SLIDE 8
But content consumption on mobile is very different to desktop computers
In a given week:
44% of U.S. adults used a smartphone to access news
64% who did so on a desktop computer
78% top US news publishers get more traffic from mobile than desktop computers
But only 20% see mobile visitors spending more time per visit than desktop
mobile users are less engaged, and dropping off

SLIDE 9
Part of the problem may be the bad experiences that many users have when accessing mobile news sites
in recent research:
more than half of all data came from ads and other content filtered by adblockers
slowest home page in the study was Boston.com, averaging 39 seconds to load on a very fast 4G connection, including 31 seconds just to load ads.
the average user expects pages to load in two seconds or less. After three seconds, up to 40% users abandon the site.

SLIDE 10
Let’s look at the rise of social media.
An average U.S. consumer spends 44 minutes per day on Facebook alone.
News apps? All they get is 4 minutes a day
If Facebook were a country, it would be larger than China or India with its 1.5 billion active monthly users worldwide.

SLIDE 11
Social networks and messaging apps are the most engaging apps.
WhatsApp users connect 37 times a day
WeChat: 29
Japanese Line, 26
Israeli Viber, 20
Facebook: 20

SLIDE 12
Social networks and messaging apps have become top sources of news for their users, and top sources of traffic to news publishers.
Of U.S. adults, 35% came across news stories on Facebook and via other social networks last year
From July last year, Facebook referred more traffic to the top 400 publishers than Google:
38% of all referrals are from Facebook
35% of all referrals from Google

SLIDE 13

SLIDE 14
So what’s the big problem with news sites?
All too familiar to you. Many problems can be attributed to third-party scripts:
network hogging, means that content can’t be prioritised over other assets
CPU hogging
scroll-jacking,
render blocking,
forcing relayout
unused resources waste bandwidth
large download size
big problem for people on cheaper contracts, roaming, third-world
multiple duplicated tracking libraries, each with their own event listeners:
Stakeholders at your company probably don’t even use them!
misbehaving iframes
lack of HTTPS support
Intrusive adverts

SLIDE 15

SLIDE 16
May 2015: Facebook Instant Articles launched
April this year, opened up to all publishers worldwide
September 2015, Apple’s News app launched
pre-installed on iOS
more than 100 publishers worldwide.
October 2015, Twitter launched Moments
a magazine-like story format
curated streams of Tweets with photos, videos, links to articles
February 2016, Google publicly launched Accelerated Mobile Pages
open standard intended to speed up the mobile Web
Google only one of the stakeholders
There are plenty of other platforms emerging too

SLIDE 17
Why do these internet giants line up at publishers’ doors and ask for news?
As a category of content, ’The News’ passes the famous Silicon Valley “toothbrush test.” CLICK

“When deciding whether Google should spend millions or even billions of dollars in acquiring a new company, its chief executive, Larry Page, asks whether the acquisition passes the toothbrush test: Is it something you will use once or twice a day, and does it make your life better?”

SLIDE 18
Facebook and other Social Media giants want to make the experience of consuming content online more seamless.
They realise that, if they can offer the same content locally, they can:
improve the user experience
improve performance
improve engagement
sell better targeted adverts with all their demographic data
keep the user inside their app

SLIDE 19
Three main types of Distributed Content:
On-platform Aggregators
glorified RSS
Apple News,
On-platform hosts
host the editorial content to allow their users to interact with it
Facebook Instant Articles
Twitter Moments
Off-platform open standard
Accelerated Mobile Pages

SLIDE 20

SLIDE 21
Aggregation of RSS feeds from publishers
Captive audience of millions of iOS users
Pre-installed app
Markup in Apple News Format:
proprietary, complex and buggy layout as you’d expect from a new markup language
Opaque analytics
aggregated
can’t track visits which start on Apple News
Just get a generic ‘Apple News’ referrer
paywall is coming with a 30% cut, but we can’t properly engage with the users

SLIDE 22

SLIDE 23
Demo
Our Social media editor said:
“so, why can’t we just make our website like this? Like, um, fast?”
Supported in Native apps for iOS and Android
Hijacks URL clicks
Lightening bolt means that the publisher has made an Instant Article version available
Loads in-app, near instantly, rather than loading the URL
No active promotion from FB, but because they increase engagement (sharing, commenting, liking), they appear more often in News Feeds
Proprietary HTML format
H3 doesn’t exist!
Text outside is ignored
iframes with inline content!
other magic!
CSS is removed, instead there’s a very basic style editor
Publish via RSS or push to an API

SLIDE 24
Apple News and Instant Articles are similar:
Essentially, fancy news aggregators
with custom renderers
built on top of proprietary syndication formats
in-app experiences:
neither allows users to view outside the native apps
even during our QA process!
either RSS or API driven:
need to update your CMS to produce RSS or push to API
Requires you to push content updates too - important for corrections and legal
both keen to improve performance, but…
they are focussed on making articles look and feel modern.
Slick & smooth interactions,
embracing native hardware acceleration to immerse user.

SLIDE 25
Imagine you’re Google, and you observe Apple and Facebook
negotiating deals with publishers
setting up walled gardens of news content

What do you do?

Google’s incentive here is obvious:
It makes its money on advertising
the vast majority of that advertising is on the open web
If Facebook provides a much better mobile experience than the open web does
those advertising opportunities, and that user data,
CLICK disappears into Zuckerberg’s Walled Garden of Earthly Delights, as Nieman Lab called it

SLIDE 26
AMP is more to do with changing the mobile web than building a new platform or app
AMP isn’t a business partnership
like Instant Articles or Apple News are
tries to do something more significant: change the way that the web is built
killing off some technologies
promoting others

SLIDE 27
“we want to make it easier to create documents that are fast-by-default”
Open-source
Web-based (works equally in system browser, in-app browsers, webviews)
Complete HTML documents which the publisher hosts.
Can optionally be cached on CDN, where documents are:
validated
parsed
partially rendered by the AMP runtime.
far more about performance than about aesthetics or interaction patterns
Smashing Magazine nice intro to AMP

SLIDE 28
Some elements forbidden:
Frames, embeds, objects, applets all forbidden
Some new elements to replace forbidden elements
amp-img: optimised for viewport size, scroll position, connectivity
amp-video & amp-audio: lazy-load based on viewport position
amp-iframe: enforce security and performance by sandboxing & controlling where iframes may appear
And to introduce new functionality:
amp-ad: ad loading managed by AMP in the same way as videos and images, and no JS can run in parent document to cause reflow and unnecessary layout calculations
amp-analytics and amp-pixel: centralised push for all metrics with unlimited consumers
amp-font: control timeouts and fallbacks
Others like carousel, lightbox, fit-text, install serviceworker, social media plugins
These are genuine Custom Elements
which collaborate with the AMP runtime
to prioritise resource loading
and enforce security

SLIDE 29
designed specifically to work well across mobile devices, rather than fully responsive
Styled with standard CSS to a great extent
Some limitations:
All CSS is inline
Limited to 50K
!important is banned to ensure that AMP can control element sizing

SLIDE 30
Layout:
The runtime must be able to infer the size of all externally loaded resources before they are actually loaded
so that a final layout can be calculated as quickly as possible.
All externally loaded resources must have width, height and one of the supported layout properties
Responsive layout will reserve a space relative to the aspect-ratio defined for the element

SLIDE 31
JS limited to AMP runtime and AMP extensions
encapsulated interactive functionality
so publishers don’t harm performance by writing their own
Amp-iframe allows JS
with limited communication to parent document
can request a resize

SLIDE 32
manages resource loading and prioritisation
security (Fonts only over HTTPS), etc.
implements AMP components,
Centralised system for layout
optionally, includes a runtime validator for AMP HTML by adding a hash to the URL: #development=1
We get exposure in the Google Webmaster console of validation failures
Chrome Extension to highlight an AMP alternative for a page, and show validation errors

SLIDE 33
Google advocates using their AMP cache: a proxy-based CDN with extra features
Anyone may run their own AMP cache on their own CDN, or publishers can use Google’s for free

SLIDE 34
AMP documents have a 0.7 second median load time
99% of load times are < 8 seconds
could be better, but
Non-AMP is 22 seconds at the 99th percentile

SLIDE 35
8 second timeout fades in the page without JS or fonts if those resources have failed to load
Space reserved for components using explicit width & height to set aspect ratio, plus a layout type
All scripts are async
but the main runtime can still render the page before they load
grey boxes, loading indicators, YouTube poster images are drawn in a reserved space
ZERO scripts block font downloads
Components are allowed to resize after initial render, if:
If below the viewport, fine
If above the viewport, fine, and scroll-top is adjusted so the content in view remains in exactly the same place relative to the viewport
If in the viewport, no resizing allowed unless following a user interaction (so requires a button to be shown)
DOM mutation batching (FastDOM)
How to support unlimited analytics providers but guarantee good performance?
Separate instrumentation from reporting
so AMP runtime reports events like “click”, “scroll = 25%”, etc.

SLIDE 36
Prerender is used to make Search Carousel pages load instantly
but not for all of each page, as that’s a huge waste of resources
Only what’s in the viewport.
‘With prerendering, you go from fast to instant`
For the carousel when prerendering, only load content which is visible, and only using the minimum network and CPU to give the impression of loaded
Carousel sends a postMessage call to the iframe to say “now you’re visible, load everything for real”
Preconnect API used extensively

SLIDE 37
You can see that we serve the pages directly
…but that we also make use of Google’s CDN
Google currently presents AMP documents in its Search Carousel
supported on mobile devices only
they’re rolling out slowly across the world
Nothing to stop other organisations setting up a CDN, or presenting AMP documents

SLIDE 38
Imagine you’re a publisher
What do you do about distributed content?
Do you keep on making new versions of every article in a new format?
Do you still have control?

SLIDE 39
huge new markets to develop growth
BUT, the gatekeepers decide all the rules
new content formats
Lose control of QA by publishing through a platform
Each new platform requires a lot of work to transform content, QA, adapt CMS, investigate publishing workflow
they keep users on their platforms rather than sending them to publishers’ websites
advertisers can sell directly to users on a social network
they have richer demographic data
why should they still work with the publishers?
Brands lose their identity when presented in this way
Do readers ever know who they’re reading?
Analytics is kept under the platform’s control.
Aggregated (mention Redshift work),
lack of demographic data
In-app browsers (FIA, Twitter moments) don’t share cookies
so can’t track signed-in users
Paywall implementations are slow in coming,
hard for the FT to experiment and assess effectiveness
Totally against the ‘progressive web app’ philosophy - more like an ‘m.’ site (see Andrew’s post)

SLIDE 40
What does AMP offer that we cannot offer ourselves already?
It’s not offering “how to make a fast website”…
we can do that already.
Instead, it’s a verification that the site will perform well
in terms of:
render speed,
jank,
network usage
mobile UX.

SLIDE 41
The AMP project freely admits that their results will never be as good as websites which are CLICK hand-tuned for the best performance.

SLIDE 42

Time to talk about the Content Performance Policy
Draft spec in the works
main author is Yoav Weiss, who spoke last month
contributions from other experts you will have heard of:
Tim Kadlec, Ilya Grigorik

Bruce has already told us the heartwarming story of the Picture Element.

I like to think that if Bruce is the King of Responsive Images, Yoav is the queen, having implemented the Picture Element in Blink & Webkit

SLIDE 43
Site authors gain control over the performance of their pages
You would define a policy using dedicated directives in a HTTP header
like no hijacking of scroll events
The browser could then view this as a “promise” that the site adheres to the specified policy
in this case, that it doesn’t hijack any scroll events
If the site then tried to break its promise, the browser would make sure that it cannot
e.g. ignore attempts to cancel the scroll event
An embedder, such as a search engine or a social network app, can then be certain that the “promises” provided by the developers are enforced, and the user experience on the site is guaranteed not to suffer from these anti-patterns.
For example, if Google could be 100% sure that a website had good performance, it could put it in the Search Carousel rather than requiring an AMP page
borrows heavily from the syntax of Content-Security-Policy
CSP defines the Content-Security-Policy HTTP header that allows you to create a whitelist of sources of trusted content
instructs the browser to only execute or render resources from those sources
you can instruct the browser to POST a JSON violation report back to you
so your can monitor a new policy without enforcing it

SLIDE 44
Three groups of directives:
enforcing performance best practices and opting into the fast path
No synchronous external scripts
nor blocking external stylesheets
Size limited inline scripts and styles
No text-blocking external Web fonts
I.e. no web fonts, or a fallback mechanism in place
No default-overriding events
Scroll and click-hijacking are classic
Document layout must not be dependent on resource download
Avoid content shifting by using intrinsic dimensions to layout before resources are available
No layout thrashing
reads and writes batched

SLIDE 45
2nd directive: reducing the consumption of resources
Lazy loading of images
Only download what’s needed
Very long viewports on mobile - why waste bandwidth?
Resource size limits
Limit total allowable download size for a page
Throttling CPU consumption on the main thread
Don’t let iframes hog CPU

SLIDE 46
3rd directive: enforcing mobile-friendly user experience
No auto playing video and audio
Debateable - how to allow ‘acceptable’ adverts?
Certain page components are limited to paint only on certain parts of the viewport
Prevent 3rd-party scripts from hijacking the viewport with a modal dialog, for example
No overlays
no popunders

SLIDE 47
Tools which can help us now:
Intersection Observer (lazy-load when ad/image comes into view)
Passive Event Listeners
listen to scroll events with no option to preventDefault means no blocking necessary
FastDOM
used extensively in the FT web app
At the FT we’re working on a Chrome extension to experiment with the CPP, reporting violations of as many rules as we can detect

SLIDE 48
In summary:
Internet giants want to improve user experience when consuming news
and keep control over their journey and data
Large platforms have a strangle-hold over publishers
Google pushed back with AMP
open-source,
on and of the web,
prioritising performance over everything
All these solutions are new formats, or variants of HTML,
requiring a fragmented rather than ‘progressive’ approach to content production
But, all provide a consistently good user experience.
AMP technology in particular should feed into browser vendors
Disruption for advertising and analytics in particular.
If all mobile web sites loaded near-instantly
would Facebook, Apple and others feel such a strong need to bring content inside their platforms

SLIDE 49

SLIDE 1 •

SLIDE 2 • Hello!

I’m going to talk to you about the battle for your content, known to us in this room as the Proxy Wars

SLIDE 3 •

SLIDE 4 • Thought experiment

Imagine you were Facebook, and every day, you watched your billions of users doing this:

SLIDE 5 • Very slow to load
not responsive
user leaves the Facebook ecosystem
From FB’s point of view, there are two big problems here:
terrible User Experience
they’re losing users to the publisher’s site

SLIDE 6 •

SLIDE 7 • Over 2 billion people with smartphones around the world
90% of time on mobile is spent in apps
mobile apps mainly for:
social networking and messaging, plus:
entertainment and games
An average U.S. user uses 27 apps a month
but spends 79% of her time on just 5 apps
These statistics have a US bias, but give an idea of the problem for mobile news consumption

SLIDE 8 • But content consumption on mobile is very different to desktop computers
In a given week:
44% of U.S. adults used a smartphone to access news
64% who did so on a desktop computer
78% top US news publishers get more traffic from mobile than desktop computers
But only 20% see mobile visitors spending more time per visit than desktop
mobile users are less engaged, and dropping off

SLIDE 9 • Part of the problem may be the bad experiences that many users have when accessing mobile news sites
in recent research:
more than half of all data came from ads and other content filtered by adblockers
slowest home page in the study was Boston.com, averaging 39 seconds to load on a very fast 4G connection, including 31 seconds just to load ads.
the average user expects pages to load in two seconds or less. After three seconds, up to 40% users abandon the site.

SLIDE 10 • Let’s look at the rise of social media.
An average U.S. consumer spends 44 minutes per day on Facebook alone.
News apps? All they get is 4 minutes a day
If Facebook were a country, it would be larger than China or India with its 1.5 billion active monthly users worldwide.

SLIDE 11 • Social networks and messaging apps are the most engaging apps.
WhatsApp users connect 37 times a day
WeChat: 29
Japanese Line, 26
Israeli Viber, 20
Facebook: 20

SLIDE 12 • Social networks and messaging apps have become top sources of news for their users
and top sources of traffic to news publishers.
Of U.S. adults, 35% came across news stories on Facebook and via other social networks last year
From July last year, Facebook referred more traffic to the top 400 publishers than Google:
38% of all referrals are from Facebook
35% of all referrals from Google

SLIDE 13 •

SLIDE 14 • So what’s the big problem with news sites?
All too familiar to you. Many problems can be attributed to third-party scripts:
network hogging, means that content can’t be prioritised over other assets
CPU hogging
scroll-jacking,
render blocking,
forcing relayout
unused resources waste bandwidth
large download size
big problem for people on cheaper contracts, roaming, third-world
multiple duplicated tracking libraries, each with their own event listeners:
Stakeholders at your company probably don’t even use them!
misbehaving iframes
lack of HTTPS support
Intrusive adverts

SLIDE 15 •

SLIDE 16 • May 2015: Facebook Instant Articles launched
April this year, opened up to all publishers worldwide
September 2015, Apple’s News app launched
pre-installed on iOS
more than 100 publishers worldwide.
October 2015, Twitter launched Moments
a magazine-like story format
curated streams of Tweets with photos, videos, links to articles
February 2016, Google publicly launched Accelerated Mobile Pages
open standard intended to speed up the mobile Web
Google only one of the stakeholders
There are plenty of other platforms emerging too

SLIDE 17 • Why do these internet giants line up at publishers’ doors and ask for news?
As a category of content, ’The News’ passes the famous Silicon Valley “toothbrush test.” CLICK

“When deciding whether Google should spend millions or even billions of dollars in acquiring a new company, its chief executive, Larry Page, asks whether the acquisition passes the toothbrush test: Is it something you will use once or twice a day, and does it make your life better?”

SLIDE 18 • Facebook and other Social Media giants want to make the experience of consuming content online more seamless.
They realise that, if they can offer the same content locally, they can:
improve the user experience
improve performance
improve engagement
sell better targeted adverts with all their demographic data
keep the user inside their app

SLIDE 19 • Three main types of Distributed Content:
On-platform Aggregators
glorified RSS
Apple News,
On-platform hosts
host the editorial content to allow their users to interact with it
Facebook Instant Articles
Twitter Moments
Off-platform open standard
Accelerated Mobile Pages

SLIDE 20 •

SLIDE 21 • Aggregation of RSS feeds from publishers
Captive audience of millions of iOS users
Pre-installed app
Markup in Apple News Format:
proprietary, complex and buggy layout as you’d expect from a new markup language
Opaque analytics
aggregated
can’t track visits which start on Apple News
Just get a generic ‘Apple News’ referrer
paywall is coming with a 30% cut, but we can’t properly engage with the users

SLIDE 22 •

SLIDE 23 • Demo
Our Social media editor said:
“so, why can’t we just make our website like this? Like, um, fast?”
Supported in Native apps for iOS and Android
Hijacks URL clicks
Lightening bolt means that the publisher has made an Instant Article version available
Loads in-app, near instantly, rather than loading the URL
No active promotion from FB, but because they increase engagement (sharing, commenting, liking), they appear more often in News Feeds
Proprietary HTML format
H3 doesn’t exist!
Text outside is ignored
iframes with inline content!
other magic!
CSS is removed, instead there’s a very basic style editor
Publish via RSS or push to an API

SLIDE 24 • Apple News and Instant Articles are similar:
Essentially, fancy news aggregators
with custom renderers
built on top of proprietary syndication formats
in-app experiences:
neither allows users to view outside the native apps
even during our QA process!
either RSS or API driven:
need to update your CMS to produce RSS or push to API
Requires you to push content updates too - important for corrections and legal
both keen to improve performance, but…
they are focussed on making articles look and feel modern.
Slick & smooth interactions,
embracing native hardware acceleration to immerse user.

SLIDE 25 • Imagine you’re Google, and you observe Apple and Facebook
negotiating deals with publishers
setting up walled gardens of news content

What do you do?

Google’s incentive here is obvious:
It makes its money on advertising
the vast majority of that advertising is on the open web
If Facebook provides a much better mobile experience than the open web does
those advertising opportunities, and that user data,
CLICK disappears into Zuckerberg’s Walled Garden of Earthly Delights, as Nieman Lab called it

SLIDE 26 • AMP is more to do with changing the mobile web than building a new platform or app
AMP isn’t a business partnership
like Instant Articles or Apple News are
tries to do something more significant: change the way that the web is built
killing off some technologies
promoting others

SLIDE 27 • “we want to make it easier to create documents that are fast-by-default”
Open-source
Web-based (works equally in system browser, in-app browsers, webviews)
Complete HTML documents which the publisher hosts.
Can optionally be cached on CDN, where documents are:
validated
parsed
partially rendered by the AMP runtime.
far more about performance than about aesthetics or interaction patterns
Smashing Magazine nice intro to AMP

SLIDE 28 • Some elements forbidden:
Frames, embeds, objects, applets all forbidden
Some new elements to replace forbidden elements
amp-img: optimised for viewport size, scroll position, connectivity
amp-video & amp-audio: lazy-load based on viewport position
amp-iframe: enforce security and performance by sandboxing & controlling where iframes may appear
And to introduce new functionality:
amp-ad: ad loading managed by AMP in the same way as videos and images, and no JS can run in parent document to cause reflow and unnecessary layout calculations
amp-analytics and amp-pixel: centralised push for all metrics with unlimited consumers
amp-font: control timeouts and fallbacks
Others like carousel, lightbox, fit-text, install serviceworker, social media plugins
These are genuine Custom Elements
which collaborate with the AMP runtime
to prioritise resource loading
and enforce security

SLIDE 29 • designed specifically to work well across mobile devices, rather than fully responsive
Styled with standard CSS to a great extent
Some limitations:
All CSS is inline
Limited to 50K
!important is banned to ensure that AMP can control element sizing

SLIDE 30 • Layout:
The runtime must be able to infer the size of all externally loaded resources before they are actually loaded
so that a final layout can be calculated as quickly as possible.
All externally loaded resources must have width, height and one of the supported layout properties
Responsive layout will reserve a space relative to the aspect-ratio defined for the element

SLIDE 31 • JS limited to AMP runtime and AMP extensions
encapsulated interactive functionality
so publishers don’t harm performance by writing their own
Amp-iframe allows JS
with limited communication to parent document
can request a resize

SLIDE 32 • manages resource loading and prioritisation
security (Fonts only over HTTPS), etc.
implements AMP components,
Centralised system for layout
optionally, includes a runtime validator for AMP HTML by adding a hash to the URL: #development=1
We get exposure in the Google Webmaster console of validation failures
Chrome Extension to highlight an AMP alternative for a page, and show validation errors

SLIDE 33 • Google advocates using their AMP cache: a proxy-based CDN with extra features
Anyone may run their own AMP cache on their own CDN, or publishers can use Google’s for free

SLIDE 34 • AMP documents have a 0.7 second median load time
99% of load times are < 8 seconds
could be better, but
Non-AMP is 22 seconds at the 99th percentile

SLIDE 35 • 8 second timeout fades in the page without JS or fonts if those resources have failed to load
Space reserved for components using explicit width & height to set aspect ratio, plus a layout type
All scripts are async
but the main runtime can still render the page before they load
grey boxes, loading indicators, YouTube poster images are drawn in a reserved space
ZERO scripts block font downloads
Components are allowed to resize after initial render, if:
If below the viewport, fine
If above the viewport, fine, and scroll-top is adjusted so the content in view remains in exactly the same place relative to the viewport
If in the viewport, no resizing allowed unless following a user interaction (so requires a button to be shown)
DOM mutation batching (FastDOM)
How to support unlimited analytics providers but guarantee good performance?
Separate instrumentation from reporting
so AMP runtime reports events like “click”, “scroll = 25%”, etc.

SLIDE 36 • Prerender is used to make Search Carousel pages load instantly
but not for all of each page, as that’s a huge waste of resources
Only what’s in the viewport.
‘With prerendering, you go from fast to instant`
For the carousel when prerendering, only load content which is visible, and only using the minimum network and CPU to give the impression of loaded
Carousel sends a postMessage call to the iframe to say “now you’re visible, load everything for real”
Preconnect API used extensively

SLIDE 37 • You can see that we serve the pages directly
…but that we also make use of Google’s CDN
Google currently presents AMP documents in its Search Carousel
supported on mobile devices only
they’re rolling out slowly across the world
Nothing to stop other organisations setting up a CDN, or presenting AMP documents

SLIDE 38 • Imagine you’re a publisher
What do you do about distributed content?
Do you keep on making new versions of every article in a new format?
Do you still have control?

SLIDE 39 • huge new markets to develop growth
BUT, the gatekeepers decide all the rules
new content formats
Lose control of QA by publishing through a platform
Each new platform requires a lot of work to transform content, QA, adapt CMS, investigate publishing workflow
they keep users on their platforms rather than sending them to publishers’ websites
advertisers can sell directly to users on a social network
they have richer demographic data
why should they still work with the publishers?
Brands lose their identity when presented in this way
Do readers ever know who they’re reading?
Analytics is kept under the platform’s control.
Aggregated (mention Redshift work),
lack of demographic data
In-app browsers (FIA, Twitter moments) don’t share cookies
so can’t track signed-in users
Paywall implementations are slow in coming,
hard for the FT to experiment and assess effectiveness
Totally against the ‘progressive web app’ philosophy - more like an ‘m.’ site (see Andrew’s post)

SLIDE 40 • What does AMP offer that we cannot offer ourselves already?
It’s not offering “how to make a fast website”…
we can do that already.
Instead, it’s a verification that the site will perform well
in terms of:
render speed,
jank,
network usage
mobile UX.

SLIDE 41 • The AMP project freely admits that their results will never be as good as websites which are CLICK hand-tuned for the best performance.

SLIDE 42 •
Time to talk about the Content Performance Policy
Draft spec in the works
main author is Yoav Weiss, who spoke last month
contributions from other experts you will have heard of:
Tim Kadlec, Ilya Grigorik

Bruce has already told us the heartwarming story of the Picture Element.

I like to think that if Bruce is the King of Responsive Images, Yoav is the queen, having implemented the Picture Element in Blink & Webkit

SLIDE 43 • Site authors gain control over the performance of their pages
You would define a policy using dedicated directives in a HTTP header
like no hijacking of scroll events
The browser could then view this as a “promise” that the site adheres to the specified policy
in this case, that it doesn’t hijack any scroll events
If the site then tried to break its promise, the browser would make sure that it cannot
e.g. ignore attempts to cancel the scroll event
An embedder, such as a search engine or a social network app, can then be certain that the “promises” provided by the developers are enforced, and the user experience on the site is guaranteed not to suffer from these anti-patterns.
For example, if Google could be 100% sure that a website had good performance, it could put it in the Search Carousel rather than requiring an AMP page
borrows heavily from the syntax of Content-Security-Policy
CSP defines the Content-Security-Policy HTTP header that allows you to create a whitelist of sources of trusted content
instructs the browser to only execute or render resources from those sources
you can instruct the browser to POST a JSON violation report back to you
so your can monitor a new policy without enforcing it

SLIDE 44 • Three groups of directives:
enforcing performance best practices and opting into the fast path
No synchronous external scripts
nor blocking external stylesheets
Size limited inline scripts and styles
No text-blocking external Web fonts
I.e. no web fonts, or a fallback mechanism in place
No default-overriding events
Scroll and click-hijacking are classic
Document layout must not be dependent on resource download
Avoid content shifting by using intrinsic dimensions to layout before resources are available
No layout thrashing
reads and writes batched

SLIDE 45 • 2nd directive: reducing the consumption of resources
Lazy loading of images
Only download what’s needed
Very long viewports on mobile - why waste bandwidth?
Resource size limits
Limit total allowable download size for a page
Throttling CPU consumption on the main thread
Don’t let iframes hog CPU

SLIDE 46 • 3rd directive: enforcing mobile-friendly user experience
No auto playing video and audio
Debateable - how to allow ‘acceptable’ adverts?
Certain page components are limited to paint only on certain parts of the viewport
Prevent 3rd-party scripts from hijacking the viewport with a modal dialog, for example
No overlays
no popunders

SLIDE 47 • Tools which can help us now:
Intersection Observer (lazy-load when ad/image comes into view)
Passive Event Listeners
listen to scroll events with no option to preventDefault means no blocking necessary
FastDOM
used extensively in the FT web app
At the FT we’re working on a Chrome extension to experiment with the CPP, reporting violations of as many rules as we can detect

SLIDE 48 • In summary:
Internet giants want to improve user experience when consuming news
and keep control over their journey and data
Large platforms have a strangle-hold over publishers
Google pushed back with AMP
open-source,
on and of the web,
prioritising performance over everything
All these solutions are new formats, or variants of HTML,
requiring a fragmented rather than ‘progressive’ approach to content production
But, all provide a consistently good user experience.
AMP technology in particular should feed into browser vendors
Disruption for advertising and analytics in particular.
If all mobile web sites loaded near-instantly
would Facebook, Apple and others feel such a strong need to bring content inside their platforms

SLIDE 49 •

George Crawford

June 16, 2016
Tweet

More Decks by George Crawford

Other Decks in Technology

Transcript

  1. View Slide

  2. @georgeocrawford
    Proxy Wars

    View Slide

  3. @georgeocrawford
    George Crawford
    Developer at the Financial Times
    app.ft.com
    Next version of FT.com
    Distributed Content team
    Live in Bristol
    FT Web App
    GitHub: Financial Times
    About me

    View Slide

  4. @georgeocrawford
    Imagine you were Facebook…

    View Slide

  5. ⚡Imagine you were Facebook…

    View Slide

  6. @georgeocrawford
    How big is the problem?

    View Slide

  7. @georgeocrawford
    over 2 billion people with smartphones
    90% of time on mobile spent in apps
    27 apps a month
    79% of the time in just 5 apps
    All data from INMA: Evaluating Distributed Content in the News Media Ecosystem
    How big is the problem?

    View Slide

  8. @georgeocrawford
    44% use a smartphone to access news
    64% on a desktop computer
    78% publishers get more traffic from mobile
    20% see longer visits on mobile
    How big is the problem?
    All data from INMA: Evaluating Distributed Content in the News Media Ecosystem

    View Slide

  9. @georgeocrawford
    More than 50% of downloaded data was
    from adverts and ‘adblocked content’
    www.boston.com:
    39 seconds on fast 4G
    31 seconds just to load adverts
    Users expect pages to load in <2s
    After 3 seconds, 40% abandon the site
    How big is the problem?
    All data from INMA: Evaluating Distributed Content in the News Media Ecosystem

    View Slide

  10. @georgeocrawford
    44 minutes per day on Facebook
    4 minutes per day in news apps
    1.5 billion active monthly users
    How big is the problem?
    All data from INMA: Evaluating Distributed Content in the News Media Ecosystem

    View Slide

  11. @georgeocrawford
    How big is the problem?
    All data from INMA: Evaluating Distributed Content in the News Media Ecosystem
    WhatsApp
    37
    WeChat
    29
    Viber
    20
    Line
    26
    Facebook
    20

    View Slide

  12. @georgeocrawford
    35% of adults found news on social networks
    38% of referrals from Facebook
    35% of referrals from Google
    How big is the problem?
    All data from INMA: Evaluating Distributed Content in the News Media Ecosystem

    View Slide

  13. @georgeocrawford
    Why are news sites so slow?

    View Slide

  14. @georgeocrawford
    Network hogging
    CPU hogging
    Scroll-jacking
    Render blocking
    Relayout forcing
    Wasted bandwidth
    Huge downloads
    Duplicated tracking
    Event listeners
    Bad iframes
    Lack of HTTPS
    Intrusive adverts
    Why are news sites so slow?

    View Slide

  15. @georgeocrawford
    Distributed Content

    View Slide

  16. @georgeocrawford
    May 2015: Facebook Instant Articles
    September 2015: Apple News
    October 2015: Twitter Moments
    February 2016: Accelerated Mobile Pages
    Distributed Content

    View Slide

  17. @georgeocrawford
    Passes the Toothbrush Test:
    “Is it something you will use once or twice a
    day, and does it make your life better?”
    NY Times: In Silicon Valley, Mergers Must Meet the Toothbrush Test
    Distributed Content

    View Slide

  18. @georgeocrawford
    User Experience
    Performance
    Engagement
    Demographic Ad Targeting
    Keep the user inside the app
    Distributed Content

    View Slide

  19. @georgeocrawford
    Aggregators (Apple News)
    Hosts (Facebook Instant Articles, Twitter Moments)
    Off-platform open standards (AMP)
    Distributed Content

    View Slide

  20. @georgeocrawford
    Apple News

    View Slide

  21. ⚡Apple News

    View Slide

  22. @georgeocrawford
    Facebook Instant Articles

    View Slide

  23. ⚡Facebook Instant Articles

    View Slide

  24. @georgeocrawford
    fancy news aggregators
    custom renderers
    proprietary syndication formats
    in-app experience
    RSS/API driven
    improved performance, but…
    …focus is on immersive UX
    Distributed Content

    View Slide

  25. @georgeocrawford
    ‘Zuckerberg’s Walled Garden of Earthly Delights’
    Imagine you were Google…
    Nieman Lab: Get AMP’d

    View Slide

  26. @georgeocrawford
    Accelerated Mobile Pages

    View Slide

  27. @georgeocrawford
    “Documents that are fast-by-default”
    Open-Source framework
    Web-based, standard HTML page
    Hosted by the publisher
    Optional caching on an AMP CDN
    All about performance
    Smashing Magazine has a nice introduction
    AMP: A new approach to web performance
    Smashing Magazine: Turn Your AMP Up To 11
    Accelerated Mobile Pages

    View Slide

  28. @georgeocrawford
    Forbidden elements:
    frame, frameset, object, applet
    Replaced elements:
    amp-img, amp-video, amp-audio, amp-iframe
    New functionality:
    amp-ad, amp-analytics, amp-pixel, amp-font
    AMP HTML Format
    AMP HTML

    View Slide

  29. @georgeocrawford
    Standard CSS, with some limitations
    Inlined in a single block in the <head><br/>Limited to 50kB<br/>!important forbidden<br/>AMP CSS Format<br/>AMP CSS<br/>

    View Slide

  30. @georgeocrawford
    nodisplay
    fixed
    responsive
    fixed-height
    fill
    container
    AMP Layouts
    AMP Layout

    View Slide

  31. @georgeocrawford
    limited to AMP runtime and AMP extensions
    Don’t trust the users!
    amp-iframe allows inline JS
    Can request a resize
    AMP Iframes and Media
    AMP Scripts

    View Slide

  32. @georgeocrawford
    Resource loading and prioritisation
    Security
    Implements AMP components
    Centralised layout system
    Runtime validator: #development=1
    AMP Technical Overview
    AMP Chrome Extension
    AMP Runtime

    View Slide

  33. @georgeocrawford Google AMP Cache
    AMP Cache
    Proxy-based CDN with extra features
    Anyone can run one
    Google’s is free!
    The document, images & JS all load
    from a single origin, over HTTP2

    View Slide

  34. @georgeocrawford
    Why is AMP so fast?

    View Slide

  35. @georgeocrawford
    AMP Technical Overview
    FastDOM
    Why is AMP so fast?
    8-second timeout to basic HTML
    external resources reserve space
    all scripts are async
    resizing allowed when off-viewport, or on user
    interaction
    DOM Mutation Batching
    Separates analytics instrumentation from reporting

    View Slide

  36. @georgeocrawford YouTube: How AMP achieves its speed
    Why is AMP so fast?
    Google Search Carousel uses prerendering
    “With prerendering, you go from fast to instant”
    Only load visible content
    Minimum use of network and CPU
    Only gives the impression of ‘loaded’
    postMessage to the iframe says ‘now load it for real’
    Preconnect API

    View Slide

  37. ⚡Accelerated Mobile Pages

    View Slide

  38. @georgeocrawford
    Imagine you were a publisher…

    View Slide

  39. @georgeocrawford
    Huge new markets
    Platforms decide the rules
    New content formats
    Fewer visits to the publishers’ sites
    Advertisers can sell directly, with
    richer user data
    Brands lose their identity
    Opaque analytics
    No shared state with browsers
    Primitive/missing paywalls
    Not progressive
    Imagine you were a publisher…
    Andrew Betts: Progressively Less Progressive

    View Slide

  40. @georgeocrawford
    Imagine you could see into the future…

    View Slide

  41. @georgeocrawford YouTube: How AMP achieves its speed
    Imagine you could see into the future…

    View Slide

  42. @georgeocrawford
    Content Performance Policy
    Indiegogo: Implement the picture element in Blink!

    View Slide

  43. @georgeocrawford
    Authors gain control over performance
    Directives are defined with HTTP headers
    Browser vendors and embed platforms can
    make decisions when policies are broken
    Inspired by Content Security Policy
    Report-only mode
    Content Performance Policy
    W3C: Content Security Policy
    WICG: Content Performance Policy

    View Slide

  44. @georgeocrawford
    Content Performance Policy
    WICG: Content Performance Policy
    No synchronous external scripts
    No blocking stylesheets
    Inline scripts and CSS limited in
    size
    No text-blocking external fonts
    (avoid web fonts, or use a
    fallback mechanism like font-
    display: swap)
    No preventDefault in event
    listeners
    Document layout not
    dependent on resource
    downloads
    No layout thrashing

    View Slide

  45. @georgeocrawford
    Content Performance Policy
    Lazy-load images
    Only load resources which will be seen
    Limit size and number of downloaded resources
    Throttle CPU consumption on the main thread
    WICG: Content Performance Policy

    View Slide

  46. @georgeocrawford
    Content Performance Policy
    No auto-play on video and audio
    Page components can only paint to
    allocated areas of the document
    Prevent hijacking of the viewport
    No overlays, modal dialogs
    No popunders
    WICG: Content Performance Policy

    View Slide

  47. @georgeocrawford
    Imagine you could see into the future…
    Intersection Observer
    Passive Event Listeners
    FastDOM
    Chrome Extension (WIP!)
    WICG: Intersection Observer
    WICG: Passive Event Listeners
    GitHub: FastDOM
    GitHub: FTLabs CPP Extension

    View Slide

  48. @georgeocrawford
    Imagine you could see into the future…

    View Slide

  49. Thanks!
    @georgeocrawford

    View Slide