Proxy Wars: The Battle For Your Content

Proxy Wars: The Battle For Your Content

We’re seeing an important shift in the world of web publishing, and it’s driven in part by performance. As web pages become heavier and more complex, and as the public demand a broader spectrum of sources for their news, internet giants such as Facebook, Google and Apple have seen an opportunity to step in. In return for the promise of an immersive, tailor-made and lightening fast web experience, they offer to promote, curate and host your content for you. But who’s winning here, and at what cost?

First presented at Bristol Web Performance on 16/6/2016: http://www.meetup.com/bristolwebperf/events/230315192/.

George is a Senior Developer at the Financial Times, and currently engaged with the company’s drive to increase ‘reach’ into new readership markets. He’ll describe how they went about implementing offerings for Apple News, Google AMP and Facebook Instant Articles, the technical issues they presented, and how they’re performing against the early promises.

-------------------

Speaker Notes

-------------------

SLIDE 1

SLIDE 2
Hello!
I’m going to talk to you about the battle for your content, known to us in this room as the Proxy Wars

SLIDE 3

SLIDE 4
Thought experiment:
Imagine you were Facebook, and every day, you watched your billions of users doing this:

SLIDE 5

Very slow to load
not responsive
user leaves the Facebook ecosystem

From FB’s point of view, there are two big problems here:

terrible User Experience
they’re losing users to the publisher’s site

SLIDE 6

SLIDE 7
Over 2 billion people with smartphones around the world
90% of time on mobile is spent in apps
mobile apps mainly for:
social networking and messaging, plus entertainment and games
An average U.S. user uses 27 apps a month
but spends 79% of her time on just 5 apps
These statistics have a US bias, but give an idea of the problem for mobile news consumption

SLIDE 8
But content consumption on mobile is very different to desktop computers
In a given week:
44% of U.S. adults used a smartphone to access news
64% who did so on a desktop computer
78% top US news publishers get more traffic from mobile than desktop computers
But only 20% see mobile visitors spending more time per visit than desktop
mobile users are less engaged, and dropping off

SLIDE 9
Part of the problem may be the bad experiences that many users have when accessing mobile news sites
in recent research:
more than half of all data came from ads and other content filtered by adblockers
slowest home page in the study was Boston.com, averaging 39 seconds to load on a very fast 4G connection, including 31 seconds just to load ads.
the average user expects pages to load in two seconds or less. After three seconds, up to 40% users abandon the site.

SLIDE 10
Let’s look at the rise of social media.
An average U.S. consumer spends 44 minutes per day on Facebook alone.
News apps? All they get is 4 minutes a day
If Facebook were a country, it would be larger than China or India with its 1.5 billion active monthly users worldwide.

SLIDE 11
Social networks and messaging apps are the most engaging apps.
WhatsApp users connect 37 times a day
WeChat: 29
Japanese Line, 26
Israeli Viber, 20
Facebook: 20

SLIDE 12
Social networks and messaging apps have become top sources of news for their users, and top sources of traffic to news publishers.
Of U.S. adults, 35% came across news stories on Facebook and via other social networks last year
From July last year, Facebook referred more traffic to the top 400 publishers than Google:
38% of all referrals are from Facebook
35% of all referrals from Google

SLIDE 13

SLIDE 14
So what’s the big problem with news sites?
All too familiar to you. Many problems can be attributed to third-party scripts:
network hogging, means that content can’t be prioritised over other assets
CPU hogging
scroll-jacking,
render blocking,
forcing relayout
unused resources waste bandwidth
large download size
big problem for people on cheaper contracts, roaming, third-world
multiple duplicated tracking libraries, each with their own event listeners:
Stakeholders at your company probably don’t even use them!
misbehaving iframes
lack of HTTPS support
Intrusive adverts

SLIDE 15

SLIDE 16
May 2015: Facebook Instant Articles launched
April this year, opened up to all publishers worldwide
September 2015, Apple’s News app launched
pre-installed on iOS
more than 100 publishers worldwide.
October 2015, Twitter launched Moments
a magazine-like story format
curated streams of Tweets with photos, videos, links to articles
February 2016, Google publicly launched Accelerated Mobile Pages
open standard intended to speed up the mobile Web
Google only one of the stakeholders
There are plenty of other platforms emerging too

SLIDE 17
Why do these internet giants line up at publishers’ doors and ask for news?
As a category of content, ’The News’ passes the famous Silicon Valley “toothbrush test.” CLICK

“When deciding whether Google should spend millions or even billions of dollars in acquiring a new company, its chief executive, Larry Page, asks whether the acquisition passes the toothbrush test: Is it something you will use once or twice a day, and does it make your life better?”

SLIDE 18
Facebook and other Social Media giants want to make the experience of consuming content online more seamless.
They realise that, if they can offer the same content locally, they can:
improve the user experience
improve performance
improve engagement
sell better targeted adverts with all their demographic data
keep the user inside their app

SLIDE 19
Three main types of Distributed Content:
On-platform Aggregators
glorified RSS
Apple News,
On-platform hosts
host the editorial content to allow their users to interact with it
Facebook Instant Articles
Twitter Moments
Off-platform open standard
Accelerated Mobile Pages

SLIDE 20

SLIDE 21
Aggregation of RSS feeds from publishers
Captive audience of millions of iOS users
Pre-installed app
Markup in Apple News Format:
proprietary, complex and buggy layout as you’d expect from a new markup language
Opaque analytics
aggregated
can’t track visits which start on Apple News
Just get a generic ‘Apple News’ referrer
paywall is coming with a 30% cut, but we can’t properly engage with the users

SLIDE 22

SLIDE 23
Demo
Our Social media editor said:
“so, why can’t we just make our website like this? Like, um, fast?”
Supported in Native apps for iOS and Android
Hijacks URL clicks
Lightening bolt means that the publisher has made an Instant Article version available
Loads in-app, near instantly, rather than loading the URL
No active promotion from FB, but because they increase engagement (sharing, commenting, liking), they appear more often in News Feeds
Proprietary HTML format
H3 doesn’t exist!
Text outside

is ignored
iframes with inline content!
other magic!
CSS is removed, instead there’s a very basic style editor
Publish via RSS or push to an API

SLIDE 24
Apple News and Instant Articles are similar:
Essentially, fancy news aggregators
with custom renderers
built on top of proprietary syndication formats
in-app experiences:
neither allows users to view outside the native apps
even during our QA process!
either RSS or API driven:
need to update your CMS to produce RSS or push to API
Requires you to push content updates too - important for corrections and legal
both keen to improve performance, but…
they are focussed on making articles look and feel modern.
Slick & smooth interactions,
embracing native hardware acceleration to immerse user.

SLIDE 25
Imagine you’re Google, and you observe Apple and Facebook
negotiating deals with publishers
setting up walled gardens of news content

What do you do?

Google’s incentive here is obvious:
It makes its money on advertising
the vast majority of that advertising is on the open web
If Facebook provides a much better mobile experience than the open web does
those advertising opportunities, and that user data,
CLICK disappears into Zuckerberg’s Walled Garden of Earthly Delights, as Nieman Lab called it

SLIDE 26
AMP is more to do with changing the mobile web than building a new platform or app
AMP isn’t a business partnership
like Instant Articles or Apple News are
tries to do something more significant: change the way that the web is built
killing off some technologies
promoting others

SLIDE 27
“we want to make it easier to create documents that are fast-by-default”
Open-source
Web-based (works equally in system browser, in-app browsers, webviews)
Complete HTML documents which the publisher hosts.
Can optionally be cached on CDN, where documents are:
validated
parsed
partially rendered by the AMP runtime.
far more about performance than about aesthetics or interaction patterns
Smashing Magazine nice intro to AMP

SLIDE 28
Some elements forbidden:
Frames, embeds, objects, applets all forbidden
Some new elements to replace forbidden elements
amp-img: optimised for viewport size, scroll position, connectivity
amp-video & amp-audio: lazy-load based on viewport position
amp-iframe: enforce security and performance by sandboxing & controlling where iframes may appear
And to introduce new functionality:
amp-ad: ad loading managed by AMP in the same way as videos and images, and no JS can run in parent document to cause reflow and unnecessary layout calculations
amp-analytics and amp-pixel: centralised push for all metrics with unlimited consumers
amp-font: control timeouts and fallbacks
Others like carousel, lightbox, fit-text, install serviceworker, social media plugins
These are genuine Custom Elements
which collaborate with the AMP runtime
to prioritise resource loading
and enforce security

SLIDE 29
designed specifically to work well across mobile devices, rather than fully responsive
Styled with standard CSS to a great extent
Some limitations:
All CSS is inline
Limited to 50K
!important is banned to ensure that AMP can control element sizing

SLIDE 30
Layout:
The runtime must be able to infer the size of all externally loaded resources before they are actually loaded
so that a final layout can be calculated as quickly as possible.
All externally loaded resources must have width, height and one of the supported layout properties
Responsive layout will reserve a space relative to the aspect-ratio defined for the element

SLIDE 31
JS limited to AMP runtime and AMP extensions
encapsulated interactive functionality
so publishers don’t harm performance by writing their own
Amp-iframe allows JS
with limited communication to parent document
can request a resize

SLIDE 32
manages resource loading and prioritisation
security (Fonts only over HTTPS), etc.
implements AMP components,
Centralised system for layout
optionally, includes a runtime validator for AMP HTML by adding a hash to the URL: #development=1
We get exposure in the Google Webmaster console of validation failures
Chrome Extension to highlight an AMP alternative for a page, and show validation errors

SLIDE 33
Google advocates using their AMP cache: a proxy-based CDN with extra features
Anyone may run their own AMP cache on their own CDN, or publishers can use Google’s for free

SLIDE 34
AMP documents have a 0.7 second median load time
99% of load times are is ignored
iframes with inline content!
other magic!
CSS is removed, instead there’s a very basic style editor
Publish via RSS or push to an API

SLIDE 24 • Apple News and Instant Articles are similar:
Essentially, fancy news aggregators
with custom renderers
built on top of proprietary syndication formats
in-app experiences:
neither allows users to view outside the native apps
even during our QA process!
either RSS or API driven:
need to update your CMS to produce RSS or push to API
Requires you to push content updates too - important for corrections and legal
both keen to improve performance, but…
they are focussed on making articles look and feel modern.
Slick & smooth interactions,
embracing native hardware acceleration to immerse user.

SLIDE 25 • Imagine you’re Google, and you observe Apple and Facebook
negotiating deals with publishers
setting up walled gardens of news content

What do you do?

Google’s incentive here is obvious:
It makes its money on advertising
the vast majority of that advertising is on the open web
If Facebook provides a much better mobile experience than the open web does
those advertising opportunities, and that user data,
CLICK disappears into Zuckerberg’s Walled Garden of Earthly Delights, as Nieman Lab called it

SLIDE 26 • AMP is more to do with changing the mobile web than building a new platform or app
AMP isn’t a business partnership
like Instant Articles or Apple News are
tries to do something more significant: change the way that the web is built
killing off some technologies
promoting others

SLIDE 27 • “we want to make it easier to create documents that are fast-by-default”
Open-source
Web-based (works equally in system browser, in-app browsers, webviews)
Complete HTML documents which the publisher hosts.
Can optionally be cached on CDN, where documents are:
validated
parsed
partially rendered by the AMP runtime.
far more about performance than about aesthetics or interaction patterns
Smashing Magazine nice intro to AMP

SLIDE 28 • Some elements forbidden:
Frames, embeds, objects, applets all forbidden
Some new elements to replace forbidden elements
amp-img: optimised for viewport size, scroll position, connectivity
amp-video & amp-audio: lazy-load based on viewport position
amp-iframe: enforce security and performance by sandboxing & controlling where iframes may appear
And to introduce new functionality:
amp-ad: ad loading managed by AMP in the same way as videos and images, and no JS can run in parent document to cause reflow and unnecessary layout calculations
amp-analytics and amp-pixel: centralised push for all metrics with unlimited consumers
amp-font: control timeouts and fallbacks
Others like carousel, lightbox, fit-text, install serviceworker, social media plugins
These are genuine Custom Elements
which collaborate with the AMP runtime
to prioritise resource loading
and enforce security

SLIDE 29 • designed specifically to work well across mobile devices, rather than fully responsive
Styled with standard CSS to a great extent
Some limitations:
All CSS is inline
Limited to 50K
!important is banned to ensure that AMP can control element sizing

SLIDE 30 • Layout:
The runtime must be able to infer the size of all externally loaded resources before they are actually loaded
so that a final layout can be calculated as quickly as possible.
All externally loaded resources must have width, height and one of the supported layout properties
Responsive layout will reserve a space relative to the aspect-ratio defined for the element

SLIDE 31 • JS limited to AMP runtime and AMP extensions
encapsulated interactive functionality
so publishers don’t harm performance by writing their own
Amp-iframe allows JS
with limited communication to parent document
can request a resize

SLIDE 32 • manages resource loading and prioritisation
security (Fonts only over HTTPS), etc.
implements AMP components,
Centralised system for layout
optionally, includes a runtime validator for AMP HTML by adding a hash to the URL: #development=1
We get exposure in the Google Webmaster console of validation failures
Chrome Extension to highlight an AMP alternative for a page, and show validation errors

SLIDE 33 • Google advocates using their AMP cache: a proxy-based CDN with extra features
Anyone may run their own AMP cache on their own CDN, or publishers can use Google’s for free

SLIDE 34 • AMP documents have a 0.7 second median load time
99% of load times are

Ac3c5e78511a155771294f5a4a7cee0c?s=128

George Crawford

June 16, 2016
Tweet