HTTPS is Hard - Speaker Deck

Slide 1

Slide 1 text

HTTPS is Hard Steve Workman

Slide 2

Slide 2 text

“We’re a business directory, why do we need to be secure?” Me, to Dan Applequist, January 2015 @steveworkman HTTPS is Hard #fstoco

Slide 3

Slide 3 text

“Think about what queries your users put through that every day, legal counsel, family planning clinics, as well as the regular plumbers and hairdressers. They search for it locally, and that is all personally identifiable. If I were a hacker intercepting this traffic I could work out some pretty interesting stuff about you” Dan Applequist, correcting me, January 2015 @steveworkman HTTPS is Hard #fstoco

Slide 4

Slide 4 text

“Google is pushing hard on this, they made it a ranking factor to encourage the big guys to change. If you’re selling this to your boss, that’s what you’ll major on” Dan Applequist, selling it, January 2015 @steveworkman HTTPS is Hard #fstoco

Slide 5

Slide 5 text

@steveworkman HTTPS is Hard #fstoco

Slide 6

Slide 6 text

@steveworkman HTTPS is Hard #fstoco Engineering Security Operations Product Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 7

Slide 7 text

@steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 8

Slide 8 text

@steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 9

Slide 9 text

@steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 10

Slide 10 text

@steveworkman HTTPS is Hard #fstoco http ://www.yell.com S Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 11

Slide 11 text

See what breaks q Some internal URLs, including the canonical URLs q All adverts q Adobe Analytics q The entire reviews section @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 12

Slide 12 text

Fixing things ü Some internal URLs, including the canonical URLs q All adverts q Adobe Analytics q The entire reviews section @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 13

Slide 13 text

Securing Adverts: • AOL/Yahoo’s Advertising network • Can easily serve their scripts over HTTPS • Adverts will then be served over HTTPS • Or at least they should be • You can be your own worst enemy here @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 14

Slide 14 text

IAB are changing their ways • 80% of the industry supports HTTPS • In October 2015, they admitted they messed up • http://www.iab.com/news/lean/ • Light • Encrypted • Ad Choice Supported • Non-Invasive @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 15

Slide 15 text

@steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Advertising Adtech Ad Sales team Engineering Security Operations Product

Slide 16

Slide 16 text

Fixing things ü Some internal URLs, including the canonical URLs ü All adverts q Adobe Analytics q The entire reviews section @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 17

Slide 17 text

Third Party 2: Adobe Analytics • Checked our implementation – no joy • Contact Adobe • Enabled first-party domains • Supply certificates • Very cautiously updated to the latest version @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 18

Slide 18 text

@steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Advertising Adtech Ad Sales team Engineering Security Operations Product Analytics Team Adobe

Slide 19

Slide 19 text

Fixing things ü Some internal URLs, including the canonical URLs ü All adverts ü Adobe Analytics q The entire reviews section @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 20

Slide 20 text

“What’s in that shadowy place over there?” @steveworkman HTTPS is Hard #fstoco That’s the reviews system, you must never go there

Slide 21

Slide 21 text

Fixing things ü Some internal URLs, including the canonical URLs ü All adverts ü Adobe Analytics ü The entire reviews section @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 22

Slide 22 text

Acquire Certificates @steveworkman HTTPS is Hard #fstoco • Self-signed • Domain Validated • Extended Validation

Slide 23

Slide 23 text

Why EV Certificates? • It’s a mark of trust in the organisation • It’s not much more expensive than a regular certificate • It’s the only type of certificate that turns the padlock green in Edge • Important for the perception of security @steveworkman HTTPS is Hard #fstoco

Slide 24

Slide 24 text

EV certification isn’t hard, it takes time • More levels of scrutiny and manual steps takes the time • Had to update our domain records due to corporate name changes • Took a total of 4 weeks @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 25

Slide 25 text

@steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Advertising Adtech Ad Sales team Engineering Security Operations Product Analytics Team Adobe Legal Companies House

Slide 26

Slide 26 text

Other third parties • Anti-scraping tool • Costs money to do with EV cert for a private IP • Video hosting CDN • Costs money – host didn’t support SNI • Cross-region agreement means this is still in progress @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 27

Slide 27 text

The Business Case • Capital Expenditure (spending money) isn’t easy for many developers • Lots will have never written a business case before • Depending on your organisation, this may not be trivial and can take time and effort to push it through @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 28

Slide 28 text

Pre-live performance concerns • Is TLS Fast Yet? • Yes, it is: www.Istlsfastyet.com • Monitor our performance with RUM tools • Terminate the connection at load balancer (closer to user) • Ensure it is up to date @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 29

Slide 29 text

@steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Advertising Adtech Ad Sales team Engineering Security Operations Product Analytics Team Adobe Legal Companies House Anti-scrape CDN CDO CEO

Slide 30

Slide 30 text

The Big Day @steveworkman HTTPS is Hard #fstoco • Sitemaps (~10M links) • Robots.txt • Google Search Console • 301 redirects for HTTP traffic at the network edge (the flip) Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 31

Slide 31 text

The aftermath @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 32

Slide 32 text

Java silently stopped sending requests • Java only has some standard Root CA certificates by default • Without these, requests over HTTPS will fail silently • Upgrading Java wholesale is full of risk, simpler to install missing CAs • Pro tip: Always have an internal non-HTTPS route @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 33

Slide 33 text

What does HTTPS do to your Google search ranking? @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 34

Slide 34 text

@steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec • HTTPS is 1 factor out of 200+ and is a “tie-break” factor • It correlates +0.04 to ranking - not strong • https://moz.com/search-ranking-factors/correlations

Slide 35

Slide 35 text

Search ranking can be affected • Wired chose to use 302 redirects initially, causing drops in search ranking • Once they switched to 301 redirects, ranking losses stopped @steveworkman HTTPS is Hard #fstoco Source: https://www.wired.com/2016/09/wired-completely-encrypted/ Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 36

Slide 36 text

Google re-indexing took over 6 months @steveworkman HTTPS is Hard #fstoco 21/06/2015 21/07/2015 21/08/2015 21/09/2015 21/10/2015 21/11/2015 21/12/2015 21/01/2016 % of pages indexed on Google %HTTP % HTTPS Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 37

Slide 37 text

TLS Performance @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Desktop devices Mobile devices

Slide 38

Slide 38 text

HTTPS is Fast, but it is not Free Un-tuned HTTPS will add 100-200ms to your first render time, and more than that at the extremes of connectivity @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 39

Slide 39 text

What’s wrong here? • Anti-scrape server isn’t as optimised as it could be – Window Scaling, OCSP stapling, TLS False Start all off • Together they add 2 round-trips to each handshake • So, the impact should theoretically be 30-60ms, not 100- 200ms @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Anti-scrape Origin

Slide 40

Slide 40 text

“I’ve stopped receiving traffic from your site” @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 41

Slide 41 text

HTTP Referrer • 99% of our customer’s websites are served over HTTP, and of that 1%, a quarter of those are Facebook pages. @steveworkman HTTPS is Hard #fstoco From / To HTTP HTTPS HTTP Pass Pass HTTPS Do not pass Pass Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 42

Slide 42 text

Referrer Policy @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 43

Slide 43 text

Content Security Policy Level 2 @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 44

Slide 44 text

We chose to educate our customers instead @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Slide 45

Slide 45 text

Slide 46

Slide 46 text

HTTPS is not a technology problem, it is a people problem, and that problem is incentives @steveworkman HTTPS is Hard #fstoco

Slide 47

Slide 47 text

Good News Everyone! The internet has listened and is changing for the better @steveworkman HTTPS is Hard #fstoco

Slide 48

Slide 48 text

Problem Certificates aren’t free There’s a performance impact CDNs should offer TLS for free Solution HTTP/2 Most do for DV certificates @steveworkman HTTPS is Hard #fstoco

Slide 49

Slide 49 text

The migration cost is too high Without HTTPS you can’t have @steveworkman HTTPS is Hard #fstoco Privileged Features Geolocation Webcam Microphone Notifications Device motion & orientation Progressive Web Apps Service Worker AMP

Slide 50

Slide 50 text

@steveworkman HTTPS is Hard #fstoco

Slide 51

Slide 51 text

What’s next for Yell? • Work with third-party providers to improve TCP performance • HTTP/2 • HTTP Strict Transport Security (HSTS) • Create a Content Security Policy (CSP) • Ensure server Cookies set with httpsOnly flag • Complete the transition and update our CDN @steveworkman HTTPS is Hard #fstoco

Slide 52

Slide 52 text

@steveworkman HTTPS is Hard #fstoco https://observatory.mozilla.org

Slide 53

Slide 53 text

Thank you Twitter: @steveworkman Slides: https://speakerdeck.com/steveworkman/https-is-hard Epic Blog post: https://blog.yell.com/2016/03/https-is-hard/ @steveworkman HTTPS is Hard #fstoco