HTTPS is Hard

HTTPS is Hard

Here, Yell’s Head of Web Engineering, Steve Workman, looks back over Yell.com‘s nine-month transition to HTTPS, to raise awareness of the issues with the move in the industry and to make the adoption process easier for other engineering teams.

First presented at Front End London on 31st March 2016, updated for Breaking Borders on June 15th 2016 and updated again for Full Stack Toronto, October 18th 2016

2fdb5b62030270813e22f5e17d16f6b9?s=128

Steve Workman

March 31, 2016
Tweet

Transcript

  1. 2.

    “We’re a business directory, why do we need to be

    secure?” Me, to Dan Applequist, January 2015 @steveworkman HTTPS is Hard #fstoco
  2. 3.

    “Think about what queries your users put through that every

    day, legal counsel, family planning clinics, as well as the regular plumbers and hairdressers. They search for it locally, and that is all personally identifiable. If I were a hacker intercepting this traffic I could work out some pretty interesting stuff about you” Dan Applequist, correcting me, January 2015 @steveworkman HTTPS is Hard #fstoco
  3. 4.

    “Google is pushing hard on this, they made it a

    ranking factor to encourage the big guys to change. If you’re selling this to your boss, that’s what you’ll major on” Dan Applequist, selling it, January 2015 @steveworkman HTTPS is Hard #fstoco
  4. 10.
  5. 11.

    See what breaks q Some internal URLs, including the canonical

    URLs q All adverts q Adobe Analytics q The entire reviews section @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  6. 12.

    Fixing things ü Some internal URLs, including the canonical URLs

    q All adverts q Adobe Analytics q The entire reviews section @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  7. 13.

    Securing Adverts: • AOL/Yahoo’s Advertising network • Can easily serve

    their scripts over HTTPS • Adverts will then be served over HTTPS • Or at least they should be • You can be your own worst enemy here @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  8. 14.

    IAB are changing their ways • 80% of the industry

    supports HTTPS • In October 2015, they admitted they messed up • http://www.iab.com/news/lean/ • Light • Encrypted • Ad Choice Supported • Non-Invasive @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  9. 15.

    @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May

    Jun Jul Aug Sep Oct Nov Dec Advertising Adtech Ad Sales team Engineering Security Operations Product
  10. 16.

    Fixing things ü Some internal URLs, including the canonical URLs

    ü All adverts q Adobe Analytics q The entire reviews section @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  11. 17.

    Third Party 2: Adobe Analytics • Checked our implementation –

    no joy • Contact Adobe • Enabled first-party domains • Supply certificates • Very cautiously updated to the latest version @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  12. 18.

    @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May

    Jun Jul Aug Sep Oct Nov Dec Advertising Adtech Ad Sales team Engineering Security Operations Product Analytics Team Adobe
  13. 19.

    Fixing things ü Some internal URLs, including the canonical URLs

    ü All adverts ü Adobe Analytics q The entire reviews section @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  14. 20.

    “What’s in that shadowy place over there?” @steveworkman HTTPS is

    Hard #fstoco That’s the reviews system, you must never go there
  15. 21.

    Fixing things ü Some internal URLs, including the canonical URLs

    ü All adverts ü Adobe Analytics ü The entire reviews section @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  16. 23.

    Why EV Certificates? • It’s a mark of trust in

    the organisation • It’s not much more expensive than a regular certificate • It’s the only type of certificate that turns the padlock green in Edge • Important for the perception of security @steveworkman HTTPS is Hard #fstoco
  17. 24.

    EV certification isn’t hard, it takes time • More levels

    of scrutiny and manual steps takes the time • Had to update our domain records due to corporate name changes • Took a total of 4 weeks @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  18. 25.

    @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May

    Jun Jul Aug Sep Oct Nov Dec Advertising Adtech Ad Sales team Engineering Security Operations Product Analytics Team Adobe Legal Companies House
  19. 26.

    Other third parties • Anti-scraping tool • Costs money to

    do with EV cert for a private IP • Video hosting CDN • Costs money – host didn’t support SNI • Cross-region agreement means this is still in progress @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  20. 27.

    The Business Case • Capital Expenditure (spending money) isn’t easy

    for many developers • Lots will have never written a business case before • Depending on your organisation, this may not be trivial and can take time and effort to push it through @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  21. 28.

    Pre-live performance concerns • Is TLS Fast Yet? • Yes,

    it is: www.Istlsfastyet.com • Monitor our performance with RUM tools • Terminate the connection at load balancer (closer to user) • Ensure it is up to date @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  22. 29.

    @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May

    Jun Jul Aug Sep Oct Nov Dec Advertising Adtech Ad Sales team Engineering Security Operations Product Analytics Team Adobe Legal Companies House Anti-scrape CDN CDO CEO
  23. 30.

    The Big Day @steveworkman HTTPS is Hard #fstoco • Sitemaps

    (~10M links) • Robots.txt • Google Search Console • 301 redirects for HTTP traffic at the network edge (the flip) Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  24. 31.
  25. 32.

    Java silently stopped sending requests • Java only has some

    standard Root CA certificates by default • Without these, requests over HTTPS will fail silently • Upgrading Java wholesale is full of risk, simpler to install missing CAs • Pro tip: Always have an internal non-HTTPS route @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  26. 33.

    What does HTTPS do to your Google search ranking? @steveworkman

    HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  27. 34.

    @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May

    Jun Jul Aug Sep Oct Nov Dec • HTTPS is 1 factor out of 200+ and is a “tie-break” factor • It correlates +0.04 to ranking - not strong • https://moz.com/search-ranking-factors/correlations
  28. 35.

    Search ranking can be affected • Wired chose to use

    302 redirects initially, causing drops in search ranking • Once they switched to 301 redirects, ranking losses stopped @steveworkman HTTPS is Hard #fstoco Source: https://www.wired.com/2016/09/wired-completely-encrypted/ Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  29. 36.

    Google re-indexing took over 6 months @steveworkman HTTPS is Hard

    #fstoco 21/06/2015 21/07/2015 21/08/2015 21/09/2015 21/10/2015 21/11/2015 21/12/2015 21/01/2016 % of pages indexed on Google %HTTP % HTTPS Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  30. 37.

    TLS Performance @steveworkman HTTPS is Hard #fstoco Jan Feb Mar

    Apr May Jun Jul Aug Sep Oct Nov Dec Desktop devices Mobile devices
  31. 38.

    HTTPS is Fast, but it is not Free Un-tuned HTTPS

    will add 100-200ms to your first render time, and more than that at the extremes of connectivity @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  32. 39.

    What’s wrong here? • Anti-scrape server isn’t as optimised as

    it could be – Window Scaling, OCSP stapling, TLS False Start all off • Together they add 2 round-trips to each handshake • So, the impact should theoretically be 30-60ms, not 100- 200ms @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Anti-scrape Origin
  33. 40.

    “I’ve stopped receiving traffic from your site” @steveworkman HTTPS is

    Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  34. 41.

    HTTP Referrer • 99% of our customer’s websites are served

    over HTTP, and of that 1%, a quarter of those are Facebook pages. @steveworkman HTTPS is Hard #fstoco From / To HTTP HTTPS HTTP Pass Pass HTTPS Do not pass Pass Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  35. 43.

    Content Security Policy Level 2 @steveworkman HTTPS is Hard #fstoco

    Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  36. 44.

    We chose to educate our customers instead @steveworkman HTTPS is

    Hard #fstoco Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
  37. 45.

    @steveworkman HTTPS is Hard #fstoco Jan Feb Mar Apr May

    Jun Jul Aug Sep Oct Nov Dec Advertising Adtech Ad Sales team Engineering Security Operations Product Analytics Team Adobe Legal Companies House Anti-scrape CDN CDO CEO Sales Marketing Customer Services Telesales
  38. 46.

    HTTPS is not a technology problem, it is a people

    problem, and that problem is incentives @steveworkman HTTPS is Hard #fstoco
  39. 47.

    Good News Everyone! The internet has listened and is changing

    for the better @steveworkman HTTPS is Hard #fstoco
  40. 48.

    Problem Certificates aren’t free There’s a performance impact CDNs should

    offer TLS for free Solution HTTP/2 Most do for DV certificates @steveworkman HTTPS is Hard #fstoco
  41. 49.

    The migration cost is too high Without HTTPS you can’t

    have @steveworkman HTTPS is Hard #fstoco Privileged Features Geolocation Webcam Microphone Notifications Device motion & orientation Progressive Web Apps Service Worker AMP
  42. 51.

    What’s next for Yell? • Work with third-party providers to

    improve TCP performance • HTTP/2 • HTTP Strict Transport Security (HSTS) • Create a Content Security Policy (CSP) • Ensure server Cookies set with httpsOnly flag • Complete the transition and update our CDN @steveworkman HTTPS is Hard #fstoco