Scaling a culture of performance

In a consistently moving organization, prioritizing performance can sometimes be a challenge.

There is always something to do, that new feature that needs to be released by the end of the month, that bug that needs to be fixed before the end of the day, or that production issue that needs to be addressed right away.

How do we fit in performance related work in all of this?

Over the past year, Priceline has worked to create a renewed culture of performance. We will be exploring how we approached implementing a performance mindset within our teams introducing key learnings, challenges and takeaways.

We will also be talking about our journey using modern web performance tools and metrics and how they are helping us improve not only the site speed but also the business and the user experience.

Transcript

1. Scaling a culture of performance NYC Web Performance meetup - Jan 29th, 2019 Malek Hakim malek.hakim@priceline.com
2. Priceline?
3. We are a travel site, since 1997.
4. In 2018, we saved people +$1Billion on travel.
5. #PerfMatters
6. +50 ms → -20% traffic @ Google +100 ms → -1% sales @ Amazon More?
7. Speed ranks #1 in the UX hierarchy Speed Matters - Vol 3 The speed it takes to load the page. 75% 66% 61% 58% 24% How easy it is to find what I’m looking for. How well the site fits my screen. How easy the site to use. How attractive the site looks.
8. Culture is Invisible
9. 1. Communication: Do we talk about it? 2. Metrics: How is it being evaluated? 3. Tools: Can we measure it effectively? 4. Workflow: Is it part of our job? Culture is invisible
10. Performance is a process.
11. Your site is changing.
12. You will be getting more mobile users % of mobile page views by country, mPulse 2018
13. Users are using technology that you are probably not testing Load time by device type (Android), mPulse July 2017-August 2018
14. Set performance quality standards from the start.
15. Performance culture drives innovation.
16. • Performance is a process. • Your site always need to be performant. • The world around your site changes and will definitely affect performance. • Set quality standards from the start. • A culture of performance drives innovation. Why does a culture of performance matter?
17. Performance culture change is a journey Past: This could be faster. Now: I can make this faster.
18. Regression Improvement Web performance cycle
19. Performance Time
20. New Feature Clean up Upgrade Performance Time A great culture tries to reduce the time to close the cycle
21. Organizational alignment Measurement Technical Approach How do we create an effective performance culture? Buy-in Support Analysis Visibility Alerting Engineering
22. Organizational Alignment 1
23. 46% of organizations expressed obstacles in aligning internal teams around common technologies. State of Kubernetes 2018 by Heptio
24. In the era of microservices, teams are autonomous, easy to get siloed Hotel UX team Hotel Services team A/B platform team
25. Find Allies.
26. Culture starts as a movement.
27. Solving alignment by creating tribes and guilds.
28. The Performance Ambassadors team provides a cross-functional forum to surface and discuss challenges and opportunities to further drive overall culture of performance and quality. Performance Ambassador group UX Services Performance ambassador group
29. Goal: Make performance contagious
30. Case study: Internal Common Components
31. Teams agree on a common API, workflow..
32. Build it as a community
33. Design system bundle size Optimize together
34. People can optimize only what they understand well
35. Teams will go in different places And you should not block them. Don’t panic!
36. What about business?
37. Example of slides shown in a business review meeting
38. Performance is not a feature, it’s part of software DNA
39. However, sell it as a feature if you need to
40. Measurement 2
41. Performance X Experimentation
42. Looking for the right metric.
43. Choose what works for you for your user • Use your analytics tool to look where your users are • Look up what are most of the visited pages, then go setup measurement. • Organize the user critical path to conversion
44. Metrics Fatigue Various performance talk slides
45. User experience is a story, so is performance
46. Shared metrics Navigation starts TTFB First Paint Time To First Interactive Visually Complete Time To Interactive Ready to use metrics First impression metrics CORE Server time
47. Define a common language that everyone understands
48. Encourage teams to use custom metrics Navigation starts Time To Search Form Visible CUSTOM Use custom User Timing metric for more granular examples
49. after-page-load experience Not only loading is important
50. Detect bad experiences for real users https://developers.google.com/web/fundamentals/performance/user-centric-performance-metrics#tracking_long_tasks • Track long tasks • Track input latency • Report client API call latency Vanilla Js input latency
51. RUM or Synthetic? RUM Synthetic What it does Gather metrics from real user experiences Emulated users in a controlled environment Environment Field metrics Lab metrics Reporters Real users Technology Uses modern browsers APIs and custom code Uses browser APIs + Video Capture Purpose General improvements and outliers exploration Benchmarking and monitoring
52. RUM or Synthetic? They complement each other RUM Synthetic Alerting Browser Diversity Object level Visibility to all pages
53. Use RUM to benchmark internally Use Synthetic to benchmark externally
54. An A/B test got turned on affecting TTI Annotate Performance trends with notable events
55. Measurement challenges: Visually Ready How is this being measured? • Using different techniques leads to different results • Accurate measurement is limited to image elements
56. Measurement challenges: Interactivity: what’s right for me? Metric Tool Description First Interactive WebpageTest (Synthetic) This is the earliest point after the main content has rendered where there is at least a 5-second window where the page is responsive First CPU Idle (previously First Interactive) Lighthouse (Synthetic) It is the first moment when a website is minimally interactive, source First Input Delay Lighthouse (Synthetic) First Input Delay is the duration between the first meaningful event’s hardware timestamp, and when the event started being processed Estimated Input Latency Boomerang (continuity plugin) For the first interaction on the page, how responsive was it?
57. Measurement challenges: Interactivity: what’s right for me? Metric Tool Description Time To Consistently Interactive WebpageTest (Synthetic) This is the point where the page can be expected to always be responsive to input. Time To Interactive Lighthouse (Synthetic) The load is finished, the main thread work is done. • Network to be quiet • Long tasks to be over • Using LongTasks API Time To Interactive Boomerang (continuity plugin) First CPU Idle after 500ms without considering the network to be idle domInteractive Navigation Timing (Level 2) A time value equal to the time immediately before the user agent sets the current document readiness of the current document to "interactive"
58. Your site is always delivering good and bad experience at the same time Long tail People in elevator, subway.. Okay speed 0-2 seconds 2-5 seconds <5 seconds Not bad speed Average? Median? p90? p95? p99?
59. Technical Approach 3
60. Start small 1. Start with low hanging fruits 2. Then apply known advanced techniques 3. Then only think about architectural changes Low g f u s Ad a c c iq Ar it ra h s Implementation speed
61. Optimizing the logo svg after a redesign • ~10KB reduced across all apps • Render start by 19% • TTFB by 30% • Hundreds in bookings increased a day AFTER BEFORE
62. Low hanging fruits: - Clean up unused code - Reduce JS bundle size - Change resource loading priorities - Load critical assets first - Call your API very early during page load - Turn on optimization flags - Enable gzip
63. Advanced Techniques Techniques known in the industry for the technologies we use • Code Splitting • Smart Prefetching of API calls • Intelligent Caching techniques
64. Educate
65. Document
66. Celebrate Even by a Slack post
67. Some examples of what’s next for us? • Better integration of performance data with business data. • Make performance budgets part of the developer workflow (CI/CD, deployment gates..) • Improve our internal A/B testing platform to support canary releases.
68. Remember! start small, measure and show the impact
69. #PerfCultureMatters
70. That’s it! Questions? @hakimelekk
71. References • Speed Matters Vol 3 - Mustafa Kurtuldu & Lionel Mora • WPO Stats - Tammy Everts, Tim Kadlec • Real User Measurement Insights - Paul Calvano • Building a performance engineering team - Kwame Thomison • Effective Performance Engineering - Todd DeCapua, Shane Evans • Time To Interactive – Measuring more of the user experience - Pat Meenan • Image Custom Metric - Steve Souders • Spotify Squad framework - Henrik Kniberg • RUM & Synthetic - Tom Gerhard