Tobin Titus

Getting content to the user fast is one the chief concerns of web performance. Performance best practices for graphics currently include a number of recommendations that are usually aimed at reducing network utilization. These recommendations are important. However, as our landscape continues to shift to lower-cost devices on mobile networks, we face more challenges than we would have ever expected.

This sessions will cover physical limitations of these devices, choices of graphics formats, and how those limitations affect not just graphics, but your entire site or application performance.

Transcript

1. IMAGECON 2017 The Physics of Fast Graphics Tobin Titus @tobint
2. IMAGECON 2017 The Impact of Images  100 resource requests, 18 Domains, 33 TCP Connections 54 7 23 9 7 Images CSS JavaScript HTML Other 1627 81 408 48 101 Images CSS JavaScript HTML Other  2.467 MB Total Size, 51% Cacheable
3. IMAGECON 2017 Seemingly simplistic … Download Decode Display
4. IMAGECON 2017 .. but surprisingly complex Network Disk CPU Memory Memory Bus GPU Display Battery GB
5. IMAGECON 2017 Responsiveness makes it more complex!
6. IMAGECON 2017 Download Getting data to your device
7. © 2012 mass:werk
8. IMAGECON 2017 Bandwidth vs Throughput Bandwidth Throughput
9. IMAGECON 2017 Bandwidth vs Throughput Bandwidth Throughput
10. IMAGECON 2017 Bandwidth vs Throughput Bandwidth Throughput
11. IMAGECON 2017 Global Mobile Network Speeds 0.1 3.3 10.8 17.4 0 2 4 6 8 10 12 14 16 18 20 Speeds (Mbps) 2G 3G WiFI 4G LTE Source: State of LTE – Nov 2016
12. IMAGECON 2017 Overhead Plays Significant Role NYC to: Latency KM Miles Vacuum Fiber RTT 1 Packet Seattle 3,870 2,405 12.9 19.7 39.3 157.6 London 5,576 3,465 18.6 28.3 56.6 226.4 Beijing 11,001 6,835 36.7 55.9 111.7 447.2 TCP TLS Data
13. IMAGECON 2017 Last Mile Latency Source: Measuring Broadband America - 2016
14. IMAGECON 2017 Decode Unpacking the Downloaded Image File
15. IMAGECON 2017 Download Size vs Memory Size Download sizes may seem small, but image memory adds up very quickly 683 x 1024 RGBA 557 KB width * height * 4 2.67 MB Decoded
16. IMAGECON 2017 Memory Utilization Responsible memory management is crucial when targeting low- cost devices with your sites and apps - 20 40 60 80 100 120 140 160 180 200 Megabytes 5000 sites’ image memory utilization
17. Images: Downloading Decoding
18. Images: Continue Downloading Decoding
19. Images : Decoding Decoding
20. Processing Images: Processing Decoding
21. Images: Copy to the GPU Processing Decoding
22. Images: Download to Display Processing Decoding
23. Images: Download to Display Processing Decoding
24. IMAGECON 2017 Display Showing Your Images to The User
25. IMAGECON 2017 Image Formats Matter Image formats use different means to depict colors RGB to YCbCr Color Conversion Chroma Subsampling DCT / Quantization Huffman Encoding
26. Chroma Subsampling + Original Luma (Y) Chroma (Cb) Chroma (Cr) = + + 4:4:4 Original Luma (Y) Chroma (Cb) Chroma (Cr) = + 4:2:0 Original Luma (Y) Chroma (Cb) Chroma (Cr) = + + 4:2:2 62.5% less memory than RGB (no subsampling)
27. JPEG images were previously decoded, upsampled and color-converted on the CPU before copying them to the GPU JPEG – From Download to Display Huffman Decoding Dequantization / iDCT Chroma Upsampling YCbCr to RGB Color Conversion
28. IMAGECON 2017 Drain! Powering your Device without Overheating
29. Power Consumption Let it rest! Power efficiency can drain your users’ battery and decrease satisfaction with your application CPU Utilization GPU Utilization VSync
30. Power Consumption Let it rest! Power efficiency can drain your users’ battery and decrease satisfaction with your application CPU Utilization GPU Utilization VSync
31. Power Consumption Let it rest! Power efficiency can drain your users’ battery and decrease satisfaction with your application CPU Utilization GPU Utilization VSync
32. IMAGECON 2017 Memory And Power Consumption Maintaining memory consumes power and generates heat. The sites to the left of the graph use far less power than the sites on the right. - 20 40 60 80 100 120 140 160 180 200 Megabytes 5000 sites’ image memory utilization
33. IMAGECON 2017 Navigation App or Site Network / GPS CPU GPU Memory GB
34. IMAGECON 2017 Resources Overview Concepts High Performance Websites Steve Souders, September 2007 Event Faster Websites: Best Practices Steve Souders, June 2009 High Performance Browser Networking Ilya Grigorik, September 2013 A Picture Costs a Thousand Words Guy Podjamy, November 2013 High Performance Images Colin Bendell, Tim Kadlec, Yoav Weiss, Guy Podjarny, Kick Doyle & Mike McCall November 2016 Microsoft Guidance High-Performance Web Platform Tobin Titus, Build 2014 Web Runtime Performance Tobin Titus, Build 2013 Performance Tricks to Make Apps & Sites Faster Jatinder Mann, Build 2012 50 Performance Tricks for Windows Store Apps Jason Weber, Build 2011 W3C Web Performance Web Performance Working Group Homepage Blog Posts 1) JPEGXR updates (comparisons to WebP) 2) Debugging/Tuning Web Sites w/F12 Developer Tools 3) Measuring Performance with ETW/XPerf 4) Measuring Performance in Lab Environments 5) Browser Subsystems Overview 6) What Common Benchmarks Measure Tools Windows Performance Toolkit Fiddler Web Debugger Contact Tobin Titus Twitter: @tobint Email: tobint@microsoft.com
35. IMAGECON 2017