Multi-screen experiences for the mobile web

You have probably seen these new smartphones with foldable screens, like the Samsung Galaxy Fold, or with two screens like the Surface Duo. They're already on the market, so it's up to you to update your applications for these devices! ... 🤔 But how? Where to start ? Fortunately, the web is constantly evolving and has already started adapting to account for these new devices, thanks to new APIs.

In this talk, we'll explore how you can leverage multiple screens to create new user experiences for your web apps, and dive into new experimental web features. We'll also have a look at how we can adapt existing apps for these new devices while trying to avoid dealing with the quirkiness of new CSS and JavaScript primitives.

Transcript

1. Building multi-screen experiences Building multi-screen experiences For the web For the web | | Yohan Lasorsa Yohan Lasorsa | | Olivier Leplus Olivier Leplus
2. Who are we? Yohan Lasorsa  @sinedied Olivier Leplus  @olivierleplus
3. Once upon a time... @sinedied | @olivierleplus
4. @sinedied | @olivierleplus
5. Then came new devices... @sinedied | @olivierleplus
7. And someone had an idea. @sinedied | @olivierleplus
8. @sinedied | @olivierleplus Your web apps shall be responsive! “
9. Responsive web design was born. @sinedied | @olivierleplus
10. But something wasn't right! @sinedied | @olivierleplus
11. @sinedied | @olivierleplus
12. Someone else had another idea. @sinedied | @olivierleplus
13. @sinedied | @olivierleplus
14. Then came new devices. @sinedied | @olivierleplus
15. @sinedied | @olivierleplus
16. Not this one. ( It's older than the web! ) @sinedied | @olivierleplus
17. @sinedied | @olivierleplus
18. ( Still a bit wrong. This one came before the mobile web. ) @sinedied | @olivierleplus
19. @sinedied | @olivierleplus
20. @sinedied | @olivierleplus
21. @sinedied | @olivierleplus
22. What if... @sinedied | @olivierleplus
24. Or not. @sinedied | @olivierleplus
25. Back to now. @sinedied | @olivierleplus
26. So, where to begin? @sinedied | @olivierleplus
27. Another responsive design target? @sinedied | @olivierleplus
28. ⚠️ ⚠️ Experimental APIs ahead Experimental APIs ahead @sinedied | @olivierleplus @sinedied | @olivierleplus
29. So, media queries it is! @media (screen-spanning: single-fold-vertical) { ... } @media (screen-spanning: single-fold-horizontal) { ... } @sinedied | @olivierleplus aka.ms/foldable/css-primitives aka.ms/docs/css-primitives
30. Environment variables @sinedied | @olivierleplus
31. But... things moves fast @sinedied | @olivierleplus
32. @media (horizontal-viewport-segments:2) and @media (vertical-viewport-segments:1); @media (horizontal-viewport-segments:1) and @media (vertical-viewport-segments:2); New media queries... @sinedied | @olivierleplus
33. @media (horizontal-viewport-segments:2) and @media (vertical-viewport-segments:1); @media (horizontal-viewport-segments:1) and @media (vertical-viewport-segments:2); New media queries... @sinedied | @olivierleplus With new env vars: https://github.com/w3c/csswg-drafts/pull/6474/
34. ...Opening up new possibilities? @media (horizontal-viewport-segments:2) and @media (vertical-viewport-segments:2); @sinedied | @olivierleplus
35. ...Opening up new possibilities? @media (horizontal-viewport-segments:2) and @media (vertical-viewport-segments:2); @sinedied | @olivierleplus
36. ...Opening up new possibilities? @media (horizontal-viewport-segments:2) and @media (vertical-viewport-segments:2); @sinedied | @olivierleplus
37. Let's play a game @sinedied | @olivierleplus
38. @media (horizontal-viewport-segments: 2) { body { background-color: yellow; } } @sinedied | @olivierleplus
39. @media (horizontal-viewport-segments: 2) { body { background-color: yellow; } } @sinedied | @olivierleplus
40. What about the JavaScript side? @sinedied | @olivierleplus
41. Window Segments API const screenSegments = window.getWindowSegments(); @sinedied | @olivierleplus aka.ms/foldable/window-segments aka.ms/docs/window-segments
42. Window Segments API const screenSegments = window.getWindowSegments(); 👆 But that was before community feedback... @sinedied | @olivierleplus aka.ms/foldable/window-segments aka.ms/docs/window-segments
43. After community feedback... @sinedied | @olivierleplus
44. Visual Viewport Window Segments API const screenSegments = window.visualViewport.segments; @sinedied | @olivierleplus aka.ms/foldable/window-segments aka.ms/docs/window-segments
45. Visual Viewport Window Segments API const screenSegments = window.visualViewport.segments; // => null || DOMRect[] if (screenSegments.length > 1) { // It's a foldable device ! } @sinedied | @olivierleplus aka.ms/foldable/window-segments aka.ms/docs/window-segments
46. Visual Viewport Window Segments API const screenSegments = window.visualViewport.segments; // => null || DOMRect[] if (screenSegments.length > 1) { // It's a foldable device ! for (let i = 0; i < screenSegments.length; i++) { console.log(`Screen segment ${i}:`); console.log('- width : ' + screenSegments[i].width); console.log('- height : ' + screenSegments[i].height); console.log('- x origin: ' + screenSegments[i].x); console.log('- y origin: ' + screenSegments[i].y); } } @sinedied | @olivierleplus aka.ms/foldable/window-segments aka.ms/docs/window-segments
47. Visual Viewport Window Segments API const screenSegments = window.visualViewport.segments; // => null || DOMRect[] if (screenSegments.length > 1) { // It's a foldable device ! for (let i = 0; i < screenSegments.length; i++) { console.log(`Screen segment ${i}:`); console.log('- width : ' + screenSegments[i].width); console.log('- height : ' + screenSegments[i].height); console.log('- x origin: ' + screenSegments[i].x); console.log('- y origin: ' + screenSegments[i].y); } } @sinedied | @olivierleplus aka.ms/foldable/window-segments aka.ms/docs/window-segments
48. window.visualViewport.segments is immutable window.addEventListener("resize") => { const segments = window.visualViewport.segments; console.log(segments.length); } @sinedied | @olivierleplus
49. A little bit of gym 🤸‍♀️ @sinedied | @olivierleplus
50. Device Posture API w3.org/TR/device-posture/ Device: single-screen (foldable or not), dual-screen Posture: no-fold, laptop, flat, tent, tablet, book @sinedied | @olivierleplus
51. Device Posture API navigator.devicePosture.addEventListener("change", () => { console.log(`The current posture is: ${navigator.devicePosture.type}!`); }) @sinedied | @olivierleplus
52. Device Posture API navigator.devicePosture.addEventListener("change", () => { console.log(`The current posture is: ${navigator.devicePosture.type}!`); }) @media (device-posture: laptop) and (screen-spanning: single-fold-horizontal) { /* Where the magic happens! */ } .videocall-controls .videocall-area @sinedied | @olivierleplus
53. Demos @sinedied | @olivierleplus
54. What about UI/UX? @sinedied | @olivierleplus
55. Design patterns? 🤔 @sinedied | @olivierleplus
56. List / details @sinedied | @olivierleplus
57. Companion pane @sinedied | @olivierleplus
58. Dual view @sinedied | @olivierleplus
59. Dual view interaction @sinedied | @olivierleplus
60. Setting up for development @sinedied | @olivierleplus
61. Enable browser dual-screen emulation aka.ms/foldable/dev-setup 1. chrome://flags or edge://flags 2. Enable Experimental Web Platform features 3. Open dev tools settings 4. In Experiments tab, tick Emulation: Support dual screen mode @sinedied | @olivierleplus
62. Android Surface Duo emulator aka.ms/foldable/dev-setup ⚠️ Highly recommended! @sinedied | @olivierleplus
63. About polyfills... @sinedied | @olivierleplus
64. @sinedied | @olivierleplus
65. Current polyfills status Feature Implementation Polyfill CSS Spanning ✅ 🧪 ✅ Window Segments ✅ 🧪 ✅ Device Posture ⛔ ✅ ✋ CSS Viewport Window Segments ⛔ ⛔ @sinedied | @olivierleplus
66. What about some code? @sinedied | @olivierleplus
67. const end = { message : 'Thank you!', slides : 'bit.ly/foldable-dev', links : 'aka.ms/foldable-dev' }; alert('Questions?'); {    } @sinedied | @olivierleplus