Pointer Events Working Group update / TPAC 2023 / Patrick H. Lauke

Update about Pointer Events Level 3 work for the upcoming W3C Technical Plenary and Advisory Committee (TPAC) 2023 in Seville

Video recording

w3c.github.io/pointerevents
www.w3.org/TR/pointerevents
www.w3.org/2023/09/TPAC

patrickhlauke.github.io/touch/w3c_tpac2023_pewg

Cross-posted from www.w3.org/2023/09/TPAC/group-updates.html#pointer-events

Transcript

1. Pointer Events Working Group update changes and additions in Pointer Events Level 3 Patrick H. Lauke / TPAC 2023
2. What are Pointer Events To better address devices with different input types, this specification defines a more abstract form of input, called a pointer. A higher level event model based on mouse events, but also covering pen, touch, and other (future) pointing devices. Latest stable recommendation: Pointer Events Level 2 (4 April 2019) Work has since been ongoing towards Pointer Events Level 3 (Editor's draft August 2023)
3. Pointer Events Level 3 Beyond clarifications for undocumented cases/scenarios that came out of implementation experience and developer adoption, Level 3 includes a few new features: ▪   pointerrawupdate event to better react to fast movements ▪   getCoalescedEvents() and getPredictedEvents() methods ▪   altitudeAngle and azimuthAngle properties ▪  bonus: redefinition of click , auxclick , and contextmenu
4. pointerrawupdate
5. The problem For performance reasons, user agents MAY delay the dispatch of pointermove events (as they already do for mousemove ). For very fast pointer movements, user agents will generally "coalesce" individual small movements into a single pointermove event. While good for performance, this can be problematic for scenarios where authors want to accurately track high-frequency pointer movements – for instance, drawing applications.
6. Demo: basic drawing application using pointermove
7. pointerrawupdate The new pointerrawupdate event aims help make these applications work smoother. Compared to the pointermove event, user agents SHOULD dispatch pointerrawupdate as soon as possible.
8. Demo: basic drawing application using pointerrawupdate
9. pointerrawupdate May have a performance impact – authors should keep code executed in response to pointerrawupdate to a minimum (e.g. just store coordinates) Note: even though pointerrawupdate should fire as soon as possible, the user agent may still coalesce a few individual events/changes.
10. getCoalescedEvents()
11. The problem For very fast pointer movements, user agents will generally "coalesce" individual small movements into a single pointermove or pointerrawupdate event Again, for certain applications, authors may want access to all the separate events that were coalesced by the user agent
12. getCoalescedEvents() The new getCoalescedEvents() method gives authors access to all the raw position/property changes that were coalesced into an event Best of both worlds – allows for increased granularity, without incurring additional performance penalties: ▪  listen to regular pointermove (or pointerrawupdate ) events ▪  then process all the coalesced events for that event
13. foo.addEventListener("pointermove", (e)=> { if (e.getCoalescedEvents) { for (let ec of e.getCoalescedEvents()) { // access the coalesced event properties // like clientX/clientY (more granular) } } else { // fallback: use the pointermove event's // properties instead } });
14. Demo: basic drawing application using pointermove and getCoalescedEvents()
15. getPredictedEvents()
16. The problem Even with the use of pointerrawupdate and getCoalescedEvents() , certain applications – such as drawing applications – may still exhibit perceived latency. There will always be a gap, no matter how small, between an event being dispatched and the application reacting to it.
17. getPredictedEvents() Some user agents have built-in algorithms which, after a series of confirmed pointer movements, can predict likely future movements. The new getPredictedEvents() method gives authors access to these predicted events These can be helpful in scenarios like drawing applications: draw ahead to predicted positions to reduce perceived latency (but discard these speculative/predicted points when the real points are received).
18. foo.addEventListener("pointermove", (e)=> { // regular processing of the event, // and/or any coalesced events if (e.getPredictedEvents) { for (let ep of e.getPredictedEvents()) { // do something with the predicted events, // such as speculatively drawing ahead } } });
19. Demo: basic drawing application using pointermove and getPredictedEvents()
20. Out of scope Both getCoalescedEvents() and getPredictedEvents() only define the methods/API to access coalesced and predicted events The Pointer Events specification itself does not define how events are coalesced or predicted – this is left up to individual implementations (operating system / user agent dependent)
21. altitudeAngle / azimuthAngle
22. The problem The original Pointer Events specification defined tiltX and tiltY properties to convey the orientation of a stylus These properties are, admittedly, not very intuitive for developers
23. tiltX : The plane angle (in degrees, in the range of [-90,90]) between the Y-Z plane and the plane containing both the transducer (e.g. pen/stylus) axis and the Y axis … tiltY : The plane angle … between the X-Z plane and the plane containing both the transducer … axis and the X axis …
24. altitudeAngle / azimuthAngle Pointer Events Level 3 "borrows" the altitudeAngle and azimuthAngle properties from Touch Events (introduced when Apple expanded Touch Events to support Pencil on iPad)
25. altitudeAngle : The altitude (in radians) of the transducer (e.g. pen/stylus), in the range [0,π/2] — where 0 is parallel to the surface (X-Y plane), and π/2 is perpendicular to the surface. … azimuthAngle : The azimuth angle (in radians) of the transducer …, in the range [0, 2π] — where 0 represents a transducer whose cap is pointing in the direction of increasing X values (point to "3 o'clock" if looking straight down) on the X-Y plane, and the values progressively increase when going clockwise …
26. altitudeAngle / azimuthAngle User agents MUST provide both the classic tiltX / tiltY and the new altitudeAngle / azimuthAngle properties The specification includes algorithm for converting between the two sets
27. Demo: pen tracker (using WebGL)
28. Bonus: click / auxclick / contextmenu
29. Redefining events in UI Events The specification redefines click , auxclick , and contextmenu as Pointer Events This change is already in the latest UI Events working draft Possible new application: determine what type of input caused one of these events to be fired using the pointerType property
30. Demo: basic button with pointerType check
31. Thank you… Currently working on last few blockers for Level 3 and Web Platform Tests Hoping to go to Candidate Recommendation (CR) shortly ▪  Pointer Events Level 3 (Editor's Draft) ▪  github.com/w3c/pointerevents