Share
Embed
Transcript
273 slides
-
getting touchy AN INTRODUCTION TO TOUCH AND POINTER EVENTS Patrick
H. Lauke / Frontend NE / 2 March 2017 -
about me... • senior accessibility consultant at The Paciello Group
• previously developer relations at Opera • co-editor Touch Events Level 2 • WG chair and co-editor Pointer Events Level 2 -
compatibility mouse events (mouseenter) > mouseover > mousemove* > mousedown
> (focus) > mouseup > click * only a single sacrificial mousemove event fired -
on first tap (mouseenter) > mouseover > mousemove > mousedown
> (focus) > mouseup > click subsequent taps mousemove > mousedown > mouseup > click tapping away (mouseout) > (blur) focus / blur only on focusable elements; subtle differences between browsers Mobile/tablet touchscreen activation/tap event order -
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html less of a problem in modern browsers, but for
iOS < 9.3 or older Androids still relevant -
touch events introduced by Apple, adopted in Chrome/Firefox/Opera (and belatedly
IE/Edge – more on that later) www.w3.org/TR/touch-events -
events fired on tap touchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover > mousemove > mousedown > (focus) > mouseup > click (mouse events only fired for single-finger tap) -
on first tap touchstart > [touchmove]+ > touchend > (mouseenter)
> mouseover > mousemove > mousedown > (focus) > mouseup > click subsequent taps touchstart > [touchmove]+ > touchend > mousemove > mousedown > mouseup > click tapping away mouseout > (mouseleave) > (blur) -
• too many touchmove events prevent mouse compatibility events after
touchend (not considered a "clean" tap) • too many touchmove events on activatable elements can lead to touchcancel (in old Chrome/Browser versions) • not all browsers consistently send touchmove • differences in focus / blur and some mouse compatibility events (e.g. mouseenter / mouseleave ) • some events may only fire when tapping away to another focusable element (e.g. blur ) some browsers outright weird... -
Browser/Android 4.3 mouseover > mousemove > touchstart > touchend >
mousedown > mouseup > click Browser/Blackberry PlayBook 2.0 touchstart > mouseover > mousemove > mousedown > touchend > mouseup > click UC Browser 10.8/Android 6 mouseover > mousemove > touchstart > (touchmove)+ > touchend > mousedown > focus > mouseup > click -
/ feature detection for touch events */ if ( 'ontouchstart'
in window ) { /* some clever stuff here */ } /* older browsers have flaky support so more hacky tests needed...use Modernizr.touch or similar */ -
/* conditional "touch OR mouse/keyboard" event binding */ if ('ontouchstart'
in window) { // set up event listeners for touch ... } else { // set up event listeners for mouse/keyboard ... } -
@patrick_h_lauke showing a naive "touch or mouse" issue on Flickr
(which has since been fixed) -
Bug 888304 - touch-events on Firefox-desktop should be disabled until
we can support them properly -
Android + mouse – like touch (mouse emulating touch) touchstart
> touchend > mouseover > mousemove > mousedown > (focus) > mouseup > click -
Blackberry PlayBook 2.0 + mouse – like desktop mouseover >
mousedown > (mousemove)+ > mouseup > click -
Blackberry Leap (BBOS 10.1) + mouse – like desktop mouseover
> mousedown > (mousemove)+ > mouseup > click -
Windows 10 Mobile/Microsoft Edge + mouse – like desktop mouse
events (+ pointer events) + click -
iOS keyboard only works in same situations as on-screen keyboard
(e.g. text inputs, URL entry) -
iOS + keyboard – similar to touch focus ¡ touchstart
> touchend > (mouseenter) > mouseover > mousemove > mousedown > blur > mouseup > click -
further scenarios? • desktop with external touchscreen • touchscreen laptop
with non-touch second screen • touchscreen laptop with trackpad/mouse • ...and other permutations? -
/* feature detection for touch events */ if ('ontouchstart' in
window) { /* browser supports touch events but user is not necessarily using touch (exclusively) */ /* it could be a mobile, tablet, desktop, fridge ... */ } -
/* MQ4 Interaction Media Features*/ pointer: none | coarse |
fine hover: none | hover any-pointer: none | coarse | fine any-hover: none | hover hover: on-demand / any-hover: on-demand removed in recent drafts -
/* Naive uses of Interaction Media Features */ @media (pointer:
fine) { /* primary input has fine pointer precision... so make all buttons/controls small? */ } @media (hover: hover) { /* primary input has hover...so we can rely on it? */ } /* pointer and hover only check "primary" input, but what if there's a secondary input that lacks capabilities? */ -
/* Better uses of Interaction Media Features */ @media (any-pointer:
coarse) { /* at least one input has coarse pointer precision... provide larger buttons/controls for touch */ } /* test for *any* "least capable" inputs (primary or not) */ Limitation: [mediaqueries-4] any-hover can't be used to detect mixed hover and non-hover capable pointers. Also, pointer / hover / any-pointer / any-hover don't cover non-pointer inputs (e.g. keyboards) – always assume non-hover-capable inputs @media (any-hover: none) { /* at least one input lacks hover capability... don't rely on it or avoid altogether */ } -
if in doubt, offer a way to switch interfaces... or
just always use a touch-optimised interface -
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html less of a problem in modern browsers, but for
iOS < 9.3 or older Androids still relevant -
touch / mouse events delay touchstart > [touchmove]+ > touchend
> [300ms delay] (mouseenter) > mouseover > mousemove > mousedown > (focus) > mouseup > click -
touchend for a control that fires after finger lifted (but
this can result in events firing after zoom/scroll) -
/* DON'T DO THIS: conditional "touch OR mouse/keyboard" event binding
*/ if ('ontouchstart' in window) { // set up event listeners for touch foo.addEventListener('touchend', ...); ... } else { // set up event listeners for mouse/keyboard foo.addEventListener('click', ...); ... } -
/* DO THIS: doubled-up "touch AND mouse/keyboard" event binding */
// set up event listeners for touch foo.addEventListener('touchend', ...); // set up event listeners for mouse/keyboard foo.addEventListener('click', ...); /* but this would fire our function twice for touch? */ patrickhlauke.github.io/touch/tests/event-listener_naive-event-doubling.html -
/* DO THIS: doubled-up "touch AND mouse/keyboard" event binding */
// set up event listeners for touch foo.addEventListener('touchend', function(e) { // prevent compatibility mouse events and click e.preventDefault(); ... }); // set up event listeners for mouse/keyboard foo.addEventListener('click', ...); patrickhlauke.github.io/touch/tests/event-listener_naive-event-doubling-fixed.html -
Changeset 191072 - Web pages with unscalable viewports shouldn't have
a single tap delay (from iOS 9.3 onwards) -
@thomasfuchs - Safari/iOS10beta3 ignores unscalable viewports (no official Apple documentation
about the change...but the 300ms delay is back) -
Chrome 32+ / Android: content="width=device-width" suppresses double-tap-to-zoom, still allows pinch
zoom Google Developers: 300ms tap delay, gone away -
suppressing 300ms delay if your code does rely on handling
click / mouse events: • "mobile optimised" <meta name="viewport" content="width=device-width"> • add touch-action:manipulation in CSS (part of Pointer Events) • use helper library like fastclick.js for older browsers • make your viewport non-scalable... -
events fired on tap touchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover > mousemove* > mousedown > (focus) > mouseup > click * mouse event emulation fires only a single mousemove too many touchmove events prevent mouse compatibility events after touchend -
£ar posX, posY; function positionHandler(e) { posX = e.clientX ;
posY = e.clientY ; } canvas.addEventListener(' mousemove ', positionHandler, false); -
¤ar posX, posY; function positionHandler(e) { posX = e.clientX ;
posY = e.clientY ; } canvas.addEventListener(' mousemove ', positionHandler, false); canvas.addEventListener(' touchmove ', positionHandler, false); /* but this won't work for touch... */ -
interface MouseEvent : UIEvent { readonly attribute long screenX ;
readonly attribute long screenY ; readonly attribute long clientX ; readonly attribute long clientY ; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute unsigned short button; readonly attribute EventTarget relatedTarget; // [DOM4] UI Events readonly attribute unsigned short buttons; }; www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent www.w3.org/TR/uievents/#events-mouseevents -
partial interface MouseEvent { readonly attribute double screenX; readonly attribute
double screenY; readonly attribute double pageX ; readonly attribute double pageY ; readonly attribute double clientX; readonly attribute double clientY; readonly attribute double x ; readonly attribute double y ; readonly attribute double offsetX ; readonly attribute double offsetY ; }; www.w3.org/TR/cssom-view/#extensions-to-the-mouseevent- interface -
interface TouchEvent : UIEvent { readonly attribute TouchList touches ;
readonly attribute TouchList targetTouches ; readonly attribute TouchList changedTouches ; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; }; www.w3.org/TR/touch-events/#touchevent-interface -
interface Touch { readonly attribute long identifier; readonly attribute EventTarget
target; readonly attribute long screenX ; readonly attribute long screenY ; readonly attribute long clientX ; readonly attribute long clientY ; readonly attribute long pageX ; readonly attribute long pageY ; }; www.w3.org/TR/touch-events/#touch-interface -
TouchList differences touches all touch points on screen targetTouches all
touch points that started on the element changedTouches touch points that caused the event to fire -
¥ar posX, posY; function positionHandler(e) { if ((e.clientX)&&(e.clientY)) { posX
= e.clientX; posY = e.clientY; } else if (e.targetTouches) { posX = e.targetTouches[0].clientX; posY = e.targetTouches[0].clientY; e.preventDefault() ; } } canvas.addEventListener('mousemove', positionHandler, false ); canvas.addEventListener('touchmove', positionHandler, false ); -
/* Swipe detection from basic principles */ Δt = end.time
- start.time; Δx = end.x - start.x; Δy = end.y - start.y; if (( Δt > timeThreshold ) || ( (Δx + Δy) < distanceThreshold )) { /* too slow or movement too small */ } else { /* it's a swipe! - use ¦x and ¦y to determine direction - pythagoras √( ¦x² + ¦y² ) for distance - distance/¦t to determine speed */ } -
do heavy lifting (drawing etc.) separately, avoid queueing (e.g. using
setTimeout and/or requestAnimationFrame ) -
interface TouchEvent : UIEvent { readonly attribute TouchList touches ;
readonly attribute TouchList targetTouches ; readonly attribute TouchList changedTouches ; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; }; www.w3.org/TR/touch-events/#touchevent-interface -
/* iterate over relevant TouchList */ for (i=0; i< e.targetTouches
.length; i++) { ... posX = e.targetTouches[i].clientX ; posY = e.targetTouches[i].clientY ; ... } -
/* iOS/Safari/WebView has gesture events for size/rotation, not part of
the W3C Touch Events spec. */ /* gesturestart / gesturechange / gestureend */ foo.addEventListener('gesturechange', function(e) { /* e.scale e.rotation */ /* values can be plugged directly into CSS transforms etc */ }); /* not supported in Chrome/Firefox/Opera */ iOS Developer Library - Safari Web Content Guide - Handling gesture events -
/* Pinch/rotation detection from basic principles */ Δx = x2
- x1; Δy = y2 - y1; /* pythagoras √( §x² + §y² ) to calculate distance */ Δd = Math.sqrt( (Δx * Δx) + (Δy * Δy) ); /* trigonometry to calculate angle */ α = Math.atan2( Δy, Δx ); Mozilla Developer Network: Math.atan2() -
not defined in spec (yet), but de facto yes in
most modern browsers patrickhlauke.github.io/touch/gesture-touch -
/* Extension to touch objects */ partial interface Touch {
readonly attribute float radiusX; readonly attribute float radiusY; readonly attribute float rotationAngle; readonly attribute float force; }; -
/* Touch Events – contact geometry */ partial interface Touch
{ readonly attribute float radiusX ; readonly attribute float radiusY ; readonly attribute float rotationAngle ; readonly attribute float force; }; -
/* Touch Events – force */ partial interface Touch {
readonly attribute float radiusX; readonly attribute float radiusY; readonly attribute float rotationAngle; readonly attribute float force ; }; force : value in range 0 – 1 . if no hardware support 0 (some devices, e.g. Nexus 10, "fake" force based on radiusX / radiusY ) -
patrickhlauke.github.io/touch/tracker/tracker-force-pressure.html iPhone 6s with 3D Touch supports force ... (Safari
and WKWebView, e.g. Chrome/iOS, but not UIWebView, e.g. Firefox/iOS) -
W3C Touch Events Level 2 (Editor's Draft) (merges errata, touch
events extensions, fractional touch coordinates) -
...and some new inputs (though currently mapped to mouse) Building
Xbox One Apps using HTML and JavaScript “ouTube: Xbox One Edge Browser sends Pointer Events -
work now continues to enhance/expand Pointer Events... W3C Pointer Events
Level 2 (Editor's Draft) -
Apple Pencil currently indistinguishable from touch in browser / JavaScript
(no tilt information, fires touch events) -
events fired on tap (Edge) mousemove* > pointerover > mouseover
> pointerenter > mouseenter > pointerdown > mousedown > focus gotpointercapture > pointermove > mousemove > pointerup > mouseup > lostpointercapture > click > pointerout > mouseout > pointerleave > mouseleave mouse events fired inline with pointer events (for a primary pointer, e.g. first finger on screen) -
IE10/IE11 quirks • vendor-prefixed in IE10 ( MSPointerDown ..., navigator.msMaxTouchPoints
, -ms-touch-action ) • unprefixed in IE11 (but prefixed versions mapped for compatibility) • event order (when click is fired) incorrect in IE10/IE11 see W3C Pointer Events WG mailing list - Jacob Rossi (Microsoft) -
Chrome, Edge (on mobile), IE11 (Windows Phone 8.1update1) support both
Touch Events and Pointer Events w3c.github.io/pointerevents/#mapping-for-devices-that-do-not-support-hover -
about:flags in Microsoft Edge to turn on touch events on
desktop (e.g. touch-enabled laptops) – off by default for site compatibility -
... when touch events also supported (Edge) pointerover > pointerenter
> pointerdown > touchstart > pointerup > touchend > mouseover > mouseenter > mousemove > mousedown > focus > mouseup > click > pointerout > pointerleave Specific order of events is not defined in the spec in this case – will vary between browsers... only problem if handling both pointer events and mouse events (which is nonsensical) -
/* Pointer Events extend Mouse Events vs Touch Events and
their new objects / TouchLists / Touches */ interface PointerEvent : MouseEvent { readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure; readonly attribute float tangentialPressure; /* Level 2 */ readonly attribute long tiltX; readonly attribute long tiltY; readonly attribute long twist; /* Level 2 */ readonly attribute DOMString pointerType; readonly attribute boolean isPrimary; } -
handling mouse input is exactly the same as traditional mouse
events (only change pointer* instead of mouse* event names) -
handling touch or stylus is also exactly the same as
traditional mouse events (mouse code can be adapted to touch/stylus quickly) -
/* detecting pointer events support */ if ( window.PointerEvent )
{ /* some clever stuff here but this covers touch, stylus, mouse, etc */ } /* still listen to click for keyboard! */ -
/* detect maximum number of touch points */ if (
navigator.maxTouchPoints > 0 ) { /* device with a touchscreen */ } if ( navigator.maxTouchPoints > 1 ) { /* multitouch-capable device */ } "you can detect a touchscreen" ...but user may still use other input mechanisms -
pointer / mouse events and delay ©©© [300ms delay] click
... 300ms delay just before click event -
• double-up pointerup and click listeners? • prevent code firing
twice with preventDefault ? won't work: preventDefault() stops mouse compatibility events, but click is not considered mouse compatibility event -
CSS property what action should the browser handle? touch-action: auto
| none | [ pan-x || pan-y ] | manipulation www.w3.org/TR/pointerevents/#the-touch-action-css-property only determines default touch action, does not stop compatibility mouse events nor click -
Pointer Events Level 2 expanded set of values (useful for
pull-to-refresh, carousels, etc) touch-action: auto | none | [ [ pan-x | pan-left | pan-right ] || [ pan-y | pan-up | pan-down ] ] | manipulation w3c.github.io/pointerevents/#the-touch-action-css-property -
Windows 10 build 15007 on Mobile ignores unscalable viewports no
delay until user zooms for first time... -
/* touch events: separate handling */ foo.addEventListener('touchmove', ... , false);
foo.addEventListener('mousemove', ... , false); /* pointer events: single listener for mouse, stylus, touch */ foo.addEventListener(' pointermove ', ... , false); -
no need for separate mouse or touch code to get
x / y coords -
/* Reminder: Touch Events need separate code for x /
y coords */ function positionHandler(e) { if ((e.clientX)&&(e.clientY)) { posX = e.clientX; posY = e.clientY; } else if (e.targetTouches) { posX = e.targetTouches[0].clientX; posY = e.targetTouches[0].clientY; ... } } canvas.addEventListener('mousemove', positionHandler, false ); canvas.addEventListener('touchmove', positionHandler, false ); -
/* Pointer Events extend Mouse Events */ foo.addEventListener(' pointermove ',
function(e) { ... posX = e.clientX ; posY = e.clientY ; ... }, false); www.w3.org/TR/pointerevents/#pointerevent-interface -
3D Rotator modified to use Pointer Events minimal code changes,
as Pointer Events extend mouse events -
foo.addEventListener('pointermove', function(e) { ... switch( e.pointerType ) { case '
mouse ': ... break; case ' pen ': ... break; case ' touch ': ... break; default : /* future-proof */ } ... } , false); -
/* PointerEvents don't have the handy TouchList objects, so we
have to replicate something similar... */ ar points = []; switch (e.type) { case ' pointerdown ': /* add to the array */ break; case ' pointermove ': /* update the relevant array entry's x and y */ break; case ' pointerup ': case ' pointercancel ': /* remove the relevant array entry */ break; } -
/* like iOS/Safari, IE/Win has higher-level gestures , but these
are not part of the W3C Pointer Events spec. Replicate these from basic principles again... */ MSDN IE10 Developer Guide: Gesture object and events -
/* Pointer Events - pressure */ interface PointerEvent : MouseEvent
{ readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure ; readonly attribute float tangentialPressure; readonly attribute long tiltX; readonly attribute long tiltY; readonly attribute long twist; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary; } pressure : value in range 0 – 1 . if no hardware support, 0.5 in active button state, 0 otherwise -
hovering stylus • hardware-dependant • pointermove fires •
pressure == 0 (non-active button state) • track pointerdown / pointerup to be safe -
/* Pointer Events - contact geometry */ interface PointerEvent :
MouseEvent { readonly attribute long pointerId; readonly attribute long width ; readonly attribute long height ; readonly attribute float pressure; readonly attribute float tangentialPressure; readonly attribute long tiltX; readonly attribute long tiltY; readonly attribute long twist; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary; } if hardware can't detect contact geometry, either 0 or "best guess" (e.g. for mouse/stylus, return width / height of 1 ) -
/* Pointer Events - tilt */ interface PointerEvent : MouseEvent
{ readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure; readonly attribute float tangentialPressure; readonly attribute long tiltX ; readonly attribute long tiltY ; readonly attribute long twist; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary; } tiltX / tiltY : value in degrees -90 – 90 . returns 0 if hardware does not support tilt -
pointermove fires if any property changes, not just x/y position
( width , height , tiltX , tiltY , pressure ) -
/* cover all cases (hat-tip Stu Cox) */ if (window.PointerEvent)
{ /* bind to Pointer Events: pointerdown, pointerup, etc */ } else { /* bind to mouse events: mousedown, mouseup, etc */ if ('ontouchstart' in window) { /* bind to Touch Events: touchstart, touchend, etc */ } } /* bind to keyboard / click */ -
adding jQuery PEP * <script src="https:code.jquery.compep0.4.1pep.js"><script> * need to
use custom touch-action attribute, not CSS (yet) * <button touch-action="none" >...<div> -
Chrome DevTools / Device Mode & Mobile Emulation correctly emulates
touch and pointer events -
no touch emulation, nor touch events + pointer events (like
on real Windows 10 Mobile) emulation, in Edge/F12 Tools