Getting touchy - Everything you (n)ever wanted to know about touch and pointer events / JavaScript Days 2017

Slide 1

Slide 1 text

getting touchy EVERYTHING YOU (N)EVER WANTED TO KNOW ABOUT TOUCH AND POINTER EVENTS Patrick H. Lauke / JavaScript Days 2017 / 22 March 2017 / Munich

Slide 2

Slide 2 text

about me... • senior accessibility consultant at The Paciello Group • previously developer relations at Opera • co-editor Touch Events Level 2 • WG chair and co-editor Pointer Events Level 2

Slide 3

Slide 3 text

github.com/patrickhlauke/getting-touchy-presentation "evergreen" expanded version of this presentation (and branches for speciﬁc conferences)

Slide 4

Slide 4 text

patrickhlauke.github.io/touch

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Touch/pointer events test results

Slide 7

Slide 7 text

CodePen: Basic slider - from mouse-only to mouse/touch/stylus enabled

Slide 8

Slide 8 text

my JavaScript sucks... (but will hopefully convey the right concepts)

Slide 9

Slide 9 text

“how can I make my website work on touch devices?”

Slide 10

Slide 10 text

you don't need touch events browsers emulate regular mouse events

Slide 11

Slide 11 text

patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html

Slide 12

Slide 12 text

patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html

Slide 13

Slide 13 text

compatibility mouse events (mouseenter) > mouseover > mousemove* > mousedown > (focus) > mouseup > click * only a single “sacriﬁcial” mousemove event ﬁred

Slide 14

Slide 14 text

on ﬁrst tap (mouseenter) > mouseover > mousemove > mousedown > (focus) > mouseup > click subsequent taps mousemove > mousedown > mouseup > click tapping away (mouseout) > (blur) focus / blur only on focusable elements; subtle differences between browsers Mobile/tablet touchscreen activation/tap event order

Slide 15

Slide 15 text

emulation works, but is limiting/problematic

Slide 16

Slide 16 text

1. delayed event dispatch 2. mousemove doesn't track

Slide 17

Slide 17 text

1. delayed event dispatch 2. mousemove doesn't track

Slide 18

Slide 18 text

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html less of a problem in modern browsers, but for iOS < 9.3 or older Androids still relevant

Slide 19

Slide 19 text

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

Slide 20

Slide 20 text

1. delayed event dispatch 2. mousemove doesn't track

Slide 21

Slide 21 text

patrickhlauke.github.io/touch/particle/2

Slide 22

Slide 22 text

patrickhlauke.github.io/touch/particle/2

Slide 23

Slide 23 text

“we need to go deeper...”

Slide 24

Slide 24 text

touch events introduced by Apple, adopted in Chrome/Firefox/Opera (and belatedly IE/Edge – more on that later) www.w3.org/TR/touch-events

Slide 25

Slide 25 text

caniuse.com: Can I use touch events?

Slide 26

Slide 26 text

touchstart touchmove touchend touchcancel

Slide 27

Slide 27 text

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

Slide 28

Slide 28 text

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html Bug 128534 - 'mouseenter' mouse compat event not ﬁred...

Slide 29

Slide 29 text

events fired on tap touchstart > [touchmove]+ > touchend > (mouseenter) > mouseover > mousemove > mousedown > (focus) > mouseup > click (mouse events only fired for single-finger tap)

Slide 30

Slide 30 text

on ﬁrst tap touchstart > [touchmove]+ > touchend > (mouseenter) > mouseover > mousemove > mousedown > (focus) > mouseup > click subsequent taps touchstart > [touchmove]+ > touchend > mousemove > mousedown > mouseup > click tapping away mouseout > (mouseleave) > (blur)

Slide 31

Slide 31 text

• too many touchmove events prevent mouse compatibility events after touchend (not considered a "clean" tap) • too many touchmove events on activatable elements can lead to touchcancel (in old Chrome/Browser versions) • not all browsers consistently send touchmove • differences in focus / blur and some mouse compatibility events (e.g. mouseenter / mouseleave ) • some events may only ﬁre when tapping away to another focusable element (e.g. blur ) some browsers outright weird...

Slide 32

Slide 32 text

Browser/Android 4.3 mouseover > mousemove > touchstart > touchend > mousedown > mouseup > click Browser/Blackberry PlayBook 2.0 touchstart > mouseover > mousemove > mousedown > touchend > mouseup > click UC Browser 10.8/Android 6 mouseover > mousemove > touchstart > (touchmove)+ > touchend > mousedown > focus > mouseup > click

Slide 33

Slide 33 text

Touch/pointer events test results

Slide 34

Slide 34 text

shouldn't affect your code, unless you're expecting a very speciﬁc sequence...

Slide 35

Slide 35 text

simple feature detection for touch events

Slide 36

Slide 36 text

/* feature detection for touch events */ if ( 'ontouchstart' in window ) { /* some clever stuff here */ } /* older browsers have flaky support so more hacky tests needed...use Modernizr.touch or similar */

Slide 37

Slide 37 text

patrickhlauke.github.io/touch/tests/touch-feature-detect.html

Slide 38

Slide 38 text

/* conditional "touch OR mouse/keyboard" event binding */ if ('ontouchstart' in window) { // set up event listeners for touch ... } else { // set up event listeners for mouse/keyboard ... }

Slide 39

Slide 39 text

don't make it touch-exclusive

Slide 40

Slide 40 text

hybrid devices touch + mouse + keyboard

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

@patrick_h_lauke showing a naive "touch or mouse" issue on Flickr (which has since been ﬁxed)

Slide 44

Slide 44 text

Bug 888304 - touch-events on Firefox-desktop should be disabled until we can support them...

Slide 45

Slide 45 text

Issue 392584: TouchEvent API should be enabled consistently

Slide 46

Slide 46 text

Chrome now returns window.TouchEvent on non-touch devices patrickhlauke.github.io/touch/tests/touch-feature-detect.html

Slide 47

Slide 47 text

even on "mobile" we can have multiple inputs...

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

Windows 10 "Continuum" (mobile device acting as desktop)

Slide 51

Slide 51 text

Android + mouse – like touch (mouse emulating touch) touchstart > touchend > mouseover > mousemove > mousedown > (focus) > mouseup > click

Slide 52

Slide 52 text

Android + Chrome 58 + mouse – like desktop mouse events (+ pointer events) + click

Slide 53

Slide 53 text

Blackberry PlayBook 2.0 + mouse – like desktop mouseover > mousedown > (mousemove)+ > mouseup > click

Slide 54

Slide 54 text

Blackberry Leap (BBOS 10.1) + mouse – like desktop mouseover > mousedown > (mousemove)+ > mouseup > click

Slide 55

Slide 55 text

Windows 10 Mobile/Microsoft Edge + mouse – like desktop mouse events (+ pointer events) + click

Slide 56

Slide 56 text

Windows 10 Mobile/Microsoft Edge + keyboard – like desktop focus / click / blur

Slide 57

Slide 57 text

Android + keyboard – like desktop focus / click / blur

Slide 58

Slide 58 text

iOS keyboard only works in same situations as on-screen keyboard (e.g. text inputs, URL entry)

Slide 59

Slide 59 text

VoiceOver enables full keyboard access on iOS

Slide 60

Slide 60 text

iOS + keyboard – similar to touch (using VO + SPACE ) focus / touchstart > touchend > (mouseenter) > mouseover > mousemove > mousedown > blur > mouseup > click

Slide 61

Slide 61 text

iOS + keyboard – like desktop (using ENTER ) focus / click / blur

Slide 62

Slide 62 text

mobile Assistive Technologies (e.g. screen readers on touchscreen devices)

Slide 63

Slide 63 text

iOS + VoiceOver (with/without keyboard) – similar to touch focus / touchstart > touchend > (mouseenter) > mouseover > mousemove > mousedown > blur > mouseup > click

Slide 64

Slide 64 text

Android 4.3/Chrome + TalkBack – keyboard/mouse hybrid focus / blur > mousedown > mouseup > click > focus

Slide 65

Slide 65 text

Android 6.1/Chrome + TalkBack – like touch touchstart > touchend > mouseover > mouseenter > mousemove > mousedown > focus > mouseup > click

Slide 66

Slide 66 text

Android 6.1/Firefox + TalkBack – similar to touch touchstart > mousedown > focus > touchend > mouseup > click

Slide 67

Slide 67 text

further scenarios? • desktop with external touchscreen • touchscreen laptop with non-touch second screen • touchscreen laptop with trackpad/mouse • ...and other permutations?

Slide 68

Slide 68 text

note on trackpad gestures • trackpads that allow multi-finger gestures (e.g. recent MacBooks, Magic Mouse/Trackpad, Windows precision trackpad, ASUS Smart Gesture, Wacom Intuos Pro gestures etc) don't fire touch events / expose touch points • these gestures are handled directly by the OS/driver and don't reach the browser as raw touches • Wacom Intuos Pro touch input exposed as mouse to OS • OS X/Safari 9.1+ does fire Apple's proprietary gesture* events (for pinch-to-zoom, rotation)

Slide 69

Slide 69 text

no way to detect these cases...

Slide 70

Slide 70 text

Modernizr.touch detects touch events not touch devices

Slide 71

Slide 71 text

Modernizr: Detecting a mouse user

Slide 72

Slide 72 text

Stu Cox: You can't detect a touchscreen

Slide 73

Slide 73 text

hacks.mozilla.org - Detecting touch [...]

Slide 74

Slide 74 text

/* feature detection for touch events */ if ('ontouchstart' in window) { /* browser supports touch events but user is not necessarily using touch (exclusively) */ /* it could be a mobile, tablet, desktop, fridge ... */ }

Slide 75

Slide 75 text

touch or mouse or keyboard

Slide 76

Slide 76 text

touch and mouse and keyboard

Slide 77

Slide 77 text

what about CSS4 Media Queries?

Slide 78

Slide 78 text

W3C Media Queries Level 4 (Editor's Draft)

Slide 79

Slide 79 text

Interaction Media Features pointer / hover relate to “primary” pointing device any-pointer / any-hover relate to all pointing devices /* the primary input is ... */ @media (pointer: fine) { /* a mouse, stylus, ... */ } @media (pointer: coarse) { /* a touchscreen, ... */ } @media (pointer: none) { /* not a pointer (e.g. d-pad) */ } @media (hover: hover) { /* hover-capable */ } @media (hover: none) { /* not hover-capable */ } hover: on-demand / any-hover: on-demand removed in recent drafts

Slide 80

Slide 80 text

Interaction Media Features pointer / hover relate to “primary” pointing device any-pointer / any-hover relate to all pointing devices /* across all detected pointing devices at least one is ... */ @media (any-pointer: fine) { /* a mouse, stylus, ... */ } @media (any-pointer: coarse) { /* a touchscreen, ... */ } @media (any-pointer: none) { /* none of them are pointers */ } @media (any-hover: hover) { /* hover-capable */ } @media (any-hover: none) { /* none of them are hover-capable */ } hover: on-demand / any-hover: on-demand removed in recent drafts

Slide 81

Slide 81 text

caniuse.com: Can I use interaction media features?

Slide 82

Slide 82 text

Media Features and JavaScript if (window.matchMedia("(any-pointer:coarse)").matches) { ... } /* listen for dynamic changes */ window.matchMedia("(any-pointer:coarse)")↵ .onchange = function(e) { ... } window.matchMedia("(any-pointer:coarse)")↵ .addEventListener("change", function(e) { ... } , false } window.matchMedia("(any-pointer:coarse)")↵ .removeEventListener("change", function(e) { ... } , false }

Slide 83

Slide 83 text

caniuse.com: Can I use matchMedia ?

Slide 84

Slide 84 text

dev.opera - Interaction Media Features and their potential (for incorrect assumptions)

Slide 85

Slide 85 text

patrickhlauke.github.io/touch/pointer-hover-any-pointer-any-hover primary input is unchanged (and Chrome/Android required full restart – Issue 442418)

Slide 86

Slide 86 text

/* Naive uses of Interaction Media Features */ @media (pointer: fine) { /* primary input has fine pointer precision... so make all buttons/controls small? */ } @media (hover: hover) { /* primary input has hover...so we can rely on it? */ } /* pointer and hover only check "primary" input, but what if there's a secondary input that lacks capabilities? */

Slide 87

Slide 87 text

/* Better uses of Interaction Media Features */ @media (any-pointer: coarse) { /* at least one input has coarse pointer precision... provide larger buttons/controls for touch */ } /* test for *any* "least capable" inputs (primary or not) */ Limitation: [mediaqueries-4] any-hover can't be used to detect mixed hover and non-hover capable pointers. Also, pointer / hover / any-pointer / any-hover don't cover non-pointer inputs (e.g. keyboards) – always assume non-hover-capable inputs @media (any-hover: none) { /* at least one input lacks hover capability... don't rely on it or avoid altogether */ }

Slide 88

Slide 88 text

detect which input the users is using right now...

Slide 89

Slide 89 text

GitHub: ten1seven / what-input

Slide 90

Slide 90 text

Demo: What Input?

Slide 91

Slide 91 text

if in doubt, offer a way to switch interfaces... or just always use a touch-optimised interface

Slide 92

Slide 92 text

Slide 93

Slide 93 text

touch events vs limitations/problems

Slide 94

Slide 94 text

1. delayed event dispatch 2. mousemove doesn't track

Slide 95

Slide 95 text

1. delayed event dispatch 2. mousemove doesn't track

Slide 96

Slide 96 text

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html less of a problem in modern browsers, but for iOS < 9.3 or older Androids still relevant

Slide 97

Slide 97 text

why the delay? double-tap to zoom (mostly anyway)

Slide 98

Slide 98 text

when does the delay happen?

Slide 99

Slide 99 text

patrickhlauke.github.io/touch/tests/event-listener.html

Slide 100

Slide 100 text

patrickhlauke.github.io/touch/tests/event-listener.html

Slide 101

Slide 101 text

touch / mouse events delay touchstart > [touchmove]+ > touchend > [300ms delay] (mouseenter) > mouseover > mousemove > mousedown > (focus) > mouseup > click

Slide 102

Slide 102 text

“how can we make it feel responsive like a native app?”

Slide 103

Slide 103 text

react to events ﬁred before the 300ms delay...

Slide 104

Slide 104 text

touchstart for an “immediate” control (e.g. ﬁre/jump button on a game)

Slide 105

Slide 105 text

touchend for a control that fires after finger lifted (but this can result in events firing after zoom/scroll)

Slide 106

Slide 106 text

don't make it touch-exclusive

Slide 107

Slide 107 text

/* DON'T DO THIS: conditional "touch OR mouse/keyboard" event binding */ if ('ontouchstart' in window) { // set up event listeners for touch foo.addEventListener('touchend', ...); ... } else { // set up event listeners for mouse/keyboard foo.addEventListener('click', ...); ... }

Slide 108

Slide 108 text

patrickhlauke.github.io/touch/tests/event-listener_naive-touch-or-mouse.html

Slide 109

Slide 109 text

/* DO THIS: doubled-up "touch AND mouse/keyboard" event binding */ // set up event listeners for touch foo.addEventListener('touchend', ...); // set up event listeners for mouse/keyboard foo.addEventListener('click', ...); /* but this would fire our function twice for touch? */ patrickhlauke.github.io/touch/tests/event-listener_naive-event-doubling.html

Slide 110

Slide 110 text

/* DO THIS: doubled-up "touch AND mouse/keyboard" event binding */ // set up event listeners for touch foo.addEventListener('touchend', function(e) { // prevent compatibility mouse events and click e.preventDefault(); ... }); // set up event listeners for mouse/keyboard foo.addEventListener('click', ...); patrickhlauke.github.io/touch/tests/event-listener_naive-event-doubling-ﬁxed.html

Slide 111

Slide 111 text

preventDefault() kills scrolling, pinch/zoom, etc

Slide 112

Slide 112 text

apply preventDefault() carefully (just on buttons/links, not entire page)

Slide 113

Slide 113 text

github.com/ftlabs/fastclick

Slide 114

Slide 114 text

patrickhlauke.github.io/touch/fastclick YouTube: iOS/Safari 300ms click delay: vanilla Bootstrap and using fastclick.js

Slide 115

Slide 115 text

patrickhlauke.github.io/touch/fastclick/fastclick.html YouTube: iOS/Safari 300ms click delay: vanilla Bootstrap and using fastclick.js

Slide 116

Slide 116 text

browsers working to remove double-tap to zoom delay

Slide 117

Slide 117 text

non-scalable/zoomable viewport and "double-tap to zoom"

Slide 118

Slide 118 text

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

Slide 119

Slide 119 text

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

Slide 120

Slide 120 text

... content="minimum-scale=1, maximum-scale=1" patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

Slide 121

Slide 121 text

... content="minimum-scale=1, maximum-scale=1" patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

Slide 122

Slide 122 text

Changeset 191072 - Web pages with unscalable viewports shouldn't have a single tap delay (from iOS 9.3 onwards)

Slide 123

Slide 123 text

what about accessibility?

Slide 124

Slide 124 text

Chrome: Settings > Accessibility > Force enable zoom

Slide 125

Slide 125 text

Opera: Settings > Force enable zoom

Slide 126

Slide 126 text

Firefox: Settings > Accessibility > Always enable zoom

Slide 127

Slide 127 text

Samsung Internet: Internet settings > Manual zoom

Slide 128

Slide 128 text

"Force enable zoom" reintroduces delay – a fair compromise? patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

Slide 129

Slide 129 text

@thomasfuchs - Safari/iOS10beta3 ignores unscalable viewports (no ofﬁcial Apple documentation about the change...but the 300ms delay is back)

Slide 130

Slide 130 text

Windows 10 build 15007 on Mobile ignores unscalable viewports

Slide 131

Slide 131 text

"mobile optimised" viewport and "double-tap to zoom"

Slide 132

Slide 132 text

Chrome 32+ / Android: content="width=device-width" suppresses double-tap-to-zoom, still allows pinch zoom Google Developers: 300ms tap delay, gone away

Slide 133

Slide 133 text

Bug 941995 - Remove 300ms [...] on "responsive" pages [RESOLVED FIXED]

Slide 134

Slide 134 text

Bug 150604 - Implement viewport-width-based fast-click heuristic (from iOS 9.3 onwards)

Slide 135

Slide 135 text

Bug 151077 - Fast-clicking should trigger when scale is equal to explicitly set initial scale (from iOS 9.3 onwards)

Slide 136

Slide 136 text

WebKit blog: More Responsive Tapping on iOS

Slide 137

Slide 137 text

patrickhlauke.github.io/touch/tests/results/#suppressing-300ms-delay

Slide 138

Slide 138 text

suppressing 300ms delay if your code does rely on handling click / mouse events: • "mobile optimised" • add touch-action:manipulation in CSS (part of Pointer Events) • use helper library like fastclick.js for older browsers • make your viewport non-scalable...

Slide 139

Slide 139 text

1. delayed event dispatch 2. mousemove doesn't track

Slide 140

Slide 140 text

patrickhlauke.github.io/touch/particle/2

Slide 141

Slide 141 text

patrickhlauke.github.io/touch/particle/2

Slide 142

Slide 142 text

events ﬁred on tap touchstart > [touchmove]+ > touchend > (mouseenter) > mouseover > mousemove* > mousedown > (focus) > mouseup > click * mouse event emulation ﬁres only a single mousemove too many touchmove events prevent mouse compatibility events after touchend

Slide 143

Slide 143 text

doubling up handling of mousemove and touchmove

Slide 144

Slide 144 text

var posX, posY; function positionHandler(e) { posX = e.clientX ; posY = e.clientY ; } canvas.addEventListener(' mousemove ', positionHandler, false);

Slide 145

Slide 145 text

var posX, posY; function positionHandler(e) { posX = e.clientX ; posY = e.clientY ; } canvas.addEventListener(' mousemove ', positionHandler, false); canvas.addEventListener(' touchmove ', positionHandler, false); /* but this won't work for touch... */

Slide 146

Slide 146 text

interface MouseEvent : UIEvent { readonly attribute long screenX ; readonly attribute long screenY ; readonly attribute long clientX ; readonly attribute long clientY ; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute unsigned short button; readonly attribute EventTarget relatedTarget; // [DOM4] UI Events readonly attribute unsigned short buttons; }; www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent www.w3.org/TR/uievents/#events-mouseevents

Slide 147

Slide 147 text

partial interface MouseEvent { readonly attribute double screenX; readonly attribute double screenY; readonly attribute double pageX ; readonly attribute double pageY ; readonly attribute double clientX; readonly attribute double clientY; readonly attribute double x ; readonly attribute double y ; readonly attribute double offsetX ; readonly attribute double offsetY ; }; www.w3.org/TR/cssom-view/#extensions-to-the-mouseevent- interface

Slide 148

Slide 148 text

interface TouchEvent : UIEvent { readonly attribute TouchList touches ; readonly attribute TouchList targetTouches ; readonly attribute TouchList changedTouches ; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; }; www.w3.org/TR/touch-events/#touchevent-interface

Slide 149

Slide 149 text

interface Touch { readonly attribute long identifier; readonly attribute EventTarget target; readonly attribute long screenX ; readonly attribute long screenY ; readonly attribute long clientX ; readonly attribute long clientY ; readonly attribute long pageX ; readonly attribute long pageY ; }; www.w3.org/TR/touch-events/#touch-interface

Slide 150

Slide 150 text

TouchList differences touches all touch points on screen targetTouches all touch points that started on the element changedTouches touch points that caused the event to ﬁre

Slide 151

Slide 151 text

changedTouches depending on event: • for touchstart , all new touch points that became active • for touchmove , all touch points that changed/moved since last event • for touchend / touchcancel , touch points that were removed (and are not active anymore at this point)

Slide 152

Slide 152 text

patrickhlauke.github.io/touch/touchlist-objects

Slide 153

Slide 153 text

var posX, posY; function positionHandler(e) { if ((e.clientX)&&(e.clientY)) { posX = e.clientX; posY = e.clientY; } else if (e.targetTouches) { posX = e.targetTouches[0].clientX; posY = e.targetTouches[0].clientY; e.preventDefault() ; } } canvas.addEventListener('mousemove', positionHandler, false ); canvas.addEventListener('touchmove', positionHandler, false );

Slide 154

Slide 154 text

TouchList collections order • order of individual Touch objects in a TouchList can change • e.g. targetTouches[0] not guaranteed to always be the same finger when dealing with multitouch • not problematic for single-finger interactions, but use identifier property for each Touch to explicitly track a particular touch point / finger in multitouch

Slide 155

Slide 155 text

patrickhlauke.github.io/touch/particle/3

Slide 156

Slide 156 text

patrickhlauke.github.io/touch/particle/4a

Slide 157

Slide 157 text

patrickhlauke.github.io/touch/paranoid_0.5 www.splintered.co.uk/experiments/archives/paranoid_0.5

Slide 158

Slide 158 text

patrickhlauke.github.io/touch/picture-slider

Slide 159

Slide 159 text

implicit capture • touch events have implicit capture: once you start a touch movement on an element, events keep ﬁring to the element even when moving outside the element's boundaries • compare to mouse events, which require tricks to track mouse

Slide 160

Slide 160 text

(3) Naive mouse-driven fake slider (doesn't "capture") Demo: without tricks, slider won't work when moving mouse outside

Slide 161

Slide 161 text

(4) Basic mouse-driven fake slider Exercise/demo: expand mouse-driven code to also work with touch

Slide 162

Slide 162 text

(5) Basic mouse- and touch-driven fake slider

Slide 163

Slide 163 text

tracking ﬁnger movement over time ... swipe gestures

Slide 164

Slide 164 text

patrickhlauke.github.io/touch/swipe

Slide 165

Slide 165 text

patrickhlauke.github.io/touch/swipe

Slide 166

Slide 166 text

/* Swipe detection from basic principles */ Δt = end.time - start.time; Δx = end.x - start.x; Δy = end.y - start.y; if (( Δt > timeThreshold ) || ( (Δx + Δy) < distanceThreshold )) { /* too slow or movement too small */ } else { /* it's a swipe! - use Δx and Δy to determine direction - pythagoras √( Δx² + Δy² ) for distance - distance/Δt to determine speed */ }

Slide 167

Slide 167 text

don't forget mouse/keyboard!

Slide 168

Slide 168 text

bradfrostweb.com/demo/mobile-ﬁrst

Slide 169

Slide 169 text

touchmove ﬁres...a lot!

Slide 170

Slide 170 text

do absolute minimum on each touchmove (usually: store coordinates)

Slide 171

Slide 171 text

do heavy lifting (drawing etc.) separately, avoid queueing (e.g. using setTimeout and/or requestAnimationFrame )

Slide 172

Slide 172 text

debounce / throttle events

Slide 173

Slide 173 text

GitHub: m-gagne / limit.js

Slide 174

Slide 174 text

Function.prototype.debounce = function (milliseconds, context) { var baseFunction = this, timer = null, wait = milliseconds; return function () { var self = context || this, args = arguments; function complete() { baseFunction.apply(self, args); timer = null; } if (timer) { clearTimeout(timer); } timer = setTimeout(complete, wait); }; }; window.addEventListener('touchmove', myFunction.debounce(250) );

Slide 175

Slide 175 text

Function.prototype.throttle = function (milliseconds, context) { var baseFunction = this, lastEventTimestamp = null, limit = milliseconds; return function () { var self = context || this, args = arguments, now = Date.now(); if (!lastEventTimestamp || now - lastEventTimestamp >= limit) { lastEventTimestamp = now; baseFunction.apply(self, args); } }; }; window.addEventListener('touchmove', myFunction.throttle(250) );

Slide 176

Slide 176 text

patrickhlauke.github.io/touch/touch-limit

Slide 177

Slide 177 text

why stop at a single point? multitouch support

Slide 178

Slide 178 text

Slide 179

Slide 179 text

/* iterate over relevant TouchList */ for (i=0; i< e.targetTouches .length; i++) { ... posX = e.targetTouches[i].clientX ; posY = e.targetTouches[i].clientY ; ... }

Slide 180

Slide 180 text

patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html

Slide 181

Slide 181 text

iOS/iPad preventDefault() can't override 4-ﬁnger gestures

Slide 182

Slide 182 text

iOS7+ Safari/WebView preventDefault() can't override edge swipes

Slide 183

Slide 183 text

YouTube: iOS/Safari (iPhone) can't prevent edge swipe gestures

Slide 184

Slide 184 text

Chrome/iOS (WebView) custom swipe, also can't be prevented patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html

Slide 185

Slide 185 text

multitouch gestures

Slide 186

Slide 186 text

/* iOS/Safari/WebView has gesture events for size/rotation, not part of the W3C Touch Events spec */ /* gesturestart / gesturechange / gestureend */ foo.addEventListener('gesturechange', function(e) { /* e.scale e.rotation */ /* values can be plugged directly into CSS transforms etc */ }); /* not supported in other browsers, but potentially useful for iOS exclusive content (hybrid apps/webviews) */ iOS Developer Library - Safari Web Content Guide - Handling gesture events

Slide 187

Slide 187 text

patrickhlauke.github.io/touch/iosgestures

Slide 188

Slide 188 text

patrickhlauke.github.io/touch/iosgestures/image.html

Slide 189

Slide 189 text

/* Pinch/rotation detection from basic principles */ Δx = x2 - x1; Δy = y2 - y1; /* pythagoras √( Δx² + Δy² ) to calculate distance */ Δd = Math.sqrt( (Δx * Δx) + (Δy * Δy) ); /* trigonometry to calculate angle */ α = Math.atan2( Δy, Δx ); Mozilla Developer Network: Math.atan2()

Slide 190

Slide 190 text

patrickhlauke.github.io/touch/pinch-zoom-rotate

Slide 191

Slide 191 text

patrickhlauke.github.io/touch/picture-organiser

Slide 192

Slide 192 text

not all old/cheap devices/OSs support multitouch!

Slide 193

Slide 193 text

HTC Hero – Android 2.1

Slide 194

Slide 194 text

LG Optimus 2X – Android 2.3.7

Slide 195

Slide 195 text

ZTE Open – Firefox OS 1.1

Slide 196

Slide 196 text

Touch Events speciﬁcation grey areas...

Slide 197

Slide 197 text

do touch events ﬁre during scroll/zoom?

Slide 198

Slide 198 text

different models and behaviours, particularly in old browsers YouTube: Touch Events and scrolling/zooming (playlist)

Slide 199

Slide 199 text

e.g. older versions of Chrome ﬁre touchcancel on scroll/zoom YouTube: Google Developers - Mobile Web Thursdays: Performance on Mobile

Slide 200

Slide 200 text

not deﬁned in spec (yet), but de facto yes in most modern browsers patrickhlauke.github.io/touch/gesture-touch

Slide 201

Slide 201 text

touchmove and scroll • useful for common interactions like pull to refresh – let browser (over)scroll and track movement • has scroll performance impact, as browser is blocked on event listener in case it cancels the scroll (see passive event listeners)

Slide 202

Slide 202 text

non-standard iOS quirks...

Slide 203

Slide 203 text

making generic elements clickable...

Slide 204

Slide 204 text

Historically (until end of 2016?), Apple suggested adding dummy onclick="" handlers iOS Developer Library - Safari Web Content Guide - Handling Events

Slide 205

Slide 205 text

dummy onclick="" handlers are unnecessary (at least since iOS6) Apple quietly dropped this from their documentation... patrickhlauke.github.io/touch/ios-clickable/

Slide 206

Slide 206 text

mouse + click event bubbling on touch (important when using event delegation)

Slide 207

Slide 207 text

Quirksmode: Mouse event bubbling in iOS

Slide 208

Slide 208 text

patrickhlauke.github.io/touch/bubbling/button.html

Slide 209

Slide 209 text

patrickhlauke.github.io/touch/bubbling/link.html

Slide 210

Slide 210 text

patrickhlauke.github.io/touch/bubbling/div.html

Slide 211

Slide 211 text

patrickhlauke.github.io/touch/bubbling/div-onclick-target.html

Slide 212

Slide 212 text

patrickhlauke.github.io/touch/bubbling/div-onclick-ancestor.html

Slide 213

Slide 213 text

patrickhlauke.github.io/touch/bubbling/div-addeventlistener-target.html

Slide 214

Slide 214 text

patrickhlauke.github.io/touch/bubbling/div-addeventlistener-ancestor.html

Slide 215

Slide 215 text

patrickhlauke.github.io/touch/bubbling/div-cursorpointer-target.html

Slide 216

Slide 216 text

patrickhlauke.github.io/touch/bubbling/div-cursorpointer-ancestor.html

Slide 217

Slide 217 text

mouse + click bubbling in iOS • target element is a link or form control • target or any ancestor (up to but not including body ) has explicit mouse or click handler (even if only empty function) • target or any ancestor (up to and including document ) has cursor:pointer Quirksmode: Mouse event bubbling in iOS

Slide 218

Slide 218 text

iOS quirks in a nutshell if your code does rely on handling mouse events: • you don't need dummy click handlers to make arbitrary (non- interactive) elements react to mouse events (such as mouseover , mousedown , mouseup ...) • you do need to do extra work if you rely on event bubbling / event delegation of mouse events (including click )

Slide 219

Slide 219 text

Touch Events extensions...

Slide 220

Slide 220 text

W3C Touch Events Extensions WG Note

Slide 221

Slide 221 text

/* Extension to touch objects */ partial interface Touch { readonly attribute float radiusX; readonly attribute float radiusY; readonly attribute float rotationAngle; readonly attribute float force; };

Slide 222

Slide 222 text

Google Developers: Using rotationAngle and touchRadius

Slide 223

Slide 223 text

/* Touch Events – contact geometry */ partial interface Touch { readonly attribute float radiusX ; readonly attribute float radiusY ; readonly attribute float rotationAngle ; readonly attribute float force; };

Slide 224

Slide 224 text

patrickhlauke.github.io/touch/tracker/tracker-radius-rotationangle.html YouTube: Touch Events: radiusX, radiusY and rotationAngle

Slide 225

Slide 225 text

Rick Byers - Paint (with rotationAngle and touchRadius)

Slide 226

Slide 226 text

/* Touch Events – force */ partial interface Touch { readonly attribute float radiusX; readonly attribute float radiusY; readonly attribute float rotationAngle; readonly attribute float force ; }; force : value in range 0 – 1 . if no hardware support 0 (some devices, e.g. Nexus 10, "fake" force based on radiusX / radiusY )

Slide 227

Slide 227 text

patrickhlauke.github.io/touch/tracker/tracker-force-pressure.html iPhone 6s with 3D Touch supports force ... (Safari and WKWebView, e.g. Chrome/iOS, but not UIWebView, e.g. Firefox/iOS)

Slide 228

Slide 228 text

patrickhlauke.github.io/touch/tracker/tracker-force-pressure.html ...while in non-3D Touch models (e.g. iPhone 5c) force == 0

Slide 229

Slide 229 text

3D Touch ≠ Force Touch • Force Touch is Apple's proprietary extension to mouse events • only aimed at force-enabled trackpads (new Apple models) • new events: webkitmouseforcewillbegin , webkitmouseforcedown , webkitmouseforceup , webkitmouseforcechanged • mouse events have additional webkitForce property Safari Developer Library: Responding to Force Touch Events

Slide 230

Slide 230 text

Google Developers: Precision Touch for Precise Gestures

Slide 231

Slide 231 text

Bug 133180 - Touch events should allow for device-pixel accuracy

Slide 232

Slide 232 text

interface Touch { readonly attribute long identifier; readonly attribute EventTarget target; readonly attribute long float screenX; readonly attribute long float screenY; readonly attribute long float clientX; readonly attribute long float clientY; readonly attribute long float pageX; readonly attribute long float pageY; };

Slide 233

Slide 233 text

W3C Touch Events Community Group

Slide 234

Slide 234 text

W3C Web Events WG - Touch Events errata (early effort to clarify grey areas, simplify language used)

Slide 235

Slide 235 text

W3C Touch Events Level 2 (Editor's Draft) (merges errata, touch events extensions, fractional touch coordinates)

Slide 236

Slide 236 text

Pointer Events

Slide 237

Slide 237 text

up to IE9 (Win7 / WinPhone7.5) only mouse events

Slide 238

Slide 238 text

in IE10 Microsoft introduced Pointer Events

Slide 239

Slide 239 text

uniﬁes mouse, touch and pen input into a single event model

Slide 240

Slide 240 text

...and some new inputs (though currently mapped to mouse) Building Xbox One Apps using HTML and JavaScript YouTube: Xbox One Edge Browser sends Pointer Events

Slide 241

Slide 241 text

not just some “not invented here” technology

Slide 242

Slide 242 text

submitted by Microsoft as W3C Candidate REC 09 May 2013

Slide 243

Slide 243 text

Pointer Events - W3C REC 24 February 2015

Slide 244

Slide 244 text

work now continues to enhance/expand Pointer Events... W3C Pointer Events Level 2 (Editor's Draft)

Slide 245

Slide 245 text

caniuse.com: Can I use pointer events?

Slide 246

Slide 246 text

Bug 822898 - Implement pointer events

Slide 247

Slide 247 text

about:config / dom.w3c_pointer_events.enabled

Slide 248

Slide 248 text

...what about Apple?

Slide 249

Slide 249 text

Microsoft submitted a proof of concept WebKit patch html5labs.interoperabilitybridges.com/prototypes/...

Slide 250

Slide 250 text

Bug 105463 - Implement pointer events RESOLVED WONTFIX

Slide 251

Slide 251 text

Maciej Stachowiak - [webkit-dev] pointer events speciﬁcation - ﬁrst editors draft

Slide 252

Slide 252 text

@patrick_h_lauke paraphrasing Apple's stance on Pointer Events...

Slide 253

Slide 253 text

No content

Slide 254

Slide 254 text

Apple Pencil currently indistinguishable from touch in browser / JavaScript (no tilt information, ﬁres touch events)

Slide 255

Slide 255 text

the anatomy of Pointer Events (sequence, event object, ...)

Slide 256

Slide 256 text

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

Slide 257

Slide 257 text

events fired on tap (Edge) mousemove* > pointerover > mouseover > pointerenter > mouseenter > pointerdown > mousedown > focus gotpointercapture > pointermove > mousemove > pointerup > mouseup > lostpointercapture > click > pointerout > mouseout > pointerleave > mouseleave mouse events fired “inline” with pointer events (for a primary pointer, e.g. first finger on screen)

Slide 258

Slide 258 text

IE10/IE11 quirks • vendor-prefixed in IE10 ( MSPointerDown ..., navigator.msMaxTouchPoints , -ms-touch-action ) • unprefixed in IE11 (but prefixed versions mapped for compatibility) • event order (when click is fired) incorrect in IE10/IE11 see W3C Pointer Events WG mailing list - Jacob Rossi (Microsoft)

Slide 259

Slide 259 text

Chrome, Edge (on mobile), IE11 (Windows Phone 8.1update1) support both Touch Events and Pointer Events w3c.github.io/pointerevents/#mapping-for-devices-that-do-not-support-hover

Slide 260

Slide 260 text

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

Slide 261

Slide 261 text

about:flags in Microsoft Edge to turn on touch events on desktop (e.g. touch-enabled laptops) – off by default for site compatibility

Slide 262

Slide 262 text

... when touch events also supported (Edge) pointerover > pointerenter > pointerdown > touchstart > pointerup > touchend > mouseover > mouseenter > mousemove > mousedown > focus > mouseup > click > pointerout > pointerleave Speciﬁc order of events is not deﬁned in the spec in this case – will vary between browsers... only problem if handling both pointer events and mouse events (which is nonsensical)

Slide 263

Slide 263 text

/* Pointer Events extend Mouse Events vs Touch Events and their new objects / TouchLists / Touches */ interface PointerEvent : MouseEvent { readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure; readonly attribute float tangentialPressure; /* Level 2 */ readonly attribute long tiltX; readonly attribute long tiltY; readonly attribute long twist; /* Level 2 */ readonly attribute DOMString pointerType; readonly attribute boolean isPrimary; }

Slide 264

Slide 264 text

handling mouse input is exactly the same as traditional mouse events (only change pointer* instead of mouse* event names)

Slide 265

Slide 265 text

handling touch or stylus is also exactly the same as traditional mouse events (mouse code can be adapted to touch/stylus quickly)

Slide 266

Slide 266 text

simple feature detection for pointer events

Slide 267

Slide 267 text

/* detecting pointer events support */ if ( window.PointerEvent ) { /* some clever stuff here but this covers touch, stylus, mouse, etc */ } /* still listen to click for keyboard! */

Slide 268

Slide 268 text

"don't forget about keyboard users" note in Pointer Events spec

Slide 269

Slide 269 text

/* detect maximum number of touch points */ if ( navigator.maxTouchPoints > 0 ) { /* device with a touchscreen */ } if ( navigator.maxTouchPoints > 1 ) { /* multitouch-capable device */ } "you can detect a touchscreen" ...but user may still use other input mechanisms

Slide 270

Slide 270 text

patrickhlauke.github.io/touch/tests/pointer-multitouch-detect.html

Slide 271

Slide 271 text

do pointer events ﬁre during scroll/zoom?

Slide 272

Slide 272 text

w3c.github.io/pointerevents/#the-pointercancel-event

Slide 273

Slide 273 text

once a browser handles scrolling, it sends pointercancel patrickhlauke.github.io/touch/gesture-touch/pointerevents.html

Slide 274

Slide 274 text

pointer events vs limitations/problems of mouse event emulation

Slide 275

Slide 275 text

1. delayed event dispatch 2. mousemove doesn't track

Slide 276

Slide 276 text

1. delayed event dispatch 2. mousemove doesn't track

Slide 277

Slide 277 text

patrickhlauke.github.io/touch/tests/event-listener.html (IE11/WinPhone 8.1 Update no optimization for width=device-width )

Slide 278

Slide 278 text

patrickhlauke.github.io/touch/tests/event-listener.html (IE/Win8 has double-tap to zoom, so problem on desktop too)

Slide 279

Slide 279 text

patrickhlauke.github.io/touch/tests/event-listener.html (Microsoft Edge/Win10 has double-tap to zoom, so problem on desktop too)

Slide 280

Slide 280 text

pointer / mouse events and delay ... [300ms delay] click ... 300ms delay just before click event

Slide 281

Slide 281 text

“how can we make it feel responsive like a native app?”

Slide 282

Slide 282 text

we could try a similar approach to touch events...

Slide 283

Slide 283 text

• double-up pointerup and click listeners? • prevent code ﬁring twice with preventDefault ? won't work: preventDefault() stops mouse compatibility events, but click is not considered mouse compatibility event

Slide 284

Slide 284 text

patrickhlauke.github.io/touch/tests/event-listener.html

Slide 285

Slide 285 text

a more declarative approach with touch-action

Slide 286

Slide 286 text

touch-action what action should the browser handle? touch-action: auto /* default */ touch-action: none touch-action: pan-x touch-action: pan-y touch-action: manipulation /* pan/zoom */ touch-action: pan-x pan-y /* can be combined */ www.w3.org/TR/pointerevents/#the-touch-action-css-property only determines default touch action, does not stop compatibility mouse events nor click

Slide 287

Slide 287 text

Pointer Events Level 2 expanded set of values (useful for pull-to-refresh, carousels, etc) touch-action: pan-left touch-action: pan-right touch-action: pan-up touch-action: pan-down touch-action: pan-left pan-down /* can be combined */ touch-action: pan-x pan-down /* can be combined */ w3c.github.io/pointerevents/#the-touch-action-css-property new values only determine allowed pan direction at the start of the interaction; once panning starts, user can also move in opposite direction along same axis

Slide 288

Slide 288 text

compat.spec.whatwg.org adds extra value pinch-zoom

Slide 289

Slide 289 text

patrickhlauke.github.io/touch/touch-action-scrolling

Slide 290

Slide 290 text

although it's called "touch-action", it applies to any pointer type that pans/zooms (e.g. Samsung Note + stylus, Chrome <58/Android + mouse) w3c.github.io/pointerevents/#declaring-candidate-regions-for-default-touch-behaviors

Slide 291

Slide 291 text

touch-action:none (suppress all default browser behaviour)

Slide 292

Slide 292 text

patrickhlauke.github.io/touch/tests/event-listener_touch[...]

Slide 293

Slide 293 text

patrickhlauke.github.io/touch/tests/event-listener_touch[...]

Slide 294

Slide 294 text

touch-action:none kills scrolling, long-press, pinch/zoom

Slide 295

Slide 295 text

touch-action:manipulation (suppress double-tap-to-zoom)

Slide 296

Slide 296 text

patrickhlauke.github.io/touch/tests/event-listener_touch[...]

Slide 297

Slide 297 text

patrickhlauke.github.io/touch/tests/event-listener_touch[...]

Slide 298

Slide 298 text

caniuse.com: Can I use touch-action?

Slide 299

Slide 299 text

browsers working to remove double-tap to zoom delay (when page not zoomable)

Slide 300

Slide 300 text

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

Slide 301

Slide 301 text

No content

Slide 302

Slide 302 text

"Allow zooming on all web content" reintroduces delay patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

Slide 303

Slide 303 text

"Allow zooming..." option currently missing in Win 10 Mobile

Slide 304

Slide 304 text

Windows 10 build 15007 on Mobile ignores unscalable viewports no delay until user zooms for ﬁrst time...

Slide 305

Slide 305 text

width=device-width still has delay in IE11/WinPhone 8.1 Update patrickhlauke.github.io/touch/tests/event-listener_width-device-width.html

Slide 306

Slide 306 text

Microsoft Edge removes delay for width=device-width patrickhlauke.github.io/touch/tests/event-listener_width-device-width.html

Slide 307

Slide 307 text

patrickhlauke.github.io/touch/tests/results/#suppressing-300ms-delay

Slide 308

Slide 308 text

1. delayed event dispatch 2. mousemove doesn't track

Slide 309

Slide 309 text

patrickhlauke.github.io/touch/particle/2 mousemove / pointermove ﬁre, but browser scroll action takes over

Slide 310

Slide 310 text

you can "fake" it with touch-action:none and listen for mouse events...

Slide 311

Slide 311 text

patrickhlauke.github.io/touch/particle/2a (does not work in Microsoft Edge/Windows 10 Mobile due to touch events support)

Slide 312

Slide 312 text

better: just listen to pointermove...

Slide 313

Slide 313 text

no need for separate mouse or touch event listeners

Slide 314

Slide 314 text

/* touch events: separate handling */ foo.addEventListener('touchmove', ... , false); foo.addEventListener('mousemove', ... , false); /* pointer events: single listener for mouse, stylus, touch */ foo.addEventListener(' pointermove ', ... , false);

Slide 315

Slide 315 text

no need for separate mouse or touch code to get x / y coords

Slide 316

Slide 316 text

/* Reminder: Touch Events need separate code for x / y coords */ function positionHandler(e) { if ((e.clientX)&&(e.clientY)) { posX = e.clientX; posY = e.clientY; } else if (e.targetTouches) { posX = e.targetTouches[0].clientX; posY = e.targetTouches[0].clientY; ... } } canvas.addEventListener('mousemove', positionHandler, false ); canvas.addEventListener('touchmove', positionHandler, false );

Slide 317

Slide 317 text

/* Pointer Events extend Mouse Events */ foo.addEventListener(' pointermove ', function(e) { ... posX = e.clientX ; posY = e.clientY ; ... }, false); www.w3.org/TR/pointerevents/#pointerevent-interface

Slide 318

Slide 318 text

3D Rotator by Creative Punch coded to only use mouse events

Slide 319

Slide 319 text

3D Rotator modiﬁed to use Pointer Events minimal code changes, as Pointer Events extend mouse events

Slide 320

Slide 320 text

(4) Basic mouse-driven fake slider Exercise/demo: convert mouse-driven code to use Pointer Events

Slide 321

Slide 321 text

(6) Basic pointer events-driven fake slider

Slide 322

Slide 322 text

but you can distinguish mouse or touch or stylus

Slide 323

Slide 323 text

foo.addEventListener('pointermove', function(e) { ... switch( e.pointerType ) { case ' mouse ': ... break; case ' pen ': ... break; case ' touch ': ... break; default : /* future-proof */ } ... } , false);

Slide 324

Slide 324 text

/* in IE11/Edge, pointerType returns a string in IE10, the return type is long */ MSPOINTER_TYPE_TOUCH: 0x00000002 MSPOINTER_TYPE_PEN: 0x00000003 MSPOINTER_TYPE_MOUSE: 0x00000004 MSDN: IE Dev Center - API reference - pointerType property

Slide 325

Slide 325 text

what about multitouch?

Slide 326

Slide 326 text

/* PointerEvents don't have the handy TouchList objects, so we have to replicate something similar... */ var points = []; switch (e.type) { case ' pointerdown ': /* add to the array */ break; case ' pointermove ': /* update the relevant array entry's x and y */ break; case ' pointerup ': case ' pointercancel ': /* remove the relevant array entry */ break; }

Slide 327

Slide 327 text

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html (note multiple isPrimary pointers, per pointer type)

Slide 328

Slide 328 text

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer-hud.html

Slide 329

Slide 329 text

simultaneous use of inputs is hardware-dependent (e.g. Surface 3 "palm blocking" prevents concurrent touch/stylus/mouse, but not touch/external mouse/external stylus)

Slide 330

Slide 330 text

YouTube: Pointer Events support multiple stylus/pen inputs simultaneously

Slide 331

Slide 331 text

/* like iOS/Safari, IE/Edge have higher-level gestures , but these are not part of the W3C Pointer Events spec */ foo.addEventListener("MSGestureStart", ... , false) foo.addEventListener("MSGestureEnd", ... , false) foo.addEventListener("MSGestureChange", ... , false) foo.addEventListener("MSInertiaStart", ... , false) foo.addEventListener("MSGestureTap", ... , false) foo.addEventListener("MSGestureHold", ... , false) /* not supported in other browsers, but potentially useful for IE/Edge exclusive content (e.g. UWP packaged/hosted/hybrid web app); for web, replicate these from basic principles again... */ MSDN IE10 Developer Guide: Gesture object and events

Slide 332

Slide 332 text

extended capabilities (if supported by hardware)

Slide 333

Slide 333 text

/* Pointer Events - pressure */ interface PointerEvent : MouseEvent { readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure ; readonly attribute float tangentialPressure; readonly attribute long tiltX; readonly attribute long tiltY; readonly attribute long twist; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary; } pressure : value in range 0 – 1 . if no hardware support, 0.5 in active button state, 0 otherwise

Slide 334

Slide 334 text

patrickhlauke.github.io/touch/tracker/... YouTube: Touch tracker with Surface 3 pen

Slide 335

Slide 335 text

hovering stylus • hardware-dependant • pointermove ﬁres • pressure == 0 (non-active button state) • track pointerdown / pointerup to be safe

Slide 336

Slide 336 text

/* Pointer Events - contact geometry */ interface PointerEvent : MouseEvent { readonly attribute long pointerId; readonly attribute long width ; readonly attribute long height ; readonly attribute float pressure; readonly attribute float tangentialPressure; readonly attribute long tiltX; readonly attribute long tiltY; readonly attribute long twist; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary; } if hardware can't detect contact geometry, either 0 or "best guess" (e.g. for mouse/stylus, return width / height of 1 )

Slide 337

Slide 337 text

patrickhlauke.github.io/touch/tracker/tracker-width-height.html YouTube: Pointer Events width and height ...

Slide 338

Slide 338 text

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer-hud.html (Nokia Lumia 520: pressure is 0.5 - active state - and width / height is 0 )

Slide 339

Slide 339 text

/* Pointer Events - tilt */ interface PointerEvent : MouseEvent { readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure; readonly attribute float tangentialPressure; readonly attribute long tiltX ; readonly attribute long tiltY ; readonly attribute long twist; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary; } tiltX / tiltY : value in degrees -90 – 90 . returns 0 if hardware does not support tilt

Slide 340

Slide 340 text

patrickhlauke.github.io/touch/pen-tracker YouTube: ...pen with tilt, pressure and hover support

Slide 341

Slide 341 text

patrickhlauke.github.io/touch/pen-tracker YouTube: Chrome on Samsung Galaxy Note 4 - Pointer Events and stylus

Slide 342

Slide 342 text

pointermove ﬁres if any property changes, not just x/y position ( width , height , tiltX , tiltY , pressure )

Slide 343

Slide 343 text

pointer capture (implicit vs explicit)

Slide 344

Slide 344 text

• touch events have implicit capture: once you start a touch movement on an element, events keep ﬁring to the element even when moving outside the element's boundaries • for mouse / stylus: pointer events like mouse events: only ﬁre events to an element while pointer inside it (but allow explicit capture) • for touch: pointer events level 2 use implicit capture (for compatibility with touch events) Pointer Events Level 2: Implicit Pointer Capture

Slide 345

Slide 345 text

/* events related to pointer capture */ foo.addEventListener("gotpointercapture", ... , false) foo.addEventListener("lostpointercapture", ... , false) /* methods related to pointer capture */ element.setPointerCapture(pointerId) element.releasePointerCapture(pointerId) if (element.hasPointerCapture(pointerId)) { ... }

Slide 346

Slide 346 text

/* example of how to capture a pointer explicitly */ element.addEventListener('pointerdown', function(e) { this. setPointerCapture(e.pointerId) ; }, false } /* capture automatically released on pointerup / pointercancel or explicitly with releasePointerCapture() */ patrickhlauke.github.io/touch/tests/pointercapture.html

Slide 347

Slide 347 text

setPointerCapture can simplify mouse input e.g. draggable interfaces/sliders, use instead of "bind mousemove to document / body on mousedown "

Slide 348

Slide 348 text

(6) Basic pointer events-driven fake slider Exercise/demo: simplify code to use pointer capture

Slide 349

Slide 349 text

(7) Basic pointer events-driven fake slider with setPointerCapture

Slide 350

Slide 350 text

pointer events as the future?

Slide 351

Slide 351 text

transitional event handling (until all browsers support pointer events)

Slide 352

Slide 352 text

/* cover all cases (hat-tip Stu Cox) */ if (window.PointerEvent) { /* bind to Pointer Events: pointerdown, pointerup, etc */ } else { /* bind to mouse events: mousedown, mouseup, etc */ if ('ontouchstart' in window) { /* bind to Touch Events: touchstart, touchend, etc */ } } /* bind to keyboard / click */

Slide 353

Slide 353 text

polyﬁlls for pointer events (code for tomorrow, today)

Slide 354

Slide 354 text

GitHub - jQuery Pointer Events Polyﬁll (PEP)

Slide 355

Slide 355 text

/* adding jQuery PEP */ /* need to use custom touch-action attribute, not CSS (yet) */ ...

Slide 356

Slide 356 text

patrickhlauke.github.io/touch/pep-draw

Slide 357

Slide 357 text

patrickhlauke.github.io/touch/pep/listener/event-listener.html

Slide 358

Slide 358 text

events ﬁred on tap with PEP pointerover > pointerenter > pointerdown > touchstart > pointerup > pointerout > pointerleave > touchend > mouseover > mouseenter > mousemove > mousedown > mouseup > click essentially, relevant Pointer Events before touchstart and touchend

Slide 359

Slide 359 text

3D Rotator modiﬁed to use Pointer Events minimal code changes, as Pointer Events extend mouse events

Slide 360

Slide 360 text

3D Rotator modiﬁed to use Pointer Events (and PEP)

Slide 361

Slide 361 text

(7) Basic pointer events-driven fake slider with setPointerCapture Exercise/demo: make slider work in non-Pointer-Events browsers

Slide 362

Slide 362 text

(8) Basic pointer events-driven fake slider with setPointerCapture and PEP

Slide 363

Slide 363 text

utility libraries (because life is too short to hand-code gesture support)

Slide 364

Slide 364 text

Hammer.js

Slide 365

Slide 365 text

Hammer.js

Slide 366

Slide 366 text

patrickhlauke.github.io/touch/swipe

Slide 367

Slide 367 text

/* Hammer's high-level events example */ var element = document.getElementById('test_el'); var hammertime = new Hammer(element); hammertime.on("swipe", function(event) { /* handle horizontal swipes */ });

Slide 368

Slide 368 text

patrickhlauke.github.io/touch/hammer/swipe

Slide 369

Slide 369 text

debugging/testing

Slide 370

Slide 370 text

nothing beats having real devices...

Slide 371

Slide 371 text

BrowserStack (includes Physical Devices)

Slide 372

Slide 372 text

No content

Slide 373

Slide 373 text

Chrome DevTools / Remote Debugging Android Devices Chrome DevTools / Using the Console / Monitor events

Slide 374

Slide 374 text

enable touch events even without a touchscreen (to test "naive" 'ontouchstart' in window code) chrome://ﬂags/#touch-events

Slide 375

Slide 375 text

Chrome DevTools / Device Mode & Mobile Emulation correctly emulates touch and pointer events

Slide 376

Slide 376 text

Chrome DevTools / Device Mode & Mobile Emulation correctly emulates pointer events (with pointerType=="touch" )

Slide 377

Slide 377 text

emulate a touch interface (even without mobile emulation)

Slide 378

Slide 378 text

Firefox Developer Tools > Responsive Design Mode correctly emulates touch events

Slide 379

Slide 379 text

no touch emulation, nor touch events + pointer events (like on real Windows 10 Mobile) emulation, in Edge/F12 Tools

Slide 380

Slide 380 text

no touch emulation in Safari "Responsive Design Mode"

Slide 381

Slide 381 text

bonus material

Slide 382

Slide 382 text

no concept of hover on touch devices iOS fakes it, Samsung Galaxy Note II/Pro + stylus, ...

Slide 383

Slide 383 text

iOS Developer Library - Safari Web Content Guide - Handling Events while this works ok, Safari is the only browser doing it...

Slide 384

Slide 384 text

YouTube: Samsung Galaxy Note Pro 12.2 stylus hover requires advanced stylus, which is uncommon...

Slide 385

Slide 385 text

MSDN: Hover touch support

Slide 386

Slide 386 text

Channel9 MSDN - IE11: Using Hover Menus with Touch

Slide 387

Slide 387 text

MSDN: Using aria-haspopup to simulate hover on touch-enabled devices non-standard use of the aria-haspopup attribute...

Slide 388

Slide 388 text

avoid hover interfaces? arguable ... it's a user experience decision

Slide 389

Slide 389 text

complement for touch! same considerations as for keyboard users...

Slide 390

Slide 390 text

Input Device Capabilities (Draft Community Group Report)

Slide 391

Slide 391 text

Input Device Capabilities • currently limited usefulness - only has firesTouchEvents property • limited use cases (see Identifying touch events that will trigger a tap / click, Identifying mouse events derived from touch events and Google Developer: Input Device Capabilities) • easier to switch to Pointer Events (and add PEP)?

Slide 392

Slide 392 text

myButton.addEventListener('touchstart', addHighlight, false); myButton.addEventListener('touchend', removeHighlight, false); myButton.addEventListener('mousedown', addHighlight, false); myButton.addEventListener('mouseup', removeHighlight, false); myButton.addEventListener('click', someFunction, false); /* addHighlight/removeHighlight will fire twice for touch (first for touch, then for mouse compatibility events); can't use e.preventDefault(), as we want to rely on 'click' */

Slide 393

Slide 393 text

myButton.addEventListener('touchstart', addHighlight, false); myButton.addEventListener('touchend', removeHighlight, false); myButton.addEventListener('mousedown', function(e) { if (!e.sourceCapabilities.firesTouchEvents) { addHighlight(e); } } , false); myButton.addEventListener('mouseup', function(e) { if (!e.sourceCapabilities.firesTouchEvents) { removeHighlight(e); } } , false); myButton.addEventListener('click', someFunction, false); /* same function for touch and mouse, but if mouse check if source input fires touch events (i.e. this is a mouse compat event) */

Slide 394

Slide 394 text

Passive Event Listeners (EventListenerOptions)

Slide 395

Slide 395 text

Leap Motion Controller

Slide 396

Slide 396 text

Leap Motion • requires custom code (JavaScript, Java, C++, ...) using proprietary API (e.g. Leap Motion: JavaScript SDK Documentation) • does not ﬁre touch nor mouse events (third party native apps in Leap Motion App Store available to simulate native mouse) • (personally ﬁnd it quite inaccurate and horrible)

Slide 397

Slide 397 text

/* rough proof of concept that fires programmatic Pointer Events for each "pointable" (finger / tool) */ e = new PointerEvent('pointermove', { pointerId: ... , width: ... , height: ... , pressure: ... , tiltX: ... , tiltY: ... , pointerType: 'leap', isPrimary: false, clientX: ... , clientY: ... , screenX: ... , screenY: ... }); /* dispatch event to whatever element is at the coordinates (no support for pointer capture at this stage) */ document.elementFromPoint(x, y).dispatchEvent(e);

Slide 398

Slide 398 text

YouTube: Leap Motion generating programmatic Pointer Events patrickhlauke.github.io/touch/leapmotion/tracker

Slide 399

Slide 399 text

Slide 400

Slide 400 text

• Matt Gaunt – Touch Feedback for Mobile Sites • Jonathan Stark – FastActive • Stephen Woods – HTML5 Touch Interfaces • YouTube: Stephen Woods – Responsive HTML5 Touch Interfaces • Chris Wilson + Paul Kinlan – Touch And Mouse: Together Again For The First Time • Ryan Fioravanti – Creating Fast Buttons for Mobile Web Applications • Boris Smus – Multi-touch Web Development • Boris Smus – Generalized input on the cross-device web • Boris Smus – Interactive touch laptop experiments

Slide 401

Slide 401 text

• Rick Byers + Boris Smus (Google I/O) – Point, Click, Tap, Touch - Building Multi-Device Web Interfaces • Grant Goodale – Touch Events • W3C – Touch Events Extensions • Mozilla Developer Network – Touch Events • WebPlatform.org – Pointer Events • Rick Byers – The best way to avoid the dreaded 300ms click delay is to disable double-tap zoom • Chromium Issue 152149: All touch-event related APIs should exist if touch support is enabled at runtime

Slide 402

Slide 402 text

• Tim Kadlec – Avoiding the 300ms Click Delay, Accessibly • David Rousset - Unifying touch and mouse [...] • Microsoft – Pointer events updates (differences between IE10-IE11) • Patrick H. Lauke – Webseiten zum Anfassen • Patrick H. Lauke – Drei unter einem Dach: Pointer-Events für Maus, Stylus und Touchscreen • Patrick H. Lauke – Erweiterte Interaktionsmöglichkeiten mit Touchscreens • Patrick H. Lauke – Make your site work on touch devices • Stu Cox – You can't detect a touchscreen

Slide 403

Slide 403 text

• Microsoft – Hover touch support (IE10/IE11) • W3C Media Queries Level 4 – pointer • Stu Cox – The Good & Bad of Level 4 Media Queries • Peter-Paul Koch – Touch table • Peter-Paul Koch – Preventing the touch default • Peter-Paul Koch – Mouse event bubbling in iOS • YouTube: Edge Conference (Feb 2013 London) – Panel: Input • Edge Conference (Mar 2014 London) – Panel: Pointers and Interactions • Trent Walton – Device-Agnostic

Slide 404

Slide 404 text

• Stu Cox – The Golden Pattern for Handling Touch Input • Matt Gaunt – ‘Focusing’ on the Web Today • Mobiscroll – Working with touch events • Peter-Paul Koch – The iOS event cascade and innerHTML • Patrick H. Lauke – Make your site work on touch devices • Scott Jehl (Filament Group) – Tappy: A custom tap event handler • Yehuda Katz – Touch events on the web are fundamentally unﬁnished • Google Developers: A More Compatible, Smoother Touch

Slide 405

Slide 405 text

• Andrea Giammarchi – PointerEvents No More • YouTube: Matt Gaunt (Google) - Touch in a Web App? No Problem • Luke Wroblewski – How to Communicate Hidden Gestures • David Washington - Designing custom touch interactions • David Washington - Building pull-to-refresh • Andy Peatling - JavaScript Pull to Refresh for the Web • Tilo Mitra – The State of Gestures • YouTube: Tilo Mitra (YUI) – The State of Gestures

Slide 406

Slide 406 text

• Rob Larsen - The Uncertain Web: Pointer Event Polyﬁll and Chrome Fragmentation • Detlev Fisher - Implications of new input modes (touch, speech, gestures) for the Web Content Accessibility Guidelines • Ralph Thomas - Towards declarative touch interactions • Windows Dev Center: Windows desktop applications > Guidelines > Interaction (touch, keyboard, mouse, pen) • Microsoft Open Technologies - jQuery Adopts Pointer Events

Slide 407

Slide 407 text

• jQuery blog - Getting on Point • IEBlog - Pointer Events W3C Recommendation, Interoperable Touch, and Removing the Dreaded 300ms Tap Delay • Microsoft Open Technologies - Pointer Events is now a W3C Standard • Patrick H. Lauke (The Paciello Group) - Pointer Events advance to W3C Recommendation • Jacob Rossi - The Input Shouldn't Matter • hacks.mozilla.org - Pointer Events now in Firefox Nightly

Slide 408

Slide 408 text

• Rick Byers – Interoperability Case Studies at BlinkOn 5 (touch-action) • Mozilla Developer Network – Pointer Events • Christian Saylor – A Touch of Brilliance: Why 3D Touch Matters • Lukas Mathis – Usability on Mobile Is Getting Worse • Rocket Insights – Initial thoughts on 3D Touch • Wiser Coder – Disable “pull to refresh” in Chrome for Android

Slide 409

Slide 409 text

• Google Developers / Web Fundamentals: Add touch to your site • Alex Gibson – Different ways to trigger touchcancel in mobile browsers • BBC Global Experience Language: How to design for touch • Google BlinkOn 5: Lightning talks / Mustaq Ahmed – Pointer Events in Chrome implementation update • A Book Apart: Josh Clark – Designing for Touch • Kickstarter – The Sensel Morph: Interaction Evolved

Slide 410

Slide 410 text

• Ruadhán O'Donoghue – The HTML5 Pointer Events API: Combining touch, mouse, and pen • Jordan Staniscia – Hover is dead. Long live hover. • Jason Grigsby – Four Truths About Input • Jason Grigsby – Adapting to Input • David Corbacho – Debouncing and Throttling Explained • YouTube: Pre-Touch Sensing for Mobile Interaction (Microsoft)

Slide 411

Slide 411 text

• YouTube: HTML5DevConf: Jacob Rossi, "Pointer Events, Panning & Zooming, and Gestures" • YouTube: Jacob Rossi, "Pointing Forward" at W3Conf 2013 • YouTube: Tim Park: Pointing Forward (updated) – JSConf.Asia 2013 • Flickity – touch, responsive, ﬂickable carousels • Google Developers / Update: Once Upon an Event Listener • Google Developers / Update: Improving Scroll Performance with Passive Event Listeners • Google Developers / Update: Making touch scrolling fast by default

Slide 412

Slide 412 text

• Google Developers / Update: Pointing the Way Forward • Google Developers / Update: Touch Action Options • Universal Stylus Initiative (USI) • Adobe Creative Cloud: Andrew Smyk – Holistic Adaptive Design: Factors to Consider in a Multi-Screen World • YouTube: Drawboard on Surface Studio with Surface Dial • AirBar – Touch without a touchscreen • GitHub: Rafael Pedicini (Rafrex) – Detect It

Slide 413

Slide 413 text

youtube.com/watch?v=AZKpByV5764

Slide 414

Slide 414 text

get in touch @patrick_h_lauke github.com/patrickhlauke/touch patrickhlauke.github.io/getting-touchy-presentation slideshare.net/redux paciellogroup.com splintered.co.uk