The State of Gestures

This talk is a technical deep dive into the world of gesture events. I'll introduce the work that has been and is currently being done in YUI Gestures and how you can use gesture events to create multi-device user experiences.

Transcript

1. STATE GESTURES THE OF Tilo Mitra I work on YUI & Pure @ Yahoo
2. Challenges of developing multi-device user experiences OR and what we’re doing about it.
3. Evolving World of Inputs
4. Let’s assume we want to “fast-click” on a button.
5. Mouse Keyboard
6. node.on('click', function (e) { ... });
7. Touch Accelerometer Gyroscope
8. if (window && ("ontouchstart" in window)) { node.on('touchend', DoSomething); } else { node.on('mouseup', DoSomething); }
9. Mouse + Touch
10. if (window && ("ontouchstart" in window)) { node.on('touchend', DoSomething); } else { node.on('mouseup', DoSomething); } This doesn’t get executed, so we can’t interact with mouse* events.
11. if (window && ("ontouchstart" in window)) { node.on(['touchend', 'mouseup'], function (e) { //so mouse* doesn’t fire if touchend fires e.preventDefault(); doSomething(e); }); }
12. if (window && ("ontouchstart" in window)) { node.on(['touchend', 'mouseup'], function (e) { //so mouse* doesn’t fire if touchend fires e.preventDefault(); doSomething(e); }); } else if (win && ("msPointerEnabled" in win.navigator)) { node.on('MSPointerUp', doSomething); }
13. if (window && ("ontouchstart" in window)) { node.on(['touchend', 'mouseup'], function (e) { //so mouse* doesn’t fire if touchend fires e.preventDefault(); doSomething(e); }); } else if (win && ("msPointerEnabled" in win.navigator)) { node.on('MSPointerUp', doSomething); } else { node.on('mouseup', doSomething); }
14. if (window && ("ontouchstart" in window)) { node.on(['touchend', 'mouseup'], function (e) { //so mouse* doesn’t fire if touchend fires e.preventDefault(); doSomething(e); }); } else if (win && ("msPointerEnabled" in win.navigator)) { node.on('MSPointerUp', doSomething); } else { node.on('mouseup', doSomething); } This gets complicated
15. This gets complicated What to do for more complex gestures? preventDefault() is heavy-handed
16. http://jsbin.com/IKegUbu/1 See what events your browser fires
17. Desktop Browser iOS Android IE10+ mousedown mousemove mouseup click touchstart touchmove touchend click touchstart mousedown touchmove mousemove touchend mouseup click MSPointerDown mousedown MSPointerMove mousemove MSPointerUp mouseup click
18. RWD CSS HTML /
19. RWD CSS HTML / JavaScript
20. re·spon·sive ADJECTIVE quick to react or respond
21. Decouple events from inputs
22. Decouple events from inputs finger mouse pen {{ some event }} Your App
23. Sound familiar? Basically what the Pointer Spec is about http://www.w3.org/Submission/pointer-events/
24. A pointer can be any point of contact on the screen made by a mouse cursor, pen, touch (including multi-touch), or other pointing input device. This model makes it easy to write sites and applications that work well no matter what hardware the user has.
25. Polyfill?
26. Polyfill? Why not improve gesturemove* events?
27. "gesturemovestart", "gesturemove", and "gesturemoveend" are events that serve as abstractions over mouse and touch events, forking internally based on the client device. From the Docs:
28. touchstart mousedown MSPointerDown gesturemovestart touchmove mousemove MSPointerMove touchend mouseup MSPointerUp gesturemove gesturemoveend
29. touchstart mousedown MSPointerDown gesturemovestart touchmove mousemove MSPointerMove touchend mouseup MSPointerUp gesturemove gesturemoveend Your App tap flick move/drag transforms
30. Improvements to event-move to make it work for mouse, touch and devices that support both Pull #1309 Code in Progress
31. Common fast-click use case, with e.preventDefault() support event-tap Available Today
32. EVENT UPDATES PAYLOAD STATUS STATUS tap Dual Listener Support, e.preventDefault() - Available Today Available Today flick flickup, flickdown flickleft, flickright direction velocity PULL #1323 PULL #1323 gesturemove* Dual Listener Support direction deltaX deltaY Pull #1309 See Demo move moveup, movedown moveleft, moveright, ability to lock axis direction deltaX deltaY See Demo See Demo NEW APIs
33. Demo CAROUSEL WITH NEW GESTURE EVENTS http://bit.ly/gesture-events
34. WHAT WE DO NOW 1. Set up some gesturemove* and flick listeners 2. Store the [x1,y1] of gesturemovestart 3. Compare [x1,y1] to [x2,y2] from gesturemove to determine direction + delta 4. Compare [x1,y1] to [x2,y2] from gesturemoveend to determine direction + delta 5. In callbacks, branch logic based on direction 6. Make sure that these do not get called twice in devices that support mouse and touch, but yet, also works with both.
35. WITH NEW APIs 1. Listen for move, flickleft, flickright, and gesturemoveend event 2. Inside move callback, get info with e.drag.direction and e.drag.deltaX 3. Transition to previous and next panels with flickleft and flickright 4. Inside gesturemoveend callback, get info with e.gesture.deltaX and e.gesture.dirX to decide whether or not to transition panels.
36. Going Forward
37. Pull #1309 gesturemove* improvements Pull #1323 flick* improvements tilomitra/yui3/gesture- integration Build of YUI with all of these modules Demo http://bit.ly/gesture-events
38. Check it out & Let us know what you think
39. THANKS @tilomitra GitHub & Twitter