The State of Gestures

39416e8a4f8a9c9677a0a67da51f2207?s=47 Tilo
November 06, 2013

The State of Gestures

This talk is a technical deep dive into the world of gesture events. I'll introduce the work that has been and is currently being done in YUI Gestures and how you can use gesture events to create multi-device user experiences.

39416e8a4f8a9c9677a0a67da51f2207?s=128

Tilo

November 06, 2013
Tweet

Transcript

  1. STATE GESTURES THE OF Tilo Mitra I work on YUI

    & Pure @ Yahoo
  2. Challenges of developing multi-device user experiences OR and what we’re

    doing about it.
  3. Evolving World of Inputs

  4. Let’s assume we want to “fast-click” on a button.

  5. Mouse Keyboard

  6. node.on('click', function (e) { ... });

  7. Touch Accelerometer Gyroscope

  8. if (window && ("ontouchstart" in window)) { node.on('touchend', DoSomething); }

    else { node.on('mouseup', DoSomething); }
  9. Mouse + Touch

  10. if (window && ("ontouchstart" in window)) { node.on('touchend', DoSomething); }

    else { node.on('mouseup', DoSomething); } This doesn’t get executed, so we can’t interact with mouse* events.
  11. if (window && ("ontouchstart" in window)) { node.on(['touchend', 'mouseup'], function

    (e) { //so mouse* doesn’t fire if touchend fires e.preventDefault(); doSomething(e); }); }
  12. if (window && ("ontouchstart" in window)) { node.on(['touchend', 'mouseup'], function

    (e) { //so mouse* doesn’t fire if touchend fires e.preventDefault(); doSomething(e); }); } else if (win && ("msPointerEnabled" in win.navigator)) { node.on('MSPointerUp', doSomething); }
  13. if (window && ("ontouchstart" in window)) { node.on(['touchend', 'mouseup'], function

    (e) { //so mouse* doesn’t fire if touchend fires e.preventDefault(); doSomething(e); }); } else if (win && ("msPointerEnabled" in win.navigator)) { node.on('MSPointerUp', doSomething); } else { node.on('mouseup', doSomething); }
  14. if (window && ("ontouchstart" in window)) { node.on(['touchend', 'mouseup'], function

    (e) { //so mouse* doesn’t fire if touchend fires e.preventDefault(); doSomething(e); }); } else if (win && ("msPointerEnabled" in win.navigator)) { node.on('MSPointerUp', doSomething); } else { node.on('mouseup', doSomething); } This gets complicated
  15. This gets complicated What to do for more complex gestures?

    preventDefault() is heavy-handed
  16. http://jsbin.com/IKegUbu/1 See what events your browser fires

  17. Desktop Browser iOS Android IE10+ mousedown mousemove mouseup click touchstart

    touchmove touchend click touchstart mousedown touchmove mousemove touchend mouseup click MSPointerDown mousedown MSPointerMove mousemove MSPointerUp mouseup click
  18. RWD CSS HTML /

  19. RWD CSS HTML / JavaScript

  20. re·spon·sive ADJECTIVE quick to react or respond

  21. Decouple events from inputs

  22. Decouple events from inputs finger mouse pen {{ some event

    }} Your App
  23. Sound familiar? Basically what the Pointer Spec is about http://www.w3.org/Submission/pointer-events/

  24. A pointer can be any point of contact on the

    screen made by a mouse cursor, pen, touch (including multi-touch), or other pointing input device. This model makes it easy to write sites and applications that work well no matter what hardware the user has.
  25. Polyfill?

  26. Polyfill? Why not improve gesturemove* events?

  27. "gesturemovestart", "gesturemove", and "gesturemoveend" are events that serve as abstractions

    over mouse and touch events, forking internally based on the client device. From the Docs:
  28. touchstart mousedown MSPointerDown gesturemovestart touchmove mousemove MSPointerMove touchend mouseup MSPointerUp

    gesturemove gesturemoveend
  29. touchstart mousedown MSPointerDown gesturemovestart touchmove mousemove MSPointerMove touchend mouseup MSPointerUp

    gesturemove gesturemoveend Your App tap flick move/drag transforms
  30. Improvements to event-move to make it work for mouse, touch

    and devices that support both Pull #1309 Code in Progress
  31. Common fast-click use case, with e.preventDefault() support event-tap Available Today

  32. EVENT UPDATES PAYLOAD STATUS STATUS tap Dual Listener Support, e.preventDefault()

    - Available Today Available Today flick flickup, flickdown flickleft, flickright direction velocity PULL #1323 PULL #1323 gesturemove* Dual Listener Support direction deltaX deltaY Pull #1309 See Demo move moveup, movedown moveleft, moveright, ability to lock axis direction deltaX deltaY See Demo See Demo NEW APIs
  33. Demo CAROUSEL WITH NEW GESTURE EVENTS http://bit.ly/gesture-events

  34. WHAT WE DO NOW 1. Set up some gesturemove* and

    flick listeners 2. Store the [x1,y1] of gesturemovestart 3. Compare [x1,y1] to [x2,y2] from gesturemove to determine direction + delta 4. Compare [x1,y1] to [x2,y2] from gesturemoveend to determine direction + delta 5. In callbacks, branch logic based on direction 6. Make sure that these do not get called twice in devices that support mouse and touch, but yet, also works with both.
  35. WITH NEW APIs 1. Listen for move, flickleft, flickright, and

    gesturemoveend event 2. Inside move callback, get info with e.drag.direction and e.drag.deltaX 3. Transition to previous and next panels with flickleft and flickright 4. Inside gesturemoveend callback, get info with e.gesture.deltaX and e.gesture.dirX to decide whether or not to transition panels.
  36. Going Forward

  37. Pull #1309 gesturemove* improvements Pull #1323 flick* improvements tilomitra/yui3/gesture- integration

    Build of YUI with all of these modules Demo http://bit.ly/gesture-events
  38. Check it out & Let us know what you think

  39. THANKS @tilomitra GitHub & Twitter