Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Understanding Android Gestures

7bf2baf0d8d5d7d96e6b67c5d567e3f9?s=47 Caren
October 28, 2017

Understanding Android Gestures

For developers working with custom views and touches for the first time, it could sometimes be difficult to decipher what goes on when a user touches the screen. In this talk we’ll trace through the steps to see how Android decides which view should handle a touch, and how the system differentiates between swipes and clicks. We’ll start by visually examining how a view hierarchy is laid out and examine the multiple methods involved when the system receives a touch. Then we’ll look into the different kinds of gestures that are categorized in Android and how the system finally decides to call the ubiquitous onClickListener().

7bf2baf0d8d5d7d96e6b67c5d567e3f9?s=128

Caren

October 28, 2017
Tweet

More Decks by Caren

Other Decks in Programming

Transcript

  1. Understanding Gestures on Android Caren Chang

  2. button.setOnClickListener(new View.OnClickListener() {
 @Override
 public boolean onClick() {
 
 }


    });
  3. button.setOnClickListener(new View.OnClickListener() {
 @Override
 public boolean onClick() {
 
 }


    }); button.setOnLongClickListener(new View.OnLongClickListener() {
 @Override
 public boolean onLongClick(View v) {
 return false;
 }
 });
  4. How are the Android APIs working under the hood?

  5. Motivations

  6. Motivations UI for an oven != UI for phone

  7. Motivations UI for an oven != UI for phone Remove

    “feeling” of using a phone
  8. Motivations UI for an oven != UI for phone Remove

    “feeling” of using a phone Psychology of clicking
  9. Motivations UI for an oven != UI for phone Remove

    “feeling” of using a phone Psychology of clicking Angling
  10. Motivations for you

  11. Motivations for you Understand your phone

  12. Motivations for you Understand your phone Customize for the behaviors

    you want
  13. Motivations for you Understand your phone Customize for the behaviors

    you want Create better user experience
  14. Motivations for you Understand your phone Customize for the behaviors

    you want Create better user experience Be less stressed when developing and debugging
  15. What happens when a user interacts with their phone?

  16. What happens when a user interacts with their phone? Initiate

    ‘contact’
  17. What happens when a user interacts with their phone? Initiate

    ‘contact’ Moves finger around
  18. What happens when a user interacts with their phone? Initiate

    ‘contact’ Moves finger around Lifts finger
  19. What happens when a user interacts with their phone? Initiate

    ‘contact’ ACTION_DOWN Moves finger around ACTION_MOVE Lifts finger ACTION_UP
  20. MotionEvent

  21. MotionEvent Every action on the screen is translated into MotionEvent

  22. MotionEvent Every action on the screen is translated into 


    MotionEvent Description movements with an action code and
 set of axis values
  23. MotionEvent Every action on the screen is translated into 


    MotionEvent Description movements with an action code and 
 set of axis values Action code = up, down, move
  24. MotionEvent Every action on the screen is translated into 


    MotionEvent Description movements with an action code and 
 set of axis values Action code = up, down, move Axis values = touch position
  25. MotionEvent

  26. MotionEvent getAction() -> ACTION_DOWN getX() -> 400 getY() -> 630

  27. MotionEvent 0, 0 maxX, 0 maxX, maxY maxY, 0

  28. MotionEvent 0, 0 maxX, 0 maxX, maxY maxY, 0

  29. MotionEvent 0, 0 maxX, 0 maxX, maxY maxY, 0

  30. Screen receives a touch event MotionEvent

  31. Screen receives a touch event Using the (x , y)

    coordinate, 
 the MotionEvent is sent to the 
 appropriate child view(s) MotionEvent
  32. Screen receives a touch event Using the (x , y)

    coordinate, 
 the MotionEvent is sent to the 
 appropriate child view(s) The view can decide how to 
 handle the MotionEvent MotionEvent
  33. Decide? What does the view decide?

  34. Decide? What does the view decide? Views can decide if

    they want to take action
  35. Decide? What does the view decide? boolean onTouchEvent() Views can

    decide if they want to take action
  36. Decide? What does the view decide? boolean onTouchEvent() Views can

    decide if they want to take action Returns true if the event was handled, false otherwise
  37. public class CustomView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent

    motionEvent) {
 
 switch (motionEvent.getAction()) {
 case MotionEvent.ACTION_DOWN:
 Log.i("touch", "down event");
 break;
 case MotionEvent.ACTION_MOVE:
 Log.i("touch", "move event");
 break;
 case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 }
 return false;
 }
 }
  38. public class CustomView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent

    motionEvent) {
 
 switch (motionEvent.getAction()) {
 case MotionEvent.ACTION_DOWN:
 Log.i("touch", "down event");
 break;
 case MotionEvent.ACTION_MOVE:
 Log.i("touch", "move event");
 break;
 case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 }
 return false;
 }
 }
  39. public class CustomView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent

    motionEvent) {
 
 switch (motionEvent.getAction()) {
 case MotionEvent.ACTION_DOWN:
 Log.i("touch", "down event");
 break;
 case MotionEvent.ACTION_MOVE:
 Log.i("touch", "move event");
 break;
 case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 }
 return false;
 }
 }
  40. public class CustomView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent

    motionEvent) {
 
 switch (motionEvent.getAction()) {
 case MotionEvent.ACTION_DOWN:
 Log.i("touch", "down event");
 break;
 case MotionEvent.ACTION_MOVE:
 Log.i("touch", "move event");
 break;
 case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 }
 return false;
 }
 }
  41. public class CustomView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent

    motionEvent) {
 
 switch (motionEvent.getAction()) {
 case MotionEvent.ACTION_DOWN:
 Log.i("touch", "down event");
 break;
 case MotionEvent.ACTION_MOVE:
 Log.i("touch", "move event");
 break;
 case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 }
 return false;
 }
 }
  42. Creating our own click listener

  43. Creating our own click listener What does it take for

    a series of motion events to be considered a “click” or a “long click” ?
  44. Creating our own click listener What does it take for

    a series of motion events to 
 be considered a “click” or a “long click” ? time it takes from the first down event till the last up event
  45. Creating our own click listener What does it take for

    a series of motion events to 
 be considered a “click” or a “long click” ? time it takes from the first down event till the last up event difference in distance in between down and up
  46. custom ‘click’ listener

  47. custom ‘click’ listener case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 long eventEndTime

    = System.currentTimeMillis();
 
 // time in ms when the first down event registered
 long eventStartTime = motionEvent.getDownTime();
 long totalTimeElapsed = eventEndTime - eventStartTime;
 
 if (totalTimeElapsed < clickTimeoutMs) {
 performClick();
 } else if (totalTimeElapsed < longClickTimeoutMs) {
 performLongClick();
 } else {
 Log.i(TAG, "click action timed out");
 }
  48. custom ‘click’ listener case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 long eventEndTime

    = System.currentTimeMillis();
 
 // time in ms when the first down event registered
 long eventStartTime = motionEvent.getDownTime();
 long totalTimeElapsed = eventEndTime - eventStartTime;
 
 if (totalTimeElapsed < clickTimeoutMs) {
 performClick();
 } else if (totalTimeElapsed < longClickTimeoutMs) {
 performLongClick();
 } else {
 Log.i(TAG, "click action timed out");
 }
  49. custom ‘click’ listener case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 long eventEndTime

    = System.currentTimeMillis();
 
 // time in ms when the first down event registered
 long eventStartTime = motionEvent.getDownTime();
 long totalTimeElapsed = eventEndTime - eventStartTime;
 
 if (totalTimeElapsed < clickTimeoutMs) {
 performClick();
 } else if (totalTimeElapsed < longClickTimeoutMs) {
 performLongClick();
 } else {
 Log.i(TAG, "click action timed out");
 }
  50. custom ‘click’ listener case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 long eventEndTime

    = System.currentTimeMillis();
 
 // time in ms when the first down event registered
 long eventStartTime = motionEvent.getDownTime();
 long totalTimeElapsed = eventEndTime - eventStartTime;
 
 if (totalTimeElapsed < clickTimeoutMs) {
 performClick();
 } else if (totalTimeElapsed < longClickTimeoutMs) {
 performLongClick();
 } else {
 Log.i(TAG, "click action timed out");
 }
  51. custom ‘click’ listener public class CustomView extends View {
 


    
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 … // input code we had previously
 }
 
 return false;
 }
 }
  52. custom ‘click’ listener public class CustomView extends View {
 


    
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 … // input code we had previously
 }
 
 return false;
 }
 }
  53. custom ‘click’ listener public class CustomView extends View {
 


    
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 … // input code we had previously
 }
 
 return false;
 }
 }
  54. custom ‘click’ listener public class CustomView extends View {
 


    
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 … // input code we had previously
 }
 
 return false;
 }
 }
  55. swipe down to dismiss

  56. swipe down to dismiss 1. direction of the touch movement

  57. swipe down to dismiss 1. direction of the touch movement

    2. distance of the touch movement
  58. swipe down to dismiss @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {


    
 if (motionEvent.getAction() == MotionEvent.ACTION_DOWN) {
 initialDownYCoord = motionEvent.getRawY();
 }
 
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 if (motionEvent.getRawY() - initialDownYCoord > SWIPE_DOWN_RANGE) { // swiped down
 startAnimation(slide_down); // slide_down is a predefined animation
 return true;
 }
 }
 
 return super.onTouchEvent(motionEvent);
 }
  59. swipe down to dismiss @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {


    
 if (motionEvent.getAction() == MotionEvent.ACTION_DOWN) {
 initialDownYCoord = motionEvent.getRawY();
 }
 
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 if (motionEvent.getRawY() - initialDownYCoord > SWIPE_DOWN_RANGE) { // swiped down
 startAnimation(slide_down); // slide_down is a predefined animation
 return true;
 }
 }
 
 return super.onTouchEvent(motionEvent);
 }
  60. swipe down to dismiss @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {


    
 if (motionEvent.getAction() == MotionEvent.ACTION_DOWN) {
 initialDownYCoord = motionEvent.getRawY();
 }
 
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 if (motionEvent.getRawY() - initialDownYCoord > SWIPE_DOWN_RANGE) { // swiped down
 startAnimation(slide_down); // slide_down is a predefined animation
 return true;
 }
 }
 
 return super.onTouchEvent(motionEvent);
 }
  61. swipe down to dismiss @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {


    
 if (motionEvent.getAction() == MotionEvent.ACTION_DOWN) {
 initialDownYCoord = motionEvent.getRawY();
 }
 
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 if (motionEvent.getRawY() - initialDownYCoord > SWIPE_DOWN_RANGE) { // swiped down
 startAnimation(slide_down); // slide_down is a predefined animation
 return true;
 }
 }
 
 return super.onTouchEvent(motionEvent);
 }
  62. Why have we been extending the View class?

  63. Why have we been extending the View class? Based on

    what we’ve done so far, we could have simplified things with something such as :
 view.setOnTouchListener(new View.OnTouchListener() {
 @Override
 public boolean onTouch(View v, MotionEvent event) {
 return false;
 }
 });
  64. Why have we been extending the View class? Based on

    what we’ve done so far, we could have simplified 
 things with something such as :
 view.setOnTouchListener(new View.OnTouchListener() {
 @Override
 public boolean onTouch(View v, MotionEvent event) {
 return false;
 }
 }); While onTouchListener is great , we get more control when we creating a custom view class
  65. Extra methods available when implementing custom view classes onInterceptTouchEvent() dispatchTouchEvent()

  66. boolean onInterceptTouchEvent()

  67. boolean onInterceptTouchEvent() Called whenever a touch event occurs If this

    method returns true, the MotionEvent will not be passed to child Can be used to ‘listen’ to events and act accordingly
  68. boolean dispatchTouchEvent()

  69. boolean dispatchTouchEvent() Passes the motion event to the target view

    (or this view if it’s the target)
  70. boolean dispatchTouchEvent() Passes the motion event to the target view

    (or this view if it’s the target) Returns true if handled by current view, false otherwise
  71. For every MotionEvent : dispatchTouchEvent() is called in the order

    of Activity down to Button If FrameLayout.dispatchTouchEvent() returned true, Button would not call dispatchTouchEvent() Instead, FrameLayout.onTouchEvent() would be called Activity FrameLayout Button
  72. For every MotionEvent : dispatchTouchEvent() is called in the order

    of Activity down to Button If FrameLayout.dispatchTouchEvent() returned true, Button would not call dispatchTouchEvent() Instead, FrameLayout.onTouchEvent() would be called Activity FrameLayout Button
  73. For every MotionEvent : dispatchTouchEvent() is called in the order

    of Activity down to Button If FrameLayout.dispatchTouchEvent() returned true, Button would not call dispatchTouchEvent() Instead, FrameLayout.onTouchEvent() would be called Activity FrameLayout Button
  74. If dispatchTouchEvent() gets to Button, and there’s no more child

    views, Button.onTouchEvent() would be called Activity FrameLayout Button
  75. If dispatchTouchEvent() gets to Button, and there’s no more child

    views, Button.onTouchEvent() would be called onTouchEvent() is then called from Button to FrameLayout to Activity until one of the views decides to handle it Activity FrameLayout Button
  76. How MotionEvents are propagated Activity.dispatchTouchEvent() FrameLayout.dispatchTouchEvent() Button.onTouchEvent() false Activity.onTouchEvent() FrameLayout.onTouchEvent()

    Button.dispatchTouchEvent() false
  77. How MotionEvents are propagated Activity.dispatchTouchEvent() FrameLayout.dispatchTouchEvent() Button.onTouchEvent() Activity.onTouchEvent() true FrameLayout.onTouchEvent()

    Button.dispatchTouchEvent()
  78. How MotionEvents are propagated Activity.dispatchTouchEvent() FrameLayout.dispatchTouchEvent() Button.onTouchEvent() false Activity.onTouchEvent() FrameLayout.onTouchEvent()

    Button.dispatchTouchEvent() false false
  79. How MotionEvents are propagated Activity.dispatchTouchEvent() FrameLayout.dispatchTouchEvent() Button.onTouchEvent() false Activity.onTouchEvent() FrameLayout.onTouchEvent()

    Button.dispatchTouchEvent() true false
  80. github.com/calren/viewsAndTouches

  81. github.com/calren/viewsAndTouches @Override
 public boolean dispatchTouchEvent(MotionEvent motionEvent) {
 Log.i("touchevent", "dispatchTouchEvent ACTIVITY");


    return super.dispatchTouchEvent(motionEvent);
 }
 
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 Log.i("touchevent", "onTouchEvent ACTIVITY");
 return super.onTouchEvent(motionEvent);
 }
  82. None
  83. None
  84. Things to consider when customizing behavior

  85. Things to consider when customizing behavior Most common gestures are

    already implemented (tap, doubleTap, fling)
  86. Things to consider when customizing behavior Most common gestures are

    already implemented (tap, doubleTap, fling) Try to base values off base values defined by Android TOUCH_SLOP, TAP_TIMEOUT, PAGING_TOUCH_SLOP
  87. In Summary… Custom touch event handling is complicated

  88. In Summary… public boolean dispatchTouchEvent(MotionEvent event) {
 // If the

    event should be handled by accessibility focus first.
 if (event.isTargetAccessibilityFocus()) {
 // We don't have focus or no virtual descendant has it, do not handle the event.
 if (!isAccessibilityFocusedViewOrHost()) {
 return false;
 }
 // We have focus and got the event, then use normal event dispatch.
 event.setTargetAccessibilityFocus(false);
 }
 
 boolean result = false;
 
 if (mInputEventConsistencyVerifier != null) {
 mInputEventConsistencyVerifier.onTouchEvent(event, 0);
 }
 
 final int actionMasked = event.getActionMasked();
 if (actionMasked == MotionEvent.ACTION_DOWN) {
 // Defensive cleanup for new gesture
 stopNestedScroll();
 }
 
 if (onFilterTouchEventForSecurity(event)) {
 if ((mViewFlags & ENABLED_MASK) == ENABLED && handleScrollBarDragging(event)) {
 result = true;
 }
 //noinspection SimplifiableIfStatement
 ListenerInfo li = mListenerInfo;
 if (li != null && li.mOnTouchListener != null
 && (mViewFlags & ENABLED_MASK) == ENABLED
 && li.mOnTouchListener.onTouch(this, event)) {
 result = true;
 }
 
 if (!result && onTouchEvent(event)) {
 result = true;
 }
 }
 
 if (!result && mInputEventConsistencyVerifier != null) {
 mInputEventConsistencyVerifier.onUnhandledEvent(event, 0);
 }
 
 // Clean up after nested scrolls if this is the end of a gesture;
 // also cancel it if we tried an ACTION_DOWN but we didn't want the rest
 // of the gesture.
 if (actionMasked == MotionEvent.ACTION_UP ||
 actionMasked == MotionEvent.ACTION_CANCEL ||
 (actionMasked == MotionEvent.ACTION_DOWN && !result)) {
 stopNestedScroll();
 }
 
 return result;

  89. In Summary… Custom touch event handling is complicated All touch

    events on Android are packaged into a MotionEvent
  90. In Summary… Custom touch event handling is complicated All touch

    events on Android are packaged into a MotionEvent Touch events are passed down the view hierarchy until an interested view handles it onTouchEvent()
  91. In Summary… Custom touch event handling is complicated All touch

    events on Android are packaged into a MotionEvent Touch events are passed down the view hierarchy until an interested view handles it onTouchEvent() Events can be intercepted and redirected as needed
 dispatchTouchEvent() and onInterceptTouchEvent()
  92. Other great talks Writing Custom Views for Android Adam Powell,

    Romain Guy Measure, Layout, Draw, Repeat: Custom Views and ViewGroups Huyen Tue Dao Making Sense of the Touch System Philippe Breault Mastering the Android Touch System Dave Smith
  93. Thanks! github.com/calren/viewsAndTouches @calren24