Understanding Android Gestures

For developers working with custom views and touches for the first time, it could sometimes be difficult to decipher what goes on when a user touches the screen. In this talk we’ll trace through the steps to see how Android decides which view should handle a touch, and how the system differentiates between swipes and clicks. We’ll start by visually examining how a view hierarchy is laid out and examine the multiple methods involved when the system receives a touch. Then we’ll look into the different kinds of gestures that are categorized in Android and how the system finally decides to call the ubiquitous onClickListener().

Transcript

1. Understanding Gestures on Android Caren Chang
2. button.setOnClickListener(new View.OnClickListener() {
 @Override
 public boolean onClick() {
 
 }
 });
3. button.setOnClickListener(new View.OnClickListener() {
 @Override
 public boolean onClick() {
 
 }
 }); button.setOnLongClickListener(new View.OnLongClickListener() {
 @Override
 public boolean onLongClick(View v) {
 return false;
 }
 });
4. How are the Android APIs working under the hood?
5. Motivations
6. Motivations UI for an oven != UI for phone
7. Motivations UI for an oven != UI for phone Remove “feeling” of using a phone
8. Motivations UI for an oven != UI for phone Remove “feeling” of using a phone Psychology of clicking
9. Motivations UI for an oven != UI for phone Remove “feeling” of using a phone Psychology of clicking Angling
10. Motivations for you
11. Motivations for you Understand your phone
12. Motivations for you Understand your phone Customize for the behaviors you want
13. Motivations for you Understand your phone Customize for the behaviors you want Create better user experience
14. Motivations for you Understand your phone Customize for the behaviors you want Create better user experience Be less stressed when developing and debugging
15. What happens when a user interacts with their phone?
16. What happens when a user interacts with their phone? Initiate ‘contact’
17. What happens when a user interacts with their phone? Initiate ‘contact’ Moves finger around
18. What happens when a user interacts with their phone? Initiate ‘contact’ Moves finger around Lifts finger
19. What happens when a user interacts with their phone? Initiate ‘contact’ ACTION_DOWN Moves finger around ACTION_MOVE Lifts finger ACTION_UP
20. MotionEvent
21. MotionEvent Every action on the screen is translated into MotionEvent
22. MotionEvent Every action on the screen is translated into 
 MotionEvent Description movements with an action code and
 set of axis values
23. MotionEvent Every action on the screen is translated into 
 MotionEvent Description movements with an action code and 
 set of axis values Action code = up, down, move
24. MotionEvent Every action on the screen is translated into 
 MotionEvent Description movements with an action code and 
 set of axis values Action code = up, down, move Axis values = touch position
25. MotionEvent
26. MotionEvent getAction() -> ACTION_DOWN getX() -> 400 getY() -> 630
27. MotionEvent 0, 0 maxX, 0 maxX, maxY maxY, 0
28. MotionEvent 0, 0 maxX, 0 maxX, maxY maxY, 0
29. MotionEvent 0, 0 maxX, 0 maxX, maxY maxY, 0
30. Screen receives a touch event MotionEvent
31. Screen receives a touch event Using the (x , y) coordinate, 
 the MotionEvent is sent to the 
 appropriate child view(s) MotionEvent
32. Screen receives a touch event Using the (x , y) coordinate, 
 the MotionEvent is sent to the 
 appropriate child view(s) The view can decide how to 
 handle the MotionEvent MotionEvent
33. Decide? What does the view decide?
34. Decide? What does the view decide? Views can decide if they want to take action
35. Decide? What does the view decide? boolean onTouchEvent() Views can decide if they want to take action
36. Decide? What does the view decide? boolean onTouchEvent() Views can decide if they want to take action Returns true if the event was handled, false otherwise
37. public class CustomView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 
 switch (motionEvent.getAction()) {
 case MotionEvent.ACTION_DOWN:
 Log.i("touch", "down event");
 break;
 case MotionEvent.ACTION_MOVE:
 Log.i("touch", "move event");
 break;
 case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 }
 return false;
 }
 }
38. public class CustomView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 
 switch (motionEvent.getAction()) {
 case MotionEvent.ACTION_DOWN:
 Log.i("touch", "down event");
 break;
 case MotionEvent.ACTION_MOVE:
 Log.i("touch", "move event");
 break;
 case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 }
 return false;
 }
 }
39. public class CustomView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 
 switch (motionEvent.getAction()) {
 case MotionEvent.ACTION_DOWN:
 Log.i("touch", "down event");
 break;
 case MotionEvent.ACTION_MOVE:
 Log.i("touch", "move event");
 break;
 case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 }
 return false;
 }
 }
40. public class CustomView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 
 switch (motionEvent.getAction()) {
 case MotionEvent.ACTION_DOWN:
 Log.i("touch", "down event");
 break;
 case MotionEvent.ACTION_MOVE:
 Log.i("touch", "move event");
 break;
 case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 }
 return false;
 }
 }
41. public class CustomView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 
 switch (motionEvent.getAction()) {
 case MotionEvent.ACTION_DOWN:
 Log.i("touch", "down event");
 break;
 case MotionEvent.ACTION_MOVE:
 Log.i("touch", "move event");
 break;
 case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 }
 return false;
 }
 }
42. Creating our own click listener
43. Creating our own click listener What does it take for a series of motion events to be considered a “click” or a “long click” ?
44. Creating our own click listener What does it take for a series of motion events to 
 be considered a “click” or a “long click” ? time it takes from the first down event till the last up event
45. Creating our own click listener What does it take for a series of motion events to 
 be considered a “click” or a “long click” ? time it takes from the first down event till the last up event difference in distance in between down and up
46. custom ‘click’ listener
47. custom ‘click’ listener case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 long eventEndTime = System.currentTimeMillis();
 
 // time in ms when the first down event registered
 long eventStartTime = motionEvent.getDownTime();
 long totalTimeElapsed = eventEndTime - eventStartTime;
 
 if (totalTimeElapsed < clickTimeoutMs) {
 performClick();
 } else if (totalTimeElapsed < longClickTimeoutMs) {
 performLongClick();
 } else {
 Log.i(TAG, "click action timed out");
 }
48. custom ‘click’ listener case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 long eventEndTime = System.currentTimeMillis();
 
 // time in ms when the first down event registered
 long eventStartTime = motionEvent.getDownTime();
 long totalTimeElapsed = eventEndTime - eventStartTime;
 
 if (totalTimeElapsed < clickTimeoutMs) {
 performClick();
 } else if (totalTimeElapsed < longClickTimeoutMs) {
 performLongClick();
 } else {
 Log.i(TAG, "click action timed out");
 }
49. custom ‘click’ listener case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 long eventEndTime = System.currentTimeMillis();
 
 // time in ms when the first down event registered
 long eventStartTime = motionEvent.getDownTime();
 long totalTimeElapsed = eventEndTime - eventStartTime;
 
 if (totalTimeElapsed < clickTimeoutMs) {
 performClick();
 } else if (totalTimeElapsed < longClickTimeoutMs) {
 performLongClick();
 } else {
 Log.i(TAG, "click action timed out");
 }
50. custom ‘click’ listener case MotionEvent.ACTION_UP:
 Log.i("touch", "up event");
 long eventEndTime = System.currentTimeMillis();
 
 // time in ms when the first down event registered
 long eventStartTime = motionEvent.getDownTime();
 long totalTimeElapsed = eventEndTime - eventStartTime;
 
 if (totalTimeElapsed < clickTimeoutMs) {
 performClick();
 } else if (totalTimeElapsed < longClickTimeoutMs) {
 performLongClick();
 } else {
 Log.i(TAG, "click action timed out");
 }
51. custom ‘click’ listener public class CustomView extends View {
 
 
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 … // input code we had previously
 }
 
 return false;
 }
 }
52. custom ‘click’ listener public class CustomView extends View {
 
 
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 … // input code we had previously
 }
 
 return false;
 }
 }
53. custom ‘click’ listener public class CustomView extends View {
 
 
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 … // input code we had previously
 }
 
 return false;
 }
 }
54. custom ‘click’ listener public class CustomView extends View {
 
 
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 … // input code we had previously
 }
 
 return false;
 }
 }
55. swipe down to dismiss
56. swipe down to dismiss 1. direction of the touch movement
57. swipe down to dismiss 1. direction of the touch movement 2. distance of the touch movement
58. swipe down to dismiss @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 
 if (motionEvent.getAction() == MotionEvent.ACTION_DOWN) {
 initialDownYCoord = motionEvent.getRawY();
 }
 
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 if (motionEvent.getRawY() - initialDownYCoord > SWIPE_DOWN_RANGE) { // swiped down
 startAnimation(slide_down); // slide_down is a predefined animation
 return true;
 }
 }
 
 return super.onTouchEvent(motionEvent);
 }
59. swipe down to dismiss @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 
 if (motionEvent.getAction() == MotionEvent.ACTION_DOWN) {
 initialDownYCoord = motionEvent.getRawY();
 }
 
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 if (motionEvent.getRawY() - initialDownYCoord > SWIPE_DOWN_RANGE) { // swiped down
 startAnimation(slide_down); // slide_down is a predefined animation
 return true;
 }
 }
 
 return super.onTouchEvent(motionEvent);
 }
60. swipe down to dismiss @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 
 if (motionEvent.getAction() == MotionEvent.ACTION_DOWN) {
 initialDownYCoord = motionEvent.getRawY();
 }
 
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 if (motionEvent.getRawY() - initialDownYCoord > SWIPE_DOWN_RANGE) { // swiped down
 startAnimation(slide_down); // slide_down is a predefined animation
 return true;
 }
 }
 
 return super.onTouchEvent(motionEvent);
 }
61. swipe down to dismiss @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 
 if (motionEvent.getAction() == MotionEvent.ACTION_DOWN) {
 initialDownYCoord = motionEvent.getRawY();
 }
 
 if (motionEvent.getAction() == MotionEvent.ACTION_UP && isTouchInView(motionEvent)) {
 if (motionEvent.getRawY() - initialDownYCoord > SWIPE_DOWN_RANGE) { // swiped down
 startAnimation(slide_down); // slide_down is a predefined animation
 return true;
 }
 }
 
 return super.onTouchEvent(motionEvent);
 }
62. Why have we been extending the View class?
63. Why have we been extending the View class? Based on what we’ve done so far, we could have simplified things with something such as :
 view.setOnTouchListener(new View.OnTouchListener() {
 @Override
 public boolean onTouch(View v, MotionEvent event) {
 return false;
 }
 });
64. Why have we been extending the View class? Based on what we’ve done so far, we could have simplified 
 things with something such as :
 view.setOnTouchListener(new View.OnTouchListener() {
 @Override
 public boolean onTouch(View v, MotionEvent event) {
 return false;
 }
 }); While onTouchListener is great , we get more control when we creating a custom view class
65. Extra methods available when implementing custom view classes onInterceptTouchEvent() dispatchTouchEvent()
66. boolean onInterceptTouchEvent()
67. boolean onInterceptTouchEvent() Called whenever a touch event occurs If this method returns true, the MotionEvent will not be passed to child Can be used to ‘listen’ to events and act accordingly
68. boolean dispatchTouchEvent()
69. boolean dispatchTouchEvent() Passes the motion event to the target view (or this view if it’s the target)
70. boolean dispatchTouchEvent() Passes the motion event to the target view (or this view if it’s the target) Returns true if handled by current view, false otherwise
71. For every MotionEvent : dispatchTouchEvent() is called in the order of Activity down to Button If FrameLayout.dispatchTouchEvent() returned true, Button would not call dispatchTouchEvent() Instead, FrameLayout.onTouchEvent() would be called Activity FrameLayout Button
72. For every MotionEvent : dispatchTouchEvent() is called in the order of Activity down to Button If FrameLayout.dispatchTouchEvent() returned true, Button would not call dispatchTouchEvent() Instead, FrameLayout.onTouchEvent() would be called Activity FrameLayout Button
73. For every MotionEvent : dispatchTouchEvent() is called in the order of Activity down to Button If FrameLayout.dispatchTouchEvent() returned true, Button would not call dispatchTouchEvent() Instead, FrameLayout.onTouchEvent() would be called Activity FrameLayout Button
74. If dispatchTouchEvent() gets to Button, and there’s no more child views, Button.onTouchEvent() would be called Activity FrameLayout Button
75. If dispatchTouchEvent() gets to Button, and there’s no more child views, Button.onTouchEvent() would be called onTouchEvent() is then called from Button to FrameLayout to Activity until one of the views decides to handle it Activity FrameLayout Button
76. How MotionEvents are propagated Activity.dispatchTouchEvent() FrameLayout.dispatchTouchEvent() Button.onTouchEvent() false Activity.onTouchEvent() FrameLayout.onTouchEvent() Button.dispatchTouchEvent() false
77. How MotionEvents are propagated Activity.dispatchTouchEvent() FrameLayout.dispatchTouchEvent() Button.onTouchEvent() Activity.onTouchEvent() true FrameLayout.onTouchEvent() Button.dispatchTouchEvent()
78. How MotionEvents are propagated Activity.dispatchTouchEvent() FrameLayout.dispatchTouchEvent() Button.onTouchEvent() false Activity.onTouchEvent() FrameLayout.onTouchEvent() Button.dispatchTouchEvent() false false
79. How MotionEvents are propagated Activity.dispatchTouchEvent() FrameLayout.dispatchTouchEvent() Button.onTouchEvent() false Activity.onTouchEvent() FrameLayout.onTouchEvent() Button.dispatchTouchEvent() true false
80. github.com/calren/viewsAndTouches
81. github.com/calren/viewsAndTouches @Override
 public boolean dispatchTouchEvent(MotionEvent motionEvent) {
 Log.i("touchevent", "dispatchTouchEvent ACTIVITY");
 return super.dispatchTouchEvent(motionEvent);
 }
 
 @Override
 public boolean onTouchEvent(MotionEvent motionEvent) {
 Log.i("touchevent", "onTouchEvent ACTIVITY");
 return super.onTouchEvent(motionEvent);
 }
84. Things to consider when customizing behavior
85. Things to consider when customizing behavior Most common gestures are already implemented (tap, doubleTap, fling)
86. Things to consider when customizing behavior Most common gestures are already implemented (tap, doubleTap, fling) Try to base values off base values defined by Android TOUCH_SLOP, TAP_TIMEOUT, PAGING_TOUCH_SLOP
87. In Summary… Custom touch event handling is complicated
88. In Summary… public boolean dispatchTouchEvent(MotionEvent event) {
 // If the event should be handled by accessibility focus first.
 if (event.isTargetAccessibilityFocus()) {
 // We don't have focus or no virtual descendant has it, do not handle the event.
 if (!isAccessibilityFocusedViewOrHost()) {
 return false;
 }
 // We have focus and got the event, then use normal event dispatch.
 event.setTargetAccessibilityFocus(false);
 }
 
 boolean result = false;
 
 if (mInputEventConsistencyVerifier != null) {
 mInputEventConsistencyVerifier.onTouchEvent(event, 0);
 }
 
 final int actionMasked = event.getActionMasked();
 if (actionMasked == MotionEvent.ACTION_DOWN) {
 // Defensive cleanup for new gesture
 stopNestedScroll();
 }
 
 if (onFilterTouchEventForSecurity(event)) {
 if ((mViewFlags & ENABLED_MASK) == ENABLED && handleScrollBarDragging(event)) {
 result = true;
 }
 //noinspection SimplifiableIfStatement
 ListenerInfo li = mListenerInfo;
 if (li != null && li.mOnTouchListener != null
 && (mViewFlags & ENABLED_MASK) == ENABLED
 && li.mOnTouchListener.onTouch(this, event)) {
 result = true;
 }
 
 if (!result && onTouchEvent(event)) {
 result = true;
 }
 }
 
 if (!result && mInputEventConsistencyVerifier != null) {
 mInputEventConsistencyVerifier.onUnhandledEvent(event, 0);
 }
 
 // Clean up after nested scrolls if this is the end of a gesture;
 // also cancel it if we tried an ACTION_DOWN but we didn't want the rest
 // of the gesture.
 if (actionMasked == MotionEvent.ACTION_UP ||
 actionMasked == MotionEvent.ACTION_CANCEL ||
 (actionMasked == MotionEvent.ACTION_DOWN && !result)) {
 stopNestedScroll();
 }
 
 return result;

89. In Summary… Custom touch event handling is complicated All touch events on Android are packaged into a MotionEvent
90. In Summary… Custom touch event handling is complicated All touch events on Android are packaged into a MotionEvent Touch events are passed down the view hierarchy until an interested view handles it onTouchEvent()
91. In Summary… Custom touch event handling is complicated All touch events on Android are packaged into a MotionEvent Touch events are passed down the view hierarchy until an interested view handles it onTouchEvent() Events can be intercepted and redirected as needed
 dispatchTouchEvent() and onInterceptTouchEvent()
92. Other great talks Writing Custom Views for Android Adam Powell, Romain Guy Measure, Layout, Draw, Repeat: Custom Views and ViewGroups Huyen Tue Dao Making Sense of the Touch System Philippe Breault Mastering the Android Touch System Dave Smith
93. Thanks! github.com/calren/viewsAndTouches @calren24