Making Sense of the touch system

Touch is pretty simple, right? You set an onClickListener and you are all set! But then... what happens when you want to drag stuff? What happens when you want to add gestures? How do you manage multiple views fighting for the focus? What is this pointer index thing? How do you animate a view after a fling and keep it natural? What is a touch slop?

In this talk, we will take a good look at what happens under the hood when your fingers touch the screen and how you can use this to create interesting ways to interact with your app!

Transcript

1. Philippe Breault droidcon NYC 2015 www.developerphil.com +PhilippeBreault @pbreault Making Sense of the Touch System
2. Click Listeners view.setOnClickListener(new OnClickListener() {
 @Override
 public void onClick(View v) {
 …
 }
 });

3. Implement a Button Challenge #1
4. Where? myView.setOnTouchListener(new OnTouchListener() {
 public boolean onTouch(View v, MotionEvent event) {
 … return false;
 }
 }); Listener
5. Where? public class MyView extends View {
 @Override
 public boolean onTouchEvent(MotionEvent event) {
 …
 return super.onTouchEvent(event);
 }
 }
 Override
6. Action MotionEvent ACTION_DOWN
 ACTION_UP ACTION_MOVE Data getX() getY()
7. The Code public boolean onTouchEvent(MotionEvent event) {
 super.onTouchEvent(event);
 switch (event.getActionMasked()) {
 case MotionEvent.ACTION_DOWN:
 setBackgroundColor(RED);
 break;
 
 case MotionEvent.ACTION_UP:
 setBackgroundColor(BLUE);
 if (isInside(event.getX(), event.getY()))
 toast("Click!");
 break;
 }
 
 return true;
 }
8. The Code public boolean onTouchEvent(MotionEvent event) {
 super.onTouchEvent(event);
 switch (event.getActionMasked()) {
 case MotionEvent.ACTION_DOWN:
 setBackgroundColor(RED);
 break;
 
 case MotionEvent.ACTION_UP:
 setBackgroundColor(BLUE);
 if (isInside(event.getX(), event.getY()))
 toast("Click!");
 break;
 }
 
 return true;
 }
9. The Code public boolean onTouchEvent(MotionEvent event) {
 super.onTouchEvent(event);
 switch (event.getActionMasked()) {
 case MotionEvent.ACTION_DOWN:
 setBackgroundColor(RED);
 break;
 
 case MotionEvent.ACTION_UP:
 setBackgroundColor(BLUE);
 if (isInside(event.getX(), event.getY()))
 toast("Click!");
 break;
 }
 
 return true;
 }
10. The Code public boolean onTouchEvent(MotionEvent event) {
 super.onTouchEvent(event);
 switch (event.getActionMasked()) {
 case MotionEvent.ACTION_DOWN:
 setBackgroundColor(RED);
 break;
 
 case MotionEvent.ACTION_UP:
 setBackgroundColor(BLUE);
 if (isInside(event.getX(), event.getY()))
 toast("Click!");
 break;
 }
 
 return true;
 }
11. The Code public boolean onTouchEvent(MotionEvent event) {
 super.onTouchEvent(event);
 switch (event.getActionMasked()) {
 case MotionEvent.ACTION_DOWN:
 setBackgroundColor(RED);
 break;
 
 case MotionEvent.ACTION_UP:
 setBackgroundColor(BLUE);
 if (isInside(event.getX(), event.getY()))
 toast("Click!");
 break;
 }
 
 return true;
 }
12. Implement a ScrollView Challenge #2
13. Action MotionEvent ACTION_MOVE ACTION_DOWN
 ACTION_UP Data getX() getY()
14. The Code @Override
 public boolean onTouchEvent(MotionEvent event) {
 super.onTouchEvent(event);
 
 switch (event.getActionMasked()) {
 case MotionEvent.ACTION_DOWN:
 mPreviousY = event.getY();
 break;
 
 case MotionEvent.ACTION_MOVE:
 float distance = mPreviousY - event.getY();
 scrollBy(0, Math.round(distance));
 mPreviousY = event.getY();
 break;
 }
 
 return true;
 }
15. The Code @Override
 public boolean onTouchEvent(MotionEvent event) {
 super.onTouchEvent(event);
 
 switch (event.getActionMasked()) {
 case MotionEvent.ACTION_DOWN:
 mPreviousY = event.getY();
 break;
 
 case MotionEvent.ACTION_MOVE:
 float distance = mPreviousY - event.getY();
 scrollBy(0, Math.round(distance));
 mPreviousY = event.getY();
 break;
 }
 
 return true;
 }
16. The Code @Override
 public boolean onTouchEvent(MotionEvent event) {
 super.onTouchEvent(event);
 
 switch (event.getActionMasked()) {
 case MotionEvent.ACTION_DOWN:
 mPreviousY = event.getY();
 break;
 
 case MotionEvent.ACTION_MOVE:
 float distance = mPreviousY - event.getY();
 scrollBy(0, Math.round(distance));
 mPreviousY = event.getY();
 break;
 }
 
 return true;
 }
17. Problem Buttons break scrolling!
18. Problem Buttons break scrolling!
19. Gesture ACTION_UP ACTION_DOWN …
20. Top Level View receives the gesture Touch Target Receives all MotionEvents for the rest of the gesture
21. Top Level View receives the gesture Touch Target Receives all MotionEvents for the rest of the gesture
22. Touch Target @Override
 public boolean onTouchEvent(MotionEvent event) {
 boolean handled = super.onTouchEvent(event);
 
 if (canHandleIt()) {
 // do stuff
 handled = true;
 }
 
 return handled;
 }
23. Touch Interception @Override
 public boolean onInterceptTouchEvent(MotionEvent event) {
 return super.onInterceptTouchEvent(ev);
 } @Override
 public boolean onTouchEvent(MotionEvent event) {
 . . .
 return super.onTouchEvent(ev);
 }
24. Touch Interception
25. • See MotionEvents before children Touch Interception
26. • See MotionEvents before children • Steal MotionEvents if needed Touch Interception
27. • See MotionEvents before children • Steal MotionEvents if needed • Only available in ViewGroups Touch Interception
28. • See MotionEvents before children • Steal MotionEvents if needed • Only available in ViewGroups • Interceptee receives ACTION_CANCEL Touch Interception
29. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down
30. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down
31. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down
32. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down Touch Target!
33. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down Touch Target! move
34. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down Touch Target! move
35. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down Touch Target! move
36. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down Touch Target! move cancel
37. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down Touch Target! move cancel
38. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down Touch Target! move cancel move
39. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down Touch Target! move cancel move
40. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down Touch Target! move cancel move up
41. Touch Interception ScrollView Button onInterceptTouchEvent onTouchEvent onTouchEvent down Touch Target! move cancel move up
42. The Code @Override
 public boolean onInterceptTouchEvent(MotionEvent ev) {
 boolean handled = super.onInterceptTouchEvent(ev);
 
 if (ev.getActionMasked() == ACTION_MOVE) {
 handled = true;
 mPreviousY = ev.getY();
 }
 return handled;
 }
43. The Code @Override
 public boolean onInterceptTouchEvent(MotionEvent ev) {
 boolean handled = super.onInterceptTouchEvent(ev);
 
 if (ev.getActionMasked() == ACTION_MOVE) {
 handled = true;
 mPreviousY = ev.getY();
 }
 return handled;
 }
44. The Code @Override
 public boolean onInterceptTouchEvent(MotionEvent ev) {
 boolean handled = super.onInterceptTouchEvent(ev);
 
 if (ev.getActionMasked() == ACTION_MOVE) {
 handled = true;
 mPreviousY = ev.getY();
 }
 return handled;
 } The state of the button is not restored when we scroll!
45. The Code 
 public boolean onTouchEvent(MotionEvent event) {
 super.onTouchEvent(event);
 switch (event.getActionMasked()) {
 case ACTION_DOWN:
 setPressedState();
 break;
 case ACTION_CANCEL:
 restoreState();
 break;
 case ACTION_UP:
 restoreState();
 if (isInside(event.getX(), event.getY())) {
 toast("Click!");
 }
 break;
 }
 
 return true;
 }
46. The Code 
 public boolean onTouchEvent(MotionEvent event) {
 super.onTouchEvent(event);
 switch (event.getActionMasked()) {
 case ACTION_DOWN:
 setPressedState();
 break;
 case ACTION_CANCEL:
 restoreState();
 break;
 case ACTION_UP:
 restoreState();
 if (isInside(event.getX(), event.getY())) {
 toast("Click!");
 }
 break;
 }
 
 return true;
 }
47. The Code
48. The Code Your fingers will move
 Allow some movement
49. ViewConfiguration mTouchSlop = ViewConfiguration.get(getContext()).getScaledTouchSlop();
50. ViewConfiguration int getScaledTouchSlop() int getScaledDoubleTapSlop()
 int getScaledEdgeSlop()
 int getScaledFadingEdgeLength()
 int getScaledMaximumFlingVelocity()
 int getScaledMinimumFlingVelocity()
 int getScaledOverflingDistance()
 int getScaledOverscrollDistance()
 int getScaledPagingTouchSlop()
 int getScaledScrollBarSize()
 int getScaledWindowTouchSlop()
51. ViewConfiguration int getScaledTouchSlop() int getScaledDoubleTapSlop()
 int getScaledEdgeSlop()
 int getScaledFadingEdgeLength()
 int getScaledMaximumFlingVelocity()
 int getScaledMinimumFlingVelocity()
 int getScaledOverflingDistance()
 int getScaledOverscrollDistance()
 int getScaledPagingTouchSlop()
 int getScaledScrollBarSize()
 int getScaledWindowTouchSlop() Lots of useful values
52. The Code @Override
 public boolean onInterceptTouchEvent(MotionEvent ev) {
 boolean handled = super.onInterceptTouchEvent(ev);
 
 switch (ev.getActionMasked()) {
 case ACTION_DOWN:
 initialDown = ev.getY();
 break;
 case ACTION_MOVE:
 float dy = ev.getY() - initialDown;
 if (Math.abs(dy) > touchSlop) {
 mPreviousY = ev.getY();
 handled = true;
 }
 break;
 }
 
 return handled;
 }
53. The Code @Override
 public boolean onInterceptTouchEvent(MotionEvent ev) {
 boolean handled = super.onInterceptTouchEvent(ev);
 
 switch (ev.getActionMasked()) {
 case ACTION_DOWN:
 initialDown = ev.getY();
 break;
 case ACTION_MOVE:
 float dy = ev.getY() - initialDown;
 if (Math.abs(dy) > touchSlop) {
 mPreviousY = ev.getY();
 handled = true;
 }
 break;
 }
 
 return handled;
 }
54. The Code @Override
 public boolean onInterceptTouchEvent(MotionEvent ev) {
 boolean handled = super.onInterceptTouchEvent(ev);
 
 switch (ev.getActionMasked()) {
 case ACTION_DOWN:
 initialDown = ev.getY();
 break;
 case ACTION_MOVE:
 float dy = ev.getY() - initialDown;
 if (Math.abs(dy) > touchSlop) {
 mPreviousY = ev.getY();
 handled = true;
 }
 break;
 }
 
 return handled;
 }
55. The Code @Override
 public boolean onInterceptTouchEvent(MotionEvent ev) {
 boolean handled = super.onInterceptTouchEvent(ev);
 
 switch (ev.getActionMasked()) {
 case ACTION_DOWN:
 initialDown = ev.getY();
 break;
 case ACTION_MOVE:
 float dy = ev.getY() - initialDown;
 if (Math.abs(dy) > touchSlop) {
 mPreviousY = ev.getY();
 handled = true;
 }
 break;
 }
 
 return handled;
 }
56. Implement Flinging Challenge #3
57. • Computes Velocity in pixels/milliseconds • Used to calculate a natural motion VelocityTracker velocityTracker = VelocityTracker.obtain();
 velocityTracker.addMovement(event);
 velocityTracker.computeCurrentVelocity(1);
 velocityTracker.getXVelocity();
 velocityTracker.recycle();
58. VelocityTracker @Override
 public boolean onTouchEvent(MotionEvent event) {
 switch (event.getActionMasked()) {
 case ACTION_UP:
 velocityTracker.computeCurrentVelocity(1);
 float xVelocity = velocityTracker.getXVelocity();
 float yVelocity = velocityTracker.getYVelocity();
 handleFling(xVelocity, yVelocity);
 break;
 } 
 velocityTracker.addMovement(event);
 
 return super.onTouchEvent(event);
 }
59. VelocityTracker @Override
 public boolean onTouchEvent(MotionEvent event) {
 switch (event.getActionMasked()) {
 case ACTION_UP:
 velocityTracker.computeCurrentVelocity(1);
 float xVelocity = velocityTracker.getXVelocity();
 float yVelocity = velocityTracker.getYVelocity();
 handleFling(xVelocity, yVelocity);
 break;
 } 
 velocityTracker.addMovement(event);
 
 return super.onTouchEvent(event);
 }
60. VelocityTracker @Override
 public boolean onTouchEvent(MotionEvent event) {
 switch (event.getActionMasked()) {
 case ACTION_UP:
 velocityTracker.computeCurrentVelocity(1);
 float xVelocity = velocityTracker.getXVelocity();
 float yVelocity = velocityTracker.getYVelocity();
 handleFling(xVelocity, yVelocity);
 break;
 } 
 velocityTracker.addMovement(event);
 
 return super.onTouchEvent(event);
 }
61. • Natural scrolling behaviour for the platform • Computes values • The view has to apply the values Scroller
62. Scroller private void onFling(float velocityX, float velocityY) { scroller.fling(getScrollX(), getScrollY(),
 (int) -velocityX, (int) -velocityY,
 minScrollX, maxScrollX,
 minScrollY, maxScrollY);
 invalidate();
 }
 
 @Override
 public void computeScroll() {
 super.computeScroll();
 if (!scroller.isFinished()) {
 scroller.computeScrollOffset();
 scrollTo(scroller.getCurrX(), scroller.getCurrY());
 postInvalidateOnAnimation();
 }
 }
63. Scroller private void onFling(float velocityX, float velocityY) { scroller.fling(getScrollX(), getScrollY(),
 (int) -velocityX, (int) -velocityY,
 minScrollX, maxScrollX,
 minScrollY, maxScrollY);
 invalidate();
 }
 
 @Override
 public void computeScroll() {
 super.computeScroll();
 if (!scroller.isFinished()) {
 scroller.computeScrollOffset();
 scrollTo(scroller.getCurrX(), scroller.getCurrY());
 postInvalidateOnAnimation();
 }
 }
64. Implement a Drawing View Challenge #4
65. Simple Drawing 
 case MotionEvent.ACTION_DOWN:
 path = new Path();
 path.setLastPoint(event.getX(), event.getY());
 break;
 case MotionEvent.ACTION_MOVE:
 path.lineTo(event.getX(), event.getY()); break;
 }
66. Simple Drawing 
 case MotionEvent.ACTION_DOWN:
 path = new Path();
 path.setLastPoint(event.getX(), event.getY());
 break;
 case MotionEvent.ACTION_MOVE:
 path.lineTo(event.getX(), event.getY()); break;
 }
67. Simple Drawing Breaks if the app lags!
68. Simple Drawing Breaks if the app lags!
69. • History of ACTION_MOVE between frames • Useful for timing dependent gestures Batching event.getHistorySize() event.getHistoricalX(i) event.getHistoricalY(i)
70. Simple Drawing 
 case MotionEvent.ACTION_DOWN:
 path = new Path();
 path.setLastPoint(event.getX(), event.getY());
 break;
 case MotionEvent.ACTION_MOVE: for (int i = 0; i < event.getHistorySize(); i++) {
 path.lineTo(event.getHistoricalX(i), event.getHistoricalY(i));
 }
 path.lineTo(event.getX(), event.getY()); break;
 }
71. Simple Drawing 
 case MotionEvent.ACTION_DOWN:
 path = new Path();
 path.setLastPoint(event.getX(), event.getY());
 break;
 case MotionEvent.ACTION_MOVE: for (int i = 0; i < event.getHistorySize(); i++) {
 path.lineTo(event.getHistoricalX(i), event.getHistoricalY(i));
 }
 path.lineTo(event.getX(), event.getY()); break;
 }
72. Implement Multi-Touch Challenge #5
73. Multi-Touch
74. • A finger is considered a pointer Multi-Touch
75. • A finger is considered a pointer • Each pointer has an id Multi-Touch
76. • A finger is considered a pointer • Each pointer has an id • Each MotionEvent contains all pointers Multi-Touch
77. ACTION_UP ACTION_DOWN ACTION_POINTER_DOWN ACTION_POINTER_UP Pointer 1 Pointer 2 Multi-touch Gesture
78. Action MotionEvent ACTION_DOWN ACTION_MOVE ACTION_UP ACTION_CANCEL ACTION_POINTER_DOWN ACTION_POINTER_UP Data findPointerIndex() getPointerId() getX(pointerIndex) getY(pointerIndex)
79. Index vs Id Index Id Fake Phone! 0 1 2
80. Index vs Id 0 Index Id Fake Phone! 0 1 2 0
81. Index vs Id 0 Index Id 1 Fake Phone! 0 1 2 0 1
82. Index vs Id 0 Index Id 1 2 Fake Phone! 0 1 2 0 1 2
83. Index vs Id 0 Index Id 2 Fake Phone! 0 1 2 0 2
84. Index vs Id 0 Index Id 2 Fake Phone! 0 1 2 0 2
85. Focal Point switch (event.getActionMasked()) {
 case ACTION_DOWN:
 pointerId1 = event.getPointerId(0);
 break;
 case ACTION_POINTER_DOWN:
 pointerId2 = event.getPointerId(event.getActionIndex());
 case ACTION_MOVE:
 int index1 = event.findPointerIndex(pointerId1);
 int index2 = event.findPointerIndex(pointerId2);
 if (index1 != -1 && index2 != -1) {
 focalX = (event.getX(index1) + event.getX(index2) / 2f);
 focalY = (event.getY(index1) + event.getY(index2) / 2f);
 }
 break;
 }

86. Focal Point switch (event.getActionMasked()) {
 case ACTION_DOWN:
 pointerId1 = event.getPointerId(0);
 break;
 case ACTION_POINTER_DOWN:
 pointerId2 = event.getPointerId(event.getActionIndex());
 case ACTION_MOVE:
 int index1 = event.findPointerIndex(pointerId1);
 int index2 = event.findPointerIndex(pointerId2);
 if (index1 != -1 && index2 != -1) {
 focalX = (event.getX(index1) + event.getX(index2) / 2f);
 focalY = (event.getY(index1) + event.getY(index2) / 2f);
 }
 break;
 }

87. • Accumulates touch events and notifies you • Does the hard work for you • Multitouch ready Gesture Detector
88. Gesture Detector @Override
 public boolean onTouchEvent(MotionEvent event) {
 
 gestureDetector.onTouchEvent(event);
 
 return super.onTouchEvent(event);
 }
89. Gesture Detector public interface OnGestureListener {
 
 boolean onDown(MotionEvent e);
 
 void onShowPress(MotionEvent e);
 
 boolean onSingleTapUp(MotionEvent e);
 
 boolean onScroll(MotionEvent e1, MotionEvent e2, 
 float distanceX, float distanceY);
 
 void onLongPress(MotionEvent e);
 
 boolean onFling(MotionEvent e1, MotionEvent e2, 
 float velocityX, float velocityY);
 }
90. ScaleGestureDetectorCompat new ScaleGestureDetector.SimpleOnScaleGestureListener() {
 
 @Override
 public boolean onScaleBegin(ScaleGestureDetector detector) {
 return true;
 }
 
 @Override
 public boolean onScale(ScaleGestureDetector detector) {
 detector.getScaleFactor();
 detector.getCurrentSpanX();
 detector.getCurrentSpanY();
 detector.getFocusX();
 detector.getFocusY();
 return true;
 }
 }
91. Misc #notachallenge
92. • Activity will dispatch MotionEvents to its root layout • Viewgroups will dispatch touch to their children Dispatch
93. Activity ViewGroup View dispatchTouchEvent dispatchTouchEvent dispatchTouchEvent onInterceptTouchEvent onTouchEvent onTouchEvent onTouchEvent Dispatch
94. • Increase the touch area of a child • Delegates a portion of the parent to the child TouchDelegate
95. TouchDelegate button.post(new Runnable() {
 @Override
 public void run() {
 Rect hitRect = new Rect();
 button.getHitRect(hitRect);
 hitRect.inset(-100, -100);
 if (button.getParent() instanceof ViewGroup) {
 ViewGroup parent = (ViewGroup) button.getParent();
 parent.setTouchDelegate(new TouchDelegate(hitRect, button));
 }
 }
 });
96. button.post(new Runnable() {
 @Override
 public void run() {
 Rect hitRect = new Rect();
 button.getHitRect(hitRect);
 hitRect.inset(-100, -100);
 if (button.getParent() instanceof ViewGroup) {
 ViewGroup parent = (ViewGroup) button.getParent();
 parent.setTouchDelegate(new TouchDelegate(hitRect, button));
 }
 }
 }); TouchDelegate
97. button.post(new Runnable() {
 @Override
 public void run() {
 Rect hitRect = new Rect();
 button.getHitRect(hitRect);
 hitRect.inset(-100, -100);
 if (button.getParent() instanceof ViewGroup) {
 ViewGroup parent = (ViewGroup) button.getParent();
 parent.setTouchDelegate(new TouchDelegate(hitRect, button));
 }
 }
 }); TouchDelegate
98. button.post(new Runnable() {
 @Override
 public void run() {
 Rect hitRect = new Rect();
 button.getHitRect(hitRect);
 hitRect.inset(-100, -100);
 if (button.getParent() instanceof ViewGroup) {
 ViewGroup parent = (ViewGroup) button.getParent();
 parent.setTouchDelegate(new TouchDelegate(hitRect, button));
 }
 }
 }); TouchDelegate
99. button.post(new Runnable() {
 @Override
 public void run() {
 Rect hitRect = new Rect();
 button.getHitRect(hitRect);
 hitRect.inset(-100, -100);
 if (button.getParent() instanceof ViewGroup) {
 ViewGroup parent = (ViewGroup) button.getParent();
 parent.setTouchDelegate(new TouchDelegate(hitRect, button));
 }
 }
 }); TouchDelegate
100. button.post(new Runnable() {
 @Override
 public void run() {
 Rect hitRect = new Rect();
 button.getHitRect(hitRect);
 hitRect.inset(-100, -100);
 if (button.getParent() instanceof ViewGroup) {
 ViewGroup parent = (ViewGroup) button.getParent();
 parent.setTouchDelegate(new TouchDelegate(hitRect, button));
 }
 }
 }); TouchDelegate
101. button.post(new Runnable() {
 @Override
 public void run() {
 Rect hitRect = new Rect();
 button.getHitRect(hitRect);
 hitRect.inset(-100, -100);
 if (button.getParent() instanceof ViewGroup) {
 ViewGroup parent = (ViewGroup) button.getParent();
 parent.setTouchDelegate(new TouchDelegate(hitRect, button));
 }
 }
 }); TouchDelegate
102. • Part of View • Handle Non-Touch Events • Mouse Hover • Joystick OnGenericMotionEvent public boolean onGenericMotionEvent(MotionEvent event)
103. • Touch multiple views at the same time 
 with different fingers • On by default • Per layout Split Touch Events <LinearLayout
 android:layout_width="wrap_content"
 android:layout_height="wrap_content"
 android:splitMotionEvents="false"
 >
 …
 </LinearLayout>

104. Philippe Breault droidcon NYC 2015 www.developerphil.com +PhilippeBreault @pbreault THE END Questions?