Share
Embed
The Challenges Behind a Better Sticker Input Experience
Jiaheng Huang
LINE Fukuoka Development H Team Software Engineer
Yu Yang
LINE Fukuoka Development H Team Android Engineer
https://linedevday.linecorp.com/2020/ja/sessions/7642
https://linedevday.linecorp.com/2020/en/sessions/7642
LINE DevDay 2020
November 26, 2020
Transcript
87 slides
-
Agenda › An introduction of the new LINE sticker keyboard
› A deep dive on the rendering performance issue on Android › Create smooth expand & collapse animations on iOS -
Our initial solution… <androidx.recyclerview.widget.RecyclerView android:id="@+id/sticker_grid_view" android:layout_width="match_parent" android:layout_height="wrap_content"/>
<androidx.recyclerview.widget.RecyclerView android:id=“@+id/author_latest_stickers_view" android:layout_width="match_parent" android:layout_height="wrap_content" /> -
Our initial solution… <androidx.core.widget.NestedScrollView android:id="@+id/nested_scroll_view" android:layout_width="match_parent" android:layout_height="match_parent">
<LinearLayout android:layout_width="match_parent" android:layout_height="wrap_content" android:orientation="vertical"> <androidx.recyclerview.widget.RecyclerView android:id="@+id/sticker_grid_view" android:layout_width="match_parent" android:layout_height="wrap_content"/> <androidx.recyclerview.widget.RecyclerView android:id="@+id/author_latest_stickers_view" android:layout_width="match_parent" android:layout_height="wrap_content" /> </LinearLayout> </androidx.core.widget.NestedScrollView> -
How Android draws views? ViewGroup View View DecorView ViewGroup Pre-order
traversal of the layout tree -
How Android draws views? ViewRootImpl.java private void performTraversals() {
... int childWidthMeasureSpec = getRootMeasureSpec(mWidth, lp.width); int childHeightMeasureSpec = getRootMeasureSpec(mHeight, lp.height); ... performMeasure(childWidthMeasureSpec, childHeightMeasureSpec); ... performLayout(lp, mWidth, mHeight); ... performDraw(); } -
How Android draws views? ViewRootImpl.java private void performTraversals() {
... int childWidthMeasureSpec = getRootMeasureSpec(mWidth, lp.width); int childHeightMeasureSpec = getRootMeasureSpec(mHeight, lp.height); ... performMeasure(childWidthMeasureSpec, childHeightMeasureSpec); ... performLayout(lp, mWidth, mHeight); ... performDraw(); } -
How Android draws views? ViewRootImpl.java private void performTraversals() {
... int childWidthMeasureSpec = getRootMeasureSpec(mWidth, lp.width); int childHeightMeasureSpec = getRootMeasureSpec(mHeight, lp.height); ... performMeasure(childWidthMeasureSpec, childHeightMeasureSpec); ... performLayout(lp, mWidth, mHeight); ... performDraw(); } -
How Android draws views? ViewRootImpl.java private void performTraversals() {
... int childWidthMeasureSpec = getRootMeasureSpec(mWidth, lp.width); int childHeightMeasureSpec = getRootMeasureSpec(mHeight, lp.height); ... performMeasure(childWidthMeasureSpec, childHeightMeasureSpec); ... performLayout(lp, mWidth, mHeight); ... performDraw(); } -
How Android draws views? Measure Layout Draw performTraversals For a
child view to be measured, two specifications are required: -
How Android draws views? Measure Layout Draw performTraversals › MATCH_PARENT
› WRAP_CONTENT › an exact number (e.g.,100dp) ViewGroup.LayoutParams For a child view to be measured, two specifications are required: -
How Android draws views? Measure Layout Draw performTraversals › MATCH_PARENT
› WRAP_CONTENT › an exact number (e.g.,100dp) ViewGroup.LayoutParams MeasureSpec › UNSPECIFIED › EXACTLY › AT MOST For a child view to be measured, two specifications are required: -
How a RecycleView inside a NestedScrollView is measured? NestedScrollView.java protected
void measureChild(..., int parentWidthMeasureSpec, int parentHeightMeasureSpec) { } -
How a RecycleView inside a NestedScrollView is measured? NestedScrollView.java protected
void measureChild(..., int parentWidthMeasureSpec, int parentHeightMeasureSpec) { ... childWidthMeasureSpec = getChildMeasureSpec(...); childHeightMeasureSpec = MeasureSpec.makeMeasureSpec(0, MeasureSpec.UNSPECIFIED); child.measure(childWidthMeasureSpec, childHeightMeasureSpec); } -
How a RecycleView inside a NestedScrollView is measured? NestedScrollView.java protected
void measureChild(..., int parentWidthMeasureSpec, int parentHeightMeasureSpec) { ... childWidthMeasureSpec = getChildMeasureSpec(...); childHeightMeasureSpec = MeasureSpec.makeMeasureSpec(0, MeasureSpec.UNSPECIFIED); child.measure(childWidthMeasureSpec, childHeightMeasureSpec); } -
How a RecycleView inside a NestedScrollView is measured? NestedScrollView.java protected
void measureChild(..., int parentWidthMeasureSpec, int parentHeightMeasureSpec) { ... childWidthMeasureSpec = getChildMeasureSpec(...); childHeightMeasureSpec = MeasureSpec.makeMeasureSpec(0, MeasureSpec.UNSPECIFIED); child.measure(childWidthMeasureSpec, childHeightMeasureSpec); } -
How a RecycleView inside a NestedScrollView is measured? RecyclerView.java
protected void onMeasure(int widthSpec, int heightSpec) { ... final int widthMode = MeasureSpec.getMode(widthSpec); final int heightMode = MeasureSpec.getMode(heightSpec); mLayout.onMeasure(mRecycler, mState, widthSpec, heightSpec); final boolean measureSpecModeIsExactly = widthMode == MeasureSpec.EXACTLY && heightMode == MeasureSpec.EXACTLY; if (measureSpecModeIsExactly || mAdapter == null) return; ... mLayout.setMeasuredDimensionFromChildren(widthSpec, heightSpec); ... } -
How a RecycleView inside a NestedScrollView is measured? RecyclerView.java
protected void onMeasure(int widthSpec, int heightSpec) { ... final int widthMode = MeasureSpec.getMode(widthSpec); final int heightMode = MeasureSpec.getMode(heightSpec); mLayout.onMeasure(mRecycler, mState, widthSpec, heightSpec); final boolean measureSpecModeIsExactly = widthMode == MeasureSpec.EXACTLY && heightMode == MeasureSpec.EXACTLY; if (measureSpecModeIsExactly || mAdapter == null) return; ... mLayout.setMeasuredDimensionFromChildren(widthSpec, heightSpec); ... } -
How a RecycleView inside a NestedScrollView is measured? RecyclerView.java
void setMeasuredDimensionFromChildren(int widthSpec, int heightSpec) { ... for (int i = 0; i < count; i++) { View child = getChildAt(i); final Rect bounds = mRecyclerView.mTempRect; getDecoratedBoundsWithMargins(child, bounds); if (bounds.left < minX) minX = bounds.left; if (bounds.right > maxX) maxX = bounds.right; if (bounds.top < minY) minY = bounds.top; if (bounds.bottom > maxY) maxY = bounds.bottom; } mRecyclerView.mTempRect.set(minX, minY, maxX, maxY); setMeasuredDimension(mRecyclerView.mTempRect, widthSpec, heightSpec); } -
How a RecycleView inside a NestedScrollView is measured? RecyclerView.java
void setMeasuredDimensionFromChildren(int widthSpec, int heightSpec) { ... for (int i = 0; i < count; i++) { View child = getChildAt(i); final Rect bounds = mRecyclerView.mTempRect; getDecoratedBoundsWithMargins(child, bounds); if (bounds.left < minX) minX = bounds.left; if (bounds.right > maxX) maxX = bounds.right; if (bounds.top < minY) minY = bounds.top; if (bounds.bottom > maxY) maxY = bounds.bottom; } mRecyclerView.mTempRect.set(minX, minY, maxX, maxY); setMeasuredDimension(mRecyclerView.mTempRect, widthSpec, heightSpec); } -
How a RecycleView inside a NestedScrollView is measured? RecyclerView.java
public void setMeasuredDimension(Rect childrenBounds, int wSpec, int hSpec) { int usedWidth = childrenBounds.width() + getPaddingLeft() + getPaddingRight(); int usedHeight = childrenBounds.height() + getPaddingTop() + getPaddingBottom(); int width = chooseSize(wSpec, usedWidth, getMinimumWidth()); int height = chooseSize(hSpec, usedHeight, getMinimumHeight()); setMeasuredDimension(width, height); } -
How a RecycleView inside a NestedScrollView is measured? RecyclerView.java
public void setMeasuredDimension(Rect childrenBounds, int wSpec, int hSpec) { int usedWidth = childrenBounds.width() + getPaddingLeft() + getPaddingRight(); int usedHeight = childrenBounds.height() + getPaddingTop() + getPaddingBottom(); int width = chooseSize(wSpec, usedWidth, getMinimumWidth()); int height = chooseSize(hSpec, usedHeight, getMinimumHeight()); setMeasuredDimension(width, height); } -
How a RecycleView inside a NestedScrollView is measured? RecyclerView.java
public void setMeasuredDimension(Rect childrenBounds, int wSpec, int hSpec) { int usedWidth = childrenBounds.width() + getPaddingLeft() + getPaddingRight(); int usedHeight = childrenBounds.height() + getPaddingTop() + getPaddingBottom(); int width = chooseSize(wSpec, usedWidth, getMinimumWidth()); int height = chooseSize(hSpec, usedHeight, getMinimumHeight()); setMeasuredDimension(width, height); } -
How a RecycleView inside a NestedScrollView is measured? RecyclerView.java
public static int chooseSize(int spec, int desired, int min) { final int mode = View.MeasureSpec.getMode(spec); final int size = View.MeasureSpec.getSize(spec); switch (mode) { case View.MeasureSpec.EXACTLY: return size; case View.MeasureSpec.AT_MOST: return Math.min(size, Math.max(desired, min)); case View.MeasureSpec.UNSPECIFIED: default: return Math.max(desired, min); } } -
How a RecycleView inside a NestedScrollView is measured? RecyclerView.java
public static int chooseSize(int spec, int desired, int min) { final int mode = View.MeasureSpec.getMode(spec); final int size = View.MeasureSpec.getSize(spec); switch (mode) { case View.MeasureSpec.EXACTLY: return size; case View.MeasureSpec.AT_MOST: return Math.min(size, Math.max(desired, min)); case View.MeasureSpec.UNSPECIFIED: default: return Math.max(desired, min); } } -
How a RecycleView inside a NestedScrollView is measured? RecyclerView.java
public static int chooseSize(int spec, int desired, int min) { final int mode = View.MeasureSpec.getMode(spec); final int size = View.MeasureSpec.getSize(spec); switch (mode) { case View.MeasureSpec.EXACTLY: return size; case View.MeasureSpec.AT_MOST: return Math.min(size, Math.max(desired, min)); case View.MeasureSpec.UNSPECIFIED: default: return Math.max(desired, min); } } -
Our initial solution… So, the height of the RecyclerView would
not be restricted, and its height would be the total height of all child view items and the paddings (if any). In other words, all child view items will be rendered at once. -
The combination of LayoutParams and MeasureSpec EXACTLY AT_MOST UNSPECIFIED DP
EXACTLY childSize EXACTLY childSize EXACTLY childSize MATCH_PARENT EXACTLY parentSize AT_MOST parentSize UNSPECIFIED 0 WRAP_CONTENT AT_MOST parentSize AT_MOST parentSize UNSPECIFIED 0 -
The combination of LayoutParams and MeasureSpec EXACTLY AT_MOST UNSPECIFIED DP
EXACTLY childSize EXACTLY childSize EXACTLY childSize MATCH_PARENT EXACTLY parentSize AT_MOST parentSize UNSPECIFIED 0 WRAP_CONTENT AT_MOST parentSize AT_MOST parentSize UNSPECIFIED 0 -
The combination of LayoutParams and MeasureSpec EXACTLY AT_MOST UNSPECIFIED DP
EXACTLY childSize EXACTLY childSize EXACTLY childSize MATCH_PARENT EXACTLY parentSize AT_MOST parentSize UNSPECIFIED 0 WRAP_CONTENT AT_MOST parentSize AT_MOST parentSize UNSPECIFIED 0 -
How this problem could be solved? › Use an exact
height value instead of WRAP_CONTENT for the RecyclerView. -
How this problem could be solved? › DO NOT wrap
a RecyclerView inside a NestedScrollView. Instead, use a single RecyclerView with multiple view types to achieve complex UI. › Use an exact height value instead of WRAP_CONTENT for the RecyclerView. -
How this problem could be solved? › DO NOT wrap
a RecyclerView inside a NestedScrollView. Instead, use a single RecyclerView with multiple view types to achieve complex UI. › Luckily LINE client has its own customized adapter called RecyclerViewModelAdapter to easily support multiple view types in a RecyclerView. › Use an exact height value instead of WRAP_CONTENT for the RecyclerView. -
Recap › When MATCH_PARENT or WRAP_CONTENT is specified for a
RecyclerView’s height inside a NestedScrollView, all list view items would rendered at once. › Discussed how a RecyclerView inside a NestedScrollView is measured. -
Recap › When MATCH_PARENT or WRAP_CONTENT is specified for a
RecyclerView’s height inside a NestedScrollView, all list view items would rendered at once. › Discussed how a RecyclerView inside a NestedScrollView is measured. › DO NOT put a RecyclerView inside a NestedScrollView unless the height is known in advance. -
Challenge We don’t want to change keyboard height during expanding
animation, otherwise the input bar will go up Keyboard is not full screen height, the expand animation will go beyond keyboard area -
How to implement this? Calculate layout frame by frame ›
Too much verbose calculation -
How to implement this? UIPropertyAnimator? Calculate layout frame by frame
› Too much verbose calculation -
UIPropertyAnimator › Introduced in iOS 10 › Can modify animation
dynamically › eg. Scrub through a animation by modifying the fractionComplete property -
How to implement this? › Need to handle hitTest(_:with:) through
out the view hierarchy UIPropertyAnimator Calculate layout frame by frame -
How to implement this? › Need to handle hitTest(_:with:) through
out the view hierarchy › Not a good place to handle background dimming logic UIPropertyAnimator Calculate layout frame by frame -
How to implement this? How about presenting a view controller
here? › Present view controller works on keyboard -
How to implement this? How about presenting a view controller
here? › Present view controller works on keyboard › Still can take advantage of the powerful UIPropertyAnimator inside UIViewControllerTransitioningDelegate -
How to implement this? How about presenting a view controller
here? › Present view controller works on keyboard › Still can take advantage of the powerful UIPropertyAnimator inside UIViewControllerTransitioningDelegate › No need to handle hitTest(_:with:) -
How to implement this? How about presenting a view controller
here? › Present view controller works on keyboard › Still can take advantage of the powerful UIPropertyAnimator inside UIViewControllerTransitioningDelegate › No need to handle hitTest(_:with:) › Can handle background dimming logic and device rotation easily in UIPresentationController -
Implementation In View Controller Define two states enum State {
case expanded, collapsed } -
Implementation In View Controller Layout UI during view controller appear
and disappear using transition coordinator Define two states enum State { case expanded, collapsed } transitionCoordinator?.animate(alongsideTransition: { [weak self] (_) in self?.configUI(.expanded) }, completion: nil) -
› Applied iOS 13 default presenting animation › Has interactive
dismiss but no interactive present › Incorrect initial frame of presentation Demo -
UIViewControllerTransitioningDelegate UIViewController TransitioningDelegate Presented View Controller Presenting View Controller Animator
Interactive Animator Presentation Controller -
Implementation In UIViewControllerTransitioningDelegate Create a custom Presentation Controller func presentationController(forPresented
presented: UIViewController, presenting: UIViewController?, source: UIViewController) -> UIPresentationController? › Define final layout of presented view controller › Add an extra view to do dimming background -
Presentation Controller Define final layout class PresentationController: UIPresentationController { private
var topSpacing: CGFloat override var frameOfPresentedViewInContainerView: CGRect { let containerSize = containerView!.bounds.size var frame: CGRect = .zero frame.size = size( forChildContentContainer: presentedViewController, withParentContainerSize: containerSize ) return frame } override func size( forChildContentContainer container: UIContentContainer, withParentContainerSize parentSize: CGSize ) -> CGSize { let size = CGSize(width: parentSize.width, height: parentSize.height - topSpacing) return size } } -
Presentation Controller Add dimming view class PresentationController: UIPresentationController { private
let dimmingView: UIView override func presentationTransitionWillBegin() { containerView?.insertSubview(dimmingView, at: 0) ... presentedViewController.transitionCoordinator?.animate(alongsideTransition: { _ in self.dimmingView.alpha = 0.7 }) } override func presentationTransitionDidEnd(_ completed: Bool) { if !completed { dimmingView.removeFromSuperview() } } override func dismissalTransitionWillBegin() { presentedViewController.transitionCoordinator?.animate(alongsideTransition: { _ in self.dimmingView.alpha = 0.0 }) } override func dismissalTransitionDidEnd(_ completed: Bool) { if completed { dimmingView.removeFromSuperview() } } } -
Implementation In UIViewControllerTransitioningDelegate Create an Animator Object Create a custom
Presentation Controller func animationController(forDismissed dismissed: UIViewController) -> UIViewControllerAnimatedTransitioning? func animationController(forPresented presented: UIViewController, presenting: UIViewController, source: UIViewController) -> UIViewControllerAnimatedTransitioning? { › Set correct initial frame for presented view controller in Animator › Use UIPropertyAnimator to define a interruptible animation -
Animator class TagSearchResultTransitionAnimator: NSObject, UIViewControllerAnimatedTransitioning { let isPresenting: Bool private
func transitionAnimator(using transitionContext: UIViewControllerContextTransitioning) -> UIViewImplicitlyAnimating { // Set Start position if isPresenting { container.addSubview(presentedView) presentedView.frame = startFrame } let springTiming = UISpringTimingParameters(…) let animator = UIViewPropertyAnimator(duration: duration, timingParameters: springTiming) animator.addAnimations { presentedView.frame = self.isPresenting ? finalFrame : startFrame } animator.addCompletion { position in let didComplete = !transitionContext.transitionWasCancelled // After a failed presentation or successful dismissal, remove the view. if self.isPresenting && !didComplete || !self.isPresenting && didComplete { presentedView.removeFromSuperview() } transitionContext.completeTransition(didComplete) } return animator } } -
Implementation In UIViewControllerTransitioningDelegate Create an Animator Object Create a custom
Presentation Controller Create an Interactive Animator › The easiest way to create an interactive animator is to subclass UIPercentDrivenInteractiveTransition class and add gesture recognizer event-handling code to your subclass func interactionControllerForPresentation(using animator: UIViewControllerAnimatedTransitioning) -> UIViewControllerInteractiveTransitioning? func interactionControllerForDismissal(using animator: UIViewControllerAnimatedTransitioning) -> UIViewControllerInteractiveTransitioning? -
Interactive Animator class PresentInteractor: UIPercentDrivenInteractiveTransition { init(gestureView: UIView) { super.init()
panGestureRecognizer = UIPanGestureRecognizer(target: self, action: #selector(handlePan(_:))) gestureView.addGestureRecognizer(panGestureRecognizer) } @objc private func handlePan(_ panGestureRecognizer: UIPanGestureRecognizer) { switch panGestureRecognizer.state { case .possible: break case .began: presentViewController() case .changed: shouldCompleteTransition = percentage > 0.1 update(percentage) case .ended: if shouldCompleteTransition { finish() } else { cancel() } case .cancelled, .failed: cancel() @unknown default: break } } } -
Implementation One Last Step Don’t let user notice there’re two
view controllers › Sync status when start expanding and end collapsing struct ViewState { let contentOffset: CGPoint } protocol ViewStateSyncDelegate { var initialViewStateForExpandedViewController: ViewState { get } func expandedViewDidFinishCollapsing(_ viewState: ViewState) } -
Recap › ViewController presenting works on Keyboard › Use transition
coordinator to define appear and disappear animation › Use UIViewControllerTransitioningDelegate to create custom interactive transition