Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ANDROID AT LARGE: HOW TO BRING OPTIMISED EXPERIENCES TO THE BIG SCREEN

ANDROID AT LARGE: HOW TO BRING OPTIMISED EXPERIENCES TO THE BIG SCREEN

With Android making its way to new form factors, and OEMs pushing the boundaries with new technology, it’s time to take a look at what these environments bring to you and your users. It’s crucial that your developer workflow accounts for resizability, new display sizes and aspect ratios, multi display, and app continuity to provide your users the best experience no matter where they are. The days of taking the easy road with screenOrientation="portrait" are quickly coming to an end.

In this talk we discuss the challenges faced when looking to support Android on various platforms, and how to make sure that you’re providing a great experience on all of these form-factors. You will leave this talk with an actionable checklist that helps stabilise your app and user experience on any device.

3dfa913cbcfe896af79e0484084c316b?s=128

Pietro F. Maggi

November 27, 2020
Tweet

Transcript

  1. Android at Large Optimized Experiences for the Big Screen Pietro

    F. Maggi @pfmaggi Kenneth Ford @KennethFSWE
  2. Android comes in all sizes

  3. Phones Tablets Foldable Phones Desktop Environments

  4. One size does not fit all! Handle different screen sizes

    & aspect ratios
  5. Different contexts of use Quick actions vs longer sessions Gaming,

    Media, Social, Productivity Immersive Quick actions Multi-tasking Communication, Music Productivity, Communication
  6. App Continuity Configuration changes are unavoidable

  7. One APK for all of these devices? Android App Bundles

    to the rescue
  8. UX & UI considerations

  9. New Aspect Ratios Test & support non-standard aspect ratios If

    necessary, use new minAspectRatio or existing maxAspectRatio attributes to restrict the ratios your app supports.
  10. Layouts You have more space, use it! Think about how

    you can surface more information or make your users more efficient Larger screens provide the opportunity for more immersive experiences No one size fits all solution Check out our Material Studies for inspiration! material.io/design/material-studies/
  11. material.io/design/material-studies/reply.html

  12. material.io/design/material-studies/rally.html

  13. Input Don’t assume touch input. Also true on phones (accessibility

    needs, desktop modes). Mouse + Keyboard =
  14. KeyboardFocus.kt // Standard view myView.setFocusable(true) // Override default arrow mapping

    myView.setNextFocusLeftId(R.id.view_to_left) myView.setNextFocusRightId(R.id.view_to_right) myView.setNextFocusTopId(R.id.view_to_top) myView.setNextFocusBottomId(R.id.view_to_bottom) // Override default tab mapping myView.setNextFocusForwardId(R.id.next_view)
  15. EnterKey.kt // Listen for release of enter key myView.setOnKeyListener {

    v, keyCode, event -> if (event.action == KeyEvent.ACTION_UP && keyCode == KeyEvent.KEYCODE_ENTER) { // Insert behavior here true } else { false } }
  16. EnterKey.kt // Listen for release of enter key myView.setOnKeyListener {

    v, keyCode, event -> if (event.action == KeyEvent.ACTION_UP && keyCode == KeyEvent.KEYCODE_ENTER) { // Insert behavior here true } else { false } }
  17. EnterKey.kt // Listen for release of enter key myView.setOnKeyListener {

    v, keyCode, event -> if (event.action == KeyEvent.ACTION_UP && keyCode == KeyEvent.KEYCODE_ENTER) { // Insert behavior here true } else { false } }
  18. Ctrl_Z.kt override fun dispatchKeyShortcutEvent(event: KeyEvent?): Boolean { if (event?.keyCode ==

    KeyEvent.KEYCODE_Z && event.hasModifiers(KeyEvent.META_CTRL_ON)) { // Undo action return true } return super.dispatchKeyShortcutEvent(event) }
  19. Ctrl_Shift_Z.kt override fun dispatchKeyShortcutEvent(event: KeyEvent?): Boolean { if (event?.keyCode ==

    KeyEvent.KEYCODE_Z && event.hasModifiers(KeyEvent.META_CTRL_ON | KeyEvent.META_SHIFT_ON)) { // Redo action return true } return super.dispatchKeyShortcutEvent(event) }
  20. Pointers Long press action == right click actions onContextClickListener Change

    colors/elevation when hovering onHoverListener Update mouse pointers on hover View.pointerIcon
  21. yourView.setOnHoverListener{ view, motionEvent -> when (motionEvent.actionMasked) { MotionEvent.ACTION_HOVER_ENTER -> {

    // UI Change } MotionEvent.ACTION_HOVER_EXIT -> { // Revert } } } Highlight Elements on Hover
  22. yourView.setOnHoverListener { view, motionEvent -> view.pointerIcon = PointerIcon.getSystemIcon(context, PointerIcon.TYPE_HAND) false

    } Update Mouse Pointer Icons github.com/chromeos/pointer-icon-sample
  23. App continuity

  24. App Continuity Seamlessly continue what you’re doing! The ability to

    fold and unfold devices brings more spotlight to configuration changes and how your app handles them. Your app should restore the same state and location the user was in before folding or unfolding.
  25. App Continuity Beyond foldables, the ability to save and restore

    state through configuration changes is a priority on all platforms. Correctly handling rotation or free-form resizing of windows not only requires the ability to save and restore state, but to be able to re-draw and re-layout with minimal jank. Beyond foldables, the ability to save and restore state through configuration changes is a priority on all platforms.
  26. android:resizeableActivity Indicates whether an activity supports multi-window and multi-display environment.

    However, setting this to false does not mean the activity never needs to resize because of orientation changes, free-form resizing or display folding.
  27. Activity on external screen when folded Compat mode Activity running

    in compat mode when unfolded Android 10 has a new compatibility mode for activities that support neither multi-window nor orientation changes.
  28. Handling configuration changes Unfolding triggers a configuration change for smallestScreenSize,

    screenLayout and screenSize. • Use onSaveInstanceState and ViewModel, or • Handle the configuration change without restarting via handleConfigChange=”..” For the best experience, declare resizeableActivity=”true”.
  29. Multi-Window

  30. Android 10 Resumed Resumed Resumed Multi-resume In multi-window, all top

    focusable activities in visible stacks are now in the RESUMED state. Activity can still end up in the PAUSED state if: • There is a transparent activity on top • It’s not currently focusable (PiP) In multi-window, all top focusable activities in visible stacks are now in the RESUMED state.
  31. Resources While multiple apps are resumed, some will disconnect from

    available resources. Watch for camera availability callbacks to continue use. resizeableActivity=false does not guarantee camera access. While multiple apps are resumed, some will disconnect from available resources.
  32. protected void onTopResumedActivityChanged(boolean topResumed) { if (topResumed) { // Top

    resumed activity // Can be a signal to re-acquire exclusive resources } else { // No longer the top resumed activity } } New lifecycle callbacks for top-resumed
  33. Drag-n-drop With multi-window becoming more common, consider adding drag-n-drop support

    for text and images.
  34. Multi-Display

  35. Activities on secondary screens Keep in mind that the following

    things may occur: • Context update • Window resize • Resource Changes
  36. // Get current display val activityDisplay = activity.windowManager.defaultDisplay // Get

    current window metrics val windowMetrics = DisplayMetrics() activityDisplay.getMetrics(windowMetrics) // … or val windowMetrics = activity.resources.displayMetrics Activity vs Application context Context contains information about display // Get current display val activityDisplay = activity.windowManager.defaultDisplay // Get current window metrics val windowMetrics = DisplayMetrics() activityDisplay.getMetrics(windowMetrics) // … or val windowMetrics = activity.resources.displayMetrics // Show toast on the current display Toast.makeText(activity, text, duration).show()
  37. Configuration changes If activity handles configuration change, it will be

    notified onConfigurationChanged(newConfig: Configuration?) If it doesn’t, it will be relaunched. onCreate and onConfigurationChanged are good points to check what is the current display for an activity. android:configChanges="density|orientation|screenLayout |screenSize|smallestScreenSize |touchscreen"
  38. Multiple instances FLAG_ACTIVITY_NEW_TASK|FLAG_ACTIVITY_MULTIPLE_TASK

  39. Multiple instances FLAG_ACTIVITY_NEW_TASK|FLAG_ACTIVITY_MULTIPLE_TASK Resumed Resumed

  40. // Get available displays val dm = getSystemService(Context.DISPLAY_SERVICE) as DisplayManager

    val displays = dm.displays // Check their characteristics Display.flags // To check if it’s a secure display, etc. Display.metrics // To get the size, resolution and density Display.state // To check if it’s ON/OFF Using secondary screens
  41. // Check if launch is allowed activityManager.isActivityStartAllowedOnDisplay(context, displayId, intent) //

    Launch on specific display val options = ActivityOptions.makeBasic() options.launchDisplayId = targetDisplay.displayId startActivity(intent, options.toBundle()) Using secondary screens
  42. Multiple display densities • •

  43. android { bundle { language { enableSplit = true //

    true by default. } density { // The app bundle should not support configuration APKs for // display densities. These resources are instead packaged // with each base and dynamic feature APK. enableSplit = false } abi { enableSplit = true // true by default. } } } Android App Bundles and multiple-display
  44. Launchers android.intent.category.SECONDARY_HOME

  45. Jetpack Window Manager

  46. Jetpack Window Manager Provide a single API surface for different

    types of foldable devices coming to the market. Target the entire category, not a single model.
  47. Features @IntDef({ // A fold on the screen without a

    physical gap. TYPE_FOLD, // A physical separation with a hinge that // allows two display panels to fold. TYPE_HINGE, }) public @interface Type{} github.com/android/user-interface-samples/tree/master/WindowManager
  48. Postures @IntDef({ POSTURE_UNKNOWN, POSTURE_CLOSED, POSTURE_HALF_OPENED, POSTURE_OPENED, POSTURE_FLIPPED }) public @interface

    Posture{} github.com/android/user-interface-samples/tree/master/WindowManager
  49. Usage dependencies { implementation "androidx.window:window:1.0.0-alpha01" } github.com/android/user-interface-samples/tree/master/WindowManager var windowManager =

    WindowManager( this /* context */, null /* windowBackend */) val displayFeatures = windowManager.windowLayoutInfo.displayFeatures windowManager.registerDeviceStateChangeCallback( mainThreadExecutor /* Executor */, callback /* Consumer<DeviceState> */)
  50. Mocking - WindowBackend class MidScreenFoldBackend : WindowBackend { override fun

    getDeviceState() = { /* */ } override fun getWindowLayoutInfo(context: Context): WindowLayoutInfo { /* */ } override fun registerDeviceStateChangeCallback( executor: Executor, callback: Consumer<DeviceState> ) { /* */ } override fun unregisterDeviceStateChangeCallback(callback: Consumer<DeviceState>) { /* */ } override fun registerLayoutChangeCallback( context: Context, executor: Executor, callback: Consumer<WindowLayoutInfo> ) { /* */ } override fun unregisterLayoutChangeCallback(callback: Consumer<WindowLayoutInfo>) { /* */ } } github.com/android/user-interface-samples/tree/master/WindowManager var windowManager = WindowManager( this /* context */, null /* windowBackend */)
  51. Testing

  52. Android Emulator Multi-Touch Multi-core support Quick Boot OpenGL ES 3.0

    Multiple instances Display cutout Battery mode Headless build
  53. Size Resolution Unfolded 8” 2200x2480 Folded 6.6” 1148x2480 Size Resolution

    Unfolded 7.3” 1536x2152 Folded 4.6” 840x1960 7.3” 7.3” and 8” Emulators 8” Android Emulator
  54. App continuity Compat mode Multi-resume Emulator and Android 10

  55. 6.7” Horizontal Fold-in Android Emulator

  56. 7.4” Rollable Android Emulator androidstudio.googleblog.com/2020/09/emulator-3013-canary.html

  57. 13.5” Freeform Android Emulator

  58. Multiple Displays Android Emulator

  59. ...and More Developing for Android 11 with the Android Emulator

    medium.com/androiddevelopers/developing-for-android-11-with-the-android-emulator-a9486af2d7ef
  60. Thank You! Pietro F. Maggi @pfmaggi