$30 off During Our Annual Pro Sale. View Details »

詳解 Android Auto - 使い方からそれを支える技術まで -

詳解 Android Auto - 使い方からそれを支える技術まで -

Keishin Yokomaku

February 09, 2018
Tweet

More Decks by Keishin Yokomaku

Other Decks in Technology

Transcript

  1. ৄղ Android Auto
    - ࢖͍ํ͔ΒͦΕΛࢧ͑Δٕज़·Ͱ -
    Keishin Yokomaku / DroidKaigi 2018

    View Slide

  2. Keishin Yokomaku
    2
    ৄղ Android Auto
    Drivemode, Inc / Engineer
    @KeithYokoma: GitHub / Qiita / Twitter
    DroidKaigi 2018

    View Slide

  3. Android Auto
    3
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  4. 4
    ৄղ Android Auto
    Photo by Maurizio Pesce: http://bit.ly/2higwKJ

    View Slide

  5. 5
    ৄղ Android Auto
    Photo by Maurizio Pesce: http://bit.ly/2zsY6BS

    View Slide

  6. Features
    6
    ৄղ Android Auto
    Audio Messages Calls Navigation
    DroidKaigi 2018

    View Slide

  7. Features
    7
    ৄղ Android Auto
    Audio Messages Calls Navigation
    Voice actions
    DroidKaigi 2018

    View Slide

  8. Features
    8
    ৄղ Android Auto
    Audio Messages Calls Navigation
    Voice actions
    DroidKaigi 2018

    View Slide

  9. How do they work?
    DroidKaigi 2018

    View Slide

  10. Chapters in this talk
    ▸ Audio Freature Framework
    ▸ Messaging Feature Framework
    ▸ Wrap-up
    10
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  11. Audio Feature
    Framework
    DroidKaigi 2018

    View Slide

  12. Audio: Characters in this framework
    12
    ৄղ Android Auto
    DroidKaigi 2018
    Apps providing media contents
    Apps connecting to media app

    (incl. Android Auto and others)
    Android system service

    View Slide

  13. How AndroidAuto works
    13
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  14. How AndroidAuto works
    14
    ৄղ Android Auto
    IPC
    DroidKaigi 2018

    View Slide

  15. MediaSession in media app
    15
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession

    View Slide

  16. MediaSession in media app
    16
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaController
    MediaSession.Token

    View Slide

  17. Playback control flow
    17
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaController
    MC.TransportControls
    MS.Callback

    View Slide

  18. Playback control flow
    18
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaController
    MediaPlayer
    MS.Callback
    MC.TransportControls

    View Slide

  19. MediaController package
    19
    ৄղ Android Auto
    ✔android.media.session.MediaController
    ✘ android.widget.MediaController
    DroidKaigi 2018

    View Slide

  20. IPC between media app and AndroidAuto
    20
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaController
    MediaSession.Token

    View Slide

  21. IPC between media app and AndroidAuto
    21
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaController
    MS.Callback
    MC.TransportControls

    View Slide

  22. IPC between media app and AndroidAuto
    22
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaController
    MS.Callback
    MC.TransportControls

    View Slide

  23. ৄղ Android Auto
    Creating MediaSession
    class MusicService : Service() {
    lateinit var session: MediaSessionCompat
    override fun onCreate() {
    session = MediaSessionCompat(this, “music”)
    session.setCallback(object : Callback() {
    override fun onPlay() {
    // start playing music!
    }
    // ……
    })
    session.isActive = true
    }
    }
    23
    DroidKaigi 2018

    View Slide

  24. ৄղ Android Auto
    class MusicService : Service() {
    lateinit var session: MediaSessionCompat
    override fun onCreate() {
    session = MediaSessionCompat(this, “music”)
    session.setCallback(object : Callback() {
    override fun onPlay() {
    // start playing music!
    }
    // ……
    })
    session.isActive = true
    }
    }
    24
    DroidKaigi 2018
    Creating MediaSession

    View Slide

  25. ৄղ Android Auto
    Implement Callback methods
    class MusicService : Service() {
    lateinit var session: MediaSessionCompat
    override fun onCreate() {
    session = MediaSessionCompat(this, “music”)
    session.setCallback(object : Callback() {
    override fun onPlay() {
    // start playing music!
    }
    // ……
    })
    session.isActive = true
    }
    }
    25
    DroidKaigi 2018

    View Slide

  26. ৄղ Android Auto
    Activate MediaSession
    class MusicService : Service() {
    lateinit var session: MediaSessionCompat
    override fun onCreate() {
    session = MediaSessionCompat(this, “music”)
    session.setCallback(object : Callback() {
    override fun onPlay() {
    // start playing music!
    }
    // ……
    })
    session.isActive = true
    }
    }
    26
    DroidKaigi 2018

    View Slide

  27. IPC between media app and AndroidAuto
    27
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaController
    MediaSession.Token

    View Slide

  28. ৄղ Android Auto
    Emit MediaSession.Token via Binder
    class MusicService : Service() {
    lateinit var session: MediaSessionCompat
    private val binder: MusicBinder = MusicService()
    override fun onBind(intent: Intent) = binder
    fun getToken() = session.sessionToken
    inner class MusicBinder : Binder() {
    MusicService getService() = this@MusicService
    }
    }
    28
    DroidKaigi 2018

    View Slide

  29. ৄղ Android Auto
    Emit MediaSession.Token via Binder
    class MusicService : Service() {
    lateinit var session: MediaSessionCompat
    private val binder: MusicBinder = MusicService()
    override fun onBind(intent: Intent) = binder
    fun getToken() = session.sessionToken
    inner class MusicBinder : Binder() {
    MusicService getService() = this@MusicService
    }
    }
    29
    DroidKaigi 2018

    View Slide

  30. ৄղ Android Auto
    Emit MediaSession.Token via Binder
    class MusicService : Service() {
    lateinit var session: MediaSessionCompat
    private val binder: MusicBinder = MusicService()
    override fun onBind(intent: Intent) = binder
    fun getToken() = session.sessionToken
    inner class MusicBinder : Binder() {
    MusicService getService() = this@MusicService
    }
    }
    30
    DroidKaigi 2018

    View Slide

  31. ৄղ Android Auto
    Emit MediaSession.Token via Binder
    class MusicService : Service() {
    lateinit var session: MediaSessionCompat
    private val binder: MusicBinder = MusicService()
    override fun onBind(intent: Intent) = binder
    fun getToken() = session.sessionToken
    inner class MusicBinder : Binder() {
    MusicService getService() = this@MusicService
    }
    }
    31
    DroidKaigi 2018
    MediaSession.Token : Parcelable

    View Slide

  32. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    val conn: ServiceConnection = // …
    override fun onCreate(savedState: Bundle) {
    super.onCreate(savedState)
    val intent = Intent(this, MusicService::class)
    bindService(intent, conn, BIND_AUTO_CREATE)
    }
    }
    32
    DroidKaigi 2018

    View Slide

  33. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    val conn: ServiceConnection = // …
    override fun onCreate(savedState: Bundle) {
    super.onCreate(savedState)
    val intent = Intent(this, MusicService::class)
    bindService(intent, conn, BIND_AUTO_CREATE)
    }
    }
    33
    DroidKaigi 2018

    View Slide

  34. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    val conn: ServiceConnection = // …
    override fun onDestroy() {
    if (bound) {
    unbindService(conn)
    bound = false
    }
    super.onDestroy()
    }
    }
    34
    DroidKaigi 2018

    View Slide

  35. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    val conn: ServiceConnection = // …
    override fun onDestroy() {
    if (bound) {
    unbindService(conn)
    bound = false
    }
    super.onDestroy()
    }
    }
    35
    DroidKaigi 2018

    View Slide

  36. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    val conn = object : ServiceConnection() {
    override fun onServiceConnected(
    cn: ComponentName, s: IBinder) {
    val binder = s as MusicBinder
    service = binder.getService()
    bound = true
    }
    }
    }
    36
    DroidKaigi 2018

    View Slide

  37. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    val conn = object : ServiceConnection() {
    override fun onServiceConnected(
    cn: ComponentName, s: IBinder) {
    val binder = s as MusicBinder
    service = binder.getService()
    bound = true
    }
    }
    }
    37
    DroidKaigi 2018

    View Slide

  38. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    val conn = object : ServiceConnection() {
    override fun onServiceConnected(
    cn: ComponentName, s: IBinder) {
    val binder = s as MusicBinder
    service = binder.getService()
    bound = true
    }
    }
    }
    38
    DroidKaigi 2018

    View Slide

  39. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    val conn = object : ServiceConnection() {
    override fun onServiceConnected(
    cn: ComponentName, s: IBinder) {
    val binder = s as MusicBinder
    service = binder.getService()
    bound = true
    }
    }
    }
    39
    DroidKaigi 2018

    View Slide

  40. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    val conn = object : ServiceConnection() {
    override fun onServiceConnected(
    cn: ComponentName, s: IBinder) {
    val binder = s as MusicBinder
    service = binder.getService()
    bound = true
    }
    }
    }
    40
    DroidKaigi 2018

    View Slide

  41. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    var mediaController: MediaController? = null
    fun buildMediaController() {
    ɹ service?.let {
    mediaController = MediaControllerCompat(
    this@MusicActivity, it.getToken())
    mediaController.prepare()
    }
    }
    }
    41
    DroidKaigi 2018

    View Slide

  42. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    var mediaController: MediaController? = null
    fun buildMediaController() {
    ɹ service?.let {
    mediaController = MediaControllerCompat(
    this@MusicActivity, it.getToken())
    mediaController.prepare()
    }
    }
    }
    42
    DroidKaigi 2018

    View Slide

  43. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    var mediaController: MediaController? = null
    fun buildMediaController() {
    ɹ service?.let {
    mediaController = MediaControllerCompat(
    this@MusicActivity, it.getToken())
    mediaController.prepare()
    }
    }
    }
    43
    DroidKaigi 2018

    View Slide

  44. ৄղ Android Auto
    Create MediaController from Token
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var bound: Boolean = false
    var mediaController: MediaController? = null
    fun buildMediaController() {
    ɹ service?.let {
    mediaController = MediaControllerCompat(
    this@MusicActivity, it.getToken())
    mediaController.prepare()
    }
    }
    }
    44
    DroidKaigi 2018

    View Slide

  45. ৄղ Android Auto
    Playback control via MC.TransportControls
    class MusicActivity : Activity() {
    var mediaController: MediaController? // …
    fun onPlayClick() {
    mediaController.transportControls.play()
    }
    fun onPauseClick() {
    mediaController.transportControls.pause()
    }
    }
    45
    DroidKaigi 2018

    View Slide

  46. ৄղ Android Auto
    Playback control via MC.TransportControls
    class MusicActivity : Activity() {
    var mediaController: MediaController? // …
    fun onPlayClick() {
    mediaController.transportControls.play()
    }
    fun onPauseClick() {
    mediaController.transportControls.pause()
    }
    }
    46
    DroidKaigi 2018

    View Slide

  47. Activate MediaSession
    47
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession

    View Slide

  48. Activate MediaSession
    48
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSessionManager

    View Slide

  49. Activate MediaSession
    49
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSessionManager
    MediaSessionService
    MediaSessionRecord
    ……

    View Slide

  50. Activate MediaSession
    50
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSessionManager
    MediaSessionService
    MediaSessionRecord
    MediaSessionRecord
    ……

    View Slide

  51. Activate MediaSession
    51
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSessionManager
    MediaSessionService
    MediaSessionRecord
    MediaSessionRecord
    ……
    MediaSessionRecord

    View Slide

  52. Prepare IPC via MediaController
    52
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    MSR.getController()

    View Slide

  53. Prepare IPC via MediaController
    53
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    ISessionController

    View Slide

  54. Prepare IPC via MediaController
    54
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    ISessionController
    MediaController

    View Slide

  55. Prepare IPC via MediaController
    55
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    ISessionController
    MediaController
    ISessionController

    View Slide

  56. Playback control via IPC
    56
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    ISessionController
    MediaController
    ISessionController
    TransportControls
    play(), pause(), …

    View Slide

  57. Playback control via IPC
    57
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    ISessionController
    MediaController
    ISessionController
    TransportControls
    play(), pause(), …
    play(), pause(),

    View Slide

  58. Playback control via IPC
    58
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    ISessionController
    MediaController
    ISessionController
    TransportControls
    play(), pause(), …
    play(), pause(),
    MS.Callback
    onPlay(), …

    View Slide

  59. What if the media app is dead…?
    59
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    ISessionController
    MediaController
    ISessionController
    TransportControls
    play(), pause(), …
    play(), pause(),
    Process is dead!

    View Slide

  60. What if the media app is dead…?
    60
    ৄղ Android Auto
    Fatal exception:
    DeadObjectException
    DroidKaigi 2018

    View Slide

  61. When remote process is dead
    61
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    ISessionController
    MediaController
    ISessionController
    Process is dead!

    View Slide

  62. When remote process is dead
    62
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    ISessionController
    MediaController
    ISessionController
    Process is dead!
    binderDied()
    ⇣

    onDestroy()

    View Slide

  63. When remote process is dead
    63
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    MediaSession.Token
    ISessionController
    MediaController
    ISessionController
    Process is dead!
    Callback
    onSessionDestroy()

    View Slide

  64. ৄղ Android Auto
    Detect remote process death
    class MusicActivity : Activity() {
    var binder: IBinder? = null
    val recipient: IBinder.DeathRecipient = // …
    val conn = object : ServiceConnection() {
    override fun onServiceConnected(
    cn: ComponentName, b: IBinder) {
    binder = b
    b.linkToDeath(recipient, 0)
    }
    }
    }
    64
    DroidKaigi 2018

    View Slide

  65. ৄղ Android Auto
    Detect remote process death
    class MusicActivity : Activity() {
    var binder: IBinder? = null
    val recipient: IBinder.DeathRecipient = // …
    val conn = object : ServiceConnection() {
    override fun onServiceConnected(
    cn: ComponentName, b: IBinder) {
    binder = b
    b.linkToDeath(recipient, 0)
    }
    }
    }
    65
    DroidKaigi 2018

    View Slide

  66. ৄղ Android Auto
    Detect remote process death
    class MusicActivity : Activity() {
    var binder: IBinder? = null
    val recipient = object : IBinder.DeathRecipient {
    override fun binderDied() {
    binder?.unlinkToDeath(this@DeathRecipient)
    }
    }
    val conn: ServiceConnection = // …
    }
    66
    DroidKaigi 2018

    View Slide

  67. IBinder.DeathRecipient vs ServiceConnection
    ▸ IBinder.DeathRecipient#binderDied
    ▸ Monitor remote process death
    ▸ ServiceConnection#onServiceDisconnected
    ▸ Called from IBinder.DeathRecipient#binderDied
    ▸ Only called when service is bound to remote
    67
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  68. ৄղ Android Auto
    Get notified from media app
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var mediaController: MediaController? = null
    val callback: MediaController.Callback = //…
    fun buildMediaController() {
    ɹ service?.let {
    mediaController = // …
    mediaController.registerCallback(callback)
    }
    }
    }
    68
    DroidKaigi 2018

    View Slide

  69. ৄղ Android Auto
    Get notified from media app
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var mediaController: MediaController? = null
    val callback: MediaController.Callback =
    object : MediaController.Callback() {
    fun onSessionDestroy() {
    // no more interaction from here!
    }
    }
    }
    69
    DroidKaigi 2018

    View Slide

  70. ৄղ Android Auto
    Get notified from media app
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var mediaController: MediaController? = null
    val callback: MediaController.Callback =
    object : MediaController.Callback() {
    fun onPlaybackStateChanged(
    state: PlaybackStateCompat) {
    // get new playback state!
    }
    }
    }
    70
    DroidKaigi 2018

    View Slide

  71. ৄղ Android Auto
    Get notified from media app
    class MusicActivity : Activity() {
    var service: MusicService? = null
    var mediaController: MediaController? = null
    val callback: MediaController.Callback =
    object : MediaController.Callback() {
    fun onMetadataChanged(
    data: MediaMetadataCompat) {
    // get new media information!
    }
    }
    }
    71
    DroidKaigi 2018

    View Slide

  72. Playback control IPC on Lock Screen in 4.x
    72
    ৄղ Android Auto
    DroidKaigi 2018
    RemoteControlClient
    AudioManager
    AudioService
    LockScreen

    View Slide

  73. Playback control IPC on Lock Screen in 4.x
    73
    ৄղ Android Auto
    DroidKaigi 2018
    RemoteControlClient
    AudioManager
    AudioService
    RemoteControlDisplay
    LockScreen

    View Slide

  74. Playback control IPC on Lock Screen in 4.x
    74
    ৄղ Android Auto
    DroidKaigi 2018
    RemoteControlClient
    AudioManager
    AudioService
    RemoteControlDisplay
    LockScreen

    View Slide

  75. Playback control IPC on Lock Screen in 4.4
    75
    ৄղ Android Auto
    DroidKaigi 2018
    RemoteControlClient
    AudioManager
    AudioService
    RemoteController
    LockScreen
    NotificationListener
    Service

    View Slide

  76. Playback control IPC on Lock Screen in 5+
    76
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    NotificationManager
    Service
    MediaController
    LockScreen
    MediaStyle Notification
    NotificationListener
    Service

    View Slide

  77. Playback control IPC on Lock Screen in 5+
    77
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSession
    NotificationManager
    Service
    MediaController
    LockScreen
    MediaStyle Notification
    NotificationListener
    Service

    View Slide

  78. Get active sessions for the package
    78
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSessionManager
    MediaSessionService
    MediaSessionRecord
    MediaSessionRecord
    ……
    getActiveSessions()

    View Slide

  79. Get active sessions for the package
    79
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSessionManager
    MediaSessionService
    MediaSessionRecord
    MediaSessionRecord
    ……
    getSessions()

    View Slide

  80. Get active sessions for the package
    80
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSessionManager
    MediaSessionService
    MediaSessionRecord
    MediaSessionRecord
    ……
    ISessionController

    View Slide

  81. Get active sessions for the package
    81
    ৄղ Android Auto
    DroidKaigi 2018
    MediaSessionManager
    MediaSessionService
    MediaSessionRecord
    MediaSessionRecord
    ……
    MediaController

    View Slide

  82. ৄղ Android Auto
    Get notified from media app
    class MusicActivity : Activity() {
    override fun onCreate(savedState: Bundle) {
    super.onCreate(savedState)
    val manager: MediaSessionManager =
    context.getSystemService(
    MEDIA_SESSION_SERVICE)
    val controllers: List =
    manager.getActiveSessions(null)
    // do whatever you want with media controller…
    }
    }
    82
    DroidKaigi 2018

    View Slide

  83. ৄղ Android Auto
    Get notified from media app
    class MusicActivity : Activity() {
    override fun onCreate(savedState: Bundle) {
    super.onCreate(savedState)
    val manager: MediaSessionManager =
    context.getSystemService(
    MEDIA_SESSION_SERVICE)
    val controllers: List =
    manager.getActiveSessions(null)
    // do whatever you want with media controller…
    }
    }
    83
    DroidKaigi 2018

    View Slide

  84. ৄղ Android Auto
    Get notified from media app
    class MusicActivity : Activity() {
    override fun onCreate(savedState: Bundle) {
    super.onCreate(savedState)
    val manager: MediaSessionManager =
    context.getSystemService(
    MEDIA_SESSION_SERVICE)
    val controllers: List =
    manager.getActiveSessions(null)
    // do whatever you want with media controller…
    }
    }
    84
    DroidKaigi 2018

    View Slide

  85. ৄղ Android Auto
    Get notified from media app
    class MusicActivity : Activity() {
    override fun onCreate(savedState: Bundle) {
    super.onCreate(savedState)
    val manager: MediaSessionManager =
    context.getSystemService(
    MEDIA_SESSION_SERVICE)
    val controllers: List =
    manager.getActiveSessions(null)
    // do whatever you want with media controller…
    }
    }
    85
    DroidKaigi 2018

    View Slide

  86. How AndroidAuto works: MediaBrowser
    86
    ৄղ Android Auto
    IPC
    DroidKaigi 2018

    View Slide

  87. MediaBrowser and MediaBrowserService
    87
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService
    MediaBrowser

    View Slide

  88. Connect to MediaBrowserService
    88
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService
    MediaBrowser
    connect()

    View Slide

  89. Connect to MediaBrowserService
    89
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService
    MediaBrowser
    MB.ConnectionCallback#
    onConnected()

    View Slide

  90. ৄղ Android Auto
    Connect to MediaBrowserService
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val service = ComponentName(
    this, BrowserService::class)
    val cb: MediaBrowserCompat.ConnectionCallback = …
    override fun onCreate(savedState: Bundle) {
    super.onCreate(savedState)
    browser = MediaBrowserCompat(
    this, service, cb, Bundle())
    browser.connect()
    }
    }
    90
    DroidKaigi 2018

    View Slide

  91. ৄղ Android Auto
    Connect to MediaBrowserService
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val service = ComponentName(
    this, BrowserService::class)
    val cb: MediaBrowserCompat.ConnectionCallback = …
    override fun onCreate(savedState: Bundle) {
    super.onCreate(savedState)
    browser = MediaBrowserCompat(
    this, service, cb, Bundle())
    browser.connect()
    }
    }
    91
    DroidKaigi 2018

    View Slide

  92. ৄղ Android Auto
    Connect to MediaBrowserService
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val service = ComponentName(
    this, BrowserService::class)
    val cb: MediaBrowserCompat.ConnectionCallback = …
    override fun onCreate(savedState: Bundle) {
    super.onCreate(savedState)
    browser = MediaBrowserCompat(
    this, service, cb, Bundle())
    browser.connect()
    }
    }
    92
    DroidKaigi 2018

    View Slide

  93. ৄղ Android Auto
    Connect to MediaBrowserService
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val cb = object : MBC.ConnectionCallback {
    override fun onConnected() {
    // you can start subscribing/searching
    // media contents after this callback!
    }
    override fun onConnectionFailed() {
    // cannot connect to remote service!
    }
    }
    }
    93
    DroidKaigi 2018

    View Slide

  94. ৄղ Android Auto
    Connect to MediaBrowserService
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val cb = object : MBC.ConnectionCallback {
    override fun onConnected() {
    // you can start subscribing/searching
    // media contents after this callback!
    }
    override fun onConnectionFailed() {
    // cannot connect to remote service!
    }
    }
    }
    94
    DroidKaigi 2018

    View Slide

  95. ৄղ Android Auto
    Connect to MediaBrowserService
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val cb = object : MBC.ConnectionCallback {
    override fun onConnected() {
    // you can start subscribing/searching
    // media contents after this callback!
    }
    override fun onConnectionFailed() {
    // cannot connect to remote service!
    }
    }
    }
    95
    DroidKaigi 2018

    View Slide

  96. ৄղ Android Auto
    Verify connection from MediaBrowser
    class BrowserService : MediaBrowserServiceCompat() {
    override fun onGetRoot(
    clientPkg: String,
    clientUid: Int,
    hints: Bundle?): BrowserRoot? {
    // called on connect
    // return null if refusing connection
    }
    }
    96
    DroidKaigi 2018

    View Slide

  97. Connection verification by
    97
    ৄղ Android Auto
    ✔Verify signature of the client
    ✔Verify the client package name
    DroidKaigi 2018
    ✔Verify the client process user id

    View Slide

  98. Connection verification flow
    98
    ৄղ Android Auto
    DroidKaigi 2018
    Allow access?
    Client is
    self?
    Next Step
    Allow
    YES
    No

    View Slide

  99. No
    Connection verification flow
    99
    ৄղ Android Auto
    DroidKaigi 2018
    Allow access?
    Client is
    self?
    Next Step
    Allow
    clientUid == Process.myUid()
    YES

    View Slide

  100. No
    Connection verification flow
    100
    ৄղ Android Auto
    DroidKaigi 2018
    Allow access?
    Client is
    system?
    Next Step
    Allow
    YES

    View Slide

  101. No
    Connection verification flow
    101
    ৄղ Android Auto
    DroidKaigi 2018
    Allow access?
    Client is
    system?
    Next Step
    Allow
    clientUid == Process.SYSTEM_UID
    YES

    View Slide

  102. Connection verification flow
    102
    ৄղ Android Auto
    DroidKaigi 2018
    Allow access?
    Client has
    valid signature?
    Refuse
    Allow
    YES
    No

    View Slide

  103. ৄղ Android Auto
    List of allowed client packages and signatures


    name=“Android Auto”
    release=“false”
    package=“com.google.android.projection.gearhead”
    >
    Base64 encoded signature here


    103
    DroidKaigi 2018

    View Slide

  104. ৄղ Android Auto
    List of allowed client packages and signatures


    name=“Android Auto”
    release=“false”
    package=“com.google.android.projection.gearhead”
    >
    Base64 encoded signature here


    104
    DroidKaigi 2018

    View Slide

  105. ৄղ Android Auto
    List of allowed client packages and signatures


    name=“Android Auto”
    release=“false”
    package=“com.google.android.projection.gearhead”
    >
    Base64 encoded signature here


    105
    DroidKaigi 2018

    View Slide

  106. ৄղ Android Auto
    Generate your app signature
    $ keytool -export -alias androiddebugkey -keystore
    ~/.android/debug.keystore | base64
    Password: xxxx
    Base64 encoded signature
    106
    DroidKaigi 2018

    View Slide

  107. ৄղ Android Auto
    Generate your app signature
    $ keytool -export -alias androiddebugkey -keystore
    ~/.android/debug.keystore | base64
    Password: xxxx
    Base64 encoded signature
    107
    DroidKaigi 2018

    View Slide

  108. ৄղ Android Auto
    Verify connection from MediaBrowser
    class BrowserService : MediaBrowserServiceCompat() {
    override fun onGetRoot(
    clientPkg: String,
    clientUid: Int,
    hints: Bundle?): BrowserRoot? {
    val validPackage: Boolean = // validate caller!
    if (validPackage) {
    return MBSC.BrowserRoot(“__ROOT__”, null)
    }
    return null
    }
    }
    108
    DroidKaigi 2018

    View Slide

  109. ৄղ Android Auto
    Media contents structure
    109
    DroidKaigi 2018
    __ROOT__
    __FAVORITE__
    __RECOMMEND__
    __SUBSCRIPTION__
    __FAVORITE__/1
    __FAVORITE__/2

    View Slide

  110. ৄղ Android Auto
    Media contents structure
    110
    DroidKaigi 2018
    __ROOT__
    __FAVORITE__
    __RECOMMEND__
    __SUBSCRIPTION__
    __FAVORITE__/1
    __FAVORITE__/2
    BrowserRoot

    View Slide

  111. ৄղ Android Auto
    Media contents structure
    111
    DroidKaigi 2018
    __ROOT__
    __FAVORITE__
    __RECOMMEND__
    __SUBSCRIPTION__
    __FAVORITE__/1
    __FAVORITE__/2
    MediaItem
    BROWSABLE
    MediaItem
    PLAYABLE

    View Slide

  112. Subscribe media content
    112
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService
    MediaBrowser
    subscribe(id, callback)

    View Slide

  113. Subscribe media content
    113
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService
    MediaBrowser
    MB.SubscriptionCallback#
    onChildrenLoaded()

    View Slide

  114. ৄղ Android Auto
    Load top level items
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val scb: MBC.SubscriptionCallback = // …
    fun loadRootItems() {
    val rootId: String = browser.getRoot()
    browser.subscribe(rootId, scb)
    }
    }
    114
    DroidKaigi 2018

    View Slide

  115. ৄղ Android Auto
    Load top level items
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val scb: MBC.SubscriptionCallback = // …
    fun loadRootItems() {
    val rootId: String = browser.getRoot()
    browser.subscribe(rootId, scb)
    }
    }
    115
    DroidKaigi 2018

    View Slide

  116. ৄղ Android Auto
    Load top level items
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val scb: MBC.SubscriptionCallback = // …
    fun loadRootItems() {
    val rootId: String = browser.getRoot()
    browser.subscribe(rootId, scb)
    }
    }
    116
    DroidKaigi 2018

    View Slide

  117. ৄղ Android Auto
    Load top level items
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val scb = object : MBC.SubscriptionCallback() {
    override fun onChildrenLoaded(
    parentId: String, list: List) {
    // show media list under the parentId item
    }
    }
    }
    117
    DroidKaigi 2018

    View Slide

  118. ৄղ Android Auto
    Load top level items
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val scb = object : MBC.SubscriptionCallback() {
    override fun onChildrenLoaded(
    parentId: String, list: List) {
    // show media list under the parentId item
    }
    }
    }
    118
    DroidKaigi 2018

    View Slide

  119. ৄղ Android Auto
    On select item: Load the next list
    119
    DroidKaigi 2018
    __ROOT__
    __FAVORITE__
    __RECOMMEND__
    __SUBSCRIPTION__
    __FAVORITE__/1
    __FAVORITE__/2
    MediaItem
    BROWSABLE

    View Slide

  120. ৄղ Android Auto
    On select item: Start playing media content
    120
    DroidKaigi 2018
    __ROOT__
    __FAVORITE__
    __RECOMMEND__
    __SUBSCRIPTION__
    __FAVORITE__/1
    __FAVORITE__/2
    MediaItem
    PLAYABLE

    View Slide

  121. ৄղ Android Auto
    Load media items synchronously
    class BrowserService : MediaBrowserServiceCompat() {
    override fun onLoadChildren(
    parentId: String,
    result: Result>) {
    val list: List = // load media items
    result.sendResult(list)
    }
    }
    121
    DroidKaigi 2018

    View Slide

  122. ৄղ Android Auto
    Load media items synchronously
    class BrowserService : MediaBrowserServiceCompat() {
    override fun onLoadChildren(
    parentId: String,
    result: Result>) {
    val list: List = // load media items
    result.sendResult(list)
    }
    }
    122
    DroidKaigi 2018

    View Slide

  123. ৄղ Android Auto
    Load media items asynchronously
    class BrowserService : MediaBrowserServiceCompat() {
    override fun onLoadChildren(
    parentId: String,
    result: Result>) {
    result.detach()
    Single.create {
    val list: List = // …
    it.onSuccess(list)
    }.subscribeOn(schedulers.io).subscribe { list ->
    result.sendResult(list)
    }
    }
    }
    123
    DroidKaigi 2018

    View Slide

  124. ৄղ Android Auto
    Load media items asynchronously
    class BrowserService : MediaBrowserServiceCompat() {
    override fun onLoadChildren(
    parentId: String,
    result: Result>) {
    result.detach()
    Single.create {
    val list: List = // …
    it.onSuccess(list)
    }.subscribeOn(schedulers.io).subscribe { list ->
    result.sendResult(list)
    }
    }
    }
    124
    DroidKaigi 2018

    View Slide

  125. ৄղ Android Auto
    Load media items asynchronously
    class BrowserService : MediaBrowserServiceCompat() {
    override fun onLoadChildren(
    parentId: String,
    result: Result>) {
    result.detach()
    Single.create {
    val list: List = // …
    it.onSuccess(list)
    }.subscribeOn(schedulers.io).subscribe { list ->
    result.sendResult(list)
    }
    }
    }
    125
    DroidKaigi 2018

    View Slide

  126. ৄղ Android Auto
    Handle item selection
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val scb: MBC.SubscriptionCallback = // …
    fun onItemSelected(item: MediaItem) {
    if (item.isBrowsable()) {
    // start loading more item under the item
    } else if (item.isPlayable() {
    // start playing item
    }
    }
    }
    126
    DroidKaigi 2018

    View Slide

  127. ৄղ Android Auto
    Handle item selection
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val scb: MBC.SubscriptionCallback = // …
    fun onItemSelected(item: MediaItem) {
    if (item.isBrowsable()) {
    // start loading more item under the item
    } else if (item.isPlayable() {
    // start playing item
    }
    }
    }
    127
    DroidKaigi 2018

    View Slide

  128. ৄղ Android Auto
    Handle item selection: Load more!
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val scb: MBC.SubscriptionCallback = // …
    fun onItemSelected(item: MediaItem) {
    if (item.isBrowsable()) {
    // start loading more item under the item
    } else if (item.isPlayable() {
    // start playing item
    }
    }
    }
    128
    DroidKaigi 2018

    View Slide

  129. ৄղ Android Auto
    Handle item selection: Start playing!
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val scb: MBC.SubscriptionCallback = // …
    fun onItemSelected(item: MediaItem) {
    if (item.isBrowsable()) {
    // start loading more item under the item
    } else if (item.isPlayable() {
    // start playing item
    }
    }
    }
    129
    DroidKaigi 2018

    View Slide

  130. ৄղ Android Auto
    Handle item selection: Load more!
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val scb: MBC.SubscriptionCallback = // …
    fun onItemSelected(item: MediaItem) {
    if (item.isBrowsable()) {
    browser.subscribe(item.mediaId, scb)
    }
    }
    }
    130
    DroidKaigi 2018

    View Slide

  131. ৄղ Android Auto
    Handle item selection: Start playing!
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    var controller: MediaController? = null
    fun onItemSelected(item: MediaItem) {
    if (item.isPlayable() {
    val token = browser.sessionToken
    controller = MediaController(this, token)
    controller.transportControls.playFromMediaId(
    item.mediaId, Bundle())
    }
    }
    }
    131
    DroidKaigi 2018

    View Slide

  132. ৄղ Android Auto
    Search items
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val qcb = object : MBC.SearchCallback() {
    override fun onSearchResult(query: String,
    extra: Bundle, res: List) {
    // show query result to the list!
    }
    }
    fun onSearchItem(query: String) {
    browser.search(query, Bundle(), qcb)
    }
    }
    132
    DroidKaigi 2018

    View Slide

  133. ৄղ Android Auto
    Search items
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val qcb = object : MBC.SearchCallback() {
    override fun onSearchResult(query: String,
    extra: Bundle, res: List) {
    // show query result to the list!
    }
    }
    fun onSearchItem(query: String) {
    browser.search(query, Bundle(), qcb)
    }
    }
    133
    DroidKaigi 2018

    View Slide

  134. ৄղ Android Auto
    Search items
    class MusicActivity : Activity() {
    lateinit var browser: MediaBrowserCompat
    val qcb = object : MBC.SearchCallback() {
    override fun onSearchResult(query: String,
    extra: Bundle, res: List) {
    // show query result to the list!
    }
    }
    fun onSearchItem(query: String) {
    browser.search(query, Bundle(), qcb)
    }
    }
    134
    DroidKaigi 2018

    View Slide

  135. ৄղ Android Auto
    Handle search query synchronously
    class BrowserService : MediaBrowserServiceCompat() {
    override fun onSearch(
    query: String, extras: Bundle,
    result: Result>) {
    val list: List = // …
    result.sendResult(list)
    }
    }
    135
    DroidKaigi 2018

    View Slide

  136. ৄղ Android Auto
    Handle search query asynchronously
    class BrowserService : MediaBrowserServiceCompat() {
    override fun onSearch(
    query: String, extras: Bundle,
    result: Result>) {
    result.detach()
    Single.create {
    val list: List = // …
    it.onSuccess(list)
    }.subscribeOn(schedulers.io).subscribe { list ->
    result.sendResult(list)
    }
    }
    }
    136
    DroidKaigi 2018

    View Slide

  137. Wrap up: Audio Feature Framework
    137
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService MediaBrowser
    MediaController
    MediaSession
    MediaPlayer

    View Slide

  138. Wrap up: Audio Feature Framework
    138
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService MediaBrowser
    MediaController
    MediaSession
    MediaPlayer
    Create new session

    View Slide

  139. Wrap up: Audio Feature Framework
    139
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService MediaBrowser
    MediaController
    MediaSession
    MediaPlayer
    Provide session token

    View Slide

  140. Wrap up: Audio Feature Framework
    140
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService MediaBrowser
    MediaController
    MediaSession
    MediaPlayer
    Connect to service

    View Slide

  141. Wrap up: Audio Feature Framework
    141
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService MediaBrowser
    MediaController
    MediaSession
    MediaPlayer
    Provide media info and token

    View Slide

  142. Wrap up: Audio Feature Framework
    142
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService MediaBrowser
    MediaController
    MediaSession
    MediaPlayer
    Refuse connection if not OK

    View Slide

  143. Wrap up: Audio Feature Framework
    143
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService MediaBrowser
    MediaController
    MediaSession
    MediaPlayer
    Create controller from token

    View Slide

  144. Wrap up: Audio Feature Framework
    144
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService MediaBrowser
    MediaController
    MediaSession
    MediaPlayer
    Playback control via
    TransportControls

    View Slide

  145. Wrap up: Audio Feature Framework
    145
    ৄղ Android Auto
    DroidKaigi 2018
    MediaBrowserService MediaBrowser
    MediaController
    MediaSession
    MediaPlayer
    Actual playback control

    View Slide

  146. Messaging Feature
    Framework
    DroidKaigi 2018

    View Slide

  147. Messaging: Characters in this framework
    147
    ৄղ Android Auto
    DroidKaigi 2018
    Messenger apps
    Apps listening to incoming messages

    (incl. Android Auto and others)
    Android system service

    View Slide

  148. Overview - Messaging Feature Framework
    148
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  149. NOTIFICATION
    Overview - Messaging Feature Framework
    149
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  150. NOTIFICATION
    Overview - Messaging Feature Framework
    150
    ৄղ Android Auto
    DroidKaigi 2018
    NotificationManager#notify(id, n)

    View Slide

  151. Message Notification for Android Auto
    151
    ৄղ Android Auto
    DroidKaigi 2018
    NotificationCompat.CarExtender
    Visual settings(color and icon)
    Unread/Read status
    Voice input for reply

    View Slide

  152. ৄղ Android Auto
    Building Notification for Android Auto
    val n = NotificationCompat.Builder(context)

    .extend(CarExtender()
    .setColor(R.color.car_notification)
    .setLargeIcon(iconBitmap)
    .setUnreadConversation(
    CarExtender.UnreadConversation.Builder(“test”)
    .addMessage(“This is a message.”)
    .setReadPendingIntent(readPi)
    .setReplyAction(replyPi, remoteInput)
    .setLatestTimestamp(timestamp)
    .build())
    ).build()
    152
    DroidKaigi 2018

    View Slide

  153. ৄղ Android Auto
    Building Notification for Android Auto
    val n = NotificationCompat.Builder(context)

    .extend(CarExtender()
    .setColor(R.color.car_notification)
    .setLargeIcon(iconBitmap)
    .setUnreadConversation(
    CarExtender.UnreadConversation.Builder(“test”)
    .addMessage(“This is a message.”)
    .setReadPendingIntent(readPi)
    .setReplyAction(replyPi, remoteInput)
    .setLatestTimestamp(timestamp)
    .build())
    ).build()
    153
    DroidKaigi 2018

    View Slide

  154. ৄղ Android Auto
    Building Notification for Android Auto
    val n = NotificationCompat.Builder(context)

    .extend(CarExtender()
    .setColor(R.color.car_notification)
    .setLargeIcon(iconBitmap)
    .setUnreadConversation(
    CarExtender.UnreadConversation.Builder(“test”)
    .addMessage(“This is a message.”)
    .setReadPendingIntent(readPi)
    .setReplyAction(replyPi, remoteInput)
    .setLatestTimestamp(timestamp)
    .build())
    ).build()
    154
    DroidKaigi 2018

    View Slide

  155. ৄղ Android Auto
    Building Notification for Android Auto
    val n = NotificationCompat.Builder(context)

    .extend(CarExtender()
    .setColor(R.color.car_notification)
    .setLargeIcon(iconBitmap)
    .setUnreadConversation(
    CarExtender.UnreadConversation.Builder(“test”)
    .addMessage(“This is a message.”)
    .setReadPendingIntent(readPi)
    .setReplyAction(replyPi, remoteInput)
    .setLatestTimestamp(timestamp)
    .build())
    ).build()
    155
    DroidKaigi 2018

    View Slide

  156. ৄղ Android Auto
    Building Notification for Android Auto
    val n = NotificationCompat.Builder(context)

    .extend(CarExtender()
    .setColor(R.color.car_notification)
    .setLargeIcon(iconBitmap)
    .setUnreadConversation(
    CarExtender.UnreadConversation.Builder(“test”)
    .addMessage(“This is a message.”)
    .setReadPendingIntent(readPi)
    .setReplyAction(replyPi, remoteInput)
    .setLatestTimestamp(timestamp)
    .build())
    ).build()
    156
    DroidKaigi 2018

    View Slide

  157. ৄղ Android Auto
    Building Notification for Android Auto
    val n = NotificationCompat.Builder(context)

    .extend(CarExtender()
    .setColor(R.color.car_notification)
    .setLargeIcon(iconBitmap)
    .setUnreadConversation(
    CarExtender.UnreadConversation.Builder(“test”)
    .addMessage(“This is a message.”)
    .setReadPendingIntent(readPi)
    .setReplyAction(replyPi, remoteInput)
    .setLatestTimestamp(timestamp)
    .build())
    ).build()
    157
    DroidKaigi 2018

    View Slide

  158. ৄղ Android Auto
    Building Notification for Android Auto
    val n = NotificationCompat.Builder(context)

    .extend(CarExtender()
    .setColor(R.color.car_notification)
    .setLargeIcon(iconBitmap)
    .setUnreadConversation(
    CarExtender.UnreadConversation.Builder(“test”)
    .addMessage(“This is a message.”)
    .setReadPendingIntent(readPi)
    .setReplyAction(replyPi, remoteInput)
    .setLatestTimestamp(timestamp)
    .build())
    ).build()
    158
    DroidKaigi 2018

    View Slide

  159. ৄղ Android Auto
    Building Notification for Android Auto
    val n = NotificationCompat.Builder(context)

    .extend(CarExtender()
    .setColor(R.color.car_notification)
    .setLargeIcon(iconBitmap)
    .setUnreadConversation(
    CarExtender.UnreadConversation.Builder(“test”)
    .addMessage(“This is a message.”)
    .setReadPendingIntent(readPi)
    .setReplyAction(replyPi, remoteInput)
    .setLatestTimestamp(timestamp)
    .build())
    ).build()
    159
    DroidKaigi 2018

    View Slide

  160. ৄղ Android Auto
    Building Notification for Android Auto
    val n = NotificationCompat.Builder(context)

    .extend(CarExtender()
    .setColor(R.color.car_notification)
    .setLargeIcon(iconBitmap)
    .setUnreadConversation(
    CarExtender.UnreadConversation.Builder(“test”)
    .addMessage(“This is a message.”)
    .setReadPendingIntent(readPi)
    .setReplyAction(replyPi, remoteInput)
    .setLatestTimestamp(timestamp)
    .build())
    ).build()
    160
    DroidKaigi 2018

    View Slide

  161. ৄղ Android Auto
    Read/Unread status update
    val conversationId: Int = //ϝοηʔδͷ΍ΓऔΓΛද͢ID
    val intent = Intent(“your.app.action.READ_MESSAGE”)
    .addFlags(Intent.FLAG_INCLUDE_STOPPED_PACKAGE)
    .putExtra(“conversation_id”, conversationId);
    val readPi = PendingIntent.getBroadcast(
    applicationContext,
    conversationId,
    intent,
    PendingIntent.FLAG_UPDATE_CURRENT)
    161
    DroidKaigi 2018

    View Slide

  162. ৄղ Android Auto
    Read/Unread status update
    val conversationId: Int = //ϝοηʔδͷ΍ΓऔΓΛද͢ID
    val intent = Intent(“your.app.action.READ_MESSAGE”)
    .addFlags(Intent.FLAG_INCLUDE_STOPPED_PACKAGE)
    .putExtra(“conversation_id”, conversationId);
    val readPi = PendingIntent.getBroadcast(
    applicationContext,
    conversationId,
    intent,
    PendingIntent.FLAG_UPDATE_CURRENT)
    162
    DroidKaigi 2018

    View Slide

  163. ৄղ Android Auto
    Read/Unread status update
    val conversationId: Int = //ϝοηʔδͷ΍ΓऔΓΛද͢ID
    val intent = Intent(“your.app.action.READ_MESSAGE”)
    .addFlags(Intent.FLAG_INCLUDE_STOPPED_PACKAGE)
    .putExtra(“conversation_id”, conversationId);
    val readPi = PendingIntent.getBroadcast(
    applicationContext,
    conversationId,
    intent,
    PendingIntent.FLAG_UPDATE_CURRENT)
    163
    DroidKaigi 2018

    View Slide

  164. ৄղ Android Auto
    Read/Unread status update
    val conversationId: Int = //ϝοηʔδͷ΍ΓऔΓΛද͢ID
    val intent = Intent(“your.app.action.READ_MESSAGE”)
    .addFlags(Intent.FLAG_INCLUDE_STOPPED_PACKAGE)
    .putExtra(“conversation_id”, conversationId);
    val readPi = PendingIntent.getBroadcast(
    applicationContext,
    conversationId,
    intent,
    PendingIntent.FLAG_UPDATE_CURRENT)
    164
    DroidKaigi 2018

    View Slide

  165. ৄղ Android Auto
    Reply by voice input in Android Auto
    val conversationId: Int = //ϝοηʔδͷ΍ΓऔΓΛද͢ID
    val input = RemoteInput.Builder(“key_voice_reply”)

    val intent = Intent(“your.app.action.REPLY_MESSAGE”)

    val replyPi = PendingIntent.getBroadcast(
    applicationContext,
    conversationId,
    intent,
    PendingIntent.FLAG_UPDATE_CURRENT)
    165
    DroidKaigi 2018

    View Slide

  166. ৄղ Android Auto
    Reply by voice input in Android Auto
    val conversationId: Int = //ϝοηʔδͷ΍ΓऔΓΛද͢ID
    val input = RemoteInput.Builder(“key_voice_reply”)

    val intent = Intent(“your.app.action.REPLY_MESSAGE”)

    val replyPi = PendingIntent.getBroadcast(
    applicationContext,
    conversationId,
    intent,
    PendingIntent.FLAG_UPDATE_CURRENT)
    166
    DroidKaigi 2018

    View Slide

  167. ৄղ Android Auto
    Reply by voice input in Android Auto
    val conversationId: Int = //ϝοηʔδͷ΍ΓऔΓΛද͢ID
    val input = RemoteInput.Builder(“key_voice_reply”)
    .setLabel(“Reply by voice”)
    .build()
    val intent = Intent(“your.app.action.REPLY_MESSAGE”)

    val replyPi = PendingIntent.getBroadcast(
    applicationContext,
    conversationId,
    intent,
    PendingIntent.FLAG_UPDATE_CURRENT)
    167
    DroidKaigi 2018

    View Slide

  168. ৄղ Android Auto
    Reply by voice input in Android Auto
    val conversationId: Int = //ϝοηʔδͷ΍ΓऔΓΛද͢ID
    val input = RemoteInput.Builder(“key_voice_reply”)

    val intent = Intent(“your.app.action.REPLY_MESSAGE”)
    .addFlags(Intent.FLAG_INCLUDE_STOPPED_PACKAGES)
    .putExtra(“conversation_id”, conversationId)
    val replyPi = PendingIntent.getBroadcast(
    applicationContext,
    conversationId,
    intent,
    PendingIntent.FLAG_UPDATE_CURRENT)
    168
    DroidKaigi 2018

    View Slide

  169. ৄղ Android Auto
    Reply by voice input in Android Auto
    val conversationId: Int = //ϝοηʔδͷ΍ΓऔΓΛද͢ID
    val input = RemoteInput.Builder(“key_voice_reply”)

    val intent = Intent(“your.app.action.REPLY_MESSAGE”)

    val replyPi = PendingIntent.getBroadcast(
    applicationContext,
    conversationId,
    intent,
    PendingIntent.FLAG_UPDATE_CURRENT)
    169
    DroidKaigi 2018

    View Slide

  170. ৄղ Android Auto
    Receive voice input result from Android Auto
    class ReplyReceiver : BroadcastReceiver() {
    override fun onReceive(c: Context, i: Intent) {
    val conversationId =
    i.getIntExtra(“conversation_id”)
    val result = RemoteInput.getResultsFromIntent(i)
    result?.let {
    val text =
    it.getCharSequence(“key_voice_reply”)
    … // sending reply
    }
    }
    }
    170
    DroidKaigi 2018

    View Slide

  171. ৄղ Android Auto
    Receive voice input result from Android Auto
    class ReplyReceiver : BroadcastReceiver() {
    override fun onReceive(c: Context, i: Intent) {
    val conversationId =
    i.getIntExtra(“conversation_id”)
    val result = RemoteInput.getResultsFromIntent(i)
    result?.let {
    val text =
    it.getCharSequence(“key_voice_reply”)
    … // sending reply
    }
    }
    }
    171
    DroidKaigi 2018

    View Slide

  172. ৄղ Android Auto
    Receive voice input result from Android Auto
    class ReplyReceiver : BroadcastReceiver() {
    override fun onReceive(c: Context, i: Intent) {
    val conversationId =
    i.getIntExtra(“conversation_id”)
    val result = RemoteInput.getResultsFromIntent(i)
    result?.let {
    val text =
    it.getCharSequence(“key_voice_reply”)
    … // sending reply
    }
    }
    }
    172
    DroidKaigi 2018

    View Slide

  173. NOTIFICATION
    Overview - Messaging Feature Framework
    173
    ৄղ Android Auto
    DroidKaigi 2018
    ?

    View Slide

  174. NOTIFICATION
    Overview - Messaging Feature Framework
    174
    ৄղ Android Auto
    DroidKaigi 2018
    NotificationListenerService

    View Slide

  175. Special permission for accessing notifications
    175
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  176. Special permission for accessing notifications
    176
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  177. Special permission for accessing notifications
    177
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  178. NotificationListenerService
    178
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  179. NotificationListenerService
    179
    ৄղ Android Auto
    DroidKaigi 2018
    NotificationManager
    NotificationManagerService
    NotificationListenerService
    IPC
    IPC

    View Slide

  180. NotificationListenerService
    180
    ৄղ Android Auto
    DroidKaigi 2018
    NotificationManager
    NotificationManagerService
    NotificationListenerService
    notify
    IPC
    IPC

    View Slide

  181. NotificationListenerService
    181
    ৄղ Android Auto
    DroidKaigi 2018
    NotificationManager
    NotificationManagerService
    NotificationListenerService
    enqueueNotificationWithTag
    IPC
    IPC

    View Slide

  182. NotificationListenerService
    182
    ৄղ Android Auto
    DroidKaigi 2018
    NotificationManager
    NotificationManagerService
    NotificationListenerService
    onNotificationPosted
    IPC
    IPC

    View Slide

  183. ৄղ Android Auto
    Reading car-extended notification
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val n: Notification = sbn.getNotification()
    val extras: Bundle = n.extras
    val extensions: Bundle = extras.getBundle(
    “android.car.EXTENSIONS”)
    }
    }
    183
    DroidKaigi 2018

    View Slide

  184. ৄղ Android Auto
    Reading car-extended notification
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val n: Notification = sbn.getNotification()
    val extras: Bundle = n.extras
    val extensions: Bundle = extras.getBundle(
    “android.car.EXTENSIONS”)
    }
    }
    184
    DroidKaigi 2018

    View Slide

  185. ৄղ Android Auto
    Reading car-extended notification
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val n: Notification = sbn.getNotification()
    val extras: Bundle = n.extras
    val extensions: Bundle = extras.getBundle(
    “android.car.EXTENSIONS”)
    }
    }
    185
    DroidKaigi 2018

    View Slide

  186. ৄղ Android Auto
    Reading car-extended notification
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val n: Notification = sbn.getNotification()
    val extras: Bundle = n.extras
    val extensions: Bundle = extras.getBundle(
    “android.car.EXTENSIONS”)
    }
    }
    186
    DroidKaigi 2018

    View Slide

  187. ৄղ Android Auto
    Reading car-extended notification
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val n: Notification = sbn.getNotification()
    val extras: Bundle = n.extras
    val extensions: Bundle = extras.getBundle(
    “android.car.EXTENSIONS”)
    }
    }
    187
    DroidKaigi 2018

    View Slide

  188. ৄղ Android Auto
    Reading car-extended notification
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val n: Notification = sbn.getNotification()
    val extras: Bundle = n.extras
    val extensions: Bundle = extras.getBundle(
    “android.car.EXTENSIONS”)
    }
    }
    188
    DroidKaigi 2018

    View Slide

  189. ৄղ Android Auto
    Get UnreadConversation data
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val extensions: Bundle = // ……
    // UnreadConversation
    val conv: Bundle = extensions.getBundle(
    “car_conversation”)
    }
    }
    189
    DroidKaigi 2018

    View Slide

  190. ৄղ Android Auto
    Get UnreadConversation data
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val conv: Bundle = // ……
    // Array of Bundles containing messages
    val messages: Array =
    conv.getParcelableArray(“messages”)
    }
    }
    190
    DroidKaigi 2018

    View Slide

  191. ৄղ Android Auto
    Prepare for sending reply
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val conv: Bundle = // ……
    val messages: Array = // ……
    // Reply message container
    val input: RemoteInput =
    conv.getParcelable(“remote_input”)
    }
    }
    191
    DroidKaigi 2018

    View Slide

  192. ৄղ Android Auto
    Prepare for sending reply
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val conv: Bundle = // ……
    val messages: Array = // ……
    val input: RemoteInput = // ……
    val replyPi: PendingIntent =
    conv.getParcelable(“on_reply”)
    }
    }
    192
    DroidKaigi 2018

    View Slide

  193. ৄղ Android Auto
    Prepare for sending reply
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val conv: Bundle = // ……
    val messages: Array = // ……
    val input: RemoteInput = // ……
    val replyPi: PendingIntent = // ……
    val readPi: PendingIntent =
    conv.getParcelable(“on_read”)
    }
    }
    193
    DroidKaigi 2018

    View Slide

  194. ৄղ Android Auto
    Reading incoming message texts
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val messages: Array = // ……
    val list: List = messages.filter {
    it is Bundle
    }.map {
    (it as Bundle).getString(“text”)
    }
    }
    }
    194
    DroidKaigi 2018

    View Slide

  195. ৄղ Android Auto
    Reading incoming message texts
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val messages: Array = // ……
    val list: List = messages.filter {
    it is Bundle
    }.map {
    (it as Bundle).getString(“text”)
    }
    }
    }
    195
    DroidKaigi 2018

    View Slide

  196. ৄղ Android Auto
    Reading incoming message texts
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val messages: Array = // ……
    val list: List = messages.filter {
    it is Bundle
    }.map {
    (it as Bundle).getString(“text”)
    }
    }
    }
    196
    DroidKaigi 2018

    View Slide

  197. ৄղ Android Auto
    Finish reading texts
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val readPi: PendingIntent = // ……
    readPi.send()
    }
    }
    197
    DroidKaigi 2018

    View Slide

  198. ৄղ Android Auto
    Sending reply message
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val replyMessage: String = “reply body”
    val input: RemoteInput =
    conv.getParcelable(“remote_input”)
    val key = input.getResultKey()
    val intent: Intent = Intent()
    val bundle: Bundle = Bundle()
    bundle.putString(key, replyMessage)
    }
    }
    198
    DroidKaigi 2018

    View Slide

  199. ৄղ Android Auto
    Sending reply message
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val replyMessage: String = “reply body”
    val input: RemoteInput =
    conv.getParcelable(“remote_input”)
    val key = input.getResultKey()
    val intent: Intent = Intent()
    val bundle: Bundle = Bundle()
    bundle.putString(key, replyMessage)
    }
    }
    199
    DroidKaigi 2018

    View Slide

  200. ৄղ Android Auto
    Sending reply message
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val replyMessage: String = “reply body”
    val input: RemoteInput =
    conv.getParcelable(“remote_input”)
    val key = input.getResultKey()
    val intent: Intent = Intent()
    val bundle: Bundle = Bundle()
    bundle.putString(key, replyMessage)
    }
    }
    200
    DroidKaigi 2018

    View Slide

  201. ৄղ Android Auto
    Sending reply message
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val replyMessage: String = “reply body”
    val input: RemoteInput =
    conv.getParcelable(“remote_input”)
    val key = input.getResultKey()
    val intent: Intent = Intent()
    val bundle: Bundle = Bundle()
    bundle.putString(key, replyMessage)
    }
    }
    201
    DroidKaigi 2018

    View Slide

  202. ৄղ Android Auto
    Sending reply message
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val key = input.getResultKey()
    val intent: Intent = Intent()
    val bundle: Bundle = Bundle()
    val replyInput: RemoteInput =
    RemoteInput.Builder(key).build()
    RemoteInput.addResultsToIntent(
    arrayOf(replyInput), intent, bundle)
    }
    }
    202
    DroidKaigi 2018

    View Slide

  203. ৄղ Android Auto
    Sending reply message
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val key = input.getResultKey()
    val intent: Intent = Intent()
    val bundle: Bundle = Bundle()
    val replyInput: RemoteInput =
    RemoteInput.Builder(key).build()
    RemoteInput.addResultsToIntent(
    arrayOf(replyInput), intent, bundle)
    }
    }
    203
    DroidKaigi 2018

    View Slide

  204. ৄղ Android Auto
    Sending reply message
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val key = input.getResultKey()
    val intent: Intent = Intent()
    val bundle: Bundle = Bundle()
    val replyInput: RemoteInput =
    RemoteInput.Builder(key).build()
    RemoteInput.addResultsToIntent(
    arrayOf(replyInput), intent, bundle)
    }
    }
    204
    DroidKaigi 2018

    View Slide

  205. ৄղ Android Auto
    Sending reply message
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val key = input.getResultKey()
    val intent: Intent = Intent()
    val replyPi: PendingIntent = // ……
    replyPi.send(context, 0, intent)
    }
    }
    205
    DroidKaigi 2018

    View Slide

  206. ৄղ Android Auto
    Sending reply message
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val key = input.getResultKey()
    val intent: Intent = Intent()
    val replyPi: PendingIntent = // ……
    replyPi.send(context, 0, intent)
    }
    }
    206
    DroidKaigi 2018

    View Slide

  207. ৄղ Android Auto
    Sending reply message
    class MyListener : NotificationListenerService {
    override fun onNotificationPosted(
    sbn: StatusBarNotification) {
    val key = input.getResultKey()
    val intent: Intent = Intent()
    val replyPi: PendingIntent = // ……
    replyPi.send(context, 0, intent)
    }
    }
    207
    DroidKaigi 2018

    View Slide

  208. ৄղ Android Auto
    Wrap up: Messaging Feature Framework
    208
    SomeMessagingModel Notification
    ListenerService
    ReplyReceiver
    DroidKaigi 2018

    View Slide

  209. ৄղ Android Auto
    Wrap up: Messaging Feature Framework
    209
    SomeMessagingModel Notification
    ListenerService
    ReplyReceiver
    Notification
    with PendingIntent
    DroidKaigi 2018

    View Slide

  210. ৄղ Android Auto
    Wrap up: Messaging Feature Framework
    210
    SomeMessagingModel Notification
    ListenerService
    ReplyReceiver
    PendingIntent#send
    DroidKaigi 2018

    View Slide

  211. ৄղ Android Auto
    Wrap up: Messaging Feature Framework
    211
    SomeMessagingModel Notification
    ListenerService
    ReplyReceiver
    Send reply message
    DroidKaigi 2018

    View Slide

  212. ৄղ Android Auto
    Wrap up: Messaging Feature Framework
    212
    SomeMessagingModel Notification
    ListenerService
    ReadReceiver
    DroidKaigi 2018

    View Slide

  213. ৄղ Android Auto
    Wrap up: Messaging Feature Framework
    213
    SomeMessagingModel Notification
    ListenerService
    ReadReceiver
    Notification
    with PendingIntent
    DroidKaigi 2018

    View Slide

  214. ৄղ Android Auto
    Wrap up: Messaging Feature Framework
    214
    SomeMessagingModel Notification
    ListenerService
    ReadReceiver
    PendingIntent#send
    DroidKaigi 2018

    View Slide

  215. ৄղ Android Auto
    Wrap up: Messaging Feature Framework
    215
    SomeMessagingModel Notification
    ListenerService
    ReadReceiver
    Set read flag, etc…
    DroidKaigi 2018

    View Slide

  216. Wrap up
    DroidKaigi 2018

    View Slide

  217. Wrap up: Overview
    217
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  218. Wrap up: Media playback control
    218
    ৄղ Android Auto
    IPC with
    MediaSession / MediaController
    DroidKaigi 2018

    View Slide

  219. Wrap up: Media browsing
    219
    ৄղ Android Auto
    Callback to MediaBrowser
    Bind to MediaBrowserService
    DroidKaigi 2018

    View Slide

  220. Wrap up: Media browsing
    220
    ৄղ Android Auto
    Callback to MediaBrowser
    Bind to MediaBrowserService
    DroidKaigi 2018

    View Slide

  221. Wrap up: Media browsing
    221
    ৄղ Android Auto
    Callback to MediaBrowser
    Bind to MediaBrowserService
    DroidKaigi 2018

    View Slide

  222. Wrap up: Incoming message
    222
    ৄղ Android Auto
    Notification
    DroidKaigi 2018

    View Slide

  223. Wrap up: Incoming message
    223
    ৄղ Android Auto
    Notification
    Notification
    DroidKaigi 2018

    View Slide

  224. Wrap up: Replying to a message
    224
    ৄղ Android Auto
    Broadcast by PendingIntent
    DroidKaigi 2018

    View Slide

  225. Wrap up: Behind the scenes of Android Auto
    ▸ Interprocess Communication(IPC)
    ▸ The Binder
    ▸ Android Auto is based on the wrapped classes
    ▸ MediaSession and MediaController
    ▸ MediaBrowser and MediaBrowserService
    ▸ Google Assistant is also based on these classes!
    225
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  226. Wrap up: References
    ▸ More about IPC in Android
    ▸ Inside Binder
    ▸ https://www.slideshare.net/l_b__/binderandroid
    ▸ Deep Dive into Android IPC/Binder Framework
    ▸ Android Binder
    226
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  227. Wrap up: References
    ▸ API Docs: Messaging features
    ▸ Notification.CarExtender
    ▸ Notification.CarExtender.UnreadConversation
    ▸ NotificationListenerService
    227
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  228. Wrap up: References
    ▸ API Docs: Media features
    ▸ Notification.MediaStyle
    ▸ MediaBrowser and MediaBrowserService
    ▸ MediaSession and MediaController
    ▸ MediaSessionManager
    228
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  229. Wrap up: Sample Project
    ▸ Media Browser sample
    ▸ github/googlesamples/MediaBrowserService
    ▸ github/googlesamples/UniversalMusicPlayer
    ▸ TechBooster
    ▸ C90ΞϯυϩΠυΞΧσϛΞ for Audio Framework
    ▸ C91ͳͳ͍ΖAndroid for Messaging Framework
    229
    ৄղ Android Auto
    DroidKaigi 2018

    View Slide

  230. ৄղ Android Auto
    - ࢖͍ํ͔ΒͦΕΛࢧ͑Δٕज़·Ͱ -
    Keishin Yokomaku / DroidKaigi 2018

    View Slide

  231. We are hiring!
    231
    Drivemode
    DroidKaigi 2018

    View Slide