Pro Yearly is on sale from $80 to $50! »

Media streaming on Android @droidkaigi 2018

A374f41eab3f73c50d8bab0652cb207a?s=47 TakuSemba
February 08, 2018

Media streaming on Android @droidkaigi 2018

A374f41eab3f73c50d8bab0652cb207a?s=128

TakuSemba

February 08, 2018
Tweet

Transcript

  1. Media streaming on Android TakuSemba CyberAgent.Inc

  2. @takusemba https://github.com/TakuSemba

  3. Player

  4. API level 1 API level 14 Simple use cases Advanced

    use cases Black Box Customizable MediaPlayer ExoPlayer
  5. ExoPlayer dependencies { implementation ‘com.google.android.exoplayer:exoplayer:x.x.x’ }

  6. ExoPlayer dependencies { implementation ‘com.google.android.exoplayer:exoplayer-core:x.x.x’ implementation ‘com.google.android.exoplayer:exoplayer-ui:x.x.x’ implementation ‘com.google.android.exoplayer:exoplayer-hls:x.x.x’ }

  7. ExoPlayer dependencies { implementation ‘com.google.android.exoplayer:exoplayer-core:x.x.x’ implementation ‘com.google.android.exoplayer:exoplayer-ui:x.x.x’ implementation ‘com.google.android.exoplayer:exoplayer-hls:x.x.x’ implementation

    ‘com.google.android.exoplayer:extension-okhttp:x.x.x’ }
  8. Streaming

  9. Streaming HTTP video

  10. Streaming video

  11. Streaming 1min 1sec × 60files video .mp4 .mp4 video

  12. Streaming time video1 video2 video3 video4

  13. Streaming time video1 video2 video3 video4

  14. time Streaming video1 video2 video3 video4 video1

  15. Streaming time video1 video2 video3 video4 video1 video2

  16. Streaming time video1 video2 video3 video4 video1 video2 video3

  17. time video1 video2 video3 video4 Streaming video1 video2 video3 video4

  18. Streaming

  19. 1 pixel = 24 bits (1byte for red, blue, green)

    Streaming 1 pixel
  20. 1 pixel = 24 bits (1byte for red, blue, green)

    1 frame = 49m bits (1080px * 1920px * 24bits) Streaming 1 frame
  21. 1 pixel = 24 bits (1byte for red, blue, green)

    1 frame = 49m bits (1080px * 1920px * 24bits) 1 second = 1492m bits (30fps * 49m bits) Streaming 1 second
  22. 1 pixel = 24 bits (1byte for red, blue, green)

    1 frame = 49m bits (1080px * 1920px * 24bits) 1 second = 1492m bits (30fps * 49m bits) 1492m bits Streaming 1 second
  23. 1 pixel = 24 bits (1byte for red, blue, green)

    1 frame = 49m bits (1080px * 1920px * 24bits) 1 second = 1492m bits (30fps * 49m bits) 1492m bits > 60m bps (4G LTE) Streaming 1 second
  24. 1 pixel = 24 bits (1byte for red, blue, green)

    1 frame = 49m bits (1080px * 1920px * 24bits) 1 second = 1492m bits (30fps * 49m bits) 1492m bits > 60m bps (4G LTE) codec (h264) Streaming 1 second
  25. 1 pixel = 24 bits (1byte for red, blue, green)

    1 frame = 49m bits (1080px * 1920px * 24bits) 1 second = 1492m bits (30fps * 49m bits) 1492m bits > 60m bps (4G LTE) 14.92m bits < 60m bps (4G LTE) codec (h264) Streaming 1 second
  26. Resize This Window Resize This Window https://developer.android.com/guide/topics/media/media-formats.html

  27. HLS

  28. master playlist media playlists ts files .m3u8 .mp4 .mp4 .m3u8

    .mp4 .mp4 .ts
  29. master playlist media playlists ts files .m3u8 .mp4 .mp4 .m3u8

    .mp4 .mp4 .ts
  30. client master playlist .m3u8

  31. HTTP client master playlist .m3u8

  32. Master Playlist #EXTM3U #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=300000 http://mycompany.com/240p.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=1400000 http://mycompany.com/480p.m3u8 …

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=2400000 http://mycompany.com/720p.m3u8 …
  33. Master Playlist #EXTM3U #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=300000 http://mycompany.com/240p.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=1400000 http://mycompany.com/480p.m3u8 …

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=2400000 http://mycompany.com/720p.m3u8 … #EXTM3U
  34. Master Playlist #EXTM3U #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=300000 http://mycompany.com/240p.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=1400000 http://mycompany.com/480p.m3u8 …

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=2400000 http://mycompany.com/720p.m3u8 … #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=300000 http://mycompany.com/240p.m3u8
  35. Master Playlist #EXTM3U #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=300000 http://mycompany.com/240p.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=1400000 http://mycompany.com/480p.m3u8 …

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=2400000 http://mycompany.com/720p.m3u8 … #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=300000
  36. Master Playlist #EXTM3U #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=300000 http://mycompany.com/240p.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=1400000 http://mycompany.com/480p.m3u8 …

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=2400000 http://mycompany.com/720p.m3u8 … http://mycompany.com/240p.m3u8
  37. master playlist media playlists ts files .m3u8 .mp4 .mp4 .m3u8

    .mp4 .mp4 .ts
  38. client 480p 240p 720p HTTP .m3u8 .m3u8 .m3u8

  39. Media Playlist #EXT-X-VERSION:3 #EXTM3U #EXT-X-TARGETDURATION:10 #EXT-X-MEDIA-SEQUENCE:1 #EXTINF:10.0, http://media.example.com/segment1.ts #EXTINF:9.0, http://media.example.com/segment2.ts

    #EXTINF:7.5, http://media.example.com/segment3.ts #EXTINF:8.5, http://media.example.com/segment4.ts …
  40. Media Playlist #EXT-X-VERSION:3 #EXTM3U #EXT-X-TARGETDURATION:10 #EXT-X-MEDIA-SEQUENCE:1 #EXTINF:10.0, http://media.example.com/segment1.ts #EXTINF:9.0, http://media.example.com/segment2.ts

    #EXTINF:7.5, http://media.example.com/segment3.ts #EXTINF:8.5, http://media.example.com/segment4.ts … #EXTINF:10.0, http://media.example.com/segment1.ts
  41. Media Playlist #EXT-X-VERSION:3 #EXTM3U #EXT-X-TARGETDURATION:10 #EXT-X-MEDIA-SEQUENCE:1 #EXTINF:10.0, http://media.example.com/segment1.ts #EXTINF:9.0, http://media.example.com/segment2.ts

    #EXTINF:7.5, http://media.example.com/segment3.ts #EXTINF:8.5, http://media.example.com/segment4.ts … #EXTINF:10.0,
  42. Media Playlist #EXT-X-VERSION:3 #EXTM3U #EXT-X-TARGETDURATION:10 #EXT-X-MEDIA-SEQUENCE:1 #EXTINF:10.0, http://media.example.com/segment1.ts #EXTINF:9.0, http://media.example.com/segment2.ts

    #EXTINF:7.5, http://media.example.com/segment3.ts #EXTINF:8.5, http://media.example.com/segment4.ts … http://media.example.com/segment1.ts
  43. Media Playlist #EXT-X-VERSION:3 #EXTM3U #EXT-X-TARGETDURATION:10 #EXT-X-MEDIA-SEQUENCE:1 #EXTINF:10.0, http://media.example.com/segment1.ts #EXTINF:9.0, http://media.example.com/segment2.ts

    #EXTINF:7.5, http://media.example.com/segment3.ts #EXTINF:8.5, http://media.example.com/segment4.ts … http://media.example.com/segment2.ts
  44. Media Playlist #EXT-X-VERSION:3 #EXTM3U #EXT-X-TARGETDURATION:10 #EXT-X-MEDIA-SEQUENCE:1 #EXTINF:10.0, http://media.example.com/segment1.ts #EXTINF:9.0, http://media.example.com/segment2.ts

    #EXTINF:7.5, http://media.example.com/segment3.ts #EXTINF:8.5, http://media.example.com/segment4.ts … http://media.example.com/segment3.ts
  45. master playlist media playlists ts files .m3u8 .mp4 .mp4 .m3u8

    .mp4 .mp4 .ts
  46. client 480p 240p 720p HTTP .m3u8 .m3u8 .m3u8 .mp4 .mp4

    .ts .mp4 .mp4 .ts .mp4 .mp4 .ts
  47. client 480p 240p 720p HTTP .m3u8 .m3u8 .m3u8 .mp4 .mp4

    .ts .mp4 .mp4 .ts .mp4 .mp4 .ts
  48. HTTP client 480p 240p 720p .m3u8 .m3u8 .m3u8 .mp4 .mp4

    .ts .mp4 .mp4 .ts .mp4 .mp4 .ts
  49. HTTP client 480p 240p 720p .m3u8 .m3u8 .m3u8 .mp4 .mp4

    .ts .mp4 .mp4 .ts .mp4 .mp4 .ts
  50. MPEG-DASH

  51. MPD fragmented mp4 file Initialization file .mpd .mp4 .mp4

  52. MPD fragmented mp4 file Initialization file .mpd .mp4 .mp4

  53. MPD fragmented mp4 file Initialization file .mpd .mp4 .mp4

  54. MP4 ftyp: file type moov: header data mdat: audio or

    video data
  55. ftyp: file type moof: header data sidx: index of moof

    + mdat Fragmented MP4 mdat: audio or video data
  56. Fragmented MP4 fragmented mp4 segment files .mp4 .mp4 .mp4 .m4s

  57. MPD fragmented mp4 file Initialization file .mpd .mp4 .mp4

  58. MPD <MPD xmlns="urn:mpeg:dash:schema:mpd:2011" minBufferTime="PT1.500S" type="static" mediaPresentationDuration="PT0H3M10.024S" maxSegmentDuration="PT0H0M10.010S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011"> <Period duration="PT0H3M10.024S">

    <AdaptationSet segmentAlignment="true" maxWidth="1280" maxHeight="720" maxFrameRate="24000/1001" par="16:9" lang="und" subsegmentAlignment="true" subsegmentStartsWithSAP="1"> <Representation id="1" mimeType="video/mp4" codecs="avc1.64001F" width="1280" height="720" frameRate="24000/1001" sar="1:1" startWithSAP="1" bandwidth="400823"> <BaseURL>400k.mp4</BaseURL> <SegmentBase indexRangeExact="true" indexRange="919-1262"> <Initialization range="0-918"/> </SegmentBase> … </Representation> </AdaptationSet> <AdaptationSet segmentAlignment="true" lang="und" subsegmentAlignment="true" subsegmentStartsWithSAP="1"> <Representation id="3" mimeType="audio/mp4" codecs="mp4a.40.2" startWithSAP="1" bandwidth="197469"> <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/> <BaseURL>output_audio_dashinit.mp4</BaseURL> <SegmentBase indexRangeExact="true" indexRange="861-1132"> <Initialization range="0-860"/> </SegmentBase> </Representation> </AdaptationSet> </Period> </MPD>
  59. MPD <MPD> <Period> <AdaptationSet> <Representation> <Segment> </Segment> </Representation> </AdaptationSet> </Period>

    </MPD> ɾlive or onDemand ? ɾresolution ? ɾbyte range or segment path
  60. ExoPlayer

  61. val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) val playerView

    = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player = player val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true
  62. val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) val playerView

    = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player = player val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() )
  63. val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) val playerView

    = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player = player val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true val playerView = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player = player
  64. val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) val playerView

    = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player = player val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null)
  65. val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) val playerView

    = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player = player val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true player.prepare(mediaSource) player.playWhenReady = true
  66. val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) val playerView

    = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player = player val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true // clean up player.release()
  67. val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) val playerView

    = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player = player val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true
  68. val player = ExoPlayerFactory.newSimpleInstance(…) val playerView = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player =

    player val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true
  69. val player = ExoPlayerFactory.newSimpleInstance(…) val playerView = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player =

    player val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true
  70. val player = ExoPlayerFactory.newSimpleInstance(…) val playerView = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player =

    player val okHttpClient = OkHttpClient.Builder() .connectTimeout(10L, TimeUnit.SECONDS) .writeTimeout(10L, TimeUnit.SECONDS) .readTimeout(30L, TimeUnit.SECONDS) .build() val dataSourceFactory = DefaultDataSourceFactory(context, "user-agent") val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true
  71. val player = ExoPlayerFactory.newSimpleInstance(…) val playerView = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player =

    player val okHttpClient = OkHttpClient.Builder() .connectTimeout(10L, TimeUnit.SECONDS) .writeTimeout(10L, TimeUnit.SECONDS) .readTimeout(30L, TimeUnit.SECONDS) .build() val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true
  72. val player = ExoPlayerFactory.newSimpleInstance(…) val playerView = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player =

    player val okHttpClient = OkHttpClient.Builder() .connectTimeout(10L, TimeUnit.SECONDS) .writeTimeout(10L, TimeUnit.SECONDS) .readTimeout(30L, TimeUnit.SECONDS) .build() val dataSourceFactory = OkHttpDataSourceFactory(okHttpClient, “user-agent”, null) val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true
  73. val player = ExoPlayerFactory.newSimpleInstance(…) val playerView = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player =

    player val okHttpClient = OkHttpClient(…) val dataSourceFactory = OkHttpDataSourceFactory(okHttpClient, “user-agent”, null) val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true
  74. val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) val playerView

    = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player = player val okHttpClient = OkHttpClient(…) val dataSourceFactory = OkHttpDataSourceFactory(okHttpClient, “user-agent”, null) val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true
  75. val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) val playerView

    = findViewById<SimpleExoPlayerView>(R.id.player_view) playerView.player = player val okHttpClient = OkHttpClient(…) val dataSourceFactory = OkHttpDataSourceFactory(okHttpClient, “user-agent”, null) val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady = true DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl()
  76. Renderer val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() )

  77. Renderer val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) DefaultRenderersFactory(context)

  78. Renderer public class MyRenderer implements Renderer { // make your

    own renderer }
  79. Renderer public class MyRenderer implements Renderer { // make your

    own renderer } DefaultRenderersFactory(..., MyRenderer())
  80. TrackSelector val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() )

  81. TrackSelector val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) DefaultTrackSelector(),

  82. TrackSelector val selector = DefaultTrackSelector()

  83. TrackSelector val factory = AdaptiveTrackSelection.Factory( DefaultBandwidthMeter(), DEFAULT_MAX_INITIAL_BITRATE, DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS,

    DEFAULT_BANDWIDTH_FRACTION)
  84. TrackSelector val factory = AdaptiveTrackSelection.Factory( DefaultBandwidthMeter(), DEFAULT_MAX_INITIAL_BITRATE, DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS,

    DEFAULT_BANDWIDTH_FRACTION) val selector = DefaultTrackSelector(factory)
  85. TrackSelector val selector = DefaultTrackSelector(factory) val factory = AdaptiveTrackSelection.Factory( DefaultBandwidthMeter(),

    HIEST_BITRATE, DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS, DEFAULT_BANDWIDTH_FRACTION)
  86. client 480p 240p 720p HTTP .m3u8 .m3u8 .m3u8

  87. client 480p 240p 720p .m3u8 .m3u8 .m3u8 HTTP

  88. TrackSelector val selector = DefaultTrackSelector(factory) val factory = AdaptiveTrackSelection.Factory( DefaultBandwidthMeter(),

    DEFAULT_MAX_INITIAL_BITRATE, DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS, DEFAULT_BANDWIDTH_FRACTION)
  89. TrackSelector val selector = DefaultTrackSelector(factory) val factory = AdaptiveTrackSelection.Factory( DefaultBandwidthMeter(),

    DEFAULT_MAX_INITIAL_BITRATE, DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS, DEFAULT_BANDWIDTH_FRACTION) DefaultBandwidthMeter(),
  90. TrackSelector public final class DefaultBandwidthMeter implements BandwidthMeter, TransferListener<Object> { @Override

    public long getBitrateEstimate() { return currentBitrate; } @Override public void onTransferStart(Object source, DataSpec dataSpec) { } @Override public void onBytesTransferred(Object source, int bytesTransferred) { } @Override public void onTransferEnd(Object source) { } }
  91. TrackSelector public final class DefaultBandwidthMeter implements BandwidthMeter, TransferListener<Object> { @Override

    public long getBitrateEstimate() { return currentBitrate; } @Override public void onTransferStart(Object source, DataSpec dataSpec) { } @Override public void onBytesTransferred(Object source, int bytesTransferred) { } @Override public void onTransferEnd(Object source) { } } @Override public long getBitrateEstimate() { return currentBitrate; }
  92. TrackSelector public final class MyBandwidthMeter implements BandwidthMeter, TransferListener<Object> { @Override

    public long getBitrateEstimate() { return Math.min(currentBitrate, limitBitrate); } @Override public void onTransferStart(Object source, DataSpec dataSpec) { } @Override public void onBytesTransferred(Object source, int bytesTransferred) { } @Override public void onTransferEnd(Object source) { } } @Override public long getBitrateEstimate() { return Math.min(currentBitrate, limitBitrate); } MyBandwidthMeter
  93. TrackSelector val selector = DefaultTrackSelector(factory) val factory = AdaptiveTrackSelection.Factory( MyBandwidthMeter(),

    DEFAULT_MAX_INITIAL_BITRATE, DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS, DEFAULT_BANDWIDTH_FRACTION)
  94. TrackSelector val selector = DefaultTrackSelector(factory) val factory = AdaptiveTrackSelection.Factory( MyBandwidthMeter(),

    DEFAULT_MAX_INITIAL_BITRATE, DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS, DEFAULT_BANDWIDTH_FRACTION) MyBandwidthMeter(),
  95. client 480p 240p 720p .m3u8 .m3u8 .m3u8 HTTP

  96. client 480p 240p 720p .m3u8 .m3u8 .m3u8 HTTP

  97. TrackSelector public final class MyBandwidthMeter implements BandwidthMeter, TransferListener<Object> { @Override

    public long getBitrateEstimate() { return Math.max(currentBitrate, limitBitrate); } @Override public void onTransferStart(Object source, DataSpec dataSpec) { } @Override public void onBytesTransferred(Object source, int bytesTransferred) { } @Override public void onTransferEnd(Object source) { } }
  98. TrackSelector public final class MyBandwidthMeter implements BandwidthMeter, TransferListener<Object> { @Override

    public long getBitrateEstimate() { return Math.max(currentBitrate, limitBitrate); } @Override public void onTransferStart(Object source, DataSpec dataSpec) { } @Override public void onBytesTransferred(Object source, int bytesTransferred) { } @Override public void onTransferEnd(Object source) { } } return Math.max(currentBitrate, limitBitrate); MyBandwidthMeter
  99. client 480p 240p 720p HTTP .m3u8 .m3u8 .m3u8

  100. client 480p 240p 720p HTTP .m3u8 .m3u8 .m3u8

  101. LoadControl val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() )

  102. LoadControl val player = ExoPlayerFactory.newSimpleInstance( DefaultRenderersFactory(context), DefaultTrackSelector(), DefaultLoadControl() ) DefaultLoadControl()

  103. LoadControl val loadControl = DefaultLoadControl()

  104. LoadControl val loadControl = DefaultLoadControl( DefaultAllocator(true, C.DEFAULT_BUFFER_SEGMENT_SIZE), DEFAULT_MIN_BUFFER_MS, DEFAULT_MAX_BUFFER_MS, DEFAULT_BUFFER_FOR_PLAYBACK_MS,

    DEFAULT_BUFFER_FOR_PLAYBACK_AFTER_REBUFFER_MS )
  105. LoadControl val loadControl = DefaultLoadControl( DefaultAllocator(true, C.DEFAULT_BUFFER_SEGMENT_SIZE), DEFAULT_MIN_BUFFER_MS, DEFAULT_MAX_BUFFER_MS, DEFAULT_BUFFER_FOR_PLAYBACK_MS,

    DEFAULT_BUFFER_FOR_PLAYBACK_AFTER_REBUFFER_MS ) DEFAULT_MIN_BUFFER_MS, DEFAULT_MAX_BUFFER_MS, DEFAULT_BUFFER_FOR_PLAYBACK_MS, DEFAULT_BUFFER_FOR_PLAYBACK_AFTER_REBUFFER_MS
  106. MediaSource val mediaSource = HlsMediaSource(uri, dataSourceFactory, handler, null) player.prepare(mediaSource) player.playWhenReady

    = true
  107. MediaSource val mediaSource = DashMediaSource(Uri, …) player.prepare(mediaSource) player.playWhenReady = true

  108. MediaSource val mediaSource = ExtractorMediaSource(Uri, …) player.prepare(mediaSource) player.playWhenReady = true

  109. MediaSource val firstSource = HlsMediaSource(firstUri, …) val secondSource = HlsMediaSource(secondUri,

    …) val mediaSource = ConcatenatingMediaSource(firstSource, secondSource) // play firstSource, then secondSource player.prepare(mediaSource) player.playWhenReady = true
  110. Metadata

  111. Metadata video

  112. Metadata video createdAt? title? thumbnail? genre?

  113. Metadata ɾID3 tag

  114. Metadata ɾID3 tag .ts id = 1000 place = droidkaigi

  115. Metadata

  116. Metadata class Logger : MetadataOutput { override fun onMetadata(metadata: Metadata)

    { (1..metadata.length()) .map { metadata[it] } .filterIsInstance<TextInformationFrame>() .forEach { print("metadata: ${it.value}") } } } // metadata: id=1000,place=droidkaigi
  117. Metadata ɾID3 tag ɾemsg box

  118. Metadata ɾID3 tag id = 1000 place = droidkaigi ɾemsg

    box .mp4
  119. Metadata class Logger : MetadataOutput { override fun onMetadata(metadata: Metadata)

    { (1..metadata.length()) .map { metadata[it] } .filterIsInstance<EventMessage>() .forEach { print("metadata: ${it.value}") } } } // metadata: id=1000,place=droidkaigi
  120. Publish

  121. UDP TCP 1:1 1:n low latency high latency WebRTC RTMP

  122. RTMP RTMP client server

  123. RTMP RTMP client server users RTMP

  124. RTMP RTMP client server users HLS MPEG-DASH

  125. RTMP ɾdeveloped by Adobe. not “open” protocol ɾRed5 started rtmp

    in 2005 ɾWowza started rtmp in 2007 ɾrtmpdump released in 2008 ɾrtmp specification released 2012
  126. RTMP “we will stop updating and distributing the Flash Player

    at the end of 2020” -Adobe
  127. RTMP RTMP

  128. RTMP RTMP

  129. RTMP RTMP

  130. RTMP RTMP

  131. RTMP PRO CON ɾsmall header (basically 1, 4, 8, 12

    bytes) ɾrealtime ɾdifficult to cache ɾno multiple resolutions
  132. RTMP RTMP

  133. RTMP RTMP RTMP

  134. RTMP RTMP

  135. RTMP dependencies { implementation ‘com.google.android.exoplayer:exoplayer-core:x.x.x’ implementation ‘com.google.android.exoplayer:exoplayer-ui:x.x.x’ implementation ‘com.google.android.exoplayer:extension-rtmp:x.x.x’ }

  136. val mediaSource = ExtractorMediaSource( Uri.parse("rtmp://xxx.xx.xx.xxx:1935/path/stream_name") RtmpDataSourceFactory(), DefaultExtractorsFactory(), null, null) player.prepare(mediaSource)

    player.playWhenReady = true
  137. val mediaSource = ExtractorMediaSource( Uri.parse("rtmp://xxx.xx.xx.xxx:1935/path/stream_name") RtmpDataSourceFactory(), DefaultExtractorsFactory(), null, null) player.prepare(mediaSource)

    player.playWhenReady = true Uri.parse("rtmp://xxx.xx.xx.xxx:1935/path/stream_name")
  138. val mediaSource = ExtractorMediaSource( Uri.parse("rtmp://xxx.xx.xx.xxx:1935/path/stream_name") RtmpDataSourceFactory(), DefaultExtractorsFactory(), null, null) player.prepare(mediaSource)

    player.playWhenReady = true RtmpDataSourceFactory(),
  139. public final class RtmpDataSource implements DataSource { private RtmpClient rtmpClient;

    ... @Override public int read(byte[] buffer, int offset, int readLength) throws IOException { int bytesRead = rtmpClient.read(buffer, offset, readLength); ... return bytesRead; } ... }
  140. public final class RtmpDataSource implements DataSource { private RtmpClient rtmpClient;

    ... @Override public int read(byte[] buffer, int offset, int readLength) throws IOException { int bytesRead = rtmpClient.read(buffer, offset, readLength); ... return bytesRead; } ... } private RtmpClient rtmpClient; rtmpClient.read(buffer, offset, readLength);
  141. public class RtmpClient { … public int read(byte[] data, int

    offset, int size) throws IOException { return nativeRead(data, offset, size, rtmpPointer); } private native int nativeRead(byte[] data, int offset, int size, long rtmpPointer) throws IOException … }
  142. public class RtmpClient { … public int read(byte[] data, int

    offset, int size) throws IOException { return nativeRead(data, offset, size, rtmpPointer); } private native int nativeRead(byte[] data, int offset, int size, long rtmpPointer) throws IOException … } private native int nativeRead(byte[] data, int offset, int size, long rtmpPointer) public class RtmpClient
  143. RTMP RTMP

  144. RTMP ɾcapture data ɾencode data ɾpublish data

  145. RTMP ɾcapture data ɾencode data ɾpublish data

  146. RTMP ɾcapture data ɾencode data ɾpublish data val audioRecord =

    AudioRecord(...) Audio Data
  147. RTMP ɾcapture data ɾencode data ɾpublish data val audioRecord =

    AudioRecord(...) // camera2 API val manager : CameraManager manager.openCamera(...) // camera API val camera = Camera.(info) Audio Data Video Data
  148. RTMP ɾcapture data ɾencode data ɾpublish data

  149. RTMP // MediaCodec.java val encoder = MediaCodec(…) encoder.start() val inputBuffers

    = encoder.getInputBuffer(index) // or val surface = encoder.createInputSurface() // get encoded data val outputBuffers = encoder.getOutputBuffer(index) ɾcapture data ɾencode data ɾpublish data
  150. RTMP // MediaCodec.java val encoder = MediaCodec(…) encoder.start() val inputBuffers

    = encoder.getInputBuffer(index) // or val surface = encoder.createInputSurface() // get encoded data val outputBuffers = encoder.getOutputBuffer(index) ɾcapture data ɾencode data ɾpublish data // MediaCodec.java val encoder = MediaCodec(…) encoder.start()
  151. RTMP ɾcapture data ɾencode data ɾpublish data // MediaCodec.java val

    encoder = MediaCodec(…) encoder.start() val inputBuffers = encoder.getInputBuffer(index) // or val surface = encoder.createInputSurface() // get encoded data val outputBuffers = encoder.getOutputBuffer(index) val inputBuffers = encoder.getInputBuffer(index) // or val surface = encoder.createInputSurface()
  152. RTMP ɾcapture data ɾencode data ɾpublish data // MediaCodec.java val

    encoder = MediaCodec(…) encoder.start() val inputBuffers = encoder.getInputBuffer(index) // or val surface = encoder.createInputSurface() // get encoded data val outputBuffers = encoder.getOutputBuffer(index) // get encoded data val outputBuffers = encoder.getOutputBuffer(index)
  153. RTMP ɾcapture data ɾencode data ɾpublish data

  154. RTMP // rtmpdump public native int writeVideo( byte[] data, int

    offset, int length, int timestamp ) ɾcapture data ɾencode data ɾpublish data
  155. RTMP // rtmpdump public native int writeVideo( byte[] data, int

    offset, int length, int timestamp ) // Socket.java val socket = Socket(“192.xx.xx.xxx", 1935) val outputStream = DataOutputStream(…) outputStream.write(byteArray) ɾcapture data ɾencode data ɾpublish data
  156. https://github.com/TakuSemba/RtmpPublisher

  157. Demo

  158. Demo RTMP Server HLS RTMP Publisher Player Player

  159. Demo rtmp://160.16.54.236:1935/live/takusemba http://160.16.54.236/hls/takusemba.m3u8 RTMP HLS

  160. Media streaming on Android https://github.com/takusemba https://twitter.com/takusemba