Using Exoplayer

Transcript

1. 1.

   Using Exoplayer Effie Barak (@CodingChick)

2. 2.

   None
3. 3.

   What is Exoplayer ...and why is everyone talking about it

4. 4.

   Udemy's Story

5. 5.

   Media Player Simple integration, no extensibility mediaPlayer.setDataSource(url); mediaPlayer.prepare(); mediaPlayer.start();

6. 6.

   @Override public boolean onError(MediaPlayer mp, int what, int extra) {

   mediaPlayerError = true; if(what == -38) { //This exception just Happens for varios unexplainable reasons UdemyMediaPlayer.this.prepareAsync(); } return false; }
7. 7.

   This beautiful state machine is a lie

8. 8.

   Exoplayer Very extensible, but steeper implementation curve

9. 9.

   Solves these problems • Open source, written in Java •

   Built on top of Media Codec • Handles HLS correctly
10. 10.

    Extensible and/ or supports • Background playing • Subtitles •

    Variable playing speed • Variable resolutions
11. 11.

    Exoplayer basics

12. 12.

    Tips to get started: • ExoPlayer is written in Java.

    • The sample app is a good place to start. • The default implementaions are Good Implementaions.
13. 13.

    1st part The player player = ExoPlayer.Factory.newInstance( PlayerConstants.RENDERER_COUNT, MIN_BUFFER_MS, MIN_REBUFFER_MS);

    playerControl = new PlayerControl(player);
14. 14.

    Adding state listeners public abstract class UdemyBaseExoplayer implements ExoPlayer.Listener, ChunkSampleSource.EventListener,

    HlsSampleSource.EventListener, DefaultBandwidthMeter.EventListener, MediaCodecVideoTrackRenderer.EventListener, MediaCodecAudioTrackRenderer.EventListener player.addListener(this);
15. 15.

    2nd part The Core

16. 16.

    The journey of a Stream

17. 17.

    Allocator allocator = new DefaultAllocator(BUFFER_SEGMENT_SIZE); Handler mainHandler = player.getMainHandler(); DefaultBandwidthMeter

    bandwidthMeter = new DefaultBandwidthMeter(mainHandler, null); DataSource dataSource = new DefaultUriDataSource( context, bandwidthMeter, Util.getUserAgent(mContext, Constants.UDEMY_NAME)); ExtractorSampleSource sampleSource = new ExtractorSampleSource(uri, dataSource, allocator, BUFFER_SEGMENT_COUNT * BUFFER_SEGMENT_SIZE, mainHandler, player, 0); MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer(context, sampleSource, MediaCodecSelector.DEFAULT, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT, 5000, mainHandler, player, 50); MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer(sampleSource, MediaCodecSelector.DEFAULT, null, true, mainHandler, player, AudioCapabilities.getCapabilities(context), AudioManager.STREAM_MUSIC); TrackRenderer[] renderers = new TrackRenderer[PlayerConstants.RENDERER_COUNT]; renderers[PlayerConstants.TYPE_VIDEO] = videoRenderer; renderers[PlayerConstants.TYPE_AUDIO] = audioRenderer; player.onRenderers(renderers, bandwidthMeter);
18. 18.

    // Allocator allocator = new DefaultAllocator(BUFFER_SEGMENT_SIZE); // Handler mainHandler =

    player.getMainHandler(); // DefaultBandwidthMeter bandwidthMeter = new DefaultBandwidthMeter(mainHandler, null); // DataSource dataSource = new DefaultUriDataSource(context, bandwidthMeter, // Util.getUserAgent(mContext, Constants.UDEMY_NAME)); ExtractorSampleSource sampleSource = new ExtractorSampleSource(uri, dataSource, allocator, BUFFER_SEGMENT_COUNT * BUFFER_SEGMENT_SIZE, mainHandler, player, 0); MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer(context, sampleSource, MediaCodecSelector.DEFAULT, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT, 5000, mainHandler, player, 50); MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer(sampleSource, MediaCodecSelector.DEFAULT, null, true, mainHandler, player, AudioCapabilities.getCapabilities(context), AudioManager.STREAM_MUSIC); TrackRenderer[] renderers = new TrackRenderer[PlayerConstants.RENDERER_COUNT]; renderers[PlayerConstants.TYPE_VIDEO] = videoRenderer; renderers[PlayerConstants.TYPE_AUDIO] = audioRenderer; player.onRenderers(renderers, bandwidthMeter);
19. 19.

    Build extractors DefaultUriDataSource uriDataSource = new DefaultUriDataSource (context, bandwidthMeter, userAgent);

    ExtractorSampleSource sampleSource = new ExtractorSampleSource (uri, uriDataSource, allocator, PlayerConstants.BUFFER_SEGMENT_COUNT * PlayerConstants.BUFFER_SEGMENT_SIZE);
20. 20.

    Build renderers TrackRenderer[] renderers = new TrackRenderer[PlayerConstants.RENDERER_COUNT]; MediaCodecAudioTrackRenderer audioRenderer =

    new MediaCodecAudioTrackRenderer( sampleSource, MediaCodecSelector.DEFAULT, null, true, player.getMainHandler(), player, AudioCapabilities.getCapabilities(context), AudioManager.STREAM_MUSIC); renderers[PlayerConstants.TYPE_AUDIO] = audioRenderer;
21. 21.

    Connect renderers to the player player.prepare(renderers);

22. 22.

    Udemy customizations to basic structure • We decreased the buffer

    size after getting OOM exceptions on low end devices. public static final int BUFFER_SEGMENT_SIZE = 16 * 1024; // Original value was 64 * 1024 public static final int VIDEO_BUFFER_SEGMENTS = 50; // Original value was 200 public static final int AUDIO_BUFFER_SEGMENTS = 20; // Original value was 54 public static final int BUFFER_SEGMENT_COUNT = 64; // Original value was 256
23. 23.

    Udemy customizations to basic structure • We decreased the buffer

    size after getting OOM exceptions on low end devices. • The ExtractorSampleSource gets a list of possible extractors to work with (mp3 and mp4 only) mp4Extractor = new Mp4Extractor(); mp3Extractor = new Mp3Extractor(); sampleSource = new ExtractorSampleSource(..., mp4Extractor, mp3Extractor);
24. 24.

    !

25. 25.

    HLS

26. 26.

    None
27. 27.

    HLS File

28. 28.

    None
29. 29.

    DefaultBandwidthMeter bandwidthMeter = new DefaultBandwidthMeter(); PtsTimestampAdjusterProvider timestampAdjusterProvider = new PtsTimestampAdjusterProvider();

    HlsChunkSource chunkSource = new HlsChunkSource(..., uriDataSource, url,..., bandwidthMeter, timestampAdjusterProvider, HlsChunkSource.ADAPTIVE_MODE_SPLICE);
30. 30.

    sampleSource = new HlsSampleSource(chunkSource, ...);

31. 31.

    None
32. 32.

    The good news HlsChunkSource chunkSource = new HlsChunkSource(..., HlsChunkSource.ADAPTIVE_MODE_NONE);

33. 33.

    The bad news // The index in variants of the

    currently selected variant. private int selectedVariantIndex; public void getChunkOperation(...) { int nextVariantIndex; ... if (adaptiveMode == ADAPTIVE_MODE_NONE) { nextVariantIndex = selectedVariantIndex; switchingVariantSpliced = false; } else { ... } ...
34. 34.

    !

35. 35.

    Background Media Playing

36. 36.

    Clear the surface by sending a message to ExoPlayer that

    sets the surface to null player.blockingSendMessage( videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, null);
37. 37.

    Create a service to keep the app alive. <service android:name="com.udemy.android.player.exoplayer.UdemyExoplayerService"

    android:exported="true" android:label="@string/app_name" android:enabled="true"> <intent-filter> <action android:name="ccom.udemy.android.player.exoplayer.UdemyExoplayerService"> </action> </intent-filter> </service> Both the view and the service control the same instance of the player, stored globally.
38. 38.

    When the app resumes set the surface again setPlayerSurface(surfaceView.getHolder().getSurface()); And

    in the player public void setSurface(Surface surface) { this.surface = surface; pushSurface(false); }
39. 39.

    Subtitles • Support .srt • Don't crash the video in

    case of error • Support multiple formats such as UTF-8
40. 40.

    Feed subtitles manually :(

41. 41.

    public void displayExoplayerSubtitles( File file, final MediaController.MediaPlayerControl playerControl, final ViewGroup

    subtitleLayout, final Context context) { convertFileCaptionList(file, context); runnableCode = new Runnable() { @Override public void run() { displayForPosition(playerControl.getCurrentPosition(), subtitleLayout, context); handler.postDelayed(runnableCode, 200); } }; handler.post(runnableCode); }
42. 42.

    Subtitles conversion https://github.com/JDaren/subtitleConverter

43. 43.

    Variable playback speeds

44. 44.

    Video always follows Audio

45. 45.

    Sonic library (The Java edition) https://github.com/waywardgeek/sonic/blob/master/ Sonic.java

46. 46.

    Implementation

47. 47.

    public class VariableSpeedAudioRenderer extends MediaCodecAudioTrackRenderer

48. 48.

    Method to override private byte[] sonicInputBuffer; private byte[] sonicOutputBuffer; @Override

    protected void onOutputFormatChanged(final MediaFormat format) {
49. 49.

    Method body // Two samples per frame * 2 to

    support audio speeds down to 0.5 final int bufferSizeBytes = SAMPLES_PER_CODEC_FRAME * 2 * 2 * channelCount; this.sonicInputBuffer = new byte[bufferSizeBytes]; this.sonicOutputBuffer = new byte[bufferSizeBytes]; this.sonic = new Sonic( format.getInteger(MediaFormat.KEY_SAMPLE_RATE), format.getInteger(MediaFormat.KEY_CHANNEL_COUNT)); this.lastInternalBuffer = ByteBuffer.wrap(sonicOutputBuffer, 0, 0); sonic.flushStream(); sonic.setSpeed(audioSpeed);
50. 50.

    @Override protected boolean processOutputBuffer(..., final ByteBuffer buffer,...)

51. 51.

    private ByteBuffer lastInternalBuffer; buffer.get(sonicInputBuffer, 0, bytesToRead); sonic.writeBytesToStream(sonicInputBuffer, bytesToRead); sonic.readBytesFromStream(sonicOutputBuffer, sonicOutputBuffer.length);

    return super.processOutputBuffer(..., lastInternalBuffer, ...);
52. 52.

    Some optimization if (bufferIndex == lastSeenBufferIndex) { return super.processOutputBuffer(..., lastInternalBuffer,

    ..., bufferIndex, ...); } else { lastSeenBufferIndex = bufferIndex; }
53. 53.

    To change the speed in the middle of consuming a

    buffer if (wasSpeedChanged) { sonic.flushStream(); sonic.setSpeed(audioSpeed); }
54. 54.

    Thank you!