[Anton Minashkin] ExoPlayer: Media playback without pain (almost...)

Presentation from GDG DevFest Ukraine 2016.
Learn more at: https://devfest.gdg.org.ua

Transcript

1. ExoPlayer: Media playback without pain (almost...) Anton Minashkin
2. #dfua What’s wrong with MediaPlayer? • State machine. Without state. • Device specific issue. • Documentation
3. #dfua What’s wrong with MediaPlayer? • Buffer • Network • Cache/Pre-caching
4. #dfua Pros and cons Support for Dynamic Adaptive Streaming over HTTP (DASH) and SmoothStreaming, neither of which are are supported by MediaPlayer (it also supports HTTP Live Streaming (HLS), FMP4, MP4, M4A, MKV, WebM, MP3, AAC, MPEG-TS, MPEG-PS, OGG, FLV and WAV).
5. #dfua Pros and cons Support for advanced HLS features, such as correct handling of #EXT-X-DISCONTINUITY tags.
6. #dfua Pros and cons The ability to customize and extend the player to suit your use case. ExoPlayer is designed specifically with this in mind, and allows many components to be replaced with custom implementations.
7. #dfua Pros and cons Easily update the player along with your application. Because ExoPlayer is a library that you include in your application apk, you have control over which version you use and you can easily update to a newer version as part of a regular application update.
8. #dfua Pros and cons Fewer device specific issues.
9. #dfua Pros and cons ExoPlayer’s standard audio and video components rely on Android’s MediaCodec API, which was released in Android 4.1 (API level 16). Hence they do not work on earlier versions of Android.
10. #dfua Pros and cons /** * Returns the current state of the player. * * @return One of the {@code STATE} constants defined in this interface. */ public int getPlaybackState();
11. #dfua Design
12. #dfua TrackRenderer A TrackRenderer plays a specific type of media, such as video, audio or text. The ExoPlayer class invokes methods on its TrackRenderer instances from a single playback thread, and by doing so causes each type of media be rendered as the global playback position is advanced.
13. #dfua TrackRenderer // 1. Instantiate the player. player = ExoPlayer.Factory.newInstance(RENDERER_COUNT); // 2. Construct renderers. MediaCodecVideoTrackRenderer videoRenderer = ... MediaCodecAudioTrackRenderer audioRenderer = ...
14. #dfua TrackRenderer // 3. Inject the renderers through prepare. player.prepare(videoRenderer, audioRenderer); // 4. Pass the surface to the video renderer. player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, surface);
15. #dfua TrackRenderer // 5. Start playback. player.setPlayWhenReady(true); ... player.release(); // Don’t forget to release when done!
16. #dfua SampleSource • ExtractorSampleSource – For formats such as FMP4, MP4, M4A, MKV, WebM, MP3, AAC, MPEG-TS, MPEG-PS, OGG, FLV and WAV. • ChunkSampleSource – For DASH and SmoothStreaming playbacks. • HlsSampleSource – For HLS playbacks.
17. #dfua DataSource • DefaultUriDataSource – For playing media that can be either local or loaded over the network. • AssetDataSource – For playing media stored in the assets folder of the application’s apk.
18. #dfua Traditional media playbacks Allocator allocator = new DefaultAllocator(BUFFER_SEGMENT_SIZE); DataSource dataSource = new DefaultUriDataSource(context, null, userAgent); ExtractorSampleSource sampleSource = new ExtractorSampleSource( uri, dataSource, allocator, BUFFER_SEGMENT_COUNT * BUFFER_SEGMENT_SIZE);
19. #dfua Traditional media playbacks MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer(context, sampleSource, MediaCodecSelector.DEFAULT, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT); MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer(sampleSource, MediaCodecSelector.DEFAULT);
20. #dfua Adaptive media playbacks ExoPlayer supports adaptive streaming, which allows the quality of the media data to be adjusted during playback based on the network conditions
21. #dfua DASH and SmoothStreaming
22. #dfua HLS
23. #dfua Player events During playback, your app can listen for events generated by ExoPlayer that indicate the overall state of the player. These events are useful as triggers for updating the app user interface such as playback controls. Many ExoPlayer components also report their own component specific low level events, which can be useful for performance monitoring.
24. #dfua High level events ExoPlayer allows instances of ExoPlayer.Listener to be added and removed using its addListener() and removeListener() methods. Registered listeners are notified of changes in playback state, as well as when errors occur that cause playback to fail.
25. #dfua Low level events In addition to high level listeners, many of the individual components provided by the ExoPlayer library allow their own event listeners.
26. #dfua Sending messages to components player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, surface); … player.blockingSendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, null);
27. #dfua Customization TrackRenderer – You may want to implement a custom TrackRenderer to handle media types other than audio and video.
28. #dfua Customization Extractor – If you need to support a container format not currently supported by the ExoPlayer library, consider implementing a custom Extractor class
29. #dfua Customization SampleSource – Implementing a custom SampleSource class may be appropriate if you wish to obtain media samples to feed to renderers in a custom way.
30. #dfua Customization DataSource – ExoPlayer’s upstream package already contains a number of DataSource implementations for different use cases.
31. #dfua Customization guidelines If a custom component needs to report events back to the app, we recommend that you do so using the same model as existing ExoPlayer components, where an event listener is passed together with a Handler to the constructor of the component.
32. #dfua Customization guidelines We recommended that custom components use the same model as existing ExoPlayer components to allow reconfiguration by the app during playback
33. #dfua ExoPlayer 2.x Solve design limitations that became apparent as new features were added to ExoPlayer 1.x, and remove the compromises that were made to work around them.
34. #dfua ExoPlayer 2.x Provide a base on which desired features such as DASH multi-period manifest support, gapless audio playbacks and seeking in live playbacks can be cleanly implemented.
35. #dfua ExoPlayer 2.x Provide a higher level API for simple use cases, without sacrificing flexibility for advanced use cases.
36. #dfua Summary: When you should NOT use? • Simple case • No networking • Static data • Video is NOT the main feature
37. #dfua Summary: When you SHOULD use? • Your app is a player • Customization needed • More control needed
38. Anton Minashkin @antonminashin Questions? Thank you!