Playing Videos on Android? ExoPlayer2 at your service!

Playing videos might be easy at first but this can get very tricky when you have to support multiple video/audio formats or offline medias. But no worries ExoPlayer2 is there for you!

This session will cover the basics of the ExoPlayer such as how to play a video thanks to the library and its core features. We'll go through its architecture, how to set it up and what video format ExoPlayer supports from classic MP4 to adaptive streams like HLS or DASH with protected content. A large overview of its capabilities will be done through this talk.

Then we will have a look at some interesting extensions and features such as the download manager or the ffmpeg extension for more audio formats support, building playlists or even managing you own DataSource for playing content on SMB servers.

Transcript

1. Julien Salvi Playing videos on Android? Exoplayer2 at your service! Place your screenshot here
2. Android addict since Froyo PAUG, Punk & IPA! You can find me at @JulienSalvi Julien Salvi Senior Android Engineer @ Innovorder
3. × Video consumption on mobile × ExoPlayer2: core features and extensions × ExoPlayer2 in action! × Go beyond with ExoPlayer
4. 1. Video consumption on mobile Gonna watch them all!
5. “Over 50% of consumption of video is taking place on mobile devices!”* *According to the Q2 Global Video Index report from Brightcove
6. Place your screenshot here Major video player on Android
7. × Open Source project started in 2014 × Based on MediaCodec API introduced in Android 4.1 (API 16) × Lots of features and extensions × Regular updates and reactive developers ExoPlayer A powerful media player for Android
8. Place your screenshot here Trusted by many Companies
9. 2. Exoplayer2: core features and extensions Deep dive into ExoPlayer2
10. × Big rewrite in 2016 to solve design limitations and workaround × Add a full support of DASH streams × Provide a better API for more complex use cases From ExoPlayer to Exoplayer2 ExoPlayer2 at your service!
11. ExoPlayer in a nutshell 10+ Extensions Supports DRM 3 10+ Audio formats MP3, AAC, Flac, MIDI, Opus, WAV... Text formats 4+ Adaptive streaming 3 10+ Video formats MP4, FMP4, FLV, WebM, MKV, Ogg...
12. Add ExoPlayer dependencies to your project repositories { google() jcenter() } dependencies { implementation 'com.google.android.exoplayer:exoplayer-core:2.11.3' implementation 'com.google.android.exoplayer:exoplayer-dash:2.11.3' implementation 'com.google.android.exoplayer:exoplayer-hls:2.11.3' implementation 'com.google.android.exoplayer:exoplayer-smoothstreaming:2.11.3' implementation 'com.google.android.exoplayer:exoplayer-ui:2.11.3' implementation 'com.google.android.exoplayer:extension-okhttp:2.11.3' } ExoPlayer2
13. Basic setup with ExoPlayer2 player = new SimpleExoPlayer.Builder(this).build(); // Attach player to the PlayerView playerView.setPlayer(player); // Create default data source DefaultDataSourceFactory dataSourceFactory = new DefaultDataSourceFactory( this, Util.getUserAgent(this, "ExoPlayerDevFestBrest")); // Create media source ExtractorMediaSource mediaSource = new ExtractorMediaSource.Factory(dataSourceFactory).createMediaSource(Uri.parse(URL)); // Prepare player and start when ready player.prepare(mediaSource); player.setPlayWhenReady(true);
14. × Progressive download (MP4) or dynamic streams (DASH) × Dynamic playlist, shuffle & repeat mode, stream clipping, side-loading source... × Downloader for offline playback × UI components with custom layouts ExoPlayer2: core features The Force is strong with ExoPlayer
15. ExoPlayer2: core features
16. × Defined the media to be played × Injected when preparing the player × ExoPlayer provide default MediaSource: - ProgressiveMediaSource - DashMediaSource - HlsMediaSource, - SsMediaSource ExoPlayer2: Media source Core components
17. Build MediaSource int type = PlayerUtils.inferContentType(mVideoUrl); switch (type) { case C.TYPE_DASH: return new DashMediaSource.Factory( new DefaultDashChunkSource.Factory(mediaDataSourceFactory), buildOkDataSourceFactory(true)).createMediaSource(uri); case C.TYPE_HLS: return new HlsMediaSource.Factory(buildOkDataSourceFactory(true)).createMediaSource(uri); case C.TYPE_OTHER: return new ProgressiveMediaSource.Factory(mediaDataSourceFactory).createMediaSource(uri); default: return null; }
18. × Build advanced and custom videos playbacks × Clip, loop or merge different media source × Build your own composition with ConcatenatingMediaSource ExoPlayer2: Media Source MediaSource compositions
19. Side-loaded subtitles MediaSource[] mediaSources = new MediaSource[subsSize]; for (int i = 0; i < subsSize; i++) { String subsUrl = mCaptions.get(i).url; mediaSources[i + 1] = new SingleSampleMediaSource.Factory(mediaDataSourceFactory) .createMediaSource(Uri.parse(subsUrl), Format.createTextSampleFormat("", PlayerUtils.getSubtitleMimeType(subsUrl), C.SELECTION_FLAG_AUTOSELECT, null, null), C.TIME_UNSET); } MergingMediaSource mergingMediaSource = new MergingMediaSource(videoSource, mediaSources); player.prepare(mergingMediaSource, playerPosition == 0, playerPosition == 0);
20. Custom MediaSource composition val ms1 = ExtractorMediaSource.Factory(dataSourceFactory).createMediaSource(Uri.parse(URL)); val ms2 = ExtractorMediaSource.Factory(dataSourceFactory).createMediaSource(Uri.parse(URL2)); val ms3 = ExtractorMediaSource.Factory(dataSourceFactory).createMediaSource(Uri.parse(URL3)); val clippingMediaSource = ClippingMediaSource(ms1, 30000000, 35000000); val clippingMediaSource2 = ClippingMediaSource(ms2, 35000000, 40000000); val loopingMediaSource2 = LoopingMediaSource(ms2, 2); val finalMediaSource = ConcatenatingMediaSource(clippingMediaSource, loopingMediaSource2, ms3);
21. × Components that render video, audio, text or metadata. × Consumes media from the MediaSource attached to the player × Injected when the player is created ExoPlayer2: Renderers Core components
22. × Selects tracks provided by the MediaSource to be consumed by each Render × You can control the max video size, the prefered audio language and more × ExoPlayer provides a default TrackSelector × Injected when the player is created ExoPlayer2: Track Selector Core components
23. × Controls when the MediaSource buffers and how much media is buffered × ExoPlayer provides a default LoadControl but can be custom × Injected when the player is created ExoPlayer2: Load control Core components
24. Build TrackSelector, Renderers and LoadControl // Init track selector val trackSelectorParameters = DefaultTrackSelector.ParametersBuilder().build() TrackSelection.Factory trackFactory = AdaptiveTrackSelection.Factory(BANDWIDTH_METER) val trackSelector = DefaultTrackSelector(trackFactory) trackSelector.setParameters(trackSelectorParameters) // Build audio, video and text renderers val renderersFactory = RenderersFactory( getApplicationContext(), DefaultRenderersFactory.EXTENSION_RENDERER_MODE_PREFER) // Init player with its renderers, track selector and DRM session manager. player = YourExoPlayer(renderersFactory, trackSelector, DefaultLoadControl(), drmSession)
25. × Basic UI for displaying and controlling your media × Customize all UI components with your own layout × /!\ Use SurfaceView over TextureView when possible /!\ ExoPlayer2: UI Components Customize all the things
26. ExoPlayer2: Event logger Track all the things! EventLogger: state [0.00, 0.00, window=0, true, BUFFERING] EventLogger: state [0.92, 0.04, window=0, period=0, true, READY] EventLogger: state [11.53, 10.60, window=0, period=0, false, READY] EventLogger: state [14.26, 10.60, window=0, period=0, true, READY] EventLogger: state [131.89, 128.27, window=0, period=0, true, ENDED] × Add EventLogger to player × Track playback state, get player information, media tracks or decoder selection × Know what’s going on real time
27. Exoplayer2 For more awesomeness! 27
28. × Supporting new audio or video formats (FLAC, ffmpeg, VP9, Opus…) × Supporting new stream or data sources (OkHttp, RTMP, cronet…) × Adding new features (Leanback, cast, GVR…) ExoPlayer2: extensions Extensions are there for...
29. × Most of the extensions can be added to your gradle dependencies × Ffmpeg, Opus or VP9 extension has to be built with the NDK before to be adding to your project × Use EXTENSION_RENDERER_MODE_ON for using renderer extensions ExoPlayer2: extensions Add extensions to your project
30. GVR extension: spatialized audio public class VrRenderersFactory extends DefaultRenderersFactory { private GvrAudioProcessor gvrAudioProcessor; public VrRenderersFactory(Context context, int extensionRendererMode) { super(context, extensionRendererMode); } @Override protected AudioProcessor[] buildAudioProcessors() { gvrAudioProcessor = new GvrAudioProcessor(); return new AudioProcessor[] {gvrAudioProcessor}; } public void setHeadOrientation(float... headOrientation) { gvrAudioProcessor.updateOrientation(headOrientation[0], headOrientation[1], headOrientation[2], headOrientation[3]); } }
31. ExoPlayer This library rocks because... × Supports a lot of video, audio and text formats × Highly customisable from core components to UI × DRM support from API 19 × Playlist and download management
32. Exoplayer This library is not so great because... × Consumes more battery than MediaPlayer × Might increase your APK size × Might be difficult to customize (eg. Renderers)
33. 3. ExoPlayer2 in action! Unlimited poweeeeer!
34. × Introduced in ExoPlayer 2.5.0 × Wraps the Google Interactive Media Ads (IMA) SDK to display ads × Pre-roll or mid-roll ads, no rebuffering, VAST/VMAP ad tags support ! ExoPlayer2: IMA extension (ads) Ads Max: fury roll!
35. × Client-side or server-side ad insertion × Easy integration with ExoPlayer UI components × Alternative: use ClippingMediaSource to build your own mid-roll ads ExoPlayer2: IMA extension (ads) Ads Max: fury roll!
36. Setup IMA ads loader player = new SimpleExoPlayer.Builder(this).build(); ... MediaSource mediaSource = ... releaseAdsLoader(); loadedAdTagUri = Uri.parse(ADS_URI); mediaSource = createAdsMediaSource(mediaSource, loadedAdTagUri); // Attach ads loader to player if (adsLoader != null) { adsLoader.setPlayer(player); } // Prepare player and start when ready player.prepare(mediaSource); player.setPlayWhenReady(true);
37. Setup IMA ads loader player = new SimpleExoPlayer.Builder(this).build(); ... MediaSource mediaSource = ... releaseAdsLoader(); loadedAdTagUri = Uri.parse(ADS_URI); mediaSource = createAdsMediaSource(mediaSource, loadedAdTagUri); // Attach ads loader to player if (adsLoader != null) { adsLoader.setPlayer(player); } // Prepare player and start when ready player.prepare(mediaSource); player.setPlayWhenReady(true);
38. Create ads media source MediaSourceFactory adMediaSourceFactory = new MediaSourceFactory() { @Override public MediaSourceFactory setDrmSessionManager(DrmSessionManager<?> manager) { return this; } @Override public MediaSource createMediaSource(Uri uri) { ... } @Override public int[] getSupportedTypes() { return new int[] {C.TYPE_OTHER}; } }; adsLoader = new ImaAdsLoader(this, adTagUri); return new AdsMediaSource(mediaSource, adMediaSourceFactory, adsLoader, playerView);
39. Create ads media source MediaSourceFactory adMediaSourceFactory = new MediaSourceFactory() { @Override public MediaSourceFactory setDrmSessionManager(DrmSessionManager<?> manager) { return this; } @Override public MediaSource createMediaSource(Uri uri) { ... } @Override public int[] getSupportedTypes() { return new int[] {C.TYPE_OTHER}; } }; adsLoader = new ImaAdsLoader(this, adTagUri); return new AdsMediaSource(mediaSource, adMediaSourceFactory, adsLoader, playerView);
40. Create ads media source MediaSourceFactory adMediaSourceFactory = new MediaSourceFactory() { @Override public MediaSourceFactory setDrmSessionManager(DrmSessionManager<?> manager) { return this; } @Override public MediaSource createMediaSource(Uri uri) { ... } @Override public int[] getSupportedTypes() { return new int[] {C.TYPE_OTHER}; } }; adsLoader = new ImaAdsLoader(this, adTagUri); return new AdsMediaSource(mediaSource, adMediaSourceFactory, adsLoader, playerView);
41. ExoPlayer2: IMA extension (ads)
42. 4. Go beyond with exoplayer Amazing stuff ahead!
43. Go beyond with exoplayer ExoPlayer for a VR/AR application Video plugin for Game Engines Build your own extensions
44. Create your own extension × Build your own DataSource for enable the connection with the server × Inject the SmbDataSource when building the media sources × Stream your content! Playing video from SMB2/3
45. Build the SmbDataSourceFactory val smbFile = "smb://<user>:<pwd>@<host>/<video_path>/<video>.mkv" class SmbDataSourceFactory(private val fileUrl: String) : DataSource.Factory { override fun createDataSource(): DataSource { val dataSpec = DataSpec(Uri.parse(fileUrl)) return SmbDataSource(dataSpec) } }
46. Build the SmbDataSource class SmbDataSource constructor(private val dataSpec: DataSpec) : DataSource { override fun open(dataSpec: DataSpec): Long override fun read(buffer: ByteArray, offset: Int, readLength: Int): Int override fun getUri(): Uri override fun close() }
47. Build the SmbDataSource class SmbDataSource constructor(private val dataSpec: DataSpec) : DataSource { private var hostname: String? = null private var username: String? = null private var password: String? = null private var shareName: String? = null private var path: String? = null private var smbClient: SMBClient? = null private var inputStream: InputStream? = null . }
48. Build the SmbDataSource override fun open(dataSpec: DataSpec): Long { smbClient = SMBClient() val connection = smbClient!!.connect(hostname) val ac = AuthenticationContext(username, password, "WORKGROUP") val session = connection.authenticate(ac) // Connect to Share val share = session.connectShare(shareName) as DiskShare val s = HashSet<SMB2ShareAccess>() s.add(SMB2ShareAccess.ALL.iterator().next()) // this is to get READ only val remoteSmbjFile = share.openFile(path, ...) inputStream = remoteSmbjFile.getInputStream() // Process inputStream here opened = true return bytesRemaining }
49. Build the SmbDataSource override fun open(dataSpec: DataSpec): Long { smbClient = SMBClient() val connection = smbClient!!.connect(hostname) val ac = AuthenticationContext(username, password, "WORKGROUP") val session = connection.authenticate(ac) // Connect to Share val share = session.connectShare(shareName) as DiskShare val s = HashSet<SMB2ShareAccess>() s.add(SMB2ShareAccess.ALL.iterator().next()) // this is to get READ only val remoteSmbjFile = share.openFile(path, ...) inputStream = remoteSmbjFile.getInputStream() // Process inputStream here opened = true return bytesRemaining }
50. Build the SmbDataSource override fun read(buffer: ByteArray, offset: Int, readLength: Int): Int { ... val bytesToRead = if (bytesRemaining == C.LENGTH_UNSET.toLong()) { readLength } else { Math.min(bytesRemaining, readLength.toLong()).toInt() } val bytesRead = inputStream.read(buffer, offset, bytesToRead) ... return bytesRead }
51. Inject the DataSource private fun buildMediaSource(uri: Uri): MediaSource { ... if (PlayerUtils.isSmb(videoUrl)) { return new ExtractorMediaSource.Factory( new SmbDataSourceFactory(mVideoUrl)).createMediaSource(uri); } ... } private fun preparePlayer() { player.prepare(buildMediaSource(Uri.parse(videoUrl)), true, true); }
52. Inject the DataSource private fun buildMediaSource(uri: Uri): MediaSource { ... if (PlayerUtils.isSmb(videoUrl)) { return new ExtractorMediaSource.Factory( new SmbDataSourceFactory(mVideoUrl)).createMediaSource(uri); } ... } private fun preparePlayer() { player.prepare(buildMediaSource(Uri.parse(videoUrl)), true, true); }
53. Exoplayer for VR apps × Use the GVR for building your VR application × OpenGL ES for your 3D objects × Attach a SurfaceTexture to your 3D object
54. Exoplayer Plugin for Unity For example you can use ExoPlayer for building a reliable player for Unity thanks to the JNI
55. DEMO 55
56. Want to know more? https://exoplayer.dev ExoPlayer documentation https://medium.com/google-exoplayer ExoPlayer developer blog https://medium.com/@cinemur/73ec7e83dd5c ExoPlayer for building powerful VR players on Cardboard and GearVR
57. Trug’ ! Have fun with ExoPlayer and Android! Any questions? @JulienSalvi