Streaming the Droid

Transcript

1. Streaming to the Droid David Gonzalez Android Software Craftsman
2. Who is this guy?
3. What is he going to talk about?
5. Streaming protocols » Apple’s HTTP Live Streaming (HLS) » Adobe’s HTTP Dynamic Streaming (HDS) » Dynamic Adaptive Streaming over HTTP (aka MPEG- DASH) » Real Time Streaming Protocol (RTSP)
6. HLS Support
7. 2.3 Gingerbread No native support
8. 3.0 Honeycomb HoneyWHAT?
9. Solutions » Commercial: Adobe Prime Time, etc » Native: Custom native solution
10. Let's talk about Native TextureView vs SurfaceView
11. SurfaceView Creates a separate window Current VideoView is a SurfaceView
12. TextureView behaves as a regular View. This key difference allows a TextureView to be moved, transformed, animated, etc
13. Our approach /** * Displays a video file. The VideoView class * can load images from various sources (such as resources or content * providers), takes care of computing its measurement from the video so that * it can be used in any layout manager, and provides various display options * such as scaling and tinting.<p> * <p/> * <em>Note: VideoView does not retain its full state when going into the * background.</em> In particular, it does not restore the current play state, * play position, selected tracks added via * {@link android.app.Activity#onSaveInstanceState} and * {@link android.app.Activity#onRestoreInstanceState}.<p> * Also note that the audio session id (from {@link #getAudioSessionId}) may * change from its previously returned value when the VideoView is restored. */ public class TextureVideoView extends android.view.TextureView implements Player {
14. Preparing for playing mMediaPlayer.setDataSource(getContext(), mUri, mHeaders); mMediaPlayer.setSurface(new Surface(mSurfaceTexture)); mMediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC); mMediaPlayer.setScreenOnWhilePlaying(true); mMediaPlayer.prepareAsync(); // we don't set the target state here either, but preserve the target state that was there before. mCurrentState = STATE_PREPARING;
16. OnPrepared Listener mMediaPlayer.setOnPreparedListener(mPreparedListener); private MediaPlayer.OnPreparedListener mPreparedListener = new MediaPlayer.OnPreparedListener() { @Override public void onPrepared(final MediaPlayer mp) { mCurrentState = STATE_PREPARED; mCanPause = true; mCanSeekBack = true; mCanSeekForward = true; if (mOnPreparedListener != null) { mOnPreparedListener.onPrepared(mMediaPlayer); } if (mConcertPlayerController != null) { mConcertPlayerController.setEnabled(true); } if (seekToPosition != 0) { seekTo(seekToPosition); } ... } };
17. OnPrepared Listener videoSizeCalculator.setVideoSize(mp.getVideoWidth(), mp.getVideoHeight()); if (videoSizeCalculator.hasASizeYet()) { // We didn't actually change the size (it was already at the size we need), // so we won't get a "surface changed" callback, // so start the video here instead of in the callback. if (mTargetState == STATE_PLAYING) { start(); showMediaController(); } else if (pausedAt(seekToPosition)) { showStickyMediaController(); } } else { // We don't know the video size yet, but should start anyway. // The video size might be reported to us later. if (mTargetState == STATE_PLAYING) { start();
18. OnErrorListener Listener mMediaPlayer.setOnErrorListener(mErrorListener); private OnErrorListener mErrorListener = new OnErrorListener() { @Override public boolean onError(final MediaPlayer mp, final int frameworkError, final int implError) { if (mCurrentState == STATE_ERROR) { return true; } mCurrentState = STATE_ERROR; mTargetState = STATE_ERROR; hideMediaController(); if (allowPlayStateToHandle(frameworkError)) { return true; } if (allowErrorListenerToHandle(frameworkError, implError)) { return true; } handleError(frameworkError); return true; } };
19. OnErrorListener Listener switch (what) { case -1004: errorMessage += "MEDIA_ERROR_IO"; case -1007: errorMessage += "MEDIA_ERROR_MALFORMED"; case 200: errorMessage += "MEDIA_ERROR_NOT_VALID_FOR_PROGRESSIVE_PLAYBACK"; case 100: errorMessage += "MEDIA_ERROR_SERVER_DIED"; case 1: errorMessage += "MEDIA_ERROR_UNKNOWN"; case -1010: errorMessage += "MEDIA_ERROR_UNSUPPORTED"; default: errorMessage += "MEDIA_ERROR_UNKNOWN"; }
20. OnErrorListener Listener switch (extra) { case 800: errorMessage += "MEDIA_INFO_BAD_INTERLEAVING"; case 702: errorMessage += "MEDIA_INFO_BUFFERING_END"; case 701: errorMessage += "MEDIA_INFO_METADATA_UPDATE"; case 802: errorMessage += "MEDIA_INFO_METADATA_UPDATE"; case 801: errorMessage += "MEDIA_INFO_NOT_SEEKABLE"; case 1: errorMessage += "MEDIA_INFO_UNKNOWN"; case 3: errorMessage += "MEDIA_INFO_VIDEO_RENDERING_START"; case 700: errorMessage += "MEDIA_INFO_VIDEO_TRACK_LAGGING"; case -110: errorMessage += "MEDIA_ERROR_TIMED_OUT"; default: errorMessage += "MEDIA_INFO_UNKNOWN"; }
21. OnInfoListener Listener mMediaPlayer.setOnInfoListener(mInfoListener); private final OnInfoListener onInfoToPlayStateListener = new OnInfoListener() { @Override public boolean onInfo(final MediaPlayer mp, final int what, final int extra) { if (noPlayStateListener()) { return false; } if (MediaPlayer.MEDIA_INFO_VIDEO_RENDERING_START == what) { onPlayStateListener.onFirstVideoFrameRendered(); onPlayStateListener.onPlay(); } if (MediaPlayer.MEDIA_INFO_BUFFERING_START == what) { onPlayStateListener.onBuffer(); } if (MediaPlayer.MEDIA_INFO_BUFFERING_END == what) { onPlayStateListener.onPlay(); } return false; } }
22. Custom listeners mMediaPlayer.setOnVideoSizeChangedListener(mSizeChangedListener); mMediaPlayer.setOnCompletionListener(mCompletionListener); mMediaPlayer.setOnBufferingUpdateListener(mBufferingUpdateListener);
25. Custom Controller /** * Controller to manage syncing the ui models with the UI Controls and MediaPlayer. * <p/> * Note that the ui models have a narrow scope (i.e. chapter list, piece navigation), * their interaction is orchestrated by this controller.ø * <p/> * It's actually a view currently, as is the android MediaController. * (which is a bit odd and should be subject to change.) */ public final class ConcertPlayerController extends FrameLayout implements VideoTouchRoot.OnTouchReceiver, TextureVideoView.VideoController, TextureVideoView.OnPlayStateListener {
26. Custom Controller @Override protected void onCreate(final Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_play_concert); ... bindViews(); } private void attachMediaController() { if (mMediaPlayer != null && mConcertPlayerController != null) { mConcertPlayerController.setMediaPlayer(this); View anchorView = this.getParent() instanceof View ? (View) this.getParent() : this; mConcertPlayerController.setAnchorView(anchorView); mConcertPlayerController.setEnabled(isInPlaybackState()); } }
27. System UI private void setSystemUiVisibility(final boolean visible) { // always use the full screen estate to avoid resizing flicker int newVis = View.SYSTEM_UI_FLAG_LAYOUT_STABLE | View.SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN | View.SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION; if (!visible) { newVis |= View.SYSTEM_UI_FLAG_LOW_PROFILE | View.SYSTEM_UI_FLAG_FULLSCREEN | View.SYSTEM_UI_FLAG_HIDE_NAVIGATION; } final View decorView = getWindow().getDecorView(); decorView.setSystemUiVisibility(newVis); decorView.setOnSystemUiVisibilityChangeListener(new View.OnSystemUiVisibilityChangeListener() { @Override public void onSystemUiVisibilityChange(final int visibility) { if ((visibility & View.SYSTEM_UI_FLAG_LOW_PROFILE) == 0) { //no low_profile flag means show controls concertPlayerController.show(); } } }); }
31. Don't trust the device
32. LESSONS LEARNED » Test on every possible device » Encrypted Video Streams are always a problem » Delay between Video / Audio » minSDK 14
34. Further reading https://github.com/novoda/fenster https://github.com/google/grafika
35. THANKS