Google Cast: A Field Guide

A field guide for working with Google Cast

Transcript

1. Google Cast: A Field Guide Caleb Smith @co_obec
2. Happy Birthday! • 2013 Jul. Chromecast • 2015 Mar. Google Cast • 2015 May Game Manager, Remote Display • 2015 Sep. 2nd Gen. (Audio, too!) • 2015 Dec. Multi-room Audio • Recently Speaker and TV Partnerships
3. Terms • Senders • Mobile devices • Browsers • Receivers • Chromecast • Speakers • TVs
4. Framework • Discovery and Connection • Message Transfer • Framework APIs • Media • Games • Remote Display
5. Framework • Discovery and Connection • Message Transfer • Framework APIs • Media • Games • Remote Display
6. UX • Established Platform • User Expectations • UX Guidelines • UI Components • UX Expectations
7. Media App https://github.com/googlecast/CastVideos-android
8. Media App https://github.com/googlecast/CastVideos-android
9. Where to Start? • Start from Scratch
10. Where to Start? • Start from Scratch • Cast Companion Library (CCL) • NEW - Cast SDK v3
11. Cast Companion Library (CCL) • Large Developer and App Usage • Officially Supported Library • Common Boilerplate • Customization • Android Only • Continued Support
12. Cast SDK v3 • CCL (-Expanded*) • Android and iOS • Part of Play Services • Flexible • Customizable / Themes
13. Configure public class CastOptionsProvider implements OptionsProvider {
 
 @Override
 public CastOptions getCastOptions(Context context) {
 return new CastOptions.Builder()
 .setReceiverApplicationId("12345678")
 .build();
 }
 
 @Override
 public List<SessionProvider> getAdditionalSessionProviders( Context appContext) {
 return null;
 }
 
 } <!-- AndroidManifest.xml -->
 <meta-data
 android:name="com.google.android.gms.cast.framework.OPTIONS_PROVIDER_CLASS_NAME"
 android:value="cast.demo.app.CastOptionsProvider"/>

14. Configure public class CastOptionsProvider implements OptionsProvider {
 
 @Override
 public CastOptions getCastOptions(Context context) {
 return new CastOptions.Builder()
 .setReceiverApplicationId("12345678")
 .build();
 }
 
 @Override
 public List<SessionProvider> getAdditionalSessionProviders( Context appContext) {
 return null;
 }
 
 } <!-- AndroidManifest.xml -->
 <meta-data
 android:name="com.google.android.gms.cast.framework.OPTIONS_PROVIDER_CLASS_NAME"
 android:value="cast.demo.app.CastOptionsProvider"/>

15. Configure public class CastOptionsProvider implements OptionsProvider {
 
 @Override
 public CastOptions getCastOptions(Context context) {
 return new CastOptions.Builder()
 .setReceiverApplicationId("12345678")
 .build();
 }
 
 @Override
 public List<SessionProvider> getAdditionalSessionProviders( Context appContext) {
 return null;
 }
 
 } <!-- AndroidManifest.xml -->
 <meta-data
 android:name="com.google.android.gms.cast.framework.OPTIONS_PROVIDER_CLASS_NAME"
 android:value="cast.demo.app.CastOptionsProvider"/>

16. Add Cast Icon <!-- main.xml --> <?xml version="1.0" encoding="utf-8"?>
 <menu xmlns:android="http://schemas.android.com/apk/res/android"
 xmlns:app="http://schemas.android.com/apk/res-auto">
 
 <item
 android:id="@+id/cast_route_item"
 android:title="@string/cast_item_title"
 app:actionProviderClass="android.support.v7.app.MediaRouteActionProvider"
 app:showAsAction="always" />
 
 </menu> // MainActivity.java @Override
 public boolean onCreateOptionsMenu(Menu menu) {
 super.onCreateOptionsMenu(menu);
 getMenuInflater().inflate(R.menu.main, menu);
 CastButtonFactory.setUpMediaRouteButton(this, menu, R.id.cast_route_item);
 return true;
 }
17. Add Cast Icon <!-- main.xml --> <?xml version="1.0" encoding="utf-8"?>
 <menu xmlns:android="http://schemas.android.com/apk/res/android"
 xmlns:app="http://schemas.android.com/apk/res-auto">
 
 <item
 android:id="@+id/cast_route_item"
 android:title="@string/cast_item_title"
 app:actionProviderClass="android.support.v7.app.MediaRouteActionProvider"
 app:showAsAction="always" />
 
 </menu> // MainActivity.java @Override
 public boolean onCreateOptionsMenu(Menu menu) {
 super.onCreateOptionsMenu(menu);
 getMenuInflater().inflate(R.menu.main, menu);
 CastButtonFactory.setUpMediaRouteButton(this, menu, R.id.cast_route_item);
 return true;
 }
18. Add Cast Icon <!-- main.xml --> <?xml version="1.0" encoding="utf-8"?>
 <menu xmlns:android="http://schemas.android.com/apk/res/android"
 xmlns:app="http://schemas.android.com/apk/res-auto">
 
 <item
 android:id="@+id/cast_route_item"
 android:title="@string/cast_item_title"
 app:actionProviderClass="android.support.v7.app.MediaRouteActionProvider"
 app:showAsAction="always" />
 
 </menu> // MainActivity.java @Override
 public boolean onCreateOptionsMenu(Menu menu) {
 super.onCreateOptionsMenu(menu);
 getMenuInflater().inflate(R.menu.main, menu);
 CastButtonFactory.setUpMediaRouteButton(this, menu, R.id.cast_route_item);
 return true;
 }
19. Ready to Connect
20. What did we just do? MenuItem menuItem = menu.findItem(R.id.cast_route_item);
 
 MediaRouteActionProvider actionProvider = (MediaRouteActionProvider) MenuItemCompat.getActionProvider(menuItem);
 
 CastContext castContext = CastContext.getSharedInstance(MainActivity.this); 
 MediaRouteSelector routeSelector = castContext.zzaij();
 actionProvider.setRouteSelector(routeSelector);
21. What did we just do? MenuItem menuItem = menu.findItem(R.id.cast_route_item);
 
 MediaRouteActionProvider actionProvider = (MediaRouteActionProvider) MenuItemCompat.getActionProvider(menuItem);
 
 CastContext castContext = CastContext.getSharedInstance(MainActivity.this); 
 MediaRouteSelector routeSelector = castContext.zzaij();
 actionProvider.setRouteSelector(routeSelector);
22. Cast Context • Singleton • Lazily Loaded • Configured with Options Provider • Centralized Access
23. Cast Context Listeners private CastStateListener castStateListener = new CastStateListener() {
 @Override
 public void onCastStateChanged(int castState) {
 switch (castState) {
 case CastState.CONNECTED:
 case CastState.CONNECTING:
 case CastState.NOT_CONNECTED: // devices available
 case CastState.NO_DEVICES_AVAILABLE:
 }
 }
 }; private AppVisibilityListener appVisibilityListener = new AppVisibilityListener() {
 @Override
 public void onAppEnteredForeground() { }
 
 @Override
 public void onAppEnteredBackground() { }
 };
24. Cast Context Listeners castContext = CastContext.getSharedInstance(this); // any context, anywhere
 
 castContext.addCastStateListener(castStateListener);
 castContext.addAppVisibilityListener(appVisibilityListener);

25. Session Manager SessionManager sessionManager = castContext.getSessionManager(); // SessionManagerListener<CastSession> - Session Lifecycle Callbacks, Errors
 sessionManager.addSessionManagerListener( sessionManagerListener, CastSession.class); castSession = sessionManager.getCurrentCastSession();
26. Session Manager SessionManager sessionManager = castContext.getSessionManager(); // SessionManagerListener<CastSession> - Session Lifecycle Callbacks, Errors
 sessionManager.addSessionManagerListener( sessionManagerListener, CastSession.class); castSession = sessionManager.getCurrentCastSession();
27. Cast Session • Application Info • Device Info • Access to RemoteMediaClient • Access to Messaging
28. Cast Session • Application Info • Device Info • Access to RemoteMediaClient • Access to Messaging
29. Configure Media // CastOptionsProvider.java NotificationOptions notificationOptions = new NotificationOptions.Builder()
 .setActions(Arrays.asList( // up to 5
 MediaIntentReceiver.ACTION_TOGGLE_PLAYBACK,
 MediaIntentReceiver.ACTION_SKIP_NEXT,
 MediaIntentReceiver.ACTION_DISCONNECT
 ), new int[] { 0, 2 }) // Show in compat (condensed) view
 .setSkipStepMs(NotificationOptions.SKIP_STEP_TEN_SECONDS_IN_MS)
 .setTargetActivityClassName( ExpandedControlsActivity.class.getName())
 .build();
30. Configure Media // CastOptionsProvider.java NotificationOptions notificationOptions = new NotificationOptions.Builder()
 .setActions(Arrays.asList( // up to 5
 MediaIntentReceiver.ACTION_TOGGLE_PLAYBACK,
 MediaIntentReceiver.ACTION_SKIP_NEXT,
 MediaIntentReceiver.ACTION_DISCONNECT
 ), new int[] { 0, 2 }) // Show in compat (condensed) view
 .setSkipStepMs(NotificationOptions.SKIP_STEP_TEN_SECONDS_IN_MS)
 .setTargetActivityClassName( ExpandedControlsActivity.class.getName())
 .build();
31. Configure Media // CastOptionsProvider.java NotificationOptions notificationOptions = new NotificationOptions.Builder()
 .setActions(Arrays.asList( // up to 5
 MediaIntentReceiver.ACTION_TOGGLE_PLAYBACK,
 MediaIntentReceiver.ACTION_SKIP_NEXT,
 MediaIntentReceiver.ACTION_DISCONNECT
 ), new int[] { 0, 2 }) // Show in compat (condensed) view
 .setSkipStepMs(NotificationOptions.SKIP_STEP_TEN_SECONDS_IN_MS)
 .setTargetActivityClassName( ExpandedControlsActivity.class.getName())
 .build();
32. Configure Media // CastOptionsProvider.java NotificationOptions notificationOptions = new NotificationOptions.Builder()
 .setActions(Arrays.asList( // up to 5
 MediaIntentReceiver.ACTION_TOGGLE_PLAYBACK,
 MediaIntentReceiver.ACTION_SKIP_NEXT,
 MediaIntentReceiver.ACTION_DISCONNECT
 ), new int[] { 0, 2 }) // Show in compat (condensed) view
 .setSkipStepMs(NotificationOptions.SKIP_STEP_TEN_SECONDS_IN_MS)
 .setTargetActivityClassName( ExpandedControlsActivity.class.getName())
 .build();
33. Configure Media // CastOptionsProvider.java CastMediaOptions mediaOptions = new CastMediaOptions.Builder()
 .setNotificationOptions(notificationOptions) .setExpandedControllerActivityClassName( ExpandedControlsActivity.class.getName()) // careful!
 //.setMediaIntentReceiverClassName(CustomReceiver.java) .setImagePicker(new ImagePickerImpl())
 .build(); castOptionsBuilder.setCastMediaOptions(mediaOptions)
 .build();
34. Configure Media // CastOptionsProvider.java CastMediaOptions mediaOptions = new CastMediaOptions.Builder()
 .setNotificationOptions(notificationOptions) .setExpandedControllerActivityClassName( ExpandedControlsActivity.class.getName()) // careful!
 //.setMediaIntentReceiverClassName(CustomReceiver.java) .setImagePicker(new ImagePickerImpl())
 .build(); castOptionsBuilder.setCastMediaOptions(mediaOptions)
 .build();
35. Configure Media // CastOptionsProvider.java CastMediaOptions mediaOptions = new CastMediaOptions.Builder()
 .setNotificationOptions(notificationOptions) .setExpandedControllerActivityClassName( ExpandedControlsActivity.class.getName()) // careful!
 //.setMediaIntentReceiverClassName(CustomReceiver.java) .setImagePicker(new ImagePickerImpl())
 .build(); castOptionsBuilder.setCastMediaOptions(mediaOptions)
 .build();
36. Configure Media // CastOptionsProvider.java CastMediaOptions mediaOptions = new CastMediaOptions.Builder()
 .setNotificationOptions(notificationOptions) .setExpandedControllerActivityClassName( ExpandedControlsActivity.class.getName()) // careful!
 //.setMediaIntentReceiverClassName(CustomReceiver.java) .setImagePicker(new ImagePickerImpl())
 .build(); castOptionsBuilder.setCastMediaOptions(mediaOptions)
 .build();
37. Configure Media // CastOptionsProvider.java CastMediaOptions mediaOptions = new CastMediaOptions.Builder()
 .setNotificationOptions(notificationOptions) .setExpandedControllerActivityClassName( ExpandedControlsActivity.class.getName()) // careful!
 //.setMediaIntentReceiverClassName(CustomReceiver.java) .setImagePicker(new ImagePickerImpl())
 .build(); castOptionsBuilder.setCastMediaOptions(mediaOptions)
 .build();
38. Image Picker public class ImagePickerImpl extends ImagePicker {
 
 @Override
 public WebImage onPickImage(MediaMetadata mediaMetadata, int type) {
 
 int imageIndex;
 
 if (type == IMAGE_TYPE_MEDIA_ROUTE_CONTROLLER_DIALOG_BACKGROUND) {
 imageIndex = 0;
 } else {
 imageIndex = 1;
 }
 
 if (mediaMetadata.hasImages() && mediaMetadata.getImages().size() > imageIndex) {
 return mediaMetadata.getImages().get(imageIndex);
 }
 
 return null;
 }
 
 }
39. Media Configured • Volume Controls ✓ • Image Selection ✓ • Notification ✓ • Lock Screen Background ✓
40. Creating Media MediaMetadata metadata = new MediaMetadata(MediaMetadata.MEDIA_TYPE_GENERIC);
 
 metadata.putString(MediaMetadata.KEY_TITLE, "Title");
 
 WebImage thumbnail = new WebImage(Uri.parse("http://demo/thumbnail.png"));
 WebImage background = new WebImage(Uri.parse("http://demo/background.png"));
 
 metadata.addImage(thumbnail);
 metadata.addImage(background); // ordering is important!
41. Creating Media MediaMetadata metadata = new MediaMetadata(MediaMetadata.MEDIA_TYPE_USER + 1);
 metadata.putString(MediaMetadata.KEY_TITLE, "Title");
 
 // ... 
 MediaInfo info = remoteMediaClient.getMediaInfo();
 String title = info.getMetadata().getString(MediaMetadata.KEY_TITLE);
 
 TextUtils.isEmpty(title); // empty!
42. Creating Media MediaMetadata metadata = new MediaMetadata(MediaMetadata.MEDIA_TYPE_USER + 1);
 metadata.putString(MediaMetadata.KEY_TITLE, "Title");
 
 // ... 
 MediaInfo info = remoteMediaClient.getMediaInfo();
 String title = info.getMetadata().getString(MediaMetadata.KEY_TITLE);
 
 TextUtils.isEmpty(title); // empty!
43. Creating Media MediaMetadata metadata = new MediaMetadata(MediaMetadata.MEDIA_TYPE_USER + 1);
 metadata.putString(MediaMetadata.KEY_TITLE, "Title");
 
 // ... 
 MediaInfo info = remoteMediaClient.getMediaInfo();
 String title = info.getMetadata().getString(MediaMetadata.KEY_TITLE);
 
 TextUtils.isEmpty(title); // empty!
44. Creating Media MediaMetadata metadata = new MediaMetadata(MediaMetadata.MEDIA_TYPE_USER);
 metadata.putString(MediaMetadata.KEY_TITLE, "Title");
 
 // ... 
 MediaInfo info = remoteMediaClient.getMediaInfo();
 String title = info.getMetadata().getString(MediaMetadata.KEY_TITLE);
 
 TextUtils.isEmpty(title); // empty! Defined keys are reserved for defined media types
45. Creating Media String contentId = “http://demo/test.mp4"; 
 MediaInfo mediaInfo = new MediaInfo.Builder(contentId)
 .setContentType("video/mp4")
 .setStreamType(MediaInfo.STREAM_TYPE_LIVE)
 .setMetadata(metadata)
 .build();
46. Creating Media String contentId = “http://demo/test.mp4"; 
 MediaInfo mediaInfo = new MediaInfo.Builder(contentId)
 .setContentType("video/mp4")
 .setStreamType(MediaInfo.STREAM_TYPE_LIVE)
 .setMetadata(metadata)
 .build(); ← Core Object
47. Core Object • Broadcast to All Senders • Needs All Necessary Information • UI • Interaction • RemoteMediaClient → MediaInfo • GameManager → GameState
48. Custom Media Data String contentId = “http://demo/test.mp4"; 
 MediaInfo.Builder mediaInfoBuilder = new MediaInfo.Builder(contentId)
 .setContentType("video/mp4")
 .setStreamType(MediaInfo.STREAM_TYPE_LIVE)
 .setMetadata(metadata); JSONObject customJson = new JSONObject("{\"homeTeam\": \"\", \"awayTeam\": \"\"}");
 infoBuilder.setCustomData(customJson);
49. Load Media RemoteMediaClient remoteMediaClient = castSession.getRemoteMediaClient();
 
 remoteMediaClient.load(mediaInfo);
 
 remoteMediaClient.load(mediaInfo, autoStart, startPosition);
 
 remoteMediaClient.load(mediaInfo, autoStart, startPosition, customJson);

50. Updating UI RemoteMediaClient remoteMediaClient = castSession.getRemoteMediaClient();
 remoteMediaClient.pause();
 
 showPause(); // don't do this!
51. Updating UI private class RemoteMediaClientListener implements RemoteMediaClient.Listener {
 
 @Override
 public void onSendingRemoteMediaRequest() {
 showSpinner();
 }
 
 @Override
 public void onStatusUpdated() {
 MediaStatus mediaStatus = remoteMediaClient.getMediaStatus();
 switch (mediaStatus.getPlayerState()) {
 case MediaStatus.PLAYER_STATE_PAUSED:
 showPause();
 // ...
 }
 }
 
 // … RemoteMediaClient remoteMediaClient = castSession.getRemoteMediaClient(); remoteMediaClient.addListener(new RemoteMediaClientListener());
 remoteMediaClient.pause();
52. Updating UI private class RemoteMediaClientListener implements RemoteMediaClient.Listener {
 
 @Override
 public void onSendingRemoteMediaRequest() {
 showSpinner();
 }
 
 @Override
 public void onStatusUpdated() {
 MediaStatus mediaStatus = remoteMediaClient.getMediaStatus();
 switch (mediaStatus.getPlayerState()) {
 case MediaStatus.PLAYER_STATE_PAUSED:
 showPause();
 // ...
 }
 }
 
 // … RemoteMediaClient remoteMediaClient = castSession.getRemoteMediaClient(); remoteMediaClient.addListener(new RemoteMediaClientListener());
 remoteMediaClient.pause(); ← The Hard Way
53. UIMediaController // centralizes SessionManagerListener, RemoteMediaClient.Listener
 UIMediaController uiMediaController = new UIMediaController(MainActivity.this);
 
 // allows binding views to state
 uiMediaController.bindImageViewToImageOfCurrentItem(
 playingThumbnail,
 ImagePicker.IMAGE_TYPE_MINI_CONTROLLER_THUMBNAIL,
 R.drawable.cast_mini_controller_img_placeholder);
 
 uiMediaController.bindViewVisibilityToPreloadingEvent(upNextLayout, View.GONE);
 
 // called after all other RemoteMediaClient.Listener
 uiMediaController.setPostRemoteMediaClientListener(new RemoteMediaClientListener()); Remember -Expanded, this is the hook for it!
54. Custom Messages Cast.MessageReceivedCallback callback = new Cast.MessageReceivedCallback() {
 @Override
 public void onMessageReceived(CastDevice castDevice, String namespace, String message) {
 // handle message
 }
 };
 
 String namespace = "urn:x-cast:com.custom.cast";
 try {
 castSession.setMessageReceivedCallbacks(namespace, callback);
 } catch (IOException e) {
 // error!
 } // urn:x-cast:com.google.cast.media
55. Ads + Queue MediaMetadata adMetadata = new MediaMetadata(MEDIA_TYPE_AD);
 
 MediaInfo adInfo = new MediaInfo.Builder(adId)
 .setContentType("video/mp4")
 .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED)
 .setMetadata(adMetadata)
 .build();
 
 MediaQueueItem adItem= new MediaQueueItem.Builder(adInfo).build();
 MediaQueueItem mediaItem = new MediaQueueItem.Builder(mediaInfo).build();
 
 MediaQueueItem[] items = new MediaQueueItem[] { adItem, mediaItem };
 remoteMediaClient.queueLoad(items, 0, MediaStatus.REPEAT_MODE_REPEAT_OFF, customJson);
56. Ads + Queue MediaMetadata adMetadata = new MediaMetadata(MEDIA_TYPE_AD);
 
 MediaInfo adInfo = new MediaInfo.Builder(adId)
 .setContentType("video/mp4")
 .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED)
 .setMetadata(adMetadata)
 .build();
 
 MediaQueueItem adItem= new MediaQueueItem.Builder(adInfo).build();
 MediaQueueItem mediaItem = new MediaQueueItem.Builder(mediaInfo).build();
 
 MediaQueueItem[] items = new MediaQueueItem[] { adItem, mediaItem };
 remoteMediaClient.queueLoad(items, 0, MediaStatus.REPEAT_MODE_REPEAT_OFF, customJson);
57. Ads + Queue private class RemoteMediaClientListener implements RemoteMediaClient.Listener {
 
 @Override
 public void onMetadataUpdated() {
 MediaInfo info = remoteMediaClient.getMediaInfo();
 
 if (info.getMetadata().getMediaType() == MEDIA_TYPE_AD) {
 hideSeekBar(); disablePause();
 }
 // ...
 } // ...
58. Ads • Receiver: • Build Queue • Retrieve Ad URL • Receiver adjusts playback rate
59. Auth • Keep Senders Authorizing Content • Receiver Authentication • Cookies • Custom Headers • Query Parameters • Use load command custom data to pass auth token
60. Analytics • Dev Console • Device and session count, avg. playback time • Country, device type
61. Analytics • Sender • Encountered Receiver • Cast UI Interaction • Receiver • X per Session • Media Events • SDKs • JavaScript • Media Element
62. Evaluating Content • Supported Format • Content Origin • Resolution • Orientation Normalized • Use Demo Apps
63. Guest Mode • Android and iOS (2nd Gen.) • Audio Tokens • PIN • Bluetooth LE (2nd Gen.) • “Nearby Device” Route • Transparent • Cloud Relay
64. Audio Devices • Same API • Opt-In Dev Console // sender CastDevice castDevice = castSession.getCastDevice();
 castDevice.hasCapability(CastDevice.CAPABILITY_VIDEO_OUT);
 castDevice.hasCapability(CastDevice.CAPABILITY_AUDIO_OUT); castDevice.hasCapability(CastDevice.CAPABILITY_VIDEO_IN); // receiver, Chromecast Audio castReceiverManager.getDeviceCapabilities() Object { bluetooth_supported: true, display_supported: false, hi_res_audio_supported: true } // server request header CAST-DEVICE-CAPABILITIES: {“display_supported": false}
65. Debugging • chrome://inspect • cast.receiver.logger.setLevelValue(cast.receiver.LoggerLevel.DEBUG) • cast.player.api.setLoggerLevel(cast.player.api.LoggerLevel.DEBUG) • location.reload(true); • Chromecast reboot, often memory issue • Crash Logs: https://support.google.com/cast-developer/contact/google_cast_contact_us • Whitelist • Unable to Debug • No Devices
66. Happy Casting Developer Documentation https://developers.google.com/cast/ Google Cast Examples https://github.com/googlecast/ Google Cast Dev Community http://goo.gl/TPLDxj Google I/O https://www.youtube.com/watch?v=Ij9xM4Velno CORS Tips on enabling and testing http://enable-cors.org/ CORS Proxy for dev: https://www.npmjs.com/package/corsproxy Media Tools: ffprobe, exiftool, mediastreamvalidator (apple dev tool) Caleb Smith @co_obec