Making Rich Media Smarter

Slide 1

Slide 1 text

Making Rich Media Smart & Faster PROSPER OTEMUYIWA | AngularNYC - October 31, 2017

Slide 2

Slide 2 text

Google Developer Expert @unicodeveloper

Slide 3

Slide 3 text

Community Evangelist ❖ forLoop Africa ❖ Laravel Nigeria ❖ Angular Nigeria ❖ Webpack Africa

Slide 4

Slide 4 text

User Experiences Across The World With Media on the Web You don Know! - Big Shaq

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

User Experience: Loading Video on a Slow 3G Network

Slide 7

Slide 7 text

User Exp: Watching Man’s Not Hot & Device goes Offline

Slide 8

Slide 8 text

User Exp: Accessing an Image on second load & it doesn’t come up instantly.

Slide 9

Slide 9 text

“After 2 seconds of buffering, users start dropping off at around 6% per second.”

Slide 10

Slide 10 text

Rich Media Images Audio Video 62% of Internet Traffic is made up of images, with audio & video constituting a growing percentage of the bytes.

Slide 11

Slide 11 text

Anatomy of a Rich Media Experience?

Slide 12

Slide 12 text

Anatomy of a Rich Media Experience ❖ Fast Playback ❖ Great UX ❖ Delivery Of High Quality Media Assets ❖ Accessibility anywhere

Slide 13

Slide 13 text

Case Studies - Rich Media Experience URL: https://www.iheart.com iHeartRadio Audio

Slide 14

Slide 14 text

Case Studies - Rich Media Experience ❖ Progressive Web App ❖ Media Session API Usage URL: https://www.iheart.com

Slide 15

Slide 15 text

Case Studies - Rich Media Experience JioCinema URL: https://www.jiocinema.com Video

Slide 16

Slide 16 text

Case Studies - Rich Media Experience URL: https://www.jiocinema.com ❖ Progressive Web App ❖ Fast Playback ❖ Play Videos Offline ❖ Media Session API Usage

Slide 17

Slide 17 text

Case Studies - Rich Media Experience iFixit URL: https://ifixit-pwa.appspot.com Images

Slide 18

Slide 18 text

Case Studies - Rich Media Experience URL: https://ifixit-pwa.appspot.com ❖ Progressive Web App ❖ Offline-first

Slide 19

Slide 19 text

“Making Rich Media Smart & Fast…...Your turn!”

Slide 20

Slide 20 text

Woot! Woot! Yaaaaaaaaaaasss!

Slide 21

Slide 21 text

Recommended Video Players ❖ Google Shaka Player - bit.ly/shaka-offline-tutorial ➢ OSS ➢ Custom subtitle rendering ➢ Adaptive bitrate streaming ➢ Playlists ❖ Cloudinary Video Player - http://bit.ly/2iJs7X7 ➢ OSS ➢ Adaptive bitrate streaming ➢ Playlists ➢ Analytics

Slide 22

Slide 22 text

Fast Playback With Adaptive Bitrate Streaming ❖ Videos start quicker ❖ Multiple video streams with different resolutions, qualities and bitrate ❖ Video player loads the optimal stream based on user’s internet connectivity. ❖ Cloudinary’s Plug-and-play adaptive bitrate streaming with HLS & Mpeg-dash works well.

Slide 23

Slide 23 text

Fast Playback With Video Preload ❖ Faster Playback start means more people watching your video. Setting the preload attribute to auto indicates that the browser may cache enough data that complete playback is possible without requiring a stop for further buffering.

Slide 24

Slide 24 text

Fast Playback With Video Preload ❖ Link Preload Link preload is a declarative fetch that allows you to force the browser to make a request for a resource without blocking the window.onload event and while the page is downloading.

Slide 25

Slide 25 text

Fast Playback With Video Preload ❖ Preload Full Video on your website Recommended for smaller files <= 5MB

Slide 26

Slide 26 text

Smart Video Preload Considerations ❖ Check user's Battery Level

Slide 27

Slide 27 text

Smart Video Preload Considerations ❖ Check Network type & save bandwidth User’s Battery Level

Slide 28

Slide 28 text

Smart Video Preload Considerations ❖ Estimate how much space is left on the users device

Slide 29

Slide 29 text

“Great UX...”

Slide 30

Slide 30 text

Screen Orientation API “As users rotate their device, be smart and automatically request for full screen on the video to create an immersive experience. ”

Slide 31

Slide 31 text

Screen Orientation API

Slide 32

Slide 32 text

Screen Orientation API

Slide 33

Slide 33 text

Background Playback & Page Visibility Pause video when page is hidden. This happens when screen lock is active or when you switch tabs. Page Visibility API makes that possible Chrome for Android already pauses videos when page is hidden

Slide 34

Slide 34 text

Intersection Observer API Show/Hide a mute button based on video visibility on the page.

Slide 35

Slide 35 text

Media Session API ” With the Media Session API, you can also customize media notifications by providing metadata for the currently playing video. It also allows you to handle media related events such as seeking or track changing which may come from notifications or media keys” Without Media Session With Media Session

Slide 36

Slide 36 text

Media Session API

Slide 37

Slide 37 text

Media Session API

Slide 38

Slide 38 text

Media Session API “If your web app provides a playlist, you may want to allow the user to navigate through your playlist directly from the media notification with some "Previous Track" and "Next Track" icons”.

Slide 39

Slide 39 text

Image & Video Transformations ” Manipulations and transformations you might want to apply to your media elements”

Slide 40

Slide 40 text

Image & Video Transformations ” Manipulations and transformations you might want to apply to your media elements”

Slide 41

Slide 41 text

Image & Video Transformations

Slide 42

Slide 42 text

Image & Video Transformations

Slide 43

Slide 43 text

Image & Video Transformations ” Concatenate video with images”

Slide 44

Slide 44 text

Image & Video Transformations ” Generating video thumbnails”

Slide 45

Slide 45 text

Image & Video Transformations ” Generating Animated GIFs”

Slide 46

Slide 46 text

Face Detection & Overlay ” Face Detection & Overlay” https://cloudydesk.com/face/

Slide 47

Slide 47 text

Image & Video Transformations https://github.com/cloudinary/cloudinary_angular ”Your Reference Guide & Transformation Overlord is in this Project”

Slide 48

Slide 48 text

“Delivering High Quality Media assets to Users? How?”

Slide 49

Slide 49 text

Fast Delivery “Deliver your media assets via CDNs”. CDNs offer: ❖ Decrease in server load. ❖ Lower Network latency and packet loss. ❖ Higher Availability. ❖ Storage and security. Akamai Fastly Cloudinary - They all offer CDN delivery

Slide 50

Slide 50 text

“Accessibility everywhere - Offline? Yeah”

Slide 51

Slide 51 text

Offline “You can cache all your media assets and serve them offline with:” JavaScript libraries for Offline Caching https://workboxjs.org/

Slide 52

Slide 52 text

Background Sync “Background sync lets you defer actions until the user has stable connectivity. This is useful for ensuring that whatever the user wants to send, is actually sent.” In summary, messages are sent and retrieved in the background. When a user navigates away, goes offline or closes the tab, background sync allows actions that have been initiated to still get completed when the user comes online such as chat messages, photo uploads, updates to documents, e.t.c

Slide 53

Slide 53 text

Background Sync “Background sync lets you defer actions until the user has stable connectivity. This is useful for ensuring that whatever the user wants to send, is actually sent.”

Slide 54

Slide 54 text

Background Fetch ”Provides a service worker based upload/download mechanism which is persistent across service worker and browser restarts. It provides an interface to query, pause, and cancel background downloads/uploads” Sample Implementation ❖ http://bit.ly/2zf2EvQ Very experimental & still in development http://bit.ly/2zNMeGY