Implementing Audio on Android

7bf2baf0d8d5d7d96e6b67c5d567e3f9?s=47 Caren
November 05, 2017
55

Implementing Audio on Android

7bf2baf0d8d5d7d96e6b67c5d567e3f9?s=128

Caren

November 05, 2017
Tweet

Transcript

  1. Implementing Audio for Android Caren Chang

  2. Motivation

  3. Motivation Creating a more interactive experience

  4. Motivation Creating a more interactive experience Allow for personalization

  5. Motivation Creating a more interactive experience Allow for personalization Guide

    users
  6. How is Audio Processed?

  7. How is Audio Processed? Latency - Time delay as a

    signal passes through a system
  8. Latency - Time delay as a signal passes through a

    system How is Audio Processed?
  9. Latency - Time delay as a signal passes through a

    system How is Audio Processed?
  10. Latency - Time delay as a signal passes through a

    system How is Audio Processed?
  11. How is Audio Processed? Application App audio source

  12. How is Audio Processed? Application Audio Framework App audio source

    Resample Effects Mixer
  13. How is Audio Processed? Application Audio Framework Device Hardware App

    audio source Resample Effects Mixer DAC
  14. How is Audio Processed? Audio hardware consumes data at constant

    rate
  15. How is Audio Processed? Audio hardware consumes data at constant

    rate When we pass audio files to be played, we have to make sure it gets processed in time to meet the deadlines
  16. How is Audio Processed? Audio hardware consumes data at constant

    rate When we pass audio files to be played, we have to make sure it gets processed in time to meet the deadlines Otherwise, when deadlines are missed, you can get silent sounds or weird ‘static’ sounds
  17. How is Audio Processed?

  18. How is Audio Processed?

  19. How is Audio Processed?

  20. How is Audio Processed?

  21. How is Audio Processed?

  22. Implementing Audio Playback

  23. Implementing Audio Playback MediaPlayer SoundPool ExoPlayer

  24. Implementing Audio Playback MediaPlayer SoundPool ExoPlayer

  25. MediaPlayer

  26. MediaPlayer Easily fetch, decode and play audio with minimal setup

  27. MediaPlayer Easily fetch, decode and play audio with minimal setup

    Supports :
 Local resources
 Internal URIs (from content resolver)
 External URLs (http streaming)
  28. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();


    try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
  29. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();


    try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
  30. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();


    try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
  31. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();


    try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
  32. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();


    try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
  33. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();


    try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
  34. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();


    try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
  35. MediaPlayer

  36. MediaPlayer MediaPlayer is state based, can easily caused undesired behaviors

  37. MediaPlayer MediaPlayer is state based, can easily caused undesired behaviors

    mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start(); or mediaPlayer.pause() or mediaPlayer.seekTo(); IDLE
  38. MediaPlayer MediaPlayer is state based, can easily caused undesired behaviors

    mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start(); or mediaPlayer.pause() or mediaPlayer.seekTo(); INITIALIZED
  39. MediaPlayer MediaPlayer is state based, can easily caused undesired behaviors

    mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start(); or mediaPlayer.pause() or mediaPlayer.seekTo(); PREPARED
  40. MediaPlayer MediaPlayer is state based, can easily caused undesired behaviors

    mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start(); or mediaPlayer.pause() or mediaPlayer.seekTo();
  41. MediaPlayer : Common Errors Not handling onError

  42. MediaPlayer : Common Errors Not handling onError Performing actions in

    wrong state
  43. MediaPlayer : Common Errors Not handling onError Performing actions in

    wrong state Unsupported file types
  44. MediaPlayer : Common Errors Not handling onError Performing actions in

    wrong state Unsupported file types Not releasing after use
  45. None
  46. Implementing Audio Playback MediaPlayer SoundPool ExoPlayer

  47. SoundPool

  48. SoundPool Uses the MediaPlayer service to decode the audio

  49. SoundPool Uses the MediaPlayer service to decode the audio Collection

    of samples from the APK that can be loaded into memory
  50. SoundPool Uses the MediaPlayer service to decode the audio Collection

    of samples from the APK that can be loaded into memory Able to manage number of audio streams
 IE : able to do sound mixing
  51. SoundPool Uses the MediaPlayer service to decode the audio Collection

    of samples from the APK that can be loaded into memory Able to manage number of audio streams
 IE : able to do sound mixing Can set priority when playing sound
  52. SoundPool SoundPool mSoundPool = new SoundPool.Builder()
 .setAudioAttributes(mAudioAttributes)
 .setMaxStreams(MAX_SOUND_STREAMS)
 .build();
 


    int loadedSoundId = mSoundPool.load(audioFilePath, 1);
 
 mSoundPool.play(loadedSoundId,
 volumeToPlaySound, volumeToPlaySound,
 REGULAR_PRIORITY_SOUND, NO_LOOP, DEFAULT_PLAYBACK_RATE);
  53. SoundPool SoundPool mSoundPool = new SoundPool.Builder()
 .setAudioAttributes(mAudioAttributes)
 .setMaxStreams(MAX_SOUND_STREAMS)
 .build();
 


    int loadedSoundId = mSoundPool.load(audioFilePath, 1);
 
 mSoundPool.play(loadedSoundId,
 volumeToPlaySound, volumeToPlaySound,
 REGULAR_PRIORITY_SOUND, NO_LOOP, DEFAULT_PLAYBACK_RATE);
  54. SoundPool SoundPool mSoundPool = new SoundPool.Builder()
 .setAudioAttributes(mAudioAttributes)
 .setMaxStreams(MAX_SOUND_STREAMS)
 .build();
 


    int loadedSoundId = mSoundPool.load(audioFilePath, 1);
 
 mSoundPool.play(loadedSoundId,
 volumeToPlaySound, volumeToPlaySound,
 REGULAR_PRIORITY_SOUND, NO_LOOP, DEFAULT_PLAYBACK_RATE);
  55. AudioService.java

  56. Implementing Audio Playback MediaPlayer SoundPool ExoPlayer

  57. ExoPlayer

  58. ExoPlayer Open source project (not part of Android framework)

  59. ExoPlayer Open source project (not part of Android framework) Pros:


    Ability to update player along with application
  60. ExoPlayer Open source project (not part of Android framework) Pros:


    Ability to update player along with application Fewer device specific issues across different versions of Android
  61. ExoPlayer Open source project (not part of Android framework) Pros:


    Ability to update player along with application Fewer device specific issues across different versions of Android Ability to customize and extend the player
  62. ExoPlayer : Talks ExoPlayer : Flexible Media Playback for Android


    Google IO ‘17 Advanced Exoplayer
 Effie Barak
  63. Being a good (App) Citizen

  64. AudioFocus

  65. AudioFocus Audio from different apps can play simultaneously (system does

    not have control over which streams play)
  66. AudioFocus Audio from different apps can play simultaneously (system does

    not have control over which streams play) If individual apps don’t properly handle audio focus it will result in a bad user experience
  67. AudioFocus : Scenario

  68. AudioFocus : Scenario User is currently in your app and

    playing audio
  69. AudioFocus : Scenario User is currently in your app and

    playing audio User goes into another media app that plays audio
  70. AudioFocus : Scenario User is currently in your app and

    playing audio User goes into another media app that plays audio If not properly handled, both apps will now play their audio stream and nothing will be heard
  71. AudioFocus : Proper Flow

  72. AudioFocus : Proper Flow Request audio focus

  73. AudioFocus : Proper Flow Request audio focus Start playback after

    audio focus is successfully granted
  74. AudioFocus : Proper Flow Request audio focus Start playback after

    audio focus is successfully granted Handle audio focus changes through onAudioFocusChangedListener()
  75. AudioFocus : Intentions

  76. AudioFocus : Intentions AUDIOFOCUS_GAIN : Indefinite AUDIOFOCUS_TRANSIENT : temporary (phone

    call) AUDIOFOCUS_TRANSIENT_MAY_DUCK : temporary (notification)
  77. AudioFocus : Implementation AudioFocusRequest mAudioFocusRequest =
 new AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN)
 .setAudioAttributes(mAudioAttributes)
 .setAcceptsDelayedFocusGain(true)


    .setOnAudioFocusChangeListener(...)
 .build();
 
 int focusRequest = mAudioManager.requestAudioFocus(mAudioFocusRequest);
  78. AudioFocus : Implementation AudioFocusRequest mAudioFocusRequest =
 new AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN)
 .setAudioAttributes(mAudioAttributes)
 .setAcceptsDelayedFocusGain(true)


    .setOnAudioFocusChangeListener(...)
 .build();
 
 int focusRequest = mAudioManager.requestAudioFocus(mAudioFocusRequest);
  79. AudioFocus : Implementation AudioFocusRequest mAudioFocusRequest =
 new AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN)
 .setAudioAttributes(mAudioAttributes)
 .setAcceptsDelayedFocusGain(true)


    .setOnAudioFocusChangeListener(...)
 .build();
 
 int focusRequest = mAudioManager.requestAudioFocus(mAudioFocusRequest);
  80. AudioFocus : Implementation AudioFocusRequest mAudioFocusRequest =
 new AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN)
 .setAudioAttributes(mAudioAttributes)
 .setAcceptsDelayedFocusGain(true)


    .setOnAudioFocusChangeListener(...)
 .build();
 
 int focusRequest = mAudioManager.requestAudioFocus(mAudioFocusRequest);

  81. Sounds on the June Oven

  82. Sounds on the June Oven Common sounds vs Special sounds

  83. Sounds on the June Oven Common sounds vs Special sounds

    Themes
  84. Sounds on the June Oven Common sounds vs Special sounds

    Themes Emergency sounds
  85. Sounds on the June Oven Common sounds (tap, cancel, accept,

    back) are eagerly loaded into a SoundPool for low latency playback.
  86. Sounds on the June Oven Common sounds (tap, cancel, accept,

    back) are eagerly loaded into a SoundPool for low latency playback. All other sounds are played with the MediaPlayer
  87. Sounds on the June Oven Common sounds (tap, cancel, accept,

    back) are eagerly loaded into a SoundPool for low latency playback. All other sounds are played with the MediaPlayer Sounds can request to be played immediately
  88. Problems Encountered

  89. Problems Encountered Creating pleasant experience on sliders

  90. Problems Encountered Creating pleasant experience on sliders ‘Wake’ sound

  91. Problems Encountered Creating pleasant experience on sliders ‘Wake’ sound Playing

    sound on theme changed
  92. Thanks! @calren24