Implementing Audio on Android

Transcript

1. Implementing Audio for Android Caren Chang
2. Motivation
3. Motivation Creating a more interactive experience
4. Motivation Creating a more interactive experience Allow for personalization
5. Motivation Creating a more interactive experience Allow for personalization Guide users
6. How is Audio Processed?
7. How is Audio Processed? Latency - Time delay as a signal passes through a system
8. Latency - Time delay as a signal passes through a system How is Audio Processed?
9. Latency - Time delay as a signal passes through a system How is Audio Processed?
10. Latency - Time delay as a signal passes through a system How is Audio Processed?
11. How is Audio Processed? Application App audio source
12. How is Audio Processed? Application Audio Framework App audio source Resample Effects Mixer
13. How is Audio Processed? Application Audio Framework Device Hardware App audio source Resample Effects Mixer DAC
14. How is Audio Processed? Audio hardware consumes data at constant rate
15. How is Audio Processed? Audio hardware consumes data at constant rate When we pass audio files to be played, we have to make sure it gets processed in time to meet the deadlines
16. How is Audio Processed? Audio hardware consumes data at constant rate When we pass audio files to be played, we have to make sure it gets processed in time to meet the deadlines Otherwise, when deadlines are missed, you can get silent sounds or weird ‘static’ sounds
17. How is Audio Processed?
18. How is Audio Processed?
19. How is Audio Processed?
20. How is Audio Processed?
21. How is Audio Processed?
22. Implementing Audio Playback
23. Implementing Audio Playback MediaPlayer SoundPool ExoPlayer
24. Implementing Audio Playback MediaPlayer SoundPool ExoPlayer
25. MediaPlayer
26. MediaPlayer Easily fetch, decode and play audio with minimal setup
27. MediaPlayer Easily fetch, decode and play audio with minimal setup Supports :
 Local resources
 Internal URIs (from content resolver)
 External URLs (http streaming)
28. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();
 try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
29. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();
 try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
30. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();
 try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
31. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();
 try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
32. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();
 try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
33. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();
 try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
34. MediaPlayer void playAudioFile(String audioFilePath) {
 MediaPlayer mediaPlayer = new MediaPlayer();
 try {
 mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start();
 } catch (IOException e) {
 Log.e(TAG, "Error playing audio file: " + audioFilePath);
 mediaPlayer.release();
 }
 }
35. MediaPlayer
36. MediaPlayer MediaPlayer is state based, can easily caused undesired behaviors
37. MediaPlayer MediaPlayer is state based, can easily caused undesired behaviors mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start(); or mediaPlayer.pause() or mediaPlayer.seekTo(); IDLE
38. MediaPlayer MediaPlayer is state based, can easily caused undesired behaviors mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start(); or mediaPlayer.pause() or mediaPlayer.seekTo(); INITIALIZED
39. MediaPlayer MediaPlayer is state based, can easily caused undesired behaviors mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start(); or mediaPlayer.pause() or mediaPlayer.seekTo(); PREPARED
40. MediaPlayer MediaPlayer is state based, can easily caused undesired behaviors mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
 mediaPlayer.setDataSource(audioFilePath);
 mediaPlayer.prepare();
 mediaPlayer.start(); or mediaPlayer.pause() or mediaPlayer.seekTo();
41. MediaPlayer : Common Errors Not handling onError
42. MediaPlayer : Common Errors Not handling onError Performing actions in wrong state
43. MediaPlayer : Common Errors Not handling onError Performing actions in wrong state Unsupported file types
44. MediaPlayer : Common Errors Not handling onError Performing actions in wrong state Unsupported file types Not releasing after use
46. Implementing Audio Playback MediaPlayer SoundPool ExoPlayer
47. SoundPool
48. SoundPool Uses the MediaPlayer service to decode the audio
49. SoundPool Uses the MediaPlayer service to decode the audio Collection of samples from the APK that can be loaded into memory
50. SoundPool Uses the MediaPlayer service to decode the audio Collection of samples from the APK that can be loaded into memory Able to manage number of audio streams
 IE : able to do sound mixing
51. SoundPool Uses the MediaPlayer service to decode the audio Collection of samples from the APK that can be loaded into memory Able to manage number of audio streams
 IE : able to do sound mixing Can set priority when playing sound
52. SoundPool SoundPool mSoundPool = new SoundPool.Builder()
 .setAudioAttributes(mAudioAttributes)
 .setMaxStreams(MAX_SOUND_STREAMS)
 .build();
 
 int loadedSoundId = mSoundPool.load(audioFilePath, 1);
 
 mSoundPool.play(loadedSoundId,
 volumeToPlaySound, volumeToPlaySound,
 REGULAR_PRIORITY_SOUND, NO_LOOP, DEFAULT_PLAYBACK_RATE);
53. SoundPool SoundPool mSoundPool = new SoundPool.Builder()
 .setAudioAttributes(mAudioAttributes)
 .setMaxStreams(MAX_SOUND_STREAMS)
 .build();
 
 int loadedSoundId = mSoundPool.load(audioFilePath, 1);
 
 mSoundPool.play(loadedSoundId,
 volumeToPlaySound, volumeToPlaySound,
 REGULAR_PRIORITY_SOUND, NO_LOOP, DEFAULT_PLAYBACK_RATE);
54. SoundPool SoundPool mSoundPool = new SoundPool.Builder()
 .setAudioAttributes(mAudioAttributes)
 .setMaxStreams(MAX_SOUND_STREAMS)
 .build();
 
 int loadedSoundId = mSoundPool.load(audioFilePath, 1);
 
 mSoundPool.play(loadedSoundId,
 volumeToPlaySound, volumeToPlaySound,
 REGULAR_PRIORITY_SOUND, NO_LOOP, DEFAULT_PLAYBACK_RATE);
55. AudioService.java
56. Implementing Audio Playback MediaPlayer SoundPool ExoPlayer
57. ExoPlayer
58. ExoPlayer Open source project (not part of Android framework)
59. ExoPlayer Open source project (not part of Android framework) Pros:
 Ability to update player along with application
60. ExoPlayer Open source project (not part of Android framework) Pros:
 Ability to update player along with application Fewer device specific issues across different versions of Android
61. ExoPlayer Open source project (not part of Android framework) Pros:
 Ability to update player along with application Fewer device specific issues across different versions of Android Ability to customize and extend the player
62. ExoPlayer : Talks ExoPlayer : Flexible Media Playback for Android
 Google IO ‘17 Advanced Exoplayer
 Effie Barak
63. Being a good (App) Citizen
64. AudioFocus
65. AudioFocus Audio from different apps can play simultaneously (system does not have control over which streams play)
66. AudioFocus Audio from different apps can play simultaneously (system does not have control over which streams play) If individual apps don’t properly handle audio focus it will result in a bad user experience
67. AudioFocus : Scenario
68. AudioFocus : Scenario User is currently in your app and playing audio
69. AudioFocus : Scenario User is currently in your app and playing audio User goes into another media app that plays audio
70. AudioFocus : Scenario User is currently in your app and playing audio User goes into another media app that plays audio If not properly handled, both apps will now play their audio stream and nothing will be heard
71. AudioFocus : Proper Flow
72. AudioFocus : Proper Flow Request audio focus
73. AudioFocus : Proper Flow Request audio focus Start playback after audio focus is successfully granted
74. AudioFocus : Proper Flow Request audio focus Start playback after audio focus is successfully granted Handle audio focus changes through onAudioFocusChangedListener()
75. AudioFocus : Intentions
76. AudioFocus : Intentions AUDIOFOCUS_GAIN : Indefinite AUDIOFOCUS_TRANSIENT : temporary (phone call) AUDIOFOCUS_TRANSIENT_MAY_DUCK : temporary (notification)
77. AudioFocus : Implementation AudioFocusRequest mAudioFocusRequest =
 new AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN)
 .setAudioAttributes(mAudioAttributes)
 .setAcceptsDelayedFocusGain(true)
 .setOnAudioFocusChangeListener(...)
 .build();
 
 int focusRequest = mAudioManager.requestAudioFocus(mAudioFocusRequest);
78. AudioFocus : Implementation AudioFocusRequest mAudioFocusRequest =
 new AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN)
 .setAudioAttributes(mAudioAttributes)
 .setAcceptsDelayedFocusGain(true)
 .setOnAudioFocusChangeListener(...)
 .build();
 
 int focusRequest = mAudioManager.requestAudioFocus(mAudioFocusRequest);
79. AudioFocus : Implementation AudioFocusRequest mAudioFocusRequest =
 new AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN)
 .setAudioAttributes(mAudioAttributes)
 .setAcceptsDelayedFocusGain(true)
 .setOnAudioFocusChangeListener(...)
 .build();
 
 int focusRequest = mAudioManager.requestAudioFocus(mAudioFocusRequest);
80. AudioFocus : Implementation AudioFocusRequest mAudioFocusRequest =
 new AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN)
 .setAudioAttributes(mAudioAttributes)
 .setAcceptsDelayedFocusGain(true)
 .setOnAudioFocusChangeListener(...)
 .build();
 
 int focusRequest = mAudioManager.requestAudioFocus(mAudioFocusRequest);

81. Sounds on the June Oven
82. Sounds on the June Oven Common sounds vs Special sounds
83. Sounds on the June Oven Common sounds vs Special sounds Themes
84. Sounds on the June Oven Common sounds vs Special sounds Themes Emergency sounds
85. Sounds on the June Oven Common sounds (tap, cancel, accept, back) are eagerly loaded into a SoundPool for low latency playback.
86. Sounds on the June Oven Common sounds (tap, cancel, accept, back) are eagerly loaded into a SoundPool for low latency playback. All other sounds are played with the MediaPlayer
87. Sounds on the June Oven Common sounds (tap, cancel, accept, back) are eagerly loaded into a SoundPool for low latency playback. All other sounds are played with the MediaPlayer Sounds can request to be played immediately
88. Problems Encountered
89. Problems Encountered Creating pleasant experience on sliders
90. Problems Encountered Creating pleasant experience on sliders ‘Wake’ sound
91. Problems Encountered Creating pleasant experience on sliders ‘Wake’ sound Playing sound on theme changed
92. Thanks! @calren24