Don Turner - Real-time audio with the 100 oscillator synthesizer

Transcript

1. Don Turner Developer Advocate - Android Audio @donturner Real-time audio with the 100 oscillator synthesizer
2. 1 2 3 Demonstrate a new audio API: Oboe Obtain lowest latency Optimise CPU performance To Do DroidCon Berlin - 2018
4. onAudioReady Audio rendering objects control statements Oscillator Mixer AudioStream Oboe library github.com/google/oboe App architecture 3 Synth audio out audio frames tap DroidCon Berlin - 2018
5. Why use Oboe? 1 2 Write less code DroidCon Berlin - 2018
6. static SLresult OpenSLEngine() { SLresult result = SL_RESULT_SUCCESS; // create engine result = slCreateEngine(&sEngineObject, 0, NULL, 0, NULL, NULL); // realize the engine result = (*sEngineObject)->Realize(sEngineObject, SL_BOOLEAN_FALSE); // get the engine interface, which is needed in order to create other objects result = (*sEngineObject)->GetInterface(sEngineObject, SL_IID_ENGINE, &sEngineEngine); // get the output mixer result = (*sEngineEngine)->CreateOutputMix(sEngineEngine, &sOutputMixObject, 0, 0, 0); // realize the output mix result = (*sOutputMixObject)->Realize(sOutputMixObject, SL_BOOLEAN_FALSE); if (mFormat == OBOE_AUDIO_FORMAT_UNSPECIFIED){ mFormat = (__ANDROID_API__ < __ANDROID_API_L__) ? OBOE_AUDIO_FORMAT_PCM_I16 : OBOE_AUDIO_FORMAT_PCM_FLOAT; } if (mFramesPerCallback != OBOE_UNSPECIFIED) { mFramesPerBurst = mFramesPerCallback; } else if (mFramesPerBurst != OBOE_UNSPECIFIED) { // set from defaultFramesPerBurst mFramesPerCallback = mFramesPerBurst; } else { mFramesPerBurst = mFramesPerCallback = DEFAULT_FRAMES_PER_CALLBACK; } mBytesPerCallback = mFramesPerCallback * getBytesPerFrame(); mCallbackBuffer = new uint8_t[mBytesPerCallback]; SLuint32 bitsPerSample = getBytesPerSample() * OBOE_BITS_PER_BYTE; SLDataLocator_AndroidSimpleBufferQueue loc_bufq = { SL_DATALOCATOR_ANDROIDSIMPLEBUFFERQUEUE, // locatorType static_cast<SLuint32>(mBurstsPerBuffer)}; // numBuffers // SLuint32 chanMask = SL_SPEAKER_FRONT_LEFT|SL_SPEAKER_FRONT_RIGHT; SLDataFormat_PCM format_pcm = { SL_DATAFORMAT_PCM, // formatType (SLuint32) mChannelCount, // numChannels (SLuint32) (mSampleRate * OBOE_MILLIS_PER_SECOND), // milliSamplesPerSec bitsPerSample, // bitsPerSample bitsPerSample, // containerSize; (SLuint32) chanCountToChanMask(mChannelCount), // channelMask s_getDefaultByteOrder(), }; SLDataSource audioSrc = {&loc_bufq, &format_pcm}; // configure audio sink SLDataLocator_OutputMix loc_outmix = {SL_DATALOCATOR_OUTPUTMIX, sOutputMixObject}; SLDataSink audioSnk = {&loc_outmix, NULL}; const SLInterfaceID ids[] = {SL_IID_BUFFERQUEUE}; const SLboolean req[] = {SL_BOOLEAN_TRUE}; // The Player result = (*sEngineEngine)->CreateAudioPlayer(sEngineEngine, &bqPlayerObject_ , &audioSrc, &audioSnk, sizeof(ids) / sizeof(ids[0]), ids, req result = (*bqPlayerObject_)->Realize(bqPlayerObject_, SL_BOOLEAN_FALSE); result = (*bqPlayerObject_)->GetInterface(bqPlayerObject_, SL_IID_PLAY, &bqPlayerPlay_); // The BufferQueue result = (*bqPlayerObject_)->GetInterface(bqPlayerObject_, SL_IID_ANDROIDSIMPLEBUFFERQUEUE, &bq_); // The register BufferQueue callback result = (*bq_)->RegisterCallback(bq_, bqPlayerCallback, this); mSharingMode = OBOE_SHARING_MODE_SHARED; mBufferCapacityInFrames = mFramesPerBurst * mBurstsPerBuffer; return OBOE_OK; } Open a stream using OpenSL ES 60+ lines of code DroidCon Berlin - 2018
7. Open a stream using Oboe AudioStreamBuilder builder; builder.setFormat(AudioFormat::Float); builder.setSampleRate(48000); builder.setChannelCount(2); AudioStream *stream = nullptr; Result result = builder.openStream(&stream); 6 lines of code Optional DroidCon Berlin - 2018
8. Why use Oboe? 1 2 Write less code Open source DroidCon Berlin - 2018
9. 1 2 github.com/google/oboe DroidCon Berlin - 2018
10. Why use Oboe? 1 2 Write less code Open source Works with 99.2% of Android devices DroidCon Berlin - 2018
11. App Oboe AAudio OpenSL ES Under the hood API 26+ (Oreo) API 16+ (Jelly Bean) DroidCon Berlin - 2018
12. Why use Oboe? 1 2 Write less code Open source Works with 99.2% of Android devices Designed for high performance DroidCon Berlin - 2018
13. Live Coding ¯\_(ツ)_/¯
14. 1 2 3 Demonstrate Oboe API Obtain lowest latency Optimise CPU performance To Do DroidCon Berlin - 2018
15. 0ms App sends sine wave into audio stream ??ms sine wave is played through speaker or headphones Optimizing output latency Minimize this DroidCon Berlin - 2018
16. AudioStream performance modes None - Default, not optimized 1 2 Power Saving - Optimized for battery life 3 Low Latency - Optimized for latency DroidCon Berlin - 2018
17. AudioStream sharing modes Your app audio stream Audio Device Mixer Other audio streams Shared Exclusive DroidCon Berlin - 2018
18. AudioStream buffer sizes Burst Burst Burst Burst App Burst Audio device AudioStream Buffer Size = latency onAudioReady DroidCon Berlin - 2018
19. AudioStream buffer sizes App Audio device Burst Buffer Burst Burst Buffer size can also be dynamically tuned on API 26+ DroidCon Berlin - 2018
20. Live Coding ¯\_(ツ)_/¯
21. Android Pro Audio Devices 1 2 3 Round-trip audio latency of under 20ms hardware feature flag: android.hardware.audio.pro DroidCon Berlin - 2018
22. 1 2 3 Demonstrate Oboe API Obtain lowest latency Optimise CPU performance To Do DroidCon Berlin - 2018
24. Audio rendering objects Mixer Oscillator Playing multiple sounds: Mixing Oscillators 1-100 audio out AudioStream Oboe library github.com/google/oboe audio frames
25. Live Coding ¯\_(ツ)_/¯
26. 1 2 3 Demonstrate a new audio API: Oboe Obtain lowest latency Optimise CPU performance Recap DroidCon Berlin - 2018
27. DroidCon Berlin - 2018 Download Oboe and start building amazing audio apps with it today!
28. 1 3 But wait…. DroidCon Berlin - 2018 JOIN US Search: Developer Programs Engineer, Android Framework
29. Thanks github.com/google/oboe @donturner