Slide 1

Slide 1 text

AVFoundation & Audio Capture Can you hear me? Ronan Rodrigo Nunes 
 iOS Software Engineer @ Uber 
 ronanrodrigo.dev

Slide 2

Slide 2 text

Straight forward implementation

Slide 3

Slide 3 text

Is simple right? let recordingSession = AVAudioSession.sharedInstance() audioSession.setCategory(.playAndRecord, mode: .default, options: [.mixWithOthers]) recordingSession.setActive(true) recordingSession.requestRecordPermission { (allowed) in audioRecorder = AVAudioRecorder(url: filename, settings: settings) audioRecorder?.record() }

Slide 4

Slide 4 text

But… let recordingSession = AVAudioSession.sharedInstance()

Slide 5

Slide 5 text

I see AVAudioEngine… let audioEngine = AVAudioEngine() let inputNode = audioEngine.inputNode let inputFormat = inputNode.inputFormat(forBus: 0) inputNode.installTap( onBus: 0, bufferSize: size, format: inputFormat, block: handle(buffer:time:) )

Slide 6

Slide 6 text

⚠ Terminating app due to uncaught exception com.apple.coreaudio.avfaudio reason: required condition is false: format.sampleRate == hwFormat.sampleRate

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

I see AVAudioEngine… let audioEngine = AVAudioEngine() let inputNode = audioEngine.inputNode let inputFormat = inputNode.inputFormat(forBus: 0) inputNode.installTap( onBus: 0, bufferSize: size, format: inputFormat, block: handle(buffer:time:) )

Slide 9

Slide 9 text

A route change NotificationCenter.default.publisher(for: AVAudioSession.routeChangeNotification) .compactMap { notification -> AVAudioSession.RouteChangeReason? in //... } .sink(receiveValue: { [weak self] routeChangeReason in switch routeChangeReason { case .unknown: Logger.log("🏴☠ unknown") case .newDeviceAvailable: Logger.log("🏴☠ newDeviceAvailable") case .oldDeviceUnavailable: Logger.log("🏴☠ oldDeviceUnavailable") case .override: Logger.log("🏴☠ override") case .wakeFromSleep: Logger.log("🏴☠ wakeFromSleep") case .noSuitableRouteForCategory: Logger.log("🏴☠ noSuitableRouteForCategory") case .routeConfigurationChange: Logger.log("🏴☠ routeConfigurationChange") case .categoryChange: Logger.log("🏴☠ categoryChange") } })

Slide 10

Slide 10 text

Time to reset the capture audioEngine.inputNode.removeTap(onBus: 0) notificationCancellable?.cancel() notificationCancellable = nil audioEngine.stop() audioEngine = AVAudioEngine() setupAudioSession() startCapture()

Slide 11

Slide 11 text

Seems to be fixed…

Slide 12

Slide 12 text

A route change NotificationCenter.default.publisher(for: AVAudioSession.routeChangeNotification) .compactMap { notification -> AVAudioSession.RouteChangeReason? in //... } .sink(receiveValue: { [weak self] routeChangeReason in switch routeChangeReason { case .unknown: Logger.log("🏴☠ unknown") case .newDeviceAvailable: Logger.log("🏴☠ newDeviceAvailable") case .oldDeviceUnavailable: Logger.log("🏴☠ oldDeviceUnavailable") case .override: Logger.log("🏴☠ override") case .wakeFromSleep: Logger.log("🏴☠ wakeFromSleep") case .noSuitableRouteForCategory: Logger.log("🏴☠ noSuitableRouteForCategory") case .routeConfigurationChange: Logger.log("🏴☠ routeConfigurationChange") case .categoryChange: Logger.log("🏴☠ categoryChange") } })

Slide 13

Slide 13 text

Dumping the Audio Session private func dumpAudioSession() -> String { return """ preferredSampleRate: \(audioSession.preferredSampleRate) preferredInputNumberOfChannels: \(audioSession.preferredInputNumberOfChannels) maximumInputNumberOfChannels: \(audioSession.maximumInputNumberOfChannels) inputNumberOfChannels: \(audioSession.inputNumberOfChannels) isInputAvailable: \(audioSession.isInputAvailable) inputDataSources: \((audioSession.inputDataSources?.map { "\($0.dataSourceName)" })) inputDataSource: \(audioSession.inputDataSource?.dataSourceName) inputSelectedDatasource: \(audioSession.currentRoute.inputs.compactMap { $0.selectedDataSource?.dataSourceName }) sampleRate: \(audioSession.sampleRate) preferredOutputNumberOfChannels: \(audioSession.preferredOutputNumberOfChannels) outputSelectedDatasource: \(audioSession.currentRoute.outputs.compactMap { $0.selectedDataSource?.dataSourceName }) outputDataSources: \(audioSession.outputDataSources) outputDataSource: \(audioSession.outputDataSource) """ }

Slide 14

Slide 14 text

Now restart if dumpAudioSession() == requiredConfig { return } audioEngine.inputNode.removeTap(onBus: 0) notificationCancellable?.cancel() notificationCancellable = nil audioEngine.stop() audioEngine = AVAudioEngine() setupAudioSession() startCapture()

Slide 15

Slide 15 text

A new issue

Slide 16

Slide 16 text

A new issue AVAudioSession 
 is a singleton!

Slide 17

Slide 17 text

Audio resource provider engineProvider .engine(for: consumer) .compactMap { availability -> AudioResourceAudioEngine? in switch availability { case let .available(engine): return engine //... } } .subscribe(onNext: { engine in try? engine.inputNode.installTap { buffer, time in //... } })