Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Audio Capture

Audio Capture

A talk about the details to create an app which records audio

Ronan Rodrigo Nunes

August 24, 2021
Tweet

More Decks by Ronan Rodrigo Nunes

Other Decks in Programming

Transcript

  1. AVFoundation & Audio Capture Can you hear me? Ronan Rodrigo

    Nunes 
 iOS Software Engineer @ Uber 
 ronanrodrigo.dev
  2. Straight forward implementation

  3. Is simple right? let recordingSession = AVAudioSession.sharedInstance() audioSession.setCategory(.playAndRecord, mode: .default,

    options: [.mixWithOthers]) recordingSession.setActive(true) recordingSession.requestRecordPermission { (allowed) in audioRecorder = AVAudioRecorder(url: filename, settings: settings) audioRecorder?.record() }
  4. But… let recordingSession = AVAudioSession.sharedInstance()

  5. I see AVAudioEngine… let audioEngine = AVAudioEngine() let inputNode =

    audioEngine.inputNode let inputFormat = inputNode.inputFormat(forBus: 0) inputNode.installTap( onBus: 0, bufferSize: size, format: inputFormat, block: handle(buffer:time:) )
  6. ⚠ Terminating app due to uncaught exception com.apple.coreaudio.avfaudio reason: required

    condition is false: format.sampleRate == hwFormat.sampleRate
  7. None
  8. I see AVAudioEngine… let audioEngine = AVAudioEngine() let inputNode =

    audioEngine.inputNode let inputFormat = inputNode.inputFormat(forBus: 0) inputNode.installTap( onBus: 0, bufferSize: size, format: inputFormat, block: handle(buffer:time:) )
  9. A route change NotificationCenter.default.publisher(for: AVAudioSession.routeChangeNotification) .compactMap { notification -> AVAudioSession.RouteChangeReason?

    in //... } .sink(receiveValue: { [weak self] routeChangeReason in switch routeChangeReason { case .unknown: Logger.log("🏴☠ unknown") case .newDeviceAvailable: Logger.log("🏴☠ newDeviceAvailable") case .oldDeviceUnavailable: Logger.log("🏴☠ oldDeviceUnavailable") case .override: Logger.log("🏴☠ override") case .wakeFromSleep: Logger.log("🏴☠ wakeFromSleep") case .noSuitableRouteForCategory: Logger.log("🏴☠ noSuitableRouteForCategory") case .routeConfigurationChange: Logger.log("🏴☠ routeConfigurationChange") case .categoryChange: Logger.log("🏴☠ categoryChange") } })
  10. Time to reset the capture audioEngine.inputNode.removeTap(onBus: 0) notificationCancellable?.cancel() notificationCancellable =

    nil audioEngine.stop() audioEngine = AVAudioEngine() setupAudioSession() startCapture()
  11. Seems to be fixed…

  12. A route change NotificationCenter.default.publisher(for: AVAudioSession.routeChangeNotification) .compactMap { notification -> AVAudioSession.RouteChangeReason?

    in //... } .sink(receiveValue: { [weak self] routeChangeReason in switch routeChangeReason { case .unknown: Logger.log("🏴☠ unknown") case .newDeviceAvailable: Logger.log("🏴☠ newDeviceAvailable") case .oldDeviceUnavailable: Logger.log("🏴☠ oldDeviceUnavailable") case .override: Logger.log("🏴☠ override") case .wakeFromSleep: Logger.log("🏴☠ wakeFromSleep") case .noSuitableRouteForCategory: Logger.log("🏴☠ noSuitableRouteForCategory") case .routeConfigurationChange: Logger.log("🏴☠ routeConfigurationChange") case .categoryChange: Logger.log("🏴☠ categoryChange") } })
  13. Dumping the Audio Session private func dumpAudioSession() -> String {

    return """ preferredSampleRate: \(audioSession.preferredSampleRate) preferredInputNumberOfChannels: \(audioSession.preferredInputNumberOfChannels) maximumInputNumberOfChannels: \(audioSession.maximumInputNumberOfChannels) inputNumberOfChannels: \(audioSession.inputNumberOfChannels) isInputAvailable: \(audioSession.isInputAvailable) inputDataSources: \((audioSession.inputDataSources?.map { "\($0.dataSourceName)" })) inputDataSource: \(audioSession.inputDataSource?.dataSourceName) inputSelectedDatasource: \(audioSession.currentRoute.inputs.compactMap { $0.selectedDataSource?.dataSourceName }) sampleRate: \(audioSession.sampleRate) preferredOutputNumberOfChannels: \(audioSession.preferredOutputNumberOfChannels) outputSelectedDatasource: \(audioSession.currentRoute.outputs.compactMap { $0.selectedDataSource?.dataSourceName }) outputDataSources: \(audioSession.outputDataSources) outputDataSource: \(audioSession.outputDataSource) """ }
  14. Now restart if dumpAudioSession() == requiredConfig { return } audioEngine.inputNode.removeTap(onBus:

    0) notificationCancellable?.cancel() notificationCancellable = nil audioEngine.stop() audioEngine = AVAudioEngine() setupAudioSession() startCapture()
  15. A new issue

  16. A new issue AVAudioSession 
 is a singleton!

  17. Audio resource provider engineProvider .engine(for: consumer) .compactMap { availability ->

    AudioResourceAudioEngine? in switch availability { case let .available(engine): return engine //... } } .subscribe(onNext: { engine in try? engine.inputNode.installTap { buffer, time in //... } })