Ի੠഑৴ΞϓϦʹ͓͚ΔiOSΛ࢖ͬ ͨԻ੠഑৴ͷશͯͱཪଆ iOSDC 2022 Track C 09/11 13:35ʙ

ࣗݾ঺հ Name: ԕ౻୓໻(͑Μͨ͘) Job: iOS / AndroidΤϯδχΞͳͲ Career: SIer໿6೥ 2018/3~ εϙʔπϚονϯάΞϓϦ 2019/3~ CBcloud ෺ྲྀITαʔϏε 2೥൒ 2021/12~ Voicy Twitter: @entaku_0818

Ի੠഑৴ϓϥοτϑΥʔϜ - Voicy https://corp.voicy.jp/ https://voicy.jp/ • Ի੠ίϯςϯπΛiOS/Android/WebͰఏڙ

VoicyͷΞϓϦ͸࠶ੜͱऩ࿥ͷ2छྨͷΞϓ ϦΛఏڙͯ͠·͢ ΠϯετʔϧϦϯΫ

ऩ࿥ͱ࠶ੜͰදཪҰମͰ͞·͟·ͳVoicyͷػೳ Ի੠࠶ੜ Ի੠ऩ࿥ ੜ์ૹ ίϝϯτ ϋογϡλάݕ ࡧ ച্੥ٻ ࠩ͠ೖΕ ϓϨϛΞϜ์ૹ

Ի੠αʔϏε্Ͱ͸ಛʹॏཁͳೋͭͷػೳ ίϝϯτ ϋογϡλάݕ ࡧ ച্੥ٻ ࠩ͠ೖΕ ϓϨϛΞϜ์ૹ Ի੠࠶ੜ Ի੠ऩ࿥ ੜ์ૹ

ར༻͍ͯ͠Δٕज़ Ի੠࠶ੜԻ੠ऩ࿥ ੜ์ૹ AVAudio Agora

AVAudioͱ͸ʁ

AVFAudioͱ͸ʁ https://developer.apple.com/documentation/avfaudio "7"VEJP4FTTJPO "7"VEJP3PVUJOH"SCJUFS "7"VEJP1MBZFS "7.*%*1MBZFS "7"VEJP3FDPSEFS "VEJP&OHJOF • AVFoundation͔ΒAudioͷػೳΛ੾Γग़ͨ͠΋ͷͰ͢ • iOS14.5͔Β

AVFAudioͱ͸ʁ https://developer.apple.com/documentation/avfaudio "7"VEJP4FTTJPO "7"VEJP3PVUJOH"SCJUFS "7"VEJP1MBZFS "7.*%*1MBZFS "7"VEJP3FDPSEFS "VEJP&OHJOF • VoicyͰར༻͍ͯ͠Δػೳ

Ի੠ͷऩ࿥

࿥ԻͷྲྀΕ ։࢝ ࿥Իத ऴྃ - 1tapͰ࿥ԻՄೳͳઃܭ Ի੠ͷ೾ܗ৘ใදࣔ

࿥Իݖݶ AVAudioSession.sharedInstance().requestRecordPermission { granted in self.handlePermissionGranted() } • ࿥ԻݖݶΛऔಘ https://developer.apple.com/documentation/avfaudio/avaudiosession/1616463- recordpermission

ηογϣϯઃఆ try audioSession.setCategory(.playAndRecord)ɹ try audioSession.setPreferredSampleRate(44100) try audioSession.setPreferredIOBufferDuration(defaultPreferredIOBufferDurat ion) try audioSession.setActive(true) https://developer.apple.com/documentation/avfaudio/avaudiosession • ࠶ੜऩ࿥ՄೳͳηογϣϯͰ։࢝ /P*NBHF

࿥Ի։࢝ var componentDescription = AudioComponentDescription( componentType: OSType(kAudioUnitType_Output), componentSubType: OSType(kAudioUnitSubType_RemoteIO), componentManufacturer: OSType(kAudioUnitManufacturer_Apple), componentFlags: UInt32(0), componentFlagsMask: UInt32(0) ) let component = AudioComponentFindNext(nil, &componentDescription) var tempAudioUnit: AudioUnit? AudioComponentInstanceNew(component!, &tempAudioUnit) https://developer.apple.com/documentation/audiotoolbox • audioUnitΛੜ੒

࿥Ի։࢝ var inputCallbackStruct = AURenderCallbackStruct( inputProc: recordingCallback, inputProcRefCon: UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque()) ) https://developer.apple.com/documentation/audiotoolbox • Callbackͷఆٛ

࿥Ի։࢝ // ࿥ԻͷCallback private let recordingCallback: AURenderCallback = { (inRefCon, ioActionFlags, inTimeStamp, inBusNumber, frameCount, _) -> OSStatus in audioObject.handlerBufferData(bufferList: &bufferList, frameCount: frameCount) } • όοϑΝʔ৘ใΛऔಘ

೾ܗͷऔಘ let pcmBuffer = AVAudioPCMBuffer(pcmFormat: audioFormat, frameCapacity: UInt32(bufferSize)) buffers.append(buffer) waveFormHeights.append(buffer.waveFormHeight) • औಘͨ͠όοϑΝʔ৘ใΛ΋ͱʹԻྔΛϨϯμϦϯά

೾ܗͷऔಘ 6*$PMMFDUJPO7JFX let pcmBuffer = AVAudioPCMBuffer(pcmFormat: audioFormat, frameCapacity: UInt32(bufferSize)) buffers.append(buffer) waveFormHeights.append(buffer.waveFormHeight) • औಘͨ͠όοϑΝʔ৘ใΛ΋ͱʹԻྔΛϨϯμϦϯά

࠶ฤू • ऩ࿥ͨ͠Ի੠σʔλΛ্ॻ͖

Ի੠ऩ࿥ͷऴྃ let settings: [String: Any] = [ AVSampleRateKey: Constants.Recording.sampleRate, AVFormatIDKey: kAudioFormatMPEG4AAC, // MP4ܗࣜͰอଘ AVNumberOfChannelsKey: Constants.Recording.numberOfChannel, AVEncoderAudioQualityKey: AVAudioQuality.high ] let audioFile = try AVAudioFile(forWriting: fileURL, settings: settings) • Local΁อଘ͠Ξοϓϩʔυ

AVAudioRecorder͸ར༻͍ͯ͠ͳ͍ https://developer.apple.com/documentation/avfaudio "7"VEJP4FTTJPO "7"VEJP3PVUJOH"SCJUFS "7"VEJP1MBZFS "7.*%*1MBZFS "7"VEJP3FDPSEFS "VEJP&OHJOF • ։ൃ౰ॳ͸audioUnitͰ༷ʑͳ͜ͱΛ΍Ζ͏ͱ͍ͯͨ͠ͷ͔΋͠Εͳ͍ • ͓ͦΒ͘೾ܗʹԻྔͷߴ͞Λऔಘ͢ΔͨΊʹར༻ BVEJP6OJU

Ի੠ͷฤू

࠶ฤू • αʔϏεϙϦγʔ্Ͱ͖ͳ͍Α͏ʹ͍ͯ͠·͢ʂ

Ի੠࠶ੜ

Ի੠ͷ։࢝ Ի੠࠶ੜ։࢝ Ի੠࠶ੜத

Ի੠ͷ։࢝ func playPlayingChapter() { voicePlayer.play(url: fileURL, startTime: playingChapter.lastPlaytime.msecToSec(), speed: playSpeed.rawValue) } • νϟϓλʔ୯ҐͰͷ։࢝

Ի੠ͷఀࢭ func stopPlay() { player?.stop() player = nil } • Ի੠ͷఀࢭ

background࠶ੜ try audioSession.setCategory(.playback, mode: .default, options: []) https://developer.apple.com/documentation/avfaudio/avaudiosession/category

ੜ์ૹ

Agoraͱ͸ʁ

Agoraͱ͸ʁ https://www.agora.io/en/ https://jp.vcube.com/sdk

AgoraͰ͸ϏσΦ௨࿩΍Ի੠഑৴ͷػೳΛఏڙ https://www.agora.io/en/

ͦͷதͰ΋Live Steaming ͱ Real-time MessagingΛར༻ https://www.agora.io/en/

Voicyͷੜ์ૹͱ͸ʁ Ի੠ͷ࿈ܞ 
 Live Streaming (RTC) ͓ศΓ΍λΠτϧͷ࿈ܞ RealTimeMessage (RTM)

ੜ์ૹ

ੜ์ૹͷྲྀΕ ੜ์ૹ։࢝ ࿥Ի ऴྃ

ੜ์ૹ // RTC agoraKit = AgoraRtcEngineKit.sharedEngine(withAppId: EnvironmentConfig.agoraAppId, delegate: self) agoraKit?.enableAudioVolumeIndication(200, smooth: 3, report_vad: true) agoraKit?.setChannelProfile(AgoraChannelProfile.liveBroadcasting) // RTM agoraRtmKit = AgoraRtmKit(appId: EnvironmentConfig.agoraAppId, delegate: self) agoraRtmChannel = agoraRtmKit?.createChannel(withId: liveId, delegate: self) • Agora্ͰRTCͱRTMͰͦΕͧΕνϟϯωϧΛ࡞੒

ੜ์ૹ࿥Ի // ࿥Ի͸αʔόʔαΠυͰ࣮ࢪ͍ͯ͠·͢ // ͲΜͳॲཧ͔Θ͔ΔΑ͏ʹcurlͷAPIΛهࡌ͠·ͨ͠ curl --location --request GET 'https://api.agora.io/v1/apps// cloud_recording/resourceid//sid//mode//query' \ • Agoraͷcloud_recording(࿥Ի)ͷػೳ

͓ศΓϝοηʔδͷऔಘ func rtmKit(_ kit: AgoraRtmKit, messageReceived message: AgoraRtmMessage, fromPeer peerId: String) { messageReceived(message: message, uid: peerId) } ~~~ if let message = RTMMessage(rawValue: message.text), let uid = Int(uid) { switch message { case .hands_up: delegate?.onRaiseHand(userId: uid) case .hands_down: delegate?.onDroppedHand(userId: uid) case .invite: delegate?.onGuestInvited() case .canceled_invite: delegate?.onCancelGuestInvitation() case .reject: delegate?.onInvitaionRejected(userId: uid) case .set_audience: changeListener() case .mute: delegate?.onMute(uid: uid, muted: true) case .un_mute: delegate?.onMute(uid: uid, muted: false) case .stamp: delegate?.onStampReceived() case .edit_live_info: delegate?.onEditLiveInfo() case .letter: delegate?.onReceivedLetter(userId: uid) case .return_guest_to_listener: delegate?.onReturnGuestToListener() case .start: delegate?.onLiveStarted() } } • ϝοηʔδΛऔಘ༷͠ʑͳϦΞϧλΠϜॲཧΛ࣮ݱ

ੜ์ૹ

ੜ์ૹ • AgoraͰੜ์ૹʹࢀՃ self.agoraRtmChannel?.join(completion: { code in self.agoraKit?.setClientRole(agoraClientRole) self.agoraKit?.joinChannel(byToken: entity.agoraToken.rtcToken, channelId: String(liveID), info: nil, uid: UInt(entity.uid))

ੜ์ૹ • ͓ศΓΛૹΔ agoraRtmKit?.send(AgoraRtmMessage(text: text.rawValue), toPeer: uid)

Ի੠αʔϏε্(Voicy)Ͱ͸ಛʹॏཁͳೋͭͷػೳʹ ͍ͭͯ঺հ͠·ͨ͠ ίϝϯτ ϋογϡλάݕ ࡧ ച্੥ٻ ࠩ͠ೖΕ ϓϨϛΞϜ์ૹ Ի੠࠶ੜ Ի੠ऩ࿥ ੜ์ૹ

VoicyͷԻ੠ॲཧͷࠓޙͷల๬

ࠓޙͷల๬ɹԻ੠࠶ੜ/Ի੠ऩ࿥ • Ի੠ऩ࿥ͷ඼࣭޲্ • ϚΠΫ - bluetoothϚΠΫͳͲ͸ి೾͕ׯবͯ͠͠ ·͍Ի੠͕௿͔ͬͨΓ͢Δ • ݱࡏ͸ϚΠΫΛ઀ଓ͠ͳ͍ঢ়ଶͰͷऩ࿥Λਪ঑ • ࿈ଓ࠶ੜ / ΦϑϥΠϯ࠶ੜͳͲ • Ϧεφʔ΁͞ΒʹศརͳԻ੠ମݧΛ

ࠓޙͷల๬ɹੜ์ૹ • ࿥Ի඼࣭޲্ • AgoraͷCloud-RecordingͩͱͲ͏͠ ͯ΋඼࣭͕ྑ͘ͳ͍ • େਓ਺Ͱ΋଱͑ΒΕΔੜ์ૹ΁ • AgoraͱαʔόʔαΠυؚΊͨΩϟύ γςΟͷ޲্

͓·͚

ࠓ৽͘͠Ի੠ΞϓϦΛ࡞Δͱͨ͠ΒͲ͏͢ΔΜͩΖ͏ʁ 🤔 ΍ͬͺSwiftUI͔ʁ Swift Concurrencyͱ͔৽ ػೳ΋࢖͍͍ͨͳ

ͱ͍͏Θ͚Ͱ࡞Γ·ͨ͠ʂ https://github.com/entaku0818/VoiceMemo

αϯϓϧΛࢀߟʹͯ͠࡞Γ·ͨ͠ https://github.com/pointfreeco/swift-composable-architecture/tree/main/Examples/VoiceMemos ͱ͍͏Θ͚Ͱ࡞Γ·ͨ͠ʂ

ಛʹ঺հ͍ͨ͠ͱ͜Ζ

Ի੠ॲཧͷDeleagteॲཧͷ؅ཧͷ೉͠͞ • େ͖ͳControllerͰશͯΛίϯτϩʔϧ͕ͪ͠ • ֤ը໘ͰDelegateͰॲཧΛॻ͖͕ͪ

Ի੠ͳͲͰ͸ಛʹଟ͍DelegateॲཧΛ·ͱΊ ͯΔ https://github.com/entaku0818/VoiceMemo private final class Delegate: NSObject, AVAudioPlayerDelegate, Sendable { let didFinishPlaying: @Sendable (Bool) -> Void let decodeErrorDidOccur: @Sendable (Error?) -> Void let player: AVAudioPlayer init( url: URL, didFinishPlaying: @escaping @Sendable (Bool) -> Void, decodeErrorDidOccur: @escaping @Sendable (Error?) -> Void ) throws { self.didFinishPlaying = didFinishPlaying self.decodeErrorDidOccur = decodeErrorDidOccur self.player = try AVAudioPlayer(contentsOf: url) super.init() self.player.delegate = self } func audioPlayerDidFinishPlaying(_ player: AVAudioPlayer, successfully flag: Bool) { self.didFinishPlaying(flag) } func audioPlayerDecodeErrorDidOccur(_ player: AVAudioPlayer, error: Error?) { self.decodeErrorDidOccur(error) } }

liveͱ͍͏ঢ়ଶͰDelegateΛఆٛ Ի੠։࢝࣌ͷॲཧΛެ։ https://github.com/entaku0818/VoiceMemo struct AudioPlayerClient { var play: @Sendable (URL) async throws -> Bool } extension AudioPlayerClient { static let live = Self { url in let stream = AsyncThrowingStream { continuation in do { let delegate = try Delegate( url: url, didFinishPlaying: { successful in continuation.yield(successful) continuation.finish() }, decodeErrorDidOccur: { error in continuation.finish(throwing: error) } ) delegate.player.play() continuation.onTermination = { _ in delegate.player.stop() } } catch { continuation.finish(throwing: error) } } return try await stream.first(where: { _ in true }) ?? false } } 4XJGU$PODVSSFODZ Ͱ࣮૷

ReducerͰը໘ͰඞཁͳॲཧΛఆٛ https://github.com/entaku0818/VoiceMemo switch state.mode { case .notPlaying: state.mode = .playing(progress: 0) return .run { [url = state.url] send in let start = environment.mainRunLoop.now async let playAudio: Void = send( .audioPlayerClient(TaskResult { try await environment.audioPlayer.play(url) }) ) for try await tick in environment.mainRunLoop.timer(interval: 0.5) { await send(.timerUpdated(tick.date.timeIntervalSince(start.date))) } } .cancellable(id: PlayID.self, cancelInFlight: true) case .playing: state.mode = .notPlaying return .cancel(id: PlayID.self) }

ࢀߟࢿྉ

ࢀߟࢿྉ • agoraΛ࢖ͬͯϥΠϒ഑৴ػೳΛ1ϲ݄൒ͰϦϦʔεͨ͠࿩ • εϥΠυ • ࠓ೔͔Β෼͔Δ AVAudioEngineͷશͯ • εϥΠυ • ࣮૷ͨ͘͠ͳΔԻ੠ฤू • εϥΠυ • աڈͷԻ੠ʹؔΘΔiOSDCͷηογϣϯ

ࢀߟࢿྉ • ΍ͬͺΓ࠷ޙ͸ެࣜ • https://developer.apple.com/documentation/avfaudio • https://docs.agora.io/en

એ఻ʂ

https://voicy.jp/channel/1305 VoicyͷΤϯδχΞ͕ӡӦ͍ͯ͠Δνϟϯωϧ voi-chordௌ͍͍ͯͩ͘͞ʂ

iOSΤϯδχΞΛืू͍ͯ͠·͢ • Ի੠޷͖ͳΤϯδχΞืूதͰ͢ʂڵຯ͋Δํ͸DM͍ͩ͘͞ʂ Notion Twitter: @entaku_0818