ミラティブとWebRTC - WebRTC framework の中身を覗いてみよう / WebRTC framework AudioUnit Processing

WebRTC framework の中身がわからなくて近付くのが怖かったけど、少しだけわかりあえた話

Transcript

1. ϛϥςΟϒͱWebRTC
   WebRTC framework ͷத਎Λ೷͍ͯΈΑ͏
   Naruki Chigira
   @_naru_jpn / Lib arXiv / ໌೔ϫΫνϯ2ճ໨
   

   View Slide

2. WebRTC framework
   

   View Slide

3. https://webrtc.org
   

   View Slide

4. WebRTC ͸ϛϥςΟϒͷͲ͜Ͱ࢖ΘΕ͍ͯΔͷ͔
   

   View Slide

5. ίϥϘ௨࿩
   ഑৴ऀ ࢹௌऀ
   ࢹௌऀͱձ࿩Λ͠ͳ͕Β഑৴͕Ͱ͖ΔΑ
   

   View Slide

6. ίϥϘ഑৴
   ഑৴ऀ A ഑৴ऀ B
   ഑৴ऀಉ࢜Ͱձ࿩Λ͠ͳ͕Β഑৴͕Ͱ͖ΔΑ
   

   View Slide

7. NEW! ίϥϘϚονϯά
   ഑৴ऀ A ഑৴ऀ B
   ಉ͡ήʔϜͰ഑৴͍ͯ͠Δਓಉ࢜ͰϚονϯά͢ΔʢҰ෦ެ։
   

   View Slide

8. ։ൃ͢ΔதͰͪΐͬͱࠔͬͨ͜ͱ͕͋ͬͨ
   

   View Slide

9. ഑৴ऀ ࢹௌऀ
   WebRTC ͷ઀ଓ࣌ʹΞϓϦ͕όοΫάϥ΢ϯυঢ়ଶʹ͋ΔͱɺԻ੠ͷೖग़ྗ͕Ͱ͖ͳ͘ͳͬͯ͠·͏
   ϛϥςΟϒΞϓϦ͸
   όοΫάϥ΢ϯυͰ
   ىಈத
   

   View Slide

10. όοΫάϥ΢ϯυͰ઀ଓ଴ػঢ়ଶΛ
    Ωϟϯηϧ͢ΔͳͲͯ͠ɺ੍ޚ͸Ͱ͖ͨ
    

    View Slide

11. ػೳ্͸੍ޚͰ͖͍ͯΔ͚Ͳɺ
    Ͳ͏ͯͦ͠͏ͳ͍ͬͯΔͷ͔ཧղͰ͖͍ͯͳ͍ͷͰ৮Δͷ͕ා͍!
    

    View Slide

12. WebRTC framework ͷத਎Λ೷͍ͯΈΑ͏!
    

    View Slide

13. https://webrtc.googlesource.com/src/
    

    View Slide

14. https://webrtc.googlesource.com/src/
    

    View Slide

15. https://webrtc.googlesource.com/src/+/refs/heads/main
    

    View Slide

16. Ի੠ͷ੍ޚΛ͍ͯ͠Δͱ͜ΖΛ೷͍ͯΈΑ͏!
    

    View Slide

17. https://webrtc.googlesource.com/src/+/refs/heads/main/sdk/objc/
    This directory contains the Obj-C SDK for WebRTC. This includes wrappers for the C++
    PeerConnection API and some platform speci
    fi
    c components for iOS and macOS.
    

    View Slide

18. https://webrtc.googlesource.com/src/+/refs/heads/main/sdk/objc/native/src/audio/
    .mm ͍ͬͯ͏ͷ͸ Objective-C++ Λද֦͢ுࢠͩΑ
    

    View Slide

19. https://webrtc.googlesource.com/src/+/refs/heads/main/sdk/objc/native/src/audio/voice_processing_audio_unit.mm
    AudioOutputUnitStart ΛݺͿͱԻ੠ͷೖग़ྗॲཧ͕͸͡·ΔΑ
    voice_processing_audio_unit.mm
    

    View Slide

20. https://developer.apple.com/documentation/audiotoolbox/1439763-audiooutputunitstart
    AudioOutputUnitStart ΛݺͿͱԻ੠ͷೖग़ྗॲཧ͕͸͡·ΔΑ
    

    View Slide

21. AudioOutputUnitStart ͷৼΔ෣͍Λௐ΂ͯΈΑ͏
    

    View Slide

22. https://stackover
    fl
    ow.com/questions/61312360/auaudiounit-cant-play-audio-in-the-background-error-code-561145187
    

    View Slide

23. https://developer.apple.com/documentation/avfaudio/avaudiosession/errorcode/cannotstartrecording
    

    View Slide

24. AudioOutputUnitStart ͕όοΫάϥ΢ϯυͰࣦഊ͢
    Δͷ͸ɺOSͷ࢓༷తʹ΋ͦ͏ͳ͍ͬͯΔΒ͍͠!
    

    View Slide

25. AudioOutputUnitStart ͕͍ͭݺ͹Ε͍ͯΔͷ͔΋֬ೝ͓ͯ͜͠͏
    

    View Slide

26. https://webrtc.googlesource.com/src/+/refs/heads/main/audio/audio_state.cc
    WebRTC ͷ઀ଓཱ͕֬ͨ͠ޙʹ AddReceivingStream ͕ݺ͹Εɺ

    StartPlayout ͷதͰ AudioOutputUnitStart ͕ݺ͹ΕΔ
    

    View Slide

27. ΞϓϦ͕όοΫάϥ΢ϯυঢ়ଶͩͱɺԻ੠ͷೖग़ྗॲ
    ཧͷ։࢝ʹࣦഊͯ͠͠·͏͜ͱ͕ཧղͰ͖ͨ!
    

    View Slide

28. ·ͱΊ
    όοΫάϥ΢ϯυͰԻ੠ͷ΍ΓͱΓ͕Ͱ͖ͳ͘ͳΔݱ৅͕ɺί
    ʔυϨϕϧͰཧղͰ͖ΔΑ͏ʹͳͬͨ
    ɾݪҼ͕Θ͔ͬͯɺͳͥͳͷ͔Λઆ໌Ͱ͖ΔΑ͏ʹͳͬͨ
    ɾWebRTC framework ͱগ͚ͩ͠஥ྑ͘ͳΕͨ
    

    View Slide

29. ͋Γ͕ͱ͏͍͟͝·ͨ͠!
    WE ARE HIRING! ੵۃ࠾༻தͩΑ!
    

    View Slide