AVAudioEngineでリアルタイムレンダリング

iOSDC Japan 2020

Transcript

1. :VLJ:BTPTIJNB!ZBTP@TBO
   
   "7"VEJP&OHJOFͰ
   ϦΞϧλΠϜϨϯμϦϯά
   J04%$+BQBO
   

   View Slide

2. ಺༰
   ΦʔσΟΦೖग़ྗͷ࢓૊Έ
   J04ͷΦʔσΟΦ"1*
   ҆ఆͯ͠ϦΞϧλΠϜʹϨϯμϦϯάΛ͢Δʹ͸
   IUUQTHJUIVCDPNPCKFDUJWFBVEJP"VEJP6OJU3FOEFSJOH&YBNQMF
   αϯϓϧίʔυ
   

   View Slide

3. ΦʔσΟΦೖग़ྗͷ࢓૊Έ
   

   View Slide

4. ίʔϧόοΫ
   ΦʔσΟΦೖग़ྗͷ࢓૊Έ
   ΦʔσΟΦॲཧΛ։࢝͢Δͱઐ༻ͷεϨουͰ܁Γฦ͠ίʔϧόοΫ͕ݺ͹ΕΔ
   ʢඵʹ͖ͭ਺ेճҎ্ʣ
   

   View Slide

5. ίʔϧόοΫ
   ग़ྗ
   ΦʔσΟΦೖग़ྗͷ࢓૊Έ
   ίʔϧόοΫ಺ͰσʔλΛ౉͢ͱԻ੠͕ग़ྗ͞ΕΔ
   

   View Slide

6. ίʔϧόοΫ
   ग़ྗ
   ೖྗ
   ΦʔσΟΦೖग़ྗͷ࢓૊Έ
   ίʔϧόοΫʹೖྗ͞ΕͨԻ੠ͷσʔλ͕౉ͬͯ͘Δ
   

   View Slide

7. ίʔϧόοΫ
   ग़ྗ
   ೖྗ
   ΦʔσΟΦೖग़ྗͷ࢓૊Έ
   ❌
   ίʔϧόοΫͷॲཧʹ͕͔͔࣌ؒΔͱɺԻ్͕੾ΕͨΓऔΓ͜΅ͨ͠Γ͢Δ͜
   ͱʹͳΔ
   

   View Slide

8. "7"VEJP1MBZFSɾ"7"VEJP3FDPSEFSͳͲͷߴϨϕϧͳ"1*͸ɺ͜ͷΑ͏ͳ௿Ϩ
   ϕϧॲཧΛؾʹ͠ͳͯ͘΋؆୯ʹ࢖͑ΔΑ͏ʹͳ͍ͬͯΔ
   ϦΞϧλΠϜʹࣗ༝ʹԻΛѻ͍͍ͨ৔߹͸ɺ௿ϨϕϧͳॲཧΛ͢Δඞཁ͕͋Δ
   ΦʔσΟΦೖग़ྗͷ࢓૊Έ
   

   View Slide

9. J04ͷΦʔσΟΦ"1*
   

   View Slide

10. J04ॳظͷࠒ͔Β͋Δ$ݴޠͷ"1*
    $BSCPODPNQPOFOUCBTFE"VEJP6OJUT
    "6(SBQI
    "VEJP2VFVF4FSWJDFT
    

    View Slide

11. ඇਪ঑ʹͳͬͨ"1*
    J043FMFBTF/PUFTΑΓ
    "6(SBQIʢʙJ04ʣ
    *OUFS"QQBVEJPʢʙJ04ʣ
    $BSCPODPNQPOFOUCBTFE"VEJP6OJUTʢ͍ۙ͏ͪʹ࢖͑ͳ͘ͳΔʣ
    

    View Slide

12. ௥Ճ͞Εͨ"1*
    "7"VEJP&OHJOFʢJ04ʙʣ
    "VEJP6OJU7"VEJP6OJU&YUFOTJPOTʢJ04ʙʣ
    "7"VEJP4PVSDF/PEF"7"VEJP4JOL/PEFʢJ04ʙʣ
    

    View Slide

13. J04ҎલͷϦΞϧλΠϜϨϯμϦϯά
    $ݴޠͷ"VEJP6OJU
    "6"VEJP6OJUΛܧঝ͢Δ͜ͱͰ"7"VEJP&OHJOFͰ΋Մೳʹ͸ͳΔʢϓϥάΠϯ
    ͷ࢓૊ΈͰ࡞ΔͷͰɺηοτΞοϓ͕໘౗Ͱෳࡶʣ
    

    View Slide

14. "7"VEJP4PVSDF/PEF"7"VEJP4JOL/PEF͕ొ৔
    ϓϥάΠϯͱൺ΂ͯηοτΞοϓ͕༰қʹͳͬͨ
    J04Ҏ߱ͷϦΞϧλΠϜϨϯμϦϯά
    

    View Slide

15. "7"VEJP4PVSDF/PEF
    ΦʔσΟΦΛग़ྗ͢ΔσʔλΛ౉͢
    4PVSDF/PEF 0VUQVU/PEF
    "7"VEJP&OHJOF
    3FOEFS
    #MPDL
    @interface AVAudioSourceNode : AVAudioNode
    - (instancetype)initWithRenderBlock:(AVAudioSourceNodeRenderBlock)block;
    - (instancetype)initWithFormat:(AVAudioFormat *)format
    renderBlock:(AVAudioSourceNodeRenderBlock)block;
    @end
    

    View Slide

16. "7"VEJP4JOL/PEF
    ΦʔσΟΦͷೖྗ͞ΕͨσʔλΛड͚औΔ
    *OQVU/PEF 4JOL/PEF
    "7"VEJP&OHJOF
    3FDFJWFS
    #MPDL
    @interface AVAudioSinkNode : AVAudioNode
    - (instancetype)initWithReceiverBlock:(AVAudioSinkNodeReceiverBlock)block;
    @end
    

    View Slide

17. ҆ఆͨ͠ϨϯμϦϯάΛߦ͏
    

    View Slide

18. ΦʔσΟΦ*0ͷεϨουʹ͓͍ͯ
    ϝϞϦͷ֬อΛ͠ͳ͍
    MJCEJTQBUDI͸࢖Θͳ͍
    $$ͰॲཧΛॻ͘
    "DDFMFSBUF'SBNFXPSLΛ׆༻͢Δ
    ଞͷεϨουͱͷϩοΫΛආ͚Δʢ4XJGU΍0CKFDUJWF$ͷݺͼग़͠ɺϑΝΠϧ
    ೖग़ྗͷதʹ͋Δʣ
    IUUQTQPTUEDDGPVSDPNNPONJTUBLFTJOBVEJPEFWFMPQNFOU
    

    View Slide

19. αϯϓϧίʔυ
    "7"VEJP4PVSDF/PEFαΠϯ೾ͷ࠶ੜ
    ʲ%JSFDUʳप೾਺ͷૢ࡞Λ͙͢ʹ൓ө
    ʲ%FGFSSFEʳཷΊ͍ͯΔόοϑΝʹରͯ͠प೾਺ͷૢ࡞Λ൓ө
    "7"VEJP4JOL/PEFೖྗԻͷϝʔλʔදࣔ
    IUUQTHJUIVCDPNPCKFDUJWFBVEJP"VEJP6OJU3FOEFSJOH&YBNQMF
    

    View Slide

20. αϯϓϧίʔυͷݴޠߏ੒
    3FOEFS#MPDL3FDFJWFS#MPDLͷத͸$Ͱهड़
    0CKFDUJWF$Ͱϥοϓͯ͠/PEFΛฦ͢
    "7"VEJP&OHJOFͷߏங΍6*͸4XJGUͰهड़
    

    View Slide

21. εϨουؒͷ΍ΓͱΓ
    

    View Slide

22. TUEBUPNJD
    ಛ௃
    ੔਺ܕɾුಈখ਺఺਺ܕɾϙΠϯλܕΛεϨουηʔϑʹѻ͑ΔΫϥε
    JT@MPDL@GSFF
    ؔ਺ͰϩοΫ͞ΕΔ͔औಘͰ͖Δʢࠓ࣌ͷσόΠεͳΒ·ͣ
    USVFʣ
    

    View Slide

23. ෳ਺ͷ஋ΛΞτϛοΫʹ͢Δ
    ڞ༻ମͰTUEBUPNJDͷܕͱTUSVDUΛม׵
    union Value {
    void *v;
    struct {
    float const level;
    float const peak;
    };
    };
    static_assert(sizeof(Value) == sizeof(void *), "");
    

    View Slide

24. ෳ਺ͷ஋ΛΞτϛοΫʹ͢Δ
    ڞ༻ମͰTUEBUPNJDͷܕͱTUSVDUΛม׵
    std::atomic atomicValue;
    // atomicͷ஋ʹηοτ͢Δ
    Value setValue{.level = 0.0f, .peak = 0.0f};
    atomicValue = setValue.v;
    // atomicͷ஋͔Βήοτ͢Δ
    Value getValue{.v = atomicValue};
    

    View Slide

25. TUETIBSFE@QUS
    ಛ௃
    $ͷΫϥεΛࢀরΧ΢ϯτํࣜͰอ࣋͢Δ
    ࢀরΧ΢ϯλ͸εϨουηʔϑ
    ϨϯμϦϯάதʹίϐʔ͠ͳ͍Α͏ʹؾΛ͚ͭΔ
    

    View Slide

26. TUETIBSFE@QUS
    ίϐʔͯ͠͠·͏ॻ͖ํ
    void sample(std::shared_ptr data) {
    auto localData = data;
    }
    ίϐʔ͞ΕΔ
    ͭղ์͞ΕΔ
    ίϐʔ͞ΕΔ
    

    View Slide

27. TUETIBSFE@QUS
    ίϐʔ͞Εͳ͍ॻ͖ํ
    void sample(std::shared_ptr const &data) {
    auto const &localData = data;
    }
    ίϐʔ͞Εͳ͍
    ίϐʔ͞Εͳ͍
    Կ΋ى͖ͳ͍
    

    View Slide

28. ͭͷεϨου͔Βಉ͡σʔλʹॻ͖ࠐΉ
    %FGFSSFEͷαϯϓϧ
    

    View Slide

29. ͭͷεϨου͔Βಉ͡σʔλΛѻ͏
    %FGFSSFEͷαϯϓϧ
    αΠϯ೾ੜ੒εϨου
    αΠϯ೾Λ
    ॻ͖ࠐΈ
    ग़ྗόοϑΝʹ
    ಡΈࠐΈ
    ڞ༗͢Δσʔλ ͭ
    
    ࠷ޙ·ͰಡΈࠐΜͩΒ
    ಡΈࠐΈݩΛ੾Γସ͑
    αΠϯ೾ͷ
    ॻ͖ࠐΈΛࢦࣔ
    ඵؒͷ
    ΦʔσΟΦσʔλ
    ग़ྗεϨου
    

    View Slide

30. ͭͷεϨου͔Βಉ͡σʔλΛѻ͏
    %FGFSSFEͷαϯϓϧ
    αΠϯ೾ੜ੒εϨου
    8SJUJOH
    3FBEJOH
    ΞτϛοΫͳϑϥά
    8SJUJOHͳΒ
    αΠϯ೾Λॻ͖ࠐΜͰ
    3FBEJOHʹ͢Δ
    3FBEJOHͳΒ
    ग़ྗόοϑΝʹ
    σʔλΛಡΈࠐΜͰ
    ࠷ޙ·ͰߦͬͨΒ
    8SJUJOHʹ͢Δ
    ڞ༗͢Δσʔλ ͭ
    
    ඵؒͷ
    ΦʔσΟΦσʔλ
    ग़ྗεϨου
    

    View Slide

31. ·ͱΊ
    $ݴޠͷ"1*͸ඇਪ঑ɻ࣌୅͸"7"VEJP&OHJOF
    ΦʔσΟΦͷεϨου͕ϩοΫͯ͠͠·Θͳ͍Α͏ʹؾΛ෇͚Δ
    

    View Slide