今日から分かる Android Audioの全て

KSA Online Meetup #1
https://student-kyushu.connpass.com/event/191541/

Transcript

1. ࠓ೔͔Β෼͔Δ
   Android Audioͷશͯ
   2020.10.26
   ߹ಉձࣾφΫα

   ઎෦ ॡฏ
   

   View Slide

2. ࣗݾ঺հ
   ઎෦ ॡฏ
   • ߹ಉձࣾφΫαɹ୅ද
   • ΤϯλϝͷϥΠϒ഑৴Λࢧ͍͑ͯ
   ·͢
   • झຯ͸࡞ۂͱωοτϫʔΫӡ༻
   • Android։ൃͰҰ൪࢖͍ͬͯΔ΋ͷ
   ͸MediaCodecͱVulkan
   

   View Slide

3. Pixel 3 ໰୊
   w 1JYFMͰ͸ɺ""VEJP͕ݪҼͰ࠶ੜ͞Ε
   ͳ͍໰୊͕͋ͬͨ
   

   View Slide

4. ձࣾ঺հ
   1.
   

   View Slide

5. w ೥݄ʹઃཱ
   w ຊࣾ͸౦ژ౎ઍ୅ా۠ਆాۋொ
   w ΦϑΟεʹϥοΫ͕͋Δ
   w ϝϯόʔ਺͸໊
   w ࠾༻ืूதʂ
   ߹ಉձࣾφΫα
   

   View Slide

6. w &/5&35"*.&/5ʷ5&$)/0-0(:
   w ΤϯλʔςΠϝϯτͷ෼໺Λٕज़Λ࢖ͬͯ໰୊ղܾΛ͢Δ
   ձࣾ
   w ΢Σϒ੍࡞͔Βಈը഑৴·ͰԿͰ΋΍Γ·͢
   φΫαͷࣄۀ
   

   View Slide

7. w ϥΠϒ഑৴αʔϏε
   w ిࢠॻ੶αʔϏε
   w ͳͲͳͲ
   ࣄۀྫ
   

   View Slide

8. w /-*7&͸ɺφΫα͕ఏڙ͍ͯ͠ΔιϦϡʔγϣϯͷҰͭɻ
   w /-*7&͸ɺΫϥΠΞϯτ͔ΒΠϯϑϥ·ͰશͯφΫα͕Χ
   όʔ͢Δ͜ͱ͕ग़དྷ·͢
   w ͦͷଞɺ%3.ͷରԠͳͲ΋αϙʔτ͢Δ͜ͱ͕ՄೳͰ͢
   ϥΠϒ഑৴ιϦϡʔγϣϯ NLIVE
   

   View Slide

9. w ڵຯΛ࣋ͬͨਓ͸๻ʹ࿈བྷ͍ͯͩ͘͠͞
   w ۀ຿ҕୗɾΞϧόΠτ΋େ׻ܴͰ͢
   We're Hiring!
   

   View Slide

10. AndroidͷAudioͷྺ࢙
    2.
    

    View Slide

11. w "VEJP'MJOHFS͸"VEJP4FSWFSͷͭͰ͋ΓɺΞϓϦͷԻΛ
    ؅ཧ͢Δαʔό
    w "VEJP'MJOHFS͸"35 "OESPJE3VOUJNF
    ্Ͱಈ࡞͢Δ
    w "VEJP'MJOHFS͔Β)"- )BSEXBSF"CTUSBDU-BZFS
    ʹॲ
    ཧ͞Εͯɺ࿥Ի΍࠶ੜΛߦ͏
    AudioFlinger
    

    View Slide

12. w "VEJP'MJOHFS͕"OESPJEͷ஗Ԇ໰୊ͷ
    தͰ͸ಛʹӨڹ͕େ͖͍
    w ͦͷཧ༝͸"VEJP'MJOHFSͰόοϑΝΛ
    ࡞Γ௚͔͢ΒͰ͋Δ
    AudioFlingerͷ10ms໰୊
    

    View Slide

13. w "VEJP'MJOHFSͷલʹ͸)"-͕͋Γɺ
    )"-͕"-4"ͱ΍ΓͱΓΛͯ͠σʔλΛ
    ૹड৴͢Δ
    w "-4"͸ϦϯάόοϑΝΛ࢖ͬͯɺυϥ
    Πόʔ͔Β"VEJP'MJOHFSʹૹ͍ͬͯΔ
    AudioFlingerͷ10ms໰୊
    

    View Slide

14. w "VEJP'MJOHFSͷޙʹ͸"VEJP3FDPSE
    ΍"VEJP5SBDL͕͋Δ
    w ͪ͜Β΋#ZUF#VGGFSΛ࢖ͬͯɺόο
    ϑΝΛ࡞ͬͯॻ͖ࠐΉॲཧ͕͋Δͷ
    Ͱɺ஗Ԇ͕͋Δ
    AudioFlingerͷ10ms໰୊
    

    View Slide

15. w "VEJP"1*ͷඪ४࣮૷͕0QFO4-&4
    w 0QFO4-&4Λ࢖͏͜ͱͰɺ"VEJP'MJOHFSΑΓ΋௿஗Ԇʹ
    ࿥Ի΍࠶ੜ͕Ͱ͖Δ
    OpenSL ES
    

    View Slide

16. w 0QFO4-&4ͷ໰୊఺͸ɺσόΠεʹΑͬͯ͸ඇରԠͳ͜
    ͱ͕ଟ͍ʢυϥΠόʔͷඇαϙʔτʣ
    w ·ͨɺ0QFO4-&4ͷ࣮૷΋͔ͳΓεύήοςΟίʔυʹ
    ͳ͓ͬͯΓɺ࣌ʑෆ҆ఆʹͳΔ͜ͱ΋Α͋͘Δ
    OpenSL ES͕࢖͍෺ʹͳΒͳ͍
    

    View Slide

17. w "OESPJEΑΓɺ0QFO4-&4ͷ୅ସͱͯ͠""VEJP͕ొ
    ৔ͨ͠
    w ""VEJPͰ͸ɺ͜Ε·Ͱ"VEJP'MJOHFSͷϥούʔతͳ໾ׂ
    Λ͍ͯͨ͠0QFO4-&4ͱ͸ҧ͍ɺ௚઀)"-ʹΞΫηεͰ
    ͖Δ
    AAudio
    

    View Slide

18. w ""VEJP͸0QFO4-&4ಉ༷ʹ$Ͱهड़Λߦ͏ͨΊɺ
    "35ͷ($֎Ͱ͋ΔʢͦͷͨΊϝϞϦղ์͸ࣗ෼Ͱ΍Δඞ
    ཁ͕͋Δʣ
    w ""VEJPͰ͸ɺ֤ϕϯμʔͱڠྗ͠ɺ҆ఆͯ͠ಈ࡞͢Δ͜
    ͱΛ໨ࢦͨ͠
    AAudio
    

    View Slide

19. Implementaion
    3.
    

    View Slide

20. w (PPHMF͕։ൃͨ͠ϥΠϒϥϦ
    w 0QFO4-&4ͱ""VEJPͷ૬ޓར༻Λ
    ߦ͏ͨΊͷϥούʔ
    w ࣗ෼Ͱॻ͘ͷ͕ͭΒ͍ਓ͸0CPFΛ
    ࢖͏ͱ؆୯ʹهड़Ͱ͖Δ
    Oboe
    

    View Slide

21. w 0QFO4-&4͸ࠓߋ࢖͏ඞཁ΋ͳ͍ͷͰࠓճ͸""VEJPΛ࣮
    ૷ͯ͠Έ·͠ΐ͏
    ࣗ෼Ͱ࣮૷ͯ͠ΈΔ
    

    View Slide

22. dlsymͰAAudioΛಡΈࠐΉ
    w MJCBBVEJPTP͸
    NJO4EL7FSTJPO͔Βೖ
    ͍ͬͯΔ
    

    View Slide

23. InputStreamΛ४උ͢Δ
    void AAudio_IO::setupInputStream( AAudioStreamBuilder* builder ) {
    AAudioStreamBuilder_setDeviceId ( builder, _inputDeviceId );
    AAudioStreamBuilder_setSampleRate ( builder, _sampleRate );
    AAudioStreamBuilder_setChannelCount( builder, _inputChannelCount );
    AAudioStreamBuilder_setFormat ( builder, _sampleFormat );
    AAudioStreamBuilder_setSharingMode ( builder, AAUDIO_SHARING_MODE_EXCLUSIVE );
    AAudioStreamBuilder_setPerformanceMode( builder, AAUDIO_PERFORMANCE_MODE_LOW_LATENCY );
    AAudioStreamBuilder_setDirection ( builder, AAUDIO_DIRECTION_INPUT );
    AAudioStreamBuilder_setErrorCallback ( builder, ::errorCallback, this );
    }
    

    View Slide

24. outputStreamΛ४උ͢Δ
    void AAudio_IO::setupOutputStream( AAudioStreamBuilder* builder ) {
    AAudioStreamBuilder_setDeviceId ( builder, _outputDeviceId );
    AAudioStreamBuilder_setFormat ( builder, _sampleFormat );
    AAudioStreamBuilder_setChannelCount( builder, _outputChannelCount );
    AAudioStreamBuilder_setSharingMode ( builder, AAUDIO_SHARING_MODE_EXCLUSIVE );
    AAudioStreamBuilder_setPerformanceMode( builder, AAUDIO_PERFORMANCE_MODE_LOW_LATENCY );
    AAudioStreamBuilder_setDirection ( builder, AAUDIO_DIRECTION_OUTPUT );
    AAudioStreamBuilder_setDataCallback ( builder, ::dataCallback, this );
    AAudioStreamBuilder_setErrorCallback ( builder, ::errorCallback, this );
    }
    

    View Slide

25. inputStreamΛ࡞Δ
    void AAudio_IO::createInputStream() {
    AAudioStreamBuilder* builder = createStreamBuilder();
    setupInputStream( builder );
    aaudio_result_t result =
    static_cast(AAudioStreamBuilder_openStream( builder,
    &_inputStream ));
    AAudioStreamBuilder_delete( builder );
    }
    

    View Slide

26. outputStreamΛ࡞Δ
    void AAudio_IO::createOutputStream() {
    AAudioStreamBuilder* builder = createStreamBuilder();
    setupOutputStream( builder );
    aaudio_result_t result = static_cast(AAudioStreamBuilder_openStream( builder, &_outputStream ));
    if ( result == AAUDIO_OK && _outputStream != nullptr ) {
    _sampleRate = AAudioStream_getSampleRate( _outputStream );
    _framesPerBurst = AAudioStream_getFramesPerBurst( _outputStream );
    AudioEngineProps::SAMPLE_RATE = _sampleRate;
    updateBufferSizeInFrames( AAudioStream_setBufferSizeInFrames( _outputStream, _framesPerBurst ));
    _underrunCountOutputStream = AAudioStream_getXRunCount( _outputStream );
    } else {
    return;
    }
    AAudioStreamBuilder_delete( builder );
    }
    

    View Slide

27. streamΛ։࢝ɾఀࢭ͢Δ
    void AAudio_IO::startStream( AAudioStream* stream ) {
    aaudio_result_t result = static_cast(AAudioStream_requestStart( stream ));
    if ( result != AAUDIO_OK ) {
    return;
    }
    }
    void AAudio_IO::stopStream( AAudioStream* stream ) {
    if ( stream == nullptr ) {
    return;
    }
    aaudio_result_t result = static_cast(AAudioStream_requestStop( stream ));
    }
    

    View Slide

28. w ""VEJP4USFBN#VJMEFS@TFU1FSGPSNBODF.PEFͰɺ-08@-"5&/$:ʹઃఆ͢Δͱ௒௿஗ԆʹͳΔ͕
    όοςϦʔ͕ݮΔɻ͔͠΋ɺ͜Ε͚ͩͩͱ͋·Γҙຯ͕ͳ͍ʢޙड़ʣ
    w ""VEJP͸5ISFBE4BGFͰ͸ͳ͍ɻͦͷͨΊɺผεϨου͔Βಉ࣌ʹ""VEJPΛݺͼग़͢͜ͱ͸Ͱ͖ͳ
    ͍ɻ
    w ίʔϧόοΫؔ਺Λ࢖͍ɺඇಉظͰॲཧΛ͢Δ͜ͱΛਪ঑ɻ४උ͕Ͱ͖ͨΒίʔϧόοΫͰಡΈࠐΈɾ
    ॻ͖ࠐΈ͕Ͱ͖Δɻ
    w όοϑΝαΠζΛௐ੔͢Δ΂͖ɻ࠷ॳ͸5$1ͷ8JOEPX4J[F0QUJNJ[FͷΑ͏ʹঃʑʹόοϑΝαΠ
    ζΛେ͖͍ͯ͘͘͠ํ͕Α͍ɻ
    AAudioͷͭ·͖ͮϙΠϯτ
    

    View Slide

29. w ""VEJPͷ-PX-BUFODZ.PEF͚ͩͩͱ௒௿஗Ԇʹ͸ͳΒͳ͍
    w "DUJWJUZͷ8JOEPX.BOBHFSʹ8JOEPX.BOBHFS-BZPVU1BSBNT'-"(@,&&1@4$3&&/@0/Λ
    ηοτ͢Δඞཁ͕͋Δ
    w BDUJWJUZHFU8JOEPX
    TFU4VTUBJOFE1FSGPSNBODF.PEF USVF
    ͰɺύϑΥʔϚϯεϞʔυΛ༗ޮʹ
    ͢Δ
    WindowManagerΛϋοΫ͢Δ
    

    View Slide

30. inline void optimizeThreadPerformance( const std::vector& cpuIds )
    {
    cpu_set_t mask;
    pid_t current_thread_id = gettid();
    cpu_set_t cpu_set;
    CPU_ZERO( &cpu_set );
    if ( cpuIds.empty() ) {
    int current_cpu_id = sched_getcpu();
    CPU_SET( current_cpu_id, &cpu_set );
    } else {
    for ( size_t i = 0; i < cpuIds.size(); i++ ) {
    int cpu_id = cpuIds.at( i );
    CPU_SET( cpu_id, &cpu_set );
    }
    }
    }
    CPUΞϑΟχςΟΛεϨουʹઃఆ͢Δ
    w ΞϑΟχςΟΛઃఆ͢Δ͜ͱͰɺ$16ύϑΥʔ
    ϚϯεΛۃݶ·Ͱ޲্ͤ͞Δ͜ͱ͕Ͱ͖Δ
    w ϓϩηεɾεϨου͸04ʹΑͬͯͲΕ͔ͷ
    $16ίΞ্Ͱ࣮ߦ͞ΕΔɻ$16ΞϑΟχςΟΛ
    ઃఆ͢Δ͜ͱͰɺͦͷ࣮ߦ͞ΕΔ$16ίΞΛ೚
    ҙʹ੍ݶɾࢦఆ͢Δ͜ͱ͕Ͱ͖Δɻ
    

    View Slide

31. ·ͱΊ
    4.
    

    View Slide

32. w "OESPJEͷΦʔσΟΦͷྺ࢙͸௕͘ɺෳࡶͰ͋Δ
    w 0CPFΛͱΓ͋͑ͣ࢖͓͚ͬͯ͹Α͍
    w ۃݶ·ͰΧελϚΠζ͍ͨ͠ਓ͸ຊηογϣϯͷ಺༰Λࢀ
    ߟʹɺ࣮૷ͯ͠Έ͍ͯͩ͘͞
    ·ͱΊ
    

    View Slide