Upgrade to Pro — share decks privately, control downloads, hide ads and more …

今日から分かる Android Audioの全て

今日から分かる Android Audioの全て

Shumpei Urabe

October 26, 2020
Tweet

More Decks by Shumpei Urabe

Other Decks in Programming

Transcript

  1. ࠓ೔͔Β෼͔Δ
    Android Audioͷશͯ
    2020.10.26
    ߹ಉձࣾφΫα

    ઎෦ ॡฏ

    View full-size slide

  2. ࣗݾ঺հ
    ઎෦ ॡฏ
    • ߹ಉձࣾφΫαɹ୅ද
    • ΤϯλϝͷϥΠϒ഑৴Λࢧ͍͑ͯ
    ·͢
    • झຯ͸࡞ۂͱωοτϫʔΫӡ༻
    • Android։ൃͰҰ൪࢖͍ͬͯΔ΋ͷ
    ͸MediaCodecͱVulkan

    View full-size slide

  3. Pixel 3 ໰୊
    w 1JYFMͰ͸ɺ""VEJP͕ݪҼͰ࠶ੜ͞Ε
    ͳ͍໰୊͕͋ͬͨ

    View full-size slide

  4. ձࣾ঺հ
    1.

    View full-size slide

  5. w ೥݄ʹઃཱ
    w ຊࣾ͸౦ژ౎ઍ୅ా۠ਆాۋொ
    w ΦϑΟεʹϥοΫ͕͋Δ
    w ϝϯόʔ਺͸໊
    w ࠾༻ืूதʂ
    ߹ಉձࣾφΫα

    View full-size slide

  6. w &/5&35"*.&/5ʷ5&$)/0-0(:
    w ΤϯλʔςΠϝϯτͷ෼໺Λٕज़Λ࢖ͬͯ໰୊ղܾΛ͢Δ
    ձࣾ
    w ΢Σϒ੍࡞͔Βಈը഑৴·ͰԿͰ΋΍Γ·͢
    φΫαͷࣄۀ

    View full-size slide

  7. w ϥΠϒ഑৴αʔϏε
    w ిࢠॻ੶αʔϏε
    w ͳͲͳͲ
    ࣄۀྫ

    View full-size slide

  8. w /-*7&͸ɺφΫα͕ఏڙ͍ͯ͠ΔιϦϡʔγϣϯͷҰͭɻ
    w /-*7&͸ɺΫϥΠΞϯτ͔ΒΠϯϑϥ·ͰશͯφΫα͕Χ
    όʔ͢Δ͜ͱ͕ग़དྷ·͢
    w ͦͷଞɺ%3.ͷରԠͳͲ΋αϙʔτ͢Δ͜ͱ͕ՄೳͰ͢
    ϥΠϒ഑৴ιϦϡʔγϣϯ NLIVE

    View full-size slide

  9. w ڵຯΛ࣋ͬͨਓ͸๻ʹ࿈བྷ͍ͯͩ͘͠͞
    w ۀ຿ҕୗɾΞϧόΠτ΋େ׻ܴͰ͢
    We're Hiring!

    View full-size slide

  10. AndroidͷAudioͷྺ࢙
    2.

    View full-size slide

  11. w "VEJP'MJOHFS͸"VEJP4FSWFSͷͭͰ͋ΓɺΞϓϦͷԻΛ
    ؅ཧ͢Δαʔό
    w "VEJP'MJOHFS͸"35 "OESPJE3VOUJNF
    ্Ͱಈ࡞͢Δ
    w "VEJP'MJOHFS͔Β)"- )BSEXBSF"CTUSBDU-BZFS
    ʹॲ
    ཧ͞Εͯɺ࿥Ի΍࠶ੜΛߦ͏
    AudioFlinger

    View full-size slide

  12. w "VEJP'MJOHFS͕"OESPJEͷ஗Ԇ໰୊ͷ
    தͰ͸ಛʹӨڹ͕େ͖͍
    w ͦͷཧ༝͸"VEJP'MJOHFSͰόοϑΝΛ
    ࡞Γ௚͔͢ΒͰ͋Δ
    AudioFlingerͷ10ms໰୊

    View full-size slide

  13. w "VEJP'MJOHFSͷલʹ͸)"-͕͋Γɺ
    )"-͕"-4"ͱ΍ΓͱΓΛͯ͠σʔλΛ
    ૹड৴͢Δ
    w "-4"͸ϦϯάόοϑΝΛ࢖ͬͯɺυϥ
    Πόʔ͔Β"VEJP'MJOHFSʹૹ͍ͬͯΔ
    AudioFlingerͷ10ms໰୊

    View full-size slide

  14. w "VEJP'MJOHFSͷޙʹ͸"VEJP3FDPSE
    ΍"VEJP5SBDL͕͋Δ
    w ͪ͜Β΋#ZUF#VGGFSΛ࢖ͬͯɺόο
    ϑΝΛ࡞ͬͯॻ͖ࠐΉॲཧ͕͋Δͷ
    Ͱɺ஗Ԇ͕͋Δ
    AudioFlingerͷ10ms໰୊

    View full-size slide

  15. w "VEJP"1*ͷඪ४࣮૷͕0QFO4-&4
    w 0QFO4-&4Λ࢖͏͜ͱͰɺ"VEJP'MJOHFSΑΓ΋௿஗Ԇʹ
    ࿥Ի΍࠶ੜ͕Ͱ͖Δ
    OpenSL ES

    View full-size slide

  16. w 0QFO4-&4ͷ໰୊఺͸ɺσόΠεʹΑͬͯ͸ඇରԠͳ͜
    ͱ͕ଟ͍ʢυϥΠόʔͷඇαϙʔτʣ
    w ·ͨɺ0QFO4-&4ͷ࣮૷΋͔ͳΓεύήοςΟίʔυʹ
    ͳ͓ͬͯΓɺ࣌ʑෆ҆ఆʹͳΔ͜ͱ΋Α͋͘Δ
    OpenSL ES͕࢖͍෺ʹͳΒͳ͍

    View full-size slide

  17. w "OESPJEΑΓɺ0QFO4-&4ͷ୅ସͱͯ͠""VEJP͕ొ
    ৔ͨ͠
    w ""VEJPͰ͸ɺ͜Ε·Ͱ"VEJP'MJOHFSͷϥούʔతͳ໾ׂ
    Λ͍ͯͨ͠0QFO4-&4ͱ͸ҧ͍ɺ௚઀)"-ʹΞΫηεͰ
    ͖Δ
    AAudio

    View full-size slide

  18. w ""VEJP͸0QFO4-&4ಉ༷ʹ$Ͱهड़Λߦ͏ͨΊɺ
    "35ͷ($֎Ͱ͋ΔʢͦͷͨΊϝϞϦղ์͸ࣗ෼Ͱ΍Δඞ
    ཁ͕͋Δʣ
    w ""VEJPͰ͸ɺ֤ϕϯμʔͱڠྗ͠ɺ҆ఆͯ͠ಈ࡞͢Δ͜
    ͱΛ໨ࢦͨ͠
    AAudio

    View full-size slide

  19. Implementaion
    3.

    View full-size slide

  20. w (PPHMF͕։ൃͨ͠ϥΠϒϥϦ
    w 0QFO4-&4ͱ""VEJPͷ૬ޓར༻Λ
    ߦ͏ͨΊͷϥούʔ
    w ࣗ෼Ͱॻ͘ͷ͕ͭΒ͍ਓ͸0CPFΛ
    ࢖͏ͱ؆୯ʹهड़Ͱ͖Δ
    Oboe

    View full-size slide

  21. w 0QFO4-&4͸ࠓߋ࢖͏ඞཁ΋ͳ͍ͷͰࠓճ͸""VEJPΛ࣮
    ૷ͯ͠Έ·͠ΐ͏
    ࣗ෼Ͱ࣮૷ͯ͠ΈΔ

    View full-size slide

  22. dlsymͰAAudioΛಡΈࠐΉ
    w MJCBBVEJPTP͸
    NJO4EL7FSTJPO͔Βೖ
    ͍ͬͯΔ

    View full-size slide

  23. InputStreamΛ४උ͢Δ
    void AAudio_IO::setupInputStream( AAudioStreamBuilder* builder ) {
    AAudioStreamBuilder_setDeviceId ( builder, _inputDeviceId );
    AAudioStreamBuilder_setSampleRate ( builder, _sampleRate );
    AAudioStreamBuilder_setChannelCount( builder, _inputChannelCount );
    AAudioStreamBuilder_setFormat ( builder, _sampleFormat );
    AAudioStreamBuilder_setSharingMode ( builder, AAUDIO_SHARING_MODE_EXCLUSIVE );
    AAudioStreamBuilder_setPerformanceMode( builder, AAUDIO_PERFORMANCE_MODE_LOW_LATENCY );
    AAudioStreamBuilder_setDirection ( builder, AAUDIO_DIRECTION_INPUT );
    AAudioStreamBuilder_setErrorCallback ( builder, ::errorCallback, this );
    }

    View full-size slide

  24. outputStreamΛ४උ͢Δ
    void AAudio_IO::setupOutputStream( AAudioStreamBuilder* builder ) {
    AAudioStreamBuilder_setDeviceId ( builder, _outputDeviceId );
    AAudioStreamBuilder_setFormat ( builder, _sampleFormat );
    AAudioStreamBuilder_setChannelCount( builder, _outputChannelCount );
    AAudioStreamBuilder_setSharingMode ( builder, AAUDIO_SHARING_MODE_EXCLUSIVE );
    AAudioStreamBuilder_setPerformanceMode( builder, AAUDIO_PERFORMANCE_MODE_LOW_LATENCY );
    AAudioStreamBuilder_setDirection ( builder, AAUDIO_DIRECTION_OUTPUT );
    AAudioStreamBuilder_setDataCallback ( builder, ::dataCallback, this );
    AAudioStreamBuilder_setErrorCallback ( builder, ::errorCallback, this );
    }

    View full-size slide

  25. inputStreamΛ࡞Δ
    void AAudio_IO::createInputStream() {
    AAudioStreamBuilder* builder = createStreamBuilder();
    setupInputStream( builder );
    aaudio_result_t result =
    static_cast(AAudioStreamBuilder_openStream( builder,
    &_inputStream ));
    AAudioStreamBuilder_delete( builder );
    }

    View full-size slide

  26. outputStreamΛ࡞Δ
    void AAudio_IO::createOutputStream() {
    AAudioStreamBuilder* builder = createStreamBuilder();
    setupOutputStream( builder );
    aaudio_result_t result = static_cast(AAudioStreamBuilder_openStream( builder, &_outputStream ));
    if ( result == AAUDIO_OK && _outputStream != nullptr ) {
    _sampleRate = AAudioStream_getSampleRate( _outputStream );
    _framesPerBurst = AAudioStream_getFramesPerBurst( _outputStream );
    AudioEngineProps::SAMPLE_RATE = _sampleRate;
    updateBufferSizeInFrames( AAudioStream_setBufferSizeInFrames( _outputStream, _framesPerBurst ));
    _underrunCountOutputStream = AAudioStream_getXRunCount( _outputStream );
    } else {
    return;
    }
    AAudioStreamBuilder_delete( builder );
    }

    View full-size slide

  27. streamΛ։࢝ɾఀࢭ͢Δ
    void AAudio_IO::startStream( AAudioStream* stream ) {
    aaudio_result_t result = static_cast(AAudioStream_requestStart( stream ));
    if ( result != AAUDIO_OK ) {
    return;
    }
    }
    void AAudio_IO::stopStream( AAudioStream* stream ) {
    if ( stream == nullptr ) {
    return;
    }
    aaudio_result_t result = static_cast(AAudioStream_requestStop( stream ));
    }

    View full-size slide

  28. w ""VEJP4USFBN#[email protected]Ͱɺ-08@-"5&/$:ʹઃఆ͢Δͱ௒௿஗ԆʹͳΔ͕
    όοςϦʔ͕ݮΔɻ͔͠΋ɺ͜Ε͚ͩͩͱ͋·Γҙຯ͕ͳ͍ʢޙड़ʣ
    w ""VEJP͸5ISFBE4BGFͰ͸ͳ͍ɻͦͷͨΊɺผεϨου͔Βಉ࣌ʹ""VEJPΛݺͼग़͢͜ͱ͸Ͱ͖ͳ
    ͍ɻ
    w ίʔϧόοΫؔ਺Λ࢖͍ɺඇಉظͰॲཧΛ͢Δ͜ͱΛਪ঑ɻ४උ͕Ͱ͖ͨΒίʔϧόοΫͰಡΈࠐΈɾ
    ॻ͖ࠐΈ͕Ͱ͖Δɻ
    w όοϑΝαΠζΛௐ੔͢Δ΂͖ɻ࠷ॳ͸5$1ͷ8JOEPX4J[F0QUJNJ[FͷΑ͏ʹঃʑʹόοϑΝαΠ
    ζΛେ͖͍ͯ͘͘͠ํ͕Α͍ɻ
    AAudioͷͭ·͖ͮϙΠϯτ

    View full-size slide

  29. w ""VEJPͷ-PX-BUFODZ.PEF͚ͩͩͱ௒௿஗Ԇʹ͸ͳΒͳ͍
    w "DUJWJUZͷ8JOEPX.BOBHFSʹ8JOEPX.BOBHFS-BZPVU1BSBNT'-"(@,&&1@4$3&&/@0/Λ
    ηοτ͢Δඞཁ͕͋Δ
    w BDUJWJUZHFU8JOEPX
    TFU4VTUBJOFE1FSGPSNBODF.PEF USVF
    ͰɺύϑΥʔϚϯεϞʔυΛ༗ޮʹ
    ͢Δ
    WindowManagerΛϋοΫ͢Δ

    View full-size slide

  30. inline void optimizeThreadPerformance( const std::vector& cpuIds )
    {
    cpu_set_t mask;
    pid_t current_thread_id = gettid();
    cpu_set_t cpu_set;
    CPU_ZERO( &cpu_set );
    if ( cpuIds.empty() ) {
    int current_cpu_id = sched_getcpu();
    CPU_SET( current_cpu_id, &cpu_set );
    } else {
    for ( size_t i = 0; i < cpuIds.size(); i++ ) {
    int cpu_id = cpuIds.at( i );
    CPU_SET( cpu_id, &cpu_set );
    }
    }
    }
    CPUΞϑΟχςΟΛεϨουʹઃఆ͢Δ
    w ΞϑΟχςΟΛઃఆ͢Δ͜ͱͰɺ$16ύϑΥʔ
    ϚϯεΛۃݶ·Ͱ޲্ͤ͞Δ͜ͱ͕Ͱ͖Δ
    w ϓϩηεɾεϨου͸04ʹΑͬͯͲΕ͔ͷ
    $16ίΞ্Ͱ࣮ߦ͞ΕΔɻ$16ΞϑΟχςΟΛ
    ઃఆ͢Δ͜ͱͰɺͦͷ࣮ߦ͞ΕΔ$16ίΞΛ೚
    ҙʹ੍ݶɾࢦఆ͢Δ͜ͱ͕Ͱ͖Δɻ

    View full-size slide

  31. w "OESPJEͷΦʔσΟΦͷྺ࢙͸௕͘ɺෳࡶͰ͋Δ
    w 0CPFΛͱΓ͋͑ͣ࢖͓͚ͬͯ͹Α͍
    w ۃݶ·ͰΧελϚΠζ͍ͨ͠ਓ͸ຊηογϣϯͷ಺༰Λࢀ
    ߟʹɺ࣮૷ͯ͠Έ͍ͯͩ͘͞
    ·ͱΊ

    View full-size slide