Pro Yearly is on sale from $80 to $50! »

今日から分かる Android Audioの全て

今日から分かる Android Audioの全て

Cb4d62dd7c1d325863696c45e9b6484d?s=128

Shumpei Urabe

October 26, 2020
Tweet

Transcript

  1. ࠓ೔͔Β෼͔Δ Android Audioͷશͯ 2020.10.26 ߹ಉձࣾφΫα
 ઎෦ ॡฏ

  2. ࣗݾ঺հ ઎෦ ॡฏ • ߹ಉձࣾφΫαɹ୅ද • ΤϯλϝͷϥΠϒ഑৴Λࢧ͍͑ͯ ·͢ • झຯ͸࡞ۂͱωοτϫʔΫӡ༻

    • Android։ൃͰҰ൪࢖͍ͬͯΔ΋ͷ ͸MediaCodecͱVulkan
  3. Pixel 3 ໰୊ w 1JYFMͰ͸ɺ""VEJP͕ݪҼͰ࠶ੜ͞Ε ͳ͍໰୊͕͋ͬͨ

  4. ձࣾ঺հ 1.

  5. w ೥݄ʹઃཱ w ຊࣾ͸౦ژ౎ઍ୅ా۠ਆాۋொ w ΦϑΟεʹϥοΫ͕͋Δ w ϝϯόʔ਺͸໊ w ࠾༻ืूதʂ

    ߹ಉձࣾφΫα
  6. w &/5&35"*.&/5ʷ5&$)/0-0(: w ΤϯλʔςΠϝϯτͷ෼໺Λٕज़Λ࢖ͬͯ໰୊ղܾΛ͢Δ ձࣾ w ΢Σϒ੍࡞͔Βಈը഑৴·ͰԿͰ΋΍Γ·͢ φΫαͷࣄۀ

  7. w ϥΠϒ഑৴αʔϏε w ిࢠॻ੶αʔϏε w ͳͲͳͲ ࣄۀྫ

  8. w /-*7&͸ɺφΫα͕ఏڙ͍ͯ͠ΔιϦϡʔγϣϯͷҰͭɻ w /-*7&͸ɺΫϥΠΞϯτ͔ΒΠϯϑϥ·ͰશͯφΫα͕Χ όʔ͢Δ͜ͱ͕ग़དྷ·͢ w ͦͷଞɺ%3.ͷରԠͳͲ΋αϙʔτ͢Δ͜ͱ͕ՄೳͰ͢ ϥΠϒ഑৴ιϦϡʔγϣϯ NLIVE

  9. w ڵຯΛ࣋ͬͨਓ͸๻ʹ࿈བྷ͍ͯͩ͘͠͞ w ۀ຿ҕୗɾΞϧόΠτ΋େ׻ܴͰ͢ We're Hiring!

  10. AndroidͷAudioͷྺ࢙ 2.

  11. w "VEJP'MJOHFS͸"VEJP4FSWFSͷͭͰ͋ΓɺΞϓϦͷԻΛ ؅ཧ͢Δαʔό w "VEJP'MJOHFS͸"35 "OESPJE3VOUJNF ্Ͱಈ࡞͢Δ w "VEJP'MJOHFS͔Β)"- )BSEXBSF"CTUSBDU-BZFS

    ʹॲ ཧ͞Εͯɺ࿥Ի΍࠶ੜΛߦ͏ AudioFlinger
  12. w "VEJP'MJOHFS͕"OESPJEͷ஗Ԇ໰୊ͷ தͰ͸ಛʹӨڹ͕େ͖͍ w ͦͷཧ༝͸"VEJP'MJOHFSͰόοϑΝΛ ࡞Γ௚͔͢ΒͰ͋Δ AudioFlingerͷ10ms໰୊

  13. w "VEJP'MJOHFSͷલʹ͸)"-͕͋Γɺ )"-͕"-4"ͱ΍ΓͱΓΛͯ͠σʔλΛ ૹड৴͢Δ w "-4"͸ϦϯάόοϑΝΛ࢖ͬͯɺυϥ Πόʔ͔Β"VEJP'MJOHFSʹૹ͍ͬͯΔ AudioFlingerͷ10ms໰୊

  14. w "VEJP'MJOHFSͷޙʹ͸"VEJP3FDPSE ΍"VEJP5SBDL͕͋Δ w ͪ͜Β΋#ZUF#VGGFSΛ࢖ͬͯɺόο ϑΝΛ࡞ͬͯॻ͖ࠐΉॲཧ͕͋Δͷ Ͱɺ஗Ԇ͕͋Δ AudioFlingerͷ10ms໰୊

  15. w "VEJP"1*ͷඪ४࣮૷͕0QFO4-&4 w 0QFO4-&4Λ࢖͏͜ͱͰɺ"VEJP'MJOHFSΑΓ΋௿஗Ԇʹ ࿥Ի΍࠶ੜ͕Ͱ͖Δ OpenSL ES

  16. w 0QFO4-&4ͷ໰୊఺͸ɺσόΠεʹΑͬͯ͸ඇରԠͳ͜ ͱ͕ଟ͍ʢυϥΠόʔͷඇαϙʔτʣ w ·ͨɺ0QFO4-&4ͷ࣮૷΋͔ͳΓεύήοςΟίʔυʹ ͳ͓ͬͯΓɺ࣌ʑෆ҆ఆʹͳΔ͜ͱ΋Α͋͘Δ OpenSL ES͕࢖͍෺ʹͳΒͳ͍

  17. w "OESPJEΑΓɺ0QFO4-&4ͷ୅ସͱͯ͠""VEJP͕ొ ৔ͨ͠ w ""VEJPͰ͸ɺ͜Ε·Ͱ"VEJP'MJOHFSͷϥούʔతͳ໾ׂ Λ͍ͯͨ͠0QFO4-&4ͱ͸ҧ͍ɺ௚઀)"-ʹΞΫηεͰ ͖Δ AAudio

  18. w ""VEJP͸0QFO4-&4ಉ༷ʹ$ Ͱهड़Λߦ͏ͨΊɺ "35ͷ($֎Ͱ͋ΔʢͦͷͨΊϝϞϦղ์͸ࣗ෼Ͱ΍Δඞ ཁ͕͋Δʣ w ""VEJPͰ͸ɺ֤ϕϯμʔͱڠྗ͠ɺ҆ఆͯ͠ಈ࡞͢Δ͜ ͱΛ໨ࢦͨ͠ AAudio

  19. Implementaion 3.

  20. w (PPHMF͕։ൃͨ͠ϥΠϒϥϦ w 0QFO4-&4ͱ""VEJPͷ૬ޓར༻Λ ߦ͏ͨΊͷϥούʔ w ࣗ෼Ͱॻ͘ͷ͕ͭΒ͍ਓ͸0CPFΛ ࢖͏ͱ؆୯ʹهड़Ͱ͖Δ Oboe

  21. w 0QFO4-&4͸ࠓߋ࢖͏ඞཁ΋ͳ͍ͷͰࠓճ͸""VEJPΛ࣮ ૷ͯ͠Έ·͠ΐ͏ ࣗ෼Ͱ࣮૷ͯ͠ΈΔ

  22. dlsymͰAAudioΛಡΈࠐΉ w MJCBBVEJPTP͸ NJO4EL7FSTJPO͔Βೖ ͍ͬͯΔ

  23. InputStreamΛ४උ͢Δ void AAudio_IO::setupInputStream( AAudioStreamBuilder* builder ) { AAudioStreamBuilder_setDeviceId ( builder,

    _inputDeviceId ); AAudioStreamBuilder_setSampleRate ( builder, _sampleRate ); AAudioStreamBuilder_setChannelCount( builder, _inputChannelCount ); AAudioStreamBuilder_setFormat ( builder, _sampleFormat ); AAudioStreamBuilder_setSharingMode ( builder, AAUDIO_SHARING_MODE_EXCLUSIVE ); AAudioStreamBuilder_setPerformanceMode( builder, AAUDIO_PERFORMANCE_MODE_LOW_LATENCY ); AAudioStreamBuilder_setDirection ( builder, AAUDIO_DIRECTION_INPUT ); AAudioStreamBuilder_setErrorCallback ( builder, ::errorCallback, this ); }
  24. outputStreamΛ४උ͢Δ void AAudio_IO::setupOutputStream( AAudioStreamBuilder* builder ) { AAudioStreamBuilder_setDeviceId ( builder,

    _outputDeviceId ); AAudioStreamBuilder_setFormat ( builder, _sampleFormat ); AAudioStreamBuilder_setChannelCount( builder, _outputChannelCount ); AAudioStreamBuilder_setSharingMode ( builder, AAUDIO_SHARING_MODE_EXCLUSIVE ); AAudioStreamBuilder_setPerformanceMode( builder, AAUDIO_PERFORMANCE_MODE_LOW_LATENCY ); AAudioStreamBuilder_setDirection ( builder, AAUDIO_DIRECTION_OUTPUT ); AAudioStreamBuilder_setDataCallback ( builder, ::dataCallback, this ); AAudioStreamBuilder_setErrorCallback ( builder, ::errorCallback, this ); }
  25. inputStreamΛ࡞Δ void AAudio_IO::createInputStream() { AAudioStreamBuilder* builder = createStreamBuilder(); setupInputStream( builder

    ); aaudio_result_t result = static_cast<aaudio_result_t>(AAudioStreamBuilder_openStream( builder, &_inputStream )); AAudioStreamBuilder_delete( builder ); }
  26. outputStreamΛ࡞Δ void AAudio_IO::createOutputStream() { AAudioStreamBuilder* builder = createStreamBuilder(); setupOutputStream( builder

    ); aaudio_result_t result = static_cast<aaudio_result_t>(AAudioStreamBuilder_openStream( builder, &_outputStream )); if ( result == AAUDIO_OK && _outputStream != nullptr ) { _sampleRate = AAudioStream_getSampleRate( _outputStream ); _framesPerBurst = AAudioStream_getFramesPerBurst( _outputStream ); AudioEngineProps::SAMPLE_RATE = _sampleRate; updateBufferSizeInFrames( AAudioStream_setBufferSizeInFrames( _outputStream, _framesPerBurst )); _underrunCountOutputStream = AAudioStream_getXRunCount( _outputStream ); } else { return; } AAudioStreamBuilder_delete( builder ); }
  27. streamΛ։࢝ɾఀࢭ͢Δ void AAudio_IO::startStream( AAudioStream* stream ) { aaudio_result_t result =

    static_cast<aaudio_result_t>(AAudioStream_requestStart( stream )); if ( result != AAUDIO_OK ) { return; } } void AAudio_IO::stopStream( AAudioStream* stream ) { if ( stream == nullptr ) { return; } aaudio_result_t result = static_cast<aaudio_result_t>(AAudioStream_requestStop( stream )); }
  28. w ""VEJP4USFBN#VJMEFS@TFU1FSGPSNBODF.PEFͰɺ-08@-"5&/$:ʹઃఆ͢Δͱ௒௿஗ԆʹͳΔ͕ όοςϦʔ͕ݮΔɻ͔͠΋ɺ͜Ε͚ͩͩͱ͋·Γҙຯ͕ͳ͍ʢޙड़ʣ w ""VEJP͸5ISFBE4BGFͰ͸ͳ͍ɻͦͷͨΊɺผεϨου͔Βಉ࣌ʹ""VEJPΛݺͼग़͢͜ͱ͸Ͱ͖ͳ ͍ɻ w ίʔϧόοΫؔ਺Λ࢖͍ɺඇಉظͰॲཧΛ͢Δ͜ͱΛਪ঑ɻ४උ͕Ͱ͖ͨΒίʔϧόοΫͰಡΈࠐΈɾ ॻ͖ࠐΈ͕Ͱ͖Δɻ w

    όοϑΝαΠζΛௐ੔͢Δ΂͖ɻ࠷ॳ͸5$1ͷ8JOEPX4J[F0QUJNJ[FͷΑ͏ʹঃʑʹόοϑΝαΠ ζΛେ͖͍ͯ͘͘͠ํ͕Α͍ɻ AAudioͷͭ·͖ͮϙΠϯτ
  29. w ""VEJPͷ-PX-BUFODZ.PEF͚ͩͩͱ௒௿஗Ԇʹ͸ͳΒͳ͍ w "DUJWJUZͷ8JOEPX.BOBHFSʹ8JOEPX.BOBHFS-BZPVU1BSBNT'-"(@,&&1@4$3&&/@0/Λ ηοτ͢Δඞཁ͕͋Δ w BDUJWJUZHFU8JOEPX TFU4VTUBJOFE1FSGPSNBODF.PEF USVF ͰɺύϑΥʔϚϯεϞʔυΛ༗ޮʹ

    ͢Δ WindowManagerΛϋοΫ͢Δ
  30. inline void optimizeThreadPerformance( const std::vector<int>& cpuIds ) { cpu_set_t mask;

    pid_t current_thread_id = gettid(); cpu_set_t cpu_set; CPU_ZERO( &cpu_set ); if ( cpuIds.empty() ) { int current_cpu_id = sched_getcpu(); CPU_SET( current_cpu_id, &cpu_set ); } else { for ( size_t i = 0; i < cpuIds.size(); i++ ) { int cpu_id = cpuIds.at( i ); CPU_SET( cpu_id, &cpu_set ); } } } CPUΞϑΟχςΟΛεϨουʹઃఆ͢Δ w ΞϑΟχςΟΛઃఆ͢Δ͜ͱͰɺ$16ύϑΥʔ ϚϯεΛۃݶ·Ͱ޲্ͤ͞Δ͜ͱ͕Ͱ͖Δ w ϓϩηεɾεϨου͸04ʹΑͬͯͲΕ͔ͷ $16ίΞ্Ͱ࣮ߦ͞ΕΔɻ$16ΞϑΟχςΟΛ ઃఆ͢Δ͜ͱͰɺͦͷ࣮ߦ͞ΕΔ$16ίΞΛ೚ ҙʹ੍ݶɾࢦఆ͢Δ͜ͱ͕Ͱ͖Δɻ
  31. ·ͱΊ 4.

  32. w "OESPJEͷΦʔσΟΦͷྺ࢙͸௕͘ɺෳࡶͰ͋Δ w 0CPFΛͱΓ͋͑ͣ࢖͓͚ͬͯ͹Α͍ w ۃݶ·ͰΧελϚΠζ͍ͨ͠ਓ͸ຊηογϣϯͷ಺༰Λࢀ ߟʹɺ࣮૷ͯ͠Έ͍ͯͩ͘͞ ·ͱΊ