音声情報処理に便利な (Python) パッケージやソフトウェア

Tokyo BISH Bashの資料から独立させたもの

Transcript

1. Ի੠৘ใॲཧ΍Ի੠ม׵ʹ ศརͳ (Python) ύοέʔδ΍ ιϑτ΢ΣΞͨͪ 1 ࢖༻ײͳͲࢲݟΛؚΈ·͢
2. TPY
   
   QZTPY • ίϚϯυϥΠϯ͔ΒϑΥʔϚοτม׵ͳͲΛ͓खܰʹ • ϑΥʔϚοτม׵ʢwav to mp3 ͳͲʣ • ݁߹΍ϛοΫεɺτϦϛϯά ΋Մೳ • όονॲཧ΋ָʢγΣϧεΫϦϓτͳͲʣ • Pythonϥούʔ pysox΋͋Δ • Πϯετʔϧ • brew install sox ͳͲ • pip install sox ← pysox ͷΠϯετʔϧ͸͜Ε 2
3. MJCSPTBʢ͜Ε͸ຊ౰ʹΦεεϝʣ • Ի੠/Իָͷ෼ੳʹศརͳϞδϡʔϧ͕ଗͬͨύοέʔδ • ެࣜϚχϡΞϧɾνϡʔτϦΞϧͷॆ࣮΋ॿ͔Δ • ݸਓతʹΑ͘࢖͏ػೳ • ೾ܗදࣔɺεϖΫτϩάϥϜදࣔ • Ի੠ಛ௃ྔநग़ʢର਺ϝϧεϖΫτϩάϥϜʣ • Πϯετʔϧ pip install librosa • ެࣜϖʔδ https://librosa.org/librosa/index.html 3
4. 1Z8PSME • Ի੠ͷ෼ੳ࠶߹੒Λߦ͏Ϙίʔμʔͷύοέʔδ • Ի੠Λʮ੠৭ɾ੠ͷߴ͞ɾ੠ͷ͔͢Εʯͷ֤੒෼ʹ෼ղ͠࠶߹੒ • C++൛ͷPythonϥούʔ • Ի੠ͷಛ௃ྔநग़ʹ΋࢖͑ͯศར ⇒ PySPTKʢޙड़ʣΑΓ΋඼࣭ͷΑ͍εϖΫτϧแབྷ • Πϯετʔϧ pip install pyworld • ެࣜϖʔδ https://github.com/JeremyCCHsu/Python-Wrapper-for-World-Vocoder 4
5. 1Z"VEJP • ετϦʔϜ࿥Ի / ࠶ੜʹศརͳύοέʔδ • ϦΞϧλΠϜͷԻ੠ೖྗɾԻ੠ग़ྗʹ࢖͑Δ • ϦΞϧλΠϜԻ੠ม׵ with PythonͳͲ΋Մೳ • Πϯετʔϧ • pip install pyaudio ※ཁportaudio (e.g., brew install portaudio) 5
6. 1Z"VEJPͱ1Z8PSMEͷ૊Έ߹Θͤ • ؆қ൛ͷϘΠενΣϯδϟʔ • ؆қϘΠενΣϯδϟʔͷεΫϦϓτΛվྑɿPyQt5ͷεϥΠμʔʹΑΓ ϐονͱϑΥϧϚϯτΛϦΞϧλΠϜௐ੔͢ΔػೳΛ௥Ճʢฐϒϩάʣ • banibiku • Zoom഑৴޲͚ʹ̎࣍ݩΩϟϥʹͳΓ͖Δ͜ͱΛ໨ࢦͨ͠ϓϩδΣΫτ • scripts/voice_converter.py ͕ྑ͍ײ͡ͷϘΠενΣϯδϟʔ → ฐϒϩάͷαϯϓϧεΫϦϓτͷόάϑΟοΫεؚ͕·ΕΔ 6 https://tam5917.hatenablog.com/entry/2019/04/30/213321 https://github.com/peisuke/babiniku
7. 1Z415, • Ի੠৘ใॲཧπʔϧΩοτSPTKͷPythonϥούʔ • SPTKࣗମ͸LinuxίϚϯυ܈ • Իڹಛ௃ྔநग़ʹ࢖͏ͷ͕ศར • Ի੠෼ੳ߹੒΋Ͱ͖Δ͕ɺ඼࣭ࣗମ͸WORLDͷ΄͏্͕ • Πϯετʔϧ pip install pysptk • ެࣜϖʔδ https://pysptk.readthedocs.io/en/latest/ 7
8. OONOLXJJ <OBOBNJO LBXBJJ> • DNNԻ੠߹੒ʹ໾ཱͭϞδϡʔϧΛूΊͨύοέʔδ • ͲͪΒ͔ͱ͍͏ͱݚڀ༻్ • લॲཧ΍Իڹಛ௃ྔநग़ͷΫϥε͕Ұ௨Γଗ͍ͬͯΔ • ࿦จͷ࠶ݱ࣮૷Λ͢Δͱ͖ͳͲʹେ͍ʹ໾ཱͭ • Πϯετʔϧ pip install nnmnkwii • ެࣜϖʔδ https://r9y9.github.io/nnmnkwii/stable/index.html 8
9. 1ZEVC • Pydub • ೾ܗฤूʹศརͳϞδϡʔϧΛूΊͨύοέʔδ • αϙʔτ͢ΔϑΝΠϧܗࣜ΋๛෋ʢwav, mp3, mp4, wma, aac, ...ʣ • ػೳ ੾Γग़͠ɺ෼ׂɺϛοΫεɺϑΣʔυΠϯΞ΢τɺແԻૠೖɺͳͲͳͲ • Ұ෦ͷػೳ͸ pysoxͷ΄͏͕ߴ଎ͱ͍͏ӟ?ʢະ֬ೝʣ • Πϯετʔϧ pip install pydub • ެࣜϖʔδ http://pydub.com/ 9
10. TQSPDLFU • ౷ܭత੠࣭ม׵ͷͨΊͷπʔϧΩοτ (not ύοέʔδ) • ͲͪΒ͔ͱ͍͏ͱݚڀ༻ ʢMITϥΠηϯεʣ • ݚڀͷʮϕʔεϥΠϯʯߏஙʹ࠷ద • ެࣜϖʔδ https://github.com/k2kobayashi/sprocket • ղઆ࿦จ ʰ౷ܭత੠࣭ม׵ιϑτ΢ΣΞೖ໳ʱ https://www.jstage.jst.go.jp/article/isciesci/62/2/62_69/_article/-char/ja/ • νϡʔτϦΞϧ (εϥΠυ & notebook) https://github.com/kan-bayashi/INTERSPEECH19_TUTORIAL 10
11. "VEBDJUZ
    ೖΕ͓ͯ͘ͱ҆৺ • ϑϦʔͷ೾ܗฤूιϑτɺϚϧνϓϥοτϑΥʔϜ • ๛෋ͳα΢ϯυΤϑΣΫτՃ޻ػೳ • ެࣜϖʔδ https://www.audacityteam.org/ 11
12. (16্ͰԻ੠ॲཧ͍ͨ͠Ϛϯʹ ͓͢͢Ίͷύοέʔδ 12 ͓·͚
13. UPSDIBVEJP • Pytorchެ͕ࣜαϙʔτ͍ͯ͠ΔԻ੠ॲཧܥϥΠϒϥϦ • Pytorchܥͷਂ૚ֶशϞσϧͱͷ૬ੑ͕ྑ͍ʢͦΕ͸ͦ͏ʣ • ެࣜϖʔδ https://pytorch.org/audio/stable/index.html 13
14. UGTJHOBM • TensorFlowެ͕ࣜαϙʔτ͍ͯ͠ΔԻ੠ॲཧܥͷؔ਺܈ • TFܥͷਂ૚ֶशϞσϧͱͷ૬ੑ͕ྑ͍ʢͦΕ͸ͦ͏ʣ • FFT/iFFT, DCT, MDCT, STFTͳͲ • ެࣜϖʔδ https://www.tensorflow.org/api_docs/python/tf/signal 14
15. UPSDIMJCSPTB • PytorchΛόοΫΤϯυʹͯ͠librosaΛGPU্Ͱಈ͔͢ • Πϯετʔϧ pip install torchlibrosa • ެࣜϖʔδ https://github.com/qiuqiangkong/torchlibrosa 15
16. LBQSF • Kerasʢͱ͍͏͔TFʣΛόοΫΤϯυʹͯ͠Ի੠ॲཧ͢Δ • STFT΍iSTFTɺϝϧεϖΫτϩάϥϜͳͲ • CQTͳͲ͸ͳ͍ • ެࣜϖʔδ https://github.com/keunwoochoi/kapre • Πϯετʔϧ pip install kapre 16
17. OO"VEJP • PytorchΛόοΫΤϯυʹͯ͠STFTͳͲΛGPU্Ͱಈ͔͢ • STFTɺٯSTFTɺCQTͳͲΑ͘࢖͏ಛ௃நग़ܥ͕ଗ͏ • ެࣜϖʔδ https://github.com/KinWaiCheuk/nnAudio 17
18. OO"VEJPʢ͖ͭͮʣ • ൺֱද 18