Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tokyo BISH Bash #02 音声情報処理と音声変換技術入門

Tokyo BISH Bash #02 音声情報処理と音声変換技術入門

Akira Tamamori

June 30, 2020
Tweet

More Decks by Akira Tamamori

Other Decks in Research

Transcript

  1. Ի੠ͱ͸Կ͔ʁ • ۭؒΛ఻ΘΔʮ೾ʯ • ۭؾ෼ࢠͷૈີ͕఻ΘΔʢഏ → ޴ → ޱ→ۭؒʣ •

    ίϛϡχέʔγϣϯͷʮϝσΟΞʯ • ݴޠ΍ײ৘ɺ࿩ऀੑͳͲΛ఻͑Δഔମ • จԽɾܳज़ʹ͓͚Δʮ෇ՃՁ஋ʯ • ਓͷ৺Λಈ͔͢Վএ 5 Tokyo BISH Bash ! Tokyo BISH Bash ̇ Tokyo BISH Bash ͕ྑ͍Μͩͬͯ͞
  2. Ի੠ʹؚ·ΕΔ̏ͭͷ৘ใ 6 ݴޠ৘ใ ύϥݴޠ৘ใ ඇݴޠ৘ใ ςΩετԽ͕Մೳ • ςΩετԽ͕ෆՄೳ • ࿩ऀ͕ҙਤతʹੜ੒

    • ײ৘΍ଶ౓ͳͲ • ςΩετԽ͕ෆՄೳ • ࿩ऀ͕ҙਤ͠ͳ͍ • ೥ྸ΍ੑผɺ࿩ऀੑ Ի੠ تͼͷ ײ৘ ʮ"͞Μʯ ͱ͍͏৘ใ "͞Μ
  3. Ի੠ͷੜ੒աఔʢͬ͘͟Γͱʣ • ࣮͸؅ָثͱྨࣅ 12 ઉ΍ޱͷܗΛมܗ ੠ଳͷ։ดʹΑΔ ۭؾͷৼಈ ϛοΫεʂ Ի৭ Իߴ

    ੠ʹͳͬͯग़ͯ͘Δ ์ࣹ ʮ͜Μʹͪ͸ʯ https://ahcweb01.naist.jp/lecture/2018/sp/ material/20181108_slide.pdf ࢀߟ ͜Μʹͪ͸
  4. Ի੠ͷੜ੒աఔʢप೾਺෼ੳͷ؍఺͔Βʣ • प೾਺ಛੑදݱ 13 ύϫʔ प೾਺ Ի৭ͷ෇༩ʢڞ໐ʣ ͳΊΒ͔ ύϫʔ प೾਺

    Իߴͷੜ੒ʢԻݯʣ ΪβΪβ มௐ ύϫʔ प೾਺ प೾਺ಛੑ ˞ͳΊΒ͔੒෼ɿεϖΫτϧแབྷ ˞ΪβΪβ੒෼ɿඍࡉߏ଄ ˞มௐɿ৞ΈࠐΈૢ࡞
  5. Ի੠ੜ੒աఔͷϞσϧԽʢʣ • Իݯ৴߸ΛԻڹ؅ʹ௨͢⇒ Ի੠ੜ੒ʢϑΟϧλϦϯάʂʣ 19 ༗੠Ի ແ੠Ի Իݯ Իݯ৴߸ εϖΫτϧแབྷ

    Ի੠ੜ੒ ˎ ʹ Իڹ؅ʢεϖΫτϧแབྷʣ ߹੒೾ܗ Իݯ͸HTS slide ΑΓ http://hts.sp.nitech.ac.jp/?Download ݩͷԻ੠
  6. Ի੠෼ੳ͔ΒԻ੠ੜ੒·ͰͷྲྀΕ • 23 Ի੠෼ੳ εϖΫτϧ แབྷ جຊप೾਺ ඇपظੑ ࢦඪ Ի੠ੜ੒

    ᶃԻ੠Λ෼ੳͯ͠ ಛ௃ྔϕΫτϧͷ ܥྻΛखʹೖΕΔ ʜ ʜ ෼ੳ୯Ґ ʹϑϨʔϜ ᶅੜ੒͞Εͨ೾ܗΛ ॏͶ߹ΘͤΔ ᶄ֤ϑϨʔϜͷಛ௃ྔΛ Ϙίʔμʔʹ௨͢ ᶆݩͷԻ੠೾ܗ͕ ࠶ߏ੒͞ΕΔ ʜ ෼ੳ ੜ੒
  7. ύϥϝʔλ੍ޚʹΑΔԻ੠Ճ޻ʢʣ • ෼ղͨ͠Ի੠ಛ௃ྔͷՃ޻ 25 ੠৭ ʢεϖΫτϧแབྷʣ ੠ͷߴ͞ ʢجຊप೾਺ʣ ͔͢Ε۩߹ ʢඇपظੑࢦඪʣ

    Ի੠෼ੳ Ի੠ಛ௃ྔʢύϥϝʔλʣ Ի੠ੜ੒ Ͳ͔͜ͷ#͞Μ ʮ͜Μʹͪ͸ʯ ม׵ɾՃ޻ॲཧ "͞Μ ʮ͜Μʹͪ͸ʯ
  8. ύϥϝʔλ੍ޚʹΑΔԻ੠Ճ޻ʢʣ • ϑΥϧϚϯτγϑτ ⇒ ੠ಓͷ৳ॖ ʹରԠ 27 प೾਺ ύϫʔ ࠨʹγϑτ

    ӈʹγϑτ ͜΋ͬͨʁ੠ ෺ཧతͳղऍɿ • ੠ಓ͕୹͍ 㱺 ࢠڙঁੑ • ੠ಓ͕௕͍ 㱺 େਓஉੑ ੠ Ӊ஦ਓʁ੠ ੠ಓͷ୹ॖ ੠ಓͷ৳௕ ݩͷ੠
  9. σϞϯετϨʔγϣϯ ݚڀࣨͷֶੜ͞Μ͕࡞੒ͨ͠ΞϓϦ • εϥΠμʔͰύϥϝʔλ੍ޚ • ҎԼΛϕʔεͱͯ͠࡞੒ • $ ൛ 803-%

    ˞̍ • +6$&˞̎ 29 ˞̍ɿߴ඼࣭Ϙίʔμʔͷ̍ͭ ˞̎ɿΦʔσΟΦܥΞϓϦͷ ։ൃʹదͨ͠$ ϑϨʔϜϫʔΫ εϥΠμʔͰ ੍ޚ εϖΫτϧ දࣔ Ӷҙ։ൃதͰ͢ʂ
  10. ౷ܭత੠࣭ม׵ • ౷ܭϞσϧΛར༻ͯ͠Ի੠ಛ௃ྔΛม׵ ⇒ ಛ௃ྔͷରԠؔ܎ʢࣸ૾ʣΛେྔͷσʔλͰֶश͓ͯ͘͠ඞཁ 31 Ի੠෼ੳ Ի੠ੜ੒ #͞Μʮ͜Μʹͪ͸ʯ ౷ܭϞσϧ

    ʹΑΔม׵ "͞Μʮ͜Μʹͪ͸ʯ εϖΫτϧแབྷ جຊप೾਺ ඇपظੑࢦඪ • Ψ΢εࠞ߹Ϟσϧ • χϡʔϥϧωοτ ൃ࿩಺༰Λอͪͭͭ ࿩ऀͷ੠࣭Λม׵ https://www.slideshare.net/KentaroTachibana1/ss-94259438 ࢀߟ ϘίʔμʔʹΑΔ Ի੠ੜ੒
  11. ൃలͷํ޲ੑʢʣ • ͦͷ̍ɿϘίʔμʔͷԻ੠඼࣭Λྑͯ͘͠ੑೳ޲্ ⇒ χϡʔϥϧϘίʔμʔͷొ৔ʢ2017೥Ҏ߱ʣ ࢀߟʰԻ੠೾ܗ௚઀ੜ੒ϞσϧʮχϡʔϥϧϘίʔμʯͷൺֱʱ(2019.10) ͳͲ • ͦͷ̎ɿϘίʔμʔΛ࢖Θͣʹ੠࣭ม׵ʂ ࠩ෼εϖΫτϧิਖ਼ʹجͮ͘੠࣭ม׵

    ⇒ ϘίʔμʔʹΑΔԻ࣭ͷྼԽΛܰݮՄೳɺϦΞϧλΠϜࢤ޲ 32 https://www.slideshare.net/Takuma_OKAMOTO/ss-180895505 • ཁ๬ɿߴ඼࣭ɺߴ଎ͳ߹੒ɺֶश͕༰қɺলϝϞϦԽɺɺɺ ⇒ Ի੠ม׵γεςϜʹ΋૊ΈࠐΜͰੑೳ޲্Λ໨ࢦ͢
  12. ൃలͷํ޲ੑʢʣ • ͦͷ̎ɿԻ੠ͷ௥Ճऩ࿥ʹ͔͔ΔίετΛԼ͛Δʂ • ͦͷ̏ɿ੠࣭͚ͩͰͳ͘ײ৘΋ม׵ʂ • ͦͷ̐ɿ࿩͠੠͚ͩͰͳ͘Վ੠ͷ੠࣭΋ม׵ʂ • ͦͷ̑ɿϦΞϧλΠϜ͔ͭߴ඼࣭ͳ੠࣭ม׵ʂ ʮχϡʔϥϧωοτશ੝ʯͷ࣌୅

    • χϡʔϥϧωοτͷೖྗଆͱग़ྗଆʹԿΛ࣋ͬͯ͘Δ͔͕ΧΪ • σʔλ͑͞༻ҙͰ͖Ε͹ࣸ૾ֶ͕शͰ͖Δ͔΋ʢͰ͖Δͱ͸ݴͬͯͳ͍ʣ • ৴߸ॲཧʹجͮ͘ख๏͸͜Ε͔Β΋݈ࡏʢͷ͸ͣʣ 33
  13. TPYQZTPY • ίϚϯυϥΠϯ͔ΒϑΥʔϚοτม׵ͳͲΛ͓खܰʹ • ϑΥʔϚοτม׵ʢwav to mp3 ͳͲʣ • ݁߹΍ϛοΫεɺτϦϛϯά

    ΋Մೳ • όονॲཧ΋ָʢγΣϧεΫϦϓτͳͲʣ • Pythonϥούʔ pysox΋͋Δ • Πϯετʔϧ • brew install sox ͳͲ • pip install sox ← pysox ͷΠϯετʔϧ͸͜Ε 35
  14. MJCSPTBʢ͜Ε͸ຊ౰ʹΦεεϝʣ • Ի੠/Իָͷ෼ੳʹศརͳϞδϡʔϧ͕ଗͬͨύοέʔδ • ެࣜϚχϡΞϧɾνϡʔτϦΞϧͷॆ࣮΋ॿ͔Δ • ݸਓతʹΑ͘࢖͏ػೳ • ೾ܗදࣔɺεϖΫτϩάϥϜදࣔ •

    Ի੠ಛ௃ྔநग़ʢର਺ϝϧεϖΫτϩάϥϜʣ • Πϯετʔϧ pip install librosa • ެࣜϖʔδ https://librosa.org/librosa/index.html 36
  15. 1Z8PSME • Ի੠ͷ෼ੳ࠶߹੒Λߦ͏Ϙίʔμʔͷύοέʔδ • Ի੠Λʮ੠৭ɾ੠ͷߴ͞ɾ੠ͷ͔͢Εʯͷ֤੒෼ʹ෼ղ͠࠶߹੒ • C++൛ͷPythonϥούʔ • Ի੠ͷಛ௃ྔநग़ʹ΋࢖͑ͯศར ⇒

    PySPTKʢޙड़ʣΑΓ΋඼࣭ͷΑ͍εϖΫτϧแབྷ • Πϯετʔϧ pip install pyworld • ެࣜϖʔδ https://github.com/JeremyCCHsu/Python-Wrapper-for-World-Vocoder 37
  16. 1Z"VEJP • ετϦʔϜ࿥Ի / ࠶ੜʹศརͳύοέʔδ • ϦΞϧλΠϜͷԻ੠ೖྗɾԻ੠ग़ྗʹ࢖͑Δ • ϦΞϧλΠϜԻ੠ม׵ with

    PythonͳͲ΋Մೳ • Πϯετʔϧ • pip install pyaudio ཁportaudio (e.g., brew install portaudio) 38
  17. 1Z"VEJPͱ1Z8PSMEͷ૊Έ߹Θͤ • ؆қ൛ͷϘΠενΣϯδϟʔ • ؆қϘΠενΣϯδϟʔͷεΫϦϓτΛվྑɿPyQt5ͷεϥΠμʔʹΑΓ ϐονͱϑΥϧϚϯτΛϦΞϧλΠϜௐ੔͢ΔػೳΛ௥Ճʢฐϒϩάʣ • banibiku • Zoom഑৴޲͚ʹ̎࣍ݩΩϟϥʹͳΓ͖Δ͜ͱΛ໨ࢦͨ͠ϓϩδΣΫτ

    • scripts/voice_converter.py ͕ྑ͍ײ͡ͷϘΠενΣϯδϟʔ → ฐϒϩάͷαϯϓϧεΫϦϓτͷόάϑΟοΫεؚ͕·ΕΔ 39 https://tam5917.hatenablog.com/entry/2019/04/30/213321 https://github.com/peisuke/babiniku
  18. OONOLXJJ <OBOBNJO LBXBJJ> • DNNԻ੠߹੒ʹ໾ཱͭϞδϡʔϧΛूΊͨύοέʔδ • ͲͪΒ͔ͱ͍͏ͱݚڀ༻్ • લॲཧ΍Իڹಛ௃ྔநग़ͷΫϥε͕Ұ௨Γଗ͍ͬͯΔ •

    ࿦จͷ࠶ݱ࣮૷Λ͢Δͱ͖ͳͲʹେ͍ʹ໾ཱͭ • Πϯετʔϧ pip install nnmnkwii • ެࣜϖʔδ https://r9y9.github.io/nnmnkwii/stable/index.html 44
  19. 1ZEVC • Pydub • ೾ܗฤूʹศརͳϞδϡʔϧΛूΊͨύοέʔδ • αϙʔτ͢ΔϑΝΠϧܗࣜ΋๛෋ʢwav, mp3, mp4, wma,

    aac, ...ʣ • ػೳ ੾Γग़͠ɺ෼ׂɺϛοΫεɺϑΣʔυΠϯΞ΢τɺແԻૠೖɺͳͲͳͲ • Ұ෦ͷػೳ͸ pysoxͷ΄͏͕ߴ଎ͱ͍͏ӟ?ʢະ֬ೝʣ • Πϯετʔϧ pip install pydub • ެࣜϖʔδ http://pydub.com/ 45
  20. TQSPDLFU • ౷ܭత੠࣭ม׵ͷͨΊͷπʔϧΩοτ (not ύοέʔδ) • ͲͪΒ͔ͱ͍͏ͱݚڀ༻ ʢMITϥΠηϯεʣ • ݚڀͷʮϕʔεϥΠϯʯߏஙʹ࠷ద

    • ެࣜϖʔδ https://github.com/k2kobayashi/sprocket • ղઆ࿦จ ʰ౷ܭత੠࣭ม׵ιϑτ΢ΣΞೖ໳ʱ https://www.jstage.jst.go.jp/article/isciesci/62/2/62_69/_article/-char/ja/ • νϡʔτϦΞϧ (εϥΠυ & notebook) https://github.com/kan-bayashi/INTERSPEECH19_TUTORIAL 46
  21. ͦͷଞ • Voidol • ϦΞϧλΠϜ੠࣭ม׵ʢ༗ྉʣ • ࣗ෼ͷ੠ΛλʔήοτͷΩϟϥʹม׵ • Realtime Yukarin

    • ϦΞϧλΠϜ੠࣭ม׵ • Gachikoe! Core • ϦΞϧλΠϜ੠࣭ม׵ 48 https://crimsontech.jp/apps/voidol/ https://blog.hiroshiba.jp/realtime-yukarin-introduction/ https://booth.pm/ja/items/1236505
  22. ͓·͚ • GoogleԻ੠߹੒ɺԻ੠ೝࣝɺ຋༁APIͷPythonϥούʔ ⇒ gTTS, SpeechRecognition, googletrans • ͦΕͧΕΛ͓ࢼ͠Ͱ࢖ͬͯΈΔʹ͸े෼ •

    ࢀߟ GoogleԻ੠ೝࣝͷ݁ՌΛGoogle຋༁͠ɺGoogle Text-to-SpeechͰ Ի੠ʹ໭͢PythonεΫϦϓτʢฐϒϩάʣ https://tam5917.hatenablog.com/entry/2019/04/28/191946 49