An attempt to reproduce WaveNet-based text-to-speech synthesis

MACHINE LEARNING Meetup KANSAI #2 on June 15. https://mlm-kansai.connpass.com/event/88458/

音声サンプル:
- https://r9y9.github.io/wavenet_vocoder/
- https://r9y9.github.io/blog/2018/05/20/tacotron2/

Transcript

1. 8BWF/FUח״׷窟鎘涸갈㡮さ䧭ח䮋䨌׃׋鑧
   2018/06/15 Ryuichi Yamamoto @ LINE Corp.
   MACHINE LEARNING Meetup KANSAI
   1
   

   View Slide

2. 5FYUUPTQFFDITZOUIFTJT
   2
   5FYU
   

   View Slide

3. 荈䊹稱➜ with my own text-to-speech (TTS)
   )J NZOBNFJT3ZVJDIJ:BNBNPUP*NBTPGUXBSFFOHJOFFSXPSLJOHBU-*/&
   DPSQPSBUJPO ,ZPUP
   3
   

   View Slide

4. WaveNet: 3遤תה׭
   • (PPHMF%FFQ.JOEח״׏ג涪僇
   • 傀㶷ך405"׾㣐ֹֻ♳㔐׷荈搫䚍ך넝ְㅷ颵
   • 堣唒㷕统ك٦أך窟鎘涸갈㡮さ䧭ٌرٕ
   4
   https://deepmind.com/blog/wavenet-generative-model-raw-audio/
   

   View Slide

5. WaveNet: 岚䕎ٖكٕך荈䊹㔐䌓㘗欰䧭ٌرٕ
   5
   

   View Slide

6. ֶ⠗ִ׃׋ְֿה
   • 갈㡮さ䧭UFYUUPTQFFDIכ(16אד׮㨣׭׵׸׷
   • 8BWF/FUח״׷넝ㅷ颵ז갈㡮さ䧭׮갹䓸׸ל⦐➂ד׮〳腉
   6
   

   View Slide

7. ➭䩛岀הך嫰鯰 (1/2)
   7
   Text: Scientists at the CERN laboratory say they have discovered a
   new particle
   Deep Voice3 [Ping ‘18]. (trained on LJSpeech, w/o WN)
   Tacotron 2 [Shen; ‘18]. (trained on LJSpeech , w/ WN)
   Tacotron 2 [Shen; ’18]. (trained on proprietary corpus , w/ WN)
   

   View Slide

8. ➭䩛岀הך嫰鯰 (2/2)
   8
   Text: Generative adversarial network or variational auto-encoder.
   Deep Voice3 [Ping ‘18]. (trained on LJSpeech, w/o WN)
   Tacotron 2 [Shen; ‘18]. (trained on LJSpeech , w/ WN)
   Tacotron 2 [Shen; ’18]. (trained on proprietary corpus , w/ WN)
   

   View Slide

9. Ꟛ涪ך酅⩎
   9
   

   View Slide

10. GPUך侧ָ駈׶זְ㉏겗
    10
    https://arxiv.org/abs/1712.05884
    

    View Slide

11. GPUך侧ָ駈׶זְ㉏겗ח㼎ׅ׷鍑瘶
    11
    • 孡さ
    – 〴 (595Jד갹䓸׶ת׃׋
    – ָծ湫䠬דְֽ׷ה䙼׏׋ךדծװ׶ֹת׃׋
    

    View Slide

12. 㷕统ח֮ת׶ח儗꟦ַַָ׷㉏겗
    • ⡦傈ַַ׏׋ך٥٥٥
    – رٌ갈㡮ٖكٕך넝ㅷ颵ח麦ׅ׷חכ鹈꟦ֻ׵ְכ䗳銲L
    – 傈ד׮ TPGUNBY׾⢪ִל
    ׉׸ז׶ךㅷ颵חכז׷
    • 鎘皾׃גְ׷꟦זח׃גְ׋ך٥٥٥
    – ˘Ī˘
    ˘
    12
    

    View Slide

13. 䱿锷ח֮ת׶ח儗꟦ַַָ׷㉏겗
    • 猱ך갈㡮ךさ䧭ח ⴓ PO(PPHMF$PMBCPSBUPSZ
    
    – ˘Ī˘
    ˘
    13
    

    View Slide

14. ֶ⠗ִ׃׋ַ׏׋ֿה
    • 갈㡮さ䧭כ(16אַ׵㨣׭׵׸תׅ 8BWF/FUכ㼰׃鳞ְדָׅ
    
    • 8BWF/FUדכꬊ䌢ח넝ㅷ颵ז갈㡮さ䧭ָ〳腉
    – 剑鵚ך،فٗ٦ثכ
    سً؎ٝ濼陎׉׿זחְ׵זְךדծٖحخزٓ؎
    14
    

    View Slide

15. 剑䖓ח
    15
    Thank you for coming to the machine learning meetup Kansai on June 15!
    

    View Slide

16. References
    • <%FFQ.JOEˎ>8BWF/FU"(FOFSBUJWF.PEFMGPS3BX"VEJP IUUQTEFFQNJOEDPNCMPHXBWFOFUHFOFSBUJWF
    NPEFMSBXBVEJP
    • "BSPOWBOEFO0PSE 4BOEFS%JFMFNBO )FJHB ;FO FUBM 8BWF/FU"(FOFSBUJWF.PEFMGPS
    3BX"VEJP BS9JW 4FQ
    • <1JOHˎ>8FJ1JOH ,BJOBO 1FOH "OESFX(JCJBOTLZ FUBM ˑ%FFQ7PJDF4DBMJOH5FYUUP4QFFDIXJUI
    $POWPMVUJPOBM4FRVFODF-FBSOJOH˒ 1SPDPG*$-3
    • <4IFOˏ>+POBUIBO4IFO 3VPNJOH 1BOH 3PO+8FJTT FUBM /BUVSBM5544ZOUIFTJTCZ$POEJUJPOJOH8BWF/FU
    PO.FM4QFDUSPHSBN1SFEJDUJPOT 1SPDPG*$"441
    16
    

    View Slide