論文サタデーナイト#01 SEMI-SUPERVISED LEARNING USING TEACHER-STUDENT MODELS FOR VOCAL MELODY EXTRACTION

2021年から毎週土曜に論文を読もうという試みで読みました．
ISMIR2020の半教師ありMelody Extractionの論文です．
Semi-supervised learning using teacher-student models for vocal melody extraction
Sangeun Kum, Jing-Hua Lin, Li Su, Juhan Nam
https://program.ismir2020.net/poster_1-11.html

Transcript

1. 4FNJTVQFSWJTFEMFBSOJOHVTJOH
   UFBDIFSTUVEFOUNPEFMTGPSWPDBM
   NFMPEZFYUSBDUJPO
   4BOHFVO,VN +JOH)VB-JO -J4V +VIBO/BN
   *4.*3
   ஜ೾େֶɹࢁຊ༤໵
   8FFLMZTVSWFZ
   

   View Slide

2. ԿΛ͔ͨ͠
   w .FMPEZ&YUSBDUJPOʹ͓͍ͯ/PJTZ4UVEFOUʹΑΔ൒ڭࢣ͋ΓֶशΛ༻͍
   ͨ
   w ڭࢣ͋Γֶशͷ৔߹ͱൺ΂ͯਖ਼ղ཰͕վળ
   w طଘख๏ͱͷൺֱʹ͓͍ͯ΋ɼ4P5"ͱḮ৭ͳ͍݁ՌΛࣔͨ͠
   

   View Slide

3. .FMPEZ&YUSBDUJPOͱ͸ʁ
   ൐૗෇͖ͷԻݯ͔ΒϝϩσΟϥΠϯΛݕग़͢Δ՝୊
   w Իݯ෼཭ͳͲΛڬ·ͣࠞ߹Ի͔Β௚઀ɼϝϩσΟͷ'Λ୳͢
   w ΞϊςʔγϣϯΛ͚ͭΔͷ͕೉͘͠ɼϥϕϧσʔλ͕ෆ଍͍ͯ͠Δ
   ൐૗෇͖ϘʔΧϧʢεϖΫτϩάϥϜʣ
   .FMPEZ&YUSBDUJPOͷ݁Ռʢ੨ઢʣ
   

   View Slide

4. ΩʔΞΠσΞɿ൒ڭࢣ͋Γֶशͱσʔλ֦ு
   ʙϥϕϧσʔλෆ଍ʹରԠ͢ΔͨΊʹʙ
   w Ξϊςʔγϣϯ͕͍ͭͨԻݯσʔλ͸গͳ͍͚Ͳɼɹɹɹɹɹɹɹɹɹɹɹɹɹɹ
   Իݯ͚ͩʢʹϥϕϧͳ͠σʔλʣͳΒ୔ࢁ༻ҙͰ͖ͦ͏ʂɹɹɹɹɹɹɹɹɹ
   ˠ൒ڭࢣ͋Γֶश͕࢖͑ΔͷͰ͸ʁ
   w ঎༻ԻָͷϘʔΧϧ͸ϛΩγϯάͷஈ֊Ͱ༷ʑͳΤϑΣΫτ͕͚͕͞ΕΔɹɹɹ
   ˠઌߦݚڀͰ΋ϐονγϑτ౳ʹΑΔσʔλ֦ு<,VN`ଞ>͕༻͍ΒΕ
   ͍ͯΔɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ
   ˠσʔλ֦ு΋׆༻Ͱ͖ͦ͏
   <,VN`>4,VNFUBMl.FMPEZFYUSBDUJPOPOWPDBMTFHNFOUTVTJOHNVMUJDPMVNOEFFQOFVSBMOFUXPSLT z*4.*3
   

   View Slide

5. ΩʔΞΠσΞɿ/PJTZ4UVEFOU<9JFb>
   ʙσʔλ֦ுΛར༻ͨ͠൒ڭࢣ͋Γֶशʙ
   w σʔλ֦ுΛར༻ͨ͠ɼ5FBDIFS
   4UVEFOU1BSBEJHNʹΑΔ൒ڭࢣ͋Γ
   ֶश
   w 4UVEFOUϞσϧʹ͸σʔλ֦ுΛ
   ࢪٖͨ͠ࣅϥϕϧσʔλͰֶश͠ɹɹɹɹ
   ϊΠζ΁ͷϩόετੑΛ֫ಘ͢Δ
   <9JF`>29JFFUBMl4FMGUSBJOJOHXJUIOPJTZTUVEFOUJNQSPWFTJNBHFOFUDMBTTJpDBUJPOz$713
   

   View Slide

6. σʔλ֦ுͷख๏
   3BOE"VEJP"VHNFOU
   w /PJTZ4UVEFOUͷ3BOEPN"VHNFOUͷɹɹɹɹ
   Իόʔδϣϯ
   w ϥϯμϜͳൺ཰ͰҎԼͷΤϑΣΫτΛ͔͚Δ
   w ϑΟϧλϦϯά
   w ΠίϥΠδϯά
   w ΦʔόʔυϥΠϒ
   w Ϧόʔϒ
   w ϑΣʔβʔ
   w ΤϑΣΫτ͸QZTOEGYʢIUUQTQZQJPSH
   QSPKFDUQZTOEGYʣΛར༻
   

   View Slide

7. ֶशͷΞϧΰϦζϜ
   w 3"" ɾ
   ɿ3BOE"VEJP"VHNFOU
   w %ɿϥϕϧ͖ͭσʔλ
   w 6ɿϥϕϧͳ͠σʔλ
   w 6UJMEFɿٙࣅϥϕϧ͕͚ͭΒΕͨ
   ϥϕϧͳ͠σʔλ
   w Lɿ/PJTZ4UVEFOUͷ܁Γฦ͠ճ਺
   

   View Slide

8. ࣮ݧ
   ͭͷϞσϧΛൺֱ
   w ڭࢣ͋Γʢ4VQFSWJTFE0OMZʣ
   w ൒ڭࢣ͋Γɾ%"ͳ͠ʢ#BTJD5FBDIFS4UVEFOUʣ
   w ൒ڭࢣ͋Γɾ%"͋Γʢ/PJTZ5FBDIFS4UVEFOUʣ
   w /PJTZ4UVEFOU
   

   View Slide

9. ͦΕͧΕͷଛࣦؔ਺
   В@TɿϞσϧͷύϥϝʔλ
   Y@Eɿೖྗ
   Z@Eɿਖ਼ղϥϕϧ
   )ɿΫϩεΤϯτϩϐʔଛࣦؔ਺
   /ɿϥϕϧ͖ͭσʔλͷྔ
   Y@Vɿೖྗʢϥϕϧͳ͠σʔλʣ
   Z@Vɿٖࣅϥϕϧ
   .ɿϥϕϧͳ͠σʔλͷྔ
   \Y@V^UJMEFɿೖྗʢσʔλ֦ுͨ͠ϥϕϧͳ͠σʔλʣ
   \Z@V^UJMEFɿσʔλ֦ுͨ͠σʔλΛೖྗͯ͠ಘٖͨࣅϥϕϧ
   

   View Slide

10. ֶशͤ͞ΔϞσϧ
    w ஶऀΒ͕Ҏલʹ༻͍͍ͯͨ
    .FMPEZ&YUSBDUJPOΛߦ͏$3//
    <,VN`>
    w ϝϩσΟͷ༗ແΛ౰ͯΔλεΫͱͷ
    ౷߹Ϟσϧ
    w εϖΫτϩάϥϜ͕ೖྗ
    <,VN`>4,VNFUBMl+PJOUEFUFDUJPOBOEDMBTTJpDBUJPOPGTJOHJOHWPJDFNFMPEZVTJOHDPOWPMVUJPOBMSFDVSSFOUOFVSBMOFUXPSLTz"QQMJFE4DJFODF
    

    View Slide

11. ࣮ݧ
    ͭͷઃఆΛൺֱ
    ϥϕϧͳ͠σʔλͱϥϕϧ͖ͭσʔλͷֶशํ๏Λൺֱ
    ֶश͢ΔσʔλͷαΠζΛม͑ͯൺֱ
    /PJTZ4UVEFOUͷ܁Γฦ͠ճ਺Λม͑ͯൺֱ
    .FMPEZ&YUSBDUJPOͷطଘख๏ͱͷൺֱ
    

    View Slide

12. ༻͍ͨσʔληοτ
    <4DIMÛUFS`>+4DIMÛUFSFUBMl&YQMPSJOHEBUBBVHNFOUBUJPOGPSJNQSPWFETJOHJOHWPJDFEFUFDUJPOXJUIOFVSBMOFUXPSLTz*4.*3
    w 6OMBCFMFEͷεϥογϡͷࠨͷ਺஋
    ͸<4DIMÛUFS`>ʹΑͬͯɹɹɹɹ
    ϘʔΧϧͷׂ߹͕Ҏ্ͷۂΛɹɹ
    બग़ͨ݁͠Ռ
    

    View Slide

13. ධՁج४
    w 3BX1JUDI"DDVSBDZ 31"
    ɿݕग़ͨ͠ϝϩσΟͷԻߴ͕œ൒ԻҎ಺ʹׂ͋ͬͨ߹
    w 0WFSBMM"DDVSBDZ 0"
    ɿϝϩσΟ༗ແͱԻߴͷಉఆͷਖ਼ղ཰
    w 7PJDJOH3FDBMM 73
    ɿϝϩσΟ༗ແͷݕग़ͷ3FDBMMɼऔΓ͜΅͠ͷͳ͞ɹɹɹɹɹɹɹɹɹ
    ʢߴ͍ఔྑ͍ʣ
    w 7PJDJOH'BMTF"MBSN 7'"
    ɿϝϩσΟ͕ͳ͍෦෼ΛϝϩσΟ͋Γͱݕग़ׂͨ͠߹ɹɹɹ
    ʢ௿͍ఔྑ͍ʣ
    

    View Slide

14. ࣮ݧɽख๏ͷൺֱ
    w 0"ͰධՁ
    w શͯͷσʔληοτͷฏۉͰ͸
    /PJTZ4UVEFOU͕ڭࢣ͋ΓΛ
    ɼɹɹɹɹɹɹɹɹɹɹɹ
    .FEMFZ%#Ͱ͸্ճΔ
    w /PJTZ5FBDIFS4UVEFOU͸௿Ίɹɹɹ
    ˠ࠷ॳͷஈ֊Ͱ࣭ͷ௿͍ɹɹɹɹɹ
    ٖࣅϥϕϧΛ͚ͭͯ͠·͏ͨΊʁ
    
    
    

    View Slide

15. ࣮ݧɽֶशํ๏Ͱͷൺֱ
    w 1SFUSBJOJOHɿϥϕϧͳ͠σʔλΛઌʹֶश
    w 'JOFUVOJOHɿ1SFUSBJOJOHޙʹϥϕϧ͖ͭσʔλΛ࢖͏
    w +PJOUUSBJOJOHɿ྆ํಉ࣌ʹֶश
    w +PJOUUSBJOJOHͰͷ0"͕࠷΋ߴ͍
    w 1SFUSBJOJOHͰ΋ڭࢣ͋ΓΑΓߴ͍
    ˠϥϕϧͳ͠σʔλ͸ϥϕϧ͖ͭσʔλͱ
    ࣅͨ෼෍Λ΋ͪɼֶशʹ༗༻Ͱ͋Δʁ
    

    View Slide

16. ࣮ݧɽֶशσʔλͷαΠζ
    w ϥϕϧͳ͠σʔλͷྔΛมߋͤ͞Δ
    w ϘʔΧϧͷൺ཰ʢτϥοΫʹҎ্
    Վ੠ΛؚΉ͔ʣʹΑΔબग़ͷ༗ޮੑΛ
    ௐ΂Δ
    w બग़Λߦͬͨɼ*OIPVTF
    '."@MBSHF৚͕݅ڭࢣ͋Γͱൺ΂ͯ
    ্ճΔ
    

    View Slide

17. ࣮ݧɽ/PJTZ4UVEFOUͷ܁Γฦ͠ճ਺
    w ճͷ܁Γฦ͕͠࠷΋ߴ͔ͬͨ
    ʢʣ
    w ͦΕҎ্܁Γฦͯ͠΋ੑೳ͸ɹ
    ্͕Βͣ
    

    View Slide

18. ࣮ݧɽଞख๏ͱͷൺֱ
    w 1BUDI$//<4V`>ɿ$'1දݱΛೖྗʹ3$//ϥΠΫͳྖҬબ୒Λߦ͏$//
    w %4.<#JUUOFS`>ɿ)$25Λೖྗʹ%FFQ4BMJFODFΛಘΔ$//
    w 4FH/FU<)TJFI`>ɿηϚϯςΟοΫηάϝϯςʔγϣϯΛ͢Δɹɹɹɹɹɹɹɹ
    ΤϯίʔμɾσίʔμϞσϧʢ6/FUʣ
    w +4%<,VN`>ɿϝϩσΟͷ༗ແͷݕग़ΛαϒλεΫͱ͢ΔϚϧνλεΫֶश
    <#JUUOFS`>3#JUUOFSFUBMl%FFQTBMJFODFSFQSFTFOUBUJPOTGPSGFTUJNBUJPOJOQPMZQIPOJDNVTJDz*4.*3
    <4V`>-4Vl7PDBMNFMPEZFYUSBDUJPOVTJOHQBUDICBTFE$//z*$"441
    <)TJFI>5)TJFIFUBMl"TUSFBNMJOFEFODPEFSEFDPEFSBSDIJUFDUVSFGPSNFMPEZFYUSBDUJPOz*$"441
    

    View Slide

19. ࣮ݧɽଞख๏ͱͷൺֱ
    w 31"ͱ0"ͷൺֱ
    w .FEMFZ%#Ͱ͸ఏҊ๏͕Ұ൪ྑ͍
    w ଞख๏ͱൺ΂ͯ΋4P5"ʹ͍ۙੑೳ
    

    View Slide

20. ࣮ݧɽଞख๏ͱͷൺֱ
    w 73ͱ7'"ͷൺֱ
    w "45ͷΈɼఏҊ๏ͷੑೳѱԽɹɹ
    ˠ"45ͷϘʔΧϧϛΩγϯά͕
    ଞͷσʔληοτͱൺ΂ͯɹɹɹɹ
    ୯७Ͱ͋ΔͨΊͩͱߟ࡯
    

    View Slide

21. ·ͱΊ
    w ԿΛ͔ͨ͠ɿ/PJTZ4UVEFOUʹΑΔ൒ڭࢣ͋ΓֶशΛ༻͍ͨ.FMPEZ&YUSBDUJPO
    w ߩݙɿڭࢣ͋ΓֶशͷੑೳΛ্ճΓɼΞϊςʔγϣϯΛ͚ͭΔͷ͕೉͍͠λεΫͰɹ
    ϥϕϧͳ͠σʔλΛ࢖͑Δ͜ͱ΋ࣔͨ͠
    w ख๏ͷΩϞɿ3BOE"VEJP"VHNFOUΛ༻͍ͨ/PJTZTUVEFOUֶशͱϥϕϧͳ͠σʔλ
    ͷςʔλબग़ͷॏཁੑͷݕূ
    w ༗ޮੑͷݕূɿڭࢣ͋Γֶशͱͷൺֱɼઌߦख๏ͱͷൺֱɼϥϕϧͳ͠σʔλͷ৚݅
    ͷൺֱ
    

    View Slide

22. ײ૝
    w ը૾Ͱ͏·͍ͬͨ͘ख๏͕΋͏Իָ৘ใॲཧʹ·Ͱདྷ͍ͯΔͷ͔ɽɽɽ
    w ࠓ·Ͱ ೥͘Β͍͸ϥά͕͋ͬͨʢݸਓతͳॴײʣ
    w ը૾ॲཧ΍ࣗવݴޠॲཧͷख๏΋ͪΐͬ͜ͱνΣοΫ͢Δͱྑ͍͔΋͠Εͳ͍
    w ൒ڭࢣ͋Γֶश͸ࠓޙԻָ৘ใॲཧͰ૿͖͑ͯͦ͏
    w Ի੠ʹൺ΂ͯԻָ͸σʔλͷ༻ҙ͕͠ʹ͍͘ɼΞϊςʔγϣϯ͸΋ͬͱ༻ҙ͠ʹ͍͘
    w 0OHB"DDFMγϯϙδ΢ϜʢIUUQTPOHBBDDFMKQTZNQPTJVNʣͰژେͷ٢Ҫઌੜ͕ݴͬͯͨΑ͏
    ʹɼɹɹɹੜ੒Ϟσϧܥͷख๏΋૿͑ͦ͏ʁ$7"&ʢ.NPEFMʣͱ͔ʢຊݚڀ͸ࣝผϞσϧPOMZʣ
    w ͏·͘Իָ஌ࣝͱ૊Έ߹ΘͤΔ͜ͱ΋ඞཁʹͳΔ͔΋
    w ٸܹͳϒϨʔΫεϧʔʹ͸ࢸ͍ͬͯͳ͍ʁ
    w Ϟσϧࣗମ͔ͳΓ࿅ΒΕ͍ͯͨͷͰɼ͔͜͜Β͞Βʹ୆΁ͷϒϨʔΫεϧʔΛى͜͢ͷ͸೉ͦ͠͏ɽ
    

    View Slide