#muana IRM

2021/02/26で発表したIRM×歌いにくさについて　

Transcript

1. ࠷ۙͷώοτιϯά͸ Վ͍ʹ͍͘ͷ͔ʁ ʙೝ஌తԻָཧ࿦Ͱ෼ੳͯ͠Έͨʙ Music×Analytics Meetup #4 ࢁຊ ༤໵ (@yamathcy) 2021/02/26
2. ࣗݾ঺հ • ࢁຊ ༤໵ʢ΍·΋ͱ Ώ͏΍ʣ@yamatchy • ஜ೾େֶେֶӃਤॻؗ৘ใϝσΟΞݚڀՊത࢜લظ՝ఔ2೥ • य़͔Βത࢜ޙظ՝ఔʹਐֶ༧ఆ • ݚڀ෼໺ɿԻָ৘ใॲཧʢಛʹՎʣ • ݚڀͷϞνϕɿΧϥΦέͷेീ൪୳͠Λͯ͘͠ΕΔγεςϜ͕ཉ͍͠ʂ • ݚڀςʔϚɿ • ਓ͕ײ͡ΔՎ͍ʹ͘͞ͷఆྔԽʢֶ෦ʣ • ػցֶशɾਂ૚ֶशʹΑΔՎএςΫχοΫͷ෼ੳʢӃʣ • ޷͖ͳΞʔςΟετɿΰʔϧσϯϘϯόʔ
3. ࠓ೔ͷ಺༰ • ਓͷೝ஌Λߟྀͨ͠Իָཧ࿦Ͱ͋Δɼೝ஌తԻָཧ࿦ͷҰͭ ҉ҙ࣮ݱϞσϧΛ࢖ͬͯՎͷෳࡶ͞ΛϞσϧԽ͓ͨ͠࿩
4. ಥવͰ͕͢ • ࠷ۙਓؾͷۂɼՎ͏ͷ೉͍͠ۂଟ͘ͳ͍Ͱ͔͢ʁʁ • ۂΛ࡞ͬͨຊਓͰ͢Β೉͍͠ͱݴ͍ͬͯΔέʔε΋…
5. Վ͏ͷ͕೉͍͠ϝϩσΟʁ • ͍ΖΜͳཁૉ͕͋Δ • ԻҬɿࣗ෼ͷ੠ҬΛ֎ΕΔͱ೉͍͠ • ςϯϙɿ଎͍ͱ೉͍͠ • ෳࡶ͞ɿෳࡶͩͱ೉͍͠ • ՎࢺͷൃԻͷ͠ʹ͘͞ɿטΜ͡Ό͏ • etc…
   Իߴ #1.ͱԻՁ ՎࢺͷԻӆ ʁ
6. ෳࡶ͞ʁ • ෳࡶͳϝϩσΟ #ͱ͸ →ʢೝ஌తʹʣԻΛ౰ͯͮΒ͍ϝϩσΟʁ →࣍ͷԻΛ༧ଌͮ͠Β͍Α͏ͳԻͷฒͼʁ ༧૝௨Γ😄 ༧૝֎😳
7. ҉ҙ࣮ݱϞσϧ (Implication-Realization Model) • ೝ஌తԻָཧ࿦ͷҰͭɼϝϩσΟʹର͢Δཧ࿦ • ʮਓ͸࣍ʹԿ͕ى͜Δ͔༧ଌ͠ͳ͕ΒԻָΛௌ͘ʯ • ϝϩσΟΛʮ҉ҙʯʹج͍ͮͯͦͷ෦෼ྻΛγϯϘϧԽ • ҉ҙɿ2Ի࿈ଓͯ͠ௌ͍ͨ࣌ɼͦͷ࣍ͷԻΛਓ͕Ͳ͏༧૝͢Δ͔ • ࣮ݱɿ҉ҙͷ௨Γʹͳ͔ͬͨͲ͏͔Ͱ෼ྨ Ի໨ͱԻ໨ͷ͕ࠩ
   Ͳ͏ͳ͍ͬͯΔ͔ Ի໨ͱԻ໨ͷࠩ͸ʁ
   ʢԻఔݪཧʣ ˠͱˠͷํ޲͸ಉ͡ʁ
   ʢํ޲ݪཧʣ ڱ͍ ڱ͍ ಉ͡ํ޲ ޿͍ ڱ͍ ҟͳΔํ޲ Eugene Narmour, “The Analysis and Cognition of Melodic Complexity”, The university of Chicago press ,1992
8. ਤղ FY
   ڱ͍Իఔ ʁ ࣍͸zڱΊzʹ
   l্͕Δzʁ 🤔 ҉ҙ 3Ի໨ ҉ҙ͕࣮ݱ ->γϯϘϧ”P” ҉ҙͷ͏ͪ ํ޲ݪཧ͕൱ఆ ->γϯϘϧ”IP” ҉ҙͷ͏ͪ Իఔݪཧ͕൱ఆ ->γϯϘϧ”VP” ڱΊʹ ্͕Δ ڱΊʹ Լ͕Δ ޿Ίʹ ্͕Δ
9. ਤղ P γϯϘϧɿ
10. ਤղ P γϯϘϧɿ P
11. ਤղ P γϯϘϧɿ P ID
12. ਤղ P γϯϘϧɿ P ID P
13. ਤղ P γϯϘϧɿ P ID P P
14. ਤղ P γϯϘϧɿ P ID P P γϯϘϧͷׂ߹ɹPɿ80ˋ ID: 20ˋ
15. ҉ҙ࣮ݱϞσϧͷγϯϘϧ ҉ҙͷݪཧͷ࣮ݱʹج͍ͮͯ෼ྨ جຊͷγϯϘϧ ྫ֎ͷγϯϘϧ 2ԻͰ׬݁ 1ԻͰ׬݁ rP rIP ಉํ޲Ͱ޿͍·· ٯํ޲ ڱ->޿
16. ҉ҙͷݪཧͱجຊγϯϘϧͷؔ܎ γϯϘϧ ԻఔΠϝʔδ ਐߦํ޲ Իߴࠩ Իఔݪཧ ํ޲ݪཧ 1 PS ಉํ޲ ڱˠڱ ˓ ˓ *1 PS ٯํ޲ ڱˠڱ ˓ º 71 PS ಉํ޲ ڱˠ޿ º ˓ 3 PS ٯํ޲ ޿ˠڱ ˓ ˓ *3 PS ಉํ޲ ޿ˠڱ ˓ º 73 PS ٯํ޲ ޿ˠ޿ º ˓ % ෆม ͳ͠   *% PS ٯํ޲ ڱˠڱ
    Ի໨ͷԻߴ
    ʹԻ໨ͷԻߴ ˓ º
17. ҉ҙͷݪཧͱྫ֎γϯϘϧͷؔ܎ γϯϘϧ ԻఔΠϝʔδ ਐߦํ޲ Իߴࠩ Իఔݪཧ ํ޲ݪཧ S1 PS ಉํ޲ ޿ˠ޿ º º S*1 PS ٯํ޲ ڱˠ޿ º º %ZBE  ԻͷΈ͕
    ݽཱ    .POBE  ԻͷΈ͕
    ݽཱ   
18. • ಐགɾՎ͍΍͍͢ۂɾՎ͍ʹ͍͘ۂͰ҉ҙ࣮ݱϞσϧͷɹɹɹɹɹ γϯϘϧग़ݱ౓߹͍ͷҧ͍͸ݟΒΕΔʁ ࠓճߦ͏෼ੳ
19. ෼ੳָۂ • 3܈ʹ෼͚Δ ಐག ؆୯Ί ෳࡶΊ ͲΜ͙Γ͜Ζ͜Ζ
    ͪΐ͏ͪΐ
    ϩϯυϯڮ ੈքʹҰ͚ͭͩͷՖ 4."1
    νΣϦʔεϐοπ
    ϚϦʔΰʔϧυ͍͋ ΈΐΜ ՖՐBJLP
    -FNPOถ௡ݰࢣ
    ໷ʹۦ͚Δ:0"40#*
    ന೔,JOH
    ,OV
    
20. ෼ੳख๏ 1. ָ֤ۂͷϝϩσΟϥΠϯͷ1൪ͷΈͷMIDIΛೖྗ 2. ϝϩσΟͷઌ಄͔Β҉ҙ࣮ݱϞσϧʹΑΔ෼ੳΛߦ͏ 3. 2෼ԻූҎ্ͷ௕͞ͷԻූͰϑϨʔζΛ۠੾Δ 4. 3.Ͱ༩͑ͨϑϨʔζʹॱ࣍γϯϘϧ෇༩ 5. γϯϘϧͷׂ߹Λࢉग़
21. ݁Ռɿಐག M 0% Dyad 0% D 0% P 41% R 0% ID 32% IP 27% VP 0% IR 0% VR 0% rP 0% rIP 0% E 0% ロンドン橋 M Dyad D P R ID IP VP IR M 0% Dyad 0% D 5% P 12% R 8% ID 7% IP 61% VP 0% IR 0% VR 0% rP 0% rIP 7% E 0% どんぐりころころ M Dyad D P R ID IP VP IR Ի਺͕গͳ͍ͷͰׂ߹͕ૈΊ ϩϯυϯڮ͸3छྨ M 0% Dyad 0% D 24% P 13% R 0% ID 4% IP 59% VP 0% IR 0% VR 0% rP 0% rIP 0% E 0% ちょうちょ M Dyad D P R ID IP VP IR VR
22. ݁Ռɿ؆୯Ί M 0% Dyad 0% D 29% P 17% R 3% ID 5% IP 43% VP 0% IR 0% VR 0% rP 0% rIP 3% E 0% 世界に⼀つだけの花/SMAP M Dyad D P R ID IP VP IR M 0% Dyad 1% D 14% P 24% R 2% ID 15% IP 42% VP 0% IR 0% VR 0% rP 0% rIP 2% E 0% マリーゴールド/あいみょん M Dyad D P R ID IP VP IR VR M 0% Dyad 1% D 10% P 33% R 1% ID 12% IP 41% VP 1% IR 0% VR 0% rP 0% rIP 1% E 0% チェリー/スピッツ M Dyad D P R ID IP VP IR VR rP rIP ಐགʹൺ΂ͯภΓ͕ݟΒΕͨ ->ϑϨʔζͷόϦΤʔγϣϯ͕ଟ͍ʁ
23. ݁Ռɿ೉͠Ί M 0% Dyad 1% D 26% P 12% R 3% ID 12% IP 43% VP 1% IR 0% VR 0% rP 0% rIP 2% E 0% 花⽕/AIKO M Dyad D P R ID IP VP IR VR rP rIP M 0% Dyad 1% D 7% P 19% R 5% ID 14% IP 49% VP 1% IR 0% VR 0% rP 0% rIP 4% E 0% ⽩⽇/KING GNU M Dyad D P R ID IP VP IR VR rP M 0% Dyad 8% D 3% P 40% R 4% ID 16% IP 23% VP 1% IR 1% VR 0% rP 0% rIP 4% E 0% 夜に駆ける/YOASOBI M Dyad D P R ID IP VP IR VR rP rIP M 0% Dyad 0% D 3% P 43% R 5% ID 18% IP 26% VP 2% IR 0% VR 0% rP 0% rIP 3% E 0% LEMON/⽶津⽞師 M Dyad D P R ID IP VP IR VR rP ؆୯Ί܈ʹൺ΂ͯID,R͕ଟΊɼग़ݱγϯϘϧ਺΋ଟΊ ্ԼͰෳࡶ͞ͷੑ࣭͕ҟͳΔʁ
24. ݁࿦ • ҉ҙ࣮ݱϞσϧʹΑΔ෼ੳʹΑΕ͹… • ಐགΑΓ΋γϯϘϧʹόϦΤʔγϣϯ͋Γ • ࠷ۙͷۂ͸एׯIDͱRͷγϯϘϧͷׂ߹͕ଟ͍ • ෳࡶ͞ʹ͸ɼ·͍͔ͨͭ͘ͷύλʔϯ͕͋Γͦ͏ • ΑΓଟ͘ͷۂͷ෼ੳ͕ඞཁ ຊ౰͸౷ܭతԾઆݕఆ/෼ࢄ෼ੳ·Ͱ΍ͬͯ ༗ҙʹ͕ࠩ͋Δ͔Λݟ͔͕ͨͬͨ ؒʹ߹͍·ͤΜͰͨ͠…
25. Thanks!! • ҉ҙ࣮ݱϞσϧؔ࿈ • 2020೥౓ஜ೾େֶฏլݚڀࣨଔۀͷઌഐํ • ஑ాपฏ͞ΜʢγϯϘϧ෇༩ϓϩάϥϜ࣮૷/JAZZ΁ͷద༻ʣ • ٶᖒڹ͞ΜʢϙϐϡϥʔԻָ΁ͷద༻ʣ • ັྗతͳϝϩσΟɾՎΛಧ͚͍ͯͩͬͯ͘͞Δɹɹɹɹ ΞʔςΟετͷօ༷ • ʢՎ͍ʹ͍͘ͱ͔disͬΆ͍͜ͱݴͬͯ͢Έ·ͤΜͰͨ͠… ͜͏͍͏ϝϩσΟ΋ັྗ ʹᷓΕ͍ͯͯେ޷͖Ͱ͢ʣ
26. ิॿࢿྉ ʴ ͓·͚
27. ࠓޙ • ෳࡶ͞͸Ͳ͔͜ΒʁΛ΋ͬͱৄ͘͠෼ੳ͍ͨ͠ • ίʔυਐߦͱϝϩσΟͷ૊Έ߹Θͤ • ͨ͘͞ΜςϯγϣϯίʔυΛ࢖͏ͱ೉͍͠ • ϝϩσΟ͕ϧʔτԻͰͳ͍ͱෳࡶʹײ͡Δ • ୯Ұίʔυ಺ͰԻ͕ಈ͘ͱෳࡶʹײ͡Δ • ϦζϜ+Իߴύλʔϯͷ෼ੳ • ϝϩσΟ͕ϧʔτԻͰͳ͍ͱෳࡶʹײ͡Δ • ୯Ұίʔυ಺ͰԻ͕ಈ͘ͱෳࡶʹײ͡Δ
28. ϝϩσΟϥΠϯͷ೉͠͞Λ෼ੳ͢Δ • Իָ৘ใՊֶݚڀձͰ༧උ୳ࡧωλͱͯ͠౤ߘ • Իߴ΍BPM౳ͷಛ௃ɾ౷ܭྔ+೉қ౓ධՁͱ૬ؔ෼ੳ΄͔ • Վ͍ʹ͘͞ͷܥ౷Λେ·͔ʹ௫Ή΋ɼݸਓ͕ࠩେ͖͘ɼཁҼ΋๲େͰ͋ΔͨΊ खʹෛ͓͑ͣଂೖΓঢ়ଶ • ༷ʑͳಛ௃Λߟ͑Δ΋݁ہɼෳࡶ͞ is Կʁͱ͍͏ϞϠϞϠEndʹ ࢁຊ༤໵ɼฏլৡɿՎ͍΍͢͞ɾՎ͍ʹ͘͞ʹண໨ָͨ͠ۂݕࡧγεςϜͷͨΊͷϙϐϡϥʔָۂͷՎএ೉қ౓ࢉग़ͷݕ౼ɼ৘ใॲཧֶձݚڀใࠂɼVol.2019‐MUS‐ 124ɼNo.9 (2019)
29. Q. σΟʔϓϥʔχϯάͰͳΜͱ͔ͳΒͳ͍ͷʁ • A. ଟ෼ͳΓ·ͤΜ • σΟʔϓϥʔχϯά͸ϥϕϧͱσʔλ͕ͨ͘͞Μඞཁ • ָේσʔλࣗମ͕खʹೖΓʹ͍͘ • Վ͍ʹ͘͞ʹؔ͢Δϥϕϧ৘ใ͸΋ͬͱखʹೖΓʹ͍͘ • ࠷ۙ͸ϝλσʔλ͕ͳͯ͘΋ಛ௃Λ΋ͱʹ෼͚ͯ͘ΕΔɼɹɹɹɹ ਂ૚ڭࢣͳֶ͠श͕τϨϯυͷҰͭɽଟগ͸ଠ౛ଧͪͰ͖ΔΑ͏ʹ ͳΔʁ
30. ҉ҙ࣮ݱϞσϧͷݚڀࣄྫ • ϝϩσΟੜ੒ • Yazawa, S., Hamanaka, M., & Utsuro, T. (2014, August). Melody generation system based on a theory of melody sequence. In 2014 International Conference of Advanced Informatics: Concept, Theory and Application (ICAICTA) (pp. 336-341). IEEE. • ϑϨʔζͷ෼ࢄදݱͷධՁʹ༻͍ͨ • Hirai, T., & Sawada, S. (2019). Melody2vec: Distributed representations of melodic phrases based on melody segmentation. Journal of Information Processing, 27, 278-286. • ༷ʑͳδϟϯϧͷ෼ੳ΁ͷԠ༻ • ೳొ෨, ஛઒Ղ੒, & ฏాܓೋ. (2018). ҉ҙ࣮ݱϞσϧʹج͖ͮ࡞ۂՈࣝผΛߦ͏ HMM ʹ͍ͭͯ. ݚڀใࠂԻ੠ݴޠ৘ใॲཧ (SLP), 2018(18), 1-6. • ٶᖒڹ, & ฏլৡ. (2019). ҉ҙ࣮ݱϞσϧʹجͮ͘ϙϐϡϥʔԻָͷηΫγϣϯ෼ ੳ. ୈ 81 ճશࠃେձߨԋ࿦จू, 2019(1), 371-372.
31. ͓·͚ᶄɿԻָ৘ใॲཧͰ࢖͑ΔpythonϥΠϒϥϦ • Իڹಛ௃நग़ • librosa Իڹಛ௃நग़ (https://librosa.org/) • madmom Իָಛ௃நग़ (https://madmom.readthedocs.io/en/latest/) • essentia Իָಛ௃நग़ (https://essentia.upf.edu/essentia_python_tutorial.html) • opensmile Իڹಛ௃நग़ (https://www.audeering.com/2020/10/26/introducing- opensmile-3-0/) • ָේσʔλϋϯυϦϯά • Music21 MusicXMLϥΠϒϥϦ (http://web.mit.edu/music21/) • MIDIܥϥΠϒϥϦʢmido, pretty-midi, pypianorollʣ • muspy UC San Diegoൃ Իָࣗಈੜ੒πʔϧΩοτ(https://salu133445.github.io/muspy/) • σʔληοτAPI • mirdata ԻָσʔληοτΛಡΈࠐΉAPIϥΠϒϥϦ (https://mirdata.readthedocs.io/en/ stable/)
32. ͓·͚ᶄɿԻָ৘ใॲཧͰ࢖͑ΔpythonϥΠϒϥϦ • GPUରԠͷDNN༻લॲཧϥΠϒϥϦ • nnAudio: pytorch༻ • kapre: Tensor fl ow༻
33. ͓·͚ᶄɿԻָ৘ใॲཧͰ࢖͑ΔpythonϥΠϒϥϦ • ͦͷଞ • mireval Իָ৘ใॲཧλεΫͷධՁࢦඪͷࢉग़ ʢhttps:// craffel.github.io/mir_eval/ʣ • kymatio ΢ΣʔϒϨοτࢄཚͷܭࢉ (kymat.io) • Omnizart ୆࿷ͷAcademia Sinicaൃࣗಈ࠾ේϥΠϒϥϦ (https://pypi.org/project/omnizart/) • FMP-notebook ਆνϡʔτϦΞϧ (https://www.audiolabs- erlangen.de/resources/MIR/FMP/C0/C0.html)
34. ͓·͚ᶅɿGTTM • GTTMʢGenerative Theory of Tonal Musicʣ • ָۂ͔Β֊૚తͳߏ଄ΛಘΔͨΊͷཧ࿦ • ೝ஌తԻָཧ࿦Ͱ͸҉ҙ࣮ݱϞσϧΑΓ΋ϝδϟʔ ʢͳҹ৅ʣ
35. ͓·͚ᶅɿGTTMΛߏ੒͢Δαϒཧ࿦ • ָۂͷߏ଄ΛಘΔͨΊͷ4ͭͷαϒཧ࿦Ͱߏ੒ • ߏ଄ͷ෼ੳ • 1. άϧʔϐϯάɿϑϨʔζΛάϧʔϓԽ • 2. ഥઅߏ଄ɿϏʔτΛͱΔ্Ͱͷॏཁ౓Λ෇༩ • ؆໿ • 3. λΠϜεύϯ؆໿ɿॏཁ౓ʹج͍ͮͯάϧʔϓ಺Ͱ ໦ߏ଄Λ֊૚తʹ࡞͍ͬͯ͘ • 4. Ԇ௕త؆໿ɿ࿨੠ͷஎ؇ɾۓுʹج͍ͮͯɼɹɹɹ άϧʔϓΛ௒͑ͨࢬͷ઀ଓΛߦ͏
36. ͓·͚ᶅɿGTTMʹΑΔ֊૚ߏ଄ͷ֫ಘ ϞʔπΝϧτ ϐΞϊιφλ K331. ഥઅߏ଄ άϧʔϐϯά
    ʢଟٛੑ͋Γʣ
37. ͓·͚ᶅɿGTTMʹΑΔ؆໿ GTTM - RikenAIP ΑΓ ൈਮ https://aip.riken.jp/ uploads/ 775777483bfe107f16 377730a69f330c.pdf