$30 off During Our Annual Pro Sale. View Details »

#muana IRM

Yuya Yamamoto
February 26, 2021

#muana IRM

2021/02/26で発表したIRM×歌いにくさについて 

Yuya Yamamoto

February 26, 2021
Tweet

More Decks by Yuya Yamamoto

Other Decks in Research

Transcript

  1. ࠷ۙͷώοτιϯά͸


    Վ͍ʹ͍͘ͷ͔ʁ


    ʙೝ஌తԻָཧ࿦Ͱ෼ੳͯ͠Έͨʙ
    Music×Analytics Meetup #4


    ࢁຊ ༤໵ (@yamathcy)


    2021/02/26

    View Slide

  2. ࣗݾ঺հ
    • ࢁຊ ༤໵ʢ΍·΋ͱ Ώ͏΍ʣ@yamatchy


    • ஜ೾େֶେֶӃਤॻؗ৘ใϝσΟΞݚڀՊത࢜લظ՝ఔ2೥


    • य़͔Βത࢜ޙظ՝ఔʹਐֶ༧ఆ


    • ݚڀ෼໺ɿԻָ৘ใॲཧʢಛʹՎʣ


    • ݚڀͷϞνϕɿΧϥΦέͷेീ൪୳͠Λͯ͘͠ΕΔγεςϜ͕ཉ͍͠ʂ


    • ݚڀςʔϚɿ


    • ਓ͕ײ͡ΔՎ͍ʹ͘͞ͷఆྔԽʢֶ෦ʣ


    • ػցֶशɾਂ૚ֶशʹΑΔՎএςΫχοΫͷ෼ੳʢӃʣ


    • ޷͖ͳΞʔςΟετɿΰʔϧσϯϘϯόʔ

    View Slide

  3. ࠓ೔ͷ಺༰
    • ਓͷೝ஌Λߟྀͨ͠Իָཧ࿦Ͱ͋Δɼೝ஌తԻָཧ࿦ͷҰͭ
    ҉ҙ࣮ݱϞσϧΛ࢖ͬͯՎͷෳࡶ͞ΛϞσϧԽ͓ͨ͠࿩

    View Slide

  4. ಥવͰ͕͢
    • ࠷ۙਓؾͷۂɼՎ͏ͷ೉͍͠ۂଟ͘ͳ͍Ͱ͔͢ʁʁ


    • ۂΛ࡞ͬͨຊਓͰ͢Β೉͍͠ͱݴ͍ͬͯΔέʔε΋…

    View Slide

  5. Վ͏ͷ͕೉͍͠ϝϩσΟʁ
    • ͍ΖΜͳཁૉ͕͋Δ


    • ԻҬɿࣗ෼ͷ੠ҬΛ֎ΕΔͱ೉͍͠


    • ςϯϙɿ଎͍ͱ೉͍͠


    • ෳࡶ͞ɿෳࡶͩͱ೉͍͠


    • ՎࢺͷൃԻͷ͠ʹ͘͞ɿטΜ͡Ό͏


    • etc…
    Իߴ
    #1.ͱԻՁ
    ՎࢺͷԻӆ
    ʁ

    View Slide

  6. ෳࡶ͞ʁ
    • ෳࡶͳϝϩσΟ #ͱ͸


    →ʢೝ஌తʹʣԻΛ౰ͯͮΒ͍ϝϩσΟʁ


    →࣍ͷԻΛ༧ଌͮ͠Β͍Α͏ͳԻͷฒͼʁ
    ༧૝௨Γ😄 ༧૝֎😳

    View Slide

  7. ҉ҙ࣮ݱϞσϧ (Implication-Realization Model)
    • ೝ஌తԻָཧ࿦ͷҰͭɼϝϩσΟʹର͢Δཧ࿦


    • ʮਓ͸࣍ʹԿ͕ى͜Δ͔༧ଌ͠ͳ͕ΒԻָΛௌ͘ʯ


    • ϝϩσΟΛʮ҉ҙʯʹج͍ͮͯͦͷ෦෼ྻΛγϯϘϧԽ


    • ҉ҙɿ2Ի࿈ଓͯ͠ௌ͍ͨ࣌ɼͦͷ࣍ͷԻΛਓ͕Ͳ͏༧૝͢Δ͔


    • ࣮ݱɿ҉ҙͷ௨Γʹͳ͔ͬͨͲ͏͔Ͱ෼ྨ
    Ի໨ͱԻ໨ͷ͕ࠩ
    Ͳ͏ͳ͍ͬͯΔ͔
    Ի໨ͱԻ໨ͷࠩ͸ʁ
    ʢԻఔݪཧʣ
    ˠͱˠͷํ޲͸ಉ͡ʁ
    ʢํ޲ݪཧʣ
    ڱ͍ ڱ͍ ಉ͡ํ޲
    ޿͍ ڱ͍ ҟͳΔํ޲
    Eugene Narmour, “The Analysis and Cognition of Melodic Complexity”, The university of Chicago press ,1992

    View Slide

  8. ਤղ
    FYڱ͍Իఔ
    ʁ
    ࣍͸zڱΊzʹ
    l্͕Δzʁ
    🤔
    ҉ҙ
    3Ի໨
    ҉ҙ͕࣮ݱ


    ->γϯϘϧ”P”
    ҉ҙͷ͏ͪ


    ํ޲ݪཧ͕൱ఆ


    ->γϯϘϧ”IP”
    ҉ҙͷ͏ͪ


    Իఔݪཧ͕൱ఆ


    ->γϯϘϧ”VP”
    ڱΊʹ


    ্͕Δ
    ڱΊʹ


    Լ͕Δ
    ޿Ίʹ


    ্͕Δ

    View Slide

  9. ਤղ
    P
    γϯϘϧɿ

    View Slide

  10. ਤղ
    P
    γϯϘϧɿ P

    View Slide

  11. ਤղ
    P
    γϯϘϧɿ P ID

    View Slide

  12. ਤղ
    P
    γϯϘϧɿ P ID P

    View Slide

  13. ਤղ
    P
    γϯϘϧɿ P ID P P

    View Slide

  14. ਤղ
    P
    γϯϘϧɿ P ID P P
    γϯϘϧͷׂ߹ɹPɿ80ˋ ID: 20ˋ

    View Slide

  15. ҉ҙ࣮ݱϞσϧͷγϯϘϧ
    ҉ҙͷݪཧͷ࣮ݱʹج͍ͮͯ෼ྨ
    جຊͷγϯϘϧ
    ྫ֎ͷγϯϘϧ
    2ԻͰ׬݁ 1ԻͰ׬݁
    rP rIP
    ಉํ޲Ͱ޿͍··
    ٯํ޲


    ڱ->޿

    View Slide

  16. ҉ҙͷݪཧͱجຊγϯϘϧͷؔ܎
    γϯϘϧ ԻఔΠϝʔδ ਐߦํ޲ Իߴࠩ Իఔݪཧ ํ޲ݪཧ
    1 PS ಉํ޲ ڱˠڱ ˓ ˓
    *1 PS ٯํ޲ ڱˠڱ ˓ º
    71 PS ಉํ޲ ڱˠ޿ º ˓
    3 PS ٯํ޲ ޿ˠڱ ˓ ˓
    *3 PS ಉํ޲ ޿ˠڱ ˓ º
    73 PS ٯํ޲ ޿ˠ޿ º ˓
    % ෆม ͳ͠
    *% PS ٯํ޲
    ڱˠڱ
    Ի໨ͷԻߴ
    ʹԻ໨ͷԻߴ

    ˓ º

    View Slide

  17. ҉ҙͷݪཧͱྫ֎γϯϘϧͷؔ܎
    γϯϘϧ ԻఔΠϝʔδ ਐߦํ޲ Իߴࠩ Իఔݪཧ ํ޲ݪཧ
    S1 PS ಉํ޲ ޿ˠ޿ º º
    S*1 PS ٯํ޲ ڱˠ޿ º º
    %ZBE
    ԻͷΈ͕
    ݽཱ

    .POBE
    ԻͷΈ͕
    ݽཱ

    View Slide

  18. • ಐགɾՎ͍΍͍͢ۂɾՎ͍ʹ͍͘ۂͰ҉ҙ࣮ݱϞσϧͷɹɹɹɹɹ
    γϯϘϧग़ݱ౓߹͍ͷҧ͍͸ݟΒΕΔʁ
    ࠓճߦ͏෼ੳ

    View Slide

  19. ෼ੳָۂ
    • 3܈ʹ෼͚Δ
    ಐག ؆୯Ί ෳࡶΊ
    ͲΜ͙Γ͜Ζ͜Ζ
    ͪΐ͏ͪΐ
    ϩϯυϯڮ
    ੈքʹҰ͚ͭͩͷՖ
    4."1
    νΣϦʔεϐοπ
    ϚϦʔΰʔϧυ͍͋
    ΈΐΜ
    ՖՐBJLP
    -FNPOถ௡ݰࢣ
    ໷ʹۦ͚Δ:0"40#*
    ന೔,JOH,OV

    View Slide

  20. ෼ੳख๏
    1. ָ֤ۂͷϝϩσΟϥΠϯͷ1൪ͷΈͷMIDIΛೖྗ


    2. ϝϩσΟͷઌ಄͔Β҉ҙ࣮ݱϞσϧʹΑΔ෼ੳΛߦ͏


    3. 2෼ԻූҎ্ͷ௕͞ͷԻූͰϑϨʔζΛ۠੾Δ


    4. 3.Ͱ༩͑ͨϑϨʔζʹॱ࣍γϯϘϧ෇༩


    5. γϯϘϧͷׂ߹Λࢉग़

    View Slide

  21. ݁Ռɿಐག
    M
    0%
    Dyad
    0%
    D
    0%
    P
    41%
    R
    0%
    ID
    32%
    IP
    27%
    VP
    0%
    IR
    0%
    VR
    0%
    rP
    0%
    rIP
    0%
    E
    0%
    ロンドン橋
    M
    Dyad
    D
    P
    R
    ID
    IP
    VP
    IR
    M
    0%
    Dyad
    0%
    D
    5% P
    12%
    R
    8%
    ID
    7%
    IP
    61%
    VP
    0%
    IR
    0%
    VR
    0%
    rP
    0%
    rIP
    7%
    E
    0%
    どんぐりころころ
    M
    Dyad
    D
    P
    R
    ID
    IP
    VP
    IR
    Ի਺͕গͳ͍ͷͰׂ߹͕ૈΊ ϩϯυϯڮ͸3छྨ
    M
    0%
    Dyad
    0%
    D
    24%
    P
    13%
    R
    0%
    ID
    4%
    IP
    59%
    VP
    0%
    IR
    0%
    VR
    0%
    rP
    0%
    rIP
    0%
    E
    0%
    ちょうちょ
    M
    Dyad
    D
    P
    R
    ID
    IP
    VP
    IR
    VR

    View Slide

  22. ݁Ռɿ؆୯Ί
    M
    0%
    Dyad
    0%
    D
    29%
    P
    17%
    R
    3%
    ID
    5%
    IP
    43%
    VP
    0%
    IR
    0%
    VR
    0%
    rP
    0%
    rIP
    3%
    E
    0%
    世界に⼀つだけの花/SMAP
    M
    Dyad
    D
    P
    R
    ID
    IP
    VP
    IR
    M
    0%
    Dyad
    1%
    D
    14%
    P
    24%
    R
    2%
    ID
    15%
    IP
    42%
    VP
    0%
    IR
    0%
    VR
    0%
    rP
    0%
    rIP
    2%
    E
    0%
    マリーゴールド/あいみょん
    M
    Dyad
    D
    P
    R
    ID
    IP
    VP
    IR
    VR
    M
    0%
    Dyad
    1%
    D
    10%
    P
    33%
    R
    1%
    ID
    12%
    IP
    41%
    VP
    1%
    IR
    0% VR
    0% rP
    0%
    rIP
    1%
    E
    0%
    チェリー/スピッツ
    M
    Dyad
    D
    P
    R
    ID
    IP
    VP
    IR
    VR
    rP
    rIP
    ಐགʹൺ΂ͯภΓ͕ݟΒΕͨ


    ->ϑϨʔζͷόϦΤʔγϣϯ͕ଟ͍ʁ

    View Slide

  23. ݁Ռɿ೉͠Ί
    M
    0%
    Dyad
    1%
    D
    26%
    P
    12%
    R
    3%
    ID
    12%
    IP
    43%
    VP
    1%
    IR
    0%
    VR
    0%
    rP
    0%
    rIP
    2%
    E
    0%
    花⽕/AIKO
    M
    Dyad
    D
    P
    R
    ID
    IP
    VP
    IR
    VR
    rP
    rIP
    M
    0%
    Dyad
    1%
    D
    7%
    P
    19%
    R
    5%
    ID
    14%
    IP
    49%
    VP
    1%
    IR
    0%
    VR
    0%
    rP
    0%
    rIP
    4%
    E
    0%
    ⽩⽇/KING GNU
    M
    Dyad
    D
    P
    R
    ID
    IP
    VP
    IR
    VR
    rP
    M
    0%
    Dyad
    8%
    D
    3%
    P
    40%
    R
    4%
    ID
    16%
    IP
    23%
    VP
    1%
    IR
    1%
    VR
    0%
    rP
    0%
    rIP
    4%
    E
    0%
    夜に駆ける/YOASOBI
    M
    Dyad
    D
    P
    R
    ID
    IP
    VP
    IR
    VR
    rP
    rIP
    M
    0%
    Dyad
    0%
    D
    3%
    P
    43%
    R
    5%
    ID
    18%
    IP
    26%
    VP
    2%
    IR
    0%
    VR
    0%
    rP
    0%
    rIP
    3% E
    0%
    LEMON/⽶津⽞師
    M
    Dyad
    D
    P
    R
    ID
    IP
    VP
    IR
    VR
    rP
    ؆୯Ί܈ʹൺ΂ͯID,R͕ଟΊɼग़ݱγϯϘϧ਺΋ଟΊ


    ্ԼͰෳࡶ͞ͷੑ࣭͕ҟͳΔʁ

    View Slide

  24. ݁࿦
    • ҉ҙ࣮ݱϞσϧʹΑΔ෼ੳʹΑΕ͹…


    • ಐགΑΓ΋γϯϘϧʹόϦΤʔγϣϯ͋Γ


    • ࠷ۙͷۂ͸एׯIDͱRͷγϯϘϧͷׂ߹͕ଟ͍


    • ෳࡶ͞ʹ͸ɼ·͍͔ͨͭ͘ͷύλʔϯ͕͋Γͦ͏


    • ΑΓଟ͘ͷۂͷ෼ੳ͕ඞཁ
    ຊ౰͸౷ܭతԾઆݕఆ/෼ࢄ෼ੳ·Ͱ΍ͬͯ


    ༗ҙʹ͕ࠩ͋Δ͔Λݟ͔͕ͨͬͨ


    ؒʹ߹͍·ͤΜͰͨ͠…

    View Slide

  25. Thanks!!
    • ҉ҙ࣮ݱϞσϧؔ࿈


    • 2020೥౓ஜ೾େֶฏլݚڀࣨଔۀͷઌഐํ


    • ஑ాपฏ͞ΜʢγϯϘϧ෇༩ϓϩάϥϜ࣮૷/JAZZ΁ͷద༻ʣ


    • ٶᖒڹ͞ΜʢϙϐϡϥʔԻָ΁ͷద༻ʣ


    • ັྗతͳϝϩσΟɾՎΛಧ͚͍ͯͩͬͯ͘͞Δɹɹɹɹ
    ΞʔςΟετͷօ༷


    • ʢՎ͍ʹ͍͘ͱ͔disͬΆ͍͜ͱݴͬͯ͢Έ·ͤΜͰͨ͠… ͜͏͍͏ϝϩσΟ΋ັྗ
    ʹᷓΕ͍ͯͯେ޷͖Ͱ͢ʣ

    View Slide

  26. ิॿࢿྉ ʴ ͓·͚

    View Slide

  27. ࠓޙ
    • ෳࡶ͞͸Ͳ͔͜ΒʁΛ΋ͬͱৄ͘͠෼ੳ͍ͨ͠


    • ίʔυਐߦͱϝϩσΟͷ૊Έ߹Θͤ


    • ͨ͘͞ΜςϯγϣϯίʔυΛ࢖͏ͱ೉͍͠


    • ϝϩσΟ͕ϧʔτԻͰͳ͍ͱෳࡶʹײ͡Δ


    • ୯Ұίʔυ಺ͰԻ͕ಈ͘ͱෳࡶʹײ͡Δ


    • ϦζϜ+Իߴύλʔϯͷ෼ੳ


    • ϝϩσΟ͕ϧʔτԻͰͳ͍ͱෳࡶʹײ͡Δ


    • ୯Ұίʔυ಺ͰԻ͕ಈ͘ͱෳࡶʹײ͡Δ

    View Slide

  28. ϝϩσΟϥΠϯͷ೉͠͞Λ෼ੳ͢Δ
    • Իָ৘ใՊֶݚڀձͰ༧උ୳ࡧωλͱͯ͠౤ߘ


    • Իߴ΍BPM౳ͷಛ௃ɾ౷ܭྔ+೉қ౓ධՁͱ૬ؔ෼ੳ΄͔


    • Վ͍ʹ͘͞ͷܥ౷Λେ·͔ʹ௫Ή΋ɼݸਓ͕ࠩେ͖͘ɼཁҼ΋๲େͰ͋ΔͨΊ
    खʹෛ͓͑ͣଂೖΓঢ়ଶ


    • ༷ʑͳಛ௃Λߟ͑Δ΋݁ہɼෳࡶ͞ is Կʁͱ͍͏ϞϠϞϠEndʹ
    ࢁຊ༤໵ɼฏլৡɿՎ͍΍͢͞ɾՎ͍ʹ͘͞ʹண໨ָͨ͠ۂݕࡧγεςϜͷͨΊͷϙϐϡϥʔָۂͷՎএ೉қ౓ࢉग़ͷݕ౼ɼ৘ใॲཧֶձݚڀใࠂɼVol.2019‐MUS‐
    124ɼNo.9 (2019)

    View Slide

  29. Q. σΟʔϓϥʔχϯάͰͳΜͱ͔ͳΒͳ͍ͷʁ
    • A. ଟ෼ͳΓ·ͤΜ


    • σΟʔϓϥʔχϯά͸ϥϕϧͱσʔλ͕ͨ͘͞Μඞཁ


    • ָේσʔλࣗମ͕खʹೖΓʹ͍͘


    • Վ͍ʹ͘͞ʹؔ͢Δϥϕϧ৘ใ͸΋ͬͱखʹೖΓʹ͍͘


    • ࠷ۙ͸ϝλσʔλ͕ͳͯ͘΋ಛ௃Λ΋ͱʹ෼͚ͯ͘ΕΔɼɹɹɹɹ
    ਂ૚ڭࢣͳֶ͠श͕τϨϯυͷҰͭɽଟগ͸ଠ౛ଧͪͰ͖ΔΑ͏ʹ
    ͳΔʁ

    View Slide

  30. ҉ҙ࣮ݱϞσϧͷݚڀࣄྫ
    • ϝϩσΟੜ੒


    • Yazawa, S., Hamanaka, M., & Utsuro, T. (2014, August). Melody generation system
    based on a theory of melody sequence. In 2014 International Conference of
    Advanced Informatics: Concept, Theory and Application (ICAICTA) (pp. 336-341).
    IEEE.


    • ϑϨʔζͷ෼ࢄදݱͷධՁʹ༻͍ͨ


    • Hirai, T., & Sawada, S. (2019). Melody2vec: Distributed representations of melodic
    phrases based on melody segmentation. Journal of Information Processing, 27,
    278-286.


    • ༷ʑͳδϟϯϧͷ෼ੳ΁ͷԠ༻


    • ೳొ෨, ஛઒Ղ੒, & ฏాܓೋ. (2018). ҉ҙ࣮ݱϞσϧʹج͖ͮ࡞ۂՈࣝผΛߦ͏
    HMM ʹ͍ͭͯ. ݚڀใࠂԻ੠ݴޠ৘ใॲཧ (SLP), 2018(18), 1-6.


    • ٶᖒڹ, & ฏլৡ. (2019). ҉ҙ࣮ݱϞσϧʹجͮ͘ϙϐϡϥʔԻָͷηΫγϣϯ෼
    ੳ. ୈ 81 ճશࠃେձߨԋ࿦จू, 2019(1), 371-372.

    View Slide

  31. ͓·͚ᶄɿԻָ৘ใॲཧͰ࢖͑ΔpythonϥΠϒϥϦ
    • Իڹಛ௃நग़


    • librosa Իڹಛ௃நग़ (https://librosa.org/)


    • madmom Իָಛ௃நग़ (https://madmom.readthedocs.io/en/latest/)


    • essentia Իָಛ௃நग़ (https://essentia.upf.edu/essentia_python_tutorial.html)


    • opensmile Իڹಛ௃நग़ (https://www.audeering.com/2020/10/26/introducing-
    opensmile-3-0/)


    • ָේσʔλϋϯυϦϯά


    • Music21 MusicXMLϥΠϒϥϦ (http://web.mit.edu/music21/)


    • MIDIܥϥΠϒϥϦʢmido, pretty-midi, pypianorollʣ


    • muspy UC San Diegoൃ Իָࣗಈੜ੒πʔϧΩοτ(https://salu133445.github.io/muspy/)


    • σʔληοτAPI


    • mirdata ԻָσʔληοτΛಡΈࠐΉAPIϥΠϒϥϦ (https://mirdata.readthedocs.io/en/
    stable/)

    View Slide

  32. ͓·͚ᶄɿԻָ৘ใॲཧͰ࢖͑ΔpythonϥΠϒϥϦ
    • GPUରԠͷDNN༻લॲཧϥΠϒϥϦ


    • nnAudio: pytorch༻


    • kapre: Tensor
    fl
    ow༻

    View Slide

  33. ͓·͚ᶄɿԻָ৘ใॲཧͰ࢖͑ΔpythonϥΠϒϥϦ
    • ͦͷଞ


    • mireval Իָ৘ใॲཧλεΫͷධՁࢦඪͷࢉग़ ʢhttps://
    craffel.github.io/mir_eval/ʣ


    • kymatio ΢ΣʔϒϨοτࢄཚͷܭࢉ (kymat.io)


    • Omnizart ୆࿷ͷAcademia Sinicaൃࣗಈ࠾ේϥΠϒϥϦ
    (https://pypi.org/project/omnizart/)


    • FMP-notebook ਆνϡʔτϦΞϧ (https://www.audiolabs-
    erlangen.de/resources/MIR/FMP/C0/C0.html)

    View Slide

  34. ͓·͚ᶅɿGTTM
    • GTTMʢGenerative Theory of Tonal Musicʣ


    • ָۂ͔Β֊૚తͳߏ଄ΛಘΔͨΊͷཧ࿦


    • ೝ஌తԻָཧ࿦Ͱ͸҉ҙ࣮ݱϞσϧΑΓ΋ϝδϟʔ
    ʢͳҹ৅ʣ

    View Slide

  35. ͓·͚ᶅɿGTTMΛߏ੒͢Δαϒཧ࿦
    • ָۂͷߏ଄ΛಘΔͨΊͷ4ͭͷαϒཧ࿦Ͱߏ੒


    • ߏ଄ͷ෼ੳ


    • 1. άϧʔϐϯάɿϑϨʔζΛάϧʔϓԽ


    • 2. ഥઅߏ଄ɿϏʔτΛͱΔ্Ͱͷॏཁ౓Λ෇༩


    • ؆໿


    • 3. λΠϜεύϯ؆໿ɿॏཁ౓ʹج͍ͮͯάϧʔϓ಺Ͱ
    ໦ߏ଄Λ֊૚తʹ࡞͍ͬͯ͘


    • 4. Ԇ௕త؆໿ɿ࿨੠ͷஎ؇ɾۓுʹج͍ͮͯɼɹɹɹ
    άϧʔϓΛ௒͑ͨࢬͷ઀ଓΛߦ͏

    View Slide

  36. ͓·͚ᶅɿGTTMʹΑΔ֊૚ߏ଄ͷ֫ಘ
    ϞʔπΝϧτ ϐΞϊιφλ K331.
    ഥઅߏ଄
    άϧʔϐϯά
    ʢଟٛੑ͋Γʣ

    View Slide

  37. ͓·͚ᶅɿGTTMʹΑΔ؆໿
    GTTM - RikenAIP ΑΓ
    ൈਮ


    https://aip.riken.jp/
    uploads/
    775777483bfe107f16
    377730a69f330c.pdf

    View Slide