Share
Embed
Transcript
25 slides
-
ՎএςΫχοΫͷࣝผʹ͓͚Δ hand-craftedಛྔͱਂֶश ಛྔͷൺֱ ࢁຊ ༤1, Juhan Nam2, ࣉᖒ ༸ࢠ1 ,
ฏլ ৡ1 1ɿஜେֶ, 2ɿKAIST 2021.03.17 ୈ130ճԻָใՊֶݚڀձ ޱ಄ൃද -
എܠɿՎএςΫχοΫͷࣝผ • ՎএςΫχοΫɿՎख͕ԻߴɾԻྔɾԻ৭ͳͲΛมಈͤ͞ɼɹ ࣮ݱ͢ΔՎএදݱٕ๏ • ࣝผͰ͖ΔͱɼՎखͷՎ͍ํͷཧղʹͭͳ͕Δ • ՎএςΫχοΫࣝผVocalSet [Wilkins 18]ͱ͍͏σʔληοτͷɹ
ొҎ߱ɼਂֶशʹΑΔख๏͕ޭΛೲΊͨ[Luo 20, Pishdadian 19ଞ] • Ұํɼैདྷ୯ҰͷՎএςΫχοΫࣝผʢϏϒϥʔτ[Dridger 16ଞ] ʣ Ͱઐࣝʹجͮ͘ಛʢHand-craftedಛʣ͕༻͍ΒΕͨ • ࠓճɼ͜ͷHand-craftedಛͱਂֶशಛΛൺֱ 4 -
ઌߦݚڀɿԻָࣝผͷಛྔͷൺֱ [Abeßer 19] • ϐονͷيΛྨ͢Δʹ͓͍ͯɼ Hand-craftedಛ(Pymus, Bitteli)ͱਂֶश (CNN)ʹΑΓ֫ಘͨ͠ಛྔΛൺֱ • ʮ໌ࣔతͳϞσϦϯάΛ͠ͳ͍CNNͰɼ
Hand-craftedಛྔͱಉͷࣝผྗΛ֫ಘ Մೳʯͱ͍͏݁ 5 [Abeßer 19] J. Abeßer et al. Fundamental frequency contour classi fi cation: A comparison between hand-crafted and cnn-based features. ICASSP2019 Ի৭ͷཁૉ͕ՃΘΔͱͲ͏͔ʁ ྨͷಘҙۤखʁ ͳͲΛՃͰݕূ -
ࣝผʹ༻͍ΔσʔληοτɿVocalSet [Wilkins 18] • ՎএςΫχοΫΛऩͨ͠େن σʔληοτ • உঁ20໊ɼҟͳΔ10छྨͷɹɹɹ ՎএςΫχοΫΛؚΉɹɹɹɹɹɹɹ •
૯ܭ10.1࣌ؒ • 1ϑΝΠϧʹ1ϥϕϧ • ϩϯάτʔϯɼΞϧϖδΦɼɹɹɹ εέʔϧͳͲ͋ΒΏΔϑϨʔζͰ Վএ • ϥϕϧ͕ෆۉߧ 6 -
ಛநग़ख๏1ɿHand-Craftedಛྔ 9 • Վखࣝผ [Kroher 14] Ͱ༻͍ΒΕͨಛ ͔ΒԻߴมಈͱԻ৭ʹؔ͢ΔಛྔΛ ൈਮ Ի৭ɼԻߴมಈͦΕͧΕʹMFCCɼϏϒϥʔτಛྔΛ࠾༻
MFCC 20࣍ݩ Ի৭ʹؔ͢Δಛྔ Ϗϒϥʔτಛྔ 2࣍ݩɼਂ͞(rate)ͱ͞(extent) Իߴมಈʹؔ͢Δಛྔ ->22࣍ݩͷϕΫτϧ -
࣮ݧ 11 • VocalSetͷࣝผ࣮ݧ • VocalSetΛֶश༻ɿςετ༻=8ɿ2ʹׂ • ྨثʹ྆݅ͱRandom Forest [Breiman
01]Λ ༻͍Δ • ܾఆͷ50 • ࣮ࡍʹϥϕϧͷෆۉߧΛߟྀ͢ΔBalanced Random Forest [Chen 04]Λ༻͍Δ -
݁Ռͱੳ 13 1. ͲͷςΫχοΫͷࣝผΛͲΕ͘Β͍ਖ਼ղͰ͖͔ͨ • શମɾΫϥε͝ͱͷਖ਼ղ (Accuracy) 2. ಛྔ֤ʑΛหผ͠͏ΔදݱΛ֫ಘͰ͖͍ͯΔ͔ •
tSNEʹΑΔ࣍ݩѹॖϓϩοτ 3. Ͳͷಛྔ͕ࣝผʹޮՌత͔ͩͬͨ • Random Forestͷಛྔॏཁͷ֬ೝ -
݁Ռᶃɿਖ਼ղ 14 • શମͷਖ਼ղ • Hand-craftedಛྔ: 0.710 • ਂֶशಛྔ: 0.736
• Ϋϥε͝ͱͷਖ਼ղ • Straight, Vibrato, Vocal fry ɿHand-crafted > ਂֶश • ͦΕҎ֎ͷՎএςΫχοΫ ɿ Hand-crafted < ਂֶश HC ਂ -
16 ݁ՌᶅɿRandom Forestͷಛྔॏཁ Hand-craftedಛྔͰͷɼRandom ForestಛྔॏཁΛܭࢉ ϏϒϥʔτಛྔͱMFCCͷ͕࣍ॏཁ͕ߴΊ 0 0.01 0.02 0.03
0.04 0.05 0.06 0.07 0.08 0.09 0.1 Vibrato extent Vibrato rate M FCC-3 M FCC-1 M FCC-2 M FCC-7 M FCC-10 M FCC-14 M FCC-5 M FCC-20 M FCC-6 M FCC-12 M FCC-11 M FCC-4 M FCC-9 M FCC-13 M FCC-19 M FCC-7 M FCC-18 M FCC-17 M FCC-16 M FCC-15 Hand-crafted 特徴量重要度 -
݁Ռͷߟ • ࣝผͷਖ਼ղɼtSNEϓϩοτΑΓ • ਂֶशख๏͕hand-craftedख๏ͱḮ৭ͳ͠ ɹɹɹɹɹɹ ɹ ɹ-> ઐࣝΛͦΕ΄Ͳ໌ࣔతʹ༩͑ͳͯࣝ͘ผʹ༗༻ͳಛΛ֫ಘՄೳʁ •
ಛྔॏཁͷੳΑΓ • Hand-craftedख๏ϏϒϥʔτಛྔɼMFCC࣍ͷॏཁ͕ߴ͍ɹ -> StraightVibratoͷਖ਼ղ͕ߴ͔ͬͨ͜ͱͱซͤΔͱɼ͜ͷೋͭΛݟ͚Δʹ Ϗϒϥʔτಛྔ͑͋͞ΕΑ͍ʁ -> ੑ࣭͕Α͘Θ͔͍ͬͯΔςΫχοΫʹରͯ͠Hand-craftedಛ͑Δʁ 17 -
·ͱΊ 18 • ͬͨ͜ͱ • ਂֶशʹΑΔࣗಈநग़ಛྔͷੑೳΛௐࠪ • Hand-craftedಛྔͱͷൺֱ • ݁Ռ
• ਂֶशख๏ͷਖ਼ղ͕2.6%্ճΔɼHand-craftedख๏ͱḮ৭ͳ͠ • ಛʹ௨ৗͱ͔͚ΕͨՎএςΫχοΫͰੑೳ˕ • Hand-craftedख๏ͰStraight, Vibratoʹ͓͍ͯੑೳ˕ • ಛྔॏཁ͔ΒɼϏϒϥʔτಛྔ͕ͦͷ2ͭͷࣝผʹΫϦςΟΧϧʁ • ࠓޙ • ΑΓଟ͘ͷγνϡΤʔγϣϯʹΑΔ࣮ݧʢೖྗσʔληοτͷൺֱରΛ ͬͱ૿͢ʣ -
ࠓޙͷ՝ • ൺֱ͢Δ݅ΛՃ͢Δ • Hand-crafted • Formant, Jitter, ShimmerͳͲɼ ޭࣄྫͷ͋ΔଞͷಛྔΛൺֱ݅
ʹՃ • ָثͷԋςΫχοΫࣝผͰSoTAΛୡͨ͠ख๏ [Wang 20]Ͱ͋Δ Wavelet Scattering TransformΛՃ • ਂֶश • ৴߸ܗɼSTFTεϖΫτϩάϥϜɼF0ɼೖྗಛྔΛมߋ • RNNΛCNNͷ࣍ͷʹஔ͘ͳͲɼΞʔΩςΫνϟͷมߋ 19 -
ࢀߟจݙ 20 • [Wilkins 18] J. Wilkins, P. Seetharaman, A.
Wahl, and B. Pardo,“Vo- calset: A singing voice dataset,” in ISMIR 2018, 2018. • [Abesser 19] J. Abesser and M. Muller, ”Fundamental Frequency Contour Classi fi cation: A Comparison between Hand- crafted and CNN-based Features,” in ICASSP 2019, 2019. • [Kroher 14] N. Kroher and E.Gomez. ”Automatic singer identi fi ca- tion for improvisational styles based on vibrato, timbre and statistical performance descriptors.” in ICMC-SMC 2014, 2014. • [Breiman 01] L. Breiman. ”Random forests,” Machine learning, Vol. 45, No. 1, pp. 5-32, 2001. • [Chen 04] C. Chen, A. Liaw, L. Breiman: Using Random Forest to Learn Imbalanced Data, Technical Report, No.666, 2004. • [Luo 20] Yin-Jyun Luo, Chin-Cheng Hsu, Kat Agres, and Dorien Herremans. Singing voice conversion with disentangled representations of singer and vocal technique using vari- ational autoencoders. In Proceedings of the IEEE International Conference on Acous- tics, Speech, and Signal Processing (ICASSP), pp. 3277–3281. IEEE, 2020. • [Pishdadian 19] F. Pishdadian, B. Kim, P. Seetharaman, and B. Pardo. ”Classifying non-speech vocals: Deep vs signal process- ing representations.” in Acoustic Scenes and Events 2019 Workshop (DCASE2019), 2019. • [Dridger 16] J. Driedger, S. Balke, S. Ewert, and M. Muller. ”Template-based vibrato analysis in music signals.” in ISMIR 2016, 2016. • [Wang 20] C. Wang, V. Lostanlen, E. Benetos, and E. Chew. Playing technique recognition by joint time–frequency scattering. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 881–885. IEEE, 2020. -
͜ͷWORKͷઌʹԿ͕͋Δͷ͔ʁ 22 Վ এ ς Ϋ χ ο Ϋ ࣝ
ผ Վ এ ς Ϋ χ ο Ϋ ݕ ग़ from Ξ Χ ϖ ϥ Վ এ Վ এ ς Ϋ χ ο Ϋ ݕ ग़ from ͭ ͖ Վ এ σʔληοτͷ น ͷରԠͷ น -
VocalSetʹ͓͚ΔϊΠδʔϥϕϧɾऑϥϕϧ • ϊΠδʔϥϕϧʢϥϕϧͷใ͕ਖ਼֬Ͱͳ͍ʣ • ऑϥϕϧʢग़ݱ͕Ұ෦ͷՕॴͷΈʣ 23 Breathy, ΞϧϖδΦ, உ Belt,
εέʔϧ, ঁ ໌Β͔ʹϏϒϥʔτͷ Α͏ͳԻߴมಈ͕ ΈΒΕΔ͕ɼ 7JCSBUPͷϥϕϧͳ͠ ϘʔΧϧϑϥΠΛ ग़͍ͤͯΔͷ ਫ෦ͷϑϨʔϜͷΈ