推しと始めるMIR

アイドルのファンになったことをきっかけに機械学習の分野の一つであるMIR(音楽情報検索)に入門したので、MIRについて紹介しつつ作ったものの話とかをゆるふわにしようと思います！

Transcript

1. ਪ͠ͱ࢝ΊΔMIR ͯʔͱ͘ (@tetoku_sakana) 2020-02-13 #stapy54
2. ͜Μͳײ͡ͰਐΈ·͢ • ࣗݾ঺հ • MIRͱ͸ • MIRͷ࣮ྫ঺հ • MIRͷ࢝Ίํ
3. ୭ • ͯʔͱ͘ (@tetoku_sakana) • WebΤϯδχΞ • nao_y ͞Μͷ͓༠͍ͰࢀՃͤ͞ ͍ͯͨͩ͘͜ͱʹͳΓ·ͨ͠
5. ਪ͠ࣄ ᶃ • ΦαΧφϝʔλʔ • @osakanameter • ΦαΧφͷMVͷ࠶ੜճ ਺΍ϑΥϩϫʔ਺ͷՄ ࢹԽ΍௨஌ • ެࣜϗʔϜϖʔδͷߋ৽ ৘ใ΍χϡʔε౳ͷ഑৴
6. ਪ͠ࣄ ᶄ • ΦαΧφΞʔΧΠϒ • ΦαΧφʹ·ͭΘΔ ΠϯλϏϡʔهࣄͳ ͲΛ·ͱΊͨαʔϏ ε
7. ͦΜͳ͜ΜͳͰ MIRʹೖ໳͠·ͨ͠ʂ
8. MIR
11. MIR • Music Information Retrieval • Իָ৘ใݕࡧ΍Իָ৘ใॲཧͱ༁͞ΕΔ • ݕࡧ͸͋Μ·Γؔ܎ͳ͍ • ػցֶश×Իָ • (ओʹඇੜ੒ܥͷ) Իָؔ࿈෼໺ͷ૯শ
12. MIRͷλεΫ (Ұྫ) • Իָ΍Իָ৘ใ (ௐ΍ςϯϙɺίʔυ౳) ͷݕ ग़ɾਪఆ • Իָͷࣗಈ෼ྨ (δϟϯϧ΍งғؾͳͲ) • ࣖίϐͷࣗಈԽ • ԻָͰԻָΛݕࡧ
13. MIRͷख๏ • Content-based • Իָ΍ՎࢺͳͲɺָۂσʔλΛѻ͏ • Context-based • ΞʔςΟετͷ঺հจͳͲɺָۂҎ֎ͷपล σʔλΛѻ͏
14. pythonͱMIR • librosa • madmom • essentia
15. import librosa >>> filepath = librosa.util.example_audio_file() >>> y, sr = librosa.load(filepath, offset=30, duration=5) >>> librosa.feature.mfcc(y=y, sr=sr) # MFCCͷऔಘ array([[ -5.229e+02, -4.944e+02, ..., -5.229e+02, -5.229e+02], [ 7.105e-15, 3.787e+01, ..., -7.105e-15, -7.105e-15], ..., [ 1.066e-14, -7.500e+00, ..., 1.421e-14, 1.421e-14], [ 3.109e-14, -5.058e+00, ..., 2.931e-14, 2.931e-14]])
16. ΦαΧφͷۂ͸೉͍͠…
17. ΦαΧφͷதͰ Ұ൪೉ָ͍͠ۂ͸ʁ
18. ϚεϩοΫࢦ਺
20. • Elias Pampalk et al., Proceedings of the ACM Multimedia 2002 • ָۂྨࣅ౓Λࣗݾ૊৫ԽϚοϓ(SOM)Λ࢖ͬ ͯՄࢹԽ Content-based Organization and Visualization of Music Archives
21. Իڹ৺ཧֶΛߟྀͨ͠ɺௌײ্ͷloudnessͷม ԽΛಛ௃ྔͱͯ͠நग़͢Δ “Rhythm Patterns”ͱͯ͠ఏҊ͞ΕͯΔಛ௃ྔநग़ख๏
23. ϚεϩοΫࢦ਺͕ ࢉग़Ͱ͖ͦ͏ʂ
24. ॲཧ֓ཁ 1. STFTΛ͔͚ͯ(ର਺ई౓ͷ) εϖΫτϩάϥϜ Λऔಘ 2. (1) ΛϒϩοΫʹ෼ׂ͠(rolling window)ɺͦ ΕͧΕ࣌ؒ࣠ํ޲ʹSTFTΛ͔͚ͯέϓετϩ άϥϜΛऔಘ 3. (2) ʹରͯ͠60ύʔηϯλΠϧΛٻΊΔ
27. def minmax(pattern): return (pattern - pattern.min()) / (pattern.max() - pattern.min()) # ࡶ def mathrock_index(pattern): pattern = pattern.sum(axis=0) pattern = minmax(pattern) * 100 pattern = np.diff(pattern) return np.percentile(pattern, q=90) def calc_lfp(filename): cent = CentSpectrum(win_length=2048, hop_length=512) D = librosa.amplitude_to_db(cent.proc(filename)) D_normalized = cent.normalize(D) lfp = LogarithmicFluctuationPattern(hop_length=256) return lfp.proc(D_normalized)
29. MIRͷ࢝Ίํ
30. MIRͷ࢝Ίํ • MIREX΍ISMIRͷ࿦จ • ipynb • musicinformationretrieval.com • ΢Οʔϯ޻Պେͷnbviewer
31. MIRͷ࢝Ίํ • SpotifyͷAPIΛ࢖͏ • Audio Features for a Track • Audio Analysis for a Track
32. { "danceability": 0.735, "energy": 0.578, "key": 5, "loudness": -11.84, "mode": 0, "speechiness": 0.0461, "acousticness": 0.514, "instrumentalness": 0.0902, "liveness": 0.159, "valence": 0.624, "tempo": 98.002, "type": "audio_features", "id": "06AKEBrKUckW0KREUWRnvT", "uri": "spotify:track:06AKEBrKUckW0KREUWRnvT", "track_href": “https://api.spotify.com/v1/tracks/…", "analysis_url": “https://api.spotify.com/v1/audio-analysis/…”, "duration_ms": 255349, "time_signature": 4 }
34. ·ͱΊ • ৘ใ͸গͳ͍͚ͲMIRͷෑډ͸௿͍ • ϥΠϒϥϦ͸librosa͕͓͢͢Ί • SpotifyͷAPI΋͓͢͢Ί • ΦαΧφ͸ྑ͍
35. ਪ͠ۦಈ։ൃ Ұॹʹ࢝ΊͯΈ·͠ΐ͏ ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʙ