Upgrade to Pro — share decks privately, control downloads, hide ads and more …

推しと始めるMIR

8cd09b3e65fed8fdb6072d8dfe7a7a60?s=47 てーとく
February 13, 2020

 推しと始めるMIR

アイドルのファンになったことをきっかけに機械学習の分野の一つであるMIR(音楽情報検索)に入門したので、MIRについて紹介しつつ作ったものの話とかをゆるふわにしようと思います!

8cd09b3e65fed8fdb6072d8dfe7a7a60?s=128

てーとく

February 13, 2020
Tweet

Transcript

  1. ਪ͠ͱ࢝ΊΔMIR ͯʔͱ͘ (@tetoku_sakana) 2020-02-13 #stapy54

  2. ͜Μͳײ͡ͰਐΈ·͢ • ࣗݾ঺հ • MIRͱ͸ • MIRͷ࣮ྫ঺հ • MIRͷ࢝Ίํ

  3. ୭ • ͯʔͱ͘ (@tetoku_sakana) • WebΤϯδχΞ • nao_y ͞Μͷ͓༠͍ͰࢀՃͤ͞ ͍ͯͨͩ͘͜ͱʹͳΓ·ͨ͠

  4. None
  5. ਪ͠ࣄ ᶃ • ΦαΧφϝʔλʔ • @osakanameter • ΦαΧφͷMVͷ࠶ੜճ ਺΍ϑΥϩϫʔ਺ͷՄ ࢹԽ΍௨஌

    • ެࣜϗʔϜϖʔδͷߋ৽ ৘ใ΍χϡʔε౳ͷ഑৴
  6. ਪ͠ࣄ ᶄ • ΦαΧφΞʔΧΠϒ • ΦαΧφʹ·ͭΘΔ ΠϯλϏϡʔهࣄͳ ͲΛ·ͱΊͨαʔϏ ε

  7. ͦΜͳ͜ΜͳͰ MIRʹೖ໳͠·ͨ͠ʂ

  8. MIR

  9. None
  10. None
  11. MIR • Music Information Retrieval • Իָ৘ใݕࡧ΍Իָ৘ใॲཧͱ༁͞ΕΔ • ݕࡧ͸͋Μ·Γؔ܎ͳ͍ •

    ػցֶश×Իָ • (ओʹඇੜ੒ܥͷ) Իָؔ࿈෼໺ͷ૯শ
  12. MIRͷλεΫ (Ұྫ) • Իָ΍Իָ৘ใ (ௐ΍ςϯϙɺίʔυ౳) ͷݕ ग़ɾਪఆ • Իָͷࣗಈ෼ྨ (δϟϯϧ΍งғؾͳͲ)

    • ࣖίϐͷࣗಈԽ • ԻָͰԻָΛݕࡧ
  13. MIRͷख๏ • Content-based • Իָ΍ՎࢺͳͲɺָۂσʔλΛѻ͏ • Context-based • ΞʔςΟετͷ঺հจͳͲɺָۂҎ֎ͷपล σʔλΛѻ͏

  14. pythonͱMIR • librosa • madmom • essentia

  15. import librosa >>> filepath = librosa.util.example_audio_file() >>> y, sr =

    librosa.load(filepath, offset=30, duration=5) >>> librosa.feature.mfcc(y=y, sr=sr) # MFCCͷऔಘ array([[ -5.229e+02, -4.944e+02, ..., -5.229e+02, -5.229e+02], [ 7.105e-15, 3.787e+01, ..., -7.105e-15, -7.105e-15], ..., [ 1.066e-14, -7.500e+00, ..., 1.421e-14, 1.421e-14], [ 3.109e-14, -5.058e+00, ..., 2.931e-14, 2.931e-14]])
  16. ΦαΧφͷۂ͸೉͍͠…

  17. ΦαΧφͷதͰ Ұ൪೉ָ͍͠ۂ͸ʁ

  18. ϚεϩοΫࢦ਺

  19. None
  20. • Elias Pampalk et al., Proceedings of the ACM Multimedia

    2002 • ָۂྨࣅ౓Λࣗݾ૊৫ԽϚοϓ(SOM)Λ࢖ͬ ͯՄࢹԽ Content-based Organization and Visualization of Music Archives
  21. Իڹ৺ཧֶΛߟྀͨ͠ɺௌײ্ͷloudnessͷม ԽΛಛ௃ྔͱͯ͠நग़͢Δ “Rhythm Patterns”ͱͯ͠ఏҊ͞ΕͯΔಛ௃ྔநग़ख๏

  22. None
  23. ϚεϩοΫࢦ਺͕ ࢉग़Ͱ͖ͦ͏ʂ

  24. ॲཧ֓ཁ 1. STFTΛ͔͚ͯ(ର਺ई౓ͷ) εϖΫτϩάϥϜ Λऔಘ 2. (1) ΛϒϩοΫʹ෼ׂ͠(rolling window)ɺͦ ΕͧΕ࣌ؒ࣠ํ޲ʹSTFTΛ͔͚ͯέϓετϩ

    άϥϜΛऔಘ 3. (2) ʹରͯ͠60ύʔηϯλΠϧΛٻΊΔ
  25. None
  26. None
  27. def minmax(pattern): return (pattern - pattern.min()) / (pattern.max() - pattern.min())

    # ࡶ def mathrock_index(pattern): pattern = pattern.sum(axis=0) pattern = minmax(pattern) * 100 pattern = np.diff(pattern) return np.percentile(pattern, q=90) def calc_lfp(filename): cent = CentSpectrum(win_length=2048, hop_length=512) D = librosa.amplitude_to_db(cent.proc(filename)) D_normalized = cent.normalize(D) lfp = LogarithmicFluctuationPattern(hop_length=256) return lfp.proc(D_normalized)
  28. None
  29. MIRͷ࢝Ίํ

  30. MIRͷ࢝Ίํ • MIREX΍ISMIRͷ࿦จ • ipynb • musicinformationretrieval.com • ΢Οʔϯ޻Պେͷnbviewer

  31. MIRͷ࢝Ίํ • SpotifyͷAPIΛ࢖͏ • Audio Features for a Track •

    Audio Analysis for a Track
  32. { "danceability": 0.735, "energy": 0.578, "key": 5, "loudness": -11.84, "mode":

    0, "speechiness": 0.0461, "acousticness": 0.514, "instrumentalness": 0.0902, "liveness": 0.159, "valence": 0.624, "tempo": 98.002, "type": "audio_features", "id": "06AKEBrKUckW0KREUWRnvT", "uri": "spotify:track:06AKEBrKUckW0KREUWRnvT", "track_href": “https://api.spotify.com/v1/tracks/…", "analysis_url": “https://api.spotify.com/v1/audio-analysis/…”, "duration_ms": 255349, "time_signature": 4 }
  33. None
  34. ·ͱΊ • ৘ใ͸গͳ͍͚ͲMIRͷෑډ͸௿͍ • ϥΠϒϥϦ͸librosa͕͓͢͢Ί • SpotifyͷAPI΋͓͢͢Ί • ΦαΧφ͸ྑ͍

  35. ਪ͠ۦಈ։ൃ Ұॹʹ࢝ΊͯΈ·͠ΐ͏ ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʙ