Living in Perfect Harmony: Where Music and Machine Learning Meet

The revolution of machine learning is reaching every aspect of our lives - including art and music.
In this talk, we will dive into the world of song analysis and the extraction of lyrical and musical features. We will discuss existing approaches, both in machine learning - Natural Language Processing, Digital Signal Processing - and in music theory & linguistics. Next, we will see how we can use these features in different kinds of machine learning models, and how these models can be used to solve problems in the music industry, such as song tags and song similarity.
Attend this talk to learn how your technical skills can be useful also to your hobbies.

Transcript

1. Living in Perfect Harmony: Where Music and Machine Learning Meet Yama Anin Aminof
2. Yama Anin Aminof Data Scientist @ Meta B.Sc in Math & Physics Saxophone Player Public Speaker
3. AGENDA
4. AGENDA Song Tags Song Similarity
5. AGENDA Lyrical Features Song Tags Song Similarity Musical Features
6. Use Cases
7. Use Cases
8. Song Tags
9. Song Similarity A B Ө
10. Feature Extraction
11. Domain Knowledge Technical Approach
12. Domain Knowledge Music Theory Linguistics Technical Approach
13. Domain Knowledge Music Theory Linguistics Technical Approach Natural Language Processing Digital Signal Processing
14. Feature Extraction: Lyrics
15. You walked into the party Like you were walking onto a yacht … You're so vain You probably think this song is about you You're so vain I'll bet you think this song is about you Don't you? Don't you? You had me several years ago When I was still quite naive … You're so vain You probably think this song is about you You're so vain I'll bet you think this song is about you Don't you? Don't you? Don't you? You’re So Vain / Carly Simon
16. You walked into the party Like you were walking onto a yacht … You're so vain You probably think this song is about you You're so vain I'll bet you think this song is about you Don't you? Don't you? You had me several years ago When I was still quite naive … You're so vain You probably think this song is about you You're so vain I'll bet you think this song is about you Don't you? Don't you? Don't you? You’re So Vain / Carly Simon
17. You walked into the party Like you were walking onto a yacht … You're so vain You probably think this song is about you You're so vain I'll bet you think this song is about you Don't you? Don't you? You had me several years ago When I was still quite naive … You're so vain You probably think this song is about you You're so vain I'll bet you think this song is about you Don't you? Don't you? Don't you? You’re So Vain / Carly Simon
18. You walked into the party Like you were walking onto a yacht … You're so vain You probably think this song is about you You're so vain I'll bet you think this song is about you Don't you? Don't you? You had me several years ago When I was still quite naive … You're so vain You probably think this song is about you You're so vain I'll bet you think this song is about you Don't you? Don't you? Don't you? You’re So Vain / Carly Simon
19. Levenshtein Distance https://dev.to/thormeier/algorithm-explained-levenshtein-edit-distance-5f74
20. You walked into the party Like you were walking onto a yacht … You're so vain You probably think this song is about you You're so vain I'll bet you think this song is about you Don't you? Don't you? You had me several years ago When I was still quite naive … You're so vain You probably think this song is about you You're so vain I'll bet you think this song is about you Don't you? Don't you? Don't you? You’re So Vain / Carly Simon
21. 21 V = Verse PC = Pre Chorus CH = Chorus CP = C-Part V-PC-CH V-CH-V-CH-CP V-CH-V-CH Song Structures
22. 22 Compression Ratio, by Year Colin Morris: Are Pop Lyrics Getting More Repetitive? Top 10 songs Top 100 songs
23. 23 Top 10 songs Top 100 songs Compression Ratio Compression Ratio, by Year Colin Morris: Are Pop Lyrics Getting More Repetitive?
24. 24 Top 10 songs Top 100 songs Compression Ratio Compression Ratio, by Year Colin Morris: Are Pop Lyrics Getting More Repetitive?
25. WORD EMBEDDING
26. WORD EMBEDDING
27. WORD EMBEDDING word2vec
28. WORD EMBEDDING doc2vec
29. WORD EMBEDDING Semantle
30. DATA
32. You probably think this song is about you is you this so song on about think what probably 1 2 1 0 1 0 1 1 0 1
33. You probably think this song is about you is you this so song on about think what probably 1 2 1 0 1 0 1 1 0 1 Cons: Loss of information Pros: Accessible and easy to use
37. 1) Wake me up when it's over After the ending When the damage has all been done I don't wanna be somewhere Where you can watch me as I bleed Just leave me here in pieces 2) If this world is wearing thin And you're thinking of escape I'll go anywhere with you Just wrap me up in chains But if you try to go alone Don't think I'll understand
38. 1) Backstreet Boys Wake me up when it's over After the ending When the damage has all been done I don't wanna be somewhere Where you can watch me as I bleed Just leave me here in pieces 2) Cradle Of Filth If this world is wearing thin And you're thinking of escape I'll go anywhere with you Just wrap me up in chains But if you try to go alone Don't think I'll understand
39. 1) Backstreet Boys Wake me up when it's over After the ending When the damage has all been done I don't wanna be somewhere Where you can watch me as I bleed Just leave me here in pieces 2) Shakespears Sister If this world is wearing thin And you're thinking of escape I'll go anywhere with you Just wrap me up in chains But if you try to go alone Don't think I'll understand
40. Feature Extraction: Music
41. Digital Signal Processing https://towardsdatascience.com/extract-features-of-music-75a3f9bc265d
42. Spectral Features
43. Spectral Features
44. Spectral Features
45. Spectral Features
46. Rhythm Features
47. import librosa amps, rate = librosa.load('Path_to_wav_file.wav') tempo = librosa.beat.tempo(amps, rate)[0] spectral_centroid = librosa.feature.spectral_centroid(amps, rate)
48. import librosa amps, rate = librosa.load('Path_to_wav_file.wav') tempo = librosa.beat.tempo(amps, rate)[0] spectral_centroid = librosa.feature.spectral_centroid(amps, rate)
49. Harmony & Melody Analysis
50. Harmony & Melody Analysis
51. Source Separation
52. Source Separation
53. Source Separation
54. Source Separation
55. Musical Features Harmony Unique Chords: # unique chords / # chords Melody Melodic Range: highest note - lowest note
56. SUMMARY
57. SUMMARY ▹ Song tags ▹ Song similarity ▹ Lyrics Repetition ▹ Word embedding & doc2vec ▹ Digital Signal Processing & Librosa ▹ Source Separation
58. The Potential in Using ML in the Music Industry
59. You Don’t Have to Be a DSP / NLP Specialist to Analyze Songs
60. You Don’t Have to Be a Musicologist / Linguist to Analyze Songs
61. It’s Never Too Late
62. THANK YOU! linkedin.com/in/yama-aa Credits: This presentation template was created by SlidesCarnival, Photographs by Unsplash Yama Anin Aminof