Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Making Machines that Make Music

Making Machines that Make Music

As presented in Euroclojure 2016.

Srihari Sriraman

October 26, 2016

More Decks by Srihari Sriraman

Other Decks in Technology


  1. Making Machines that Make Music Srihari Sriraman nilenso

  2. should we listen to some now?

  3. Why I do this I Sing, I do computers Bleeding

    edge research Dreamy ambitions Interesting By-products
  4. What this talk is about Melody Modelling Synthesis Generation Melody

    Modelling Synthesis Generation
  5. Melody ˈmɛlədi noun a sequence of single notes that is

    musically satisfying; a tune.
  6. Carnatic Music

  7. Carnatic Music Kalyani, Extempore MS Gopalakrishnan, Violin

  8. Khamas, Thillana Abhishek Raghuram

  9. Carnatic Music South Indian classical music Ragas, Gamakams Vocal Tradition

    Rich in compositions Extempore / Manodharma
  10. Tanpura Veena Mridangam

  11. The foundations of musical abstractions in Carnatic music

  12. Shruthi Tonic note Choice of artist Swarams are relative to

  13. Laya Rhythm concepts Similar to time signatures A rather mature

  14. Sa Ri Ga Ma Pa Da Sa Ni Swarams

  15. Sa Ri Ga Ma Pa Da Sa Ni S R

    G M P D S N R1 R2 R3 G1 G2 G3 M1 M2 D1 D2 D3 N1 N2 N3 Notation Pronunciation Variations
  16. Swarams The 12 semitones Elements of a raga Simples are

    sung Prescriptive notation
  17. Rāgā Kaapi, Extempore TM Krishna

  18. Raga Has a name Rule to ascend Rule to descend

    Not necessarily symmetric Not necessarily linear Grouped into families
  19. Raga Has a name Rule to ascend Rule to descend

    Not necessarily symmetric Not necessarily linear Grouped into families
  20. Demo of the fundamental abstractions.

  21. None
  22. Tools & Libraries

  23. Fuzzy search For indic languages Needs to be fast Primary

    stitching mechanism Helps with multi-source data
  24. A quick recap Play the scale of a raga Fuzzy

    find a raga Play a phrase Play a phrase in the context of a raga Play some prescriptive notation
  25. But.. ..that doesn’t sound like Carnatic music, does it?

  26. Synthesis

  27. Enter Melographs Me Machine

  28. another phrase Me Machine

  29. Prescriptive vs Descriptive

  30. Gamakams

  31. Sphuritam Orikai Jaaru Kampitam Sphuritam Nokku Ravai Kandippu Ullasitam Etra-jaru

    Iraka-jaru Odukkal Orikai Vali Kampitam
  32. Gamakams in SSP

  33. Gamakams in SSP

  34. Gaayaka | S, N D | N S R G

    | ((P S,,)) , ((S , S>>> S)) -((D. S. D)) ((S , S>> S))- S R ((G<< G , ,)) Subramanian, 2009 Database of phrases Automatic Gamakam feature – guided
  35. Modelling Gamakams

  36. None
  37. Me Machine Back to this…

  38. PASR Srikumar 2013 Pitch, Attack, Sustain, Release Vector specifies the

    PASR vars for each prescriptive note
  39. Me Machine Rendering PASR…

  40. Rendering PASR…

  41. Generation

  42. Random | Within a raga

  43. Random | Within a raga

  44. Get data

  45. Kosha An Open Carnatic Music Database http://github.com/ssrihari/kosha

  46. None
  47. None
  48. None
  49. Study data

  50. None
  51. Melographs

  52. Melographs kalyANi-MS-Subbulakshmi-nidhi_cAla_sukhamA-tyAgarAja3.mpeg.wav.pitch.frequencies-pitch-histogram kalyANi-Kunnakudi-R-Vaidyanathan-nidhi_cAla_sukhamA-tyAgarAja49.mpeg.wav.pitch.frequencies-pitch-histogram

  53. Pitch Histograms

  54. Pitch Histograms kalyANi-MS-Subbulakshmi-nidhi_cAla_sukhamA-tyAgarAja3.mpeg

  55. Pitch Histograms Kalyani - Vocal Kalyani - Violin Mohana -

    Mandolin Mohana - Vocal Revati - Vocal Revati - Instrumental
  56. Extract Music Information

  57. Midi Histogram Normalised Midi Histogram

  58. Tonic note identification Bellur, A., V. Ishwar, X. Serra, and

    H. A. Murthy (2012) A knowledge based signal processing approach to tonic identification in indian classical music.
 Bellur, A., and H. A. Murthy (2013) Automatic tonic identification in classical music using melodic characteristics and tuning of the drone. Srihari, S. (2016) * Pick the most frequent note, it mostly just works. * not really, no
  59. Tonic note identification

  60. Tonic note identification

  61. Swaram Histogram Kalyani S, R2, G3, M2, P, D2, N3,

    S. S., N3, D2, P, M2, G3, R2, S
  62. Kalyani S, R2, G3, M2, P, D2, N3, S. S.,

    N3, D2, P, M2, G3, R2, S Revati S, R1, M1, P, N2, S. S., N2, P, M1, R1, S Mohana S, R2, G3, P, D2, S. S., D2, P, G3, R2, S
  63. Generation with weighted probabilities

  64. None
  65. In comparison with random Random Single Swaram Weighted

  66. Melody insights #1 Tonic note is prominent Sa, and Pa

    have higher and sharper peaks Other note peaks are blunt Probabilities of all swarams in a raga are not the same Probabilities across octaves are not the same
  67. Two swaram probabilities

  68. Two swaram probabilities Prominence of adjacency Encoded rules of Arohanam

    and Avarohanam
  69. Two swaram probabilities

  70. Single swaram vs Two swarams Two Swaram Weighted Single Swaram

  71. Melody insights #2 Swarams close to each other are more

    melodious The rules of Arohanam, Avarohanam are encoded We begin to see gamakams Sometimes, the in-between is worse than either extreme
  72. Three swaram probabilities and more A simple markov chain

  73. First Order Matrix https://en.wikipedia.org/wiki/Markov_chain#Music Markov Chains in Music https://github.com/rm-hull/markov-chains Second

    Order Matrix
  74. Markov Chains in Music

  75. None
  76. Melody insights #3 Generic markov chains don’t really work LSTMs

    also don’t work, probably
  77. By-products

  78. Automatic Transcription

  79. Automatic Transcription (:..n1 :..n1 :..m1 :..m1 :..d1 :..d1 :..d3 :.g1

    :.m1 :.m1 :.m1 :.r3 :.r1 :..n3 :..d3 :..p :..g3 :..m1 :..m2 :..g2 :..m1 :..g3 :.r1 :.s :.s :..r1 : ..p :.s :..n3 :.s :.s :.g1 :.s :..d3 :..n1 :..n1 :..n1 :..r1 :.s :.s :.g1 :.r3 :.g1 :.r1 :..d3 :..d3 :..d3 :.g1 :.r1 :.s :.g1 :..r2 :..r1 :.s :.r3 :..n3 :..d3 :..d3 :..n1 :..n1 :..n1 :..n1 :..n1 :..p :.s :..n1 :..g2 :..n 3 :.r1 :.g3 :.g3 :.m1 :.r3 :.m1 :.g3 :.g3 :.p :.m1 :.m1 :..m1 :..g3 :.m1 :..r2 :..r2 :..n3 :.s :.s :.g1 :.g3 :.m2 :.p :.d2 :.m2 :.m1 :.r3 :.r3 :.g 1 :.g1 :.g1 :.r1 :..n1 :.r3 :.g3 :.s :.s :.r1 :.g1 :.r1 :..n3 :..n1 :..d3 :..d3 :..n1 :..d3 :..n1 :..n1 :..n1 :..r1 :.s :..n1)
  80. Raga Identification

  81. Goodness of fit test

  82. :base mohanam-base :samples mohanam-files (12.39 3.84 11.14 6.46 9.88 7.02

    9.41 12.61 13.22 1.58) :base mohanam-base :samples kalyani-files (10.95 28.66 25.61 15.26 27.32 21.53 16.42 18.58 24.80 23.80) :base mohanam-base :samples revati-files (46.56 57.19 65.69 55.21 38.61 78.10 56.27 42.99 70.92 58.39)
  83. Raga Identification Revati sample vs Revati base Mohana sample vs

    Revati base
  84. What next Model insights as melodic abstractions Use synthesis models

    with generative music Experiment with Rhythm Synthesise Human Voice Deep learning (Recurrent variational auto encoders)
  85. Is this music though?

  86. Behag Dasarapada Abhishek Raghuram

  87. Making Machines that Make Music Srihari Sriraman nilenso