Making machines that make music – Strangeloop 2018

Slide 1

Slide 1 text

Making machines that make music ीहर ीरामन् नलेो Srihari Sriraman nilenso

Slide 2

Slide 2 text

Why I do this I sing, I do computers Bleeding edge research A long time hobby Re-discovery

Slide 3

Slide 3 text

Structure of this talk Universality, Carnatic music A timeline of 4 phases Bridging the gap between machines and humans Understand domain, create abstractions Synthesise ⟷ Generate ⟷ Transcribe Reﬂect on melody insights

Slide 4

Slide 4 text

This is what it sounds like [play synthesised music]

Slide 5

Slide 5 text

Prominent tonic Resolves tension from musical movement Pervasive pentatonic scales Common in a lot of cultures: celtic, chinese, african, english, indian, etc 7 note scales, 12 semitones Latin, Chinese, Indian cultures have similar solfege Emotive melodies Lullabies, sing-alongs, march songs, festival songs, hymns Universality Natural phenomena observed around the world

Slide 6

Slide 6 text

Music is Math / Science / Art

Slide 7

Slide 7 text

Universality

Slide 8

Slide 8 text

Universality Natural phenomenons Prominent tonic Pervasive pentatonic scales 7 note scales, 12 semitones Emotive melodies Tonic | आधार षज - Resolves tension from musical movement. - Everyone knows when a piece of music returns to the tonic.

Slide 9

Slide 9 text

Pentatonic | औडुवम् Common in a lot of cultures: celtic, chinese, african, english, indian, etc hps://en.wikipedia.org/wiki/Pentatonic_scale#Pervasiveness Popular Bobby McFerrin demonstration of the power of the pentatonic scale hps://www.youtube.com/watch?v=ne6tB2KiZuk Universality Natural phenomenons Prominent tonic Pervasive pentatonic scales 7 note scales, 12 semitones Emotive melodies

Slide 10

Slide 10 text

Universality Natural phenomenons Prominent tonic Pervasive pentatonic scales 7 note scales, 12 semitones Emotive melodies 7 notes | स राः Latin: do re mi fa sol la si Chinese: shàng chě gōng fán liù wù yi Indian: sa ri ga ma pa da ni

Slide 11

Slide 11 text

Universality Natural phenomenons Prominent tonic Pervasive pentatonic scales 7 note scales, 12 semitones Emotive melodies Emotive melodies | रस Happy sing-alongs, lullabies, march songs, harvest songs, festival songs, hymns, funeral songs sound similar across cultures.

Slide 12

Slide 12 text

Carnatic music शाीय संगीतम्

Slide 13

Slide 13 text

Kalyani, Extempore MS Gopalakrishnan, Violin

Slide 14

Slide 14 text

Tanpura Veena Mridangam

Slide 15

Slide 15 text

Khamas, Thillana Abhishek Raghuram

Slide 16

Slide 16 text

Classical, art music - Not Tribal | Folk | Religious | Popular - Intent is artistic - Music itself is the sole focus Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions

Slide 17

Slide 17 text

Ancient, traditional music – Earliest treatise ना शा (Natya Shastra) at 500BC – लण (Lakshana Grantha): Rich tradition of Sanskrit grammar texts – Oral tradition, trained by exposure Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions

Slide 18

Slide 18 text

Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions Digitisation hype cycle

Slide 19

Slide 19 text

Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions Carnatic music Digitisation hype cycle

Slide 20

Slide 20 text

Melodic music - Has lile place for harmony - Emphasis on notes in succession - Explores melody through ragas and gamakas Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions

Slide 21

Slide 21 text

Extempore, compositions - Similar to jazz music - Renditions of century old compositions - Improvisations are generally around compositions Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions

Slide 22

Slide 22 text

Model fundamental abstractions Render prescriptive notation Model and synthesise gamaka Phase #1

Slide 23

Slide 23 text

Fundamental abstractions आधार षज, र, राग

Slide 24

Slide 24 text

Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo Adhara Shadja आधार षज | Tonic, pitch - Choice of the artist based on voice - Does not change for every song - Swaras are relative to this

Slide 25

Slide 25 text

(def adhaara-shadja {:.a 57 :.a# 58 :.b 59 :c 60 :c# 61 :db 61 :d 62 :d# 63 :eb 63 :e 64 :f 65 :f# 66 :gb 66 :g 67 :g# 68 :ab 68 :a 69 :a# 70 :bb 70 :b 71 :c. 72}) Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 26

Slide 26 text

- The 12 semitones in an octave - Solfege is sung, and used as notation - Elements of a raga Swara र | Note Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 27

Slide 27 text

Sa Ri Ga Ma Pa Da Sa Ni

Slide 28

Slide 28 text

Sa Ri Ga Ma Pa Da Sa Ni S R G M P D S N R1 R2 R3 G1 G2 G3 M1 M2 D1 D2 D3 N1 N2 N3 Notation Pronunciation Variations

Slide 29

Slide 29 text

(def madhya-sthayi-sthanams {:s 0 :r1 1 :r2 2 :r3 3 :g1 2 :g2 3 :g3 4 :m1 5 :m2 6 :p 7 :d1 8 :d2 9 :d3 10 :n1 9 :n2 10 :n3 11}) Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 30

Slide 30 text

(def sthayis {:mandra {:position :before :dots "." :difference -12} :madhya {:position :none :dots "" :difference 0} :thara {:position :after :dots "." :difference 12}}) Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 31

Slide 31 text

Prescriptive Notation s,,r g,p, d,s., n,d, p,dp mgrs rsnd s,,, Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 32

Slide 32 text

Raga राग | Melodic framework Kaapi, Extempore TM Krishna

Slide 33

Slide 33 text

Raga राग / Melodic framework - Melodic modes with added specialities - Grammar that a composition adheres to - Evokes a certain set of emotions Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 34

Slide 34 text

ragavardhini.ragams> (count ragams) 5280 ragavardhini.ragams> (into {} (take 2 ragams)) {:mayamalavagowla {:num 15 :arohanam [:s :r1 :g3 :m1 :p :d1 :n3 :s.] :avarohanam [:s. :n3 :d1 :p :m1 :g3 :r1 :s]} :gaula {:arohanam [:s :r1 :m1 :p :n3 :s.] :avarohanam [:s. :n3 :p :m1 :r1 :g3 :m1 :r1 :s] :parent-mela-name :mayamalavagowla :parent-mela-num 15}} Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 35

Slide 35 text

Scale | आरोह, अवरोह – Example: – Aarohanam: 1 2 3 5 6 8 – Avarohanam: 8 7 6 5 4 3 2 1 – They form a skeleton of the raga – Can be asymmetric, and non-linear Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 36

Slide 36 text

(:ragam (search/search-ragam “goula")) {:arohanam (:s :r1 :m1 :p :n3 :s.) :avarohanam (:s. :n3 :p :m1 :r1 :g3 :m1 :r1 :s) :name :gaula :parent-mela-name :mayamalavagowla :parent-mela-num 15} (play-arohanam-and-avarohanam (:ragam (search/search-ragam “goula"))) Lookup and play the scale Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 37

Slide 37 text

(play-phrase (phrase [:s :r2 :g3 :p :m1 :g3 :r2 :s] [ 1 1 1 1 1 1 2 4] (:lower kalams))) (play-phrase (phrase (:mechakalyani r/ragams) [:m :d :n :g :m :d :r :g :m :g :m :d :n :s.] [ 1 1 2 1 1 2 1 1 4 1 1 1 1 4] (:middle kalams))) Play a phrase in a raga Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 38

Slide 38 text

(play-notation (:bilahari r/ragams) "s,,r g,p, d,s., n,d, p,dp mgrs rs .n .d s,,,") (play-notation (:mayamalavagowla r/ragams) "s,,r g,p, d,s., n,d, p,dp mgrs rs .n .d s,,,") Play prescriptive notation Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo

Slide 39

Slide 39 text

Etymology in Sanskrit सक ् क ृ तम् (correctly done) = सं ृ तम् Speciﬁes grammar for creating new words Words can be composed (समास / Samāsa) Elegant solution to the naming problem in soware

Slide 40

Slide 40 text

मेव राजते इत र - that which shines on its own - In the context of grammar, “swara” means vowel. A consonant in Sanskrit cannot be pronounced without a vowel. Etymology Independent Resonant Swara र | Note

Slide 41

Slide 41 text

ः: अनुरणनाक: तो रयत - that which shines on its own - resonant, unlike spoken words Sangita Ratnakara, Sarngadeva, 12 CE Swara र | Note

Slide 42

Slide 42 text

Raga / राग – colour: hue, tint, dye – emotion: desire, interest, anger, melancholy, joy, delight, yearning – More than a scale: phraseology / रागवाचक Raga राग / Melodic framework

Slide 43

Slide 43 text

More than a scale – A class of ragas has a phraseology / रागवाचक – Learning a raga is akin to learning a language by exposure (say a mother tongue) – A raga is absorbed Etymology Color, Emotion More than a scale Raga राग / Melodic framework

Slide 44

Slide 44 text

So far.. ..we’ve managed to play some prescriptive notation within the context of a raga

Slide 45

Slide 45 text

But.. ..that doesn’t sound like Carnatic music

Slide 46

Slide 46 text

Enter Melographs Me Machine

Slide 47

Slide 47 text

Enter Melographs Me Machine

Slide 48

Slide 48 text

Gamaka

Slide 49

Slide 49 text

छाया ुरा याम् रो य गमये What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR Gamaka गमक / Ornamentation Movement of a note from it’s pitch towards another so that the second passes like shadow over it. Sangita samayasaara, Parsvadeva, 12CE

Slide 50

Slide 50 text

- Embellishment, ornamentation - Shake, push, glide, ﬂick - Pervasive in Carnatic music - Every swaram is a continuum What is it? Gamaka गमक / Ornamentation What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR

Slide 51

Slide 51 text

Sphuritam Jaaru What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR Gamaka गमक / Ornamentation

Slide 52

Slide 52 text

Kampitam Orikai What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR Gamaka गमक / Ornamentation

Slide 53

Slide 53 text

Prescriptive vs Descriptive What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR Gamaka गमक / Ornamentation

Slide 54

Slide 54 text

Prior art Gaayaka | Subramanian, 2009 | S, N D | N S R G | ((P S,,)) , ((S , S>>> S)) -((D. S. D)) ((S , S>> S))- S R ((G<< G , ,)) - Database of phrases - < and > increase and decrease pitch - ( and ) are speed factors; deeper is faster What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR Gamaka गमक / Ornamentation

Slide 55

Slide 55 text

PASR | Srikumar, 2013 - Pitch, Aack, Sustain, Release - Vector speciﬁes the PASR vars for each note [["^pa:2" [[4 0 0 0] [2 2 0 0] [7 2 4 0]]] ["^ma1:2" [[5 0 8 0]]] ["^ga3:2" [[5 0 6 0] [4 0.5 0 0.5] [5 1 0 0]]] ["^ga3" [[5 0 0 0.5] [4 1 1 1] [5 0.5 0 0]]] ["ma1" [[5 0 0 0.5] [4 1 1 1] [5 0.5 0 0]]] ["^ri2:2" [[5 0 0 0.5] [4 1 1 1] [5 0.5 0 0.5] [2 1 1 1] [4 0.5 0 0]]] ["^ga3" [[4 0 0 0.5] [2 1 1 1] [[5 0.2] 0.5 0 0]]] ["ri2" [[[5 0.2] 0 0 0.5] [2 1 1 1] [4 0.5 0 0]]] ["^sa:4" [[0 0 16 0]]]] - Prior art Gamaka गमक / Ornamentation What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR

Slide 56

Slide 56 text

SSP | Sangita Samradaya Pradarshini संगीत सादाय दषन What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR Gamaka गमक / Ornamentation

Slide 57

Slide 57 text

Gamaka गमक / Ornamentation What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR

Slide 58

Slide 58 text

- Frequency envelopes (g-inst sphuritam-inst (envelope [f f lf f f] [ d4 d1 d1 d4] :welch)) (g-inst jaru-inst (envelope [pf f f] [ d9 d1] :welch)) Synthesis Gamaka गमक / Ornamentation What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR

Slide 59

Slide 59 text

Back to this… Me Machine (play-phrase (string->phrase (:mohana r/ragams) "^g, ~r, ^s ^r ^g ^r" 1))

Slide 60

Slide 60 text

Rendering PASR… Gamaka गमक / Ornamentation What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR

Slide 61

Slide 61 text

Melody Insights #1 Prescriptive notation is insuicient for reproduction Raga as a framework for creating melodies is powerful Melographs make melodies tangible for observation Gamaka synthesis can be mathematically modelled as a curve

Slide 62

Slide 62 text

Generate a stream of Carnatic music Try to convert existing music into music data Study music data, look for paerns Infer characteristics of raga from music data Phase #2

Slide 63

Slide 63 text

Generation A naive approach

Slide 64

Slide 64 text

(defn gen-durations [] (let [m 16 n 8 sum (- m n)] (->> (repeatedly n #(rand-nth (range (inc sum)))) (cons sum) sort adjacent-differences (map inc)))) (defn random-swarams [swarams num] (repeatedly num #(rand-nth swarams))) (defn random-durations [jathi num] ;; [1, 2, 1, 1, 4, 1, 1, 2, 2, 4] (take num (flatten (repeatedly gen-durations)))) Generate random durations Pick swarams from a raga Generation A naive approach

Slide 65

Slide 65 text

Generate random durations Pick swarams from a raga Generation A naive approach (play-completely-random-phrase :mohana 4 100)

Slide 66

Slide 66 text

Transcription

Slide 67

Slide 67 text

Step 1. Build a massive database by scraping the internet Ragas, lyrics Ragas Compositions, Renditions Get data, study data Get data, study data Pitch histograms MIDI histograms Tonic identiﬁcation Swaram histograms Transcription Audio ﬁles to notes

Slide 68

Slide 68 text

Get data, study data Step 2. Build a user interface to search songs and ragas, so that I can see what’s in the database. Get data, study data Pitch histograms MIDI histograms Tonic identiﬁcation Swaram histograms Transcription Audio ﬁles to notes

Slide 69

Slide 69 text

Get data, study data Step 3. Download renditions of a given raga (->> "bhairavi" (db-search/renditions-in-ragam) (take 50) (download-renditions) hp://github.com/ssrihari/kosha Get data, study data Pitch histograms MIDI histograms Tonic identiﬁcation Swaram histograms Transcription Audio ﬁles to notes

Slide 70

Slide 70 text

Get data, study data Step 4. Transcribe the fundamental frequencies of vocals Get data, study data Pitch histograms MIDI histograms Tonic identiﬁcation Swaram histograms Transcription Audio ﬁles to notes

Slide 71

Slide 71 text

Get data, study data Step 5. Look at melographs, spectrograms, and try to find usable insights about melody. kalyANi-MS-Subbulakshmi-nidhi_cAla_sukhamA-tyAgarAja3.mpeg.wav.pitch.frequencies Get data, study data Pitch histograms MIDI histograms Tonic identification Swaram histograms Transcription Audio files to notes

Slide 72

Slide 72 text

Pitch histograms (defn pitch-histogram [filename] (let [frequencies (f/freqs-from-file filename) data-set (to-dataset frequencies) title (str "pitch histogram for " filename)] (with-data data-set (save (histogram (or plot-axis :freq) :density true :nbins 1200 :title title :x-label "frequency" :y-label "ocurrances") (str filename "-pitch-histogram.png") :width 1200 :height 300)))) Get data, study data Pitch histograms MIDI histograms Tonic identiﬁcation Swaram histograms Transcription Audio ﬁles to notes

Slide 73

Slide 73 text

Slide 74

Slide 74 text

kalyANi-MS-Subbulakshmi-nidhi_cAla_sukhamA-tyAgarAja3.mpeg Pitch histograms

Slide 75

Slide 75 text

Pitch histograms

Slide 76

Slide 76 text

MIDI Histogram Normalised MIDI Histogram (overtone.core/hz->midi frequency) (map #(mod % 12) frequencies)

Slide 77

Slide 77 text

Tonic identification Bellur, A., V. Ishwar, X. Serra, and H. A. Murthy (2012) A knowledge based signal processing approach to tonic identification in indian classical music. Bellur, A., and H. A. Murthy (2013) Automatic tonic identification in classical music using melodic characteristics and tuning of the drone. Srihari, S. (2016) * Pick the highest one, it mostly just works. * not really, no Get data, study data Pitch histograms MIDI histograms Tonic identification Swaram histograms Transcription Audio files to notes

Slide 78

Slide 78 text

Tonic identification (defn find-tonic-midi [freqs prominent-note] (let [midi-occurrances (midi-histogram freqs) prominent-midi-diff (->> midi-occurrances normalize-octaves (sort-by second >) ffirst)] (cond (= :s prominent-note) (+ 60 prominent-midi-diff) (= :p prominent-note) (+ 65 prominent-midi-diff) :else (+ 60 prominent-midi-diff)))) Get data, study data Pitch histograms MIDI histograms Tonic identification Swaram histograms Transcription Audio files to notes

Slide 79

Slide 79 text

Get data, study data Pitch histograms MIDI histograms Tonic identification Swaram histograms Transcription Audio files to notes Tonic identification (defn find-tonic-midi [freqs prominent-note] (let [midi-occurrances (midi-histogram freqs) prominent-midi-diff (->> midi-occurrances normalize-octaves (sort-by second >) ffirst)] (cond (= :s prominent-note) (+ 60 prominent-midi-diff) (= :p prominent-note) (+ 65 prominent-midi-diff) :else (+ 60 prominent-midi-diff))))

Slide 80

Slide 80 text

Swaram Histogram Kalyani S, R2, G3, M2, P, D2, N3, S. S., N3, D2, P, M2, G3, R2, S

Slide 81

Slide 81 text

Kalyani S, R2, G3, M2, P, D2, N3, S. S., N3, D2, P, M2, G3, R2, S Revati S, R1, M1, P, N2, S. S., N2, P, M1, R1, S Mohana S, R2, G3, P, D2, S. S., D2, P, G3, R2, S

Slide 82

Slide 82 text

Generation Not so naive now

Slide 83

Slide 83 text

(defn get-next-swaram [swaram-allocations] (let [r (rand 100)] (->> swaram-allocations (filter (fn [[swaram [b e]]] (< b r e))) first first))) One swaram probabilities One swaram probabilities Two-swaram probabilities One vs two swarams Markov chains Generation

Slide 84

Slide 84 text

Random Swaram Weighted One swaram probabilities One swaram probabilities Two-swaram probabilities One vs two swarams Markov chains Generation

Slide 85

Slide 85 text

Melody Insights #2 Tonic is prominent Sa (tonic), and Pa (5th) have higher and sharper peaks Other note peaks are blunt Probabilities of all swarams in a raga are not the same Probabilities across octaves are not the same

Slide 86

Slide 86 text

Make generative music sound continuous Extract raga grammar, use it in generation How do we detect melodic paerns? Phase #3

Slide 87

Slide 87 text

Generation Continued…

Slide 88

Slide 88 text

Two swaram probabilities {:.s ([:.s 57.35530546623794] [:.r2 6.109324758842444] [:s 5.868167202572347] [:.d2 5.466237942122186] [:.p 5.42604501607717] [:.g3 3.215434083601286]…) :.r2 ([:.s 29.53020134228188] [:.r2 21.0738255033557] [:.g3 14.49664429530201] [:.p 6.577181208053691]…) :.p ([:.p 44.131214761660694] [:.d2 16.04305484366991] [:.g3 8.14966683751922]…) …} One swaram probabilities Two swaram probabilities One vs two swarams Markov chains Generation

Slide 89

Slide 89 text

Two swaram probabilities One swaram probabilities Two swaram probabilities One vs two swarams Markov chains Generation

Slide 90

Slide 90 text

One swaram probabilities Two swaram probabilities One vs two swarams Markov chains Generation (defn swaram-generator [swaram allocations] (lazy-seq (cons swaram (swaram-generator (get-next-swaram swaram allocations) allocations)))) Two swaram probabilities

Slide 91

Slide 91 text

One swaram probabilities Two swaram probabilities One vs two swarams Markov chains Generation One vs two swarams

Slide 92

Slide 92 text

One swaram probabilities Two swaram probabilities One vs two swarams Markov chains Generation Markov chains https://en.wikipedia.org/wiki/Markov_chain#Music https://github.com/rm-hull/markov-chains

Slide 93

Slide 93 text

One swaram probabilities Two swaram probabilities One vs two swarams Markov chains Generation Markov chains https://en.wikipedia.org/wiki/Markov_chain#Music https://github.com/rm-hull/markov-chains

Slide 94

Slide 94 text

Melody Insights #3 Adjacent swarams seem more melodious Sequencing per raga rules makes it sound beer Gamakas at this level give a Carnatic texture Sometimes, the in-between is worse than either extreme Higher order Markov chains do not make it sound beer

Slide 95

Slide 95 text

Back to the drawing board Explore rhythm concepts Explore repetition Try to generate melodies Phase #4

Slide 96

Slide 96 text

Laya, and Tala Temporal discipline

Slide 97

Slide 97 text

ुत: माता लय: पता Shruti is the mother, Laya is the father - Shruti: Pitch ﬁdelity - Laya: Temporal discipline, or Tempo Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati

Slide 98

Slide 98 text

- Laya organizes music in time - A change in tempo has communicative weight Laya | लय Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati

Slide 99

Slide 99 text

TAla is the fundamental principle that binds Gita (song), Vadya (instrument) and Nria (dance). ताळ: तल थठयामत गीतम् वाम् तथानृं यथाालेततम् Sangita Ratnakara, Sarngadeva, 12 CE Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati

Slide 100

Slide 100 text

Tala | ताळ - A recurrent time frame, a cycle of beats - Derived from poetic meter, rather than dance - Hand gestures for conduction Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati

Slide 101

Slide 101 text

Tala Systems - Navasandhi talas (108) - Chapu talas (4) - Suladi Sapta talas (175) Laya, Tala Temporal discipline Etymology, Meaning Avartanam Matra, Akshara, Gati

Slide 102

Slide 102 text

Avartana | आवतन - avartana: 1 cycle - signiﬁes number of beats in 1 cycle - simhanandana tala: 128 beats per cycle - sharaba nandana tala: 79 beats per cycle Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati

Slide 103

Slide 103 text

Kriya, Anga, Jati - Anga: A subdivision of the tala - Kriya: The gesture used to refer to an anga - Jati: The number of matras in a laghu Laya, Tala Temporal discipline Etymology, Meaning Avartanam Kriya, Anga, Jati Matra, Akshara, Gati

Slide 104

Slide 104 text

Matra, Akshara, Gati - Matra: beat (within an avartana) - Akshara: division of beat (within a matra) - Gati: gait; factor dividing matra into akshara Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati

Slide 105

Slide 105 text

Transcription based on laya and gati

Slide 106

Slide 106 text

Original Record Gati: 1 Gati: 2 Gati: 4 Gati: 8 Transcription Based on laya and gati A clinical process Detail vs structure trade-os Varna and Alankara Removing outliers

Slide 107

Slide 107 text

Gati: 2 Gati: 8 Melodic structure is visible at 1/2 a matra / beat Embellishments / Gamakas are visible at 1/8th Transcription Based on laya and gati A clinical process Detail vs structure trade-os Varna and Alankara Removing outliers

Slide 108

Slide 108 text

Transcription Based on laya and gati A clinical process Detail vs structure trade-os Varna and Alankara Removing outliers Varna and Alankara - Tonal paerns at matra (beat) level is called varna - Embellishments at akshara (sub-beat) level is called alankara

Slide 109

Slide 109 text

Removing outliers - Short-lived frequencies (< akshara/2) - Non prominent swaras in raga - Low and high cutos for unnatural frequencies - Octave shis, and continuous tonics - Detecting rests Transcription Based on laya and gati A clinical process Detail vs structure trade-os Varna and Alankara Removing outliers

Slide 110

Slide 110 text

Generation Last time, promise

Slide 111

Slide 111 text

Revisiting Markov Chains - Transcribing with input BPM improves quality - Markov chains at multiple levels of granularity - So many knobs/toggles to play with Revisiting Markov chains At matra, akshara levels Embellishment chains Aavrii Generation

Slide 112

Slide 112 text

(markov/generate-and-play mohanam-swarams {:order 12 :duration-seconds 100 :playback-gati 2 :generation-gati 4 :mode :discrete :bpm 80}) (markov/generate-and-play mohanam-swarams {:order 22 :duration-seconds 100 :playback-gati 10 :generation-gati 8 :mode :continuous :bpm 80}) Revisiting Markov chains At matra, akshara levels Embellishment chains Aavrii Generation

Slide 113

Slide 113 text

Embellishment chains - Filling matra chains with alankara chains fails - Gamakas are contextual, and have their own laya Revisiting Markov chains At matra, akshara levels Embellishment chains Aavrii Generation

Slide 114

Slide 114 text

Revisiting Markov chains At matra, akshara levels Embellishment chains Aavrii Generation Repetition / Avartana Organic repetition seems hard to model, I’m on it.

Slide 115

Slide 115 text

Melody Insights #4 Time is fundamental to melody, and inseparable from frequency Humans perceive more than a beat when hearing a beat Machines understand tonal structures at 1/2 a beat Markov chains don’t work for embellishments Simplest form of repetition can be powerful

Slide 116

Slide 116 text

By-products

Slide 117

Slide 117 text

Using a simple goodness of ﬁt test By-product #1 Raga Identiﬁcation (defn raga-diff [base-osp sample-osp] (let [ordered-vals (fn [osp] (vals (sort-by first < osp))) probs (ordered-vals base-osp) table (ordered-vals sample-osp)] (chisq-test :table table :probs (map #(* 0.01 %) probs))))

Slide 118

Slide 118 text

(defn raga-diff [base-osp sample-osp] (let [ordered-vals (fn [osp] (vals (sort-by first < osp))) probs (ordered-vals base-osp) table (ordered-vals sample-osp)] (chisq-test :table table :probs (map #(* 0.01 %) probs)))) Using a simple goodness of ﬁt test By-product #1 Raga Identiﬁcation

Slide 119

Slide 119 text

By-product #2 Useful in education, and preserving music Automatic notation, playback (let [props {:bpm 87 :gati 2 :offset 60} swaras (bpm-transcribe (first samples/research-files))] (prescriptive-notation (get swaras (:gati props)) 16)) (“.s .r .g .p .d , s , .d .p .g , " ".r .s , .r , .g .p , .d , s , .d" " .p , .g , .r , .s .r .g .p , .d" " s , .d .p , .g .r .s .r .g .p ," " .d s , , .d .p , .r .s .r .d s " ".d .p .g .r .g s .p .g .p s .p .”)

Slide 120

Slide 120 text

By-product #3 An electronic rhythm accompanist Electronic tala assistant (def talas {:eka [:I] :rupaka [:O :I] :jhampa [:I :U :O] :triputa [:I :O :O] :matya [:I :O :I] :ata [:I :I :O :O] :dhruva [:I :O :I :I]}) (play-avartanams 60 4 (->> (tala-accents :chatusra :rupaka) #(mapcat combo) (repeatedly 10)))

Slide 121

Slide 121 text

Is this music though? Is it art?

Slide 122

Slide 122 text

Behag, Dasarapada Abhishek Raghuram

Slide 123

Slide 123 text

References Sangita Ratnakara of Sarngadeva – RK Shringy, Prem Lata Sharma Sangita Sampradaya Pradarsini – SRJ, NR, RSJ Brhaddesi of Sri Matanga Muni – Prem Lata Sharma Appreciating Carnatic Music – Lakshmi Sreeram South Indian Music – P. Sambamurthy Compmusic – MTG, Barcelona, Xavier Serra Gayaka - Subramanian Srikumar’s PASR thesis

Slide 124

Slide 124 text

Making machines that make music ीहर ीरामन् नलेो Srihari Sriraman nilenso

Slide 125

Slide 125 text

Shruti ुत Etymology Intervals in indian music ूयते इत ुत: - that which should be heard - the vedas, tanpura - that which can be heard - the word shruti probably means an interval here