Structure of this talk Universality, Carnatic music A timeline of 4 phases Bridging the gap between machines and humans Understand domain, create abstractions Synthesise ⟷ Generate ⟷ Transcribe Reflect on melody insights
Prominent tonic Resolves tension from musical movement Pervasive pentatonic scales Common in a lot of cultures: celtic, chinese, african, english, indian, etc 7 note scales, 12 semitones Latin, Chinese, Indian cultures have similar solfege Emotive melodies Lullabies, sing-alongs, march songs, festival songs, hymns Universality Natural phenomena observed around the world
Universality Natural phenomenons Prominent tonic Pervasive pentatonic scales 7 note scales, 12 semitones Emotive melodies Tonic | आधार षज - Resolves tension from musical movement. - Everyone knows when a piece of music returns to the tonic.
Pentatonic | औडुवम् Common in a lot of cultures: celtic, chinese, african, english, indian, etc hps://en.wikipedia.org/wiki/Pentatonic_scale#Pervasiveness Popular Bobby McFerrin demonstration of the power of the pentatonic scale hps://www.youtube.com/watch?v=ne6tB2KiZuk Universality Natural phenomenons Prominent tonic Pervasive pentatonic scales 7 note scales, 12 semitones Emotive melodies
Universality Natural phenomenons Prominent tonic Pervasive pentatonic scales 7 note scales, 12 semitones Emotive melodies 7 notes | स राः Latin: do re mi fa sol la si Chinese: shàng chě gōng fán liù wù yi Indian: sa ri ga ma pa da ni
Classical, art music - Not Tribal | Folk | Religious | Popular - Intent is artistic - Music itself is the sole focus Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions
Ancient, traditional music – Earliest treatise ना शा (Natya Shastra) at 500BC – लण (Lakshana Grantha): Rich tradition of Sanskrit grammar texts – Oral tradition, trained by exposure Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions
Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions Digitisation hype cycle
Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions Carnatic music Digitisation hype cycle
Melodic music - Has lile place for harmony - Emphasis on notes in succession - Explores melody through ragas and gamakas Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions
Extempore, compositions - Similar to jazz music - Renditions of century old compositions - Improvisations are generally around compositions Carnatic Music South Indian Classical Music Classical, art music Ancient, traditional music Melodic music Extempore, and compositions
Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo Adhara Shadja आधार षज | Tonic, pitch - Choice of the artist based on voice - Does not change for every song - Swaras are relative to this
- The 12 semitones in an octave - Solfege is sung, and used as notation - Elements of a raga Swara र | Note Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo
Raga राग / Melodic framework - Melodic modes with added specialities - Grammar that a composition adheres to - Evokes a certain set of emotions Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo
Scale | आरोह, अवरोह – Example: – Aarohanam: 1 2 3 5 6 8 – Avarohanam: 8 7 6 5 4 3 2 1 – They form a skeleton of the raga – Can be asymmetric, and non-linear Fundamentals आधार षज, र, राग Adhara Shadja / Tonic Swara / Note Raga / Melodic Framework Demo
Etymology in Sanskrit सक ् क ृ तम् (correctly done) = सं ृ तम् Specifies grammar for creating new words Words can be composed (समास / Samāsa) Elegant solution to the naming problem in soware
मेव राजते इत र - that which shines on its own - In the context of grammar, “swara” means vowel. A consonant in Sanskrit cannot be pronounced without a vowel. Etymology Independent Resonant Swara र | Note
More than a scale – A class of ragas has a phraseology / रागवाचक – Learning a raga is akin to learning a language by exposure (say a mother tongue) – A raga is absorbed Etymology Color, Emotion More than a scale Raga राग / Melodic framework
छाया ुरा याम् रो य गमये What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR Gamaka गमक / Ornamentation Movement of a note from it’s pitch towards another so that the second passes like shadow over it. Sangita samayasaara, Parsvadeva, 12CE
- Embellishment, ornamentation - Shake, push, glide, flick - Pervasive in Carnatic music - Every swaram is a continuum What is it? Gamaka गमक / Ornamentation What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR
Prior art Gaayaka | Subramanian, 2009 | S, N D | N S R G | ((P S,,)) , ((S , S>>> S)) -((D. S. D)) ((S , S>> S))- S R ((G<< G , ,)) - Database of phrases - < and > increase and decrease pitch - ( and ) are speed factors; deeper is faster What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR Gamaka गमक / Ornamentation
SSP | Sangita Samradaya Pradarshini संगीत सादाय दषन What is it? Kinds of gamaka Prescriptive vs Descriptive Prior art Synthesis Rendering PASR Gamaka गमक / Ornamentation
Melody Insights #1 Prescriptive notation is insuicient for reproduction Raga as a framework for creating melodies is powerful Melographs make melodies tangible for observation Gamaka synthesis can be mathematically modelled as a curve
Generate a stream of Carnatic music Try to convert existing music into music data Study music data, look for paerns Infer characteristics of raga from music data Phase #2
Step 1. Build a massive database by scraping the internet Ragas, lyrics Ragas Compositions, Renditions Get data, study data Get data, study data Pitch histograms MIDI histograms Tonic identification Swaram histograms Transcription Audio files to notes
Get data, study data Step 2. Build a user interface to search songs and ragas, so that I can see what’s in the database. Get data, study data Pitch histograms MIDI histograms Tonic identification Swaram histograms Transcription Audio files to notes
Get data, study data Step 3. Download renditions of a given raga (->> "bhairavi" (db-search/renditions-in-ragam) (take 50) (download-renditions) hp://github.com/ssrihari/kosha Get data, study data Pitch histograms MIDI histograms Tonic identification Swaram histograms Transcription Audio files to notes
Get data, study data Step 4. Transcribe the fundamental frequencies of vocals Get data, study data Pitch histograms MIDI histograms Tonic identification Swaram histograms Transcription Audio files to notes
Get data, study data Step 5. Look at melographs, spectrograms, and try to find usable insights about melody. kalyANi-MS-Subbulakshmi-nidhi_cAla_sukhamA-tyAgarAja3.mpeg.wav.pitch.frequencies Get data, study data Pitch histograms MIDI histograms Tonic identification Swaram histograms Transcription Audio files to notes
Tonic identification Bellur, A., V. Ishwar, X. Serra, and H. A. Murthy (2012) A knowledge based signal processing approach to tonic identification in indian classical music. Bellur, A., and H. A. Murthy (2013) Automatic tonic identification in classical music using melodic characteristics and tuning of the drone. Srihari, S. (2016) * Pick the highest one, it mostly just works. * not really, no Get data, study data Pitch histograms MIDI histograms Tonic identification Swaram histograms Transcription Audio files to notes
(defn get-next-swaram [swaram-allocations] (let [r (rand 100)] (->> swaram-allocations (filter (fn [[swaram [b e]]] (< b r e))) first first))) One swaram probabilities One swaram probabilities Two-swaram probabilities One vs two swarams Markov chains Generation
Melody Insights #2 Tonic is prominent Sa (tonic), and Pa (5th) have higher and sharper peaks Other note peaks are blunt Probabilities of all swarams in a raga are not the same Probabilities across octaves are not the same
One swaram probabilities Two swaram probabilities One vs two swarams Markov chains Generation (defn swaram-generator [swaram allocations] (lazy-seq (cons swaram (swaram-generator (get-next-swaram swaram allocations) allocations)))) Two swaram probabilities
One swaram probabilities Two swaram probabilities One vs two swarams Markov chains Generation Markov chains https://en.wikipedia.org/wiki/Markov_chain#Music https://github.com/rm-hull/markov-chains
One swaram probabilities Two swaram probabilities One vs two swarams Markov chains Generation Markov chains https://en.wikipedia.org/wiki/Markov_chain#Music https://github.com/rm-hull/markov-chains
Melody Insights #3 Adjacent swarams seem more melodious Sequencing per raga rules makes it sound beer Gamakas at this level give a Carnatic texture Sometimes, the in-between is worse than either extreme Higher order Markov chains do not make it sound beer
ुत: माता लय: पता Shruti is the mother, Laya is the father - Shruti: Pitch fidelity - Laya: Temporal discipline, or Tempo Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati
- Laya organizes music in time - A change in tempo has communicative weight Laya | लय Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati
TAla is the fundamental principle that binds Gita (song), Vadya (instrument) and Nria (dance). ताळ: तल थठयामत गीतम् वाम् तथानृं यथाालेततम् Sangita Ratnakara, Sarngadeva, 12 CE Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati
Tala | ताळ - A recurrent time frame, a cycle of beats - Derived from poetic meter, rather than dance - Hand gestures for conduction Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati
Tala Systems - Navasandhi talas (108) - Chapu talas (4) - Suladi Sapta talas (175) Laya, Tala Temporal discipline Etymology, Meaning Avartanam Matra, Akshara, Gati
Avartana | आवतन - avartana: 1 cycle - signifies number of beats in 1 cycle - simhanandana tala: 128 beats per cycle - sharaba nandana tala: 79 beats per cycle Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati
Kriya, Anga, Jati - Anga: A subdivision of the tala - Kriya: The gesture used to refer to an anga - Jati: The number of matras in a laghu Laya, Tala Temporal discipline Etymology, Meaning Avartanam Kriya, Anga, Jati Matra, Akshara, Gati
Matra, Akshara, Gati - Matra: beat (within an avartana) - Akshara: division of beat (within a matra) - Gati: gait; factor dividing matra into akshara Laya, Tala Temporal discipline Etymology, Meaning Avartana Matra, Akshara, Gati
Original Record Gati: 1 Gati: 2 Gati: 4 Gati: 8 Transcription Based on laya and gati A clinical process Detail vs structure trade-os Varna and Alankara Removing outliers
Gati: 2 Gati: 8 Melodic structure is visible at 1/2 a matra / beat Embellishments / Gamakas are visible at 1/8th Transcription Based on laya and gati A clinical process Detail vs structure trade-os Varna and Alankara Removing outliers
Transcription Based on laya and gati A clinical process Detail vs structure trade-os Varna and Alankara Removing outliers Varna and Alankara - Tonal paerns at matra (beat) level is called varna - Embellishments at akshara (sub-beat) level is called alankara
Removing outliers - Short-lived frequencies (< akshara/2) - Non prominent swaras in raga - Low and high cutos for unnatural frequencies - Octave shis, and continuous tonics - Detecting rests Transcription Based on laya and gati A clinical process Detail vs structure trade-os Varna and Alankara Removing outliers
Revisiting Markov Chains - Transcribing with input BPM improves quality - Markov chains at multiple levels of granularity - So many knobs/toggles to play with Revisiting Markov chains At matra, akshara levels Embellishment chains Aavrii Generation
Embellishment chains - Filling matra chains with alankara chains fails - Gamakas are contextual, and have their own laya Revisiting Markov chains At matra, akshara levels Embellishment chains Aavrii Generation
Revisiting Markov chains At matra, akshara levels Embellishment chains Aavrii Generation Repetition / Avartana Organic repetition seems hard to model, I’m on it.
Melody Insights #4 Time is fundamental to melody, and inseparable from frequency Humans perceive more than a beat when hearing a beat Machines understand tonal structures at 1/2 a beat Markov chains don’t work for embellishments Simplest form of repetition can be powerful
Shruti ुत Etymology Intervals in indian music ूयते इत ुत: - that which should be heard - the vedas, tanpura - that which can be heard - the word shruti probably means an interval here
srihari
.png){kind=avatar}
researchconf
mhangyo
rina_glutton
koheishinden
kei18
johannyjm
uepon73
eqpresents
yuukicammy
bakanaouji
shuntaros
bluesmoon
paulrobertlloyd
csswizardry
palkan
jlugia
chrislema
jasonvnalue
mongodb
addyosmani
marktimemedia
kneath
moore