Hiding What from Whom? A Critical Review of the History of Programming languages for Music

Slide 1

Slide 1 text

Tomoya MATSUURA/ দӜ஌໵ - International Computer Music Conference 2025 Art Media Center, Tokyo University of the Arts / [email protected] Hiding What from Whom? A critical review of programming-languages for music matsuuratomoya.com/en/ research/icmc-2025/

Slide 2

Slide 2 text

Computer Music Languages?

Slide 3

Slide 3 text

https://www.avid.com/ja/pro-tools Is this "Computer Music"? https://faust.grame.fr/ Is this programming languages made only for "Computer Music"?

Slide 4

Slide 4 text

History of ”Programming Language for Music” History of ”Computer Music Language” same?

Slide 5

Slide 5 text

Personal Motivation and Context https://mimium.org/ T. Matsuura and K. Jo, “Mimium: A Self-Extensible Programming Language for Sound and Music(2021) T. Matsuura, “Lambda-mmm: The Intermediate Representation for Synchronous Signal Processing Language Based on Lambda Calculus(2024) Why do you make a new language in 2020?

Slide 6

Slide 6 text

1960 1970 1980 1990 2000 2010 2020 1950 MUSIC I Mathews MUSIC III (UGen) IRCAM 4X Realtime-Variable DSP Pure Data SuperCollider ChucK Faust mimium Innis MUSIGOL Max/FTS UNIVAC/BINAC/ACE  the early attempts of sound generation with a computer FORTRAN Csound TidalCycles Sonic Pi Kronos MUSIC V Soul/Cmajor Is the PCM truly universal? How the MUSIC-N evolved with hiding its internal structure with Unit Generator What are the values of the programming language for music, developed after 2000?

Slide 7

Slide 7 text

Slide 8

Slide 8 text

"Almost Any Sound" and PCM from (Mathews 1963)

Slide 9

Slide 9 text

Algo“rhythmic” Listening Practices using sound in the early days of computer • Using the speaker to listen to the logic signal for debugging (Miyazaki, 2012) • By controlling the frequency of the logic, engineers produced melodies Photo of Pllot ACE from (Doornbusch 2017) • Most of the attempts were just playing famous melody but in the case of Pilot ACE(UK), accidentally, the engineers could get unique sound (Doornbusch 2017) due to the structure of the acoustic-delay memory

Slide 10

Slide 10 text

"Very Early Computer Music", Donald Davis, Resurrection The Bulletin of the Computer Conservation Society, vol. 10 pp.19-20 (1994) "The Ace Pilot Model and its successor, the Ace proper, were both capable of composing their own music and playing it on a little speaker built into the control desk. I say composing because no human had any intentional part in choosing the notes. The music was very interesting, though atonal, and began by playing rising arpeggios: these gradually became more complex and faster, like a developing fugue. They dissolved into colored noise as the complexity went beyond human understanding."

Slide 11

Slide 11 text

Non-PCM Representation? (from Miller Puckette's Keynote in ICMC 2015) If there was a DAC for this linear-segments sound representation, what the language for it like?

Slide 12

Slide 12 text

"Almost Any Sound" and PCM from (Mathews 1963) MUSIC I could produce 1 triangular wave with envelope. Could it produce... "any" sound?

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Unit Generator Concept (1963) by Max Mathews • Connecting "Unit" of the signal processing • e.g. Sinewave Oscillator, Filter, etc... • Drawing a diagram on the paper, translating it into a textual format, and then punchcard holes, were the "programming".

Slide 15

Slide 15 text

Example:IIR Bandpass Filter FLT I1 O I2 I3 P29 P30; On = I1 ⋅ Sn + I2 ⋅ On−1 − I3 ⋅ On−2 Z-1 Z-1 I1 I2 I3 + + In MUSIC V: Using surplus memory spaces for time-varying parameters coming from the score, to store signal histories. (Any indexes can be used if they weren't used in the score) S O

Slide 16

Slide 16 text

In MUSIC 11/Csound: instr 1 ; instrument with fabricated reson: la1 init 0 ; clear feedbacks la2 init 0 ; at start only i3 = exp(-6.28 * p6 / 10000) ; set coef 3 i2 = 4*i3*cos(6.283185 * p5/10000) / (1+i3); set coef 2 i1 = (1-i3) * sqrt(1-1 - i2*i2/(4*i3)) ; set coef 1 a1 rand p4 ; source signal la3 = la2 ; feedback 2 la2 = la1 ; feedback 1 la1 = ɹɹi1*a1 + i2 * la2 - i3 * la3 ; 2nd order difference eqn out ɹla1 endin instr 2 ; this instr does same as above a1 rand p4 ; source signal a1 reson a1,p5,p6,1 ; 2nd order recursicve filter endin You can use pre-built "reson" opcode(UGen), but also de fi ne an equivalent by yourself. (User De fi ned OpCode) ʢno temporary variables required for feedbackʣ

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Composer Realizer (Musical Assistants) Researcher Engineer Make Max/Pd Patches Make Max/Pd Objects with C/C++ Make Max/Pd Itself Division of labor embedded in Max/Pd • Today’s layered ecosystem of Max and Puredata directly came from the division of labor in IRCAM (Reese and Puckette, 2020) Compose with DAW Make VST Plugins w/ C/C++ Develop DAWs

Slide 20

Slide 20 text

"Any Sound You Can Imagine: Making Music/Consuming Technology", Paul Théberge, Wesleyan University Press, p.89 (1997) "Lacking adequate knowledge of the technical system, musicians increasingly found themselves drawn to prefabricated programs as a source of new sound material. (. . . ) it also suggests a reconceptualization on the part of the industry of the musician as a particular type of consumer." ...does this situation even happen in open-source sound programming tools, right?

Slide 21

Slide 21 text

Slide 22

Slide 22 text

More General & Formalized, Less Blackbox Extempore FoxDot TidalCycles Sonic Pi Overtone FAUST Kronos mimium Alternative Abstraction & Live Modi fi cation (Typically on top of SuperCollider) higher layer lower layer

Slide 23

Slide 23 text

Openness as an attitude & aesthetics from TOPLAP manifesto(2004) • We demand: • Give us access to the performer's mind, to the whole human instrument. • Obscurantism is dangerous. Show us your screens. • Programs are instruments that can change themselves • The program is to be transcended - Arti fi cial language is the way. • Code should be seen as well as heard, underlying algorithms viewed as well as their visual outcome. • Live coding is not about tools. Algorithms are thoughts. Chainsaws are tools. That's why algorithms are sometimes harder to notice than chainsaws. https://toplap.org/wiki/ManifestoDraft (emphasis by Tomoya)

Slide 24

Slide 24 text

whenmod 8 4 (slow 4) $ every 2 ((1=2) < ) $   every 3 (density 4) $ iter 4 "grey darkgrey green black" (Mclean 2014) Some SC-parasite languages do not depend on PCM sound generation

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Division of the labor in the DSL From "narrow and many step stair" to "one continuous steep slope" If there is a language that is general, e ff i cient and expressive enough, the division of labor will be gone? Engineer(Language Designer) Realizer(External UGen Designer) Language User(Composer)

Slide 27

Slide 27 text

Division of the labor in the DSL From "narrow and many step stair" to "one continuous steep slope" Engineer(Host Language Designer) Language User(Composer/Library Developer) ...or will it introduce even deeper division of labor between the language designer and the user? Maybe... a self-hostable language for music could solve this? If there is a language that is general, e ff i cient and expressive enough, the division of labor will be gone?

Slide 28

Slide 28 text

$PODMVTJPO

Slide 29

Slide 29 text

"Of Epistemic Tools: musical instruments as cognitive extensions", Thor Magnusson, Organized Sound 14-2, p173 (2009) "As opposed to the acoustic instrument maker, the designer of the composed digital instrument frames affordances through symbolic design, thereby creating a snapshot of musical theory, freezing musical culture in time."

Slide 30

Slide 30 text

• Why you make the new music language in 2020? • To make more interesting sound/music? • To make more e ffi cient/productive/useful tool? • Neither, designing a language involves extracting the form of music at the era and enabling them shareable • which sometimes present alternatives or even solidify/ fi x existing musical styles conversely. • programming language as a cultural infrastructure

Slide 31

Slide 31 text

Limitations and future works • My research re-contextualized existing literatures but more primary resources are needed • Archaeological practices programming languages for music/computer music(I have never tried the real systems developed before 2000!!) • We need to incorporate practices by non-academic language designer more • What is... "good" language? • How are we using the evaluation terms like "expressive" "e ff i cient" "general"? Are they shared between researchers?

Slide 32

Slide 32 text

Thanks! https://matsuuratomoya.com/en/research/icmc-2025/ Preprint URL https://matsuuratomoya.com https://mimium.org email: [email protected] mastodon: social.matsuuratomoya.com/@tomoya

Slide 33

Slide 33 text

4VQQMFNFOUBSZ.BUFSJBMT

Slide 34

Slide 34 text

4PVSDF %PDVNFOUBUJPO $PNNVOJUZ 4ZOUBY %FWFMPQJOH &OWJSPONFOU 3VOOJOH &OWJSPONFOU 4FNBOUJDT Compilation Cost Platform Independency Coding Cost Generality (Possible Expressions ) Execution Cost 'SPOUFOE *OUFSNFEJBUF 3FQSFTFOUBUJPO $PNQJMFS 3VOUJNF 7JTVBM 3FQSFTFOUBUJPO &EJU $PNQJMF 3FTVMU &SSPS 3VOUJNF &SSPS (FOFSBUFE 4PVOE &BTF PG *UFSBUJPO Live Modification +Developing Cost Learnability

Slide 35

Slide 35 text

Granularity of the Intermediate Representation

Slide 36

Slide 36 text

MUSIGOL(Innis 1968) First "DSL as a Library" for MUSIC real procedure Sinoscil(Time, Frequency, Amplitude); value Time, Frequency, Amplitude; real Time, Frequency, Amplitude; begin real X, XX; X := (6.2831853072 * Frequency * Time) MOD 6.2831853072; if abs(X - 3.1415926536) < 0.0315 then Sinoscil := Amplitude * (3.1415926536 - X) else if abs(X - 6.283185307) < 0.0315 then Sinoscil := Amplitude * (X - 6.2831853072) else begin if X > 3.1415926536 then X := X - 6.28318503072; XX := sign(X); X := abs(X); X := if X > 1.5707963268 then XX * 0.636619772 * (3.1415926536 - X) else X * XX * 0.636619772; XX := X * X; Sinoscil := X * (1.570794852 + XX * (-0.645920978 + XX * (0.079487663 - 0.004362476 * XX))) * Amplitude; end; end Sinoscil; Both the system and the user codes are entirely written in ALGOL 60. procedure Score; begin Boing(-3, -1.5, 20, E4); Boing(-3.5, -4, 15, Slope(-3.5, -4, CS4, C9)); ... Speaker(OUTS, 100); end of Score;