Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Crafting Instruments In Code

Crafting Instruments In Code

Joseph Wilk

May 11, 2018
Tweet

More Decks by Joseph Wilk

Other Decks in Technology

Transcript

  1. Crafting
    Instruments
    in
    code
    @josephwilk
    https://en.wikipedia.org/wiki/Silver_Apples

    View Slide

  2. @repl_electric http://www.repl-electric.com

    View Slide

  3. http://www.repl-electric.com
    @repl_electric

    View Slide

  4. http://www.repl-electric.com
    @repl_electric

    View Slide

  5. Noise
    Silver Apples

    View Slide

  6. Samples
    “Real-world noise of any sort has a very wide frequency
    range, far wider and more complex than any electronically
    generated sound.”

    Jon Hopkins
    https://en.wikipedia.org/wiki/Jon_Hopkins

    View Slide

  7. Channel
    (Right)
    Channel
    (Left)
    Frames
    44100
    1 second
    Sample
    16bits
    Frame
    n+1
    Frame
    n+3
    Frame
    n+4
    Frame n
    Frame
    n+2
    176k
    Sample
    16bits
    Amplitude of

    sound wave
    Stereo
    http://soundfile.sapp.org/doc/WaveFormat/

    View Slide

  8. Samples as DATA
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.4 0.5 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.3 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.3 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.1 0.6 0.3 0.5 0.3 0.2 0.4 0.3 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.5 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.5 0.1 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.2 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.5 0.1 0.5 0.3 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.1 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2
    [0.5 0.2 0.3 0.5 0.3 0.2
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    .3 0.2 0.4 0.5 0.5 0.2 0.3]
    .3 0.2 0.5 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3]
    [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.
    [0.5 0.2 0.3 0.5 0.
    [0.5 0.2 0.3 0.5 0.
    0.4 0.5 0.5 0.2 0.3]
    0.4 0.5 0.5 0.2 0.3]

    View Slide

  9. Fundamental frequency
    Midi note
    Root mean squared amplitude
    Spectral-centroid
    Spectral-irregularity
    Spectral-inharmonicity
    Spectral skewness
    Spectral kurtosis
    http://jamiebullock.github.io/LibXtract/documentation/

    View Slide

  10. Fundamental frequency
    Midi note
    Volume
    Brightness of Sound
    Noisiness of a sound
    How harmonic
    Spectrum skew
    Pitchiness
    http://jamiebullock.github.io/LibXtract/documentation/

    View Slide

  11. Query and use it demo
    T-sne
    “magical ability to create compelling two-dimensional “maps”
    from data with hundreds or even thousands of dimensions”
    https://lvdmaaten.github.io/tsne/

    View Slide

  12. Gene Kogan

    View Slide

  13. https://www.instagram.com/p/Bft_6KVHa9l/?taken-by=livecodez
    https://twitter.com/xululululuuum

    View Slide

  14. Patching
    https://en.wikipedia.org/wiki/Wendy_Carlos

    View Slide

  15. View Slide

  16. View Slide

  17. https://vcvrack.com/
    https://vcvrack.com/

    View Slide

  18. Code
    https://supercollider.github.io/

    View Slide

  19. Code

    View Slide

  20. Code

    View Slide

  21. View Slide

  22. Procedural
    Property based testing
    (Quickcheck)
    Genetic
    &
    https://en.wikipedia.org/wiki/%C3%89liane_Radigue

    View Slide

  23. Generators Properties
    All things that are a square

    should have 4 sides of equal length
    (fn []
    (let [color (gen/rgb-color)
    size (gen/int)]
    (build-a-square color size)))
    Property Based Testing
    https://en.wikipedia.org/wiki/QuickCheck

    View Slide

  24. Property Based Testing
    Generators
    reverb echo
    tube
    distort
    delay
    Sine
    Wave
    Square
    Wave
    Pulse
    Wave
    Sound
    Fx
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave

    View Slide

  25. Property Based Testing
    Generators
    reverb echo
    tube
    distort
    delay
    Sine
    Wave
    Square
    Wave
    Pulse
    Wave
    Sound
    Fx
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    Tube
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    reverb
    reverb
    Pulse
    Wave
    reverb reverb reverb
    Square
    Wave
    delay reverb
    Pulse
    Wave
    Pulse
    Wave
    reverb
    Pulse
    Wave
    reverb
    reverb
    Generators
    reverb echo
    tube
    distort
    delay
    Sine
    Wave
    Square
    Wave
    Pulse
    Wave
    Sound
    Fx
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    Tube
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    reverb
    reverb
    Pulse
    Wave
    reverb reverb reverb
    Square
    Wave
    delay reverb
    Pulse
    Wave
    Pulse
    Wave
    reverb
    Pulse
    Wave
    reverb
    reverb
    Generators
    reverb echo
    tube
    distort
    delay
    Sine
    Wave
    Square
    Wave
    Pulse
    Wave
    Sound
    Fx
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    Tube
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    reverb
    reverb
    Pulse
    Wave
    reverb reverb reverb
    Square
    Wave
    delay reverb
    Pulse
    Wave
    Pulse
    Wave
    reverb
    Pulse
    Wave
    reverb
    reverb
    Generators
    reverb echo
    tube
    distort
    delay
    Sine
    Wave
    Square
    Wave
    Pulse
    Wave
    Sound
    Fx
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    Tube
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    Pulse
    Wave
    delay
    Square
    Wave
    reverb reverb
    Pulse
    Wave
    Square
    Wave
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    reverb
    reverb
    Pulse
    Wave
    reverb reverb reverb
    Square
    Wave
    delay reverb
    Pulse
    Wave
    Pulse
    Wave
    reverb
    Pulse
    Wave
    reverb
    reverb
    reverb
    Pulse
    Wave
    Pulse
    Wave
    reverb
    Pulse
    Wave
    reverb
    reverb
    reverb
    Pulse
    Wave
    Pulse
    Wave
    reverb
    Pulse
    Wave
    reverb
    reverb
    reverb
    Pulse
    Wave
    Pulse
    Wave
    reverb
    Pulse
    Wave
    reverb
    reverb
    reverb
    Pulse
    Wave
    Pulse
    Wave
    reverb
    Pulse
    Wave
    reverb
    reverb
    Sound
    Fx
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Sound
    Fx
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Sound
    Fx
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Sound
    Fx
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Sound
    Fx
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Sound
    Fx
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Sound
    Fx
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Sound
    Fx
    delay
    Pulse
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave
    Square
    Wave

    View Slide

  26. Fitness ?

    View Slide

  27. https://twitter.com/Finger_smudger

    View Slide

  28. Playing
    instruments
    in
    odd
    ways
    https://en.wikipedia.org/wiki/Daphne_Oram

    View Slide

  29. Emacs Is A Operating System
    d[*_*]b
    /|!|\
    _| |_
    Is A Musical Instrument
    Yes Really

    View Slide

  30. Emacs Demo
    https://vimeo.com/265188142

    View Slide

  31. d[x_x]b
    /|!|\
    _/ \_
    https://vimeo.com/265189088
    Scratching Samples With Emacs

    View Slide

  32. View Slide

  33. Knob Twiddling
    Demo
    https://vimeo.com/269049832

    View Slide

  34. Machine
    Learning
    https://en.wikipedia.org/wiki/Else_Marie_Pade

    View Slide

  35. https://nsynthsuper.withgoogle.com/
    Neural
    Synthesizer

    View Slide

  36. Wavenet Decoder
    Amplitude
    of sound
    wave
    16
    temporal
    features
    Amplitude
    of sound
    wave
    Feedforward
    Sample Sample
    [0.5 0.2 0.3
    0.5 0.3 0.2
    0.4 0.5 0.5
    0.2 0.3]
    [0.5 0.2 0.3
    0.5 0.3 0.2
    0.4 0.5 0.5
    0.2 0.3]
    512

    View Slide

  37. https://vimeo.com/269039623

    View Slide

  38. AI
    https://chrisdonahue.github.io/wavegan/

    View Slide

  39. CreativITY
    https://en.wikipedia.org/wiki/Suzanne_Ciani

    View Slide

  40. Joseph Wilk

    View Slide