Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Crafting Instruments In Code

Crafting Instruments In Code

Joseph Wilk

May 11, 2018
Tweet

More Decks by Joseph Wilk

Other Decks in Technology

Transcript

  1. Samples “Real-world noise of any sort has a very wide

    frequency range, far wider and more complex than any electronically generated sound.” Jon Hopkins https://en.wikipedia.org/wiki/Jon_Hopkins
  2. Channel (Right) Channel (Left) Frames 44100 1 second Sample 16bits

    Frame n+1 Frame n+3 Frame n+4 Frame n Frame n+2 176k Sample 16bits Amplitude of sound wave Stereo http://soundfile.sapp.org/doc/WaveFormat/
  3. Samples as DATA [0.5 0.2 0.3 0.5 0.3 0.2 0.4

    0.5 0.5 0.2 0.3] [0.4 0.5 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.3 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.3 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.1 0.6 0.3 0.5 0.3 0.2 0.4 0.3 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.5 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.5 0.1 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.2 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.5 0.1 0.5 0.3 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.1 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] 0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 [0.5 0.2 0.3 0.5 0.3 0.2 [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] .3 0.2 0.4 0.5 0.5 0.2 0.3] .3 0.2 0.5 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0. [0.5 0.2 0.3 0.5 0. [0.5 0.2 0.3 0.5 0. 0.4 0.5 0.5 0.2 0.3] 0.4 0.5 0.5 0.2 0.3]
  4. Fundamental frequency Midi note Root mean squared amplitude Spectral-centroid Spectral-irregularity

    Spectral-inharmonicity Spectral skewness Spectral kurtosis http://jamiebullock.github.io/LibXtract/documentation/
  5. Fundamental frequency Midi note Volume Brightness of Sound Noisiness of

    a sound How harmonic Spectrum skew Pitchiness http://jamiebullock.github.io/LibXtract/documentation/
  6. Query and use it demo T-sne “magical ability to create

    compelling two-dimensional “maps” from data with hundreds or even thousands of dimensions” https://lvdmaaten.github.io/tsne/
  7. Generators Properties All things that are a square should have

    4 sides of equal length (fn [] (let [color (gen/rgb-color) size (gen/int)] (build-a-square color size))) Property Based Testing https://en.wikipedia.org/wiki/QuickCheck
  8. Property Based Testing Generators reverb echo tube distort delay Sine

    Wave Square Wave Pulse Wave Sound Fx Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave
  9. Property Based Testing Generators reverb echo tube distort delay Sine

    Wave Square Wave Pulse Wave Sound Fx Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave Tube Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave delay Pulse Wave Square Wave Square Wave Square Wave Square Wave reverb reverb Pulse Wave reverb reverb reverb Square Wave delay reverb Pulse Wave Pulse Wave reverb Pulse Wave reverb reverb Generators reverb echo tube distort delay Sine Wave Square Wave Pulse Wave Sound Fx Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave Tube Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave delay Pulse Wave Square Wave Square Wave Square Wave Square Wave reverb reverb Pulse Wave reverb reverb reverb Square Wave delay reverb Pulse Wave Pulse Wave reverb Pulse Wave reverb reverb Generators reverb echo tube distort delay Sine Wave Square Wave Pulse Wave Sound Fx Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave Tube Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave delay Pulse Wave Square Wave Square Wave Square Wave Square Wave reverb reverb Pulse Wave reverb reverb reverb Square Wave delay reverb Pulse Wave Pulse Wave reverb Pulse Wave reverb reverb Generators reverb echo tube distort delay Sine Wave Square Wave Pulse Wave Sound Fx Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave Tube Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave Pulse Wave delay Square Wave reverb reverb Pulse Wave Square Wave delay Pulse Wave Square Wave Square Wave Square Wave Square Wave reverb reverb Pulse Wave reverb reverb reverb Square Wave delay reverb Pulse Wave Pulse Wave reverb Pulse Wave reverb reverb reverb Pulse Wave Pulse Wave reverb Pulse Wave reverb reverb reverb Pulse Wave Pulse Wave reverb Pulse Wave reverb reverb reverb Pulse Wave Pulse Wave reverb Pulse Wave reverb reverb reverb Pulse Wave Pulse Wave reverb Pulse Wave reverb reverb Sound Fx delay Pulse Wave Square Wave Square Wave Square Wave Square Wave Square Wave Sound Fx delay Pulse Wave Square Wave Square Wave Square Wave Square Wave Square Wave Sound Fx delay Pulse Wave Square Wave Square Wave Square Wave Square Wave Square Wave Sound Fx delay Pulse Wave Square Wave Square Wave Square Wave Square Wave Square Wave Sound Fx delay Pulse Wave Square Wave Square Wave Square Wave Square Wave Square Wave Sound Fx delay Pulse Wave Square Wave Square Wave Square Wave Square Wave Square Wave Sound Fx delay Pulse Wave Square Wave Square Wave Square Wave Square Wave Square Wave Sound Fx delay Pulse Wave Square Wave Square Wave Square Wave Square Wave Square Wave
  10. Emacs Is A Operating System d[*_*]b /|!|\ _| |_ Is

    A Musical Instrument Yes Really
  11. Wavenet Decoder Amplitude of sound wave 16 temporal features Amplitude

    of sound wave Feedforward Sample Sample [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] [0.5 0.2 0.3 0.5 0.3 0.2 0.4 0.5 0.5 0.2 0.3] 512