Do you even jam bruh?

Do you even jam bruh?

Experiments with Web Audio API and React

Ea26ebbe66c3bb6afb5f711cfe766dff?s=128

Siddharth Kshetrapal

August 20, 2017
Tweet

Transcript

  1. DO YOU EVEN JAM BRUH?

  2. DO YOU??

  3. bit.ly/reactbangalore twitter slack

  4. @siddharthkp

  5. javascript architect @practo open source github.com/siddharthkp youtube thingy bit.ly/jsshow

  6. Web Audio API

  7. AudioContext input → effects → destination

  8. AudioContext input → effects → destination OscillatorNode AudioBuffer

  9. AudioContext input → effects → destination OscillatorNode AudioBuffer BiquadFilter GainNode

    WaveShaper
  10. AudioContext input → effects → destination OscillatorNode AudioBuffer BiquadFilter GainNode

    WaveShaper Audio MediaStream
  11. Web Audio API Full spec: https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API

  12. AudioContext input → effects → destination OscillatorNode AudioBuffer BiquadFilter GainNode

    WaveShaper Audio MediaStream
  13. AudioContext input → effects → destination OscillatorNode AudioBuffer BiquadFilter GainNode

    WaveShaper Audio MediaStream
  14. AudioContext ↓ destination ↓ effect input

  15. const context = new(window.AudioContext || window.webkitAudioContext) AudioContext ↓ destination ↓

    effect input
  16. const context = new(window.AudioContext || window.webkitAudioContext) const oscillator = context.createOscillator()

    AudioContext ↓ destination ↓ effect input
  17. const context = new(window.AudioContext || window.webkitAudioContext) const oscillator = context.createOscillator()

    oscillator.frequency.setValueAtTime(400, context.currentTime) AudioContext ↓ destination ↓ effect input
  18. const context = new(window.AudioContext || window.webkitAudioContext) const oscillator = context.createOscillator()

    oscillator.frequency.setValueAtTime(400, context.currentTime) oscillator.connect(context.destination) AudioContext ↓ destination ↓ effect input
  19. const context = new(window.AudioContext || window.webkitAudioContext) const oscillator = context.createOscillator()

    oscillator.frequency.setValueAtTime(400, context.currentTime) oscillator.connect(context.destination) oscillator.start() setTimeout(() => oscillator.stop(), 1000) AudioContext ↓ destination ↓ effect input
  20. const context = new(window.AudioContext || window.webkitAudioContext) const oscillator = context.createOscillator()

    oscillator.frequency.setValueAtTime(400, context.currentTime) const gainEffect = context.createGain() oscillator.connect(context.destination) oscillator.start() setTimeout(() => oscillator.stop(), 1000) AudioContext ↓ destination ↓ effect input
  21. const context = new(window.AudioContext || window.webkitAudioContext) const oscillator = context.createOscillator()

    oscillator.frequency.setValueAtTime(400, context.currentTime) const gainEffect = context.createGain() gainEffect.gain.setValueAtTime(10, context.currentTime) oscillator.connect(context.destination) oscillator.start() setTimeout(() => oscillator.stop(), 1000) AudioContext ↓ destination ↓ effect input
  22. const context = new(window.AudioContext || window.webkitAudioContext) const oscillator = context.createOscillator()

    oscillator.frequency.setValueAtTime(400, context.currentTime) const gainEffect = context.createGain() gainEffect.gain.setValueAtTime(10, context.currentTime) oscillator.connect(gainEffect).connect(context.destination) oscillator.start() setTimeout(() => oscillator.stop(), 1000) AudioContext ↓ destination ↓ effect input
  23. const context = new(window.AudioContext || window.webkitAudioContext) const oscillator = context.createOscillator()

    oscillator.frequency.setValueAtTime(400, context.currentTime) oscillator.type = 'square' // sine, square, triangle, sawtooth const gainEffect = context.createGain() gainEffect.gain.setValueAtTime(10, context.currentTime) oscillator.connect(gainEffect).connect(context.destination) oscillator.start() setTimeout(() => oscillator.stop(), 1000) AudioContext ↓ destination ↓ effect input
  24. AudioContext ↓ destination ↓ effect const context = new(window.AudioContext ||

    window.webkitAudioContext) const source = context.createBufferSource() const url = './audio/snare.wav' input
  25. AudioContext ↓ destination ↓ effect const context = new(window.AudioContext ||

    window.webkitAudioContext) const source = context.createBufferSource() const url = './audio/snare.wav' new BufferLoader(context, url, audioBuffer => { source.buffer = audioBuffer }).load() input
  26. AudioContext ↓ destination ↓ effect const context = new(window.AudioContext ||

    window.webkitAudioContext) const source = context.createBufferSource() const url = './audio/snare.wav' new BufferLoader(context, url, audioBuffer => { source.buffer = audioBuffer source.connect(context.destination) source.start() }).load() input
  27. None
  28. • • • • • • • •

  29. • • • • • • • • kick x

    x x x x x x x
  30. • • • • • • • • kick x-----

    x----- x----- x----- x----- x----- x----- x-----
  31. • • • • • • • • kick x

    x x x x x x x
  32. • • • • • • • • kick x

    x x x x x x x snare x x x x
  33. • • • • • • • • kick x

    x x x x x x x snare x x x x hihat x x x x x x x x
  34. • • • • • • • • kick x

    x x x x x x x snare x x x x hihat x x x x x x x x bass E E E E B B B B
  35. • • • • • • • • kick x

    x x x x x x x snare x x x x hihat x x x x x x x x bass E E E E B B B B lead E2 E2 G2 E2 D2 C2 B1
  36. React

  37. • • • • • • • • kick x

    x x x x x x x snare x x x x hihat x x x x x x x x bass E E E E B B B B lead E2 E2 G2 E2 D2 C2 B1
  38. <Song> kick x x x x x x x x

    snare x x x x hihat x x x x x x x x bass E E E E B B B B lead E2 E2 G2 E2 D2 C2 B1 </Song>
  39. <Song> <Kick> x x x x x x x x

    </Kick> snare x x x x hihat x x x x x x x x bass E E E E B B B B lead E2 E2 G2 E2 D2 C2 B1 </Song>
  40. class Kick extends React.Component { }

  41. class Kick extends React.Component { constructor () { } componentDidMount

    () { } render () { } }
  42. class Kick extends React.Component { constructor () { super(props) this.state

    = { notes: this.props.children.split(' ') } } componentDidMount () { } render () { } }
  43. class Kick extends React.Component { constructor () { super(props) this.state

    = { notes: this.props.children.split(' ') } } componentDidMount () { loadInstrument('kick').then(source => { this.setState({ source }) }) } render () { } }
  44. class Kick extends React.Component { constructor () { super(props) this.state

    = { notes: this.props.children.split(' ') } } componentDidMount () { loadInstrument('kick').then(source => { this.setState({ source }) }) } render () { this.state.notes.map((note, index) => { setTimeout(() => this.state.source.play(), index * 500) }) } }
  45. <Song> <Kick> x x x x x x x x

    </Kick> <Snare> x x x x </Snare> hihat x x x x x x x x bass E E E E B B B B lead E2 E2 G2 E2 D2 C2 B1 </Song>
  46. <Song> <Kick> x x x x x x x x

    </Kick> <Snare> x o x o x o x o </Snare> hihat x x x x x x x x bass E E E E B B B B lead E2 E2 G2 E2 D2 C2 B1 </Song>
  47. class Kick extends React.Component { constructor () { ... }

    componentDidMount () { ... } render () { this.state.notes.map((note, index) => { if (index !== 'o') { setTimeout(() => this.state.source.play(), index * 500) } }) } }
  48. <Song> <Kick> x x x x x x x x

    </Kick> <Snare> x o x o x o x o </Snare> <Hihat> x x x x x x x x </Hihat> bass E E E E B B B B lead E2 E2 G2 E2 D2 C2 B1 </Song>
  49. <Song> <Kick> x x x x x x x x

    </Kick> <Snare> x o x o x o x o </Snare> <Hihat> x x x x x x x x </Hihat> <Bass> E E E E B B B B </Bass> lead E2 E2 G2 E2 D2 C2 B1 </Song>
  50. class Bass extends React.Component { constructor () { ... }

    componentDidMount () { ... } render () { this.state.notes.map((note, index) => { setTimeout(() => { this.state.source.frequency.value = getFrequency(note) this.state.source.play()) }, index * 500) }) } }
  51. const chords = { C: 65.406, D: 73.416, E: 82.407,

    F: 87.307, G: 97.999, B: 123.471, }
  52. const chords = { C: 65.406, C1: 32.703, D: 73.416,

    D1: 36.708, E: 82.407, E1: 41.203, F: 87.307, F1: 43.654, G: 97.999, G1: 48.999, B: 123.471, B1: 61.735 }
  53. const chords = { C: 65.406, C1: 32.703, D: 73.416,

    D1: 36.708, E: 82.407, E1: 41.203, F: 87.307, F1: 43.654, G: 97.999, G1: 48.999, B: 123.471, B1: 61.735 } const getFrequency = note => { return chords[note] }
  54. <Song> <Kick> x x x x x x x x

    </Kick> <Snare> x o x o x o x o </Snare> <Hihat> x x x x x x x x </Hihat> <Bass> E E E E B B B B </Bass> <Lead> E2 E2 G2 E2 D2 C2 B1 </Lead> </Song>
  55. • • • • • • • • <Lead> E2

    E2 G2 E2 D2 C2 B1 </Lead>
  56. • • • • • • • • E2 E2

    G2 E2 D2 C2 B1 <Lead> <Note> E2 </Note> <Note> E2 </Note> <Note> G2 </Note> <Note> E2 </Note> <Note> D2 </Note> <Note> C2 </Note> <Note> B1 </Note> </Lead>
  57. • • • • • • • • E2 E2

    G2 E2 D2 C2 B1 <Lead> <Note time = "0" duration = "0.75" > E2 </Note> <Note time = "0.75" duration = "0.25" > E2 </Note> <Note time = "1" duration = "0.25" > G2 </Note> <Note time = "1.375" duration = "0.25" > E2 </Note> <Note time = "1.75" duration = "0.25" > D2 </Note> <Note time = "2" duration = "1" > C2 </Note> <Note time = "3" duration = "1" > B1 </Note> </Lead>
  58. class Lead extends React.Component { constructor () { ... }

    componentDidMount () { ... } render () { this.props.children.map((child, index) => { setTimeout(() => { this.state.source.frequency.value = getFrequency(child.props.chord) this.state.source.play()) }, child.props.time * 1000) }) } }
  59. Speech Synthesis API

  60. const synth = window.speechSynthesis const voices = synth.getVoices()

  61. const synth = window.speechSynthesis const voices = synth.getVoices() const text

    = 'Hello world' const speech = new SpeechSynthesisUtterance(text); speech.voice = voices[0]
  62. const synth = window.speechSynthesis const voices = synth.getVoices() const text

    = 'Hello world' const speech = new SpeechSynthesisUtterance(text); speech.voice = voices[53] synth.speak(speech)
  63. @siddharthkp twitter github