web audio api (htmlday osaka)

Transcript

1. Web audio apiͰ Ͱ͖Δ͜ͱ 2013/6/8 htmlday େࡕɹ ϑΟ @huydx Saturday, June 8, 2013
2. ࣗݾ঺հ ໊લɿυʔεΞϯϑΠ @huydx ग़਎ɿϕτφϜ ৬ۀɿֶੜ (ShareWis IncͰΠϯλʔϯʣ Ruby + javascript http://huydx.com Saturday, June 8, 2013
3. ͳͥࠓ೔Web audio API ʹ͍ͭͯൃදʁ Saturday, June 8, 2013
4. ΞδΣϯμ ‣ ΠϯτϩμΫγϣϯ ‣ ྺ࢙ ‣ Web audio api࢖͏ͨΊඞཁͳ஌ࣝ ‣ Web audio apiͷ঺հʗߏ੒ ‣ σϞλΠϜ ‣ Web audio apiͷϥΠϒϥϦ౳Λ঺հ ‣ ࣭໰λΠϜ Saturday, June 8, 2013
5. Web audio apiʹ͍ͭͯ Ͱ͕͢ɻɻ Saturday, June 8, 2013
6. • ࣭໰̍ɿ໊લΛฉ͍ͨ͜ͱ͕͋Δਓʂ • ࣭໰̎ɿ࢖ͬͨ͜ͱ͕͋Δਓʂ • ࣭໰̏ɿWeb audio apiͱͯ΋ৄ͍͠ਓ Saturday, June 8, 2013
7. • ࣭໰̍ɿ໊લΛฉ͍ͨ͜ͱ͕͋Δਓʂ • ࣭໰̏ɿͱͯ΋ৄ͍͠ਓʂ • ࣭໰̎ɿ࢖ͬͨࣄ͕͋Δਓʂ Saturday, June 8, 2013
8. • ࣭໰̍ɿ໊લΛฉ͍ͨ͜ͱ͕͋Δਓʂ • ࣭໰̏ɿͱͯ΋ৄ͍͠ਓʂ • ࣭໰̎ɿWeb audio apiͱͯ΋ৄ͍͠ਓ Saturday, June 8, 2013
9. Web audio apiͷྺ࢙ •Google Chrome͔Β࢝Ί •OpenALΛجͮ͘ •2011 / 15 / 12 : First Draft •2012 / 13 / 12 : ݱࡏͷ൛ Saturday, June 8, 2013
10. Saturday, June 8, 2013
11. ΢ΣϒͰAudioͷྺ࢙ʁ ‣ <bgsound> λάɹ(IE ͷΈʂʣ ‣ എܠͰԻΛ໐Β͢ ‣ <bgsound src=”audio/welcome.wav” /> ‣ ϑϥογϡ ‣ <audio>λά ‣ ࣮૷؆୯ ‣ ؆୯ͳԻ੠੍ޚͰ͖Δ ‣ Web audio API ొ৔ʂʂʂʂ Saturday, June 8, 2013
12. ͳͥWeb audio apiཁΔʁ • ෳ਺σʔλιʔεΛ༻͍ͯਫ਼ີͳ੍࣌ؒޚ • AudioͷύΠϓϥΠϯʗϧʔςΟϯά. • AudioͷετϦʔϜΛϑοΫͯ͠ɺΦϯβϑ ϥΠͰॲཧͰ͖Δ Saturday, June 8, 2013
13. ·ͣAudioͷجຊͷ஌ࣝ ෺ମͷৼಈ͕ۭؾͳͲͷৼಈʢԻ೾ʣ ͱͯ͠఻Θͬͯى͢ௌ֮ͷ಺༰ quote:wikipedia Saturday, June 8, 2013
14. ࣌ؒ࣠ ৼ෯࣠ Ի੠ͷRaw σʔλ ిࢠԽ ϑʔϦΤม׵ Saturday, June 8, 2013
15. Play time! Saturday, June 8, 2013
16. ·ͣ͸audioσʔλΛϩʔυ function init() { if (typeof AudioContext !== "undefined") { context = new AudioContext(); } else if (typeof webkitAudioContext !== "undefined") { context = new webkitAudioContext(); } else { throw new Error('AudioContext not supported. :('); } } Saturday, June 8, 2013
17. audioσʔλΛϩʔυ function init() { if (typeof AudioContext !== "undefined") { context = new AudioContext(); } else if (typeof webkitAudioContext !== "undefined") { context = new webkitAudioContext(); } else { throw new Error('AudioContext not supported. :('); } } ·ͣ͸AudioContextΛॳظԽ Saturday, June 8, 2013
18. function startSound() { // Note: this loads asynchronously var request = new XMLHttpRequest(); request.open("GET", url, true); request.responseType = "arraybuffer"; request.onload = function() { var audioData = request.response; context.decodeAudioData(audioData, function(buffer) { playmusic(buffer) }, onError); }; request.send(); } audioσʔλΛϩʔυ ࣍ʹXHRͰʢFileReaderͰ΋Α͍)σʔλऔಘ Saturday, June 8, 2013
19. ϩʔυͨ͠σʔλΛSource + DestinationΛ࡞ͬͯɺܨ͕Δ function playmusic(buffer) { sourceNode = context.createBufferSource( ); sourceNode.connect( context.destination ); sourceNode.buffer = buffer; sourceNode.noteOn( 0 ); } initSound( ); startSound( ); օେମͷྲྀΕΛ೺Ѳ͢ΔͰ͠ΐ͏ audioσʔλΛϩʔυ Saturday, June 8, 2013
20. σϞ Saturday, June 8, 2013
21. web audio apiߏ੒ ϊʔυ(AudioNode) Λܨ͕Δ͜ͱʹΑΓɺ άϥϑΛߏ੒͢Δ ֤ϊʔυ͸ Inputs͔ʗͱOutputsΛ࣋ͭ Saturday, June 8, 2013
22. Source Destination Input (audio data) Output (speaker) Audio Context Ұ൪؆୯ͳάϥϑ Saturday, June 8, 2013
23. ΋ͬͱ΍΍͍͜͠άϥϑ quote:W3C Saturday, June 8, 2013
24. ؆୯ʹݴ͏ͱ Input ॲཧ Output AudioSourceNodeʹ Audioσʔλೖྗ Saturday, June 8, 2013
25. ؆୯ʹݴ͏ͱ Input ॲཧ Output ͜͜Ͱউखʹॲཧ͢Δ Saturday, June 8, 2013
26. ॲཧϊʔυͷछྨ ‣ GainNode (རಘʣ ‣ BiquadFilterNode (ϑΟϧλʣ ‣ PannerNodeʢ̏࣍ݩAudioૢ࡞ʣ ‣ ConvolverNodeʢ৞ΈࠐΈʣ ‣ AnalyzerNodeʢ෼ੳʣ ‣ ....... Saturday, June 8, 2013
27. ؆୯ʹݴ͏ͱ Input ॲཧ Output AudioDestinationNodeʹ ௐ੔ͨ͠΋ͷΛग़ྗ Saturday, June 8, 2013
28. Play time! Saturday, June 8, 2013
29. ༡ͼΞδΣϯμ ‣࠶ੜͰ࣌ؒௐ੔ ‣֤ϊʔυछྨͰAudioௐ੔ΛͨΊͯ͠΍Δ ‣AnalyzerNode ‣ConvolverNode ‣OscillatorͰAudioੜ੒Λࢼͯ͠΍Δ Saturday, June 8, 2013
30. ࣌ؒௐ੔ • ෳ਺σʔλιʔεΛ༻͍ͯਫ਼ີͳ੍࣌ؒޚ • ඵ୯ҐͰ࠶ੜΛௐ੔Ͱ͖Δ Saturday, June 8, 2013
31. ࣌ؒௐ੔ Gun.prototype.shoot = function (type, rounds, interval, random, random2) { if (typeof random == 'undefined') { random = 0; } var time = context.currentTime; for (var i = 0; i < rounds; i++) { var source = this.makeSource(this.buffers[type]); source.playbackRate.value = 1 + Math.random() * random2; source.noteOn(time + i * interval + Math.random() * random); } } σʔϞ Saturday, June 8, 2013
32. ֤ϊʔυछྨͰAudioௐ੔ AnalyzerNode FFTͨ݁͠ՌΛج͍ͮͯղੳͳͲ͢ΔͨΊ ͷϊʔυ (fftSize) AnalyzerNodeΛ༻͍ͯɺप೾਺Λ෼͔Ε͹ ՄࢹԽ͕؆୯ʹͰ͖Δ Saturday, June 8, 2013
33. AnalyzerNode ΋ͬͱ΍΍͍͜͠ͷྫɿ http://html5-demos.appspot.com/static/webaudio/createMediaSourceElement.html http://chimera.labs.oreilly.com/books/1234000001552/ch05.html#s05_3 function getAverageVolume(array) { var vals = 0; var length = array.length; for (var i=0; i<length; i++) { vals += array[i]; } return (vals / length) } window.javascriptnode.onaudioprocess = function() { var array = new Uint8Array(analyzer.frequencyBinCount); analyzer.getByteFrequencyData(array); var average = getAverageVolume(array); ctx.clearRect(0, 0, 60, 130); ctx.fillStyle="#00FFFF"; ctx.fillRect(0, 130-average,25,130); } प೾਺Λarrayʹอଘ Saturday, June 8, 2013
34. ֤ϊʔυछྨͰAudioௐ੔ ConvolverNode ৞ΈࠐΈԋࢉͰ ෦԰ͷΤϑΣΫτ ͳͲΛ࠶ݱ͢Δ (ྫ͑͹ɿՎܶ৔౳ʣ Saturday, June 8, 2013
35. ConvolverNode function audioGraph(audioData) { var convolver; soundSource = context.createBufferSource(); soundBuffer = context.createBuffer(audioData, true); soundSource.buffer = soundBuffer; convolver = context.createConvolver(); soundSource.connect(convolver); convolver.connect(context.destination); Convolve('echo.mp3', convolver, function() {playSound()}); } function Convolve(url, convolver, callback) { var request = new XMLHttpRequest(); request.open("GET", url, true); request.responseType = "arraybuffer"; request.onload = function () { convolver.buffer = context.createBuffer(request.response, false); callback(); } request.send(); } Saturday, June 8, 2013
36. OscillatorͰAudioੜ੒(γϯηγεʣ ೾ܗͷछྨ (type) प೾਺ʢfrequency)ɹ σνϡϯʢdetuneʣ ࣗ෼ͰԻ੠Λੜ੒Ͱ͖Δʂ Saturday, June 8, 2013
37. OscillatorͰAudioੜ੒(γϯηγεʣ var osc = audioctx.createOscillator(); var gain = audioctx.createGain(); osc.connect(gain); gain.connect(audioctx.destination); function Setup() { if(play == 0) { osc.start(0); play = 1; } var type = document.getElementById("type").selectedIndex; var freq = parseFloat(document.getElementById("freq").value); var level = parseFloat(document.getElementById("level").value); osc.type = type; osc.frequency.value = freq; gain.gain.value = level; } Saturday, June 8, 2013
38. web audio apiͰ Ի੠ॲཧ͔Β Ի੠ੜ੒·Ͱ Saturday, June 8, 2013
39. Saturday, June 8, 2013
40. ࣮ࡍͷԠ༻ • ήʔϜʢAngryBird, CasinoRPG) • ΞϓϦͷUXϑΟουόοΫʢՄࢹԽʣ • MusicalΞϓϦέʔγϣϯʢDJ, γϯηγ ε౳ʣ • RealTimeͰAudioΤϑΣΫτ௥Ճ౳ Saturday, June 8, 2013
41. ϥΠϒϥϦ౳ ‣ MusicJS (ָثͷγϯηγε༻ʣ ‣ Howler.js (Web audio apiͷϥού) ‣ Dancer.js (Web audio apiͷϥούʣ ‣ ...... Saturday, June 8, 2013
42. ݁࿦ Web Audio API ͸ Saturday, June 8, 2013
43. Saturday, June 8, 2013
44. •ϋΠϨϕϧͷAPIΛఏڙ͢Δʢσʔ λͷܗࣜ౳Λߟྀ͠ͳͯ͘Α͍ʣ • ผͷεϨουͰॲཧ→GUIϒϩοΫ͠ͳ͍ • ๛ͳػೳʢϑΟϧλɺFFT౳ʣΛఏڙ Saturday, June 8, 2013
45. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·͢ Saturday, June 8, 2013