JSMad - What, Why, How (JSConf EU 2012)

As part of my work at ofmlabs, I co-developed an MP3 decoder in pure JavaScript. I was kindly invited to present it to JSConf.eu.

Transcript

1. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger jsmad What, Why, How ?
2. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What ? A pure JavaScript MP3 decoder
3. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « How do we play music on the web? »
4. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger The uncle you wish you never had
5. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger Flash 10 penetration Source : adobe.com
6. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Adobe Flash Player has 'critical' security issue, won't be addressed until next week » — digitaltrends.com
7. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « 16 month-old bug continues to crash Flash » — tuaw.com
8. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Flash has […] one of the worst security records in 2009. It is the number one reason Macs crash. » — Steve Jobs
9. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « The slightly clueless kid on the blocks. »
10. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « The second best reason to buy an i7. »
11. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What ? A port of libmad
12. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « libmad - a clean-room high-quality GPL'ed MP3 decoder »
13. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « A port of libmad to JavaScript that allows MP3 playback in browsers without any external plug-ins. »
14. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « libmad has 24-bit PCM output, does 32-bit fxed point arithmetic & is fully ISO/IEC 11172 compliant » — underbit.com
15. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « jsmad uses foating-point arithmetic, and has not been tested for ISO/IEC compliance » — @nddrylliog + @jensnockert
16. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger Why ? The patent situation
17. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « We have patents on the psycho-acoustic models underlying MPEG-1 Layer III audio encoding. » — Fraunhofer & Thomson
18. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « You have to pay us money if you use MP3 encoders or decoders in for-proft software. » — Fraunhofer & Thomson
19. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « La la la can't hear you. » — Every streaming website
20. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger OGG MP3 AAC WAV IE 9 ✘ ✔ ✔ ✘ Firefox 5 ✔ ✘ ✘ ✔ Chrome 12 ✔ ✔ ✔ ✔ Safari 5 ✘ ✔ ✔ ✔ Opera 11.5 ✔ ✘ ✘ ✔
21. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Firefox, Y U NO support MP3? » — Every web developer
22. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « We don't want to encourage proprietary formats. » — Mozilla
23. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « There, take a sticker and stop asking questions. » — Mozilla
24. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Sorry, we don't do stickers. » — Opera
25. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « MP3 is rubbish anyway ! » — Anonymous
26. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Yeah, yeah, I encode MP3 alright. » — iTunes
27. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « We developed our own psycho-acoustic models » — LAME project
28. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
29. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger Why ? Features
30. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
31. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « What do you mean, seeking ? » — HTML5 Audio
32. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Seeking beyond the buffer has worked for ages. » — Adobe Flash
33. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
34. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « What do you mean, live? » — HTML5 Audio
35. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « RTMP solved this years ago. » — Adobe Flash
36. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
37. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « What do you mean, SFX? » — HTML5 Audio
38. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Ever played a Flash game ? » — Adobe Flash
39. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « I decode MP3 streams without making assumptions about your container. » — jsmad
40. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger How ? How audio compession works
41. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
42. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
43. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
44. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
45. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
46. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
47. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger How ? How HTML5 <audio> works
48. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger HTML5 <audio> <audio src="somesong.mp3"> Oh, wow, what are you using, Lynx? </audio>
49. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger HTML5 <audio> <audio> <source src="somesong.mp3"> <source src="somesong.ogg"> </audio>
50. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What HTML5 <audio> does for you • Data input (fle://, http://) • Buffering (206 Partial Content) • Demuxing (MPEG, OGG, M4A) • Decoding (MP3, Vorbis, AAC) • Audio playback (CoreAudio, PulseAudio) • User interface
51. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What HTML5 <audio> does for you • Data input • Buffering • Demuxing • Decoding • Audio playback • User interface
52. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What jsmad does • Data input • Buffering • Demuxing • Decoding • Audio playback • User interface
53. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What jsmad.org runs • Data input • Buffering • Demuxing • Decoding = jsmad • Audio playback = sink.js • User interface
54. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger jsmad — decoding all frames var file = fileChooser.files[0]; new Mad.FileStream(file, function(input) { var mp3 = new Mad.MP3File(input); var mpeg = mp3.getMpegStream(); var frame = new Mad.Frame(); while (mpeg.error != Mad.Error.BUFLEN) { frame = Mad.Frame.decode(frame); } });
55. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger sink.js — playing audio var channels = 2; // stereo var rate = 44100; // 44.1 Khz var bufsize = 65536 * 4096; function refill(buffer) { // … } var dev = Sink(refill, channels, bufsize, rate);
56. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger sink.js + jsmad var frame = /* … */; var synth = new Mad.Synth(); function refill(buffer) { // if current frame all used up, decode new one frame = Mad.Frame.decode(frame); // synthesize new one if needed synth.frame(frame); // copy samples from synth to buffer // (TypedArray.set or for loop) } var dev = Sink(refill, channels, bufsize, rate);
57. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger How ? Low-level audio APIs
58. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
59. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « JavaScript is fast enough. » — jsmad
60. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Just give us low-level access to devices, we'll handle it from there » — jsmad
61. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger Thank you ! github.com/audiocogs jsmad.org @nddrylliog