$30 off During Our Annual Pro Sale. View Details »

JSMad - What, Why, How (JSConf EU 2012)

JSMad - What, Why, How (JSConf EU 2012)

As part of my work at ofmlabs, I co-developed an MP3 decoder in pure JavaScript. I was kindly invited to present it to JSConf.eu.

Amos Wenger

October 02, 2011
Tweet

More Decks by Amos Wenger

Other Decks in Programming

Transcript

  1. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    jsmad
    What, Why, How ?

    View Slide

  2. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    What ?
    A pure JavaScript MP3 decoder

    View Slide

  3. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « How do we play music on the web? »

    View Slide

  4. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    The uncle you wish you never had

    View Slide

  5. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    Flash 10 penetration
    Source : adobe.com

    View Slide

  6. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « Adobe Flash Player has 'critical' security issue,
    won't be addressed until next week »
    — digitaltrends.com

    View Slide

  7. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « 16 month-old bug continues
    to crash Flash »
    — tuaw.com

    View Slide

  8. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « Flash has […] one of the worst security
    records in 2009. It is the number one reason
    Macs crash. »
    — Steve Jobs

    View Slide

  9. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « The slightly clueless
    kid on the blocks. »

    View Slide

  10. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « The second best
    reason to buy an i7. »

    View Slide

  11. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    What ?
    A port of libmad

    View Slide

  12. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « libmad - a clean-room high-quality
    GPL'ed MP3 decoder »

    View Slide

  13. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « A port of libmad to JavaScript that
    allows MP3 playback in browsers
    without any external plug-ins. »

    View Slide

  14. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « libmad has 24-bit PCM output,
    does 32-bit fxed point arithmetic &
    is fully ISO/IEC 11172 compliant »
    — underbit.com

    View Slide

  15. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « jsmad uses foating-point
    arithmetic, and has not been tested
    for ISO/IEC compliance »
    — @nddrylliog + @jensnockert

    View Slide

  16. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    Why ?
    The patent situation

    View Slide

  17. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « We have patents on the psycho-acoustic models
    underlying MPEG-1 Layer III audio encoding. »
    — Fraunhofer & Thomson

    View Slide

  18. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « You have to pay us money if you use MP3 encoders
    or decoders in for-proft software. »
    — Fraunhofer & Thomson

    View Slide

  19. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « La la la can't hear you. »
    — Every streaming website

    View Slide

  20. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    OGG MP3 AAC WAV
    IE 9 ✘ ✔ ✔ ✘
    Firefox 5 ✔ ✘ ✘ ✔
    Chrome 12 ✔ ✔ ✔ ✔
    Safari 5 ✘ ✔ ✔ ✔
    Opera 11.5 ✔ ✘ ✘ ✔

    View Slide

  21. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « Firefox, Y U NO support MP3? »
    — Every web developer

    View Slide

  22. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « We don't want to encourage proprietary formats. »
    — Mozilla

    View Slide

  23. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « There, take a sticker and stop asking questions. »
    — Mozilla

    View Slide

  24. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « Sorry, we don't do stickers. »
    — Opera

    View Slide

  25. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « MP3 is rubbish anyway ! »
    — Anonymous

    View Slide

  26. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « Yeah, yeah, I encode MP3 alright. »
    — iTunes

    View Slide

  27. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « We developed our own psycho-acoustic models »
    — LAME project

    View Slide

  28. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  29. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    Why ?
    Features

    View Slide

  30. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  31. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « What do you mean, seeking ? »
    — HTML5 Audio

    View Slide

  32. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « Seeking beyond the buffer
    has worked for ages. »
    — Adobe Flash

    View Slide

  33. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  34. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « What do you mean, live? »
    — HTML5 Audio

    View Slide

  35. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « RTMP solved this years ago. »
    — Adobe Flash

    View Slide

  36. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  37. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « What do you mean, SFX? »
    — HTML5 Audio

    View Slide

  38. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « Ever played a Flash game ? »
    — Adobe Flash

    View Slide

  39. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « I decode MP3 streams without making
    assumptions about your container. »
    — jsmad

    View Slide

  40. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    How ?
    How audio compession works

    View Slide

  41. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  42. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  43. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  44. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  45. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  46. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  47. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    How ?
    How HTML5 works

    View Slide

  48. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    HTML5

    Oh, wow, what are you using, Lynx?

    View Slide

  49. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    HTML5




    View Slide

  50. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    What HTML5 does for you
    ● Data input (fle://, http://)
    ● Buffering (206 Partial Content)
    ● Demuxing (MPEG, OGG, M4A)
    ● Decoding (MP3, Vorbis, AAC)
    ● Audio playback (CoreAudio, PulseAudio)
    ● User interface

    View Slide

  51. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    What HTML5 does for you
    ● Data input
    ● Buffering
    ● Demuxing
    ● Decoding
    ● Audio playback
    ● User interface

    View Slide

  52. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    What jsmad does
    ● Data input
    ● Buffering
    ● Demuxing
    ● Decoding
    ● Audio playback
    ● User interface

    View Slide

  53. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    What jsmad.org runs
    ● Data input
    ● Buffering
    ● Demuxing
    ● Decoding = jsmad
    ● Audio playback = sink.js
    ● User interface

    View Slide

  54. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    jsmad — decoding all frames
    var file = fileChooser.files[0];
    new Mad.FileStream(file, function(input) {
    var mp3 = new Mad.MP3File(input);
    var mpeg = mp3.getMpegStream();
    var frame = new Mad.Frame();
    while (mpeg.error != Mad.Error.BUFLEN) {
    frame = Mad.Frame.decode(frame);
    }
    });

    View Slide

  55. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    sink.js — playing audio
    var channels = 2; // stereo
    var rate = 44100; // 44.1 Khz
    var bufsize = 65536 * 4096;
    function refill(buffer) {
    // …
    }
    var dev = Sink(refill, channels, bufsize, rate);

    View Slide

  56. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    sink.js + jsmad
    var frame = /* … */;
    var synth = new Mad.Synth();
    function refill(buffer) {
    // if current frame all used up, decode new one
    frame = Mad.Frame.decode(frame);
    // synthesize new one if needed
    synth.frame(frame);
    // copy samples from synth to buffer
    // (TypedArray.set or for loop)
    }
    var dev = Sink(refill, channels, bufsize, rate);

    View Slide

  57. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    How ?
    Low-level audio APIs

    View Slide

  58. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

    View Slide

  59. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « JavaScript is fast enough. »
    — jsmad

    View Slide

  60. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    « Just give us low-level access to devices, we'll
    handle it from there »
    — jsmad

    View Slide

  61. JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger
    Thank you !
    github.com/audiocogs
    jsmad.org
    @nddrylliog

    View Slide