JSMad - What, Why, How (JSConf EU 2012)

Slide 1

Slide 1 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger jsmad What, Why, How ?

Slide 2

Slide 2 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What ? A pure JavaScript MP3 decoder

Slide 3

Slide 3 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « How do we play music on the web? »

Slide 4

Slide 4 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger The uncle you wish you never had

Slide 5

Slide 5 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger Flash 10 penetration Source : adobe.com

Slide 6

Slide 6 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Adobe Flash Player has 'critical' security issue, won't be addressed until next week » — digitaltrends.com

Slide 7

Slide 7 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « 16 month-old bug continues to crash Flash » — tuaw.com

Slide 8

Slide 8 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Flash has […] one of the worst security records in 2009. It is the number one reason Macs crash. » — Steve Jobs

Slide 9

Slide 9 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « The slightly clueless kid on the blocks. »

Slide 10

Slide 10 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « The second best reason to buy an i7. »

Slide 11

Slide 11 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What ? A port of libmad

Slide 12

Slide 12 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « libmad - a clean-room high-quality GPL'ed MP3 decoder »

Slide 13

Slide 13 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « A port of libmad to JavaScript that allows MP3 playback in browsers without any external plug-ins. »

Slide 14

Slide 14 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « libmad has 24-bit PCM output, does 32-bit fxed point arithmetic & is fully ISO/IEC 11172 compliant » — underbit.com

Slide 15

Slide 15 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « jsmad uses foating-point arithmetic, and has not been tested for ISO/IEC compliance » — @nddrylliog + @jensnockert

Slide 16

Slide 16 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger Why ? The patent situation

Slide 17

Slide 17 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « We have patents on the psycho-acoustic models underlying MPEG-1 Layer III audio encoding. » — Fraunhofer & Thomson

Slide 18

Slide 18 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « You have to pay us money if you use MP3 encoders or decoders in for-proft software. » — Fraunhofer & Thomson

Slide 19

Slide 19 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « La la la can't hear you. » — Every streaming website

Slide 20

Slide 20 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger OGG MP3 AAC WAV IE 9 ✘ ✔ ✔ ✘ Firefox 5 ✔ ✘ ✘ ✔ Chrome 12 ✔ ✔ ✔ ✔ Safari 5 ✘ ✔ ✔ ✔ Opera 11.5 ✔ ✘ ✘ ✔

Slide 21

Slide 21 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Firefox, Y U NO support MP3? » — Every web developer

Slide 22

Slide 22 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « We don't want to encourage proprietary formats. » — Mozilla

Slide 23

Slide 23 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « There, take a sticker and stop asking questions. » — Mozilla

Slide 24

Slide 24 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Sorry, we don't do stickers. » — Opera

Slide 25

Slide 25 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « MP3 is rubbish anyway ! » — Anonymous

Slide 26

Slide 26 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Yeah, yeah, I encode MP3 alright. » — iTunes

Slide 27

Slide 27 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « We developed our own psycho-acoustic models » — LAME project

Slide 28

Slide 28 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 29

Slide 29 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger Why ? Features

Slide 30

Slide 30 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 31

Slide 31 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « What do you mean, seeking ? » — HTML5 Audio

Slide 32

Slide 32 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Seeking beyond the buffer has worked for ages. » — Adobe Flash

Slide 33

Slide 33 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 34

Slide 34 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « What do you mean, live? » — HTML5 Audio

Slide 35

Slide 35 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « RTMP solved this years ago. » — Adobe Flash

Slide 36

Slide 36 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 37

Slide 37 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « What do you mean, SFX? » — HTML5 Audio

Slide 38

Slide 38 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Ever played a Flash game ? » — Adobe Flash

Slide 39

Slide 39 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « I decode MP3 streams without making assumptions about your container. » — jsmad

Slide 40

Slide 40 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger How ? How audio compession works

Slide 41

Slide 41 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 42

Slide 42 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 43

Slide 43 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 44

Slide 44 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 45

Slide 45 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 46

Slide 46 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 47

Slide 47 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger How ? How HTML5 works

Slide 48

Slide 48 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger HTML5 Oh, wow, what are you using, Lynx?

Slide 49

Slide 49 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger HTML5

Slide 50

Slide 50 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What HTML5 does for you ● Data input (fle://, http://) ● Buffering (206 Partial Content) ● Demuxing (MPEG, OGG, M4A) ● Decoding (MP3, Vorbis, AAC) ● Audio playback (CoreAudio, PulseAudio) ● User interface

Slide 51

Slide 51 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What HTML5 does for you ● Data input ● Buffering ● Demuxing ● Decoding ● Audio playback ● User interface

Slide 52

Slide 52 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What jsmad does ● Data input ● Buffering ● Demuxing ● Decoding ● Audio playback ● User interface

Slide 53

Slide 53 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger What jsmad.org runs ● Data input ● Buffering ● Demuxing ● Decoding = jsmad ● Audio playback = sink.js ● User interface

Slide 54

Slide 54 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger jsmad — decoding all frames var file = fileChooser.files[0]; new Mad.FileStream(file, function(input) { var mp3 = new Mad.MP3File(input); var mpeg = mp3.getMpegStream(); var frame = new Mad.Frame(); while (mpeg.error != Mad.Error.BUFLEN) { frame = Mad.Frame.decode(frame); } });

Slide 55

Slide 55 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger sink.js — playing audio var channels = 2; // stereo var rate = 44100; // 44.1 Khz var bufsize = 65536 * 4096; function refill(buffer) { // … } var dev = Sink(refill, channels, bufsize, rate);

Slide 56

Slide 56 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger sink.js + jsmad var frame = /* … */; var synth = new Mad.Synth(); function refill(buffer) { // if current frame all used up, decode new one frame = Mad.Frame.decode(frame); // synthesize new one if needed synth.frame(frame); // copy samples from synth to buffer // (TypedArray.set or for loop) } var dev = Sink(refill, channels, bufsize, rate);

Slide 57

Slide 57 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger How ? Low-level audio APIs

Slide 58

Slide 58 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger

Slide 59

Slide 59 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « JavaScript is fast enough. » — jsmad

Slide 60

Slide 60 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger « Just give us low-level access to devices, we'll handle it from there » — jsmad

Slide 61

Slide 61 text

JSConf EU 2011 github.com/audiocogs/jsmad Amos Wenger Thank you ! github.com/audiocogs jsmad.org @nddrylliog