Tweet
Tweet

Slide 1

Slide 1 text

CRAZY COMPUTER vision stuff in the browser

Slide 2

Slide 2 text

.html .css .js

Slide 3

Slide 3 text

zenorocha

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

computer vision ? what is

Slide 10

Slide 10 text

Persistence of vision

Slide 11

Slide 11 text

Persistence of vision

Slide 12

Slide 12 text

Motion perception

Slide 13

Slide 13 text

Motion perception

Slide 14

Slide 14 text

vimeo.com/108331968

Slide 15

Slide 15 text

Face detection

Slide 16

Slide 16 text

Face recognition

Slide 17

Slide 17 text

Autonomous robots

Slide 18

Slide 18 text

Autonomous robots

Slide 19

Slide 19 text

Virtual reality

Slide 20

Slide 20 text

Augmented reality

Slide 21

Slide 21 text

Augmented reality

Slide 22

Slide 22 text

how to do that?

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

how to do that in the browser?

Slide 26

Slide 26 text

1. Access the camera

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

Request user’s webcam navigator.getUserMedia({ video: true }, onSuccess, onFail);

Slide 29

Slide 29 text

Request user’s webcam navigator.getUserMedia({ video: true }, onSuccess, onFail);

Slide 30

Slide 30 text

Request user’s webcam navigator.getUserMedia({ video: true }, onSuccess, onFail);

Slide 31

Slide 31 text

2. Reproduce in a video

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Get User Media callback function onSuccess(stream) { var video = document.querySelector('video'); video.src = URL.createObjectURL(stream); video.onloadedmetadata = function(e) { // Done }; }

Slide 34

Slide 34 text

Get User Media callback function onSuccess(stream) { var video = document.querySelector('video'); video.src = URL.createObjectURL(stream); video.onloadedmetadata = function(e) { // Done }; }

Slide 35

Slide 35 text

Get User Media callback function onSuccess(stream) { var video = document.querySelector('video'); video.src = URL.createObjectURL(stream); video.onloadedmetadata = function(e) { // Done }; }

Slide 36

Slide 36 text

Get User Media callback function onSuccess(stream) { var video = document.querySelector('video'); video.src = URL.createObjectURL(stream); video.onloadedmetadata = function(e) { // Done }; }

Slide 37

Slide 37 text

3. Obtain pixel matrix

Slide 38

Slide 38 text

Slide 39

Slide 39 text

4. Tracking algorithms

Slide 40

Slide 40 text

Request accepted Request access to the camera Reproduce
 camera Obtain pixel matrix Tracking algorithms Result Summary

Slide 41

Slide 41 text

tracking.js

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

Maira Bello Team eduardo lundgren java pablo carvalho zeno rocha

Slide 44

Slide 44 text

100% JavaScript! Open source No dependencies Simple and intuitive API Automatic setup Built-in tracking
 algorithms Easily extensible High performance Unit & performance
 regression tests ~ 7 Kb

Slide 45

Slide 45 text

tracker color

Slide 46

Slide 46 text

Demo

Slide 47

Slide 47 text

var tracker = new tracking.ColorTracker('magenta'); tracking.track('#video', tracker, { camera: true }); tracker.on('track', function(event) { event.data.forEach(function(rect) { // Your code goes here }); });

Slide 48

Slide 48 text

var tracker = new tracking.ColorTracker('magenta'); tracking.track('#video', tracker, { camera: true }); tracker.on('track', function(event) { event.data.forEach(function(rect) { // Your code goes here }); });

Slide 49

Slide 49 text

var tracker = new tracking.ColorTracker('magenta'); tracking.track('#video', tracker, { camera: true }); tracker.on('track', function(event) { event.data.forEach(function(rect) { // Your code goes here }); });

Slide 50

Slide 50 text

tracker object

Slide 51

Slide 51 text

Demo

Slide 52

Slide 52 text

var tracker = new tracking.ObjectTracker('face'); tracking.track('#video', tracker, { camera: true }); tracker.on('track', function(event) { event.data.forEach(function(rect) { // Your code goes here }); });

Slide 53

Slide 53 text

var tracker = new tracking.ObjectTracker('face'); tracking.track('#video', tracker, { camera: true }); tracker.on('track', function(event) { event.data.forEach(function(rect) { // Your code goes here }); });

Slide 54

Slide 54 text

var tracker = new tracking.ObjectTracker('face'); tracking.track('#video', tracker, { camera: true }); tracker.on('track', function(event) { event.data.forEach(function(rect) { // Your code goes here }); });

Slide 55

Slide 55 text

tracker custom

Slide 56

Slide 56 text

var MyTracker = function() { MyTracker.base(this, 'constructor'); }; tracking.inherits(MyTracker, tracking.Tracker); MyTracker.prototype.track = function(pxs, width, height) { // Your code here this.emit('track', { // Your code here }); };

Slide 57

Slide 57 text

var MyTracker = function() { MyTracker.base(this, 'constructor'); }; tracking.inherits(MyTracker, tracking.Tracker); MyTracker.prototype.track = function(pxs, width, height) { // Your code here this.emit('track', { // Your code here }); };

Slide 58

Slide 58 text

var MyTracker = function() { MyTracker.base(this, 'constructor'); }; tracking.inherits(MyTracker, tracking.Tracker); MyTracker.prototype.track = function(pxs, width, height) { // Your code here this.emit('track', { // Your code here }); };

Slide 59

Slide 59 text

var MyTracker = function() { MyTracker.base(this, 'constructor'); }; tracking.inherits(MyTracker, tracking.Tracker); MyTracker.prototype.track = function(pxs, width, height) { // Your code here this.emit('track', { // Your code here }); };

Slide 60

Slide 60 text

var tracker = new tracking.MyTracker(); tracking.track('#video', tracker, { camera: true }); tracker.on('track', function(event) { event.data.forEach(function(rect) { // Your code goes here }); });

Slide 61

Slide 61 text

var tracker = new tracking.MyTracker(); tracking.track('#video', tracker, { camera: true }); tracker.on('track', function(event) { event.data.forEach(function(rect) { // Your code goes here }); });

Slide 62

Slide 62 text

var tracker = new tracking.MyTracker(); tracking.track('#video', tracker, { camera: true }); tracker.on('track', function(event) { event.data.forEach(function(rect) { // Your code goes here }); });

Slide 63

Slide 63 text

who is using it?

Slide 64

Slide 64 text

kdzwinel.github.io/JS-face-tracking-demo

Slide 65

Slide 65 text

thedevilini.com/#masks

Slide 66

Slide 66 text

lessons learned

Slide 67

Slide 67 text

24 fps 60 fps 1000ms / 24fps = 41ms per frame

Slide 68

Slide 68 text

24 fps 60 fps 1000ms / 24fps = 41ms per frame

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

Int32Array Uint16Array Float64Array Uint8ClampedArray [ ] Typed arrays

Slide 73

Slide 73 text

jsperf.com/tracking-js-arrays

Slide 74

Slide 74 text

jsperf.com/tracking-js-arithmetic

Slide 75

Slide 75 text

Demo

Slide 76

Slide 76 text

No content

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

web components > > -

Slide 79

Slide 79 text

how we create a component nowadays?

Slide 80

Slide 80 text

1. Never create! Just use a plugin

Slide 81

Slide 81 text

2. Copy & paste someone’s code

Slide 82

Slide 82 text

3. And hope it works

Slide 83

Slide 83 text

3. And hope it works

Slide 84

Slide 84 text

web components

Slide 85

Slide 85 text

web components Templates

Slide 86

Slide 86 text

web components Templates Custom Elements

Slide 87

Slide 87 text

web components Templates Custom Elements Shadow DOM

Slide 88

Slide 88 text

web components HTML Imports Templates Custom Elements Shadow DOM

Slide 89

Slide 89 text

No content

Slide 90

Slide 90 text

var tracker = new tracking.ColorTracker('magenta'); tracking.track('#video', tracker, { camera: true }); tracker.on('track', function(event) { event.data.forEach(function(rect) { // Your code goes here }); }); Remember?

Slide 91

Slide 91 text

Using Web Components…

Slide 92

Slide 92 text

more ?

Slide 93

Slide 93 text

trackingjs.com

Slide 94

Slide 94 text

Where the magic happens Your comfort zone

Slide 95

Slide 95 text

Thanks! zenorocha.com @zenorocha

Slide 96

Slide 96 text

Thanks! zenorocha.com @zenorocha