Tracking.js

The tracking.js library which brings different computer vision algorithms and techniques into the browser environment. By using modern HTML5 specifications, we enable you to do real-time color tracking, face detection and much more.

Transcript

1. @zenorocha
   tracking.js
   A modern approach for
   Computer Vision on the web
   

   View Slide

2. View Slide

3. View Slide

4. View Slide

5. View Slide

6. Computer Vision
   

   View Slide

7. Face detection
   

   View Slide

8. Face recognition
   

   View Slide

9. Autonomous robots
   

   View Slide

10. Virtual reality
    

    View Slide

11. Augmented reality
    

    View Slide

12. How to do that?
    

    View Slide

13. OpenCV
    

    View Slide

14. ARToolkit
    

    View Slide

15. How to do that
    in the browser?
    

    View Slide

16. Access the camera
    #1
    

    View Slide

17. View Slide

18. Request user’s webcam
    navigator.getUserMedia({
    video: true
    }, onSuccess, onFail);
    

    View Slide

19. Reproduce it in a video
    #2
    

    View Slide

20. 
    

    View Slide

21. Success callback
    function onSuccess(stream) {
    var video = document.querySelector('video');
    video.src = window.URL.createObjectURL(stream);
    video.onloadedmetadata = function(e) {
    // Pronto
    };
    }
    

    View Slide

22. Obtain the pixel array
    #3
    

    View Slide

23. 
    

    View Slide

24. Tracking algorithms
    #4
    

    View Slide

25. Summary
    User request
    
    Access the
    camera
    Reproduce it
    in a video
    Tracking
    algorithms
    Obtain the
    pixel array
    Result
    

    View Slide

26. tracking.js
    

    View Slide

27. View Slide

28. Maira
    Bello
    Pablo
    Carvalho
    Zeno
    Rocha
    Team
    Eduardo
    Lundgren
    Java
    

    View Slide

29. 100% JavaScript!
    Open Source
    No dependencies
    tracking.js
    Built-in tracking
    algorithms
    Easily extensible
    Simple and intuitive API
    High performance
    Unit & performance
    regression tests
    Automatic setup ~ 7 Kb
    

    View Slide

30. Trackers
    

    View Slide

31. Color Tracker
    

    View Slide

32. Demo
    

    View Slide

33. var tracker = new tracking.ColorTracker(‘magenta’);
    !
    tracking.track('#video', tracker, {
    camera: true
    });
    !
    tracker.on('track', function(event) {
    event.data.forEach(function(rect) {
    // Seu código aqui
    });
    });
    

    View Slide

34. Object Tracker
    

    View Slide

35. Demo
    

    View Slide

36. var tracker = new tracking.ObjectTracker(‘face’);
    !
    tracking.track('#video', tracker, {
    camera: true
    });
    !
    tracker.on('track', function(event) {
    event.data.forEach(function(rect) {
    // Seu código aqui
    });
    });
    

    View Slide

37. Custom Tracker
    

    View Slide

38. var MyTracker = function() {
    MyTracker.base(this, 'constructor');
    };
    !
    tracking.inherits(MyTracker, tracking.Tracker);
    !
    MyTracker.prototype.track = function(pixels, width, height) {
    // Seu código aqui
    !
    this.emit('track', {
    // Seu código aqui
    });
    };
    

    View Slide

39. var tracker = new tracking.MyTracker();
    !
    tracking.track('#video', tracker, {
    camera: true
    });
    !
    tracker.on('track', function(event) {
    event.data.forEach(function(rect) {
    // Seu código aqui
    });
    });
    

    View Slide

40. Utilities
    

    View Slide

41. Fast
    

    View Slide

42. View Slide

43. Brief
    

    View Slide

44. View Slide

45. Lessons learned
    

    View Slide

46. 41ms
    per frame
    24 fps
    Real time
    

    View Slide

47. View Slide

48. Algorithm
    Complexity
    

    View Slide

49. Int32Array
    Uint16Array
    Float64Array
    Uint8ClampedArray
    Typed
    Arrays
    

    View Slide

50. jsperf.com/tracking-js-arrays
    

    View Slide

51. jsperf.com/tracking-js-arithmetic
    

    View Slide

52. Demo
    

    View Slide

53. Web Components
    >
    

    View Slide

54. var tracker = new tracking.ColorTracker(‘magenta’);
    !
    tracking.track('#video', tracker, {
    camera: true
    });
    !
    tracker.on('track', function(event) {
    event.data.forEach(function(rect) {
    // Seu código aqui
    });
    });
    Remember?
    

    View Slide

55. target="magenta"
    camera="true"
    ontrack="onResult">
    Using Web Components…
    

    View Slide

56. Documentation
    

    View Slide

57. trackingjs.com
    

    View Slide

58. View Slide

59. Thanks!
    trackingjs.com
    hi@zenorocha.com
    twitter.com/zenorocha
    

    View Slide