Building Motion Detecting Web Apps with JavaScript

What do my pets do when no one is around? I decided to find out. This talk covers how (and then some).

A webcam and some JavaScript using WebRTC is enough to take snapshots, but the trick is finding the snapshots worth saving. I'll share how to write an image diffing algorithm that creates a visual motion heatmap, which can be used to programmatically recognize when a snapshot has caught something happening. The interesting snapshots are then uploaded to Twitter for remote viewing.

This talk is also available in article form here: http://codersblock.com/blog/motion-detection-with-javascript/.

Demos (and links to source code repos) are here: http://diffcam.com/.

Transcript

1. BUILDING MOTION
   DETECTING WEB APPS
   WITH JAVASCRIPT
   WILL BOYD
   

   View Slide

2. Desmond Maya
   

   View Slide

3. CAT ACTIVITY
   10%
   90%
   Sleeping
   Hilarious Antics?
   

   View Slide

4. Animals Webcam
   Motion Detection Image Capture
   The CloudTM?
   

   View Slide

5. View Slide

6. View Slide

7. adapter.js
   /webrtc/adapter
   

   View Slide

8. View Slide

9. https
   http (except localhost)
   

   View Slide

10. DEMO TIME
    SIMPLE WEBCAM
    

    View Slide

11. HTML
    JAVASCRIPT
    

    View Slide

12. View Slide

13. 
    

    View Slide

14. DEMO TIME
    IMAGE DIFFING
    

    View Slide

15. HTML
    JAVASCRIPT
    

    View Slide

16. JAVASCRIPT
    

    View Slide

17. JAVASCRIPT
    [193, 76, 70, 255, 0, 0, 0, 255, …]
    

    View Slide

18. JAVASCRIPT
    [193, 76, 70, 255, 0, 0, 0, 255, …]
    

    View Slide

19. JAVASCRIPT
    [193, 76, 70, 255, 0, 0, 0, 255, …]
    PIXEL
    

    View Slide

20. R: 252
    G: 90
    B: 146
    R: 59
    G: 166
    B: 216
    R: 193

    G: 76
    B: 70
    (abs)
    

    View Slide

21. R: 0
    G: 0
    B: 0
    R: 255
    G: 255
    B: 255
    NO DIFFERENCE MOST DIFFERENCE
    

    View Slide

22. JAVASCRIPT
    

    View Slide

23. View Slide

24. View Slide

25. DEMO TIME
    MOTION HEATMAP
    

    View Slide

26. VIDEO STREAM
    

    View Slide

27. VIDEO STREAM
    

    View Slide

28. VIDEO STREAM
    0 ms 100 ms 200 ms 300 ms 400 ms
    

    View Slide

29. JAVASCRIPT
    

    View Slide

30. JAVASCRIPT
    

    View Slide

31. Animals Webcam
    Motion Detection Image Capture
    The CloudTM?
    

    View Slide

32. ‣ Remote viewing
    ‣ Accept image uploads
    ‣ Host images
    ‣ Display feed
    ‣ Privacy options
    ‣ Authentication
    ‣ Sharing
    ‣ Notifications
    

    View Slide

33. ‣ Remote viewing
    ‣ Accept image uploads
    ‣ Host images
    ‣ Display feed
    ‣ Privacy options
    ‣ Authentication
    ‣ Sharing
    ‣ Notifications
    

    View Slide

34. Page Back-End Twitter
    ajax API
    

    View Slide

35. WATCHING CONSIDERING UPLOADING
    CHILLING
    something

    moves
    after 4

    seconds
    immediately
    after 12 seconds
    

    View Slide

36. DEMO TIME
    DIFF CAM FEED
    

    View Slide

37. View Slide

38. View Slide

39. View Slide

40. ‣Raspbian
    ‣uv4l
    ‣IceWeasel
    ‣USB webcam
    

    View Slide

41. View Slide

42. View Slide

43. /lonekorean/diff-cam-engine
    DIFF CAM ENGINE
    

    View Slide

44. DIFFCAM.COM
    

    View Slide

45. /lonekorean
    WILL BOYD
    /lonekorean
    THANKS!
    

    View Slide