Building Camera based Intelligent Applications

Presented at PyData Delhi 2017

Transcript

1. Building Camera based
   Intelligent Applications
   Nabarun Pal
   

   View Slide

2. ● Final Year Undergrad @ IITR
   ● Software Engineer Intern @ rorodata
   ● Systems Engineer @ Team Robocon IITR
   ● Love my kindle and hacking through things
   About Me
   nabarun.in
   github.com/palnabarun
   

   View Slide

3. Overview
   ● Introduction
   ● Use Cases
   ● Components of the pipeline
   ● Bottlenecks & Challenges
   ● Our approach
   ○ Camera Agent
   ○ Cloud Architecture
   ○ Firefly
   ○ Dashboard
   ● Code Samples
   ● Ending notes
   

   View Slide

4. What are camera based applications?
   

   View Slide

5. What makes them intelligent?
   

   View Slide

6. Use Cases
   ● Surveillance
   ● Agricultural yield estimation
   ● Object classification
   ● Barcode recognition on conveyor
   

   View Slide

7. How would you build them?
   

   View Slide

8. Components of the pipeline
   ● Data Ingestion
   ● Data Storage
   ● Machine Learning
   ● Visualization
   

   View Slide

9. Bottlenecks and Challenges
   ● What tools to use?
   ● How to capture data?
   ● Where to store?
   ● Where should my ML code run?
   ● How should I visualize the output?
   

   View Slide

10. ● Takes 2-3 weeks for building an application
    ● Of that, only 2 days may be spent on actual machine learning
    ● Majority of the time is spent in software engineering rather than ML
    Typical Timeline
    

    View Slide

11. Time to Market!
    

    View Slide

12. Our Approach
    

    View Slide

13. Camera Agent
    ● Program running on end device
    ● Connects to pre-configured server
    ● Captures and uploads pictures/videos
    ● Remotely configurable by the server
    

    View Slide

14. Cloud Architecture
    ● Server to get data from the agents and store them
    ● Admin Dashboard to control the capture frequency, type of data captured,
    see logs and uploads from registered devices
    ● Storage provider can be changed
    ● Provides a client library for fetching data
    

    View Slide

15. Firefly
    ● Framework to deploy Python functions as REST web service
    ● Very easy and flexible to use
    ● Used to deploy ML models and pre-processing functions
    ● Easily deployable using WSGI servers like gunicorn
    ● Open Source !
    

    View Slide

16. Dashboard
    ● Variety of visualization options
    ● Data provided by the client library or a firefly function running Machine
    Learning Code
    ● Run anywhere
    ● Plotting and charting using Dash by plot.ly or any other visualization tool
    

    View Slide

17. Talk is cheap.
    Show me the code.
    - Linus Torvalds
    

    View Slide

18. View Slide

19. View Slide

20. View Slide

21. View Slide

22. View Slide

23. View Slide

24. Ending Notes
    ● End to end application development in 3-4 days
    ● Majority of time spent on ML
    ● Robust infrastructure
    ● Versatile data storage options
    ● Scalability
    

    View Slide

25. Links
    ● https://github.com/rorodata/firefly
    ● https://plot.ly/dash/
    

    View Slide

26. Thank You!
    Have a nice day.
    @nabar0n
    https://nabarun.in
    

    View Slide

27. Questions
    @nabar0n, //nabarun.in
    

    View Slide