Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Camera based Intelligent Applications

Nabarun Pal
September 02, 2017

Building Camera based Intelligent Applications

Presented at PyData Delhi 2017

Nabarun Pal

September 02, 2017
Tweet

More Decks by Nabarun Pal

Other Decks in Technology

Transcript

  1. Building Camera based
    Intelligent Applications
    Nabarun Pal

    View Slide

  2. ● Final Year Undergrad @ IITR
    ● Software Engineer Intern @ rorodata
    ● Systems Engineer @ Team Robocon IITR
    ● Love my kindle and hacking through things
    About Me
    nabarun.in
    github.com/palnabarun

    View Slide

  3. Overview
    ● Introduction
    ● Use Cases
    ● Components of the pipeline
    ● Bottlenecks & Challenges
    ● Our approach
    ○ Camera Agent
    ○ Cloud Architecture
    ○ Firefly
    ○ Dashboard
    ● Code Samples
    ● Ending notes

    View Slide

  4. What are camera based applications?

    View Slide

  5. What makes them intelligent?

    View Slide

  6. Use Cases
    ● Surveillance
    ● Agricultural yield estimation
    ● Object classification
    ● Barcode recognition on conveyor

    View Slide

  7. How would you build them?

    View Slide

  8. Components of the pipeline
    ● Data Ingestion
    ● Data Storage
    ● Machine Learning
    ● Visualization

    View Slide

  9. Bottlenecks and Challenges
    ● What tools to use?
    ● How to capture data?
    ● Where to store?
    ● Where should my ML code run?
    ● How should I visualize the output?

    View Slide

  10. ● Takes 2-3 weeks for building an application
    ● Of that, only 2 days may be spent on actual machine learning
    ● Majority of the time is spent in software engineering rather than ML
    Typical Timeline

    View Slide

  11. Time to Market!

    View Slide

  12. Our Approach

    View Slide

  13. Camera Agent
    ● Program running on end device
    ● Connects to pre-configured server
    ● Captures and uploads pictures/videos
    ● Remotely configurable by the server

    View Slide

  14. Cloud Architecture
    ● Server to get data from the agents and store them
    ● Admin Dashboard to control the capture frequency, type of data captured,
    see logs and uploads from registered devices
    ● Storage provider can be changed
    ● Provides a client library for fetching data

    View Slide

  15. Firefly
    ● Framework to deploy Python functions as REST web service
    ● Very easy and flexible to use
    ● Used to deploy ML models and pre-processing functions
    ● Easily deployable using WSGI servers like gunicorn
    ● Open Source !

    View Slide

  16. Dashboard
    ● Variety of visualization options
    ● Data provided by the client library or a firefly function running Machine
    Learning Code
    ● Run anywhere
    ● Plotting and charting using Dash by plot.ly or any other visualization tool

    View Slide

  17. Talk is cheap.
    Show me the code.
    - Linus Torvalds

    View Slide

  18. View Slide

  19. View Slide

  20. View Slide

  21. View Slide

  22. View Slide

  23. View Slide

  24. Ending Notes
    ● End to end application development in 3-4 days
    ● Majority of time spent on ML
    ● Robust infrastructure
    ● Versatile data storage options
    ● Scalability

    View Slide

  25. Links
    ● https://github.com/rorodata/firefly
    ● https://plot.ly/dash/

    View Slide

  26. Thank You!
    Have a nice day.
    @nabar0n
    https://nabarun.in

    View Slide

  27. Questions
    @nabar0n, //nabarun.in

    View Slide