Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Camera based Intelligent Applications

Nabarun Pal
September 02, 2017

Building Camera based Intelligent Applications

Presented at PyData Delhi 2017

Nabarun Pal

September 02, 2017
Tweet

More Decks by Nabarun Pal

Other Decks in Technology

Transcript

  1. Building Camera based
    Intelligent Applications
    Nabarun Pal

    View full-size slide

  2. ● Final Year Undergrad @ IITR
    ● Software Engineer Intern @ rorodata
    ● Systems Engineer @ Team Robocon IITR
    ● Love my kindle and hacking through things
    About Me
    nabarun.in
    github.com/palnabarun

    View full-size slide

  3. Overview
    ● Introduction
    ● Use Cases
    ● Components of the pipeline
    ● Bottlenecks & Challenges
    ● Our approach
    ○ Camera Agent
    ○ Cloud Architecture
    ○ Firefly
    ○ Dashboard
    ● Code Samples
    ● Ending notes

    View full-size slide

  4. What are camera based applications?

    View full-size slide

  5. What makes them intelligent?

    View full-size slide

  6. Use Cases
    ● Surveillance
    ● Agricultural yield estimation
    ● Object classification
    ● Barcode recognition on conveyor

    View full-size slide

  7. How would you build them?

    View full-size slide

  8. Components of the pipeline
    ● Data Ingestion
    ● Data Storage
    ● Machine Learning
    ● Visualization

    View full-size slide

  9. Bottlenecks and Challenges
    ● What tools to use?
    ● How to capture data?
    ● Where to store?
    ● Where should my ML code run?
    ● How should I visualize the output?

    View full-size slide

  10. ● Takes 2-3 weeks for building an application
    ● Of that, only 2 days may be spent on actual machine learning
    ● Majority of the time is spent in software engineering rather than ML
    Typical Timeline

    View full-size slide

  11. Time to Market!

    View full-size slide

  12. Our Approach

    View full-size slide

  13. Camera Agent
    ● Program running on end device
    ● Connects to pre-configured server
    ● Captures and uploads pictures/videos
    ● Remotely configurable by the server

    View full-size slide

  14. Cloud Architecture
    ● Server to get data from the agents and store them
    ● Admin Dashboard to control the capture frequency, type of data captured,
    see logs and uploads from registered devices
    ● Storage provider can be changed
    ● Provides a client library for fetching data

    View full-size slide

  15. Firefly
    ● Framework to deploy Python functions as REST web service
    ● Very easy and flexible to use
    ● Used to deploy ML models and pre-processing functions
    ● Easily deployable using WSGI servers like gunicorn
    ● Open Source !

    View full-size slide

  16. Dashboard
    ● Variety of visualization options
    ● Data provided by the client library or a firefly function running Machine
    Learning Code
    ● Run anywhere
    ● Plotting and charting using Dash by plot.ly or any other visualization tool

    View full-size slide

  17. Talk is cheap.
    Show me the code.
    - Linus Torvalds

    View full-size slide

  18. Ending Notes
    ● End to end application development in 3-4 days
    ● Majority of time spent on ML
    ● Robust infrastructure
    ● Versatile data storage options
    ● Scalability

    View full-size slide

  19. Links
    ● https://github.com/rorodata/firefly
    ● https://plot.ly/dash/

    View full-size slide

  20. Thank You!
    Have a nice day.
    @nabar0n
    https://nabarun.in

    View full-size slide

  21. Questions
    @nabar0n, //nabarun.in

    View full-size slide