Building Camera based Intelligent Applications

Presented at PyData Delhi 2017

1. Building Camera based Intelligent Applications Nabarun Pal
2. • Final Year Undergrad @ IITR • Software Engineer Intern @ rorodata • Systems Engineer @ Team Robocon IITR • Love my kindle and hacking through things About Me nabarun.in github.com/palnabarun
3. Overview • Introduction • Use Cases • Components of the pipeline • Bottlenecks & Challenges • Our approach ◦ Camera Agent ◦ Cloud Architecture ◦ Firefly ◦ Dashboard • Code Samples • Ending notes
4. What are camera based applications?
5. What makes them intelligent?
6. Use Cases • Surveillance • Agricultural yield estimation • Object classification • Barcode recognition on conveyor
7. How would you build them?
8. Components of the pipeline • Data Ingestion • Data Storage • Machine Learning • Visualization
9. Bottlenecks and Challenges • What tools to use? • How to capture data? • Where to store? • Where should my ML code run? • How should I visualize the output?
10. • Takes 2-3 weeks for building an application • Of that, only 2 days may be spent on actual machine learning • Majority of the time is spent in software engineering rather than ML Typical Timeline
11. Time to Market!
12. Our Approach
13. Camera Agent • Program running on end device • Connects to pre-configured server • Captures and uploads pictures/videos • Remotely configurable by the server
14. Cloud Architecture • Server to get data from the agents and store them • Admin Dashboard to control the capture frequency, type of data captured, see logs and uploads from registered devices • Storage provider can be changed • Provides a client library for fetching data
15. Firefly • Framework to deploy Python functions as REST web service • Very easy and flexible to use • Used to deploy ML models and pre-processing functions • Easily deployable using WSGI servers like gunicorn • Open Source !
16. Dashboard • Variety of visualization options • Data provided by the client library or a firefly function running Machine Learning Code • Run anywhere • Plotting and charting using Dash by plot.ly or any other visualization tool
17. Talk is cheap. Show me the code. - Linus Torvalds
24. Ending Notes • End to end application development in 3-4 days • Majority of time spent on ML • Robust infrastructure • Versatile data storage options • Scalability
25. Links • https://github.com/rorodata/firefly • https://plot.ly/dash/
26. Thank You! Have a nice day. @nabar0n https://nabarun.in
27. Questions @nabar0n, //nabarun.in