Tweet
Tweet

Slide 1

Slide 1 text

Building Camera based Intelligent Applications Nabarun Pal

Slide 2

Slide 2 text

● Final Year Undergrad @ IITR ● Software Engineer Intern @ rorodata ● Systems Engineer @ Team Robocon IITR ● Love my kindle and hacking through things About Me nabarun.in github.com/palnabarun

Slide 3

Slide 3 text

Overview ● Introduction ● Use Cases ● Components of the pipeline ● Bottlenecks & Challenges ● Our approach ○ Camera Agent ○ Cloud Architecture ○ Firefly ○ Dashboard ● Code Samples ● Ending notes

Slide 4

Slide 4 text

What are camera based applications?

Slide 5

Slide 5 text

What makes them intelligent?

Slide 6

Slide 6 text

Use Cases ● Surveillance ● Agricultural yield estimation ● Object classification ● Barcode recognition on conveyor

Slide 7

Slide 7 text

How would you build them?

Slide 8

Slide 8 text

Components of the pipeline ● Data Ingestion ● Data Storage ● Machine Learning ● Visualization

Slide 9

Slide 9 text

Bottlenecks and Challenges ● What tools to use? ● How to capture data? ● Where to store? ● Where should my ML code run? ● How should I visualize the output?

Slide 10

Slide 10 text

● Takes 2-3 weeks for building an application ● Of that, only 2 days may be spent on actual machine learning ● Majority of the time is spent in software engineering rather than ML Typical Timeline

Slide 11

Slide 11 text

Time to Market!

Slide 12

Slide 12 text

Our Approach

Slide 13

Slide 13 text

Camera Agent ● Program running on end device ● Connects to pre-configured server ● Captures and uploads pictures/videos ● Remotely configurable by the server

Slide 14

Slide 14 text

Cloud Architecture ● Server to get data from the agents and store them ● Admin Dashboard to control the capture frequency, type of data captured, see logs and uploads from registered devices ● Storage provider can be changed ● Provides a client library for fetching data

Slide 15

Slide 15 text

Firefly ● Framework to deploy Python functions as REST web service ● Very easy and flexible to use ● Used to deploy ML models and pre-processing functions ● Easily deployable using WSGI servers like gunicorn ● Open Source !

Slide 16

Slide 16 text

Dashboard ● Variety of visualization options ● Data provided by the client library or a firefly function running Machine Learning Code ● Run anywhere ● Plotting and charting using Dash by plot.ly or any other visualization tool

Slide 17

Slide 17 text

Talk is cheap. Show me the code. - Linus Torvalds

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

Ending Notes ● End to end application development in 3-4 days ● Majority of time spent on ML ● Robust infrastructure ● Versatile data storage options ● Scalability

Slide 25

Slide 25 text

Links ● https://github.com/rorodata/firefly ● https://plot.ly/dash/

Slide 26

Slide 26 text

Thank You! Have a nice day. @nabar0n https://nabarun.in

Slide 27

Slide 27 text

Questions @nabar0n, //nabarun.in