A Background Eraser Tool in Python

Deepika A. talks about how to build a background eraser tool in Python.

Transcript

1. A Background Eraser Tool in Python Jan. 24th 2019 - PyLadies Toronto Presented by Deepika Anandakrishnan
2. Motivation for Today’s Talk : 
 https://www.remove.bg/ 1. How did Benjamin do this with Python & Deep Learning? 2. How would I re-create this?
3. Theoretical Pipeline Pre-trained Mask R-CNN + Python User Uploads 2 Images Python + numpy + cv2 Python API Output for User Download
4. Traditional Solution 
 – Using Photoshop’s Background Eraser Tool 1. Choose The Eraser Tool 2. Choose Brush and Pencil modes and then choose a brush pre-set, and set Opacity and Flow in the options bar 3. Carefully drag through the required areas, until your brain hurts 4. Position Foreground Image on another Background Image
5. Computer Vision Solution * https://engineering.matterport.com/splash-of-color-instance-segmentation-with-mask-r-cnn-and- tensorflow-7c761e238b46 Classification There’s generally an image with a single object as the focus and the task is to say what that image is Complex Classification Tasks •Multiple Overlapping Objects •Different Backgrounds •Boundary Identification •Relation Identification
6. Deep Learning Solution
 • 2014: R-CNN - UC Berkeley • 2015: Fast R-CNN - FAIR – Facebook AI Research • 2016: Faster R-CNN - Microsoft Research • 2017: Mask R-CNN - FAIR – Facebook AI Research
 *(https://arxiv.org/abs/1703.06870) Dataset Used : COCO + VOC Other Real-Time Object Detection Algorithms: Real-time Object Detection - SSD, YOLOv2, YOLOv3, FPN
7. Output from Mask R-CNN
 Dataset Used : COCO + VOC Input Image M a s k Prediction Predicted Mask Resized Mask Mask Overlaid on Image
8. Existing Applications of Mask R-CNN 1. remove.bg ?? 2. Detecting empty parking spots in a parking lot in real time and sending a message to your phone using Twilio
 *(https://medium.com/@ageitgey/snagging-parking-spaces-with-mask-r-cnn-and-python-955f2231c400) 3. Detecting Car Damage – Dents and Scratches
 *(https://www.analyticsvidhya.com/blog/2018/07/building-mask-r-cnn-model-detecting-damage-cars-python/)
 4. Detecting Road Damage Using Smartphones – Potholes etc.
 *(https://github.com/iFun0/Pavement-RCNN) 5. Post Disaster Building Damage Detection
 *(http://summit.sfu.ca/system/files/iritems1/16986/etd9978_KRashediNia.pdf)
9. Potentially Million Dollar Ideas Using Mask R-CNN 1. Train a Mask R-CNN model to highlight which exact areas of an MRI scan correlate to certain behavioral/psychological patterns 2. Integrate with Adobe Photoshop as a Single-click Feature in Adobe Photoshop 3. Smart Mirror Installed in Retail Outlets 4. Predicting Instantly How Spectacles Look On A Customer
10. 
 “All things are subject to interpretation. Whichever interpretation prevails at a given time is a function of power and not truth” -Friedrich Nietzsche