Muse.ai demo

Transcript

1. MUSE.AI Detect and tag content in video frames Carlos Faham Insight Data Science Fellow
2. Problem • Identify scenes, people and text in a video to unlock insights • Focusing on TED talks ◦ Person identification and classification ◦ Slide extraction ◦ Scene detection and segmentation bootstraps the identification and classification steps.
3. Local Binary Pattern Histogram Face classifier Haar Cascade Filter Frontal face detection Video reel Frame Requires frontal face Low detection rate to avoid false-positives Algorithm
4. Local Binary Pattern Histogram Face classifier Haar Cascade Filter Frontal face detection Video reel Frame Requires frontal face Low detection rate to avoid false-positives Scene segmentation Hue histograms 4-dim PCA DBSCAN
5. Scene type clustering
7. Data Story / Validation Working on validation: • Can compare manually-labeled scene segmentation and quote error rate • Manually measure on-screen time, compare to the algorithm's measurement