Max-Planck-Institut für Informatik Invited Talk

Max-Planck-Institut für Informatik Invited Talk

1507f22ca8d84c83e52362f69e428698?s=128

Ozan Sener

March 01, 2013
Tweet

Transcript

  1. Dynamic Graph-Cut's for Efficient Mobile Image and Video Segmentation Ozan

    Sener Middle East Technical University
  2. Interaction Segmentation Mask Application Interactive Image and Video Segmentation Segmentation

    Mask for Rest of the Video Application
  3. [Bounding Box + Color GMM + Min-Cut/Max-Flow] = [Rother, SIGGRAPH

    05] [Approx. Bound. + Color and Spatial Distance +Dynamic Alg.] = [Li Y, SIGGRAPH 95] [Scribble + Color Histogram + Min-Cut/Max-Flow] = [Boykov Y, IJCV 06] Color GMM Path (Boundary) Cost Dynamic Algorithm Min-Cut / Max-Flow Interaction Model Defintion Minimization
  4. Mobile Touch Screen Devices ? User Centered Interaction More Interaction

    Errors Low Computational Power
  5. Coloring Gesture of Coloring a Color Book

  6. Some Details of Method Color GMMs are used as models

    Iterative EM is used [Rother, SIGGRAPH 05] Image is initially over-segmented for efficiency via SLIC algorithm [Achanta, PAMI 2012] Min-cut/Max-flow is used for energy minimization [Boykov, PAMI 2004]
  7. Min-Cut / Max-Flow Review Graph Structure Augmentin Path Algorithm

  8. Dynamic ? If the graph structure is not changing, previous

    flows can be reused in the minimization [Kohli, PAMI 2007] Throughout the interaction graph structure does not change at all, but min-cut/max-flow is solved many times. Only problem which can arise is the edge weights and it can be solved via additional flow.
  9. Temporally + Spatially Dynamic Can we extend this concept to

    spatial dimension ? At any stage only part of the whole graph containing foreground object is need to be solved. But, what is the size of this sub-graph ? If the external flow which can flow through edges of the subgraph can not change the solution, there is no need to enlarge it anymore ?. However, this is hard to achieve. Clustering supplied by GMM is generally confident; however, labeling can be wrong.
  10. Temporally + Spatially Dynamic (cont'd) Our Claim: if the labels

    of the GMM clusters can not be changed via external flows, there is no need to enlarge the subgraph. Algorithm: Start with the bounding box of the interaction and enlarges it until is satisfied for all GMM clusers.
  11. (a): Blue rectangle is bounding box of the current interaction

    Red rectangle is the computed bounding box (b): Result of Min-Cut/Max-Flow for blue rectangle in (a) (c): Result of Min-Cut/Max-Flow for red rectangle in (a) Dynamic Graph-Cut in Action
  12. Computation Time

  13. Segmentation Quality

  14. Redefinition of Video Segmentation Assume MRF energy for the initial

    frame is known. MRF energy of any other frame is linearly dependent on previous frame. (All superpixels are model assumption)
  15. Biexponential Filters Spatio-temporal distance metric should be used for robust

    video segmentation. Geodesic distance is a best candidate with high computational complexity -O(n^3)-. Bi-exponential smoother is used for high performance approximation -O(n)- [Unser M,TIP 2011]
  16. Sample Propagation

  17. Dynamic Graph-Cut for Linear Filtering

  18. Segmentation Quality

  19. Computation Time Improvement via Bilateral Graph-Cut

  20. Time vs Performance On Segtrack [Tsai BMVC 2010] dataset:

  21. Thank you for your attention.

  22. BACKUP SLIDES

  23. None
  24. Overall Picture

  25. Error-Tolerance Energy minimization can tolerate some level of error if

    hard labels are replaced with soft labels. Question: Hard Labels vs Error Tolerance Solution: Solve errors before they occur.
  26. Error-Tolerance Algorithm New Superpixel Idea is keeping a single RGB

    gaussian model for the color model of the currently interacted region. If new superpixel is not confirming the color model, wait for it to come back or accept the new region E ? GMM P ? Back To Previous GMM w/o Error GMM C ? Insert to Current GMM Discard P Create GMM w/ Error T F T T F F GMM C ? Discard P Create GMM w/o Error Find Path T F Find path means minimize
  27. Notes: False Positives are handled via path finding. False Negatives

    requires a restart. Single Color True Positive Multi Color False Positive Multi Color True Positive Error Tolerance in Action
  28. Error-Tolerance

  29. Interaction Quality Performance Easiness Entertainment Overall Proposed Method 5:1:.45 4:0:.86

    5:1:.74 4:1:.45 GrabCut 3:2:.92 4:1:.75 2:1:.61 3:1:.75 Intelligent Scissors 3:1:.51 2:1:.74 3:2:.89 2:1:.76 15 Subjects (Undergraduate Level Engineering Students) 4 Random images out of 10 images Grading in the level of 1-5 for 4 different metrics Results in the format of Median:IQR:STD P-Values (via dependent ANOVA test): 0.0005