path between two given endpoints given a pixelwise cost function Relevant cost : decreasing function of gradient magnitude short paths tend to follow edges 6 / 30
→ Rn + and two seeds ps, pe ∈ Ω located on the contour of an object Objective: ﬁnd a path γ : [0, 1] → Ω so that γ(0) = ps and γ(1) = pe that represents a piece of contour : L(γ) = 1 0 P(γ(s)) γ (s) ds. (1) P is a potential function derived from image f that should have low values for edges. L(γ) is smaller when it goes through low values of P → minimize L(γ) given a pair of starting and ending points. Objective : ﬁnd the geodesic path L(γ∗) = min γ∈A(ps ,pe ) L(γ) (2) 7 / 30
} : Ω → R+ from a point ps to any other point p ∈ Ω U{ps } (p) = d(ps, p) = min γ∈A(ps ,p) L(γ) (3) This can be done eﬃciently with the Fast Marching front-propagation algorithm (similar to Dijkstra’s shortest path algorithm on a graph) Second step : the minimal path between ps and pe can be constructed by applying a back-propagation procedure (gradient descent) starting from pe along the gradient of the map Ups until arriving at ps 8 / 30
does not take into account the surrounding regions apart from the delineated contour A contour is a curve Γ separating optimally two regions R+ and R− Each region has usually diﬀerent features from the other one Our proposal : to design new potentials based on patches as texture descriptors Patch at pi is P(pi ) = (f (pi + t), ∀t ∈ [−w, w]2)T ∈ R3(2w+1)2 14 / 30
as lying on an edge if the patch at its left is diﬀerent from the patch at its right. dc dc dl dr P1 (p) = max θ dθ,1 c (p) PAF 2 (p) = max θ AF(dθ,1 s (p)) AF ∈ {min, max, mean} dθ,1 s = {dθ,1 l , dθ,1 c , dθ,1 r } with patch diﬀerences: centered on p: dθ,δ c (p) = d P(pl,θ,δ), P(pr,θ,δ) upwind left: dθ,δ l (p) = d P(p), P(pl,θ,δ) upwind right: dθ,δ r (p) = d P(p), P(pr,θ,δ) with d(P(pi ), P(pj )) = P(pi ) − P(pj ) 2 and θ ∈ {0◦, 45◦, 90◦, 135◦} and δ the distance between the pixels along the line. 15 / 30
discriminate edge from non-edge pixels. extend P2 to a larger neighborhood obtained with δ > 1 based on a band of ns patches on each side of central pixel p avoid overlap with patch P(p) by considering pixels at a distance δ ∈ [ , + ns] PAF 3 (p) = max θ AF min δ∈[ , +ns] dθ,δ s (p), ∀s ∈ {l, c, r} dc dl dr 16 / 30
into account the previously positioned seeds given a seed ps , consider regions R+ θs (ps ) and R− θs (ps ) with angle: θs = arg min θ ∠(Ns , ps pl,θ,1 s ) normal to γs−1,s at ps : Ns = (γ∗ s−1,s (ps ))⊥/ γ∗ s−1,s (ps ) R+(p s ) R- (p s ps γ ) ⇒ should be similar to the other regions for the next points of the contour 17 / 30
consider regions R+ θs (ps ) and R− θs (ps ) ∀p ∈ Ω, compute all pairwise distances between R+ θs (ps ) and R+ θ (p), ∀θ P4(pi ) = min θ ( min q1×q2 q1∈R+ θ (p) q2∈R+ θs (ps ) d(P(q1), P(q2)) + min q1×q2 q1∈R− θ (p) q2∈R− θs (ps ) d(P(q1), P(q2))) with 5x5 patches P(·) contained within π/4 to the band deﬁned by θ to the horizon R+(ps ) R-(ps ) ps R+(p) R-(p) p ! 18 / 30
, no idea of the orientation ﬁnd the angle maximizing diﬀerences between patches in R+ θ (p1 ) and R− θ (p1 ) θ1 = arg max θ max q1×q2 q1∈R+ θ (p1) q2∈R− θ (p1) d(P(q1 ), P(q2 )) Combined patch-based potential P5 = αPmax 3 + (1 − α)P4 ⇒ note that this must be computed at each step of the extraction (since this depends on the last seed) 19 / 30
user behavior with an agent ﬁnd the longest possible path with minimum error from a seed ps the path must be contained in a tube of radius r on both sides of the ground truth realized by a dichotomous search along the ground truth object contour p s p e 2r ground truth ﬁnding longest path zoom 22 / 30
interactive image segmentation. CVPR 2010] What is the best patch size ? We considered w ∈ {1, 2, 3} corresponding to 3x3, 5x5 and 7x7 patches w = 1 w = 2 w = 3 Number of times a potential gives the minimum % of seeds ⇒ Best results are mostly always obtained with band-based potential Pmax 3 and w = 2 24 / 30
quantitative experiments were simulated with a user agent enables to improve classical gradient-based potential enables to reduce the user interaction (in terms of required points for object extraction) 29 / 30