Share
Embed
SPIE EI 2015
Three-Dimensional Image Processing, Measurement (3DIPM), and Applications
Olivier Lézoray
February 01, 2015
Transcript
29 slides
-
Mesh saliency with local adaptive patches Anass Nouri, Christophe Charrier,
Olivier Lézoray University of Caen, France SPIE Electronics-Imaging 12/02/2015 1 -
Cliquez pour modifier le style du titre 10/07/2013 Plan •
How to define Saliency ? • Modeling the 3D mesh surface • Adaptive local patches construction • Saliency computation • Results • Comparisons with the state-of-the-art • Robustness and stability of the approach • Perspectives 2 -
Cliquez pour modifier le style du titre 10/07/2013 How to
define saliency ? 3 How to define saliency ? -
Cliquez pour modifier le style du titre 10/07/2013 Intuitive definition
: An object is salient if it is easily noticeable. 4 How to define saliency ? -
Cliquez pour modifier le style du titre 10/07/2013 There is
2 attentional mechanisms in the human vision: Bottom-up: Saliency here is involuntary and it is related to the properties of the stimuli «The 3D object jumps out ». Top-down: Saliency depends on what the observer is looking for. « The ease of finding a target object ». The aim: Determine saliency related to the Bottom-up process, called also Attentional saliency. How to define saliency ? 5 -
Cliquez pour modifier le style du titre 10/07/2013 Sensibility of
the human eyes to strong fluctuations and discontinuities. If a vertex from the mesh stands out strongly from the mesh, then, it could be considered as a salient 3D point. The basis of our approach ? Flat Surface without fluctuations Flat surface with strong fluctuations 6 How to define saliency ? -
Cliquez pour modifier le style du titre 10/07/2013 Modeling the
surface of a 3D mesh Modeling the surface of a 3D mesh 7 -
Cliquez pour modifier le style du titre 10/07/2013 Goal :
Define into every vertex of the mesh a normal vector and a 2D tangent plane. ! To do so : ! the set of the vertices the set of edges connecting a couple of vertices and the weight of the edge . (vi, vj) 2 E G = (V, E, w) V E = V ⇥ V Let M a mesh represented by a non-oriented graph where: 8 w(vi, vj) Modeling the surface of a 3D mesh -
Cliquez pour modifier le style du titre 10/07/2013 3 eigen
values 3 eigen vectors normal vector directional vector directional vector Computations : Center of gravity: cov ( vi) = X j2N(vi) ( vj ˆ vi)( vj ˆ vi)T 2 R3⇥3 Covariance matrix: ~ x ~ y Guide the normal vectors outward using the minimum spanning tree of the graph (MST). 9 Modeling the surface of a 3D mesh ˆ vi = 1 |N(vi)| X j2N(vi) vj with vj ⇠ vi -
Cliquez pour modifier le style du titre 10/07/2013 Local adaptive
patch construction Construction of the local descriptor : The adaptive patches 10 -
Cliquez pour modifier le style du titre 10/07/2013 • Define
a spherical neighborhood of the target vertex: S"(vi) = vj | ||~ vj ~ vi ||2 2 " • Projection of the neighboring vertices on the 2D tangent plane : ~ v 0 j = [( ~ vj ~ vi) · ~ x ( vi) , ( ~ vj ~ vi) · ~ y ( vi)]T ~ P(vi) • Adaptive size of the patch offers better consideration of the irregularities of the surface: Td( vi) = max ( ~ v0 j , ~ v0 k )2 ~ P (vi) ( || ~ v0d j ~ v0d k ||2 2) d : represents the x or y coordinate. : the coordinate of the vector . : Euclidian norm. v0d j d ~ v0 j ||.||2 11 Local adaptive patch construction -
Cliquez pour modifier le style du titre 10/07/2013 • Division
of the patch into cells : l ⇥ l • Filling cells with the absolute value of the sum of the projections heights: H(~ v0 j ) = ||(~ vj ~ v0 j )||2 2 12 Local adaptive patch construction where is vector of the accumulated heights into the cells of the patch. H(~ v0 j ) index d = $ ~ v0d j Td( vi) /l % with vj ⇠ vi -
Cliquez pour modifier le style du titre 10/07/2013 Illustration of
a patch construction: 13 Local adaptive patch construction -
Cliquez pour modifier le style du titre 10/07/2013 Computation of
saliency Computation of saliency 14 -
Cliquez pour modifier le style du titre 10/07/2013 • Similarity
computation: ( vi) = max vk ⇠vi ( ||~ vi ~ vk ||2) scale parameter: : the weight of the edge between and . w(vi, vj) vi vj • Visual saliency: : the cardinality of the neighboring. 15 Computation of saliency w( vi , vj) = exp " || ~ H ( vi) ~ H ( vj) ||2 2 ( vi) ⇤ ( vj) # with vj ⇠ vi Saliency(vi) = ✓ 1 |N(vi)| ◆ X vi ⇠vj w(vi, vj) |N(vi)| -
Cliquez pour modifier le style du titre 10/07/2013 Results and
comparison with the state-of-the-art Results and comparisons with the state-of-the-art 16 -
Cliquez pour modifier le style du titre 10/07/2013 Results and
comparisons with the state-of-the-art 3D original mesh Gorilla: 17 -
Cliquez pour modifier le style du titre 10/07/2013 Visual saliency
on the 3D mesh Gorilla: 18 Results and comparisons with the state-of-the-art low saliency high saliency -
Cliquez pour modifier le style du titre 10/07/2013 Zoom on
the face of the Gorilla mesh: 19 Results and comparisons with the state-of-the-art -
Cliquez pour modifier le style du titre 10/07/2013 Comparisons with
the state-of-the-art Comparisons with the state-of-the-art 20 -
Cliquez pour modifier le style du titre 10/07/2013 Original 3D
mesh [Tal et al.] [Song et al.] [Lee et al.] Our approach 21 Comparisons with the state-of-the-art -
Cliquez pour modifier le style du titre 10/07/2013 Comparisons with
the state-of-the-art (2) Original 3D mesh [Tal et al.] [Song et al.] Our approach 22 -
Cliquez pour modifier le style du titre 10/07/2013 Robustness and
stability Robustness & stability 23 -
Cliquez pour modifier le style du titre 10/07/2013 Robustness and
stability Independence of our approach to any pretreatment to define saliency: ! - Simplification ! - Smoothing ! Robustness to 2 types of distortions: ! - Noise (aleatory displacement of vertices). ! - Simplification with the algorithm of Garland M. and Heckbert P.S . ! ! 24 -
Cliquez pour modifier le style du titre 10/07/2013 Robustesse &
stabilité 3D original Centaur mesh Saliency on the 3D Centaur mesh with our approach 25 Robustness and stability -
Cliquez pour modifier le style du titre 10/07/2013 3D mesh
Centaur noised 3D mesh Centaur severely noised Saliency on the noised 3D mesh Centaur with our approach Robustness to noise 26 Robustness and stability Saliency on the noised 3D mesh Centaur with our approach -
Cliquez pour modifier le style du titre 10/07/2013 3D mesh
Centaur simplified to 68.63% 3D mesh Centaur simplified to 79.5% Saliency on the 3D mesh Centaur with our approach Robustness to simplification 27 Robustness and stability Saliency on the 3D mesh Centaur with our approach -
Cliquez pour modifier le style du titre 10/07/2013 Perspectives •
Improvement of the saliency map. • Add and investigate the multi-scale aspect. Implement saliency for some applications like: 3D objects Compression : Salient regions are less compressed. (details preservation). Source coding and transmission: Allocate more bits for salient regions to preserve them while the transmission process. Best viewpoints selection : Present the most attractive areas on the 3D objects. 28 -
Cliquez pour modifier le style du titre 10/07/2013 Thanks for
your attention Mesh saliency with local adaptive patches 29