SPIE EI 2015

Three-Dimensional Image Processing, Measurement (3DIPM), and Applications

Transcript

1. Mesh saliency with local adaptive patches Anass Nouri, Christophe Charrier, Olivier Lézoray
 
 University of Caen, France SPIE Electronics-Imaging 12/02/2015 1
2. Cliquez pour modifier le style du titre 10/07/2013 Plan • How to define Saliency ?
 • Modeling the 3D mesh surface
 • Adaptive local patches construction
 • Saliency computation
 • Results 
 • Comparisons with the state-of-the-art
 • Robustness and stability of the approach
 • Perspectives 2
3. Cliquez pour modifier le style du titre 10/07/2013 How to define saliency ? 3 How to define saliency ?
4. Cliquez pour modifier le style du titre 10/07/2013 Intuitive definition : An object is salient if it is easily noticeable. 4 How to define saliency ?
5. Cliquez pour modifier le style du titre 10/07/2013 There is 2 attentional mechanisms in the human vision: Bottom-up: Saliency here is involuntary and it is related to the properties of the stimuli 
 «The 3D object jumps out ».
 
 Top-down: Saliency depends on what the observer is looking for. 
 « The ease of finding a target object ». The aim: Determine saliency related to the Bottom-up process, called also Attentional saliency. 
 How to define saliency ? 5
6. Cliquez pour modifier le style du titre 10/07/2013 Sensibility of the human eyes to strong fluctuations and discontinuities. If a vertex from the mesh stands out strongly from the mesh, then, it could be
 considered as a salient 3D point. The basis of our approach ? Flat Surface without fluctuations Flat surface with strong fluctuations 6 How to define saliency ?
7. Cliquez pour modifier le style du titre 10/07/2013 Modeling the surface of a 3D mesh Modeling the surface of a 3D mesh 7
8. Cliquez pour modifier le style du titre 10/07/2013 Goal : Define into every vertex of the mesh a normal vector and a 2D tangent plane. ! To do so : ! 
 the set of the vertices
 the set of edges connecting a couple of vertices and 
 the weight of the edge . (vi, vj) 2 E G = (V, E, w) V E = V ⇥ V Let M a mesh represented by a non-oriented graph where:
 8 w(vi, vj) Modeling the surface of a 3D mesh
9. Cliquez pour modifier le style du titre 10/07/2013 3 eigen values 3 eigen vectors normal vector directional vector directional vector Computations : Center of gravity: cov ( vi) = X j2N(vi) ( vj ˆ vi)( vj ˆ vi)T 2 R3⇥3 Covariance matrix: ~ x ~ y Guide the normal vectors outward using the minimum spanning tree of the graph
 (MST). 9 Modeling the surface of a 3D mesh ˆ vi = 1 |N(vi)| X j2N(vi) vj with vj ⇠ vi
10. Cliquez pour modifier le style du titre 10/07/2013 Local adaptive patch construction Construction of the local descriptor : 
 The adaptive patches 10
11. Cliquez pour modifier le style du titre 10/07/2013 • Define a spherical neighborhood of the target vertex:
 
 S"(vi) = vj | ||~ vj ~ vi ||2 2  " • Projection of the neighboring vertices on the 2D tangent plane : ~ v 0 j = [( ~ vj ~ vi) · ~ x ( vi) , ( ~ vj ~ vi) · ~ y ( vi)]T ~ P(vi) • Adaptive size of the patch offers better consideration of the irregularities of the surface: Td( vi) = max ( ~ v0 j , ~ v0 k )2 ~ P (vi) ( || ~ v0d j ~ v0d k ||2 2) d : represents the x or y coordinate. 
 : the coordinate of the vector . : Euclidian norm. v0d j d ~ v0 j ||.||2 11 Local adaptive patch construction
12. Cliquez pour modifier le style du titre 10/07/2013 • Division of the patch into cells :
 l ⇥ l • Filling cells with the absolute value of the sum of the projections heights: H(~ v0 j ) = ||(~ vj ~ v0 j )||2 2 12 Local adaptive patch construction where is vector of the accumulated heights into the cells of the patch. H(~ v0 j ) index d = $ ~ v0d j Td( vi) /l % with vj ⇠ vi
13. Cliquez pour modifier le style du titre 10/07/2013 Illustration of a patch construction: 13 Local adaptive patch construction
14. Cliquez pour modifier le style du titre 10/07/2013 Computation of saliency Computation of saliency 14
15. Cliquez pour modifier le style du titre 10/07/2013 • Similarity computation: ( vi) = max vk ⇠vi ( ||~ vi ~ vk ||2) scale parameter: : the weight of the edge between and . w(vi, vj) vi vj • Visual saliency: : the cardinality of the neighboring. 15 Computation of saliency w( vi , vj) = exp " || ~ H ( vi) ~ H ( vj) ||2 2 ( vi) ⇤ ( vj) # with vj ⇠ vi Saliency(vi) = ✓ 1 |N(vi)| ◆ X vi ⇠vj w(vi, vj) |N(vi)|
16. Cliquez pour modifier le style du titre 10/07/2013 Results and comparison with the
 state-of-the-art Results and comparisons with the state-of-the-art 16
17. Cliquez pour modifier le style du titre 10/07/2013 Results and comparisons with the state-of-the-art 3D original mesh Gorilla: 17
18. Cliquez pour modifier le style du titre 10/07/2013 Visual saliency on the 3D 
 mesh Gorilla: 18 Results and comparisons with the state-of-the-art low saliency high saliency
19. Cliquez pour modifier le style du titre 10/07/2013 Zoom on the face of the Gorilla mesh: 19 Results and comparisons with the state-of-the-art
20. Cliquez pour modifier le style du titre 10/07/2013 Comparisons with the state-of-the-art Comparisons with the state-of-the-art 20
21. Cliquez pour modifier le style du titre 10/07/2013 Original 3D mesh [Tal et al.] [Song et al.] [Lee et al.] Our approach 21 Comparisons with the state-of-the-art
22. Cliquez pour modifier le style du titre 10/07/2013 Comparisons with the state-of-the-art (2) Original 3D mesh [Tal et al.] [Song et al.] Our approach 22
23. Cliquez pour modifier le style du titre 10/07/2013 Robustness and stability Robustness & stability 23
24. Cliquez pour modifier le style du titre 10/07/2013 Robustness and stability Independence of our approach to any pretreatment to define saliency: ! - Simplification ! - Smoothing ! Robustness to 2 types of distortions: ! - Noise (aleatory displacement of vertices). ! - Simplification with the algorithm of Garland 
 M. and Heckbert P.S . ! ! 24
25. Cliquez pour modifier le style du titre 10/07/2013 Robustesse & stabilité 3D original Centaur mesh Saliency on the 3D Centaur mesh with our approach 25 Robustness and stability
26. Cliquez pour modifier le style du titre 10/07/2013 3D mesh Centaur 
 noised 3D mesh Centaur 
 severely noised Saliency on the noised 3D mesh Centaur 
 with our approach Robustness to noise 26 Robustness and stability Saliency on the noised 3D mesh Centaur 
 with our approach
27. Cliquez pour modifier le style du titre 10/07/2013 3D mesh Centaur 
 simplified to
 68.63% 3D mesh Centaur 
 simplified to
 79.5% Saliency on the 3D mesh Centaur 
 with our approach Robustness to
 simplification 27 Robustness and stability Saliency on the 3D mesh Centaur 
 with our approach
28. Cliquez pour modifier le style du titre 10/07/2013 Perspectives • Improvement of the saliency map.
 • Add and investigate the multi-scale aspect.
 Implement saliency for some applications like: 3D objects Compression : Salient regions are less compressed. 
 (details preservation). Source coding and transmission: Allocate more bits for salient regions to preserve them 
 while the transmission process. Best viewpoints selection : Present the most attractive areas on the 3D objects. 28
29. Cliquez pour modifier le style du titre 10/07/2013 Thanks for your attention Mesh saliency with local adaptive patches 29