huge amounts of 3D data Even with cheap hardware and software, one can easily generate 3D data: 3D data acquisition with common smartphone (e.g., Photogrammetry) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 4 / 28
new application fields have appeared Digital Forensics, Cultural Heritage, Body Scanning One often needs to know which regions from a 3D mesh are important: adaptive simplification or decimation, viewpoint selection O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 5 / 28
an image are visually more noticeable by their contrast with respect to surrounding regions Saliency for 3D meshes ? If a point from the 3D data stands out strongly from its surrounding, then, it could be considered as a salient 3D point. Our approach We locally study the geometry of the mesh vertices’ normals A local normal-based descriptor is built for each vertex according to a spiral path within a k-hop neighborhood. Geometric and spectral saliencies are defined from this descriptor and combined with a roughness measure O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 6 / 28
by a graph G = (V,E) with V = {v1,...,vm} and E ⊂ V×V vi ∼ vj is used to denote two adjacent vertices N (vi) = {vj,vj ∼ vi} gives the set of all the adjacent vertices to vi within a 1-hop (vertices that can be reached in one walk). Lim et al. have proposed local spiral hop descriptors: the surrounding vertices of one vertex can be enumerated by following a spiral O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 9 / 28
ordered set of vertices whose shortest path to vi is exactly k-hops long ⇒ R0(vi) = {vi} and R1(vi) = N (vi) k-disk(vi) = l=0,...,k Rl(vi) is the set of vertices that can be reached from vi in 0 to k walks. vi R1 7 R1 1 R1 2 R1 3 R1 4 R1 5 R1 6 R2 1 R2 2 R2 3 R2 4 R2 5 R2 6 R2 7 R2 8 R2 9 R2 10 R2 11 R2 12 R2 13 1-disk(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 10 / 28
ordered set of vertices whose shortest path to vi is exactly k-hops long ⇒ R0(vi) = {vi} and R1(vi) = N (vi) k-disk(vi) = l=0,...,k Rl(vi) is the set of vertices that can be reached from vi in 0 to k walks. R(k+1)(vi) = N (R(k)(vi))\k-disk(vi) is the set of vertices that can be reached in 1 walk from Rk(vi) without going through its k-disk (that contains vertices that can be reached from vi in 0 to k walks) vi R1 7 R1 1 R1 2 R1 3 R1 4 R1 5 R1 6 R2 1 R2 2 R2 3 R2 4 R2 5 R2 6 R2 7 R2 8 R2 9 R2 10 R2 11 R2 12 R2 13 2-ring(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 10 / 28
Sp(vi,k) is an ordered sequence from the concatenation of the ordered rings Sp(vi,k) = (vi,1-ring(vi),...,k-ring(vi)) = R0 1 (vi),R1 1 (vi),R1 2 (vi),...,Rk |Rk| (vi) It has 2 degrees of freedom: the direction (clockwise or counterclockwise) of the rings and the first chosen vertex R1 1 (vi) We fix : the orientation clockwise the initial vertex R1 1 (vi) is the one in the direction of the shortest geodesic path to vi: R1 1 (vi) = arg min vj∈N (vi) dG(vi,vj) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 11 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
Sp(vi,k) varies for the vertices Lim et al. either truncate or zero-pad each spiral depending on its size to compare two local spiral hop descriptors We propose a more accurate hierarchical comparison d(Sp(vi,k),Sp(vj,k)) = k ∑ l=0 d(Rl(vi),Rl(vj)) Two k-rings are compared by mapping the vertices of the largest ring to the smallest one: d(Rl(vi),Rl(vj)) = |Rl (vi)| ∑ n=0 d(Rl n (vi),Rl n (vj)) with n = n·|Rl (vj)| |Rl (vi)| and |Rl(vi)| > |Rl(vj)| First vertex descriptor Second vertex descriptor 0-hop 1-hop 2-hop The distance between two vertices is then the distance between their graph signal vectors: d(Rl n (vi),Rl m (vj)) = F(Rl n (vi))−F(Rl m (vj)) 2 that are vertex normal vectors: F(vi) = n(vi) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 12 / 28
important role for saliency detection Lee et al. have proposed the first mesh saliency detection based on differences between Gaussian-weighted mean curvatures We consider the more robust notion of roughness (from Wang et al.) that we extend to large neighborhoods (by taking γ powers of the Laplacian) R(vi) = κ(vi)− ∑ vj∈γ-rings(vi) Lγ i j ·κ(vj) ∑ vj∈γ-rings(vi) Lγ i j with γ-rings(vi) = γ-disk(vi)\{vi} and κ(vi) the mean curvature. O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 15 / 28
gradient at a vertex vi as the nonlocal vector of all the distances between the spiral descriptors of vi and its neighbors within its γ-rings(vi): ∇f(vi) = [d(Sp(vi,k),Sp(vj,k)),vj ∈ γ-rings(vi)]T with γ-rings(vi) = γ-disk(vi)\{vi}. One needs to have γ > k: the support of the gradient is larger than the spiral descriptor support. O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 16 / 28
is the normalized L1 norm of the gradient: GS(vi) = 1 |γ-rings(vi)| ∇f(vi) 1 = 1 |γ-rings(vi)| ∑ vj∈γ-rings(vi) d(Sp(vi,k),Sp(vj,k)) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 17 / 28
J is the outer product of the gradient: J(vi) = ∇T f(vi)·∇f(vi). Its eigenvalues provide a discriminative description of the local geometry by summarizing the distribution of the gradients The spectral saliency is defined as the Structure Tensor Total Variation (STTV): SS(vi) = |γ-rings(vi)| ∑ j=1 λ2 j with λj the eigenvalues of J(vi). O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 18 / 28
measures: Roughness : based on based on differences of curvatures, it tends to be very sensitive to small changes. Geometric saliency: smooth and robust to small changes. Spectral saliency: very localized and robust to small changes. To merge these informations altogether into a single saliency measure, we weight the roughness by the sum of the normalized geometric and spectral saliencies: S(vi) = Kσ R(vi)· GS(vi)+SS(vi) with Kσ a Gaussian smoothing. O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 20 / 28
Saliency computation on 3-rings with 2-hop spiral descriptors Sp(vi,2). Warm colors mean high values of saliency. O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 23 / 28
Saliency computation on 3-rings with 2-hop spiral descriptors Sp(vi,2). Warm colors mean high values of saliency. O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 23 / 28
Saliency computation on 3-rings with 2-hop spiral descriptors Sp(vi,2). Warm colors mean high values of saliency. O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 23 / 28
Saliency computation on 3-rings with 2-hop spiral descriptors Sp(vi,2). Warm colors mean high values of saliency. O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 23 / 28
Chen et al., saliency detection methods of Sun et al., Song et al., Song et al., Ours (first scale), Ours (second scale), Ours (third scale), Ours (fusion of the scales by max) O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 24 / 28
Chen et al., saliency detection methods of Sun et al., Song et al., Song et al., Ours (first scale), Ours (second scale), Ours (third scale), Ours (fusion of O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 24 / 28
built for each vertex according to a spiral path within a k-hop neighborhood. a hierarchical comparison between two local spiral hop descriptors a nonlocal gradient for each vertex as the vector of the descriptors’ comparison geometric and spectral saliencies from the norm of the gradient and the structure tensor total variation a saliency measure combining both saliencies weighted by a roughness measure O. L´ ezoray & A. Nouri 3D Mesh Saliency from local spiral hop descriptors 27 / 28
lezoray
kitachan_black
rs125
maosaeki
itakigawa
clustervr
olmdrd
cybozuinsideout
hashi0203
hirosaji
ayakohasegawa
koheishinden
ufuk
philhawksworth
revolveconf
rmw
keithpitt
hannesfritz
jonrohan
deanohume
morganepeng
lara
scottboms
bkeepers
