ICIP 2016

Transcript

1. Full-reference Saliency Based 3D Mesh Quality Index Anass Nouri, Christophe Charrier, Olivier Lézoray
 Normandie University, Caen, France
 1 ICIP - 2016
2. Cliquez pour modifier le style du titre 10/07/2013 Outline Outline: 1) Introduction 
 2) Quality assessment and visual saliency 
 3) Saliency comparison functions 4) Roughness comparison function 
 5) Quality Score computation 
 6) Performance in term of correlation & comparisons 
 7) Perspectives 2
3. Cliquez pour modifier le style du titre 10/07/2013 3 3D Mesh Quality Assessment Introduction 3D Mesh Quality Assessment
4. Cliquez pour modifier le style du titre 10/07/2013 4 3D Mesh Quality Assessment Full reference context: Think more about Fidelity than Quality
5. Cliquez pour modifier le style du titre 10/07/2013 5 Perceptual comparison is necessary when: ▪ Target is a human viewer ▪ Need to decide which algorithm to use ▪ Need to configure an algorithm to provide optimal results Compare results of ▪ Mesh watermarking ▪ Mesh compression ▪ Mesh filtering ▪ … 3D Mesh Quality Assessment Perceptual metrics aim at predicting the visual quality of a 3D mesh
6. Cliquez pour modifier le style du titre 10/07/2013 6 Geometric distance like Maximum root mean square error (MRMS) Original 3D mesh 
 (non distorted) Watermarked [Wang et al.]
 MRMS=0.00152 Watermarked [Cho et al.]
 MRMS=0.00152 NOT CORRELATED to human vision Goal : Design perceptual metrics that are correlated to human perception for the quality assessment of 3D meshes 3D Mesh Quality Assessment
7. Cliquez pour modifier le style du titre 10/07/2013 7 Quality Assessment & Visual saliency Quality Assessment & Visual saliency
8. Cliquez pour modifier le style du titre 10/07/2013 8 Visual Saliency: captures perceptually important regions where the human 
 visual attention is focalized Distorsion on perceptually important 
 regions (salient regions) Global quality is affected Distorsion on less perceptually 
 important regions (less salient regions) Global quality is less
 affected Quality Assessment & Visual saliency We observe that: and a distorsion is much perceived when it is located on salient areas [F. Boulos et al.]
9. Cliquez pour modifier le style du titre 10/07/2013 9 This is true for 2D images: 3D Mesh Quality Assessment distorsion on 
 salient areas distorsion on 
 less salient areas
10. Cliquez pour modifier le style du titre 10/07/2013 10 This is true for 3D meshes: Distorsion on 
 salient areas Distorsion on 
 less salient areas 3D Mesh Quality Assessment Original 3D mesh Multi-scale saliency map
 [A.Nouri et al. - 2015] Saliency
11. Cliquez pour modifier le style du titre 10/07/2013 11 Overview of our 
 FULL-REFERENCE Saliency-based 3D Mesh Quality Assessment Index
 (SMQI) 3D Mesh Quality Assessment : Overview of our approach
12. Cliquez pour modifier le style du titre 10/07/2013 12 3D Mesh Quality Assessment : Overview of our approach Overview:
13. Cliquez pour modifier le style du titre 10/07/2013 For a local neighborhood N( ) of the vertex , we define: 13 vi -the local mean on saliency: µN (vi) = 1 N(vi) X vj 2N(vi) MS(vj) -the local standard-deviation 
 on saliency: N (vi) = v u u t 1 N(vi) X vj 2N(vi) (MS(vj) µN( vi) )2 -the local covariance on saliency: N1(vi)N2(vi) = 1 |N1(vi)| X vj 2N1(vi),N2(vi) (MSM1 (vj) µN1(vi) )(MSM2 (vj) µN2(vi) ) vi 3D Mesh Quality Assessment : statistics computation with MS : Multi-scale saliency map Local statistics :
14. Cliquez pour modifier le style du titre 10/07/2013 14 3 comparison functions to quantify the deformation of the structural informations: 1) Saliency comparison: 2) Contrast comparison: 3) Structure comparison: L ( N1( vi) , N2( vi)) = || µN1(vi) µN2(vi) ||2 max ( µN1(vi), µN2(vi) ) C ( N1( vi) , N2( vi)) = || N1(vi) N2(vi) ||2 max ( N1(vi), N2(vi) ) S(N1(vi), N2(vi)) = || N1(vi) N2(vi) N1(vi)N2(vi) ||2 N1(vi) N2(vi) 3D Mesh Quality Assessment : Overview of our approach
15. Cliquez pour modifier le style du titre 10/07/2013 15 Visual Masking effect 3D Mesh Quality Assessment : visual masking effect
16. Cliquez pour modifier le style du titre 10/07/2013 16 Visual Masking defines the reduction in the visibility of one stimulus due to the simultaneous presence of another Geometric noise addition Geometric noise addition 3D Mesh Quality Assessment : visual masking effect Smooth surface Rough surface
17. Cliquez pour modifier le style du titre 10/07/2013 17 Distortions on rough regions are less visible than on smooth ones. Distortion on smooth regions Distortion on rough regions Original 3D mesh 3D Mesh Quality Assessment : visual masking effect
18. Cliquez pour modifier le style du titre 10/07/2013 18 To capture the visual masking effect, we generate a RoughnessMap original distorted R ( N1( vi) , N2( vi)) = || N1(vi) N2(vi) ||2 max( N1(vi) , N2(vi)) N1(vi) = 1 | N1( vi)| X vj 2N1(vi) RoughnessMap ( vj) 3D Mesh Quality Assessment : visual masking effect Roughness map Roughness map Roughness 4th comparison function :
19. Cliquez pour modifier le style du titre 10/07/2013 19 We combine the 4 features to provide a quality score: SMQI(M1, M2) = 0 @ 1 |V | |V | X L(N1(vi), N2(vi)) 1 A ↵ + 0 @ 1 |V | |V | X C(N1(vi), N2(vi)) 1 A + 0 @ 1 |V | |V | X S(N1(vi), N2(vi)) 1 A + 0 @ 1 |V | |V | X R(N1(vi), N2(vi)) 1 A parameters are estimated from an optimization based on genetic algorithms 3D Mesh Quality Assessment : visual masking effect
20. Cliquez pour modifier le style du titre 10/07/2013 20 Correlation with subjective scores of quality & Datasets Correlation with subjective scores & datasets
21. Cliquez pour modifier le style du titre 10/07/2013 21 Correlation with subjective scores & datasets Datasets used with subjective score of quality - Mean Opinion Score - Liris General Purpose Database: (4 reference 3D meshes)
 - Total of 3D meshes: 66 - Distorsion types: Noise addition and Smoothing - Strength: 3 - Localisation: uniformly, smooth regions, rough regions, transitional 
 regions (rough and smooth areas). - Total of human observers: 12. Liris Masking Database: (4 reference 3D meshes) - Total of 3D meshes: 24 - Distorsion types: Noise addition - Strength: 3 - Localisation: smooth regions, rough regions. - Total of human observers: 12.
22. Cliquez pour modifier le style du titre 10/07/2013 22 Performance: Spearman Correlation Correlation with subjective scores & datasets SROOC: Spearman Rank Ordered cOrrelation Coefficient
23. Cliquez pour modifier le style du titre 10/07/2013 23 Perspectives • Extension to the quality assessment of 3D meshes of different connectivities. • Amelioration of the multi-scale saliency map to better reflect the distortions affecting the considered 3D mesh. • Design of subject-rated database accounting for distorsions targeting visual salient regions and non salient ones. (We have observed that state-of-the-art’s metrics scores these kinds of distorsions in an incoherent manner).
24. Cliquez pour modifier le style du titre 10/07/2013 24 Work ongoing Thank you For more informations, please visit my web-page: http://nouri.users.greyc.fr