ISPA 2023

1
Zaineb Ibork 13th Int'l Symposium on Image and Signal Processing and Analysis ISPA 2023
No Reference 3D mesh quality assessment
using deep convolutional features
Zaineb Ibork(1,2), Anass Nouri(1), Olivier Lezoray(2), Christophe Charrier(2), Raja Touahni(1)
(1)SETIME Laboratory, Information Processing and A.I Team, Faculty of Sciences, Ibn Tofail University,
Kenitra, Morocco
(2)Normandie Univ, UNICAEN, ENSICAEN, CNRS, GREYC, 14000 Caen, France
{zaineb.ibork,anass.nouri, touahni.raja}@uit.ac.ma {olivier.lezoray, christophe.charrier}@unicaen.fr
le 18/09/2023

View Slide

2
Zaineb Ibork 13th Int'l Symposium on Image and Signal Processing and Analysis ISPA 2023 2
Zaineb Ibork
Plan Introduction and objectives
01
Methodology and Database
construction
02
Experimentations and results
04
Proposed method
03
Conclusion
05

View Slide

3
Problematic
➢ 3D Meshes can be altered by pre-processing steps as:
acquisition, compression or denoising
➢ In this context, visual quality assessment algorithms can be used to quantify
the amount of distortions that affect a 3D mesh and hence degrade its visual
rendering.
➢ Objective visual quality assessment methods can be categorized based on the
availability of a reference object:
Full-Reference(FR), No-Reference(NR) and Reduced-Reference (RR) methods.
➢ We introduce a no-reference mesh quality assessment index based on deep
convolutional features named:
DCFQI (Deep Convolutional Features Quality Index)
Objective
➢ Assess the visual quality of a 3D mesh without referring to its reference
version.

View Slide

4
Zaineb Ibork
01
construction
02
04
Proposed method
03
Conclusion
05

View Slide

5
Methodology
➢ We utilized a pre-trained CNN-based model with thirteen layers for
feature extraction and an MLP with two layers for regression, considering
2D representations of a mesh as input and generating a subjective quality
score as the output.
➢ After training, the network became capable of predicting the quality of a
mesh that was not part of the training phase.
➢ We constructed a base model using 'leave-one-model-out
cross-validation'.
➢ Subsequently, we built another model using the same method, but it was
initialized with weights from a model trained cumulatively on the
database.

View Slide

6
Databases construction: 3D mesh rendering
Armadillo’s 11 rendered views: the views in the first row are obtained by fixing θe = 0 and graduating θa by 60 degrees at
a time. We switch this process in the second row

View Slide

7
Database construction: Transformation Process from View to Patches

View Slide

8
Zaineb Ibork
01
construction
02
04
Proposed method
03
Conclusion
05

View Slide

9
The pipeline of the proposed quality assessment index that estimate
the quality of a rendered image (2D view or 2D view patch)

View Slide

10
Zaineb Ibork
01
construction
02
04
Proposed method
03
Conclusion
05

View Slide

11
Description
➢ 88 models between 40K and 50K vertices generated from
4 reference objects.
➢ Two types of distortion: noise addition and smoothing
➢ Distortions were applied with different strengths and at
four locations: on the whole model, on smooth areas, on
rough areas and on intermediate areas.
➢ Subjective evaluations were made at normal viewing
distance, using a SSIS (Single Stimulus Impairment Scale)
method with 12 observers.
➢ Each model is associated to a Mean Opinion Score after
normalization and outlier removal.
Mesh database: LIRIS/EPFL 3D Model General-Purpose
Armadillo Dyno
Venus RockerArm

View Slide

12
Mesh database: LIRIS/EPFL 3D Model General-Purpose
Smoothed mesh
Taubin15
Noised mesh
noise0015
Original mesh

View Slide

13
Resulting neural networks issued from the leave-one-mesh-out
cross-validation (LOMO-CV)

View Slide

14
Base Model
SROOC VALUES FOR THE BASE MODEL TRAINED FOR 20 EPOCHS ON Bview OR Bpatch DATABASES

View Slide

15
Base Model
SROOC VALUES FOR THE BASE MODEL TRAINED WITH AN EARLY STOPPING ON Bview AND Bpatch DATABASES

View Slide

16
Cumulative Model Training

View Slide

17
Cumulative Model results
SROOC AND PCC CUMULATIVE MODEL (CM) VS RE-TRAINED CUMULATIVE MODEL (RCM) RESULTS
TRAINED ON DATABASES BVIEW OR BPATCH

View Slide

18
Results versus SOTA
COMPARISON OF OUR PROPOSED DCFQI VIEW/PATCH BASED BASE AND CUMULATIVE MODELS (DCFQI-VBM &
DCFQI-VCM / DCFQI-PBM & DCFQI-PCM RESPECTIVELY) WITH THE STATE OF THE ART NO-REFERENCE METRICS.

View Slide

19
Zaineb Ibork
01
construction
02
04
Proposed method
03
Conclusion
05

View Slide

20
Conclusion
Future works
In this paper, we presented a no-reference mesh quality assessment approach:
➢ It renders the mesh in 2D views that can be subsequently divided in patches.
➢ From these images, deep features are extracted by the pre-trained VGG16
CNN and fed into a MLP that performs quality prediction.
➢ This base model is competitive with the state-of-the-art, even at the view
level.
➢ Finally a cumulative training has been proposed to obtain a single final model
for prediction that goes beyond the state-of-the-art.
Future works will consider
➢ Combining both view and patch predictions.
➢ The case of colored meshes.

View Slide

21
Thank you!
Any questions?
Special thanks: This work received funding from PHC TOUBKAL TBK/22/142-CAMPUS N°47259YH.

View Slide

ISPA 2023

ISPA 2023

Olivier Lézoray

More Decks by Olivier Lézoray

Other Decks in Research

Featured

Transcript