Reference-Free Image Quality Assessment for Virtual Try-On via Human Feedback

Yuki Hirakawa (ZOZO Research, Keio Univ.), Takashi Wada, Ryotaro Shimizu, Takuya Furusawa, Yuki Saito (ZOZO Research), Ryosuke Araki (ZOZO), Tianwei Chen, Fan Mo (ZOZO Research), Yoshimitsu Aoki (Keio Univ.)

Paper:
https://arxiv.org/pdf/2603.13057
Source Code & Dataset:
https://github.com/litelightlite/VTON-IQA
Demo:
https://huggingface.co/spaces/zozonext/VTON-IQA

第29回 画像の認識・理解シンポジウム MIRU2026
https://miru-committee.github.io/miru2026/

Transcript

1. ECCV2026採択 Reference-Free Image Quality Assessment for Virtual Try-On via Human Feedback Yuki Hirakawa1, 3, Takashi Wada1, Ryotaro Shimizu1, Takuya Furusawa1, Yuki Saito1, Ryosuke Araki2, Tianwei Chen1, Fan Mo1, and Yoshimitsu Aoki3 1 ZOZO Research, 2 ZOZO Inc., 3 Keio University
2. Virtual Try-On (VTON) ◼ VTON digitally simulate how a garment would look like on a given person. VTON • GAN • Diffusion ◼ A promising technology for bridging the gap between online fashion retail and physical stores. Aoki Media Sensing Lab. 1
3. VTON Still Frequently Generate Imperfect Results ◼ Displaying failed VTON results risks leading customers to make incorrect purchase decisions. The text printed on the shirt appears distorted. The neckline is depicted as a round neck instead of V-neck. VTON Aoki Media Sensing Lab. The stripe pattern on the sides is missing. 2
4. Reference-free Evaluation for VTON ◼ Score-based & Description-based evaluation of individual VTON images without ground-truth. Score-based Description-based The text printed on the shirt appears distorted. VTON The neckline is depicted as a round neck instead of V-neck. The stripe pattern on the sides is missing. Aoki Media Sensing Lab. 3
5. Reference-free Evaluation for VTON ◼ Score-based & Description-based evaluation of individual VTON images without ground-truth. Score-based Our Target Description-based The text printed on the shirt appears distorted. VTON The neckline is depicted as a round neck instead of V-neck. The stripe pattern on the sides is missing. Aoki Media Sensing Lab. 4
6. Key Requirements for Real-World Deployment ◼ No metrics enables automatic per-image evaluation without ground truth images. Reference-free Automation Per-image eval. SSIM × LPIPS × Human FID △ Ours × ×︎ ◼ Other desiderate o Lightweight and efficient o Minor errors should not be heavily penalized if the resulting VTON image looks natural. Aoki Media Sensing Lab. 5
7. VTON-QBench ◼ A large-scale human perception-aligned dataset for VTON quality evaluation. 1. Try-on image generation 2. Crowdsource 3-level evaluation 3. Dataset curation 789K Crowd workers Unreliable Reliable 431K 14 representative VTON models LLM(2) / DiT diff.(4) / U-Net diff.(5) / GAN(3) Aoki Media Sensing Lab. • • Dummy problems Repeatedly choosing the same option etc. 6
8. VTON-QBench ◼ Significant improvement of the questionnaire-level annotator agreement (Krippendorff's alpha) Discard ! Aoki Media Sensing Lab. 7
9. VTON-QBench ◼ A large-scale human perception-aligned dataset for VTON quality evaluation. 1. Try-on image generation 2. Crowdsource 3-level evaluation 3. Dataset curation 789K Crowd workers Unreliable Reliable 431K • • 14 representative VTON models LLM(2) / DiT diff.(4) / U-Net diff.(5) / GAN(3) Dummy problems Repeatedly choosing the same option etc. ◼ Dataset statistics (after curation) Aoki Media Sensing Lab. #Garment #VTON #Annotator #Annotation 13,153 62,688 13,838 431,800 8
10. Convert Discrete Annotation to Continuous MOS ◼ The final quality score S is computed as the average of numerical ratings from multiple annotators. Annotations to try-on query (G, P, V) ◼ The final quality score S provides an estimate of the average perceptual naturalness of VTON images. Aoki Media Sensing Lab. 9
11. VTON-IQA (Architecture) ◼ Interleaved Cross Attention (ICA) module is designed to measure: 1) Garment characteristic preservation; 2) Human pose, human identity, and background preservation. Quality Score in [-1. 1] Aoki Media Sensing Lab. 10
12. Generalization to Unseen Garment & Person ◼ SRCC with human judgements is evaluated on unseen garments and persons that were never seen during training. Aoki Media Sensing Lab. 11
13. Human vs. Automated Ranking of VTON Results Garment Human LPIPS SSIM VTON-IQA Aoki Media Sensing Lab. VTON 3 3 3 3 2 2 1 2 GT 1 1 2 Rank 1 12
14. Let’s Connect at Poster Session ! Session id OS2D-10 Paper Aoki Media Sensing Lab. Source Code & Dataset Demo 13