TMPA-2021: Unpaired Image-to-Image Translation using Transformer-based CycleGAN

Chongyu Gu and Maxim Gromov

Unpaired Image-to-Image Translation using Transformer-based CycleGAN

TMPA is an annual International Conference on Software Testing, Machine Learning and Complex Process Analysis. The conference will focus on the application of modern methods of data science to the analysis of software quality.

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro

Transcript

1. 1 25-27 NOVEMBER SOFTWARE TESTING, MACHINE LEARNING AND COMPLEX PROCESS ANALYSIS Unpaired Image-to-Image Translation using Transformer-based CycleGAN Chongyu Gu, Maxim Gromov
2. 2 Motivation Convolutional Layer Vision Transformer [Dosovitskiy, Alexey, et al., ICLR, 2020] Ref: https://www.ibm.com/cloud/learn/convolutional- neural-networks
3. 3 Motivation Visual results produced by TransGAN Unconditional image generation results by TransGAN [Yifan Jiang, Shiyu Chang, Zhangyang Wang, NeurIPS, 2021]
4. 4 4 The Generator and Discriminator Networks The pipeline of the pure transform-based generator and discriminator of TransCycleGAN.
5. 5 5 Architecture configuration of generator. Architecture configuration of discriminator. The Generator and Discriminator Networks
6. 6 6 Main Results Samples of the horse2zebra 64 ×64 Samples of the zebra2horse 64 ×64 Our model reaches FID of 80.54 on horse2zebra 64 ×64 and 93.05 FID on zebra2horse 64 ×64.
7. 7 7 Conclusion and Limitation We have introduced TransCycleGAN, the first pure transformer-based GAN for the task of image-to-image translation. Our experiments on the horse2zebra 64 × 64 benchmark demonstrate that the great potential of our new architecture. TransCycleGAN still has much room for exploration, such as going towards high-resolution translation tasks (e.g.,256 × 256) and experimenting on more datasets like Apple↔Orange, Summer↔Winter Yosemite, and Photo↔Art for style transfer, which is our future directions.
8. 8 Thank you very much for your attention!