Network-to-Network Translation with Conditional Invertible Neural Networks
Network-to-Network Translation with Conditional Invertible Neural Networks
Robin Rombach, Patrick Esser, Björn Ommer
NeurIPS 2020 (oral). Code at this https URL
https://arxiv.org/abs/2005.13580
Rombach∗, Patrick Esser∗, Björn Ommer IWR, HCI, Heidelberg University https://papers.nips.cc/paper/2020/file/1cfa81af29c6f2d8cacb44921722e753-Paper.pdf https://papers.nips.cc/paper/2020/file/1cfa81af29c6f2d8cacb44921722e753-Supplemental.pdf
which are • Image Classification, Segmentation (ResNet, DeepLab Series) • Question Answering (BERT, GPT-3) • Image Generation, Translation (BigGAN, StyleGAN) • Need to find new ways to reuse such expert models!!
Desired output, • : Latent representation • • To realize domain translation, it needs to be described probabilistically as sampling from • Denote , translation func, residuals D x , D y f(x) x ∈ D x z Φ = Φ(x) f(x) = Ψ(Φ(x)), g(y) = Λ(Θ(y)) p(z Θ |z Φ ) z Θ = τ(v|z Φ ) τ : v : x ∈ D x y 1 ∈ D y 5IFEPHJTDVUF 5IFEPHJTMPWFMZ y 2 ∈ D y z Φ = Φ(x) Λ(z Φ ) Λ(z Φ ) v
information of , if and are independent • Minimize v z Φ v z Φ KL(p(v|z Φ )|q(v)) standard normal distribution Achieve the goal of sampling from ɹɹ , sampled from p(z Θ |z Φ ) z Θ = τ(v|z Φ ) v q(v)
• 1. Sample from • 2. Encode into • 3. Sample from • 4. Transform • 5. Decode into x p(x) x z Φ = Φ(x) v q(v) z Θ = τ(v|z Φ ) z Θ y = Λ(z Θ ) 2 3 4 5 1