simple summary: "Learning a Probabilistic Latent Space of Object Shapes via 3D Generative- Adversarial Modeling" (3D-GAN)

Wu, Jiajun, et al. "Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling." Advances in Neural Information Processing Systems. 2016.

arxiv: https://arxiv.org/abs/1610.07584
spotlight video: https://www.youtube.com/watch?v=mfx7uAkUtCI

Transcript

1. “ Learning a Probabilistic Latent Space
 of Object Shapes via 3D Generative-
 Adversarial Modeling “ Wu J, Zhang C, Xue T, Freeman B, Tenenbaum J NIPS 2016
2. ͲΜͳ΋ͷʁ • GANͰ3DΦϒδΣΫτͷੜ੒Λߦͬͨ࿦จ • Latent SpaceͰͷิ׬ɺ଍͠ࢉɾҾ͖ࢉ • DiscriminatorΛ3DΦϒδΣΫτͷ෼ྨʹద༻ • ੩ࢭը͔Βͷ3DΦϒδΣΫτੜ੒ ( reconstruction )
3. ઌߦݚڀͱൺ΂ͯͲ͕͍͜͢͝ʁ • ैདྷͷ3DΦϒδΣΫτͷੜ੒ • ϝογϡɺεέϧτϯ • طଘͷΦϒδΣΫτ͔Β෦෼తʹऔ͖ͬͯͯ૊Έ߹ΘͤΔ
 ‎ ϦΞϧʹݟ͑Δ͕ɺࠜຊతʹ৽͍͠෺ମ͸ੜ੒Ͱ͖ͳ͍
 • ຊ࿦จͷख๏ • ϘΫηϧʢvoxelized objectʣ • GANΛ࢖༻
 ‎ ATΛ࢖༻͢Δ͜ͱͰɺ3D෺ମͷߏ଄Λ G ֶ͕शͰ͖Δ
 ‎ latent spaceͷ೚ҙͷϕΫτϧ͔Β෺ମΛੜ੒Ͱ͖Δ
4. ٕज़΍ख๏ͷΩϞ͸Ͳ͜ʁ 3D-GAN • z ͸200࣍ݩ ( p(x): Ұ༷෼෍ ) • 64x64x64ͷvoxelΛੜ੒ • DataSetʹ͸ ShapeNet Λ࢖༻ • Core: 55छྨ, 51,300Ϟσϧ • Sem: ΑΓ޿ൣͳछྨͰৄࡉͳ஫ऍ෇͖, 12,000Ϟσϧ • ֶश࣌ɺD ͸ࣝผ཰͕80%ະຬͱ͖ʹ͔͠ߋ৽͠ͳ͍ z G(z) in 3D Voxel Space 512×4×4×4 256×8×8×8 128×16×16×16 64×32×32×32
5. ٕज़΍ख๏ͷΩϞ͸Ͳ͜ʁ 3D-VAE-GAN: (2D image→3D object) • VAE-GANΛࢀߟʹ2Dը૾Λ z ʹม׵͢Δ E Λಋೖ L = L 3D-GAN + ↵ 1 L KL + ↵ 2 L recon , LKL = DKL(q(z|y) || p(z)), Lrecon = || G ( E ( y )) x || 2, L3D-GAN = log D(x) + log(1 D(G(z)))
6. Ͳ͏΍ͬͯ༗ޮͩͱݕূͨ͠ʁ • ModelNetͷ10, 40Ϋϥε෼ྨ • 3D-GAN͸ShapeNetͷ7छΛֶश
 (chairs, sofas, tables, boats, airplanes, rifles, and cars) 3D Object Classification
7. Ͳ͏΍ͬͯ༗ޮͩͱݕূͨ͠ʁ • IKEA dataset • 2Dը૾ΑΓ3D෺ମΛ෮ݩ • viewpointͷमਖ਼ͳͲΞϥΠϯϝϯτΛ ߦ͍ɺฏۉਫ਼౓Ͱൺֱ Single Image 3D reconstruction