Slide 1

Slide 1 text

GNeRF: GAN-based Neural Radiance Field without Posed Camera @peisuke 9

Slide 2

Slide 2 text

ABEJA Twitter @peisuke Github https://github.com/peisuke Qiita https://qiita.com/peisuke SlideShare https://www.slideshare.net/FujimotoKeisuke

Slide 3

Slide 3 text

• • GNeRF: GAN-based Neural Radiance Field without Posed Camera • • NeRF • • • GAN Radiance Field • RF

Slide 4

Slide 4 text

NeRF • / 5 •

Slide 5

Slide 5 text

• • (x,y,z)T (φ1,θ1)T (φ2,θ2)T (x,y,z) φ2,θ2 (x,y,z) φ1,θ1

Slide 6

Slide 6 text

NeRF • • •

Slide 7

Slide 7 text

NeRF • NeRF in the Wild • NeRF • iNeRF • iNeRF NeRF in the Wild BP

Slide 8

Slide 8 text

条件 iNeRF NeRF-- IDR GNeRF(提案⼿法) 事前学習 必要 不要 不要 不要 粗いカメラ座標 必要 必要 必要 不要 前⽅のみの制約 不要 必要 不要 不要 マスク 不要 不要 必要 不要 • •

Slide 9

Slide 9 text

• • • • • GAN Radiance Field • RF

Slide 10

Slide 10 text

• Phase A Phase B

Slide 11

Slide 11 text

RF • D E 3

Slide 12

Slide 12 text

GAN Real • • Discriminator Real • Dynamic Patch Sampling / 16x16 • Dynamic Patch Sampling

Slide 13

Slide 13 text

GAN Fake • • NeRF 16x16 • G

Slide 14

Slide 14 text

• 6 𝑹 = 𝑟!! 𝑟!" 𝑟!# 𝑟"! 𝑟"" 𝑟"# 𝑟#! 𝑟#" 𝑟## 𝒓 = 𝑟!! 𝑟!" 𝑟!# 𝑟"! 𝑟"" 𝑟"# $ • SO(3) NN • On the Continuity of Rotation Representations in Neural Networks

Slide 15

Slide 15 text

• Inversion network • Static Patch Sampling • E • Static Patch Sampling

Slide 16

Slide 16 text

G G D G

Slide 17

Slide 17 text

• • Inversion network • refinement

Slide 18

Slide 18 text

• Refinement

Slide 19

Slide 19 text

• • • • • Photometric loss

Slide 20

Slide 20 text

• • Generator • NeRF • Discriminator • Instance Normalization Spectral Normalization • Inversion network • Vision Transformer • • A • A B 1 iteration • B

Slide 21

Slide 21 text

• • COLMAP NeRF C+n • Refine C+r • Ground Truth NeRF G+n • • Synthetic-NeRF image size: 400x400 DTU 500x400 • 100 • • Peak Signal-to-Noise Ratio SSIM LPIPS • / • Single TAITAN GPU 30

Slide 22

Slide 22 text

COLMAP •

Slide 23

Slide 23 text

• COLMAP Drums

Slide 24

Slide 24 text

難しいシーン 特徴の少ない シーン

Slide 25

Slide 25 text

IDR • 3D IDR • IDR • PNSR •

Slide 26

Slide 26 text

• COLMAP • Drums Lego COLMAP

Slide 27

Slide 27 text

Ablation Study • • GAN InvNet. PhaseB • • A B A ABABABAB B

Slide 28

Slide 28 text

• • • COLMAP • •

Slide 29

Slide 29 text

• NeRF • • GAN RF • RF • • • COLMAP