Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
GNeRF: GAN-based Neural Radiance Field without Posed Camera
Search
peisuke
December 12, 2021
Technology
1
670
GNeRF: GAN-based Neural Radiance Field without Posed Camera
第9回全日本コンピュータビジョン勉強会
ICCV2021論文読み回発表資料
GNeRF: GAN-based Neural Radiance Field without Posed Camera
peisuke
December 12, 2021
Tweet
Share
More Decks by peisuke
See All by peisuke
LangGraphで始めるマルチエージェントシステム
peisuke
12
3.5k
AutoGenで作るLLM Agen
peisuke
1
1.1k
Self-RAG: Learning to Retrieve, Generate and Critique through Self-Reflections
peisuke
9
1.3k
Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
peisuke
0
7.4k
LangChain Toolsの運用と改善
peisuke
4
2.4k
TTS Skins: Speaker Conversion via ASR
peisuke
0
340
A Quantum Computational Approach to Correspondence Problems on Point Sets
peisuke
0
650
F0-Consistent Many-to-many Non-parallel Voice Conversion via Conditional Autoencoder
peisuke
0
160
YOLACT real-time instance segmentation
peisuke
1
220
Other Decks in Technology
See All in Technology
JBUG岡山 #6 WordCamp男木島の チームビルディング
takeshifurusato
0
150
Github Actions 로 Android 팀의 효율성 극대화
hadonghyun
0
160
データ分析を支える技術 生成AI再入門
ishikawa_satoru
0
380
開発生産性をむしろ向上させる セキュリティパートナーの作り方 / Dev Productivity Con 2024
flatt_security
0
370
Git 研修 Advanced【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
200
LLMアプリケーションの評価の実践と課題 ~PharmaXにおける今後の展望~
pharma_x_tech
2
160
推薦システムを本番導入する上で一番優先すべきだったこと~NewsPicks記事推薦機能の改善事例を元に~
morinota
0
130
スレットハンティングについて知っておきたいこと
hacket
0
130
AI研修【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
130
サービス開発を前に進めるために 新米リードエンジニアが 取り組んだこと / Steps Taken by a Novice Lead Engineer to Advance Service Development
nologyance
0
180
「我々はどこに向かっているのか」を問い続けるための仕組みづくり / Establishing a System for Continuous Inquiry about where we are
daitasu
0
170
Git 研修 Basic【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
310
Featured
See All Featured
Making Projects Easy
brettharned
111
5.7k
KATA
mclloyd
20
13k
Building Effective Engineering Teams - LeadDev
addyosmani
47
2.2k
Rebuilding a faster, lazier Slack
samanthasiow
78
8.5k
Navigating Team Friction
lara
181
13k
No one is an island. Learnings from fostering a developers community.
thoeni
17
2.8k
Product Roadmaps are Hard
iamctodd
PRO
48
10k
Designing the Hi-DPI Web
ddemaree
276
34k
Rails Girls Zürich Keynote
gr2m
93
13k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
360
22k
Happy Clients
brianwarren
94
6.5k
A Tale of Four Properties
chriscoyier
155
22k
Transcript
GNeRF: GAN-based Neural Radiance Field without Posed Camera @peisuke 9
ABEJA Twitter @peisuke Github https://github.com/peisuke Qiita https://qiita.com/peisuke SlideShare https://www.slideshare.net/FujimotoKeisuke
• • GNeRF: GAN-based Neural Radiance Field without Posed Camera
• • NeRF • • • GAN Radiance Field • RF
NeRF • / 5 •
• • (x,y,z)T (φ1,θ1)T (φ2,θ2)T (x,y,z) φ2,θ2 (x,y,z) φ1,θ1
NeRF • • •
NeRF • NeRF in the Wild • NeRF • iNeRF
• iNeRF NeRF in the Wild BP
条件 iNeRF NeRF-- IDR GNeRF(提案⼿法) 事前学習 必要 不要 不要 不要
粗いカメラ座標 必要 必要 必要 不要 前⽅のみの制約 不要 必要 不要 不要 マスク 不要 不要 必要 不要 • •
• • • • • GAN Radiance Field • RF
• Phase A Phase B
RF • D E 3
GAN Real • • Discriminator Real • Dynamic Patch Sampling
/ 16x16 • Dynamic Patch Sampling
GAN Fake • • NeRF 16x16 • G
• 6 𝑹 = 𝑟!! 𝑟!" 𝑟!# 𝑟"! 𝑟"" 𝑟"#
𝑟#! 𝑟#" 𝑟## 𝒓 = 𝑟!! 𝑟!" 𝑟!# 𝑟"! 𝑟"" 𝑟"# $ • SO(3) NN • On the Continuity of Rotation Representations in Neural Networks
• Inversion network • Static Patch Sampling • E •
Static Patch Sampling
G G D G
• • Inversion network • refinement
• Refinement
• • • • • Photometric loss
• • Generator • NeRF • Discriminator • Instance Normalization
Spectral Normalization • Inversion network • Vision Transformer • • A • A B 1 iteration • B
• • COLMAP NeRF C+n • Refine C+r • Ground
Truth NeRF G+n • • Synthetic-NeRF image size: 400x400 DTU 500x400 • 100 • • Peak Signal-to-Noise Ratio SSIM LPIPS • / • Single TAITAN GPU 30
COLMAP •
• COLMAP Drums
難しいシーン 特徴の少ない シーン
IDR • 3D IDR • IDR • PNSR •
• COLMAP • Drums Lego COLMAP
Ablation Study • • GAN InvNet. PhaseB • • A
B A ABABABAB B
• • • COLMAP • •
• NeRF • • GAN RF • RF • •
• COLMAP