Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Synthesizing Human Images in SIGGRAPH Asia 2021
Search
Udon
February 27, 2022
Technology
0
170
Synthesizing Human Images in SIGGRAPH Asia 2021
SIGGRAPH Asia 2021の「Synthesizing Human Images」セッションに採択された5本の論文を紹介します.
Udon
February 27, 2022
Tweet
Share
More Decks by Udon
See All by Udon
MIRU2024_招待講演_RALF_in_CVPR2024
udonda
1
340
[CVPR24 Oral] Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation
udonda
0
290
Survey of Image Editing with GANs in SIGGRAPH'21
udonda
2
850
Network-to-Network Translation with Conditional Invertible Neural Networks
udonda
1
250
DARTS: Differentiable Architecture Search
udonda
0
120
Other Decks in Technology
See All in Technology
成果を出しながら成長する、アウトプット駆動のキャッチアップ術 / Output-driven catch-up techniques to grow while producing results
aiandrox
0
180
Postman と API セキュリティ / Postman and API Security
yokawasa
0
200
MLOps の現場から
asei
6
630
ハイテク休憩
sat
PRO
2
120
バクラクのドキュメント解析技術と実データにおける課題 / layerx-ccc-winter-2024
shimacos
2
1k
TSKaigi 2024 の登壇から広がったコミュニティ活動について
tsukuha
0
160
LINEスキマニにおけるフロントエンド開発
lycorptech_jp
PRO
0
330
社外コミュニティで学び社内に活かす共に学ぶプロジェクトの実践/backlogworld2024
nishiuma
0
250
開発生産性向上! 育成を「改善」と捉えるエンジニア育成戦略
shoota
1
230
マイクロサービスにおける容易なトランザクション管理に向けて
scalar
0
110
CustomCopを使ってMongoidのコーディングルールを整えてみた
jinoketani
0
220
統計データで2024年の クラウド・インフラ動向を眺める
ysknsid25
2
830
Featured
See All Featured
GitHub's CSS Performance
jonrohan
1030
460k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
26
1.9k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
45
2.2k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
0
94
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
28
900
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
44
9.3k
Navigating Team Friction
lara
183
15k
4 Signs Your Business is Dying
shpigford
181
21k
It's Worth the Effort
3n
183
28k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
59k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
111
49k
Making the Leap to Tech Lead
cromwellryan
133
9k
Transcript
%BJDIJ)PSJUBൃද 4*((3"1)"TJB Synthesizing Human Images Session
୭ʁ 2
Synthesizing Human Images Session • URL: https://sa2021.siggraph.org/jp/attend/technical-papers/8/session/73 • 5ຊͷจ͕࠾ •
Barbershop • EyelashNet • Neural Actor • Pose with Style • SketchHairSalon 3
4
Barbershop ઃఆ — GAN inversion-based hair editing 5 Face identity
Generated image Hair color condition Hair texture condition Hair shape condition
• ͱإͷؒͷϒϨϯυΞʔςΟϑΝΫτΛੜͤ͡͞ͳ͍ੜ Barbershop ݁Ռ 6
Barbershop ख๏ 7 ͷલΛ” ” W+ F ͷޙΛ” ” W+
S C = (F, S) Face Hair ೖྗ ࠶ߏ Swap S ֤ೖྗ݅ ͷద༻ Blend ݟͨͷ࠷దԽ Face Identity Hair style
Barbershop ੍ݶ 8 • ᶃϚεΫ͕ͳ͍෦͋·Γ៉ྷʹ ͳΒͳ͍ • ᶄᶆϐΞεͷΑ͏ͳαϯϓϧ ແཧ •
ᶅᶇإΛःΔΑ͏ͳੜແཧ
9
EyelashNet ಈػ 10 ·ͭ͛ͷ3D࠶ߏࠔ ʢमਖ਼ʹ5࣌ؒఔඞཁʣ ·ͭ͛ͷMattingΛߦ͍ আڈ͔ͯ͠Βͷ3D࠶ߏ៉ྷʹͰ͖Δ with ·ͭ͛ w/o
·ͭ͛
EyelashNet ·ͭ͛σʔληοτ࡞ͷ 11 • ·ͭ͛άϦʔϯόοΫࡱӨͷΑ͏ͳํ๏ͰࡱӨͰ͖ͳ͍ • എܠʹͰͳ͘ɼલܠʹʢ·ͭ͛ʹܬޫృྉΛృͬͯࡱӨʣ
EyelashNet ·ͭ͛σʔληοτ࡞ͷ 12 • 2ຕͷࡱӨͷؒʹਓಈ͍ͯ͠·͏ͷͰ…ʁʁ • Optical flowͰҐஔ߹Θͤ XPҐஔ߹Θͤ XҐஔ߹Θͤ
EyelashNet ݁Ռ 13
14
Neural Actor ֓ཁ • ҙͷࢹɾ੍ޚՄೳͳϙʔζͰͷߴ࣭ͳਓؒ߹ͷͨΊͷख๏ • ϙʔζ<->ඪ४ۭؒͷมΛֶश 15
16 Neural Actor ܇࿅
NeuralActor ৽نࢹ߹࣮ݧ 17 ఆྔൺֱ ʢFID͕ۃʹྑ͍; γʔέϯεͰݟͨ࣌ʹ༏ΕΔʣ ఆੑൺֱ
18
Pose with Style ֓ཁ 19 • UVϚοϓ͕݅ͳStyleGAN2ʹΑΔશը૾ੜ ೖྗ ग़ྗ ਓੜ
Ծࢼண ೖྗ ग़ྗ
Pose with Style ܇࿅ 20 Ґஔ߹Θͤ 5BSHFUDPPSE HFOFSBUPS
Pose with Style ࣮ݧ — vs. Img-to-imgมϞσϧͱͷൺֱ 21 • ߴप៉ྷʹ
ੜ • ͷ༷ • إͷৄࡉ
Pose with Style ࣮ݧ — vs. StyleGAN-basedϞσϧͱͷൺֱ 22 • StylePoseGAN[Liu+
arXiv21] • Pose with StyleͷUVϚοϓʹର͢ ΔΛ΄ͱΜͲແͨ͘͠Α͏ͳ ख๏ • UVϚοϓΛ࠷େݶར༻͢Εߴ࣭
23
SketchHairSalon σʔληοτ࡞ 24 ೖྗ soft mask ࣗಈੜ खಈ •
soft maskMattingख๏Ͱ ਪ • ࣗಈੜͰฤΈࠐΈͳͲΛ දݱͰ͖ͳ͍ • खಈσʔληοτ͕ඞਢ
SketchHairSalon σʔληοτ࡞ 25 • ਪ࣌ʹ{ฤΈࠐΈ, ςΫενϟ}Λॻ͔ͤΔͷඇৗʹ໘͍͘͞ͷͰ…? • ύϥϝτϦοΫͳදݱؔΛఏҊ ฤΈࠐΈ ඇฤΈࠐΈ
SketchHairSalon ख๏ 26
SketchHairSalon ࣮ݧ 27
SketchHairSalon ੍ݶ 28 • Ԟߦ͖͕ඞཁͳܕݫ͍͠ • ແݶໟଋܕϑϥοτ ͳੜ • רࢴֶशαϯϓϧͱಉ͡
ੜ • Ԟߦ͖ߟྀͨ͠ੜͰղܾ(?)
·ͱΊ • ਓυϝΠϯͷΛѻ͏5ຊͷจΛհ • ฤू • Animatable NeRF • ·ͭ͛
• શੜ • ݻ༗ͷΛͲ͏ղܾ͢Δ͔ʁΛ͔ͳΓ۩ମతͳΞϓϩʔνͰղܾ͍ͯ͠Δ จ͕ଟ͔ͬͨ 29 Synthesizing Human Images Session