Synthesizing Human Images in SIGGRAPH Asia 2021

SIGGRAPH Asia 2021の「Synthesizing Human Images」セッションに採択された5本の論文を紹介します．

Transcript

1. 
   %BJDIJ
   )PSJUB
   
   
   
   
   ෼ൃද 4*((3"1)
   "TJB
    Synthesizing Human Images Session
2. ୭ʁ 2
3. Synthesizing Human Images Session • URL: https://sa2021.siggraph.org/jp/attend/technical-papers/8/session/73 • 5ຊͷ࿦จ͕࠾୒ • Barbershop • EyelashNet • Neural Actor • Pose with Style • SketchHairSalon 3
4. 4
5. Barbershop ໰୊ઃఆ — GAN inversion-based hair editing 5 Face identity Generated image Hair color condition Hair texture condition Hair shape condition
6. • ൅ͱإͷؒͷϒϨϯυΞʔςΟϑΝΫτΛੜͤ͡͞ͳ͍ੜ੒ Barbershop ݁Ռ 6
7. Barbershop ख๏ 7 ͷલ൒Λ” ” W+ F ͷޙ൒Λ” ” W+ S C = (F, S) Face Hair ೖྗ ࠶ߏ੒ Swap S ֤ೖྗ৚݅
 ͷద༻ Blend ݟͨ໨ͷ࠷దԽ Face Identity Hair style
8. Barbershop ੍ݶ 8 • ᶃϚεΫ͕ͳ͍෦෼͸͋·Γ៉ྷʹ ͳΒͳ͍ • ᶄᶆϐΞε΍඘ͷΑ͏ͳαϯϓϧ͸ ແཧ • ᶅᶇإΛःΔΑ͏ͳ൅͸ੜ੒ແཧ
9. 9
10. EyelashNet ಈػ 10 ·ͭ͛ͷ3D࠶ߏ੒͸ࠔ೉
 ʢमਖ਼ʹ5࣌ؒఔඞཁʣ ·ͭ͛ͷMattingΛߦ͍
 আڈ͔ͯ͠Βͷ3D࠶ߏ੒͸៉ྷʹͰ͖Δ with ·ͭ͛ w/o ·ͭ͛
11. EyelashNet ·ͭ͛σʔληοτ࡞੒ͷ໰୊ 11 • ·ͭ͛͸άϦʔϯόοΫࡱӨͷΑ͏ͳํ๏Ͱ͸ࡱӨͰ͖ͳ͍ • എܠʹ޻෉Ͱ͸ͳ͘ɼલܠʹ޻෉ʢ·ͭ͛ʹܬޫృྉΛృͬͯࡱӨʣ
12. EyelashNet ·ͭ͛σʔληοτ࡞੒ͷ໰୊ 12 • 2ຕͷࡱӨͷؒʹਓ͸ಈ͍ͯ͠·͏ͷͰ͸…ʁʁ • Optical flowͰҐஔ߹Θͤ XP
    Ґஔ߹Θͤ X
    Ґஔ߹Θͤ
13. EyelashNet ݁Ռ 13
14. 14
15. Neural Actor ֓ཁ • ೚ҙͷࢹ఺ɾ੍ޚՄೳͳϙʔζͰͷߴ඼࣭ͳਓؒ߹੒ͷͨΊͷख๏ • ϙʔζ<->ඪ४ۭؒͷม׵Λֶश 15
16. 16 Neural Actor ܇࿅
17. NeuralActor ৽نࢹ఺߹੒࣮ݧ 17 ఆྔൺֱ
 ʢFID͕ۃ୺ʹྑ͍; γʔέϯεͰݟͨ࣌ʹ༏ΕΔʣ ఆੑൺֱ
18. 18
19. Pose with Style ֓ཁ 19 • UVϚοϓ͕৚݅ͳStyleGAN2ʹΑΔશ਎ը૾ੜ੒ ೖྗ ग़ྗ ਓ෺ੜ੒ Ծ૝ࢼண ೖྗ ग़ྗ
20. Pose with Style ܇࿅ 20 Ґஔ߹Θͤ 5BSHFU
    DPPSE
    HFOFSBUPS
21. Pose with Style ࣮ݧ — vs. Img-to-imgม׵Ϟσϧͱͷൺֱ 21 • ߴप೾਺੒෼΋៉ྷʹ ੜ੒ • ෰ͷ໛༷ • إͷৄࡉ
22. Pose with Style ࣮ݧ — vs. StyleGAN-basedϞσϧͱͷൺֱ 22 • StylePoseGAN[Liu+ arXiv21] • Pose with StyleͷUVϚοϓʹର͢ Δ޻෉Λ΄ͱΜͲແͨ͘͠Α͏ͳ ख๏ • UVϚοϓΛ࠷େݶར༻͢Ε͹ߴ඼࣭
23. 23
24. SketchHairSalon σʔληοτ࡞੒ 24 ೖྗ ൅
 soft mask ࣗಈੜ੒ खಈ • ൅soft mask͸Mattingख๏Ͱ ਪ࿦ • ࣗಈੜ੒Ͱ͸ฤΈࠐΈͳͲΛ දݱͰ͖ͳ͍ • खಈσʔληοτ͕ඞਢ
25. SketchHairSalon σʔληοτ࡞੒ 25 • ਪ࿦࣌ʹ{ฤΈࠐΈ, ൅ςΫενϟ}Λॻ͔ͤΔͷ͸ඇৗʹ໘౗͍͘͞ͷͰ͸…? • ύϥϝτϦοΫͳ൅දݱؔ਺ΛఏҊ ฤΈࠐΈ ඇฤΈࠐΈ
26. SketchHairSalon ख๏ 26
27. SketchHairSalon ࣮ݧ 27
28. SketchHairSalon ੍ݶ 28 • Ԟߦ͖͕ඞཁͳ൅ܕ͸ݫ͍͠ • ແݶໟଋ൅ܕ͸ϑϥοτ
 ͳੜ੒ • רࢴ͸ֶशαϯϓϧͱಉ͡
 ੜ੒ • Ԟߦ͖΋ߟྀͨ͠൅ੜ੒Ͱղܾ(?)
29. ·ͱΊ • ਓυϝΠϯͷ໰୊Λѻ͏5ຊͷ࿦จΛ঺հ • ൅ฤू • Animatable NeRF • ·ͭ͛ • શ਎ੜ੒ • ݻ༗ͷ໰୊ΛͲ͏ղܾ͢Δ͔ʁΛ͔ͳΓ۩ମతͳΞϓϩʔνͰղܾ͍ͯ͠Δ
 ࿦จ͕ଟ͔ͬͨ 29 Synthesizing Human Images Session