$30 off During Our Annual Pro Sale. View details »

Synthesizing Human Images in SIGGRAPH Asia 2021

Udon
February 27, 2022

Synthesizing Human Images in SIGGRAPH Asia 2021

SIGGRAPH Asia 2021の「Synthesizing Human Images」セッションに採択された5本の論文を紹介します.

Udon

February 27, 2022
Tweet

More Decks by Udon

Other Decks in Technology

Transcript

  1. %BJDIJ)PSJUB෼ൃද 4*((3"1)"TJB Synthesizing Human Images Session

  2. ୭ʁ 2

  3. Synthesizing Human Images Session • URL: https://sa2021.siggraph.org/jp/attend/technical-papers/8/session/73 • 5ຊͷ࿦จ͕࠾୒ •

    Barbershop • EyelashNet • Neural Actor • Pose with Style • SketchHairSalon 3
  4. 4

  5. Barbershop ໰୊ઃఆ — GAN inversion-based hair editing 5 Face identity

    Generated image Hair color condition Hair texture condition Hair shape condition
  6. • ൅ͱإͷؒͷϒϨϯυΞʔςΟϑΝΫτΛੜͤ͡͞ͳ͍ੜ੒ Barbershop ݁Ռ 6

  7. Barbershop ख๏ 7 ͷલ൒Λ” ” W+ F ͷޙ൒Λ” ” W+

    S C = (F, S) Face Hair ೖྗ ࠶ߏ੒ Swap S ֤ೖྗ৚݅
 ͷద༻ Blend ݟͨ໨ͷ࠷దԽ Face Identity Hair style
  8. Barbershop ੍ݶ 8 • ᶃϚεΫ͕ͳ͍෦෼͸͋·Γ៉ྷʹ ͳΒͳ͍ • ᶄᶆϐΞε΍඘ͷΑ͏ͳαϯϓϧ͸ ແཧ •

    ᶅᶇإΛःΔΑ͏ͳ൅͸ੜ੒ແཧ
  9. 9

  10. EyelashNet ಈػ 10 ·ͭ͛ͷ3D࠶ߏ੒͸ࠔ೉
 ʢमਖ਼ʹ5࣌ؒఔඞཁʣ ·ͭ͛ͷMattingΛߦ͍
 আڈ͔ͯ͠Βͷ3D࠶ߏ੒͸៉ྷʹͰ͖Δ with ·ͭ͛ w/o

    ·ͭ͛
  11. EyelashNet ·ͭ͛σʔληοτ࡞੒ͷ໰୊ 11 • ·ͭ͛͸άϦʔϯόοΫࡱӨͷΑ͏ͳํ๏Ͱ͸ࡱӨͰ͖ͳ͍ • എܠʹ޻෉Ͱ͸ͳ͘ɼલܠʹ޻෉ʢ·ͭ͛ʹܬޫృྉΛృͬͯࡱӨʣ

  12. EyelashNet ·ͭ͛σʔληοτ࡞੒ͷ໰୊ 12 • 2ຕͷࡱӨͷؒʹਓ͸ಈ͍ͯ͠·͏ͷͰ͸…ʁʁ • Optical flowͰҐஔ߹Θͤ XPҐஔ߹Θͤ XҐஔ߹Θͤ

  13. EyelashNet ݁Ռ 13

  14. 14

  15. Neural Actor ֓ཁ • ೚ҙͷࢹ఺ɾ੍ޚՄೳͳϙʔζͰͷߴ඼࣭ͳਓؒ߹੒ͷͨΊͷख๏ • ϙʔζ<->ඪ४ۭؒͷม׵Λֶश 15

  16. 16 Neural Actor ܇࿅

  17. NeuralActor ৽نࢹ఺߹੒࣮ݧ 17 ఆྔൺֱ
 ʢFID͕ۃ୺ʹྑ͍; γʔέϯεͰݟͨ࣌ʹ༏ΕΔʣ ఆੑൺֱ

  18. 18

  19. Pose with Style ֓ཁ 19 • UVϚοϓ͕৚݅ͳStyleGAN2ʹΑΔશ਎ը૾ੜ੒ ೖྗ ग़ྗ ਓ෺ੜ੒

    Ծ૝ࢼண ೖྗ ग़ྗ
  20. Pose with Style ܇࿅ 20 Ґஔ߹Θͤ 5BSHFUDPPSE HFOFSBUPS

  21. Pose with Style ࣮ݧ — vs. Img-to-imgม׵Ϟσϧͱͷൺֱ 21 • ߴप೾਺੒෼΋៉ྷʹ

    ੜ੒ • ෰ͷ໛༷ • إͷৄࡉ
  22. Pose with Style ࣮ݧ — vs. StyleGAN-basedϞσϧͱͷൺֱ 22 • StylePoseGAN[Liu+

    arXiv21] • Pose with StyleͷUVϚοϓʹର͢ Δ޻෉Λ΄ͱΜͲແͨ͘͠Α͏ͳ ख๏ • UVϚοϓΛ࠷େݶར༻͢Ε͹ߴ඼࣭
  23. 23

  24. SketchHairSalon σʔληοτ࡞੒ 24 ೖྗ ൅
 soft mask ࣗಈੜ੒ खಈ •

    ൅soft mask͸Mattingख๏Ͱ ਪ࿦ • ࣗಈੜ੒Ͱ͸ฤΈࠐΈͳͲΛ දݱͰ͖ͳ͍ • खಈσʔληοτ͕ඞਢ
  25. SketchHairSalon σʔληοτ࡞੒ 25 • ਪ࿦࣌ʹ{ฤΈࠐΈ, ൅ςΫενϟ}Λॻ͔ͤΔͷ͸ඇৗʹ໘౗͍͘͞ͷͰ͸…? • ύϥϝτϦοΫͳ൅දݱؔ਺ΛఏҊ ฤΈࠐΈ ඇฤΈࠐΈ

  26. SketchHairSalon ख๏ 26

  27. SketchHairSalon ࣮ݧ 27

  28. SketchHairSalon ੍ݶ 28 • Ԟߦ͖͕ඞཁͳ൅ܕ͸ݫ͍͠ • ແݶໟଋ൅ܕ͸ϑϥοτ
 ͳੜ੒ • רࢴ͸ֶशαϯϓϧͱಉ͡


    ੜ੒ • Ԟߦ͖΋ߟྀͨ͠൅ੜ੒Ͱղܾ(?)
  29. ·ͱΊ • ਓυϝΠϯͷ໰୊Λѻ͏5ຊͷ࿦จΛ঺հ • ൅ฤू • Animatable NeRF • ·ͭ͛

    • શ਎ੜ੒ • ݻ༗ͷ໰୊ΛͲ͏ղܾ͢Δ͔ʁΛ͔ͳΓ۩ମతͳΞϓϩʔνͰղܾ͍ͯ͠Δ
 ࿦จ͕ଟ͔ͬͨ 29 Synthesizing Human Images Session