Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Virtual influencer를 위한 face swapping 기술 개발기

kakao
December 09, 2022

Virtual influencer를 위한 face swapping 기술 개발기

#DigitalHuman #GenerativeModel

카카오브레인에서 개발한 face swapping 기술을 소개합니다.

발표자 : orca.na
카카오브레인 Digital Human Studio 에서 딥러닝 기반 생성 모델을 연구하고 있는 Orca입니다.

kakao

December 09, 2022
Tweet

More Decks by kakao

Other Decks in Programming

Transcript

  1.  7JSUVBMJO fl VFODFSۆ  'BDFTXBQQJOH੉ۆ  ӝઓݽ؛੄ೠ҅  ਋ܻ੄ݽ؛

    .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO  খਵ۽ӝ؀غח੼
  2. 'BDF - TXBQQFEJNBHFWJEFPо૞۽ࢤࢿػ৔࢚ ↟ ઱য૓SFGFSFODF *%੿ࠁܳ೤ࢿ೧ࢲ࢜۽਍৔࢚ࢤࢿ 3FGFSFODFJNBHFWJEFPਗࠄ৔࢚ ↟ ߓ҃ ݠܻझఋੌ١JEFOUJUZܳઁ৻ೠաݠ૑੿ࠁ

    SFGFSFODF੿ࠁ ઁҕ 'BDFTXBQQJOH੉ۆ *%JNBHFࢎਊ੗оਗೞח঴ҷ ↟ ׀ݽন ঴ҷഋ١JEFOUJUZীҙ۲ػ੿ࠁ *%੿ࠁ ઁҕ "*
  3. (FOFSBUJWF"EWFSTBSJBM/FUXPSLT ("/ ↟ ѐਃ ↟ ૓૞э਷о૞੉޷૑ܳࢤࢿ೧ղח"*ݽ؛ ↟ 5BTL ↟ 3BOEPNJNBHFHFOFSBUJPO

    ↟ ೟णঌҊ્ܻ ↟ "EWFSTBSJBMMPTT 14 ੉޷૑୹୊,BSSBT 5FSP FUBM"OBMZ[JOHBOEJNQSPWJOHUIFJNBHFRVBMJUZPGTUZMFHBO1SPDFFEJOHTPGUIF*&&&$7'DPOGFSFODFPODPNQVUFSWJTJPOBOEQBUUFSOSFDPHOJUJPO 'BDFTXBQQJOH੉ۆ 4UZMF("/ (FOFSBU 3BOEPNDPEF 3BOEPNDPEF 3BOEPNDPEF 3BOEPNDPEF 3BOEPNJNBHF 3BOEPNJNBHF 3BOEPNJNBHF 3BOEPNJNBHF
  4. ӝઓݽ؛੄ೠ҅  ࢤࢿػ੉޷૑੄௬ܻ౭о୽࠙൤જ૑ঋ਺ ↟ അ੤ࢎۈ঴ҷࢤࢿӝࣿ਷NFHBQJYFM੉޷૑ܳࢤࢿೡࣻ੓਺ ↟ ೞ૑݅ GBDFTXBQӝࣿ਷ই૒੉ী୽࠙൤޷஖૑ޅೣ ੉޷૑୹୊,BSSBT 5FSP

    FUBM"OBMZ[JOHBOEJNQSPWJOHUIFJNBHFRVBMJUZPGTUZMFHBO1SPDFFEJOHTPGUIF*&&&$7'DPOGFSFODFPODPNQVUFSWJTJPOBOEQBUUFSOSFDPHOJUJPO 4UZMF("/੉ࢤࢿೠNFHBQJYFM੉޷૑
  5. ӝઓݽ؛੄ೠ҅ ߈৔غযঠೡBUUSJCVUFTо୽࠙൤߈৔غ૑ঋ਺ ↟ 3FGFSFODFJNBHF੄಴੿ ݠܻпب ↟ *%JNBHF੄঴ҷഋ 3FGFSFODFJNBHF *%JNBHF 'BDF

    - TXBQQFEJNBHF 3FGFSFODF *%੉޷૑୹୊,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT
  6. ("/JOWFSTJPO 1SF - USBJOFE 4UZMF("/ (FOFSBUPS 1SF - USBJOFE 4UZMF("/

    (FOFSBUPS &ODPEFS .BOJQVMBUJPO 0SJHJOBMJNBHF 3FDPOTUSVDUFEJNBHF &EJUFEJNBHF w ̂ w &YUSBDUFEDPEF .BOJQVMBUFEDPEF 3FDPOTUSVDUJPOMPTT ੉޷૑୹୊"MBMVG :VWBM 0S1BUBTIOJL BOE%BOJFM$PIFO - 0S3FTUZMF"SFTJEVBM - CBTFETUZMFHBOFODPEFSWJBJUFSBUJWFSF fi OFNFOU1SPDFFEJOHTPGUIF*&&&$7'*OUFSOBUJPOBM$POGFSFODFPO$PNQVUFS7JTJPO .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  7. ӝઓ੄ޙઁ੼ࢤࢿػ੉޷૑੄௬ܻ౭о୽࠙൤જ૑ঋ਺ ↟ 4PMVUJPO("/JOWFSTJPOӝ߈੄NPEFM٣੗ੋ 1SF - USBJOFE 
 4UZMF("/ 'BDJBM "UUSJCVUF

    &ODPEFS &YUSBDUFEDPEF 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODFJNBHF 'BDF - TXBQQFEJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 'BDF - TXBQQFEJNBHF 3FDPOTUSVDUJPO MPTT *%MPTT "EWFSTBSJBMMPTT ("/JOWFSTJPOӝ߈NPEFM .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  8. ӝઓ੄ޙઁ੼ࢤࢿػ੉޷૑੄௬ܻ౭о୽࠙൤જ૑ঋ਺ ↟ 4PMVUJPO("/JOWFSTJPOӝ߈੄NPEFM٣੗ੋ 1SF - USBJOFE 
 4UZMF("/ 'BDJBM "UUSJCVUF

    &ODPEFS &YUSBDUFEDPEF 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF "* 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF *%MPTT 3FDPOTUSVDUJPOMPTT "EWFSTBSJBMMPTT ("/JOWFSTJPOӝ߈NPEFM .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  9. 3FGFSFODFJNBHF *%JNBHF W#BTFMJOF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%੉޷૑୹୊,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ

    4UBCJMJUZ BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT ? ? W W 'BDF - TXBQQFEJNBHF 'BDF - TXBQQFEJNBHF .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  10. 3FGFSFODFJNBHF *%JNBHF W#BTFMJOF W#BTFMJOF &YUSBDUFENBQ 'BDF - TXBQQFEJNBHF 'BDF -

    TXBQQFEJNBHF 3FGFSFODF *%੉޷૑୹୊,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT ? 'BDF - TXBQQFEJNBHF W .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  11. %.. ↟ %੉޷૑۽ࠗఠ%QBSBNUFSܳ୶୹೧ࢲ%SFDPOTUSVDUJPO%NBOJQVMBUJPO %.. 
 &ODPEFS %.. 
 %FDPEFS 4IBQF

    &YQSFTTJPO 1PTF %JOQVUJNBHF %SFDPOTUSVDUJPO ੉޷૑୹୊'FOH :BP FUBM-FBSOJOHBOBOJNBUBCMFEFUBJMFE%GBDFNPEFMGSPNJO - UIF - XJMEJNBHFT"$.5SBOTBDUJPOTPO(SBQIJDT 5P(    .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  12. ӝઓ੄ޙઁ੼߈৔غযঠೡBUUSJCVUFTо୽࠙൤߈৔غ૑ঋ਺ ↟ 4PMVUJPO%..TVQFSWJTJPOਸ೟णҗ੿ী୶о 1SF - USBJOFE 
 4UZMF("/ 'BDJBM "UUSJCVUF

    &ODPEFS &YUSBDUFEDPEF 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF *%JNBHF %.. 
 &ODPEFS sref sswap sid pref pswap pid eref eswap eid 3FGFSFODFJNBHF 'BDF - TXBQQFEJNBHF &YUSBDUFENBQ 4IBQF &YQSFTTJPO 1PTF .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  13. 3FGFSFODFJNBHF *%JNBHF W#BTFMJOF W#BTFMJOF &YUSBDUFENBQ W#BTFMJOF &YUSBDUFENBQ %..TVQFSWJTJPO 'BDF -

    TXBQQFEJNBHF 'BDF - TXBQQFEJNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%੉޷૑୹୊,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  14. *EFOUJUZ­ 4IBQF­ &YQSFTTJPO­ 1PTF­ 1PTF - )/­ %FFQGBLFT  

       FaceShifter      4JNTXBQ      )J fi 'BDF      *OGP4XBQ      .FHB'4      4NPPUI4XBQ      .'*. PVST      .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  15. 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%੉޷૑୹୊,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ

    BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  16. 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%੉޷૑୹୊,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ

    BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  17. 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%੉޷૑୹୊,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ

    BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  18. 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%੉޷૑୹୊,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ

    BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  19. 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%੉޷૑୹୊,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ

    BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  20. 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%੉޷૑୹୊,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ

    BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO
  21. 'BDFTXBQQJOH 3FGFSFODF WJEFP *%JNBHF 'BDF - TXBQQFE GSBNFT 1SF -

    QSPDFTT 
 
 'BDFEFUFDUJPO 
 4NPPUIJOH $SPQQJOH "MJHONFOU 1SF - QSPDFTT 
 
 'BDFEFUFDUJPO 
 4NPPUIJOH $SPQQJOH "MJHONFOU 1PTU - QSPDFTT 
 3FUPVDI 
 6ODSPQQJOH 'BDF - TXBQQJOHQJQFMJOF GPSWJEFP 3FGFSFODF GSBNFT 'BDF - TXBQQFE WJEFP .'*. .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO