Share
Embed
Virtual influencer를 위한 face swapping 기술 개발기
#DigitalHuman #GenerativeModel
카카오브레인에서 개발한 face swapping 기술을 소개합니다.
발표자 : orca.na
카카오브레인 Digital Human Studio 에서 딥러닝 기반 생성 모델을 연구하고 있는 Orca입니다.
kakao
December 09, 2022
Transcript
47 slides
-
7JSUVBMJO fl VFODFSۆ 'BDFTXBQQJOHۆ ӝઓݽ؛ೠ҅ ܻݽ؛
.'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO খਵ۽ӝغח -
3FGFSFODFJNBHFWJEFPਗࠄ࢚ ↟ ߓ҃ ݠܻझఋੌ١JEFOUJUZܳઁ৻ೠաݠࠁ SFGFSFODFࠁ ઁҕ 'BDFTXBQQJOHۆ *%JNBHFࢎਊоਗೞחҷ ↟ ׀ݽন
ҷഋ١JEFOUJUZীҙ۲ػࠁ *%ࠁ ઁҕ "* -
'BDF - TXBQQFEJNBHFWJEFPо۽ࢤࢿػ࢚ ↟ যSFGFSFODF *%ࠁܳࢿ೧ࢲ࢜۽࢚ࢤࢿ 3FGFSFODFJNBHFWJEFPਗࠄ࢚ ↟ ߓ҃ ݠܻझఋੌ١JEFOUJUZܳઁ৻ೠաݠࠁ
SFGFSFODFࠁ ઁҕ 'BDFTXBQQJOHۆ *%JNBHFࢎਊоਗೞחҷ ↟ ׀ݽন ҷഋ١JEFOUJUZীҙ۲ػࠁ *%ࠁ ઁҕ "* -
(FOFSBUJWF"EWFSTBSJBM/FUXPSLT ("/ ↟ ѐਃ ↟ эоܳࢤࢿ೧ղח"*ݽ؛ ↟ 5BTL ↟ 3BOEPNJNBHFHFOFSBUJPO
↟ णঌҊ્ܻ ↟ "EWFSTBSJBMMPTT 14 ,BSSBT 5FSP FUBM"OBMZ[JOHBOEJNQSPWJOHUIFJNBHFRVBMJUZPGTUZMFHBO1SPDFFEJOHTPGUIF*&&&$7'DPOGFSFODFPODPNQVUFSWJTJPOBOEQBUUFSOSFDPHOJUJPO 'BDFTXBQQJOHۆ 4UZMF("/ (FOFSBU 3BOEPNDPEF 3BOEPNDPEF 3BOEPNDPEF 3BOEPNDPEF 3BOEPNJNBHF 3BOEPNJNBHF 3BOEPNJNBHF 3BOEPNJNBHF -
'BDFTXBQQJOHۆ 'BDFTXBQQJOHXJUI("/ *%JNBHF 3FGFSFODF JNBHF (FOFSBUFE JNBHF "* "EWFSTBSJBM MPTT
*%MPTT 3FDPOTUSVDUJPOMPTT -
ۢࠁੌب۽ࢎपੋ оܳࢤࢿೞب۾ण 'BDFTXBQQJOHXJUI("/ 'BDFTXBQQJOHۆ *%JNBHF 3FGFSFODF JNBHF (FOFSBUFE JNBHF "*
"EWFSTBSJBM MPTT *%MPTT 3FDPOTUSVDUJPOMPTT -
*%JNBHF*%ࠁ ׀ݽন ҷഋ ܳ ୭ೠ߈೧ࢲܳࢤࢿೞب۾ण 'BDFTXBQQJOHXJUI("/ 'BDFTXBQQJOHۆ *%JNBHF 3FGFSFODF JNBHF
(FOFSBUFE JNBHF "* "EWFSTBSJBM MPTT *%MPTT 3FDPOTUSVDUJPOMPTT -
3FGFSFODFJNBHFSFGFSFODFࠁ ߓ҃ ݠܻझఋੌ١ ܳ୭ೠ߈೧ࢲ ܳࢤࢿೞب۾ण 'BDFTXBQQJOHXJUI("/ 'BDFTXBQQJOHۆ *%JNBHF 3FGFSFODF JNBHF
(FOFSBUFE JNBHF "* "EWFSTBSJBM MPTT *%MPTT 3FDPOTUSVDUJPOMPTT -
ӝઓݽ؛ೠ҅ ࢤࢿػ௬ܻ౭о࠙જঋ ↟ അࢎۈҷࢤࢿӝࣿNFHBQJYFMܳࢤࢿೡࣻ ↟ ೞ݅ GBDFTXBQӝࣿইী࠙ޅೣ ,BSSBT 5FSP
FUBM"OBMZ[JOHBOEJNQSPWJOHUIFJNBHFRVBMJUZPGTUZMFHBO1SPDFFEJOHTPGUIF*&&&$7'DPOGFSFODFPODPNQVUFSWJTJPOBOEQBUUFSOSFDPHOJUJPO 4UZMF("/ࢤࢿೠNFHBQJYFM -
ӝઓݽ؛ೠ҅ 'BDFTFHNFOUBUJPONPEFMبਸ߉ইঠೣ ↟ 3FGFSFODFJNBHFBUUSJCVUFT ߓ҃ оੜ߈غޅೞӝٸޙ ↟ 4FHNFOUBUJPONPEFMبਸ߉ইSFGFSFODFJNBHF۽ࠗఠਃೠٜࠗ࠙ਸੜۄࢲоઉ১ 'BDFTFNHFOUBUJPOद
IUUQT HJUIVCDPNTXJUDIBCMFOPSNT$FMFC".BTL - )2 -
ӝઓݽ؛ೠ҅ ߈غযঠೡBUUSJCVUFTо࠙߈غঋ ↟ 3FGFSFODFJNBHF ݠܻпب ↟ *%JNBHFҷഋ 3FGFSFODFJNBHF *%JNBHF 'BDF
- TXBQQFEJNBHF 3FGFSFODF *%,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT -
("/JOWFSTJPO 1SF - USBJOFE 4UZMF("/ (FOFSBUPS 1SF - USBJOFE 4UZMF("/
(FOFSBUPS &ODPEFS .BOJQVMBUJPO 0SJHJOBMJNBHF 3FDPOTUSVDUFEJNBHF &EJUFEJNBHF w ̂ w &YUSBDUFEDPEF .BOJQVMBUFEDPEF 3FDPOTUSVDUJPOMPTT "MBMVG :VWBM 0S1BUBTIOJL BOE%BOJFM$PIFO - 0S3FTUZMF"SFTJEVBM - CBTFETUZMFHBOFODPEFSWJBJUFSBUJWFSF fi OFNFOU1SPDFFEJOHTPGUIF*&&&$7'*OUFSOBUJPOBM$POGFSFODFPO$PNQVUFS7JTJPO .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
ӝઓޙઁࢤࢿػ௬ܻ౭о࠙જঋ ↟ 4PMVUJPO("/JOWFSTJPOӝ߈NPEFM٣ੋ 1SF - USBJOFE 4UZMF("/ 'BDJBM "UUSJCVUF
&ODPEFS &YUSBDUFEDPEF 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODFJNBHF 'BDF - TXBQQFEJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 'BDF - TXBQQFEJNBHF 3FDPOTUSVDUJPO MPTT *%MPTT "EWFSTBSJBMMPTT ("/JOWFSTJPOӝ߈NPEFM .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
ӝઓޙઁࢤࢿػ௬ܻ౭о࠙જঋ ↟ 4PMVUJPO("/JOWFSTJPOӝ߈NPEFM٣ੋ 1SF - USBJOFE 4UZMF("/ 'BDJBM "UUSJCVUF
&ODPEFS &YUSBDUFEDPEF 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF "* 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF *%MPTT 3FDPOTUSVDUJPOMPTT "EWFSTBSJBMMPTT ("/JOWFSTJPOӝ߈NPEFM .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
3FGFSFODFJNBHF *%JNBHF W#BTFMJOF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ
4UBCJMJUZ BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT ? ? W W 'BDF - TXBQQFEJNBHF 'BDF - TXBQQFEJNBHF .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
ӝઓޙઁ'BDFTFHNFOUBUJPONPEFMبਸ߉ইঠೣ ↟ 4PMVUJPO'BDJBMBUUSJCVUFFODPEFSীࢲDPEF৻ীNBQө୶ೞب۾٣ੋ &YUSBDUFENBQ 1SF - USBJOFE 4UZMF("/ 'BDJBM
"UUSJCVUF &ODPEFS &YUSBDUFEDPEF 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
3FGFSFODFJNBHF *%JNBHF W#BTFMJOF W#BTFMJOF &YUSBDUFENBQ 'BDF - TXBQQFEJNBHF 'BDF -
TXBQQFEJNBHF 3FGFSFODF *%,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT ? 'BDF - TXBQQFEJNBHF W .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
%.. ↟ %۽ࠗఠ%QBSBNUFSܳ୶೧ࢲ%SFDPOTUSVDUJPO%NBOJQVMBUJPO %.. &ODPEFS %.. %FDPEFS 4IBQF
&YQSFTTJPO 1PTF %JOQVUJNBHF %SFDPOTUSVDUJPO 'FOH :BP FUBM-FBSOJOHBOBOJNBUBCMFEFUBJMFE%GBDFNPEFMGSPNJO - UIF - XJMEJNBHFT"$.5SBOTBDUJPOTPO(SBQIJDT 5P( .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
ӝઓޙઁ߈غযঠೡBUUSJCVUFTо࠙߈غঋ ↟ 4PMVUJPO%..TVQFSWJTJPOਸणҗী୶о 1SF - USBJOFE 4UZMF("/ 'BDJBM "UUSJCVUF
&ODPEFS &YUSBDUFEDPEF 3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF *%JNBHF %.. &ODPEFS sref sswap sid pref pswap pid eref eswap eid 3FGFSFODFJNBHF 'BDF - TXBQQFEJNBHF &YUSBDUFENBQ 4IBQF &YQSFTTJPO 1PTF .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
3FGFSFODFJNBHF *%JNBHF W#BTFMJOF W#BTFMJOF &YUSBDUFENBQ W#BTFMJOF &YUSBDUFENBQ %..TVQFSWJTJPO 'BDF -
TXBQQFEJNBHF 'BDF - TXBQQFEJNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
*EFOUJUZ 4IBQF &YQSFTTJPO 1PTF 1PTF - )/ %FFQGBLFT
FaceShifter 4JNTXBQ )J fi 'BDF *OGP4XBQ .FHB'4 4NPPUI4XBQ .'*. PVST .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ
BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ
BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ
BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ
BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ
BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
3FGFSFODFJNBHF *%JNBHF 'BDF - TXBQQFEJNBHF 3FGFSFODF *%,BSSBT 5FSP FUBM1SPHSFTTJWF(SPXJOHPG("/TGPS*NQSPWFE2VBMJUZ 4UBCJMJUZ
BOE7BSJBUJPO*OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH3FQSFTFOUBUJPOT .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO -
'BDFTXBQQJOH 3FGFSFODF WJEFP *%JNBHF 'BDF - TXBQQFE GSBNFT 1SF -
QSPDFTT 'BDFEFUFDUJPO 4NPPUIJOH $SPQQJOH "MJHONFOU 1SF - QSPDFTT 'BDFEFUFDUJPO 4NPPUIJOH $SPQQJOH "MJHONFOU 1PTU - QSPDFTT 3FUPVDI 6ODSPQQJOH 'BDF - TXBQQJOHQJQFMJOF GPSWJEFP 3FGFSFODF GSBNFT 'BDF - TXBQQFE WJEFP .'*. .'*..FHBQJYFM'BDJBM*EFOUJUZ.BOJQVMBUJPO