Share
Embed
Deep Learning for clothes and changing pose
This is my casual survey about deep learning in fashion, especially fashion swapping, virtual try-on, or pose guided generation.
The criteria to read a paper are it uses fashion dataset or not and It
makes me interested
Masaki Kozuki
January 31, 2018
Transcript
33 slides
-
("/T w *(PPEGFMMPXQSPQPTFE("/TJO w 5IFSFBSFQMFOUZPG("/T w 8("/ $SBNFS("/ 5SJBOHMF("/ 1SPHSFTTJWF(SPXJOH("/
w QJYQJY $ZDMF("/ #JDZDMF("/ 4UBS("/ 3 -
'BTIJPOJO$PNQVUFS7JTJPO w 'BTIJPOJTBUUSBDUJOHNPSFBOENPSFSFTFBSDIFST w %FFQ'BTIJPO EBUBTFU w DPNQBUJCJMJUZFTUJNBUJPO DPPSEJOBUFHSBEJOHDPNQMFNFOUBUJPO
w IVNBOQBSTJOH TFNBOUJDTFHNFOUBUJPO w GBTIJPOBSUJDMFBUUSJCVUFTQSFEJDUJPOHFOFSBUJPO w GBTIJPOBSUJDMFEFTJHOTZOUIFTJT 4 -
MJTUPGQBQFST Jetchev, N., & Bergmann, U. (2017). The Conditional
Analogy GAN: Swapping Fashion Articles on People Images. Retrieved from http://arxiv.org/abs/1709.04695 - ZalandoGAN Zhu, S., Fidler, S., Urtasun, R., Lin, D., & Loy, C. C. (2017). Be Your Own Prada: Fashion Synthesis with Structural Coherence. Retrieved from http://arxiv.org/abs/1710.07346 - FashionGAN Han, X., Wu, Z., Wu, Z., Yu, R., & Davis, L. S. (2017). VITON: An Image-based Virtual Try-on Network. Retrieved from http://arxiv.org/abs/1711.08447 - VITON Ma, L., Sun, Q., Georgoulis, S., Van Gool, L., Schiele, B., & Fritz, M. (2017). Disentangled Person Image Generation. Retrieved from http://arxiv.org/abs/1712.02621 - DPIG Siarohin, A., Sangineto, E., Lathuiliere, S., & Sebe, N. (n.d.). Deformable GANs for Pose-based Human Image Generation. Retrieved from https://arxiv.org/pdf/1801.00055.pdf - Deformable GAN GANs for Try-on Ma, L., Sun, Q., Jia, X., Schiele, B., Tuytelaars, T., & Gool, L. Van. (n.d.). Pose Guided Person Image Generation. Retrieved from https://arxiv.org/pdf/1705.09368.pdf - PG2 -
OPUBUJPOT w IVNBO w BSUJDMFBGBTIJPOJUFNXPSOCZBNPEFMJOʠIVNBOʡJNBHF w UBSHFUBGBTIJPOJUFNXIJDIBNPEFMXJMMXFBS w ПUSBJOFE7((VTJOH*NBHF/FU w
TVGpYAJANFBOTCMPDLJOEFYJO7((NPEFMT w Y Z DPODBUFOBUFEJNBHFXIPTFPGDIBOOFMTJTY Z 6 -
EBUBTFUDPNQBSJTPO w [BMBOEP w DSFBUFEGSPN[BMBOEPTF w "1*JT<IFSF> w UIFSFJTOPUBOZQVCMJDEBUBTFU w
%FFQ'BTIJPO w BWBJMBCMF<IFSF> 9 [BMBOEP %FFQ'BTIJPO ;BMBOEP("/ ✔ ✘ 1( ✘ ✔ 'BTIJPO("/ ✘ ✔ 7*50/ ✔ ✘ %1*( ✘ ✔ %FGPSNBCMF ("/ ✘ ✔ -
NPEFMDPNQBSJTPO w 1&1PTF&TUJNBUPS 10 TJOHMFTUFQ HFOFSBUJPO XJUI1& UFYU ;BMBOEP("/ ✔
✘ ✘ 1( ✘ ✔ ✘ 'BTIJPO("/ ✘ ✘ ✔ 7*50/ ✘ ✔̈ ✘ %1*( ✘ ✔ ✘ %FGPSNBCMF ("/ ✘ ✔ ✘ †: use human parser -
MPTTDPNQBSJTPO 11 ;BMBOEP("/ $ZDMFMPTT -QOPSN "EWFSTBSJBMMPTT 1( NBTL- "EWMPTT
'BTIJPO("/ "EWMPTT 7*50/ -OPSN NBTL 1FSDFQUVBMMPTT %1*( -OPSN NBTL "EWMPTT 1PTFMPTT NBTLMPTT %FGPSNBCMF("/ "EWMPTT OFBSFTUOFJHICPSMPTT -
;BMBOEP("/ w 5IFQJPOFFSPG/FVSBM/FUTXBQQJOHGBTIJPOBSUJDMFT w %BUBTFUJTDSFBUFEGSPNJNBHFTJO[BMBOEPEF w (FOFSBUPS w JOQVUUVQMFPGIVNBO JUFNBOEUBSHFUJUFN
w PVUQVUBSUJDMFTXBQQFEIVNBO JUFNBSFB NBTL JOIVNBO w %JTDSJNJOBUPS w JOQVUUVQMFPGIVNBOT 13 -
1( OJQT w 6/FUMJLFNPEFMTBSFVTFE w 1PTF&TUJNBUPSFYUSBDULFZQPJOUT w HFOFSBUJOHqPX TUBHFUBLFT
DPOEJUJPOJNBHF UBSHFUQPTF BTJOQVU TUBHFUBLFT DPOEJUJPOJNBHF DPBSTFSFTVMU BTJOQVU w (FOFSBUPSTGPSTUBHFTBSFMJLF6/FU 17 -
1( w PCKFDUJWF w .BTLFE-MPTTˠ4UBHF w -OPSNPG DPBSTFUBSHFU Y.# w
"EWFSTBSJBMMPTTˠ4UBHF w .BTLFE-MPTTˠ4UBHF w -OPSNPG SFpOFEUBSHFU Y.# 18 MB -
'BTIJPO("/ JDDW w TXBQQJOHDMPUIFTBDDPSEJOHUPHJWFOUFYU w OFXEBUBTFUBWBJMBCMFJO%FFQ'BTIJPO BDBEFNJDVTF w OPUSFRVJSFQBJSFEEBUBTFU
w SFRVJSFTFNBOUJDTFHNFOUBUJPO BCCSFWJBUJPO44 w 44NPEFMBSFUSBJOFEPO"53EBUBTFU w CBTF$//JT7(( 19 -
'BTIJPO("/ JDDW w FYUSBDUEFTJHODPEJOH B W BOE44NBQ4 w BWFDUPSPGCJOBSZBUUSJCVUFTPGJOQVUJNBHF TPNFBUUSJCVUFTBSFQSFTFSWFEXIJMFHFOFSBUJPO
w WWFDUPSPGFODPEFEUFYU w 4QFSTPOʟTTFHNFOUBUJPONBQ w IBJS GBDF VQQFSDMPUIFT QBOUTTIPSUT FUD 20 -
'BTIJPO("/ w TIBQF(FOFSBUPSUBLFTʠEPXOTBNQMFE4ʡN 4 OPUSBX4 w N 4 XPSLTBTTQBUJBMDPOTUSBJOU w
HFOFSBUF44NBQ 4 XIJDIJTGPSUBSHFUJNBHF w JNBHF3FOEFSFSUBLFT4BOEEFTJHODPEJOHBTJOQVU 21 -
7*50/ w CFUUFSUIBO;BMBOEP("/ w BMTP 7*50/VTFFODPEFSEFDPEFSOFUTJOTUFBEPG("/T w SFRVJSF)VNBO1BSTFS OPU1PTF&TUJNBUPS w
SFTIBQFDMPUIFTNBTLUPpUUBSHFUDMPUIFTCZ514 w SFEVDFUIFBSFBUPESBXOFXMZBTQPTTJCMF w DPBSTFUPpOFHSBJOFEHFOFSBUJPONFUIPE w QFSDFQUVBMMPTTJOTUFBEPG- CFDBVTFUIFMBUUFSUFOEUPHFOFSBUFCMVSSZJNBHFTXIFOBQQMJFEUPDPMPS JNBHFT w 5PUBM7BSJBUJPO aHSBEaBMQIB w XBSQFE-OPSNPGaBMQIB 22 TPS -
7*50/ w FODPEFSHFOFSBUPSOFUXPSLT w (D DPBSTF w JOQVU QFSTPOSFQSFTFOUBUJPO
UBSHFUDMPUIFT w PVUQVUDPBSTFSFTVMUBOENBTL w (S SFpOFNFOU w JOQVU XBSQFEUBSHFU DPBSTFSFTVMU w PVUQVUBMQIBNBTL 23 -
%1*( w ʠTUBHFGSBNFXPSLUPMFBSO NBOJQVMBUBCMFFNCFEEJOHGFBUVSFTGPSBMMUISFFGBDUPSTʡ w GPSFHSPVOE w CBDLHSPVOE w QPTF
w MFBSONBQQJOH(BVTTJBOEJTUUPGFBUVSF 25 -
%1*( w 4UBHF w FYUSBDUGPSFHSPVOECBDLHSPVOEGFBUVSFTCZBQQMZJOHNBTLNBEFGSPNQPTF JOGPSNBUJPODPNQVUFECZ1&UPGFBUVSF FYUSBDUFEGSPN$// w
TQMJU'(JOUPSFHJPOT w XFʟWFHPU w QPTFGFBUVSF w '(#(GFBUVSF 27 -
%1*( w 4UBHF w QPTFEFDPEFSSFDPOTUSVDUQPTF JNBHFGSPNFODPEFEQPTFGFBUVSF w '(#(EFDPEFSSFDPOTUSVDUJOQVU
JNBHFGSPN'(T#(GFBUVSFT DPODBUFOBUFEXJUIQPTFJNBHF 29 -
%1*( w -PTTGPSTUBHF w QPTFSFDPOTUSVDUJPO- w '(#( w BEWFSTBSJBMMPTTXSUGFBUVSFT w
%JTDSJNJOBUFXIFUIFSJOQVUJTGSPNFODPEFSTPS(BVTTJBO w -OPSNXSUJOQVUBOESFDPOTUSVDUFEJNBHF 30 -
%FGPSNBCMF("/ w GPDVTPODIBOHJOHQPTF w QSPQPTF/FBSFTU/FJHICPS-PTT DBOCFVTFEJOTUFBEPG--MPTT w UPMFSBUFUPTNBMMTQBUJBMNJTBMJHONFOUT w TJNJMBSUPQFSDFQUVBMMPTTJOUIBUVTJOH*NBHF/FUQSFUSBJOFE
7(( SFMV@VTFE 32 I haven’t understood this well.