$30 off During Our Annual Pro Sale. View Details »

Deep Learning Image Manipulation

Deep Learning Image Manipulation

Illustrated guide to some image manipulation methods, with demonstration.

Leszek Rybicki

May 18, 2017
Tweet

More Decks by Leszek Rybicki

Other Decks in Research

Transcript

  1. %FFQ-FBSOJOH *NBHF.BOJQVMBUJPO BOJMMVTUSBUFEHVJEF .-,JUDIFO

  2. "CPVUNF w -FT[FL3ZCJDLJ w HJUIVC!MVOBSEPH w CPSOJO1PMBOE w .-3FTFBSDIFSBU$PPLQBE w

    *MJLFOBUUP
  3. DBSFFST!DPPLQBEDPN 8BOUUPXPSLXJUIVT

  4. $POWPMVUJPOBM "SJUINFUJD OCIKE

  5. *NBHFTUPGFBUVSFT

  6. $POWPMVUJPO http://deeplearning.net/software/theano/tutorial/conv_arithmetic.html input output input output kernel

  7. 4USJEF http://deeplearning.net/software/theano/tutorial/conv_arithmetic.html 2px 2px 2px 2px

  8. 1BEEJOH http://deeplearning.net/software/theano/tutorial/conv_arithmetic.html 2px 2px

  9. 4USJEF QBEEJOH http://deeplearning.net/software/theano/tutorial/conv_arithmetic.html

  10. 5SBOTQPTFE http://deeplearning.net/software/theano/tutorial/conv_arithmetic.html simulated here with padding also called “deconvolution” “fractional

    stride”
  11. %PXOTBNQMJOH features or small resolution image convolutional layer or layers

    RGB image input output
  12. 6QTBNQMJOH upsampling CNN layer or layers RGB image features or

    small resolution image input output
  13. &ODPEFS%FDPEFS D E image in Decoder Encoder image out feature

    space
  14. 'VMMZ$POOFDUFE $MBTTJpFS approve loan reject class data or features also

    called “Dense” layer
  15. $//$MBTTJpFS food person plant other AlexNet, LeNet, VGG…

  16. 'PPE/FU ™ food not food

  17. @teenybiscuit

  18. None
  19. @teenybiscuit

  20. @teenybiscuit

  21. @teenybiscuit

  22. @teenybiscuit

  23. @teenybiscuit

  24. (FOFSBUJWF "EWFSTBSJBM /FUXPSLT

  25. Generator Discriminator https://speakerdeck.com/lunardog/deep-convolutional-voight-kampf-test “Couple of bots studying for the Turing

    Test”
  26. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks Alec

    Radford, Luke Metz, Soumith Chintala (Submitted on 19 Nov 2015 (v1), last revised 7 Jan 2016 (this version, v2)) https://arxiv.org/abs/1511.06434
  27. Generator Discriminator G MPPLTMFHJU UPUBMMZTIPQQFE D

  28. G SFBM GBLF D D(G(noise)) ˠ real (FOFSBUPSUSBJOJOH Discriminator acts

    as the teacher
  29. G SFBM GBLF D SFBM GBLF D D(G(noise)) ˠ fake

    D(photo) ˠ real %JTDSJNJOBUPSUSBJOJOH Generator provides negative examples
  30. None
  31. https://www.youtube.com/watch?v=rs3aI7bACGc ©Yota Ishida

  32. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks Alec

    Radford, Luke Metz, Soumith Chintala (Submitted on 19 Nov 2015 (v1), last revised 7 Jan 2016 (this version, v2)) https://arxiv.org/abs/1511.06434
  33. $POEJUJPOBM ("/T

  34. G NBMF GFNBMF DIJME FMEFSMZ G(noise | conditions) $POEJUJPOBM(FOFSBUPS

  35. SJHIU XSPOH NBMF GFNBMF DIJME FMEFSMZ D $POEJUJPOBM%JTDSJNJOBUPS

  36. SJHIU XSPOH NBMF GFNBMF DIJME FMEFSMZ D SJHIU XSPOH NBMF

    GFNBMF DIJME FMEFSMZ SJHIU XSPOH NBMF GFNBMF DIJME FMEFSMZ D D
  37. SJHIU XSPOH D $POEJUJPOBM("/ https://arxiv.org/abs/1411.1784 Conditional Generative Adversarial Nets Mehdi

    Mirza, Simon Osindero (Submitted on 6 Nov 2014) Generator Discriminator NBMF GFNBMF DIJME FMEFSMZ G NBMF GFNBMF DIJME FMEFSMZ same condition
  38. G NBMF GFNBMF DIJME FMEFSMZ SJHIU XSPOH NBMF GFNBMF DIJME

    FMEFSMZ D $POEJUJPOBM("/ Discriminator Generator
  39. https://www.faceapp.com/ Disclaimer: FaceApp authors don’t disclose their method. This is

    only my guess. It may have nothing to do with GANs. original
  40. original https://www.faceapp.com/

  41. https://www.faceapp.com/ original

  42. "SUJTUJD4UZMF5SBOTGFS Improved!

  43. https://prisma-ai.com/

  44. https://prisma-ai.com/ https://prisma-ai.com/

  45. https://prisma-ai.com/ https://prisma-ai.com/

  46. https://prisma-ai.com/ https://prisma-ai.com/

  47. https://arxiv.org/abs/1603.08155 transformation network loss network Gram matrices in feature space

    pre-trained content image style image
  48. “Gram matrices in feature space” https://en.wikipedia.org/wiki/Gramian_matrix

  49. https://www.youtube.com/watch?v=xVJwwWQlQ1o

  50. $ZDMF("/

  51. https://github.com/junyanz/CycleGAN

  52. https://github.com/junyanz/CycleGAN

  53. https://github.com/junyanz/CycleGAN

  54. (FOFSBUPS transformation network https://arxiv.org/abs/1603.08155

  55. GBLF IPSTF GBLF IPSTF … %JTDSJNJOBUPS fully convolutional judges patches

    
 of the input image https://arxiv.org/abs/1603.08155
  56. "EWFSTBSJBM-PTT X F G Y GBLF [FCSB GBLF [FCSB …

    GBLF IPSTF GBLF IPSTF … X(F(horse)) ˠ classify as zebra Y(F(zebra)) ˠ classify as horse
  57. $ZDMF-PTT G F G(F(image))ˠ the same image F G F(G(image))ˠ

    the same image
  58. https://www.youtube.com/watch?v=9reHvktowLY

  59. 5IF&OE