Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ENHANCE!! - Upscaling Images with Neural Networks

ENHANCE!! - Upscaling Images with Neural Networks

When characters on a TV show “enhance!” a blurry image, you probably laugh and tell your friends that it’s impossible to do that in real life. But over the past year, deep learning research has actually made this kind of possible! How will you explain this to your friends!?

In this talk, you’ll get an intuitive introduction to generative adversarial networks, a new machine learning technique that’s surprisingly good at upscaling images. You’ll learn how these systems are inspired by human art forgers, and how you can use them to do other things like transform a horse into a zebra, convert a sentence into a photo, and so much more!

7a6310a990212e2b392d95a67855afc2?s=128

Geoffrey Litt

May 07, 2017
Tweet

Transcript

  1. ENHANCE!! @geoffreylitt

  2. Entertainment Accuracy of portraying computers

  3. None
  4. ENHANCE, PLEASE

  5. None
  6. None
  7. Bicubic interpolation

  8. Bicubic interpolation

  9. DO BETTER

  10. NEURAL NETWORKS!!!

  11. Neural network upscaling

  12. Neural network training { } , { } , Training

    data
  13. Neural network training A: Neural network’s guess B: Real answer

    Per-pixel subtraction Add up all the pixels Total per-pixel difference Minimize this!
  14. Neural network training Loss function: “Minimize the total per-pixel difference

    between the two images”
  15. Neural network Code: https://github.com/david-gpu/srez

  16. Neural network Bicubic interpolation

  17. MORE. ENHANCE.

  18. A: Neural network’s guess B: Real answer Per-pixel subtraction

  19. A: Neural network’s guess B: Real answer Really high loss

  20. A: Neural network’s guess B: Real answer Lower loss

  21. A new loss function ??? “Minimize the total per-pixel difference

    between your guess and the real answer… But also make your guess look sharp and realistic” Loss = Total Per-pixel difference +
  22. GENERATIVE ADVERSARIAL NETWORKS

  23. Adversarial games I’ll develop better counterfeit detection techniques! I’ll develop

    better counterfeiting techniques!
  24. Generator GAN structure { { Training example B: Real answer

    A: Neural network’s guess
  25. Discriminator Generator GAN structure { { Training example B: Real

    answer Probability of fake Goal: differentiate fake vs. real with 100% accuracy Goal: Minimize discriminator accuracy A: Neural network’s guess
  26. Discriminator Generator GAN structure { { Training example B: Real

    answer Probability of fake A: Neural network’s guess “Here’s how you could make a better fake next time”
  27. low-res bicubic GAN Real answer

  28. GAN-trained generator Generator

  29. GAN results original GAN Per-pixel diff neural network

  30. GOOD JOB

  31. GAN Real answer low-res

  32. None
  33. None
  34. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Jun-Yan Zhu*, Taesung

    Park*, Phillip Isola, Alexei A. Efros
  35. None
  36. Image-to-Image Translation with Conditional Adversarial Networks Phillip Isola, Jun-Yan Zhu,

    Tinghui Zhou, Alexei A. Efros https://affinelayer.com/pixsrv/
  37. Image-to-Image Translation with Conditional Adversarial Networks Phillip Isola, Jun-Yan Zhu,

    Tinghui Zhou, Alexei A. Efros https://affinelayer.com/pixsrv/
  38. StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial

    Networks Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaolei Huang, Xiaogang Wang, Dimitris Metaxas
  39. Thanks! @geoffreylitt