Geoffrey Litt
May 07, 2017
700

# ENHANCE!! - Upscaling Images with Neural Networks

When characters on a TV show “enhance!” a blurry image, you probably laugh and tell your friends that it’s impossible to do that in real life. But over the past year, deep learning research has actually made this kind of possible! How will you explain this to your friends!?

In this talk, you’ll get an intuitive introduction to generative adversarial networks, a new machine learning technique that’s surprisingly good at upscaling images. You’ll learn how these systems are inspired by human art forgers, and how you can use them to do other things like transform a horse into a zebra, convert a sentence into a photo, and so much more!

May 07, 2017

## Transcript

data
10. ### Neural network training A: Neural network’s guess B: Real answer

Per-pixel subtraction Add up all the pixels Total per-pixel difference Minimize this!
11. ### Neural network training Loss function: “Minimize the total per-pixel difference

between the two images”

18. ### A new loss function ??? “Minimize the total per-pixel difference

between your guess and the real answer… But also make your guess look sharp and realistic” Loss = Total Per-pixel difference +

20. ### Adversarial games I’ll develop better counterfeit detection techniques! I’ll develop

better counterfeiting techniques!
21. ### Generator GAN structure { { Training example B: Real answer

A: Neural network’s guess
22. ### Discriminator Generator GAN structure { { Training example B: Real

answer Probability of fake Goal: differentiate fake vs. real with 100% accuracy Goal: Minimize discriminator accuracy A: Neural network’s guess
23. ### Discriminator Generator GAN structure { { Training example B: Real

answer Probability of fake A: Neural network’s guess “Here’s how you could make a better fake next time”

29. ### Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Jun-Yan Zhu*, Taesung

Park*, Phillip Isola, Alexei A. Efros
30. ### Image-to-Image Translation with Conditional Adversarial Networks Phillip Isola, Jun-Yan Zhu,

Tinghui Zhou, Alexei A. Efros https://affinelayer.com/pixsrv/
31. ### Image-to-Image Translation with Conditional Adversarial Networks Phillip Isola, Jun-Yan Zhu,

Tinghui Zhou, Alexei A. Efros https://affinelayer.com/pixsrv/
32. ### StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial

Networks Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaolei Huang, Xiaogang Wang, Dimitris Metaxas