Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ColorTransfer: Bringing Life to Black & White Photos

ColorTransfer: Bringing Life to Black & White Photos

This is the slideshow from my presentation about using convolutional neural networks for adding color to black-and-white photos.

https://www.meetup.com/Hacking-Machine-Learning/events/249681231/

Preslav Rachev

May 24, 2018
Tweet

More Decks by Preslav Rachev

Other Decks in Programming

Transcript

  1. Hi there! ! My name is Preslav I am a

    full-stack backend and mobile engineer… … and a data scientist by heart
  2. All of my sketches shared the same idea: start with

    simple rules and apply them over 1000s of iterations
  3. Challenge: Make a CNN that takes 1000s*M*N*256 of input points,

    and generates 1*M*N*2563 outputs Without spending a fortune on hardware
  4. Source: How to colorize black & white photos with just

    100 lines of neural network code Lab is a 3-dimensional representation of the visible color space: L (lightness 0-100) / a (green-red -127:127) / b (blue-yellow -127:127)
  5. Put simply, by using Lab instead of RGB, we can

    reduce our problem space 256 times! And get a BW layer for free!
  6. If we think about the problem as classification instead, an

    appropriate loss function might have a better chance at picking separate colors
  7. Challenge: Turn the a and b layers of the source

    images into a single layer of class labels
  8. By quantizing to a grid of 10, we lost a

    fraction of the visual quality, but reduced the problem space 100 times!
  9. In fact, one of the cornerstone studies in the field,

    has managed to reduce the ab space to a mere 313 distinct colors! Recall that the non-quantized ab space has 65536 possible values
  10. Having reduced the color space, we can now treat colorization

    like multi-label classification. Each pixel in the input image would represent one of 313 classes: 0, 0, 0, 0, 0, … 0, 1, 0, 0, 0 M*N 0, 0, 1, 0, 0, … 0, 0, 0, 0, 0
  11. Open Source All the Things! • Stepping on the shoulders

    of giants in the ML community • My models are still far from ideal, but I am trying to open-source every step of this journey • Repo: https://github.com/preslavrachev/nn-photo-colorization (https://bit.ly/ 2GNCq2W) • My main challenges right now are finding more time and computing resources. • Using a free Kaggle kernel to do all the training • That said, I would love to work on this project full-time • Advice: Don’t be scared by ML. Choose a topic of your own liking, and it will be your driving force.
  12. • Zhang, Richard, Phillip Isola, and Alexei A. Efros. “Colorful

    image colorization.” European Conference on Computer Vision. Springer, Cham, 2016. • https://github.com/richzhang/colorization • Wallner, E.: How to colorize black & white photos with just 100 lines of neural network code Medium.com, Oct 29, 2017 • Dahl, R.: Automatic Colorization Tinyclouds.org, Jan, 2016 • Cheng, Z., Yang, Q., Sheng, B.: Deep colorization. In: Proceedings of the IEEE International Conference on Computer Vision. (2015) 415–423 • Zhang, Richard, et al. "Real-time user-guided image colorization with learned deep priors." arXiv preprint arXiv:1705.02999 (2017). • https://github.com/junyanz/interactive-deep-colorization • Karpathy, Andrej. "Cs231n convolutional neural networks for visual recognition." Neural networks 1 (2016).