Save 37% off PRO during our Black Friday Sale! »

What if Picasso was a Robot? Teaching art to the machine

What if Picasso was a Robot? Teaching art to the machine

Machine learning is revolutionizing the world as we know it: recommender systems that know exactly what we’re into, ear phones capable of translating entire conversations on the fly and even predictive power to help us guess what the future may look like. But what about art? Can we teach machines how to be creative and come up with new art styles? In this talk you’ll see what happens when we mix computational power and the greatest works of art the world has ever seen! And guess what? All that using Javascript. Hold tight, art class is about to start!

91b0fc08849c4128f1eed542c057fd27?s=128

Isabella Silveira

October 19, 2018
Tweet

Transcript

  1. What if Picasso was a robot? Teaching art to the

    machine @silveira_bells
  2. X

  3. X

  4. X

  5. X

  6. Mini Isa

  7. X

  8. None
  9. None
  10. None
  11. X

  12. Teaching the computer how to make art ✨

  13. “What do you mean, Isa? Wtf?”

  14. Isa Silveira @silveira_bells Developer @Work&Co Carioca, math freak, slalom skater,

    peculiar sense of humor @life
  15. Art 01

  16. What is art?

  17. 1. The quality, production, expression, or realm, 
 according to

    aesthetic principles, of what is beautiful, appealing, or of 
 more than ordinary significance. [Dictionary] 2. “Art is either plagiarism or revolution.” [Paul Gauguin] 3. “Art is harmony." [Georges Seurat] 4. “[…] And it is upon this capacity of man to receive another man’s expression of feeling and experience those feelings himself, that is what the activity of art is based.” [Leo Tolstoy]
  18. Does it have to be beautiful or original?

  19. Does it have to be complex or have monetary value?

  20. Does it have to be made by an artist?

  21. What if the artist has an idea and the art

    
 is created by someone else?
  22. One and Three Chairs - Joseph Kosuth 1965

  23. None
  24. Kosuth didn’t make the chair, take the photo, or wrote

    the definition
  25. So where’s the art?

  26. Machine Learning 02

  27. What is machine learning?

  28. Machine learning is an application of artificial intelligence (AI) that

    provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Machine learning focuses on the development of computer programs that can access data and use it learn for themselves. The process of learning begins with observations or data, such as examples, direct experience, or instruction, in order to look for patterns in data and make better decisions in the future based on the examples that we provide. The primary aim is to allow the computers learn automatically without human intervention or assistance and adjust actions accordingly.
  29. It allows the machine to actually learn. With ML algorithms,

    it’s possible to parse data, learn from it and make mass predictions or classifications. Instead of creating an algorithm full of specific rules on how to execute a certain task, we train the machine with huge datasets so it can learn how to execute it on its own.
  30. We can use machine learning to draw conclusions about large

    chunks data automatically.
  31. Several kinds of algorithms to each kind of problem

  32. Our mission

  33. Generating new art images from existing art works

  34. 1. A powerful algorithm; 2. A couple thousands of paintings;

    3. A mighty processor; 4. 6 buckets of patience; 5. 2 cups of persistence; 6. Love to taste ❤
  35. 1. A powerful algorithm; 2. A couple thousands of paintings;

    3. A mighty processor; 4. 6 buckets of patience; 5. 2 cups of persistence; 6. Love to taste ❤
  36. Deep Convolutional Generative Adversarial Networks

  37. None
  38. Capable of creating new images from existing ones

  39. 2 convolutional neural networks that keep battling each other

  40. “Eita™, what do you mean?"

  41. Baby steps!

  42. Each algorithm works based
 on a model

  43. Types of models

  44. On the discriminative model we want to formulate descriptions to

    our results based on the data and parameters we have, in other words, finding patterns to classify or predict future results. On the generative model, we want to generate new samples based on what we learned about the data previously fed to the algorithm on the training part. Generative model Discriminative model Types of models
  45. In this algorithm we’ll use both
 of these models

  46. E.g.: generating images of IDs

  47. Neural Net 1 - The discriminator His job is to

    recognize if an image is a legitimate ID
  48. Neural Net 2 - The generator His job is to

    create
 images of IDs Photoshop mad skillz
  49. None
  50. They suck at the beginning

  51. Deep Conv Neural Network True! Seems legit Discriminator

  52. None
  53. Deep Conv Neural Network False! Super fake Discriminator

  54. Round 2

  55. Reversed Deep Conv Neural Network 500 numbers Generator

  56. Over and over and over

  57. None
  58. None
  59. None
  60. https://github.com/alantian/ganshowcase

  61. 1. A powerful algorithm; 2. A couple thousands of paintings;

    3. A mighty processor; 4. 6 buckets of patience; 5. 2 cups of persistence; 6. Love to taste ❤
  62. None
  63. My first idea was to filter by Picasso

  64. But there were only 1130 paintings

  65. Plan B: filtering by genre

  66. 24.832 images of landscapes

  67. DIR_PATH=“../images” DATA_FILE=“dataset.npz” SIZE=64

  68. $ ./datatool.py --task ./src --dir_path $DIR_PATH —npz_path $DATA_FILE --size $SIZE

  69. 1. A powerful algorithm; 2. A couple thousands of paintings;

    3. A mighty processor; 4. 6 buckets of patience; 5. 2 cups of persistence; 6. Love to taste ❤
  70. None
  71. 4GB GPU

  72. None
  73. 1. A powerful algorithm; 2. A couple thousands of paintings;

    3. A mighty processor; 4. 6 buckets of patience; 5. 2 cups of persistence; 6. Love to taste ❤
  74. Training

  75. DATA_FILE_SIZE_64=“dataset_64.npz” DCGAN64_OUT="./dist"

  76. $ ./chainer_dcgan.py \ --arch dcgan64 \ --image_size 64 \ --npz_path

    $DATA_FILE_SIZE_64 \ --out $DCGAN64_OUT
  77. The whole training took 16 hours

  78. None
  79. Stopped to make some adjustments, repeat cycle

  80. $ ./dcgan_chainer_to_keras.py \ --arch dcgan64 \ --chainer_model_path $DCGAN64_OUT/SmoothedGenerator.npz \ --keras_model_path

    $DCGAN64_OUT/Keras_SmoothedGenerator.h5 \ --tfjs_model_path $DCGAN64_OUT/tfjs_SmoothedGenerator
  81. $ ./dcgan_chainer_to_keras.py \ --arch dcgan64 \ --chainer_model_path $DCGAN64_OUT/SmoothedGenerator.npz \ --keras_model_path

    $DCGAN64_OUT/Keras_SmoothedGenerator.h5 \ --tfjs_model_path $DCGAN64_OUT/tfjs_SmoothedGenerator
  82. let all_model_info = { dcgan64: { description: 'DCGAN, 64x64 (16

    MB)', model_url: “/dist/tfjs_SmoothedGenerator/model.json”, model_size: 64, model_latent_dim: 128, draw_multiplier: 4, animate_frame: 200, }, };
  83. None
  84. Conclusions

  85. None
  86. 84 64x64 samples were generated

  87. High variance of quality in each generated art work

  88. Some styles are a lot harder to reproduce than others

  89. Creative Computers 03

  90. We have to get used with a future
 where machines

    can also do creative work
  91. None
  92. None
  93. https://youtu.be/LSHZ_b05W7o

  94. There’s always the training phase, but also a certain “creative

    unpredictability”
  95. None
  96. Art is making meaning

  97. What's art to you?

  98. https://github.com/bella-silveira

  99. @silveira_bells

  100. ¡Muchas gracias! ❤