What if Picasso was a Robot? Teaching art to the machine

Machine learning is revolutionizing the world as we know it: recommender systems that know exactly what we’re into, ear phones capable of translating entire conversations on the fly and even predictive power to help us guess what the future may look like. But what about art? Can we teach machines how to be creative and come up with new art styles? In this talk you’ll see what happens when we mix computational power and the greatest works of art the world has ever seen! And guess what? All that using Javascript. Hold tight, art class is about to start!

Transcript

1. What if Picasso
   was a robot?
   Teaching art to the machine
   @silveira_bells
   

   View Slide

2. X
   

   View Slide

3. X
   

   View Slide

4. X
   

   View Slide

5. X
   

   View Slide

6. Mini Isa
   

   View Slide

7. X
   

   View Slide

8. View Slide

9. View Slide

10. View Slide

11. X
    

    View Slide

12. Teaching the computer
    how to make art ✨
    

    View Slide

13. “What do you mean, Isa? Wtf?”
    

    View Slide

14. Isa Silveira
    @silveira_bells
    Developer @Work&Co
    Carioca, math freak, slalom skater,
    peculiar sense of humor @life
    

    View Slide

15. Art
    01
    

    View Slide

16. What is art?
    

    View Slide

17. 1. The quality, production, expression, or realm, 

    according to aesthetic
    principles, of what is beautiful, appealing, or of 

    more than ordinary significance. [Dictionary]
    2. “Art is either plagiarism or revolution.”
    [Paul Gauguin]
    3. “Art is harmony." [Georges Seurat]
    4. “[…] And it is upon this capacity of man to
    receive another man’s expression of feeling and
    experience those feelings himself, that is what
    the activity of art is based.” [Leo Tolstoy]
    

    View Slide

18. Does it have to be beautiful or original?
    

    View Slide

19. Does it have to be complex or
    have monetary value?
    

    View Slide

20. Does it have to be made
    by an artist?
    

    View Slide

21. What if the artist has an idea and the art 

    is created by someone else?
    

    View Slide

22. One and Three Chairs - Joseph Kosuth
    1965
    

    View Slide

23. View Slide

24. Kosuth didn’t make the chair, take the photo,
    or wrote the definition
    

    View Slide

25. So where’s the art?
    

    View Slide

26. Machine
    Learning
    02
    

    View Slide

27. What is
    machine learning?
    

    View Slide

28. Machine learning is an application of artificial intelligence (AI) that
    provides systems the ability to automatically learn and improve from
    experience without being explicitly programmed. Machine learning
    focuses on the development of computer programs that can access data
    and use it learn for themselves. The process of learning begins with
    observations or data, such as examples, direct experience, or instruction,
    in order to look for patterns in data and make better decisions in the future
    based on the examples that we provide. The primary aim is to allow the
    computers learn automatically without human intervention or assistance
    and adjust actions accordingly.
    

    View Slide

29. It allows the machine to
    actually learn.
    With ML algorithms, it’s possible to parse data, learn from it and make mass
    predictions or classifications. Instead of creating an algorithm full of specific rules
    on how to execute a certain task, we train the machine with huge datasets so it can
    learn how to execute it on its own.
    

    View Slide

30. We can use machine learning to
    draw conclusions about large
    chunks data automatically.
    

    View Slide

31. Several kinds of algorithms to each
    kind of problem
    

    View Slide

32. Our mission
    

    View Slide

33. Generating new art images from
    existing art works
    

    View Slide

34. 1. A powerful algorithm;
    2. A couple thousands of paintings;
    3. A mighty processor;
    4. 6 buckets of patience;
    5. 2 cups of persistence;
    6. Love to taste ❤
    

    View Slide

35. 1. A powerful algorithm;
    2. A couple thousands of paintings;
    3. A mighty processor;
    4. 6 buckets of patience;
    5. 2 cups of persistence;
    6. Love to taste ❤
    

    View Slide

36. Deep Convolutional Generative
    Adversarial Networks
    

    View Slide

37. View Slide

38. Capable of creating new images
    from existing ones
    

    View Slide

39. 2 convolutional neural networks
    that keep battling each other
    

    View Slide

40. “Eita™, what do you mean?"
    

    View Slide

41. Baby steps!
    

    View Slide

42. Each algorithm works based

    on a model
    

    View Slide

43. Types of models
    

    View Slide

44. On the discriminative model we
    want to formulate descriptions to
    our results based on the data and
    parameters we have, in other words,
    finding patterns to classify or
    predict future results.
    On the generative model, we want to
    generate new samples based on what we
    learned about the data previously fed to
    the algorithm on the training part.
    Generative model
    Discriminative model
    Types of models
    

    View Slide

45. In this algorithm we’ll use both

    of these models
    

    View Slide

46. E.g.: generating images of IDs
    

    View Slide

47. Neural Net 1 - The discriminator
    His job is to recognize if an
    image is a legitimate ID
    

    View Slide

48. Neural Net 2 - The generator
    His job is to create

    images of IDs
    Photoshop
    mad skillz
    

    View Slide

49. View Slide

50. They suck at the beginning
    

    View Slide

51. Deep Conv
    Neural Network
    True!
    Seems legit
    Discriminator
    

    View Slide

52. View Slide

53. Deep Conv
    Neural Network
    False!
    Super fake
    Discriminator
    

    View Slide

54. Round 2
    

    View Slide

55. Reversed
    Deep Conv
    Neural Network
    500 numbers
    Generator
    

    View Slide

56. Over and over and over
    

    View Slide

57. View Slide

58. View Slide

59. View Slide

60. https://github.com/alantian/ganshowcase
    

    View Slide

61. 1. A powerful algorithm;
    2. A couple thousands of paintings;
    3. A mighty processor;
    4. 6 buckets of patience;
    5. 2 cups of persistence;
    6. Love to taste ❤
    

    View Slide

62. View Slide

63. My first idea was to filter by Picasso
    

    View Slide

64. But there were only 1130 paintings
    

    View Slide

65. Plan B: filtering by genre
    

    View Slide

66. 24.832 images of landscapes
    

    View Slide

67. DIR_PATH=“../images”
    DATA_FILE=“dataset.npz”
    SIZE=64
    

    View Slide

68. $ ./datatool.py --task ./src --dir_path $DIR_PATH
    —npz_path $DATA_FILE --size $SIZE
    

    View Slide

69. 1. A powerful algorithm;
    2. A couple thousands of paintings;
    3. A mighty processor;
    4. 6 buckets of patience;
    5. 2 cups of persistence;
    6. Love to taste ❤
    

    View Slide

70. View Slide

71. 4GB GPU
    

    View Slide

72. View Slide

73. 1. A powerful algorithm;
    2. A couple thousands of paintings;
    3. A mighty processor;
    4. 6 buckets of patience;
    5. 2 cups of persistence;
    6. Love to taste ❤
    

    View Slide

74. Training
    

    View Slide

75. DATA_FILE_SIZE_64=“dataset_64.npz”
    DCGAN64_OUT="./dist"
    

    View Slide

76. $ ./chainer_dcgan.py \
    --arch dcgan64 \
    --image_size 64 \
    --npz_path $DATA_FILE_SIZE_64 \
    --out $DCGAN64_OUT
    

    View Slide

77. The whole training took 16 hours
    

    View Slide

78. View Slide

79. Stopped to make some
    adjustments, repeat cycle
    

    View Slide

80. $ ./dcgan_chainer_to_keras.py \
    --arch dcgan64 \
    --chainer_model_path $DCGAN64_OUT/SmoothedGenerator.npz \
    --keras_model_path $DCGAN64_OUT/Keras_SmoothedGenerator.h5 \
    --tfjs_model_path $DCGAN64_OUT/tfjs_SmoothedGenerator
    

    View Slide

81. $ ./dcgan_chainer_to_keras.py \
    --arch dcgan64 \
    --chainer_model_path $DCGAN64_OUT/SmoothedGenerator.npz \
    --keras_model_path $DCGAN64_OUT/Keras_SmoothedGenerator.h5 \
    --tfjs_model_path $DCGAN64_OUT/tfjs_SmoothedGenerator
    

    View Slide

82. let all_model_info = {
    dcgan64: {
    description: 'DCGAN, 64x64 (16 MB)',
    model_url: “/dist/tfjs_SmoothedGenerator/model.json”,
    model_size: 64,
    model_latent_dim: 128,
    draw_multiplier: 4,
    animate_frame: 200,
    },
    };
    

    View Slide

83. View Slide

84. Conclusions
    

    View Slide

85. View Slide

86. 84 64x64 samples were generated
    

    View Slide

87. High variance of quality in each
    generated art work
    

    View Slide

88. Some styles are a lot harder to
    reproduce than others
    

    View Slide

89. Creative
    Computers
    03
    

    View Slide

90. We have to get used with a future

    where machines can also do creative work
    

    View Slide

91. View Slide

92. View Slide

93. https://youtu.be/LSHZ_b05W7o
    

    View Slide

94. There’s always the training phase,
    but also a certain “creative unpredictability”
    

    View Slide

95. View Slide

96. Art is making meaning
    

    View Slide

97. What's art to you?
    

    View Slide

98. https://github.com/bella-silveira
    

    View Slide

99. @silveira_bells
    

    View Slide

100. ¡Muchas gracias! ❤
     

     View Slide