Creating Creative Machines (v0)

In which I present a quick rundown on some attempts to beat The Herval Test.

Transcript

1. Creating creative machines (sort of)
2. About me @herval (ē´val) herval.github.io
3. The Turing Test "Can a machine answer your questions so convincingly that you couldn’t tell it apart from a human?" Developed by Alan Turing in 1950
4. The Turing Test - a winner?
5. The Turing Test - a "winner"? twitter.com/tayandyou (currently offline for being a total jerk)
6. The Herval Test "Can a machine create original content so convincingly terrible that you couldn’t tell it apart from a really bad human writer?"
7. Markov chains "a random process that undergoes transitions from one state to another on a state space. The probability distribution of the next state depends only on the current state and not on the sequence of events that preceded it." Andrey Markov (1856-1922) Simple weather prediction: a sunny day is followed by another sunny day 90% of the time; a rainy day followed by another rainy day 50% of the time
8. Markovian Literature - Load text from books from the Gutenberg Project into a Markov Chain - Extract sequences of 1-3 phrases - Tweet twitter.com/markovian_lit github.com/herval/markovian_literature
9. Markovian Literature :-)
10. Markovian Literature :-( Basically a spam bot A what?!
11. Herval Test - FAILED
12. Recurrent Neural Networks A neural network where connections between units form a directed cycle. This creates an internal state of the network which allows it to exhibit dynamic temporal behavior. RNNs can use their internal memory to process arbitrary sequences of inputs. This makes them applicable to tasks such as unsegmented connected handwriting recognition or speech recognition. www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns
13. LSTM (Long Short-Term Memory) A RNN with memory Good for classifying and predicting time-series karpathy.github.io/2015/05/21/rnn-effectiveness
14. Unforgiven Swift - Train a 200 neuron LSTM with all lyrics from Metallica and Taylor Swift letter by letter - Extract sequences of random sizes - Post to Tumblr unforgiven-swift.tumblr.com github.com/herval/unforgiven-swift
15. Unforgiven Swift - training
16. Unforgiven Swift - not quite there yet Overfitted to Metallica, hey hey Not quite english... yet?
17. Haikuzao - Train an LSTM with Two layers of 100 neurons with a database of 5k Haikus - Extract Haikus - Tweet twitter.com/haikuzao github.com/herval/haikuzao
18. Haikuzao :-) Invented words
19. Haikuzao :-( Way too many invented words!
20. Next steps - Use a bigger dataset - Reinforced learning (use signals such as Likes to reinforce the network) - Evolve the topologies & learning parameters automatically (NEAT) - ??? en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies
21. References http://cs.stanford.edu/people/karpathy/convnetjs http://deeplearning4j.org http://karpathy.github.io/2015/05/21/rnn-effectiveness http://mathworld.wolfram.com/MarkovChain.html http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns