Infinite State Machine

Slides for my Clojure/conj talk on machine learning and poetry.

Transcript

1. (defn talk [m] (-> m (assoc :title "Infinite State Machine") (assoc :speaker "Eric Weinstein") (assoc :conf "Clojure/conj") (assoc :city "Baltimore, MD, USA") (assoc :date "13 October 2017")))
2. Part 0: Hello!
3. About Me (def eric-weinstein {:employer "Fox Networks Group" :github "ericqweinstein" :twitter "ericqweinstein" :website "ericweinste.in"}) 30% off with CLOJURECONJ30! ->>
5. Agenda • Poems about the machine • Poems with the machine • Poems by the machine (Markov Chain + RNN) • Questions (if we have time)
6. Part 1: About
7. Poem About the Machine • Isn’t this a programming conference? • Eliot and the objective correlative • This poem is called “Memory Leak”
8. The machine stays up all night writing poems smoking some it spawns a billion threads of execution little tales it tells itself in tale zero it sails from island to island in an archipelago linked
9. by green arrows of algae In tale one it is a blue fork of lightning whose ends reach into the head of every child in the neighborhood and install imaginary friends a sheep a robot
10. made of bottle caps The machine constructs these artificial histories but never deletes one One memory runs into the next bioluminescent algae fill the neighborhood pools a robot sails for the New World
11. in a stiff breeze The humans wake to find one billion machine poems overflowing the hard drive the last one reading I would grow human teeth of my own and dissolve them I would send signals directly into your heart
12. Part 2: With
13. Poems && Programs • Many parallels between code and poems (not just correct syntax, but style, idiom, line breaks, &c) • “REMCy.clj” is an example of an executable poem
14. ;; REMCy.clj ;; ;; Give me your hands, if we be friends. ;; -- Robin Goodfellow (def og; the window "To withstand the presence of ghosts." (let [us "Know that they all lived"; & the-window "is open"] [us, the-window])) (defn f[or-me] (let [the-ending-be " happily ever after"] [the-ending-be, or-me])) (def o "Rest now, the ending is written on the tallest tree.") (let [me og;re able " "; & open hands-inef (f able)] o (str;ange! We are haunted. (first me) (first (f able)))); &
15. Part 3: By
16. (Poem.) • Training corpuses (Keats, Shakespeare) • Markov chain • RNN (Recurrent Neural Network)
17. Corpus • Poems by John Keats (1817) • 2_076 lines, 13_660 words, 79_409 characters
18. Markov Chain • A Markov chain is a stochastic, “memoryless” process • The Markov property: we can get to our next state only knowing our current state • (You can sort of cheat by baking arbitrary history into the current state)
19. Prose Poem by A. Machine Yet this happy burthens Eric. I echo back each sting Thrown by thy vigils keep 'Mongst boughs pavillion'd, where sheeted lightning plays; Or, on a train Of whitest arms in these joys That through half veil'd face of his latest breath, While from Armida's bowers, And, warrior, it e'er will keep A fragile dew-drop on the room like something from his trappings glow Of this wonder of Despair, Strive for lo! I feel their soft "Lydian airs," And fresh sward beneath it, (For knightly casque are thirsty every hour. And can make "a sun-shine in throngs before him away, Than the goodliest view a Poet, sure shall be alone. Yet these scribblings might call my spirit shroud, Sweet too upheld the supreme of May seem a flower, into a gentle doings: They should not the eastern dimness, And his star-cheering voice sweetly they occasion; 'tis I marvel much more beautiful, more pleasing, a new sun-rise.
20. RNN • Neural network with memory (unlike a Markov chain) • DL4J: https://deeplearning4j.org/ • Shakespeare: 124_787 lines, 904_061 words, 5_589_889 characters
21. Image credit: https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/ content/image_folder_6/recurrent.jpg
22. Humans Like Chocolate, Right? Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has- weird-ideas-about-what
23. Humans Like Chocolate, Right? Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has- weird-ideas-about-what
24. To Serve (Hu)man(s) Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has- weird-ideas-about-what
25. To Serve (Hu)man(s) Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has- weird-ideas-about-what
26. To Serve (Hu)man(s) • ¼ cup white seeds • 1 cup mixture • 1 teaspoon juice • 1 chunks • ¼ lb fresh surface • ¼ teaspoon brown leaves • ½ cup with no noodles • 1 round meat in bowl Credit: https://www.dailydot.com/unclick/neural-network-recipe-generator/
27. I Dub Thee: Stargoon Credit: http://lewisandquark.tumblr.com/post/161854386267/neural-networks-can-name- guinea-pigs
28. I Dub Thee: Stargoon Credit: http://lewisandquark.tumblr.com/post/161854386267/neural-networks-can-name- guinea-pigs
29. I Dub Thee: Stargoon Credit: http://lewisandquark.tumblr.com/post/161854386267/neural-networks-can-name- guinea-pigs
30. The Unreasonable Effectiveness of Recurrent Neural Networks • Blog post by Andrej Karpathy (May 2015) • “There’s something magical about Recurrent Neural Networks (RNNs).” • http://karpathy.github.io/2015/05/21/rnn- effectiveness/ • https://github.com/karpathy/char-rnn (used to generate the recipes and guinea pig names)
31. Show Me the Code
32. 10 Mini Batches loif yeund, wochesther anftaome nooceiniIs beyloalt, thelveokk theouossir thochenplad my ard, aty lotheneud, somenh. EEPeCND. Ons, id theor oudoly Notfof aolllyLeagaoner lirchin, ithen, pay sudthas eomee sfolbfetf dotheut, Af thowrne shame, houd palid. EOTMOS.OOtir, hol mesoalgteus ou
33. 50 Mini Batches Ast beanbly earther, That is fay that hath a begwer. And, the face! Exeunt Frods. Let lingher me! Lothfy here'nst with honours, by thee. GLOUCESTER. Henply, cancut these orwerd. Plence was he likesess puince, Det, woulst of brother. He will sech the exate,
34. 160 Mini Batches SILVIA. The sleep- Villain; so shouty I desire Like his courtesy'r's thrice of Curst; And four lady, doth be hone'ring-broke of this, And so who ley subdue. My comforts. QUEEN ELIZABETH. Where's Bugidde
35. Image credit: https://bukk.it/notbad.gif
36. TL;DPA • Poems are emotive programs and programs are poems for the machine • Poems can be programs (and vice-versa!) • Machines can write poems (via Markov chains, RNNs, and more) • The future: Cortex (thanks, Joyce Xu!)
38. Questions? (def eric-weinstein {:employer "Fox Networks Group" :github "ericqweinstein" :twitter "ericqweinstein" :website "ericweinste.in"}) 30% off with CLOJURECONJ30! ->>