Infinite State Machine

Infinite State Machine

Slides for my Clojure/conj talk on machine learning and poetry.

Facce030b679bda34eb7c64885a741fc?s=128

Eric Weinstein

October 13, 2017
Tweet

Transcript

  1. (defn talk [m] (-> m (assoc :title "Infinite State Machine")

    (assoc :speaker "Eric Weinstein") (assoc :conf "Clojure/conj") (assoc :city "Baltimore, MD, USA") (assoc :date "13 October 2017")))
  2. Part 0: Hello!

  3. About Me (def eric-weinstein {:employer "Fox Networks Group" :github "ericqweinstein"

    :twitter "ericqweinstein" :website "ericweinste.in"}) 30% off with CLOJURECONJ30! ->>
  4. None
  5. Agenda • Poems about the machine • Poems with the

    machine • Poems by the machine (Markov Chain + RNN) • Questions (if we have time)
  6. Part 1: About

  7. Poem About the Machine • Isn’t this a programming conference?

    • Eliot and the objective correlative • This poem is called “Memory Leak”
  8. The machine stays up all night writing poems smoking some

    it spawns a billion threads of execution little tales it tells itself in tale zero it sails from island to island in an archipelago linked
  9. by green arrows of algae In tale one it is

    a blue fork of lightning whose ends reach into the head of every child in the neighborhood and install imaginary friends a sheep a robot
  10. made of bottle caps The machine constructs these artificial histories

    but never deletes one One memory runs into the next bioluminescent algae fill the neighborhood pools a robot sails for the New World
  11. in a stiff breeze The humans wake to find one

    billion machine poems overflowing the hard drive the last one reading I would grow human teeth of my own and dissolve them I would send signals directly into your heart
  12. Part 2: With

  13. Poems && Programs • Many parallels between code and poems

    (not just correct syntax, but style, idiom, line breaks, &c) • “REMCy.clj” is an example of an executable poem
  14. ;; REMCy.clj ;; ;; Give me your hands, if we

    be friends. ;; -- Robin Goodfellow (def og; the window "To withstand the presence of ghosts." (let [us "Know that they all lived"; & the-window "is open"] [us, the-window])) (defn f[or-me] (let [the-ending-be " happily ever after"] [the-ending-be, or-me])) (def o "Rest now, the ending is written on the tallest tree.") (let [me og;re able " "; & open hands-inef (f able)] o (str;ange! We are haunted. (first me) (first (f able)))); &
  15. Part 3: By

  16. (Poem.) • Training corpuses (Keats, Shakespeare) • Markov chain •

    RNN (Recurrent Neural Network)
  17. Corpus • Poems by John Keats (1817) • 2_076 lines,

    13_660 words, 79_409 characters
  18. Markov Chain • A Markov chain is a stochastic, “memoryless”

    process • The Markov property: we can get to our next state only knowing our current state • (You can sort of cheat by baking arbitrary history into the current state)
  19. Prose Poem by A. Machine Yet this happy burthens Eric.

    I echo back each sting Thrown by thy vigils keep 'Mongst boughs pavillion'd, where sheeted lightning plays; Or, on a train Of whitest arms in these joys That through half veil'd face of his latest breath, While from Armida's bowers, And, warrior, it e'er will keep A fragile dew-drop on the room like something from his trappings glow Of this wonder of Despair, Strive for lo! I feel their soft "Lydian airs," And fresh sward beneath it, (For knightly casque are thirsty every hour. And can make "a sun-shine in throngs before him away, Than the goodliest view a Poet, sure shall be alone. Yet these scribblings might call my spirit shroud, Sweet too upheld the supreme of May seem a flower, into a gentle doings: They should not the eastern dimness, And his star-cheering voice sweetly they occasion; 'tis I marvel much more beautiful, more pleasing, a new sun-rise.
  20. RNN • Neural network with memory (unlike a Markov chain)

    • DL4J: https://deeplearning4j.org/ • Shakespeare: 124_787 lines, 904_061 words, 5_589_889 characters
  21. Image credit: https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/ content/image_folder_6/recurrent.jpg

  22. Humans Like Chocolate, Right? Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has- weird-ideas-about-what

  23. Humans Like Chocolate, Right? Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has- weird-ideas-about-what

  24. To Serve (Hu)man(s) Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has- weird-ideas-about-what

  25. To Serve (Hu)man(s) Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has- weird-ideas-about-what

  26. To Serve (Hu)man(s) • ¼ cup white seeds • 1

    cup mixture • 1 teaspoon juice • 1 chunks • ¼ lb fresh surface • ¼ teaspoon brown leaves • ½ cup with no noodles • 1 round meat in bowl Credit: https://www.dailydot.com/unclick/neural-network-recipe-generator/
  27. I Dub Thee: Stargoon Credit: http://lewisandquark.tumblr.com/post/161854386267/neural-networks-can-name- guinea-pigs

  28. I Dub Thee: Stargoon Credit: http://lewisandquark.tumblr.com/post/161854386267/neural-networks-can-name- guinea-pigs

  29. I Dub Thee: Stargoon Credit: http://lewisandquark.tumblr.com/post/161854386267/neural-networks-can-name- guinea-pigs

  30. The Unreasonable Effectiveness of Recurrent Neural Networks • Blog post

    by Andrej Karpathy (May 2015) • “There’s something magical about Recurrent Neural Networks (RNNs).” • http://karpathy.github.io/2015/05/21/rnn- effectiveness/ • https://github.com/karpathy/char-rnn (used to generate the recipes and guinea pig names)
  31. Show Me the Code

  32. 10 Mini Batches loif yeund, wochesther anftaome nooceiniIs beyloalt, thelveokk

    theouossir thochenplad my ard, aty lotheneud, somenh. EEPeCND. Ons, id theor oudoly Notfof aolllyLeagaoner lirchin, ithen, pay sudthas eomee sfolbfetf dotheut, Af thowrne shame, houd palid. EOTMOS.OOtir, hol mesoalgteus ou
  33. 50 Mini Batches Ast beanbly earther, That is fay that

    hath a begwer. And, the face! Exeunt Frods. Let lingher me! Lothfy here'nst with honours, by thee. GLOUCESTER. Henply, cancut these orwerd. Plence was he likesess puince, Det, woulst of brother. He will sech the exate,
  34. 160 Mini Batches SILVIA. The sleep- Villain; so shouty I

    desire Like his courtesy'r's thrice of Curst; And four lady, doth be hone'ring-broke of this, And so who ley subdue. My comforts. QUEEN ELIZABETH. Where's Bugidde
  35. Image credit: https://bukk.it/notbad.gif

  36. TL;DPA • Poems are emotive programs and programs are poems

    for the machine • Poems can be programs (and vice-versa!) • Machines can write poems (via Markov chains, RNNs, and more) • The future: Cortex (thanks, Joyce Xu!)
  37. None
  38. Questions? (def eric-weinstein {:employer "Fox Networks Group" :github "ericqweinstein" :twitter

    "ericqweinstein" :website "ericweinste.in"}) 30% off with CLOJURECONJ30! ->>