Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Infinite State Machine

Infinite State Machine

Slides for my Clojure/conj talk on machine learning and poetry.

Eric Weinstein

October 13, 2017
Tweet

More Decks by Eric Weinstein

Other Decks in Technology

Transcript

  1. (defn talk [m]
    (-> m
    (assoc :title "Infinite State Machine")
    (assoc :speaker "Eric Weinstein")
    (assoc :conf "Clojure/conj")
    (assoc :city "Baltimore, MD, USA")
    (assoc :date "13 October 2017")))

    View Slide

  2. Part 0: Hello!

    View Slide

  3. About Me
    (def eric-weinstein
    {:employer "Fox Networks Group"
    :github "ericqweinstein"
    :twitter "ericqweinstein"
    :website "ericweinste.in"})
    30% off with CLOJURECONJ30! ->>

    View Slide

  4. View Slide

  5. Agenda
    • Poems about the machine
    • Poems with the machine
    • Poems by the machine (Markov Chain + RNN)
    • Questions (if we have time)

    View Slide

  6. Part 1: About

    View Slide

  7. Poem About the Machine
    • Isn’t this a programming conference?
    • Eliot and the objective correlative
    • This poem is called “Memory Leak”

    View Slide

  8. The machine stays up all night writing
    poems smoking some it spawns
    a billion threads of execution
    little tales it tells itself
    in tale zero it sails from island
    to island in an archipelago linked

    View Slide

  9. by green arrows of algae
    In tale one it is a blue fork
    of lightning whose ends reach
    into the head of every child
    in the neighborhood and install
    imaginary friends a sheep a robot

    View Slide

  10. made of bottle caps The machine
    constructs these artificial histories
    but never deletes one One memory
    runs into the next bioluminescent
    algae fill the neighborhood pools
    a robot sails for the New World

    View Slide

  11. in a stiff breeze The humans wake
    to find one billion machine poems
    overflowing the hard drive the last
    one reading I would grow human
    teeth of my own and dissolve them
    I would send signals directly into your heart

    View Slide

  12. Part 2: With

    View Slide

  13. Poems && Programs
    • Many parallels between code and poems (not just
    correct syntax, but style, idiom, line breaks, &c)
    • “REMCy.clj” is an example of an executable poem

    View Slide

  14. ;; REMCy.clj
    ;;
    ;; Give me your hands, if we be friends.
    ;; -- Robin Goodfellow
    (def og; the window
    "To withstand the presence of ghosts."
    (let [us "Know that they all lived"; &
    the-window "is open"]
    [us, the-window]))
    (defn f[or-me]
    (let [the-ending-be " happily ever after"]
    [the-ending-be, or-me]))
    (def o "Rest
    now, the ending
    is written on the
    tallest tree.")
    (let [me og;re
    able " "; & open
    hands-inef (f able)]
    o
    (str;ange! We are haunted.
    (first me) (first (f able)))); &

    View Slide

  15. Part 3: By

    View Slide

  16. (Poem.)
    • Training corpuses (Keats, Shakespeare)
    • Markov chain
    • RNN (Recurrent Neural Network)

    View Slide

  17. Corpus
    • Poems by John Keats (1817)
    • 2_076 lines, 13_660 words, 79_409 characters

    View Slide

  18. Markov Chain
    • A Markov chain is a stochastic, “memoryless”
    process
    • The Markov property: we can get to our next state
    only knowing our current state
    • (You can sort of cheat by baking arbitrary history
    into the current state)

    View Slide

  19. Prose Poem by A. Machine
    Yet this happy burthens Eric. I echo back each sting Thrown by thy
    vigils keep 'Mongst boughs pavillion'd, where sheeted lightning plays;
    Or, on a train Of whitest arms in these joys That through half veil'd
    face of his latest breath, While from Armida's bowers, And, warrior, it
    e'er will keep A fragile dew-drop on the room like something from his
    trappings glow Of this wonder of Despair, Strive for lo! I feel their
    soft "Lydian airs," And fresh sward beneath it, (For knightly casque
    are thirsty every hour. And can make "a sun-shine in throngs before him
    away, Than the goodliest view a Poet, sure shall be alone. Yet these
    scribblings might call my spirit shroud, Sweet too upheld the supreme
    of May seem a flower, into a gentle doings: They should not the eastern
    dimness, And his star-cheering voice sweetly they occasion; 'tis I
    marvel much more beautiful, more pleasing, a new sun-rise.

    View Slide

  20. RNN
    • Neural network with memory (unlike a Markov chain)
    • DL4J: https://deeplearning4j.org/
    • Shakespeare: 124_787 lines, 904_061 words,
    5_589_889 characters

    View Slide

  21. Image credit: https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/
    content/image_folder_6/recurrent.jpg

    View Slide

  22. Humans Like Chocolate, Right?
    Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has-
    weird-ideas-about-what

    View Slide

  23. Humans Like Chocolate, Right?
    Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has-
    weird-ideas-about-what

    View Slide

  24. To Serve (Hu)man(s)
    Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has-
    weird-ideas-about-what

    View Slide

  25. To Serve (Hu)man(s)
    Credit: http://lewisandquark.tumblr.com/post/140508739392/the-neural-network-has-
    weird-ideas-about-what

    View Slide

  26. To Serve (Hu)man(s)
    • ¼ cup white seeds
    • 1 cup mixture
    • 1 teaspoon juice
    • 1 chunks
    • ¼ lb fresh surface
    • ¼ teaspoon brown leaves
    • ½ cup with no noodles
    • 1 round meat in bowl
    Credit: https://www.dailydot.com/unclick/neural-network-recipe-generator/

    View Slide

  27. I Dub Thee: Stargoon
    Credit: http://lewisandquark.tumblr.com/post/161854386267/neural-networks-can-name-
    guinea-pigs

    View Slide

  28. I Dub Thee: Stargoon
    Credit: http://lewisandquark.tumblr.com/post/161854386267/neural-networks-can-name-
    guinea-pigs

    View Slide

  29. I Dub Thee: Stargoon
    Credit: http://lewisandquark.tumblr.com/post/161854386267/neural-networks-can-name-
    guinea-pigs

    View Slide

  30. The Unreasonable Effectiveness
    of Recurrent Neural Networks
    • Blog post by Andrej Karpathy (May 2015)
    • “There’s something magical about Recurrent Neural
    Networks (RNNs).”
    • http://karpathy.github.io/2015/05/21/rnn-
    effectiveness/
    • https://github.com/karpathy/char-rnn (used to
    generate the recipes and guinea pig names)

    View Slide

  31. Show Me the Code

    View Slide

  32. 10 Mini Batches
    loif yeund, wochesther anftaome nooceiniIs beyloalt,
    thelveokk
    theouossir thochenplad my ard, aty lotheneud, somenh.
    EEPeCND. Ons, id theor oudoly
    Notfof aolllyLeagaoner lirchin, ithen, pay sudthas eomee
    sfolbfetf dotheut,
    Af thowrne shame, houd palid.
    EOTMOS.OOtir, hol mesoalgteus ou

    View Slide

  33. 50 Mini Batches
    Ast beanbly earther,
    That is fay that hath a begwer.
    And, the face!
    Exeunt Frods. Let lingher me!
    Lothfy here'nst with honours, by thee.
    GLOUCESTER. Henply, cancut these orwerd.
    Plence was he likesess puince,
    Det, woulst of brother. He will sech the exate,

    View Slide

  34. 160 Mini Batches
    SILVIA. The sleep- Villain; so shouty I desire
    Like his courtesy'r's thrice of Curst;
    And four lady, doth be hone'ring-broke of this,
    And so who ley subdue. My comforts.
    QUEEN ELIZABETH. Where's Bugidde

    View Slide

  35. Image credit: https://bukk.it/notbad.gif

    View Slide

  36. TL;DPA
    • Poems are emotive programs and programs are poems
    for the machine
    • Poems can be programs (and vice-versa!)
    • Machines can write poems (via Markov chains, RNNs,
    and more)
    • The future: Cortex (thanks, Joyce Xu!)

    View Slide

  37. View Slide

  38. Questions?
    (def eric-weinstein
    {:employer "Fox Networks Group"
    :github "ericqweinstein"
    :twitter "ericqweinstein"
    :website "ericweinste.in"})
    30% off with CLOJURECONJ30! ->>

    View Slide