$30 off During Our Annual Pro Sale. View Details »

Baruco 2014: How I Built My Own Twitch-Plays-Pokémon

Baruco 2014: How I Built My Own Twitch-Plays-Pokémon

As the slides don't really say much, the content of the talk is going to be published as a series of blogposts.
http://ajipirijou.com/talks/how-i-built-my-own-tpp-part-1

This talk was given at Baruco 2014.
It explains the story of How I built my own TPP.

José Tomás Albornoz

September 12, 2014
Tweet

More Decks by José Tomás Albornoz

Other Decks in Technology

Transcript

  1. When was the last time you worked on a
    non productive project, just for the fun of it?

    View Slide

  2. February 14th
    2014

    View Slide

  3. Twitch Plays Pokémon

    View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. OMG

    View Slide

  8. OMG
    I need to make my own

    View Slide

  9. by @eljojo
    How I
    built
    my own
    Twitch
    Plays
    Pokémon

    View Slide

  10. by @eljojo
    How I
    built
    my own
    Twitch
    Plays
    Pokémon

    View Slide

  11. Saturday Morning

    View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator

    View Slide

  16. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator

    View Slide

  17. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator

    View Slide

  18. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator

    View Slide

  19. How do I control an emulator
    with Ruby?

    View Slide

  20. “How do I send keystrokes from a
    ruby app?”

    View Slide

  21. View Slide

  22. View Slide

  23. View Slide

  24. View Slide

  25. Problems with the
    Windows solution
    • WINDOWS!!!!!111!
    • I have no Windows machine
    • I want to run this on a server with no desktop
    environment

    View Slide

  26. View Slide

  27. View Slide

  28. require 'sinatra'
    mappings = {
    "a" => java.awt.event::KeyEvent::VK_Z,
    "b" => java.awt.event::KeyEvent::VK_X,
    "up" => java.awt.event::KeyEvent::VK_UP,
    "down" => java.awt.event::KeyEvent::VK_DOWN,
    }
    robot = java.awt.Robot.new
    get '/:key' do |key|
    if key_code = mappings[key.downcase]
    robot.keyPress(key_code)
    robot.delay(100)
    robot.keyRelease(key_code)
    end
    end

    View Slide

  29. View Slide

  30. View Slide

  31. Problems

    View Slide

  32. Problems
    • Simulating Keystrokes
    • Requires desktop environment
    • Latency!

    View Slide

  33. The perfect emulator
    • requires no desktop environment
    • streams the video online
    • has an API that allows to send button presses

    View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. What’s an emulator?

    View Slide

  38. View Slide

  39. View Slide

  40. View Slide

  41. $ ./emulator pokemon_red.gb

    View Slide

  42. View Slide

  43. View Slide

  44. $ ls lib
    cpu.rb
    gpu.rb
    ram.rb

    View Slide

  45. View Slide

  46. View Slide

  47. @ram = []
    rom = File.binread('pokemon.gb')
    @ram += rom.unpack('C*')

    View Slide

  48. View Slide

  49. View Slide

  50. address operation
    0x00 nop
    0x04 inc_b
    0x80 add_a_b
    0xA0 and_b
    0xC0 jp_nn
    CPU Instruction Table

    View Slide

  51. address operation
    0x00 nop
    0x04 inc_b
    0x80 add_a_b
    0xA0 and_b
    0xC0 jp_nn
    CPU Instruction Table
    class CPU
    OPS = [
    :nop, … …
    :inc_b, …
    :add_a_b, …
    :and_b, …
    :jp_nn,
    ]

    View Slide

  52. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter]
    next_instruction = OPS[next_byte]
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View Slide

  53. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte]
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View Slide

  54. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View Slide

  55. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View Slide

  56. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View Slide

  57. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View Slide

  58. Why do we care?

    View Slide

  59. we’ll write our own!

    View Slide

  60. Advantages
    • No need for keypresses

    View Slide

  61. Advantages
    • No need for keypresses
    • We can run on a headless server

    View Slide

  62. Advantages
    • No need for keypresses
    • We can run on a headless server
    • We can stream closer to real-time

    View Slide

  63. Advantages
    • No need for keypresses
    • We can run on a headless server
    • We can stream closer to real-time
    • We control the memory!

    View Slide

  64. Advantages
    • version controlling of gameplays
    • rewind game sessions

    View Slide

  65. Advantages
    • version controlling of gameplays
    • rewind game sessions
    • forking and branching

    View Slide

  66. Advantages
    • version controlling of gameplays
    • rewind game sessions
    • forking and branching
    • machine learning

    View Slide

  67. tom7.org/mario

    View Slide

  68. Advantages
    • No need for keypresses
    • We can run on a headless server
    • We can stream closer to real-time
    • We control the memory!

    View Slide

  69. View Slide

  70. View Slide

  71. View Slide

  72. View Slide

  73. if you
    want to
    succeed at
    conferences
    you
    should
    totally
    promise
    stuff
    you have
    no idea
    about
    that’s
    p good

    View Slide

  74. Taking over Mimey

    View Slide

  75. Jano González…
    … loves GameBoy
    … loves Pokémon
    … loves Ruby

    View Slide

  76. Mimey
    • GameBoy emulator
    • written in Ruby

    View Slide

  77. Mimey
    • GameBoy emulator
    • written in Ruby
    • only 10% done

    View Slide

  78. View Slide

  79. How to draw an owl

    View Slide

  80. How to draw an owl
    draw some circles

    View Slide

  81. How to draw an owl
    draw some circles add the rest of the owl

    View Slide

  82. bits and bytes

    View Slide

  83. View Slide

  84. View Slide

  85. View Slide

  86. I had no real reason to
    know what that means

    View Slide

  87. I was forced to learn

    View Slide

  88. Sven Fuchs
    • he knows about computers

    View Slide

  89. Sven Fuchs
    • he knows a lot about computers

    View Slide

  90. “ugh, I learned that 20
    years ago”

    View Slide

  91. I never learned it
    because I didn’t have to.

    View Slide

  92. Irresponsible Ruby

    View Slide

  93. Steve Klabnik
    • also knows a lot about
    computers
    • gave talk in EuRuKo last year

    View Slide

  94. “Ruby code that you wouldn't
    deploy at your day job because
    it's just too interesting or creative”
    - Steve Klabnik

    View Slide

  95. Irresponsible Ruby
    • Neat Hacks
    • Weird metaprogramming
    • Unusual techniques
    • No Unit-Tests

    View Slide

  96. “If you worry too much about
    being clean and tidy, you
    can’t push the boundaries”
    - _why

    View Slide

  97. testing irresponsibly

    View Slide

  98. CPU Steps
    Step Operation A B C D E F
    1 nop 0 15 232 22 19 10
    2 dec_d 0 15 232 21 19 10
    3 inc_b 0 16 232 21 19 10

    View Slide

  99. The specification of how the original GameBoy works
    was feature-frozen before I was born.

    View Slide

  100. View Slide

  101. View Slide

  102. Rendering the first
    image

    View Slide

  103. Patience

    View Slide

  104. Patience

    View Slide

  105. Patience

    View Slide

  106. View Slide

  107. View Slide

  108. View Slide

  109. View Slide

  110. The memory was the same!

    View Slide

  111. View Slide

  112. View Slide

  113. View Slide

  114. View Slide

  115. View Slide

  116. View Slide

  117. View Slide

  118. View Slide

  119. “It uses ruby everywhere!”
    - DHH

    View Slide

  120. View Slide

  121. gameboy emulator

    View Slide

  122. View Slide

  123. View Slide

  124. View Slide

  125. View Slide

  126. View Slide

  127. View Slide

  128. View Slide

  129. View Slide

  130. View Slide

  131. View Slide

  132. The Realization

    View Slide

  133. I learned tons of things
    that I would have never
    learned by my own

    View Slide

  134. I got out of my comfort
    zone

    View Slide

  135. It was fun!

    View Slide

  136. I can apply some of
    this to my work

    View Slide

  137. Conference Driven
    Development

    View Slide

  138. sometimes you need
    the right push

    View Slide

  139. View Slide

  140. wear the hacker hat

    View Slide

  141. write irresponsible ruby

    View Slide

  142. get back the passion
    that got you into programming

    View Slide

  143. thanks!
    @eljojo

    View Slide

  144. github.com/eljojo/mimey

    View Slide