Baruco 2014: How I Built My Own Twitch-Plays-Pokémon

As the slides don't really say much, the content of the talk is going to be published as a series of blogposts.
http://ajipirijou.com/talks/how-i-built-my-own-tpp-part-1

This talk was given at Baruco 2014.
It explains the story of How I built my own TPP.

Transcript

1. When was the last time you worked on a
   non productive project, just for the fun of it?
   

   View Slide

2. February 14th
   2014
   

   View Slide

3. Twitch Plays Pokémon
   

   View Slide

4. View Slide

5. View Slide

6. View Slide

7. OMG
   

   View Slide

8. OMG
   I need to make my own
   

   View Slide

9. by @eljojo
   How I
   built
   my own
   Twitch
   Plays
   Pokémon
   

   View Slide

10. by @eljojo
    How I
    built
    my own
    Twitch
    Plays
    Pokémon
    

    View Slide

11. Saturday Morning
    

    View Slide

12. View Slide

13. View Slide

14. View Slide

15. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator
    

    View Slide

16. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator
    

    View Slide

17. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator
    

    View Slide

18. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator
    

    View Slide

19. How do I control an emulator
    with Ruby?
    

    View Slide

20. “How do I send keystrokes from a
    ruby app?”
    

    View Slide

21. View Slide

22. View Slide

23. View Slide

24. View Slide

25. Problems with the
    Windows solution
    • WINDOWS!!!!!111!
    • I have no Windows machine
    • I want to run this on a server with no desktop
    environment
    

    View Slide

26. View Slide

27. View Slide

28. require 'sinatra'
    mappings = {
    "a" => java.awt.event::KeyEvent::VK_Z,
    "b" => java.awt.event::KeyEvent::VK_X,
    "up" => java.awt.event::KeyEvent::VK_UP,
    "down" => java.awt.event::KeyEvent::VK_DOWN,
    }
    robot = java.awt.Robot.new
    get '/:key' do |key|
    if key_code = mappings[key.downcase]
    robot.keyPress(key_code)
    robot.delay(100)
    robot.keyRelease(key_code)
    end
    end
    

    View Slide

29. View Slide

30. View Slide

31. Problems
    

    View Slide

32. Problems
    • Simulating Keystrokes
    • Requires desktop environment
    • Latency!
    

    View Slide

33. The perfect emulator
    • requires no desktop environment
    • streams the video online
    • has an API that allows to send button presses
    

    View Slide

34. View Slide

35. View Slide

36. View Slide

37. What’s an emulator?
    

    View Slide

38. View Slide

39. View Slide

40. View Slide

41. $ ./emulator pokemon_red.gb
    

    View Slide

42. View Slide

43. View Slide

44. $ ls lib
    cpu.rb
    gpu.rb
    ram.rb
    

    View Slide

45. View Slide

46. View Slide

47. @ram = []
    rom = File.binread('pokemon.gb')
    @ram += rom.unpack('C*')
    

    View Slide

48. View Slide

49. View Slide

50. address operation
    0x00 nop
    0x04 inc_b
    0x80 add_a_b
    0xA0 and_b
    0xC0 jp_nn
    CPU Instruction Table
    

    View Slide

51. address operation
    0x00 nop
    0x04 inc_b
    0x80 add_a_b
    0xA0 and_b
    0xC0 jp_nn
    CPU Instruction Table
    class CPU
    OPS = [
    :nop, … …
    :inc_b, …
    :add_a_b, …
    :and_b, …
    :jp_nn,
    ]
    

    View Slide

52. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter]
    next_instruction = OPS[next_byte]
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end
    

    View Slide

53. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte]
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end
    

    View Slide

54. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end
    

    View Slide

55. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end
    

    View Slide

56. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end
    

    View Slide

57. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end
    

    View Slide

58. Why do we care?
    

    View Slide

59. we’ll write our own!
    

    View Slide

60. Advantages
    • No need for keypresses
    

    View Slide

61. Advantages
    • No need for keypresses
    • We can run on a headless server
    

    View Slide

62. Advantages
    • No need for keypresses
    • We can run on a headless server
    • We can stream closer to real-time
    

    View Slide

63. Advantages
    • No need for keypresses
    • We can run on a headless server
    • We can stream closer to real-time
    • We control the memory!
    

    View Slide

64. Advantages
    • version controlling of gameplays
    • rewind game sessions
    

    View Slide

65. Advantages
    • version controlling of gameplays
    • rewind game sessions
    • forking and branching
    

    View Slide

66. Advantages
    • version controlling of gameplays
    • rewind game sessions
    • forking and branching
    • machine learning
    

    View Slide

67. tom7.org/mario
    

    View Slide

68. Advantages
    • No need for keypresses
    • We can run on a headless server
    • We can stream closer to real-time
    • We control the memory!
    

    View Slide

69. View Slide

70. View Slide

71. View Slide

72. View Slide

73. if you
    want to
    succeed at
    conferences
    you
    should
    totally
    promise
    stuff
    you have
    no idea
    about
    that’s
    p good
    

    View Slide

74. Taking over Mimey
    

    View Slide

75. Jano González…
    … loves GameBoy
    … loves Pokémon
    … loves Ruby
    

    View Slide

76. Mimey
    • GameBoy emulator
    • written in Ruby
    

    View Slide

77. Mimey
    • GameBoy emulator
    • written in Ruby
    • only 10% done
    

    View Slide

78. View Slide

79. How to draw an owl
    

    View Slide

80. How to draw an owl
    draw some circles
    

    View Slide

81. How to draw an owl
    draw some circles add the rest of the owl
    

    View Slide

82. bits and bytes
    

    View Slide

83. View Slide

84. View Slide

85. View Slide

86. I had no real reason to
    know what that means
    

    View Slide

87. I was forced to learn
    

    View Slide

88. Sven Fuchs
    • he knows about computers
    

    View Slide

89. Sven Fuchs
    • he knows a lot about computers
    

    View Slide

90. “ugh, I learned that 20
    years ago”
    

    View Slide

91. I never learned it
    because I didn’t have to.
    

    View Slide

92. Irresponsible Ruby
    

    View Slide

93. Steve Klabnik
    • also knows a lot about
    computers
    • gave talk in EuRuKo last year
    

    View Slide

94. “Ruby code that you wouldn't
    deploy at your day job because
    it's just too interesting or creative”
    - Steve Klabnik
    

    View Slide

95. Irresponsible Ruby
    • Neat Hacks
    • Weird metaprogramming
    • Unusual techniques
    • No Unit-Tests
    

    View Slide

96. “If you worry too much about
    being clean and tidy, you
    can’t push the boundaries”
    - _why
    

    View Slide

97. testing irresponsibly
    

    View Slide

98. CPU Steps
    Step Operation A B C D E F
    1 nop 0 15 232 22 19 10
    2 dec_d 0 15 232 21 19 10
    3 inc_b 0 16 232 21 19 10
    

    View Slide

99. The specification of how the original GameBoy works
    was feature-frozen before I was born.
    

    View Slide

100. View Slide

101. View Slide

102. Rendering the first
     image
     

     View Slide

103. Patience
     

     View Slide

104. Patience
     

     View Slide

105. Patience
     

     View Slide

106. View Slide

107. View Slide

108. View Slide

109. View Slide

110. The memory was the same!
     

     View Slide

111. View Slide

112. View Slide

113. View Slide

114. View Slide

115. View Slide

116. View Slide

117. View Slide

118. View Slide

119. “It uses ruby everywhere!”
     - DHH
     

     View Slide

120. View Slide

121. gameboy emulator
     

     View Slide

122. View Slide

123. View Slide

124. View Slide

125. View Slide

126. View Slide

127. View Slide

128. View Slide

129. View Slide

130. View Slide

131. View Slide

132. The Realization
     

     View Slide

133. I learned tons of things
     that I would have never
     learned by my own
     

     View Slide

134. I got out of my comfort
     zone
     

     View Slide

135. It was fun!
     

     View Slide

136. I can apply some of
     this to my work
     

     View Slide

137. Conference Driven
     Development
     

     View Slide

138. sometimes you need
     the right push
     

     View Slide

139. View Slide

140. wear the hacker hat
     

     View Slide

141. write irresponsible ruby
     

     View Slide

142. get back the passion
     that got you into programming
     

     View Slide

143. thanks!
     @eljojo
     

     View Slide

144. github.com/eljojo/mimey
     

     View Slide