Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Baruco 2014: How I Built My Own Twitch-Plays-Pokémon

Baruco 2014: How I Built My Own Twitch-Plays-Pokémon

As the slides don't really say much, the content of the talk is going to be published as a series of blogposts.
http://ajipirijou.com/talks/how-i-built-my-own-tpp-part-1

This talk was given at Baruco 2014.
It explains the story of How I built my own TPP.

José Tomás Albornoz

September 12, 2014
Tweet

More Decks by José Tomás Albornoz

Other Decks in Technology

Transcript

  1. When was the last time you worked on a
    non productive project, just for the fun of it?

    View full-size slide

  2. February 14th
    2014

    View full-size slide

  3. Twitch Plays Pokémon

    View full-size slide

  4. OMG
    I need to make my own

    View full-size slide

  5. by @eljojo
    How I
    built
    my own
    Twitch
    Plays
    Pokémon

    View full-size slide

  6. by @eljojo
    How I
    built
    my own
    Twitch
    Plays
    Pokémon

    View full-size slide

  7. Saturday Morning

    View full-size slide

  8. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator

    View full-size slide

  9. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator

    View full-size slide

  10. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator

    View full-size slide

  11. Ingredients List
    • 1 GameBoy Emulator
    • 1 screen-streaming app
    • 1 internet control for emulator

    View full-size slide

  12. How do I control an emulator
    with Ruby?

    View full-size slide

  13. “How do I send keystrokes from a
    ruby app?”

    View full-size slide

  14. Problems with the
    Windows solution
    • WINDOWS!!!!!111!
    • I have no Windows machine
    • I want to run this on a server with no desktop
    environment

    View full-size slide

  15. require 'sinatra'
    mappings = {
    "a" => java.awt.event::KeyEvent::VK_Z,
    "b" => java.awt.event::KeyEvent::VK_X,
    "up" => java.awt.event::KeyEvent::VK_UP,
    "down" => java.awt.event::KeyEvent::VK_DOWN,
    }
    robot = java.awt.Robot.new
    get '/:key' do |key|
    if key_code = mappings[key.downcase]
    robot.keyPress(key_code)
    robot.delay(100)
    robot.keyRelease(key_code)
    end
    end

    View full-size slide

  16. Problems
    • Simulating Keystrokes
    • Requires desktop environment
    • Latency!

    View full-size slide

  17. The perfect emulator
    • requires no desktop environment
    • streams the video online
    • has an API that allows to send button presses

    View full-size slide

  18. What’s an emulator?

    View full-size slide

  19. $ ./emulator pokemon_red.gb

    View full-size slide

  20. $ ls lib
    cpu.rb
    gpu.rb
    ram.rb

    View full-size slide

  21. @ram = []
    rom = File.binread('pokemon.gb')
    @ram += rom.unpack('C*')

    View full-size slide

  22. address operation
    0x00 nop
    0x04 inc_b
    0x80 add_a_b
    0xA0 and_b
    0xC0 jp_nn
    CPU Instruction Table

    View full-size slide

  23. address operation
    0x00 nop
    0x04 inc_b
    0x80 add_a_b
    0xA0 and_b
    0xC0 jp_nn
    CPU Instruction Table
    class CPU
    OPS = [
    :nop, … …
    :inc_b, …
    :add_a_b, …
    :and_b, …
    :jp_nn,
    ]

    View full-size slide

  24. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter]
    next_instruction = OPS[next_byte]
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View full-size slide

  25. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte]
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View full-size slide

  26. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View full-size slide

  27. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View full-size slide

  28. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View full-size slide

  29. “the” loop
    class Emulator
    def step
    next_byte = @ram[@program_counter] # => 0xC0
    next_instruction = OPS[next_byte] # => :jp_nn
    @cpu.send(next_instruction)
    @program_counter += 1
    end
    end

    View full-size slide

  30. Why do we care?

    View full-size slide

  31. we’ll write our own!

    View full-size slide

  32. Advantages
    • No need for keypresses

    View full-size slide

  33. Advantages
    • No need for keypresses
    • We can run on a headless server

    View full-size slide

  34. Advantages
    • No need for keypresses
    • We can run on a headless server
    • We can stream closer to real-time

    View full-size slide

  35. Advantages
    • No need for keypresses
    • We can run on a headless server
    • We can stream closer to real-time
    • We control the memory!

    View full-size slide

  36. Advantages
    • version controlling of gameplays
    • rewind game sessions

    View full-size slide

  37. Advantages
    • version controlling of gameplays
    • rewind game sessions
    • forking and branching

    View full-size slide

  38. Advantages
    • version controlling of gameplays
    • rewind game sessions
    • forking and branching
    • machine learning

    View full-size slide

  39. tom7.org/mario

    View full-size slide

  40. Advantages
    • No need for keypresses
    • We can run on a headless server
    • We can stream closer to real-time
    • We control the memory!

    View full-size slide

  41. if you
    want to
    succeed at
    conferences
    you
    should
    totally
    promise
    stuff
    you have
    no idea
    about
    that’s
    p good

    View full-size slide

  42. Taking over Mimey

    View full-size slide

  43. Jano González…
    … loves GameBoy
    … loves Pokémon
    … loves Ruby

    View full-size slide

  44. Mimey
    • GameBoy emulator
    • written in Ruby

    View full-size slide

  45. Mimey
    • GameBoy emulator
    • written in Ruby
    • only 10% done

    View full-size slide

  46. How to draw an owl

    View full-size slide

  47. How to draw an owl
    draw some circles

    View full-size slide

  48. How to draw an owl
    draw some circles add the rest of the owl

    View full-size slide

  49. bits and bytes

    View full-size slide

  50. I had no real reason to
    know what that means

    View full-size slide

  51. I was forced to learn

    View full-size slide

  52. Sven Fuchs
    • he knows about computers

    View full-size slide

  53. Sven Fuchs
    • he knows a lot about computers

    View full-size slide

  54. “ugh, I learned that 20
    years ago”

    View full-size slide

  55. I never learned it
    because I didn’t have to.

    View full-size slide

  56. Irresponsible Ruby

    View full-size slide

  57. Steve Klabnik
    • also knows a lot about
    computers
    • gave talk in EuRuKo last year

    View full-size slide

  58. “Ruby code that you wouldn't
    deploy at your day job because
    it's just too interesting or creative”
    - Steve Klabnik

    View full-size slide

  59. Irresponsible Ruby
    • Neat Hacks
    • Weird metaprogramming
    • Unusual techniques
    • No Unit-Tests

    View full-size slide

  60. “If you worry too much about
    being clean and tidy, you
    can’t push the boundaries”
    - _why

    View full-size slide

  61. testing irresponsibly

    View full-size slide

  62. CPU Steps
    Step Operation A B C D E F
    1 nop 0 15 232 22 19 10
    2 dec_d 0 15 232 21 19 10
    3 inc_b 0 16 232 21 19 10

    View full-size slide

  63. The specification of how the original GameBoy works
    was feature-frozen before I was born.

    View full-size slide

  64. Rendering the first
    image

    View full-size slide

  65. The memory was the same!

    View full-size slide

  66. “It uses ruby everywhere!”
    - DHH

    View full-size slide

  67. gameboy emulator

    View full-size slide

  68. The Realization

    View full-size slide

  69. I learned tons of things
    that I would have never
    learned by my own

    View full-size slide

  70. I got out of my comfort
    zone

    View full-size slide

  71. I can apply some of
    this to my work

    View full-size slide

  72. Conference Driven
    Development

    View full-size slide

  73. sometimes you need
    the right push

    View full-size slide

  74. wear the hacker hat

    View full-size slide

  75. write irresponsible ruby

    View full-size slide

  76. get back the passion
    that got you into programming

    View full-size slide

  77. thanks!
    @eljojo

    View full-size slide

  78. github.com/eljojo/mimey

    View full-size slide