Shall We Play A Game?

Shall We Play A Game?

From RubyConf 2015.

Teaching computers to play games has been a pursuit and passion for many programmers. Game playing has led to many advances in computing over the years, and the best computerized game players have gained a lot of attention from the general public (think Deep Blue and Watson).

Using the Ricochet Robots board game as an example, let's talk about what's involved in teaching a computer to play games. Along the way, we'll touch on graph search techniques, data representation, algorithms, heuristics, pruning, and optimization.

27204e228cc521c6cafed3c92b95184c?s=128

Randy Coulman

November 17, 2015
Tweet

Transcript

  1. www .codingzeal.com @codingzeal Randy Coulman Senior Software Engineer @randycoulman

  2. Computers Playing Games

  3. Computers Playing Games

  4. Computers Playing Games

  5. Computers Playing Games https://www.flickr.com/photos/atomictaco/12935316785 - CC-BY-SA 2.0

  6. Why Graphs, Trees & Algorithms? Hierarchical structures: Organization charts

  7. Why Graphs, Trees & Algorithms? Relationship Maps: Social Networks like

    LinkedIn & Facebook
  8. Why Graphs, Trees & Algorithms? Trip Planning

  9. Why Graphs, Trees & Algorithms? Network Routing

  10. Game Playing

  11. Ricochet Robots

  12. Moves to get to

  13. 1 1. Blue Right Moves to get to

  14. 1 2 1. Blue Right 2. Blue Down Moves to

    get to
  15. 1 2 3 1. Blue Right 2. Blue Down 3.

    Blue Left Moves to get to
  16. 1 2 3 4 1. Blue Right 2. Blue Down

    3. Blue Left 4. Green Right Moves to get to
  17. 1 2 3 4 5 1. Blue Right 2. Blue

    Down 3. Blue Left 4. Green Right 5. Green Down Moves to get to
  18. 1 2 3 4 5 6 1. Blue Right 2.

    Blue Down 3. Blue Left 4. Green Right 5. Green Down 6. Green Left Moves to get to
  19. 1 2 3 4 5 6 7 1. Blue Right

    2. Blue Down 3. Blue Left 4. Green Right 5. Green Down 6. Green Left 7. Green Down Moves to get to
  20. Characterizing the Problem Possible Board States (size of state space):

    252 * 251 * 250 * 249 * 248 = 976,484,376,000
  21. Characterizing the Problem Branching Factor: 9 - 20 possible moves

    from each state
  22. The Board • Board (Static: 16 x 16)

  23. The Board • Board (Static: 16 x 16) • Walls

    & Targets (changes each game)
  24. The Board • Board (Static: 16 x 16) • Walls

    & Targets (changes each game) • Goal (changes each turn)
  25. The Board • Board (Static: 16 x 16) • Walls

    & Targets (changes each game) • Goal (changes each turn) • Robot Positions (changes each move)
  26. Robot Movement 1 2 1 2 1

  27. Trees

  28. Search Algorithms See Jamis Buck’s book Basil & Fabian: Three

    Ways Through
  29. Depth-First Search

  30. Depth-First Search 1

  31. Depth-First Search 1 2

  32. Depth-First Search 1 2 3

  33. Depth-First Search 1 2 3 4

  34. Depth-First Search 1 2 3 4 5

  35. Depth-First Search 1 2 3 4 5 6

  36. Depth-First Search 1 2 3 4 5 6 7

  37. Depth-First Search 1 2 3 4 5 6 7 8

  38. Depth-First Search 1 2 3 4 5 6 7 8

    9
  39. Depth-First Search 1 2 3 4 5 6 7 8

    9 10
  40. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11
  41. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12
  42. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13
  43. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14
  44. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15
  45. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15 16
  46. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15 16
  47. def solve solve_recursively(Path.initial(state)) candidates.min_by(&:length) || Outcome.no_solution(state) end def solve_recursively(path) return

    candidates << path.to_outcome if path.solved? path.allowable_successors.each do |successor| solve_recursively(successor) end end Depth-First Search
  48. Cycles

  49. Cycles

  50. Cycles

  51. Cycles

  52. Cycles

  53. Cycles

  54. Cycles

  55. Cycles

  56. Cycles

  57. class Path def allowable_successors allowable_moves .map { |direction| successor(direction) }

    .compact end def successor(direction) next_robot = robot.moved(direction) next_robot == robot ? nil : self.class.new(next_robot, moves + [direction]) end end Cycles
  58. class Path def allowable_successors allowable_moves .map { |direction| successor(direction) }

    .compact .reject(&:cycle?) end def successor(direction) next_robot = robot.moved(direction) next_robot == robot ? nil : self.class.new(next_robot, moves + [direction], visited + [robot]) end def cycle? visited.include?(robot) end end Cycles
  59. Visited States 3 1 1 2 3 2 4

  60. Not a Tree

  61. Not a Tree

  62. Complications

  63. Complications

  64. 1 Complications

  65. 1 2 Complications

  66. 1 2 3 Complications

  67. 1 2 3 4 Complications

  68. 1 2 3 4 5 Complications

  69. Breadth-First Search

  70. Breadth-First Search 1

  71. Breadth-First Search 1 2

  72. Breadth-First Search 1 2 3

  73. Breadth-First Search 1 2 3 4

  74. Breadth-First Search 1 2 3 4 5

  75. Breadth-First Search 1 2 3 4 5 6

  76. Breadth-First Search 1 2 3 4 5 6 7

  77. Breadth-First Search 1 2 3 4 5 6 7 8

  78. Breadth-First Search 1 2 3 4 5 6 7 8

    9
  79. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10
  80. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11
  81. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12
  82. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13
  83. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14
  84. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15
  85. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15 16
  86. Breadth-First Search 1 2 5 3 7 8 4 9

    10 11 6 12 13 14 15 16
  87. Breadth-First Search def solve paths = [Path.initial(initial_state)] until paths.empty? path

    = paths.shift return path.to_outcome if path.solved? paths += path.allowable_successors end Outcome.no_solution(initial_state) end
  88. Breadth-First Search Global visited list works now

  89. BFS: Global Visited List def solve visited = Set.new paths

    = [Path.initial(initial_state)] until paths.empty? path = paths.shift return path.to_outcome if path.solved? next if visited.include?(path.state) visited << path.state paths += path.allowable_successors end Outcome.no_solution(initial_state) end
  90. Optimization Do Less Things Do Things Faster Heuristics

  91. Optimization Do Less Things: Process fewer states

  92. Optimization Do Things Faster: Less work per state

  93. Optimization Heuristics: Rules of Thumb Less certain; may work in

    some circumstances, but not others
  94. initial_state.ensure_goal_robot_first(goal) def ensure_goal_robot_first(goal) return if goal.color == :any goal_index =

    robots.index { |robot| robot.color == goal.color } robots.rotate!(goal_index) end Heuristic: Move Active Robot First
  95. Heuristic: Move Active Robot First States Considered 0 350000 700000

    1050000 1400000 Original Algorithm Active First
  96. def solve paths = [Path.initial(initial_state)] until paths.empty? path = paths.shift

    return path.to_outcome if path.solved? paths += path.allowable_successors end Outcome.no_solution(initial_state) end Do Less Things: Check for Solutions at Generation Time
  97. Do Less Things: Check for Solutions at Generation Time 1

    2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
  98. Do Less Things: Check for Solutions at Generation Time def

    solve paths = [Path.initial(initial_state)] until paths.empty? path = paths.shift successors = path.allowable_successors solution = successors.find(&:solved?) return solution.to_outcome if solution paths += successors end Outcome.no_solution(initial_state) end
  99. Do Less Things: Check for Solutions at Generation Time 1

    2 3 4 5 6 X X X X X X X X X 16
  100. Do Less Things: Check for Solutions at Generation Time 0

    800,000 1,600,000 2,400,000 3,200,000 Original Algorithm Check at Generation Total States Considered
  101. Do Things Faster: Precompute stopping cells

  102. Do Things Faster: Precompute stopping cells States / second 0

    675 1350 2025 2700 Original Algorithm Pre-compute Stops
  103. Do Less Things / Do Things Faster: Treat non-active robots

    as equivalent
  104. class BoardState def equivalence_class @equivalence_class ||= begin Set.new(robots.map do |robot|

    robot.position_hash + (robot.active?(goal) ? 1000 : 0) end) end end end def position_hash row * board.size + column end Do Less Things / Do Things Faster: Treat non-active robots as equivalent
  105. Do Less Things / Do Things Faster: Treat non-active robots

    as equivalent 0 1,000,000 2,000,000 3,000,000 4,000,000 Original Algorithm Check at Generation Robot Equivalence Total States Considered
  106. Do Less Things / Do Things Faster: Treat non-active robots

    as equivalent States / second 0 750 1500 2250 3000 Original Algorithm Pre-compute Stops Robot Equiv.
  107. class BoardState def equivalence_class @equivalence_class ||= begin Set.new(robots.map do |robot|

    robot.position_hash + (robot.active?(goal) ? 1000 : 0) end).sort! end end end Do Things Faster: Sorted Array vs Set
  108. Do Things Faster: Sorted Array vs Set States / second

    0 800 1600 2400 3200 Original Algorithm Pre-compute Stops Robot Equiv. Arrays not Sets
  109. class Robot def moved(direction, board_state) self.class.new(color, cell.next_cell(direction, board_state)) end end

    class BoardState def with_robot_moved(robot, direction) moved_robots = robots.map do |each_robot| each_robot == robot ? each_robot.moved(direction, self) : each_robot end self.class.new(moved_robots, goal) end end Do Things Faster: Less Object Creation
  110. class Robot def moved(direction, board_state) next_cell = cell.next_cell(direction, board_state) next_cell

    == cell ? self : self.class.new(color, next_cell) end end class BoardState def with_robot_moved(robot, direction) moved_robot = robot.moved(direction, self) return self if moved_robot.equal?(robot) moved_robots = robots.map do |each_robot| each_robot == robot ? moved_robot : each_robot end self.class.new(moved_robots, goal) end end Do Things Faster: Less Object Creation
  111. States / second 0 1250 2500 3750 5000 Original Algorithm

    Pre-compute Stops Robot Equiv. Arrays not Sets Less Objects Do Things Faster: Less Object Creation
  112. class BoardState def with_robot_moved(robot, direction) moved_robot = robot.moved(direction, self) return

    self if moved_robot.equal?(robot) moved_robots = robots.map do |each_robot| each_robot.equal?(robot) ? moved_robot : each_robot end self.class.new(moved_robots, goal) end end Do Things Faster: Use Object Identity Instead of Deep Equality
  113. Do Things Faster: Use Object Identity Instead of Deep Equality

    States / second 0 1750 3500 5250 7000 Original Algorithm Pre-compute Stops Robot Equiv. Arrays not Sets Less Objects Object Identity
  114. Results So Far Solving time (seconds) 0 750 1500 2250

    3000 Original Active First Check at Gen. Pre-compute Robot Equiv. Arrays not Sets Less Objects Robot Identity
  115. Best-First Search

  116. Best-First Search 1

  117. Best-First Search 1 2

  118. Best-First Search 1 2 3

  119. Best-First Search 1 2 3 4

  120. Best-First Search 1 2 3 4 5

  121. Best-First Search 1 2 3 4 5 6

  122. def solve paths = FastContainers::PriorityQueue.new(:min).tap do |paths| add_path(paths, path) end

    until paths.empty? path = paths.top; paths.pop successors = path.allowable_successors solution = successors.find(&:solved?) return solution.to_outcome if solution add_paths(paths, successors) end Outcome.no_solution(initial_state) end Best-First Search
  123. def add_paths(paths, successors) successors.each { |path| add_path(paths, path) } end

    def add_path(paths, path) paths.push(path, score(path)) end def score(path) # ... end Best-First Search
  124. What is Best? A* Algorithm

  125. A* Algorithm Each state is scored Lowest score is best

  126. A* Algorithm Score = Distance so far + Estimated distance

    left
  127. def score(path) path.length + best_estimate end def best_estimate state.active_robots.map {

    |robot| estimate_at(robot.cell) }.min end A* Algorithm
  128. A* Algorithm Must not over-estimate

  129. A* Algorithm 0

  130. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 0
  131. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 0
  132. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 0
  133. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 0
  134. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 0
  135. Future Ideas: Better Algorithms Collision Detection

  136. Future Ideas: Better Algorithms Work Backwards

  137. Work Backwards

  138. Future Ideas: Better Heuristics Move most-recently moved robots first (MRU)

  139. Future Ideas: Better Heuristics Some combination of active robot and

    MRU robots
  140. Future Ideas: Better Heuristics Sort movement directions intelligently

  141. Future Ideas: More Optimizations Pre-compute per-robot stopping positions

  142. Future Ideas: More Optimizations Use less objects and more primitive

    types
  143. Future Ideas: More Optimizations Parallelism

  144. Future Ideas: More Optimizations Other Languages

  145. Acknowledgements • Trever Yarrish of Zeal for the awesome graphics

    and visualizations • My fellow Zeals for ideas, feedback, and pairing on the solver • Michael Fogleman for some optimization ideas • Trevor Lalish-Menagh for introducing me to the game • Screen Captures from War Games. (Dir. John Badham. MGM/UA. 1983)
  146. Questions ? http://randycoulman.com http://speakerdeck.com/randycoulman http://speakerrate.com/randycoulman Code is on Github https://github.com/CodingZeal/robots

  147. www .codingzeal.com @codingzeal Thank You! Randy Coulman @randycoulman