$30 off During Our Annual Pro Sale. View Details »

Shall We Play A Game?

Shall We Play A Game?

From RubyConf 2015.

Teaching computers to play games has been a pursuit and passion for many programmers. Game playing has led to many advances in computing over the years, and the best computerized game players have gained a lot of attention from the general public (think Deep Blue and Watson).

Using the Ricochet Robots board game as an example, let's talk about what's involved in teaching a computer to play games. Along the way, we'll touch on graph search techniques, data representation, algorithms, heuristics, pruning, and optimization.

Randy Coulman

November 17, 2015
Tweet

More Decks by Randy Coulman

Other Decks in Programming

Transcript

  1. 1 2 3 1. Blue Right 2. Blue Down 3.

    Blue Left Moves to get to
  2. 1 2 3 4 1. Blue Right 2. Blue Down

    3. Blue Left 4. Green Right Moves to get to
  3. 1 2 3 4 5 1. Blue Right 2. Blue

    Down 3. Blue Left 4. Green Right 5. Green Down Moves to get to
  4. 1 2 3 4 5 6 1. Blue Right 2.

    Blue Down 3. Blue Left 4. Green Right 5. Green Down 6. Green Left Moves to get to
  5. 1 2 3 4 5 6 7 1. Blue Right

    2. Blue Down 3. Blue Left 4. Green Right 5. Green Down 6. Green Left 7. Green Down Moves to get to
  6. Characterizing the Problem Possible Board States (size of state space):

    252 * 251 * 250 * 249 * 248 = 976,484,376,000
  7. The Board • Board (Static: 16 x 16) • Walls

    & Targets (changes each game)
  8. The Board • Board (Static: 16 x 16) • Walls

    & Targets (changes each game) • Goal (changes each turn)
  9. The Board • Board (Static: 16 x 16) • Walls

    & Targets (changes each game) • Goal (changes each turn) • Robot Positions (changes each move)
  10. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15 16
  11. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15 16
  12. def solve solve_recursively(Path.initial(state)) candidates.min_by(&:length) || Outcome.no_solution(state) end def solve_recursively(path) return

    candidates << path.to_outcome if path.solved? path.allowable_successors.each do |successor| solve_recursively(successor) end end Depth-First Search
  13. class Path def allowable_successors allowable_moves .map { |direction| successor(direction) }

    .compact end def successor(direction) next_robot = robot.moved(direction) next_robot == robot ? nil : self.class.new(next_robot, moves + [direction]) end end Cycles
  14. class Path def allowable_successors allowable_moves .map { |direction| successor(direction) }

    .compact .reject(&:cycle?) end def successor(direction) next_robot = robot.moved(direction) next_robot == robot ? nil : self.class.new(next_robot, moves + [direction], visited + [robot]) end def cycle? visited.include?(robot) end end Cycles
  15. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15 16
  16. Breadth-First Search 1 2 5 3 7 8 4 9

    10 11 6 12 13 14 15 16
  17. Breadth-First Search def solve paths = [Path.initial(initial_state)] until paths.empty? path

    = paths.shift return path.to_outcome if path.solved? paths += path.allowable_successors end Outcome.no_solution(initial_state) end
  18. BFS: Global Visited List def solve visited = Set.new paths

    = [Path.initial(initial_state)] until paths.empty? path = paths.shift return path.to_outcome if path.solved? next if visited.include?(path.state) visited << path.state paths += path.allowable_successors end Outcome.no_solution(initial_state) end
  19. initial_state.ensure_goal_robot_first(goal) def ensure_goal_robot_first(goal) return if goal.color == :any goal_index =

    robots.index { |robot| robot.color == goal.color } robots.rotate!(goal_index) end Heuristic: Move Active Robot First
  20. Heuristic: Move Active Robot First States Considered 0 350000 700000

    1050000 1400000 Original Algorithm Active First
  21. def solve paths = [Path.initial(initial_state)] until paths.empty? path = paths.shift

    return path.to_outcome if path.solved? paths += path.allowable_successors end Outcome.no_solution(initial_state) end Do Less Things: Check for Solutions at Generation Time
  22. Do Less Things: Check for Solutions at Generation Time 1

    2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
  23. Do Less Things: Check for Solutions at Generation Time def

    solve paths = [Path.initial(initial_state)] until paths.empty? path = paths.shift successors = path.allowable_successors solution = successors.find(&:solved?) return solution.to_outcome if solution paths += successors end Outcome.no_solution(initial_state) end
  24. Do Less Things: Check for Solutions at Generation Time 0

    800,000 1,600,000 2,400,000 3,200,000 Original Algorithm Check at Generation Total States Considered
  25. Do Things Faster: Precompute stopping cells States / second 0

    675 1350 2025 2700 Original Algorithm Pre-compute Stops
  26. class BoardState def equivalence_class @equivalence_class ||= begin Set.new(robots.map do |robot|

    robot.position_hash + (robot.active?(goal) ? 1000 : 0) end) end end end def position_hash row * board.size + column end Do Less Things / Do Things Faster: Treat non-active robots as equivalent
  27. Do Less Things / Do Things Faster: Treat non-active robots

    as equivalent 0 1,000,000 2,000,000 3,000,000 4,000,000 Original Algorithm Check at Generation Robot Equivalence Total States Considered
  28. Do Less Things / Do Things Faster: Treat non-active robots

    as equivalent States / second 0 750 1500 2250 3000 Original Algorithm Pre-compute Stops Robot Equiv.
  29. class BoardState def equivalence_class @equivalence_class ||= begin Set.new(robots.map do |robot|

    robot.position_hash + (robot.active?(goal) ? 1000 : 0) end).sort! end end end Do Things Faster: Sorted Array vs Set
  30. Do Things Faster: Sorted Array vs Set States / second

    0 800 1600 2400 3200 Original Algorithm Pre-compute Stops Robot Equiv. Arrays not Sets
  31. class Robot def moved(direction, board_state) self.class.new(color, cell.next_cell(direction, board_state)) end end

    class BoardState def with_robot_moved(robot, direction) moved_robots = robots.map do |each_robot| each_robot == robot ? each_robot.moved(direction, self) : each_robot end self.class.new(moved_robots, goal) end end Do Things Faster: Less Object Creation
  32. class Robot def moved(direction, board_state) next_cell = cell.next_cell(direction, board_state) next_cell

    == cell ? self : self.class.new(color, next_cell) end end class BoardState def with_robot_moved(robot, direction) moved_robot = robot.moved(direction, self) return self if moved_robot.equal?(robot) moved_robots = robots.map do |each_robot| each_robot == robot ? moved_robot : each_robot end self.class.new(moved_robots, goal) end end Do Things Faster: Less Object Creation
  33. States / second 0 1250 2500 3750 5000 Original Algorithm

    Pre-compute Stops Robot Equiv. Arrays not Sets Less Objects Do Things Faster: Less Object Creation
  34. class BoardState def with_robot_moved(robot, direction) moved_robot = robot.moved(direction, self) return

    self if moved_robot.equal?(robot) moved_robots = robots.map do |each_robot| each_robot.equal?(robot) ? moved_robot : each_robot end self.class.new(moved_robots, goal) end end Do Things Faster: Use Object Identity Instead of Deep Equality
  35. Do Things Faster: Use Object Identity Instead of Deep Equality

    States / second 0 1750 3500 5250 7000 Original Algorithm Pre-compute Stops Robot Equiv. Arrays not Sets Less Objects Object Identity
  36. Results So Far Solving time (seconds) 0 750 1500 2250

    3000 Original Active First Check at Gen. Pre-compute Robot Equiv. Arrays not Sets Less Objects Robot Identity
  37. def solve paths = FastContainers::PriorityQueue.new(:min).tap do |paths| add_path(paths, path) end

    until paths.empty? path = paths.top; paths.pop successors = path.allowable_successors solution = successors.find(&:solved?) return solution.to_outcome if solution add_paths(paths, successors) end Outcome.no_solution(initial_state) end Best-First Search
  38. def add_paths(paths, successors) successors.each { |path| add_path(paths, path) } end

    def add_path(paths, path) paths.push(path, score(path)) end def score(path) # ... end Best-First Search
  39. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 0
  40. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 0
  41. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 0
  42. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 0
  43. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 0
  44. Acknowledgements • Trever Yarrish of Zeal for the awesome graphics

    and visualizations • My fellow Zeals for ideas, feedback, and pairing on the solver • Michael Fogleman for some optimization ideas • Trevor Lalish-Menagh for introducing me to the game • Screen Captures from War Games. (Dir. John Badham. MGM/UA. 1983)