Shall We Play a Game?

Shall We Play a Game?

From LoneStar Ruby 2015.

Teaching computers to play games has been a pursuit and passion for many programmers. Game playing has led to many advances in computing over the years, and the best computerized game players have gained a lot of attention from the general public (think Deep Blue and Watson).

Using the Ricochet Robots board game as an example, let's talk about what's involved in teaching a computer to play games. Along the way, we'll touch on graph search techniques, data representation, algorithms, heuristics, pruning, and optimization.

27204e228cc521c6cafed3c92b95184c?s=128

Randy Coulman

August 15, 2015
Tweet

Transcript

  1. www .codingzeal.com @codingzeal Randy Coulman Senior Software Engineer @randycoulman

  2. Computers Playing Games

  3. Computers Playing Games

  4. Computers Playing Games

  5. Why Graphs, Trees & Algorithms? Hierarchical structures: Organization charts

  6. Why Graphs, Trees & Algorithms? Relationship Maps: Social Networks like

    LinkedIn & Facebook
  7. Why Graphs, Trees & Algorithms? Trip Planning

  8. Why Graphs, Trees & Algorithms? Network Routing

  9. Game Playing

  10. Ricochet Robots

  11. Moves to get to

  12. 1 1. Blue Right Moves to get to

  13. 1 2 1. Blue Right 2. Blue Down Moves to

    get to
  14. 1 2 3 1. Blue Right 2. Blue Down 3.

    Blue Left Moves to get to
  15. 1 2 3 4 1. Blue Right 2. Blue Down

    3. Blue Left 4. Green Right Moves to get to
  16. 1 2 3 4 5 1. Blue Right 2. Blue

    Down 3. Blue Left 4. Green Right 5. Green Down Moves to get to
  17. 1 2 3 4 5 6 1. Blue Right 2.

    Blue Down 3. Blue Left 4. Green Right 5. Green Down 6. Green Left Moves to get to
  18. 1 2 3 4 5 6 7 1. Blue Right

    2. Blue Down 3. Blue Left 4. Green Right 5. Green Down 6. Green Left 7. Green Down Moves to get to
  19. Characterizing the Problem Possible Board States (size of state space):

    252 * 251 * 250 * 249 * 248 = 976,484,376,000
  20. Characterizing the Problem Branching Factor: 9 - 20 possible moves

    from each state
  21. The Board • Board (Static: 16 x 16)

  22. The Board • Board (Static: 16 x 16) • Walls

    & Targets (changes each game)
  23. The Board • Board (Static: 16 x 16) • Walls

    & Targets (changes each game) • Goal (changes each turn)
  24. The Board • Board (Static: 16 x 16) • Walls

    & Targets (changes each game) • Goal (changes each turn) • Robot Positions (changes each move)
  25. Robot Movement 1 2 1 2 1

  26. Trees

  27. Search Algorithms See Jamis Buck’s book Basil & Fabian: Three

    Ways Through
  28. Depth-First Search

  29. Depth-First Search 1

  30. Depth-First Search 1 2

  31. Depth-First Search 1 2 3

  32. Depth-First Search 1 2 3 4

  33. Depth-First Search 1 2 3 4 5

  34. Depth-First Search 1 2 3 4 5 6

  35. Depth-First Search 1 2 3 4 5 6 7

  36. Depth-First Search 1 2 3 4 5 6 7 8

  37. Depth-First Search 1 2 3 4 5 6 7 8

    9
  38. Depth-First Search 1 2 3 4 5 6 7 8

    9 10
  39. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11
  40. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12
  41. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13
  42. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14
  43. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15
  44. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15 16
  45. Depth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15 16
  46. Cycles

  47. Cycles

  48. Cycles

  49. Cycles

  50. Cycles

  51. Cycles

  52. Cycles

  53. Cycles

  54. Cycles

  55. Visited States 3 1 1 2 3 2 4

  56. Not a Tree

  57. Not a Tree

  58. Complications

  59. Complications

  60. 1 Complications

  61. 1 2 Complications

  62. 1 2 3 Complications

  63. 1 2 3 4 Complications

  64. 1 2 3 4 5 Complications

  65. Breadth-First Search

  66. Breadth-First Search 1

  67. Breadth-First Search 1 2

  68. Breadth-First Search 1 2 3

  69. Breadth-First Search 1 2 3 4

  70. Breadth-First Search 1 2 3 4 5

  71. Breadth-First Search 1 2 3 4 5 6

  72. Breadth-First Search 1 2 3 4 5 6 7

  73. Breadth-First Search 1 2 3 4 5 6 7 8

  74. Breadth-First Search 1 2 3 4 5 6 7 8

    9
  75. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10
  76. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11
  77. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12
  78. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13
  79. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14
  80. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15
  81. Breadth-First Search 1 2 3 4 5 6 7 8

    9 10 11 12 13 14 15 16
  82. Breadth-First Search 1 2 5 3 7 8 4 9

    10 11 6 12 13 14 15 16
  83. Breadth-First Search Global visited list works now

  84. Optimization Do Less Things Do Things Faster Heuristics

  85. Optimization Do Less Things: Process fewer states

  86. Optimization Do Things Faster: Less work per state

  87. Optimization Heuristics: Rules of Thumb Less certain; may work in

    some circumstances, but not others
  88. Heuristic: Move Active Robot First States Considered 0 350000 700000

    1050000 1400000 Original Algorithm Active First
  89. Do Less Things: Check for Solutions at Generation Time 1

    2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
  90. Do Less Things: Check for Solutions at Generation Time 1

    2 3 4 5 6 X X X X X X X X X 16
  91. Do Less Things: Check for Solutions at Generation Time 0

    800,000 1,600,000 2,400,000 3,200,000 Original Algorithm Check at Generation Total States Considered
  92. Do Things Faster: Precompute stopping cells

  93. Do Things Faster: Precompute stopping cells States / second 0

    675 1350 2025 2700 Original Algorithm Pre-compute Stops
  94. Do Less Things / Do Things Faster: Treat non-active robots

    as equivalent
  95. Do Less Things / Do Things Faster: Treat non-active robots

    as equivalent 0 1,000,000 2,000,000 3,000,000 4,000,000 Original Algorithm Check at Generation Robot Equivalence Total States Considered
  96. Do Less Things / Do Things Faster: Treat non-active robots

    as equivalent States / second 0 750 1500 2250 3000 Original Algorithm Pre-compute Stops Robot Equiv.
  97. Do Things Faster: Sorted Array vs Set States / second

    0 800 1600 2400 3200 Original Algorithm Pre-compute Stops Robot Equiv. Arrays not Sets
  98. Do Things Faster: Less Object Creation States / second 0

    1250 2500 3750 5000 Original Algorithm Pre-compute Stops Robot Equiv. Arrays not Sets Less Objects
  99. Do Things Faster: Use Object Identity Instead of Deep Equality

    States / second 0 1750 3500 5250 7000 Original Algorithm Pre-compute Stops Robot Equiv. Arrays not Sets Less Objects Object Identity
  100. Results So Far Solving time (seconds) 0 750 1500 2250

    3000 Original Active First Check at Gen. Pre-compute Robot Equiv. Arrays not Sets Less Objects Robot Identity
  101. Best-First Search

  102. Best-First Search 1

  103. Best-First Search 1 2

  104. Best-First Search 1 2 3

  105. Best-First Search 1 2 3 4

  106. Best-First Search 1 2 3 4 5

  107. Best-First Search 1 2 3 4 5 6

  108. What is Best? A* Algorithm

  109. A* Algorithm Each state is scored Lowest score is best

  110. A* Algorithm Score = Distance so far + Estimated distance

    left
  111. A* Algorithm Must not over-estimate

  112. A* Algorithm 0

  113. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 0
  114. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 0
  115. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 0
  116. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 0
  117. A* Algorithm 1 1 1 1 1 1 1 1

    1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 0
  118. Future Ideas: Better Algorithms Collision Detection

  119. Future Ideas: Better Algorithms Work Backwards

  120. Work Backwards

  121. Future Ideas: Better Heuristics Move most-recently moved robots first (MRU)

  122. Future Ideas: Better Heuristics Some combination of active robot and

    MRU robots
  123. Future Ideas: Better Heuristics Sort movement directions intelligently

  124. Future Ideas: More Optimizations Pre-compute per-robot stopping positions

  125. Future Ideas: More Optimizations Use less objects and more primitive

    types
  126. Future Ideas: More Optimizations Parallelism

  127. Acknowledgements • Trever Yarrish of Zeal for the awesome graphics

    and visualizations • My fellow Zeals for ideas, feedback, and pairing on the solver • Michael Fogleman for some optimization ideas • Trevor Lalish-Menagh for introducing me to the game • Screen Captures from War Games. (Dir. John Badham. MGM/UA. 1983)
  128. Questions ? http://randycoulman.com http://speakerdeck.com/randycoulman http://speakerrate.com/randycoulman Code is on Github https://github.com/CodingZeal/robots

  129. www .codingzeal.com @codingzeal Thank You! Randy Coulman @randycoulman