Shall We Play a Game?

From LoneStar Ruby 2015.

Teaching computers to play games has been a pursuit and passion for many programmers. Game playing has led to many advances in computing over the years, and the best computerized game players have gained a lot of attention from the general public (think Deep Blue and Watson).

Using the Ricochet Robots board game as an example, let's talk about what's involved in teaching a computer to play games. Along the way, we'll touch on graph search techniques, data representation, algorithms, heuristics, pruning, and optimization.

Transcript

1. www .codingzeal.com @codingzeal Randy Coulman Senior Software Engineer @randycoulman
2. Computers Playing Games
3. Computers Playing Games
4. Computers Playing Games
5. Why Graphs, Trees & Algorithms? Hierarchical structures: Organization charts
6. Why Graphs, Trees & Algorithms? Relationship Maps: Social Networks like LinkedIn & Facebook
7. Why Graphs, Trees & Algorithms? Trip Planning
8. Why Graphs, Trees & Algorithms? Network Routing
9. Game Playing
10. Ricochet Robots
11. Moves to get to
12. 1 1. Blue Right Moves to get to
13. 1 2 1. Blue Right 2. Blue Down Moves to get to
14. 1 2 3 1. Blue Right 2. Blue Down 3. Blue Left Moves to get to
15. 1 2 3 4 1. Blue Right 2. Blue Down 3. Blue Left 4. Green Right Moves to get to
16. 1 2 3 4 5 1. Blue Right 2. Blue Down 3. Blue Left 4. Green Right 5. Green Down Moves to get to
17. 1 2 3 4 5 6 1. Blue Right 2. Blue Down 3. Blue Left 4. Green Right 5. Green Down 6. Green Left Moves to get to
18. 1 2 3 4 5 6 7 1. Blue Right 2. Blue Down 3. Blue Left 4. Green Right 5. Green Down 6. Green Left 7. Green Down Moves to get to
19. Characterizing the Problem Possible Board States (size of state space): 252 * 251 * 250 * 249 * 248 = 976,484,376,000
20. Characterizing the Problem Branching Factor: 9 - 20 possible moves from each state
21. The Board • Board (Static: 16 x 16)
22. The Board • Board (Static: 16 x 16) • Walls & Targets (changes each game)
23. The Board • Board (Static: 16 x 16) • Walls & Targets (changes each game) • Goal (changes each turn)
24. The Board • Board (Static: 16 x 16) • Walls & Targets (changes each game) • Goal (changes each turn) • Robot Positions (changes each move)
25. Robot Movement 1 2 1 2 1
26. Trees
27. Search Algorithms See Jamis Buck’s book Basil & Fabian: Three Ways Through
28. Depth-First Search
29. Depth-First Search 1
30. Depth-First Search 1 2
31. Depth-First Search 1 2 3
32. Depth-First Search 1 2 3 4
33. Depth-First Search 1 2 3 4 5
34. Depth-First Search 1 2 3 4 5 6
35. Depth-First Search 1 2 3 4 5 6 7
36. Depth-First Search 1 2 3 4 5 6 7 8
37. Depth-First Search 1 2 3 4 5 6 7 8 9
38. Depth-First Search 1 2 3 4 5 6 7 8 9 10
39. Depth-First Search 1 2 3 4 5 6 7 8 9 10 11
40. Depth-First Search 1 2 3 4 5 6 7 8 9 10 11 12
41. Depth-First Search 1 2 3 4 5 6 7 8 9 10 11 12 13
42. Depth-First Search 1 2 3 4 5 6 7 8 9 10 11 12 13 14
43. Depth-First Search 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
44. Depth-First Search 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
45. Depth-First Search 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
46. Cycles
47. Cycles
48. Cycles
49. Cycles
50. Cycles
51. Cycles
52. Cycles
53. Cycles
54. Cycles
55. Visited States 3 1 1 2 3 2 4
56. Not a Tree
57. Not a Tree
58. Complications
59. Complications
60. 1 Complications
61. 1 2 Complications
62. 1 2 3 Complications
63. 1 2 3 4 Complications
64. 1 2 3 4 5 Complications
65. Breadth-First Search
66. Breadth-First Search 1
67. Breadth-First Search 1 2
68. Breadth-First Search 1 2 3
69. Breadth-First Search 1 2 3 4
70. Breadth-First Search 1 2 3 4 5
71. Breadth-First Search 1 2 3 4 5 6
72. Breadth-First Search 1 2 3 4 5 6 7
73. Breadth-First Search 1 2 3 4 5 6 7 8
74. Breadth-First Search 1 2 3 4 5 6 7 8 9
75. Breadth-First Search 1 2 3 4 5 6 7 8 9 10
76. Breadth-First Search 1 2 3 4 5 6 7 8 9 10 11
77. Breadth-First Search 1 2 3 4 5 6 7 8 9 10 11 12
78. Breadth-First Search 1 2 3 4 5 6 7 8 9 10 11 12 13
79. Breadth-First Search 1 2 3 4 5 6 7 8 9 10 11 12 13 14
80. Breadth-First Search 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
81. Breadth-First Search 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
82. Breadth-First Search 1 2 5 3 7 8 4 9 10 11 6 12 13 14 15 16
83. Breadth-First Search Global visited list works now
84. Optimization Do Less Things Do Things Faster Heuristics
85. Optimization Do Less Things: Process fewer states
86. Optimization Do Things Faster: Less work per state
87. Optimization Heuristics: Rules of Thumb Less certain; may work in some circumstances, but not others
88. Heuristic: Move Active Robot First States Considered 0 350000 700000 1050000 1400000 Original Algorithm Active First
89. Do Less Things: Check for Solutions at Generation Time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
90. Do Less Things: Check for Solutions at Generation Time 1 2 3 4 5 6 X X X X X X X X X 16
91. Do Less Things: Check for Solutions at Generation Time 0 800,000 1,600,000 2,400,000 3,200,000 Original Algorithm Check at Generation Total States Considered
92. Do Things Faster: Precompute stopping cells
93. Do Things Faster: Precompute stopping cells States / second 0 675 1350 2025 2700 Original Algorithm Pre-compute Stops
94. Do Less Things / Do Things Faster: Treat non-active robots as equivalent
95. Do Less Things / Do Things Faster: Treat non-active robots as equivalent 0 1,000,000 2,000,000 3,000,000 4,000,000 Original Algorithm Check at Generation Robot Equivalence Total States Considered
96. Do Less Things / Do Things Faster: Treat non-active robots as equivalent States / second 0 750 1500 2250 3000 Original Algorithm Pre-compute Stops Robot Equiv.
97. Do Things Faster: Sorted Array vs Set States / second 0 800 1600 2400 3200 Original Algorithm Pre-compute Stops Robot Equiv. Arrays not Sets
98. Do Things Faster: Less Object Creation States / second 0 1250 2500 3750 5000 Original Algorithm Pre-compute Stops Robot Equiv. Arrays not Sets Less Objects
99. Do Things Faster: Use Object Identity Instead of Deep Equality States / second 0 1750 3500 5250 7000 Original Algorithm Pre-compute Stops Robot Equiv. Arrays not Sets Less Objects Object Identity
100. Results So Far Solving time (seconds) 0 750 1500 2250 3000 Original Active First Check at Gen. Pre-compute Robot Equiv. Arrays not Sets Less Objects Robot Identity
101. Best-First Search
102. Best-First Search 1
103. Best-First Search 1 2
104. Best-First Search 1 2 3
105. Best-First Search 1 2 3 4
106. Best-First Search 1 2 3 4 5
107. Best-First Search 1 2 3 4 5 6
108. What is Best? A* Algorithm
109. A* Algorithm Each state is scored Lowest score is best
110. A* Algorithm Score = Distance so far + Estimated distance left
111. A* Algorithm Must not over-estimate
112. A* Algorithm 0
113. A* Algorithm 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0
114. A* Algorithm 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 0
115. A* Algorithm 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 0
116. A* Algorithm 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 0
117. A* Algorithm 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 0
118. Future Ideas: Better Algorithms Collision Detection
119. Future Ideas: Better Algorithms Work Backwards
120. Work Backwards
121. Future Ideas: Better Heuristics Move most-recently moved robots first (MRU)
122. Future Ideas: Better Heuristics Some combination of active robot and MRU robots
123. Future Ideas: Better Heuristics Sort movement directions intelligently
124. Future Ideas: More Optimizations Pre-compute per-robot stopping positions
125. Future Ideas: More Optimizations Use less objects and more primitive types
126. Future Ideas: More Optimizations Parallelism
127. Acknowledgements • Trever Yarrish of Zeal for the awesome graphics and visualizations • My fellow Zeals for ideas, feedback, and pairing on the solver • Michael Fogleman for some optimization ideas • Trevor Lalish-Menagh for introducing me to the game • Screen Captures from War Games. (Dir. John Badham. MGM/UA. 1983)
128. Questions ? http://randycoulman.com http://speakerdeck.com/randycoulman http://speakerrate.com/randycoulman Code is on Github https://github.com/CodingZeal/robots
129. www .codingzeal.com @codingzeal Thank You! Randy Coulman @randycoulman