Playing Video Games One Frame at a Time

Slide 1

Slide 1 text

1 TEST SLIDE Text is usually about this size and color It might go this far down. ^ code will be this color on this background and this example is roughly the smallest it will get. It will usually be bigger. If you can’t read it, move closer please.

Slide 2

Slide 2 text

Playing Video Games One Frame at a Time Ólafur Waage

Slide 3

Slide 3 text

Ólafur Waage Senior Software Developer - Factlines @olafurw on most social platforms 1 3

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

“ People say nothing is impossible, but I do nothing every day - Winnie the Pooh 7

Slide 8

Slide 8 text

“ People say nothing is impossible, but I do nothing every day - Winnie the Pooh 8

Slide 9

Slide 9 text

WHO AM I?

Slide 10

Slide 10 text

Worked on ATC systems for a couple of years. In the gaming industry for 7 years. As a consultant in Norway for a couple of years.

Slide 11

Slide 11 text

Worked on ATC systems for a couple of years. In the gaming industry for 7 years. As a consultant in Norway for a couple of years.

Slide 12

Slide 12 text

Worked on ATC systems for a couple of years. In the gaming industry for 7 years. As a consultant in Norway for a couple of years.

Slide 13

Slide 13 text

WHAT IS THIS TALK ABOUT?

Slide 14

Slide 14 text

What’s going on here? A good way to describe this talk is: 14

Slide 15

Slide 15 text

What’s going on here? A good way to describe this talk is: - A bit of silly fun 15

Slide 16

Slide 16 text

What’s going on here? A good way to describe this talk is: - A bit of silly fun - A bit of curiosity 16

Slide 17

Slide 17 text

What’s going on here? A good way to describe this talk is: - A bit of silly fun - A bit of curiosity - A bit of real life use cases 17

Slide 18

Slide 18 text

What’s going on here? A good way to describe this talk is: - A bit of silly fun - A bit of curiosity - A bit of real life use cases This is probably not an area a lot of people have thought about, so let’s explore together. 18

Slide 19

Slide 19 text

Last year? Also… 19

Slide 20

Slide 20 text

Last year? Also… I did a talk last year, 20

Slide 21

Slide 21 text

Last year? Also… I did a talk last year, that was way more serious than this one. 21

Slide 22

Slide 22 text

Last year? Also… I did a talk last year, that was way more serious than this one. (about how humans learn information) 22

Slide 23

Slide 23 text

Last year? Also… I did a talk last year, that was way more serious than this one. (about how humans learn information) And in that talk I said this… 23

Slide 24

Slide 24 text

"Unfortunately, in this case there is considerable evidence that in many situations this strategy…

Slide 25

Slide 25 text

"Unfortunately, in this case there is considerable evidence that in many situations this strategy (1 hour lectures)

Slide 26

Slide 26 text

"Unfortunately, in this case there is considerable evidence that in many situations this strategy (1 hour lectures) is rather poor (Bligh, 1971).

Slide 27

Slide 27 text

"Unfortunately, in this case there is considerable evidence that in many situations this strategy (1 hour lectures) is rather poor (Bligh, 1971). A lecture may inspire, motivate and inform, but rarely fulﬁls the principles for effective learning; it is certainly inadequate for developing high-level skills"

Slide 28

Slide 28 text

Last year? So, I can’t in good conscience make a talk this year that tries to effectively teach you something. 28

Slide 29

Slide 29 text

Last year? So, I can’t in good conscience make a talk this year that tries to effectively teach you something. Then instead I decided to make a fun talk that does have some real life applications that you might not have thought about. 29

Slide 30

Slide 30 text

WARNING

Slide 31

Slide 31 text

Before I start 31

Slide 32

Slide 32 text

Before I start I will do stuff to video games in this talk. 32

Slide 33

Slide 33 text

Before I start I will do stuff to video games in this talk. If you are going to do similar stuff, please, and I’m asking nicely, only do this to single player non-online games. 33

Slide 34

Slide 34 text

Before I start I will do stuff to video games in this talk. If you are going to do similar stuff, please, and I’m asking nicely, only do this to single player non-online games. I’m not going to do crazy stuff here, but I feel the need to add this warning. 34

Slide 35

Slide 35 text

BORING BACKSTORY

Slide 36

Slide 36 text

Playing video games is great fun, but I’m also a programmer.

Slide 37

Slide 37 text

Playing video games is great fun, but I’m also a programmer. How can I marry these two things together?

Slide 38

Slide 38 text

Playing video games is great fun, but I’m also a programmer. How can I marry these two things together? Well, you can make games.

Slide 39

Slide 39 text

Playing video games is great fun, but I’m also a programmer. How can I marry these two things together? Well, you can make games. But what if there was another way?

Slide 40

Slide 40 text

Games are just applications, so what can we do to control those applications remotely?

Slide 41

Slide 41 text

Games are just applications, so what can we do to control those applications remotely? What are the different challenges you will face when trying to interact with complex software like this?

Slide 42

Slide 42 text

Games are just applications, so what can we do to control those applications remotely? What are the different challenges you will face when trying to interact with complex software like this? And can some of this apply to interacting with other software?

Slide 43

Slide 43 text

Level 1

Slide 44

Slide 44 text

Level 1 What if the game you want to play supports playing it programmatically, right out of the box?

Slide 45

Slide 45 text

Slide 46

Slide 46 text

Doom is a 1993 ﬁrst person shooter.

Slide 47

Slide 47 text

Doom is a 1993 ﬁrst person shooter. And it kind of redeﬁned a few aspects of video games.

Slide 48

Slide 48 text

Doom is a 1993 first person shooter. And it kind of redefined a few aspects of video games. - The look and feel of first person shooters

Slide 49

Slide 49 text

Doom is a 1993 first person shooter. And it kind of redefined a few aspects of video games. - The look and feel of first person shooters - Online multiplayer

Slide 50

Slide 50 text

Doom is a 1993 first person shooter. And it kind of redefined a few aspects of video games. - The look and feel of first person shooters - Online multiplayer - Game modification

Slide 51

Slide 51 text

Doom is a 1993 first person shooter. And it kind of redefined a few aspects of video games. - The look and feel of first person shooters - Online multiplayer - Game modification - Video Game Speedrunning

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Whenever there is a timer, people will try to be the best, be the fastest.

Slide 54

Slide 54 text

Whenever there is a timer, people will try to be the best, be the fastest. Doom is no exception.

Slide 55

Slide 55 text

Whenever there is a timer, people will try to be the best, be the fastest. Doom is no exception. Because at the end of every stage, there was a timer.

Slide 56

Slide 56 text

Whenever there is a timer, people will try to be the best, be the fastest. Doom is no exception. Because at the end of every stage, there was a timer. And as you can see, people are still playing, still trying to get better.

Slide 57

Slide 57 text

And if you download these ﬁles, you’ll notice that they’re actually pretty small.

Slide 58

Slide 58 text

And if you download these ﬁles, you’ll notice that they’re actually pretty small. The reason is that these ﬁles basically only contain what inputs the user pressed and during what frame.

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

No content

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

This is possible because of one thing.

Slide 66

Slide 66 text

This is possible because of one thing. Doom is deterministic

Slide 67

Slide 67 text

This is possible because of one thing. Doom is deterministic Doom is a single threaded application where every single entity in the map patiently waits their turn, always in the same order.

Slide 68

Slide 68 text

This is possible because of one thing. Doom is deterministic Doom is a single threaded application where every single entity in the map patiently waits their turn, always in the same order.

Slide 69

Slide 69 text

This is possible because of one thing. Doom is deterministic Doom is a single threaded application where every single entity in the map patiently waits their turn, always in the same order. And when they want to do something random, they extract a number from this table

Slide 70

Slide 70 text

Slide 71

Slide 71 text

So what if you wanted to take Doom and convert it to C++ (or some language)

Slide 72

Slide 72 text

So what if you wanted to take Doom and convert it to C++ (or some language) How can you move forwards with that project knowing that your code changes and refactorings haven’t broken the game?

Slide 73

Slide 73 text

So what if you wanted to take Doom and convert it to C++ (or some language) How can you move forwards with that project knowing that your code changes and refactorings haven’t broken the game? We could try playing the game, or writing unit tests.

Slide 74

Slide 74 text

Slide 75

Slide 75 text

Approval testing is a methodology where you focus on comparing the output of the old known bit of code with the output of your new code.

Slide 76

Slide 76 text

Slide 77

Slide 77 text

Approval testing is a methodology where you focus on comparing the output of the old known bit of code with the output of your new code. Think of it like a git diff. If there is a difference, then something broke. This is useful when you want to do a complete rewrite or when the original source might not be available in full.

Slide 78

Slide 78 text

Doom can run the old demo ﬁles and then give you a nice statistics screen at the end.

Slide 79

Slide 79 text

Doom can run the old demo ﬁles and then give you a nice statistics screen at the end. How long you played for, how many monsters you killed, how many items you got and how many secrets you found.

Slide 80

Slide 80 text

Doom can run the old demo ﬁles and then give you a nice statistics screen at the end. How long you played for, how many monsters you killed, how many items you got and how many secrets you found. Well, if I play this demo on the new version, I should get the exact same results.

Slide 81

Slide 81 text

Another tidbit. The game doesn’t need to render the game to the screen for the simulation to run.

Slide 82

Slide 82 text

Another tidbit. The game doesn’t need to render the game to the screen for the simulation to run. The graphics are just a nice visual representation of the game state.

Slide 83

Slide 83 text

Slide 84

Slide 84 text

Another tidbit. The game doesn’t need to render the game to the screen for the simulation to run. The graphics are just a nice visual representation of the game state. So we can add (and modify) some game ﬂags where the game can run in the terminal. This also means we don’t need to run the game at a nice frame rate, we can just run it as fast as possible.

Slide 85

Slide 85 text

Slide 86

Slide 86 text

Putting 1 and 1 together. We can play the game using a known good version of the game.

Slide 87

Slide 87 text

Putting 1 and 1 together. We can play the game using a known good version of the game. Collect the resulting Stats output.

Slide 88

Slide 88 text

Putting 1 and 1 together. We can play the game using a known good version of the game. Collect the resulting Stats output. Save the demo ﬁles and the stats text ﬁles in a folder.

Slide 89

Slide 89 text

Putting 1 and 1 together. We can play the game using a known good version of the game. Collect the resulting Stats output. Save the demo files and the stats text files in a folder. Run the game, in command line using those demos and check if the resulting stats data match the corresponding text file.

Slide 90

Slide 90 text

Doom running in GitHub Actions.

Slide 91

Slide 91 text

Doom running in GitHub Actions.

Slide 92

Slide 92 text

Doom running in GitHub Actions.

Slide 93

Slide 93 text

Doom running in GitHub Actions.

Slide 94

Slide 94 text

Level 2

Slide 95

Slide 95 text

Level 2 What if the game you want to play doesn't have the support Doom does?

Slide 96

Slide 96 text

Slide 97

Slide 97 text

Bejeweled 3 is a match three game.

Slide 98

Slide 98 text

Bejeweled 3 is a match three game. Pick a gem, move it left/right or up/down to create three of a kind.

Slide 99

Slide 99 text

Bejeweled 3 is a match three game. Pick a gem, move it left/right or up/down to create three of a kind. You can also join four or more gems for bonus points.

Slide 100

Slide 100 text

Bejeweled 3 is a match three game. Pick a gem, move it left/right or up/down to create three of a kind. You can also join four or more gems for bonus points. After a match has been made, those gems go away and new ones drop down.

Slide 101

Slide 101 text

For Bejeweled 3 the plan of action is pretty simple

Slide 102

Slide 102 text

For Bejeweled 3 the plan of action is pretty simple - What Gem is where

Slide 103

Slide 103 text

For Bejeweled 3 the plan of action is pretty simple - What Gem is where - Pick a good move

Slide 104

Slide 104 text

For Bejeweled 3 the plan of action is pretty simple - What Gem is where - Pick a good move - Move that gem

Slide 105

Slide 105 text

For Bejeweled 3 the plan of action is pretty simple - What Gem is where - Pick a good move - Move that gem - Repeat

Slide 106

Slide 106 text

The Project Project Setup: - C++ - CMake - VCPKG - and OpenCV 106

Slide 107

Slide 107 text

The Project Project Setup: - C++ - CMake - VCPKG - and OpenCV 107

Slide 108

Slide 108 text

Step 1 What Gem is Where?

Slide 109

Slide 109 text

No content

Slide 110

Slide 110 text

64px Step 1 What Gem is Where? The game board is ﬁxed resolution, which means each cell is 64 by 64 pixels.

Slide 111

Slide 111 text

w: 261px h: 37px Step 1 What Gem is Where? The game board is ﬁxed resolution, which means each cell is 64 by 64 pixels. That means that the board starts at 261x37 from the top left of the game screen.

Slide 112

Slide 112 text

Step 1 What Gem is Where? The game board is ﬁxed resolution, which means each cell is 64 by 64 pixels. That means that the board starts at 261x37 from the top left of the game screen. From that information, I know exactly where each cell on the board is.

Slide 113

Slide 113 text

Step 1 What Gem is Where? But how do I know that this gem is the green one?

Slide 114

Slide 114 text

Step 1 What Gem is Where? But how do I know that this gem is the green one? I started simple, let’s take the center pixel of that gem and check the RGB color values.

Slide 115

Slide 115 text

Step 1 What Gem is Where? But how do I know that this gem is the green one? I started simple, let’s take the center pixel of that gem and check the RGB color values. But there’s a problem.

Slide 116

Slide 116 text

Step 1 What Gem is Where? But how do I know that this gem is the green one? I started simple, let’s take the center pixel of that gem and check the RGB color values. But there’s a problem.

Slide 117

Slide 117 text

Step 1 What Gem is Where? Let’s use OpenCV to our advantage.

Slide 118

Slide 118 text

Step 1 What Gem is Where? Let’s use OpenCV to our advantage.

Slide 119

Slide 119 text

Step 1 What Gem is Where? Let’s use OpenCV to our advantage. Using Template Matching we can look for our gems on the board.

Slide 120

Slide 120 text

Step 1 What Gem is Where? Let’s use OpenCV to our advantage. Using Template Matching we can look for our gems on the board. I created these nice sample images of each gem.

Slide 121

Slide 121 text

No content

Slide 122

Slide 122 text

No content

Slide 123

Slide 123 text

No content

Slide 124

Slide 124 text

Step 1 What Gem is Where? Let’s use OpenCV to our advantage. Using Template Matching we can look for our gems on the board. I created these nice sample images of each gem. So why doesn’t this work?

Slide 125

Slide 125 text

Slide 126

Slide 126 text

Step 1 What Gem is Where? Let’s go back to the idea of color.

Slide 127

Slide 127 text

Step 1 What Gem is Where? Let’s go back to the idea of color. What if I don’t check for 1 pixel in the center, but the general … hue … of the center area.

Slide 128

Slide 128 text

Step 1 What Gem is Where? Let’s go back to the idea of color. What if I don’t check for 1 pixel in the center, but the general … hue … of the center area. So I grab a 32x32 area in the center of the gem.

Slide 129

Slide 129 text

Slide 130

Slide 130 text

Slide 131

Slide 131 text

Slide 132

Slide 132 text

Step 1 What Gem is Where? So I went through and checked the Hue value of every single gem (when stationary and while spinning).

Slide 133

Slide 133 text

Step 1 What Gem is Where? So I went through and checked the Hue value of every single gem (when stationary and while spinning). And made a chart of where they overlap and where they are isolated.

Slide 134

Slide 134 text

Step 1 What Gem is Where? So I went through and checked the Hue value of every single gem (when stationary and while spinning). And made a chart of where they overlap and where they are isolated.

Slide 135

Slide 135 text

Step 1 What Gem is Where? So I went through and checked the Hue value of every single gem (when stationary and while spinning). And made a chart of where they overlap and where they are isolated. And then wrote a function that tries to disambiguate between the colors (and I used RGB to help also)

Slide 136

Slide 136 text

Slide 137

Slide 137 text

Slide 138

Slide 138 text

Slide 139

Slide 139 text

Slide 140

Slide 140 text

No content

Slide 141

Slide 141 text

Step 2 Pick a good move.

Slide 142

Slide 142 text

Step 2 Pick a good move. So what is a good move?

Slide 143

Slide 143 text

Step 2 Pick a good move. So what is a good move? Initially I was thinking of writing some cool algorithm, get some deep learning AI into the mix.

Slide 144

Slide 144 text

Step 2 Pick a good move. So what is a good move? Initially I was thinking of writing some cool algorithm, get some deep learning AI into the mix. But after the color ﬁasko, I wanted to see if something simple was enough.

Slide 145

Slide 145 text

Slide 146

Slide 146 text

Step 2 Pick a good move. Let’s start by playing the game a bit and see if anything pops out.

Slide 147

Slide 147 text

Step 2 Pick a good move. Let’s start by playing the game a bit and see if anything pops out. So I played the game a little bit. Just a bit.

Slide 148

Slide 148 text

Step 2 Pick a good move. Let’s start by playing the game a bit and see if anything pops out. So I played the game a little bit. Just a bit. And I had an idea.

Slide 149

Slide 149 text

Step 2 Pick a good move. Let’s start by playing the game a bit and see if anything pops out. So I played the game a little bit. Just a bit. And I had an idea. There aren’t actually that many moves in this game.

Slide 150

Slide 150 text

Slide 151

Slide 151 text

151

Slide 152

Slide 152 text

152

Slide 153

Slide 153 text

153

Slide 154

Slide 154 text

154

Slide 155

Slide 155 text

155

Slide 156

Slide 156 text

156

Slide 157

Slide 157 text

157

Slide 158

Slide 158 text

158

Slide 159

Slide 159 text

159

Slide 160

Slide 160 text

160

Slide 161

Slide 161 text

Step 2 Pick a good move. But which move should we pick?

Slide 162

Slide 162 text

Step 2 Pick a good move. But which move should we pick? Well a high value move is always sorted above other moves. But what if all moves have the same value?

Slide 163

Slide 163 text

Slide 164

Slide 164 text

Step 2 Pick a good move. But which move should we pick? Well a high value move is always sorted above other moves. But what if all moves have the same value? Then we pick a move that is lower on the board. Why? Because it creates more matching opportunities when tiles fall down. A move at the top only creates one.

Slide 165

Slide 165 text

Step 3 Move that gem. Now it’s time to interact with the game application directly.

Slide 166

Slide 166 text

Step 3 Move that gem. Now it’s time to interact with the game application directly. The Windows SDK has a lot of nice mouse input functions. So let’s use them.

Slide 167

Slide 167 text

Step 3 Move that gem. Now it’s time to interact with the game application directly. The Windows SDK has a lot of nice mouse input functions. So let’s use them. I basically want a function like this. I know where the gem is and I know what direction I want to move to.

Slide 168

Slide 168 text

No content

Slide 169

Slide 169 text

No content

Slide 170

Slide 170 text

No content

Slide 171

Slide 171 text

No content

Slide 172

Slide 172 text

No content

Slide 173

Slide 173 text

No content

Slide 174

Slide 174 text

No content

Slide 175

Slide 175 text

No content

Slide 176

Slide 176 text

No content

Slide 177

Slide 177 text

No content

Slide 178

Slide 178 text

No content

Slide 179

Slide 179 text

Bejeweled 3 DEMO

Slide 180

Slide 180 text

Level 3

Slide 181

Slide 181 text

Level 3 What if a game has none of these features?

Slide 182

Slide 182 text

182

Slide 183

Slide 183 text

183

Slide 184

Slide 184 text

VVVVVV Does not have built in support for giving the game the inputs.

Slide 185

Slide 185 text

VVVVVV Does not have built in support for giving the game the inputs. Does not have a simple game state.

Slide 186

Slide 186 text

VVVVVV Does not have built in support for giving the game the inputs. Does not have a simple game state. Is fast paced and you need to be pretty accurate with your movement.

Slide 187

Slide 187 text

187

Slide 188

Slide 188 text

188

Slide 189

Slide 189 text

Before I continue with the madness. There is one concept I need to talk about.

Slide 190

Slide 190 text

Before I continue with the madness. There is one concept I need to talk about. The Game Loop

Slide 191

Slide 191 text

Before I continue with the madness. There is one concept I need to talk about. The Game Loop Games are applications like any other, and they usually follow a certain set of steps.

Slide 192

Slide 192 text

Before I continue with the madness. There is one concept I need to talk about. The Game Loop Games are applications like any other, and they usually follow a certain set of steps. - Get input from user

Slide 193

Slide 193 text

Slide 194

Slide 194 text

Slide 195

Slide 195 text

There are a few caveats with these steps that makes our work harder.

Slide 196

Slide 196 text

There are a few caveats with these steps that makes our work harder. Get input from user There might be no input or if we handle it badly, we might delay it or miss it.

Slide 197

Slide 197 text

Slide 198

Slide 198 text

There are a few caveats with these steps that makes our work harder. Get input from user There might be no input or if we handle it badly, we might delay it or miss it. Update the state This can take a long time and we might multi-thread it. Draw the current state We might draw old state or we might offload the draw to happen in another thread.

Slide 199

Slide 199 text

199 Initialize

Slide 200

Slide 200 text

200 Initialize Process input

Slide 201

Slide 201 text

201 Initialize Process input Update game state

Slide 202

Slide 202 text

202 Initialize Process input Update game state Draw game state

Slide 203

Slide 203 text

203 Initialize Process input Update game state Draw game state

Slide 204

Slide 204 text

204 Initialize Process input Update game state Draw game state “Frame”

Slide 205

Slide 205 text

205 Initialize Process input Update game state Draw game state <1ms “Frame”

Slide 206

Slide 206 text

206 Initialize Process input Update game state Draw game state <1ms “Frame” Depends

Slide 207

Slide 207 text

207 Initialize Process input Update game state Draw game state <1ms “Frame” Depends Depends

Slide 208

Slide 208 text

208 Initialize Process input Update game state Draw game state “Frame” 60 fps game 16.666… ms

Slide 209

Slide 209 text

209 Initialize Process input Update game state Draw game state

Slide 210

Slide 210 text

210 Initialize Process input Update game state Draw game state At least once per 16.666 ms

Slide 211

Slide 211 text

211 Initialize Process input Update game state Draw game state *As often as we can At least once per 16.666 ms

Slide 212

Slide 212 text

212

Slide 213

Slide 213 text

213

Slide 214

Slide 214 text

214

Slide 215

Slide 215 text

215

Slide 216

Slide 216 text

216

Slide 217

Slide 217 text

217

Slide 218

Slide 218 text

218

Slide 219

Slide 219 text

219

Slide 220

Slide 220 text

220

Slide 221

Slide 221 text

221

Slide 222

Slide 222 text

222

Slide 223

Slide 223 text

223

Slide 224

Slide 224 text

VVVVVV actually has a few advantages and is why I picked this game for the project.

Slide 225

Slide 225 text

VVVVVV actually has a few advantages and is why I picked this game for the project. It’s open source, so we can just go check.

Slide 226

Slide 226 text

VVVVVV actually has a few advantages and is why I picked this game for the project. It’s open source, so we can just go check. It’s single threaded.

Slide 227

Slide 227 text

VVVVVV actually has a few advantages and is why I picked this game for the project. It’s open source, so we can just go check. It’s single threaded. It has a lot of customization for such a simple game.

Slide 228

Slide 228 text

So here is my plan Loader Target process dll

Slide 229

Slide 229 text

So here is my plan Launch the game as a paused child process of some loader. Loader

Slide 230

Slide 230 text

So here is my plan Launch the game as a paused child process of some loader. Loader Game Process

Slide 231

Slide 231 text

So here is my plan Launch the game as a paused child process of some loader. Loader Game Process dll

Slide 232

Slide 232 text

So here is my plan Launch the game as a paused child process of some loader. Hook the function that creates the DirectX Device. Loader Game Process dll

Slide 233

Slide 233 text

So here is my plan Launch the game as a paused child process of some loader. Hook the function that creates the DirectX Device. Get the address of that device. Loader Game Process dll

Slide 234

Slide 234 text

So here is my plan Launch the game as a paused child process of some loader. Hook the function that creates the DirectX Device. Get the address of that device. Intercept the game’s draw call and call our draw function ﬁrst. Loader Game Process dll

Slide 235

Slide 235 text

Slide 236

Slide 236 text

236 Initialize Process input Update game state Draw game state

Slide 237

Slide 237 text

237 Initialize Process input Update game state Draw game state

Slide 238

Slide 238 text

238 Initialize Process input Update game state Draw game state Intercept - Call game’s draw function - Send our inputs

Slide 239

Slide 239 text

239 Initialize Process input Update game state Draw game state We could hook the input polling function.

Slide 240

Slide 240 text

240 Initialize Process input Update game state Draw game state We could hook the input polling function. But we actually want to do a bit of drawing ourselves, to make a nice debug UI.

Slide 241

Slide 241 text

241

Slide 242

Slide 242 text

242

Slide 243

Slide 243 text

243

Slide 244

Slide 244 text

244

Slide 245

Slide 245 text

245

Slide 246

Slide 246 text

246

Slide 247

Slide 247 text

247

Slide 248

Slide 248 text

248

Slide 249

Slide 249 text

249

Slide 250

Slide 250 text

250

Slide 251

Slide 251 text

251

Slide 252

Slide 252 text

252

Slide 253

Slide 253 text

253

Slide 254

Slide 254 text

254

Slide 255

Slide 255 text

255

Slide 256

Slide 256 text

256

Slide 257

Slide 257 text

257

Slide 258

Slide 258 text

258

Slide 259

Slide 259 text

259

Slide 260

Slide 260 text

260

Slide 261

Slide 261 text

261

Slide 262

Slide 262 text

262

Slide 263

Slide 263 text

263

Slide 264

Slide 264 text

264

Slide 265

Slide 265 text

265

Slide 266

Slide 266 text

266

Slide 267

Slide 267 text

267

Slide 268

Slide 268 text

268

Slide 269

Slide 269 text

269

Slide 270

Slide 270 text

270

Slide 271

Slide 271 text

271

Slide 272

Slide 272 text

A summary of what we’ve done so far. Loader Game Process dll

Slide 273

Slide 273 text

A summary of what we’ve done so far. ✅ Spawn the game process paused. Loader Game Process dll

Slide 274

Slide 274 text

A summary of what we’ve done so far. ✅ Spawn the game process paused. ✅ Attach our DLL to the game process. Loader Game Process dll

Slide 275

Slide 275 text

A summary of what we’ve done so far. ✅ Spawn the game process paused. ✅ Attach our DLL to the game process. ✅ Hook the “CreateDevice” function the game is going to call once it starts. Loader Game Process dll

Slide 276

Slide 276 text

A summary of what we’ve done so far. ✅ Spawn the game process paused. ✅ Attach our DLL to the game process. ✅ Hook the “CreateDevice” function the game is going to call once it starts. Once the game calls the “CreateDevice” function, get the device it just made and hook end “EndScene” function. Loader Game Process dll

Slide 277

Slide 277 text

277

Slide 278

Slide 278 text

278

Slide 279

Slide 279 text

279

Slide 280

Slide 280 text

280

Slide 281

Slide 281 text

281

Slide 282

Slide 282 text

282

Slide 283

Slide 283 text

283

Slide 284

Slide 284 text

284

Slide 285

Slide 285 text

285

Slide 286

Slide 286 text

286

Slide 287

Slide 287 text

287

Slide 288

Slide 288 text

288

Slide 289

Slide 289 text

289

Slide 290

Slide 290 text

290

Slide 291

Slide 291 text

VVVVVV DEMO

Slide 292

Slide 292 text

Summary: Games are neat and it’s fun to play with them programmatically. Loader Game Process dll

Slide 293

Slide 293 text

Slide 294

Slide 294 text

Summary: Games are neat and it’s fun to play with them programmatically. When you have an application you don’t have the source code to, there are still ways to automatically interact with it. AKA. Yes, you can still test that old legacy app. Yes you can still do safe(er) refactoring. Loader Game Process dll

Slide 295

Slide 295 text

Slide 296

Slide 296 text

TASVideos.org Loader Game Process dll

Slide 297

Slide 297 text

TASVideos.org Loader Game Process dll

Slide 298

Slide 298 text

dsdarchive.com Loader Game Process dll

Slide 299

Slide 299 text

www.classicdoom.com/odddemos.htm Loader Game Process dll

Slide 300

Slide 300 text

VVVVVV https://github.com/olafurw/vvvvvv-hook/ Bejeweled 3 https://github.com/olafurw/bejeweled-player Doom https://github.com/patricia-gallardo/cpp-doom Loader Game Process dll

Slide 301

Slide 301 text

Ólafur Waage Senior Software Developer - Factlines @olafurw on most social platforms 1 301