Reinforcement learning in PyGame

Presentation for PyData 21 on reinforcement learning in the PyGame framework

Transcript

1. Reinforcement learning in PyGame By Daniel K Slater
2. What is reinforcement learning? • Agents are run within an environment. • As they take actions they receive feedback • They aim to maximize good feedback and minimize bad feedback • Computer games are a great way to train reinforcement learning agents. we know we can learn games from just a sequence of images, so computer agents should be able to do the same thing (given enough computational power, time and the right algorithms).
3. Deep learning in reinforcement learning • Google deepmind recently published a set of papers which have the best performance for reinforcement learning on a number of atari games such as Pong • They use a deep convolutional architecture to turn a 192x160 image into a much smaller representation of the state of the game A technique called Q-learning is then used to learn the best actions from those states
4. • http://www.arcadelearningenvironment.org/ • Framework for running agents against atari 2600 games • Lots of atari roms now supported • Used in many top research papers on reinforcement learning Arcade learning environment (ALE)
5. PyGame • http://pygame.org/ • Most popular python games framework • 1000’s of games, all free, all open source • All written in Python
6. Everything being in python is great • Its easy to make modifications to the game, such as removing a menu screen We can grab values from within the game to use for reinforcement ◦ In Pong we can get reinforcement from the scores ◦ In Tetris we can get +ve reinforcement by intercepting the removeCompleteLines function ◦ In a platform game we can grab the player's x position to encourage exploration
7. PyGamePlayer • https://github.com/DanielSlater/PyGamePlayer • Allows running of PyGame games with zero touches • Handles intercepting screen buffer and key presses • Fixes the game frame rates
8. Thank you! contact me @: http://www.danielslater.net/