Deep Reinforcement Learning DOOM

Invited talk given at Signal Media on my work applying deep reinforcement learning to the video game DOOM.

Transcript

1. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 Deep Reinforcement Learning
2. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // Second Year // iGGi Student // Using games to improve AI
3. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // Deep Neural Networks // Reinforcement Learning // CLYDE
4. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // Deep Neural Networks Output Inputs
5. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // Recurrent Cells
6. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // Language Modelling Char embedding next Char embedding LSTM Cells
7. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // Reinforcement Learning State Environment Action Reward Agent
8. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // Actor-Critic State Environment Action Reward Value Policy TD Error
9. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 π v // CLYDE - A3C
10. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // CLYDE - A3C Global Network Local Network Game Instance Local Network Game Instance Local Network Game Instance Local Network Game Instance
11. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // CLYDE - Rewards // Pickups: + 1 // Kills: +10 // Suicide: -10
12. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // TRAINING -20.0 -10.0 0.00 10.0 0.000 20.00M 40.00M 60.00M 80.00M Frags Game Ticks
13. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // Competition // Known map // Rocket Launcher // Most Frags Wins
14. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 // Results Place Team Bot 1 2 3 4 5 6 7 8 9 10 11 12 Total frags 1 F1 F1 56 62 n/a 54 47 43 47 55 50 48 50 47 559 2 The Terminators Arnold 36 34 42 36 36 45 36 39 n/a 33 36 40 413 3 CLYDE CLYDE 37 n/a 38 32 37 30 46 42 33 24 44 30 393 4 TUHO TUHO 39 25 32 31 21 19 21 n/a 35 35 24 30 312 5 5vision 5vision 15 17 10 12 8 n/a 18 15 10 16 10 11 142 6 ColbyCS ColbyMules 12 4 8 10 n/a 16 14 19 20 9 13 6 131 7 InDepth AbyssII 11 9 15 n/a 3 0 10 13 15 5 20 17 118 8 PotatoesArePrettyOk WallDestroyerXxx -20 -17 -15 -21 -16 -11 n/a -25 -5 n/a n/a n/a -130 9 Ivomi Ivomi n/a -81 -76 -65 -86 -76 -74 -62 -58 n/a n/a n/a -578
15. University of Essex Dino Ratcliffe | Signal | 9 NOV 2016 QUESTIONS?