Game AI in Game Industry

5th international game ai summer school talk
https://school.gameaibook.org/

Transcript

1. Game AI in Game Industry - Overview of Industrial Game AI – Youichiro Miyake @miyayou y.m.4160@gmail.com 5th International Game AI summer school 2023.6.29 https://www.facebook.com/youichiro.miyake https://miyayou.com
2. Youichiro Miyake Career highlights - AI Technical Advisor, “FINAL FANTASY XIV” in 2012 - Lead AI Architect, “FINAL FANTASY XV” in 2016 - AI Technical Director, “KINGDOM HEARTS III” in 2019 - QA Automation AI Technical Advisor, “FINAL FANTASY VII REMAKE” in 2020 - Director, Digital Games Research Association (DiGRA) JAPAN - Director, The Japanese Society for Artificial Intelligence Awards - 2008 Funai Award, Digital Contents Symposium 8th - 2012 Award for Programming/Development Environment Excellence, CEDEC - 2012 Youth Encouragement Award, DiGRA JAPAN - 2018 Book Award, CEDEC - 2021 Japanese Society of AI, Best Paper Award • SQEX AI Division, General Manager 20 members SQEX AI & Arts Alchemy, CTO 10members
3. Who is Youichiro Miyake ? • https://scholar.google.com/citations?user=CXqiolQAAAAJ • https://researchmap.jp/youichiro_miyake?lang=en • https://miyayou.com/
4. My Works (2004-2023) AI for Game Titles Books
5. AI Deep Neural Network 2D/3D Navigation Character System (GOAP, Emotional) Meta AI Natural Langage Programming Animation Generation Game AI (planning) Procedural Contents Generation
6. My belief •AI can change game design. •AI will make the new future of game.
7. Game AI Useful Link Lists by Youichiro Miyake • History of Game AI In Game Industry • https://www.ai-gakkai.or.jp/resource/my-bookmark/my- bookmark_vol37-no6/ • Fundamental Knowledge of Game AI in Game Industry • https://www.ai-gakkai.or.jp/resource/my-bookmark/my- bookmark_vol32-no4/
8. My English lecture on YouTube • GDC 2018 • Eos is Alive: The AI Systems of Final Fantasy XV • https://www.youtube.com/watch?v=ygNRNru1B_s • Replaying Japan • Keynote presentation: Youichiro Miyake • https://www.youtube.com/watch?v=9pIgpktZW3o
9. FFXV in Game AI Pro (Free download) • http://www.gameaipro.com/ • A Character Decision-Making System for FINAL FANTASY XV by Combining Behavior Trees and State Machines • Youichiro Miyake, Youji Shirakami, Kazuya Shimokawa, Kousuke Namiki, Tomoki Komatsu, Joudan Tatsuhiro, Prasert Prasertvithyakarn, and Takanori Yokoyama • http://www.gameaipro.com/GameAIPro3/GameAIPro3_Chapter11_A_Character_Decision- Making_System_for_FINAL_FANTASY_XV_by_Combining_Behavior_Trees_and_State_Machines .pdf • Logging Visualization in FINAL FANTASY XV • Matthew W. Johnson, Fabien Gravot, Shintaro Minamino, Ingimar Hólm Guðmundsson, Hendrik Skubch, and Youichiro Miyake • http://www.gameaipro.com/GameAIPro3/GameAIPro3_Chapter03_Logging_Visualization_in_FI NAL_FANTASY_XV.pdf • Predictive Animation Control Using Simulations and Fitted Models • Ingimar Hólm Guðmundsson, Hendrik Skubch, Fabien Gravot, and Youichiro Miyake • http://www.gameaipro.com/GameAIPro3/GameAIPro3_Chapter16_Predictive_Animation_Contr ol_Using_Simulations_and_Fitted_Models.pdf
10. FFXIV in Game AI Pro (Free download) • http://www.gameaipro.com/ • Precomputed Pathfinding for Large and Detailed Worlds on MMO Servers • Fabien Gravot, Takanori Yokoyama, Youichiro Miyake • http://www.gameaipro.com/GameAIPro/GameAIPro_Chapter20_Precomputed_Pathfinding_for _Large_and_Detailed_Worlds_on_MMO_Servers.pdf • Game Balancing AI • Game Balancing using Genetic Algorithms to Generate Player Agents • Kazuko Manabe and Youichiro Miyake • http://www.gameaipro.com/GameAIProOnlineEdition2021/GameAIProOnlineEdition2021_Chap ter17_Game_Balancing_using_Genetic_Algorithms_to_Generate_Player_Agents.pdf
11. GDC talk in 2018, 2022 • Eos is Alive: The AI Systems of 'Final Fantasy XV‘ (2018) • https://www.gdcvault.com/play/1025115/Eos-is-Alive-The-AI • AI Summit: Meta-Character-Spatial AI Dynamic Cooperative Model in Digital Game AI (2022) • https://www.gdcvault.com/play/1027641/AI-Summit-Meta- Character-Spatial
12. INTRODUCTION Chapter 2
13. How to design AI ? Player Character Rock Ground Pond For example, how to design AI in this situation ?
14. How to design AI ? Player Character Rock Ground Pond For example, how to design AI in this situation ? Ａ B （Example） If the player is in A, B goes around in B. If the player is in B, B attacks A.
15. How to design AI ? Player Character Rock Ground Pond For example, how to design AI in this situation ? Ａ B （Example） If the player is in A, B goes around in B. If the player is in B, B attack A. But this statement is not the AI’s idea. A game designer writes these ideas. = They are orders to AI. Classical AI = Scripted AI
16. Script From Scripted AI to Autonomous AI Game Designer’s idea Game Designer’s idea Knowledge Thinking Scripted AI Autonomous AI
17. Script From Scripted AI to Autonomous AI Game Designer’s idea Game Designer’s idea Knowledge Thiniking Scripted AI Autonomous AI No longer a puppet（scripted AI）. A character should be an autonomous AI which thinks by itself. Knowledge and thinking should be given to an AI.
18. Player Character Rock Ground Pond How to design AI ? Ａ B Terrain data （Way Points） Step 1 Let AI understand terrain ＝ Give terrain data to AI AI can guess it relative position to a player by using terrain data.
19. Navigation AI Thinking about positon by using terrain data Player Character Rock Ground Pond How to design AI ? Ａ B Terrain data （Way Points） Step 2 Let AI know its position by using terrain data. AI can guess its relative position to a player by using terrain data.
20. Navigation AI How to design AI ? Thinking by using terrain data Networked graph search Terrain data （Way Points） Thinking by using Terrain data Calculate the shortest path
21. Navigation AI How to design AI ? Thinking by using terrain data Networked graph search Terrain data （Way Points） Thinking by using Terrain data Calculate the shortest path
22. Navigation AI How to design AI ? Thinking by using terrain data Networked graph search Terrain data （Way Points） Thinking by using Terrain data Calculate the shortest path
23. Navigation AI How to design AI ? Make Networked Graph include LOS (Line of Sight) data. Terrain data （Way Points） Thinking by using Terrain data All points can be seen from any point in a group. All points can be seen from any point in a group.
24. Navigation AI How to design AI ? Make Networked Graph include LOS (Line of Sight) data. Terrain data （Way Points） Thinking by using Terrain data Any point of the group can not be seen any point in the group.
25. Navigation AI How to design AI ? An AI can be seen by the player. Terrain data （Way Points） Thinking by using Terrain data
26. How to design AI ? The player can not see the AI. Navigation AI Terrain data （Way Points） Thinking by using Terrain data
27. How to design AI ? The player can see the AI. Navigation AI Terrain data （Way Points） Thinking by using Terrain data
28. How to design AI ? The player can not see the AI. Navigation AI Terrain data （Way Points） Thinking by using Terrain data
29. How to design AI ? The AI will move so the player can see it. Navigation AI Terrain data （Way Points） Thinking by using Terrain data
30. How to design AI ? The AI will move so the player can not see it. Navigation AI Terrain data （Way Points） Thinking by using Terrain data
31. Thinking its positon by using terrain data Player Character Rock Ground Pond How to design AI ? Terrain data （Way Points） Step 3 Let AI think about timing when to attack. Decision Making
32. How to design AI ? Attack? Hide? Threaten? selection ＝ Decision Making Thinking Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making
33. How to design AI ? Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making Attack
34. How to design AI ? Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making Threaten (= Going to the positon where a player can see AI)
35. How to design AI ? Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making Hide（＝AI goes to the positon where a player can not see AI）
36. Thinking its positon by using terrain data Player Character Rock Ground Pond How to design AI ? Terrain data （Way Points） Step 1- 3 Let AI be intelligent. Decision Making
37. Making Autonomous AI Game Designer (Human) Terrain data（Way Points） Thinking based on Terrain data Thinking based on Object data
38. Making Autonomous AI Game Designer (Human) Terrain data（Way Points） Thinking based on Terrain data Thinking based on Object data The process to make AI be intelligent Step 1： Give knowledge Step2： Make AI think by using knowledge Step3 : Let AI do Decision Making by itself
39. Knowledge Thinking Making Autonomous AI Game Designer (Human) Intelligence ＝ Knowledge × Thinking Terrain data （Way Points） Thinking based on Terrain data Thinking based on Object data Object data Body data Thinking based on Body data Representing knowledge Thinking on knowledge
40. Intelligence Knowledge Thinking Making Autonomous AI Game Designer (Human) By adding knowledge and thinking, AI grows up to be more intelligent. Terrain data （Way Points） Thinking based on Terrain data Thinking based on Object data Object data Body data Thinking based on Body data Representing knowledge Thinking on knowledge
41. Intelligence Knowledge Thinking Making Autonomous AI Game Designer (Human) Terrain data （Way Points） Thinking based on Terrain data Thinking based on Object data Object data Body data Thinking based on Body data Representing knowledge Thinking on knowledge Intelligence a character has is called “Character AI”.
42. References GAME AI PRO 3 (2017/6) http://www.gameaipro.com/
43. FFXV AI Book is published in Japanese in 2019 • I want to publish in English. • Now finding a publisher.
44. History of Game AI in Japan
45. Classical Meta AI = Dynamic difficulty Adaptive AI 岩谷徹氏： ファンと一般のユーザーを満足させる方法のひとつは人工知能AIのような考え方です。プ レーヤースキルをプログラム側から判断して、難易度を調整していくというものです。これを私はセ ルフゲームコントロールシステムと呼んで10年以上前から開発に使っています。 － International Game Designers Panel － http://game.watch.impress.co.jp/docs/20050312/gdc_int.htm Toru Iwatani (PACMAN Creator) said : A way to satisfy game freak and game fun is AI . By judging player’s skill by program, difficulty level is dynamically adaptive to the player. That is called “Level Control System”, which had been used for 10 years.
46. Published Pac-Man specification papers Japanese Society of Artificial Intelligence,No.1, 2019. Anyone can download the article: https://jsai.ixsq.nii.ac.jp/ej/index.php?active_action=repository_view_main_item_detail&page_id=13&block_id=23&item_id=9670&item_no=1 Toru Iwatani, Mirei Takahashi, Youichiro Miyake, “Toru Iwatani, the PAC-MAN Creater, Interview about the Origin of Game AI “PAC-MAN”,” , Japanese Society of Artificial Intelligence, No.1 Vol.34, No.1 pp.86-99., 2019.
47. ① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques
48. ① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques
49. AI Monsters’ Behavior Policy: PAC-MAN • Wave Attack … 4 monsters attack to a player with all of them at once. • Two mode: Surrounding mode and dispersion mode
50. Monster’s two mode: surrounding and dispersion Surrounding mode Dispersion mode
51. 7s 20s 7s 20s 5s 20s 5s 5s 20s 5s 20s 5s 27 34 54 59 7 25 30 50 55 5 79 84 7s 20s 7s 20s 5s 27 34 54 59 7 Monster dispersion Monster Attack A B C, D (sec) (sec) (sec) 3 types of Wave Attack time table
52. ① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques
53. Monster Wave Attack Surrounding Dispersion AKA Go to the Pac-man’s position （８ｘ８dot） Go to the upper right of play field and move around it . PINK Go to 3 cell forward position to Pac-man’s position Go to the upper left of play field and move around it . CYAN Go to the symmetry positon to AKA’s position around Pac-man’s position. Go to the lower right of play field and move around it . ORANGE Within 130 dots from Pac-man, it has the same character of AKA, and Out of the 130 dots, it moves randomly. Go to the lower left of play field and mover around it . Wave Attack two modes and monsters’ behavior
54. Surrounding around a PAC-MAN
55. ① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques
56. 2 ３ ４ monsters A B C Number of Monsters to be spawn 244 30 90 ３ ４ monsters 50 ４monsters (Count of eaten foods)
57. ① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques
58. SPEED PATTERN SPEED A Ｂ Ｃ Ｄ 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 ① イジケ ワープ ② ① イジケ ワープ ② ① イジケ ワープ ② ① イジケ ワープ ②
59. 「SEAMAN」（1999, Dream Cast） Main game designer：Yutaka Saito （At the time：VIVARIUM） Interview (2017)
60. https://jsai.ixsq.nii.ac.jp/ej/?action=repository_action_common_download&item_id=8633&item_no=1&attribute_id=22&file_no=1
61. Mr.Saito speakes about the techniques of natural language conversation of SEAMAN.
62. • Not exact speech recognition • When SEAMAN cannot understand a user’s speech, he ignores it, and begins to lead the conversation by himself. • SEAMAN has some different pronunciations for one speech. It makes difference of meaning by pronouciations. SEAMAN Game AI techniques
63. Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI
64. OVERVIEW Chapter 1
65. OVERVIEW 1.1
66. Intelligent Game System Game System Intelligent Game System Transition in 1995 when 3D game begins.
67. What is digital game ? • Interactive Digital Space • The space becomes structured • AI becomes a module in a whole system
68. Level Script Navigation AI Character AI メタAI 1995 2000 2005 2010 (3D Games) 1994 2005 ( 2010頃～ （Open World Game) Spatial AI 1980 PlayStation (1994) Xbox360 (2005) PlayStation3 (2006) Scripted AI 三宅陽一郎、水野勇太、里井大輝、 「メタAI」と「AI Director」の歴史的発展、日本デジタルゲーム学会（2020年、Vol.13, No.2） LS-Model Model LCN-AI Model MCS-AI Cooperative Model MCN-AI Model
69. レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
70. レベル Character AI Enemy character Player Data Collection Spatial AI Meta-AI Order Ask & Report Control the game Support query query Brain MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
71. AI in game, and AI outside of game Game AI （outside of game） Game AI （In game） Meta-AI Character AI Navigation AI Supporting AI QA-AI Auto-balancing AI Interface-AI Data Mining Simulation Game Visualization Biofeedback
72. Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI
73. レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
74. Meta AI History 1980 1990 1980 1990 2000 Classical Meta AI Modern Meta AI Development of Character AI (Autonomous AI ) Classical Meta AI to control difficulty of the game (Weaken enemies). Modern Meta AI to design game system dynamically.
75. (Example） Xevious（Namco、1983） Enemy Table Rewinding Enemy 0 Enemy １ Enemy 2 Enemt 3 Enemy 4 Enemy 5 https://www.youtube.com/watch?v=mbSHIFUUDLg
76. Monster’s two mode: surrounding and dispersion Surrounding mode Dispersion mode
77. AI Director (Meta AI) = Left 4 Dead Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html Example : Left 4 Dead
78. Modern Meta AI Meta AI dynamically changes a game Meta AI Allocation of enemies Spawning enemies Story Generation Procedural Terrain Generation User Experience
79. Left 4 Dead (Valve) AI Director controls NPC spawning distribution and game-play-pacing.  Genre：Online Action  Developer： Valve software  Publisher : Valve software  Hardware: XBox360  Year： 2006 Using nav-mesh for recognizing and controlling Real-time status of game. Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009 http://www.valvesoftware.com/publications.html
80. Left 4 Dead – AI Director Meta AI recognize game status in real-time Predicting player’s route and spawning monsters dynamically. Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009 http://www.valvesoftware.com/publications.html
81. AI Director decide when enemies are spawned. USER’S INTENSITY ACTUAL POPULATION DESIRED POPULATION Build Up …プレイヤーの緊張度が目標値を超えるまで 敵を出現させ続ける。 Sustain Peak … 緊張度のピークを3-5秒維持するために、 敵の数を維持する。 Peak Fade … 敵の数を最小限へ減少して行く。 Relax … プレイヤーたちが安全な領域へ行くまで、30-45秒間、 敵の出現を最小限に維持する。 AI Director makes users relax, and break relax repeatedly. Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009 http://www.valvesoftware.com/publications.html Meta AI(=AI Director) observes players’ action in real-time in Left 4 Dead
82. Adaptive Dramatic Dynamic Pacing [Basic Idea] (1) Meta AI populates many enemies, enough to get user’s intention up to the value (measured by input). (2) When user’s intention goes over the value, meta AI stops the population. (3) When user becomes relaxed, go to (1). Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
83. Meta AI recognize game status via Navigation-Mesh Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
84. Active Area Set Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html The area where meta-AI populate and extinguish agents is called AAS（= Active Area Set）
85. Active Area Set Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
86. Active Area Set Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
87. Flow Distance(predicting a path) Meta- AI predict a path - where they arrive - where is back - direction Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
88. Action to AAS Meta AI spawns and extinguishes enemies in Active Area Set. Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
89. Area seen from player In the Area where player can see, Meta-AI cannot spawn enemies Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
90. Area to spawn enemies FRONT BACK Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html Meta- AI spawns enemies in the area players cannot see.
91. Monster and Item appearance frequency High Low Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html Wandereｒs (High) Mobs(Middle) Special Infected (Middle) Bosses (Low) Weapon Caches (Low) Scavenge Items (Middle)
92. Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html (1) Decide number of enemies to populate by user’s escape route length. (2) Populate enemies along the route in a area around the player. (3) When a player goes out the area, the population stops, and enemies vanish. (4) In the state of relax or when the are can be seen from a player, all enemies are eliminated from the map. Algorithm of Meta AI
93. References (1) Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009. (2) Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html (3) 三宅 陽一郎, “メタAI”,「デジタルゲームの技術」 P.186-190, ソフトバンク クリエイティブ
94. Character AI Spatial AI Meta-AI ③ ordering to move and help a player ① find a point between a player and an enemy ⑤ Pathfinding Pathfinding to a goal point ④ Go to the point and fight with enemies ② return a point
95. Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI
96. META-AI X PROCEDURAL CONTENTS GENERATION Chapter 3
97. PCG Game AI AI PCG （PCG, Procedural Contents Generation ）
98. Rogue (1980) Rect[0] Rect[0] Rect[1] Rect[0] Rect[1] Rect[2] Rect[3] http://racanhack.sourceforge.jp/rhdoc/intromaze.html
99. Terrain Generation from Brown Motion ロバート・ブラウン博士によって、１８２７年に発見された現象。 微粒が媒質（液体）の中で行う不規則な運動。 アインシュタイン博士によって、熱運動する媒質の不規則な 衝突によって引き起こされると説明された。 http://ja.wikipedia.org/wiki/%E3%83%96%E3%83%A9%E3%82%A6%E3%83%B3%E9%81%8B%E5%8B%95 Kazunori Miyata 「Procedural Contents Geneation」（CEDEC 2008）
100. Terrain Generation in Fractal (1987) http://ja.wikipedia.org/wiki/%E3%83%96%E3%83%A9%E3%82%A6%E3%83%B3%E9%81%8B%E5%8B%95 http://www.kenmusgrave.com
101. Terrain Generation in Fractal(1987) http://ja.wikipedia.org/wiki/%E3%83%96%E3%83%A9%E3%82%A6%E3%83%B3%E9%81%8B%E5%8B%95 http://www.kenmusgrave.com
102. NO MAN’S SKY (Hello Games, 2016) http://www.no-mans-sky.com/ 宇宙、星系、太陽系、惑星を自動生成する。
103. FarCry2 (Dunia Engine ) デモ 草原自動生成 時間システム 樹木自動生成 動的天候システム 動的天候システム http://www.farcry2-hq.com/downloads,18,dunia-engine-nr1.htm
104. Far Cry 2 Dunia Engine - Growing Vegetation (Far Cry HQ) https://www.youtube.com/watch?v=FI3oR6vqn1Q
105. Terrain Generation by Voronoi and Noise Jacob Olsen, Realtime Procedural Terrain Generation http://oddlabs.com/download/terrain_generation.pdf ２次元中点変位法 ボロノイ図 ＋ ＝
106. Ken Musgrave http://www.kenmusgrave.com/
107. Terragen(Planetside Software) Landscape Geneation http://www.planetside.co.uk/terragen/
108. 3D Terrain Genetation in Age of Empires III 西川善司, 「3DゲームファンのためのAGE OF EMPIRESエンジン講座(後編)こだわりの影生成と算術合成 されるディテール、次回作はXbox2?」, GAME Watch, 2005
109. L-system による街の自動生成 City Engine(central pictures) Yoav I H Parish, Pascal Müller http://www.centralpictures.com/ce/tp/paper.pdf http://www.centralpictures.com/ce/ George Kelly, Hugh McCabe, A Survey of Procedural Techniques for City Generation http://www.gamesitb.com/SurveyProcedural.pdf
110. https://www.youtube.com/watch?v=NfizT369g60
111. https://www.youtube.com/watch?v=FzoY062kY1s
112. Terraon Generation from Americas （Eidos） https://www.eidosmontreal.com/news/worldgen-painting-the-world-one-layer-at-a-time/
113. VITRUVIO CityEngine Plugin for Unreal Engine https://esri.github.io/cityengine/vitruvio
114. SimCity 4 Engine, Glassbox https://www.youtube.com/watch?v=tKOgo7EFl_w
115. Procedural Generation in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
116. Black Combination in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
117. Procedural Generation Map in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
118. Auto-analyzing the map in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
119. Start, Exit, objectives auto-distribution in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
120. Influence Map The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
121. Influence Map The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
122. Active Are Set The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
123. Meta AI （AI Director, Dynamic Adaptive Pacing) The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
124. メタＡＩ（自動適応ペーシング） Meta AI （AI Director, Dynamic Adaptive Pacing) The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
125. Summary Classical Game System Modern Game System In classical game system, game contents are fixed after the development, but in modern game system, game contents are dynamically created.
126. Summary Classical Game System Modern Game System Meta AI dynamically creates a game play flow, and gives an order to character AI and a game event. The reason why meta AI action is simple is that character AI has become autonomous.
127. Referenced Papers (1) Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009. (2) Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
128. Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847
129. Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847
130. Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847
131. CoG 2021: Adversarial Reinforcement Learning for Procedural Content Generation SEED – Electronic Arts https://www.youtube.com/watch?v=kNj0qcc6Fpg
132. Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847
133. Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847
134. Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847
135. NUVERS uses GPT3 in Animation Selection ［GDC 2023］GPT-3でNPCの会話を生成。Nuverseが「アース：リバイバル」でチャレンジする，新テクノロ ジーによるゲーム開発の“新たな一歩” https://www.4gamer.net/games/651/G065105/20230325007/
136. Earth revival (Nuverse) • Finding the animation of character by analyzing the lines 800 within 800ns. • GPT3 • Sound Generation Nuverse lets AI do all the talking and moving of NPCs in Earth: Revival https://premortem.games/2023/03/22/nuverse-lets-ai-do-all- the-talking-and-moving-of-npcs-in-earth-revival/ ジェネレーティブAIでゲーム開発支援！ GDCで発表されたリア ルな事例や取り組みを紹介 https://www.famitsu.com/news/202303/24297009.html
137. ChatGPT and Stable Diffusion usage RPG: Tales of Syn • How AI-assisted RPG Tales of Syn utilizes Stable Diffusion and ChatGPT to create assets and dialogues • https://gameworldobserver.com/2023/03/06/tales-of-syn-ai- rpg-stable-diffusion-chatgpt-game • Tales of Syn • https://talesofsyn.com/ • https://twitter.com/_hackmans_/status/1624501399383072768
138. ChatGPT と Stable Diffusion を用いた RPG: Tales of Syn • https://twitter.com/_hackmans_/status/1624501399383072768 • https://talesofsyn.com/
139. Tales of Syn Game - NPC Chat GPT3 Prototype (Tales of Syn) https://www.youtube.com/watch?v=ejw6OI4_lJw
140. Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI
141. CHARACTER AI Chapter 4
142. レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
143. FC SFC SS, PS PS2,GC,Xbox Xbox360, PS3, Wii DC （次世代） Hardware 時間軸 2005 1999 Character AI Complex AI AI and game world has become more complex Simple AI （スペースインベーダー、タイトー、1978年） （アサシンクリード、ゲームロフト、2007年）
144. (Ex) Space Invader (1978) プレイヤーの動きに関係なく、決められた動きをする （スペースインベーダー、タイトー、1978年）
145. （Ex） Prince of Persia （プリンスオブペルシャ、1989年）
146. Halo (Bungie) http://halo.bungie.net/inside/publications.aspx#pub15068
147. Environment Intelligence Artificial Intelligence＝ dynamically makes an AI’s action in harmony with artificial environment. What is intelligence？ Body （Inner Structure） Input（Sensor） Output（Action）
148. Intelligence World Environment World Effector・Body Sensor・ Body Agent Architecture Agent Architecture
149. Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module Agent Architecture
150. Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module Objective・ Phenomenon Agent Architecture
151. Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module Agent Architecture
152. World Sensor 知識 生成 Knowledg e Making 意思決定 Decision Making 運動 生成 Motion Making Information Flow 記憶 Principle of Learning Action Result Decision Making Body
153. World Sensor 知識 生成 Knowledg e Making 意思決定 Decision Making 運動 生成 Motion Making Information Flow 記憶 Principle of Learning Action Result Decision Making Body
154. Deep Learning in Ubisoft https://gdcvault.com/play/1025653/The-Alchemy-and-Science-of The Alchemy and Science of Machine Learning for Games Yves Jacquier （Ubisoft Montreal）
155. Deep Learning in Ubisoft https://gdcvault.com/play/1025653/The-Alchemy-and-Science-of The Alchemy and Science of Machine Learning for Games Yves Jacquier （Ubisoft Montreal）
156. https://gdcvault.com/play/1025653/The-Alchemy-and-Science-of 35:00- Deep Learning in Ubisoft The Alchemy and Science of Machine Learning for Games Yves Jacquier （Ubisoft Montreal）
157. Belief – Desire – Intention モデル Desire (Perceptrons) Opinions (Decision Trees) Beliefs (Attribute List) Richard Evans, “Varieties of Learning”, 11.2, AI Programming Wisdom Low Energy Source =0.2 Weight =0.8 Value = Source*Weight = 0.16 Tasty Food Source =0.4 Weight =0.2 Value = Source*Weight = 0.08 Unhappines s Source =0.7 Weight =0.2 Value = Source*Weight = 0.14 ∑ 0.16+0.08+0.14 Threshold （0～1の値に 変換） hunger Perceptron in Black & White （Lionhead,2000）
158. Black & White （Lionhead,2000） Training of creature by learning http://www.youtube.com/watch?v=2t9ULyYGN-s http://www.lionhead.com/games/black-white/ 14:20-
159. Reinforcement Learning (Ex) Fighting Game Kick Punc h Magic R_0 : Reward = Damage http://piposozai.blog76.fc2.com/ http://dear-croa.d.dooo.jp/download/illust.html
160. LEARNING TO FIGHT T. Graepel, R. Herbrich, Julian Gold Published 2004 Computer Science https://www.microsoft.com/en-us/research/wp-content/uploads/2004/01/graehergol04.pdf
161. 3 ft Q-Table THROW KICK STAND 1ft / GROUND 2ft / GROUND 3ft / GROUND 4ft / GROUND 5ft / GROUND 6ft / GROUND 1ft / KNOCKED 2ft / KNOCKED 3ft / KNOCKED 4ft / KNOCKED 5ft / KNOCKED 6ft / KNOCKED actions game states 13.2 10.2 -1.3 3.2 6.0 4.0 +10.0 Ralf Herbrich, Thore Graepel, Joaquin Quiñonero Candela Applied Games Group,Microsoft Research Cambridge "Forza, Halo, Xbox Live The Magic of Research in Microsoft Products" http://research.microsoft.com/en-us/projects/drivatar/ukstudentday.pptx
162. Early in the learning process … … after 15 minutes of learning Reward for decrease in Wulong Goth’s health Ralf Herbrich, Thore Graepel, Joaquin Quiñonero Candela Applied Games Group,Microsoft Research Cambridge "Forza, Halo, Xbox Live The Magic of Research in Microsoft Products" http://research.microsoft.com/en-us/projects/drivatar/ukstudentday.pptx
163. Early in the learning process … … after 15 minutes of learning Punishment for decrease in either player’s health Ralf Herbrich, Thore Graepel, Joaquin Quiñonero Candela Applied Games Group,Microsoft Research Cambridge "Forza, Halo, Xbox Live The Magic of Research in Microsoft Products" http://research.microsoft.com/en-us/projects/drivatar/ukstudentday.pptx
164. GENETIC ALGORITHM
165. 「Astronoka」（ENIX, 1998） https://www.jp.square-enix.com/game/detail/astronoka/ Evolving by Genetic Algorithm
166. 1998「Astronoka」 (PlayStation) Generating parameters of character specification by genetic algorithm. 三宅 陽一郎「ゲームAI技術20年の進化とこれから」 https://cedil.cesa.or.jp/cedil_sessions/view/1943 森川幸人「やってきたこと」よ Trap Jump Break Round about Creatures evolves to adapt the environment
168. Genetic Algorithm Make a group evolve in one direction First generation New generation（１００～） … Evolution by generation One generation produces a next generation
169. Genetic Algorithm 遺伝子 Gene Next Generation Parent ① Parent② Pick up two parents from base group Crossover two genes Produce new generation (selection) (crossover) (production) Present Generation The iteration makes a desired generation in the environment 遺伝子 Gene
170. Flow chart of GA A player makes traps Beginning of a day Trap battle begins Trap battle ends Evaluation Fitness value Order Delete the last two NPC Pick up two parents from NPCs with high fitness value Generate two new NPCs Generate new generation Get enough number of new NPC The day ends Generation exchange rate changes Mutation rate changes
171. ４－② Simulation and evaluation Evaluation score is proportional to how long NPC go into the field avoiding and breaking traps. G Fitness＝Score＋Time*0.3＋Enjoy param*0.5＋Trap_Score＋Safety＋HP*0.5 Time for clear G C B A B C C B C C S G A B C Start Bonus =100 Goal Bonus =50 Bonus =30 Bonus =10 Bonus =0
172. GA improvement in Game system Average of fitness A curve of fitness a day Generations (1) To give a feeling of evolution to users, the game system evolves 5 generations a day by GA. (2) To make constant speed of evolution of one day, the game system controls the number of iterations of GA evolution.
173. Neural Networks in Supreme Commander 2 (GDC 2012) Michael Robbins (Gas Powered Games) http://www.gdcvault.com/play/1015406/Off-the-Beaten-Path-Non http://www.gdcvault.com/play/1015667/Off-the-Beaten-Path-Non
174. Neural Networks in Supreme Commander 2 (GDC 2012) Michael Robbins (Gas Powered Games) http://www.gdcvault.com/play/1015406/Off-the-Beaten-Path-Non http://www.gdcvault.com/play/1015667/Off-the-Beaten-Path-Non Neural Network Application
175. Neural Networks in Supreme Commander 2 (GDC 2012) Michael Robbins (Gas Powered Games) http://www.gdcvault.com/play/1015406/Off-the-Beaten-Path-Non http://www.gdcvault.com/play/1015667/Off-the-Beaten-Path-Non
176. City Conquest Paul Tozor Postmortem: Intelligence Engine Design Systems' City Conquest https://www.gamedeveloper.com/business/postmortem-intelligence-engine-design-systems-i-city-conquest-i-
177. Build = 1,11,46,Mine,8 Build = 1,11,48,Mine,16 Build = 1,11,47,Skyc,24 Build = 1,14,42,Turr,28 Build = 1,14,45,GrSl,100808 Build = 1,14,40,Turr,171184 Build = 1,9,47,Mine, 52 Build = 1,9,49,Mine, 60 Build = 1,12,44,RktL,100836 Build = 1,10,49,Drop,100812 Build = 1,13,45,RktL,100816 Build = 1,17,44,RktL,100820 Build = 1,10,45,Skyc,100824 Upgrade = 0,13,45,RktL,100820 Build = 0,11,46,Mine,8 Build = 0,11,48,Mine,16 Build = 0,11,47,Skyc,24 Build = 0,14,42,Turr,28 Build = 0,14,40,Turr,171184 Build = 0,9,47,Mine 52 Build = 0,9,47,Mine 60 Build = 0,15,45,GrSl,100808 Build = 0,10.,49,Drop,100812 Build = 0,13,45,RktL,100816 Build = 0,17,44,RktL,100820 Build = 0,10,45,Skyc,100824 Upgrade = 0,13,45,RktL,100816 Build = 0,17,49,Drop,100832 Paul Tozor Postmortem: Intelligence Engine Design Systems' City Conquest https://www.gamedeveloper.com/business/postmortem-intelligence-engine-design-systems-i-city-conquest-i-
178. TT L_1 TT L2 TT L3 TT L4 TT L1 TT L2 TT L3 TT L4 Parent１ Parent ２ TT L_1 TT L2 TT L3 TT L4 TT L_1 TT L2 TT L3 TT L4 Child １ Child ２
179. Typical Agents in Digital Games (1)Morikawa-kun (SCE、1997年） (2)Astronoka (ENIX、1998年） (3)The Sims（EA, Maxis, 2000）
180. 1997「GANBARE MORIKAWAKUN 2GO」 (PLAYSTATION) Character Control by Neural Network 三宅 陽一郎「ゲームAI技術20年の進化とこれから」 https://cedil.cesa.or.jp/cedil_sessions/view/1943 森川幸人「やってきたこと」より引用 Action Player orders an action Player orders an action Reaction from an item Decision Making Learning from the result AI！
181. Impression Impression Impression Impression https://www.youtube.com/watch?v=CFnCd38yvD8 Article https://morikatron.ai/2022/03/gameai_history_04/
182. Color１ Color 2 Form Smell Sound Behavior … Take an action Knock Kick Take back Eat Jump Push おいしい 快い 不快 Impression from item Nourishing Enjoyable Damaged Fear Fine … Table Data … … … … Texture Neural Network with Backpropagation in game ゲーム中にBP学習される ニューラルネットワーク② Put it on the head
183. Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI
184. CHARACTER BEHAVIOR LEARNING Chapter 3-2
185. Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module
186. Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI
187. DECISION MAKING ALGORITHM Chapter 3-3
188. Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module
189. Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive
190. Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive
191. Rule Control ID 0 : IF …. THEN … ID 1 : IF …. THEN … ID 3 : IF …. THEN …. ID 4 : IF …. THEN … ID .. : ID 4 : IF …. THEN … Priority: 3 Priority: 1 Priority: 4 Priority: 5 Priority: 2 A constant priority value or dynamic priority is attached to each rule. The rule with highest priority is fired in all fired rules. The system is used by tactical thinking and character behavior. Priority: - Control Rule ＝ IF （ Action Condition） then （Action Command）
192. Rule Control ID 0 : IF …. THEN … ID 1 : IF …. THEN … ID 3 : IF …. THEN …. ID 4 : IF …. THEN … ID .. : ID 5 : IF …. THEN … Priority: 3 Priority: 1 Priority: 4 Priority: 5 Priority: 2 Thinking module picked up one rule from the game situation. Priority: - Control Thinking Module Rule ＝ IF （ Action Condition） then （Action Command）
193. Example http://gomibako.symphonic-net.com/vote.cgi?genre=act_b&rating=9 ID 0 : IF （an enemy found） THEN （escape） ID 1 : IF （cannot find any） THEN （random walk） ID 2 : IF （found item） THEN （get the item） ID 3 : IF （found door） THEN （go through it） ID 0 : IF （enemy is strong） THEN （Magic） ID 1 : IF （found many enemies） THEN （attack the weakest） ID 2 : IF （HP<20） THEN （spell refresh magic ） ID 3 : IF （found a hole） THEN （avoid it） Action game RPG
194. Decision Making Rule-based AI  Genre：Simulation  Developer： Maxis  Publisher : EA  Hardware: PC, PS3  Year： 2009 [GDC2010] Richard Evans, "Modeling Individual Personalities in The Sims 3" http://www.gdcvault.com
195. Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive
196. F.E.A.R Agent Architecture  Genre：Horror FPS  Developer： Monolith Production  Publisher : SIERRA  Hardware: Windows  Year： 2004 Agent Architecture Considerations for Real-Time Planning in Games (AIIDE 2005) http://web.media.mit.edu/~jorkin/AIIDE05_Orkin_Planning.ppt Sensors Working Memory Planner Blackboard Navigation Animation / Movement Targeting Weapons World World
197. Planning by Chaining Example kSymbol_ TargetIsDead kSymbol_ TargetIsDead Attack kSymbol_ WeaponLoaded kSymbol_ WeaponLoaded Reload kSymbol_ WeaponArmed kSymbol_ WeaponArmed Pick up a weapon None Planner Planning Action Pools
198. Replaning AI in F.E.A.R. Jeff Orkins, Three States and a Plan: The AI of FEAR http://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear.pdf
199. Hierarchical Goal-oriented decision making in Chrome Hounds Iteration: Decomposing a goal to smaller goals Goal Goal Goal Goal Goal Goal Goal Goal Goal 三宅陽一郎 「クロムハウンズにおける人工知能開発から見るゲームAIの展望」 https://cedil.cesa.or.jp/cedil_sessions/view/50
200. Chrome Hounds walk along the pat Search the position of base Find the path Go to a Base Attack the base Conquer a base Shoot Walk Stop Attack until the base is fallen Tactics Operator (Behavior) ハウンズ Strategy Hierarchical Goal-oriented decision making in Chrome Hounds 三宅陽一郎 「クロムハウンズにおける人工知能開発から見るゲームAIの展望」 https://cedil.cesa.or.jp/cedil_sessions/view/50
201. Conquer_Combas_TeamAI.avi https://www.nicovideo.jp/watch/so14536895 https://www.nicovideo.jp/watch/so14536912? d=09f8b28b-b460- 4c8d-ad6b-d5d1f2917925 Hierarchical Goal-oriented decision making in Chrome Hounds Hierarchical Goal-oriented decision making in Chrome Hounds https://www.nicovideo.jp/watch/so14536895 https://www.nicovideo.jp/watch/so14536912 https://www.nicovideo.jp/watch/so14536698 三宅陽一郎 「クロムハウンズにおける人工知能開発から見るゲームAIの展望」 https://cedil.cesa.or.jp/cedil_sessions/view/50
202. Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Decision Making is generally a very complex high-degree process. But for digital game there are 7 simple decision making algorithms. Simulation-based AI Utility-based AI 「(something)-based AI」means that an algorithm uses (something) as a unit. Task-based AI Decision Making
203. State Machine State State State State Machine（Finite State Machine） AI’s instruction is described in a state, and changes in the world and AI are described in a transition condition. A state machine has a loop structure but does not have feedback dynamics.
204. Hierarchical State Machine Two enemy characters are patrolling. One player comes to the room. AI character is described as Hierarchical Finite State Machine (HFSM).
205. (Example) Quake HFSM http://ai-depot.com/FiniteStateMachines/FSM-Practical.html Monster’s FSM Spawn Idle Attack Die Melee Melee Smash Left Right Finish spawning Lost goal Located goal Zero health Zero health
206. Quake (id Software) https://www.youtube.com/watch?v=LpEygM6ZkfI&t=350s
207. Watch Chase Chase Attack Attack Warning Command Ally Watch the exit Call an Ally Ally not found & Out of battle field Join with Ally Join with Ally Found Ally Lost enemy Found enemy Lost enemy Found enemy Patrol 10 sec. passed Hear a sound Ally Responds Transition Condition State Higher State Hierarchical State Machine
208. Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Decision Making is generally a very complex high-degree process. But for digital game there are 7 simple decision making algorithms. Simulation-based AI Utility-based AI 「(something)-based AI」means that an algorithm uses (something) as a unit. Task-based AI Decision Making
209. Behavior-based AI Behavior A behavior represent not an animation detail but a physical action.
210. Behavior-based AI Behavior Behavior Decision-Making Behavior A behavior-based AI constructs its thinking by using some behaviors.
211. (1) Hierarchical Structure of Box (2) Each box has some behaviors (3) Each box has one selection-rule (random, sequential, priority-order…) (4) Finally, one behavior is selected at end-of-tree. (5) Decision Making Behavior-based AI  Genre：SciFi-FPS  Developer： BUNGIE Studio  Publisher : Microsoft  Hardware: Xbox, Windows, Mac  Year： 2004 Behavior-Tree (has become very popular method .) Damian Isla (2005), Handling Complexity in the Halo 2 AI, GDC Proceedings., http://www.gamasutra.com/gdc2005/features/20050311/isla_01.shtml
212. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
213. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
214. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
215. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule From the possible nodes in this layer, make the highest-priority node active.
216. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
217. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule Execute possible nodes from this layer in order.
218. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
219. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
220. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
221. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
222. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule Set one possible node from this layer to be executed randomly.
223. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
224. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
225. root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule
226. Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Decision Making is generally a very complex high-degree process. But for digital game there are 7 simple decision making algorithms. Simulation-based AI Utility-based AI 「(something)-based AI」means that an algorithm uses (something) as a unit. Task-based AI Decision Making
227. State machine compared to Behavior tree • State machine＝ Steady control step by step • Behavior tree ＝ Adapt behavior fluently We want to use both good points
228. AI Graph Hierarchical Layered Architecture State machine Behavior tree A node on graph structure can reference to graph structure. IDLE MOVE FIGHT MOVE ATTACK State machine Different types of graph system can be combined.
229. Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive
230. Task based AI A big task is formed by combining some small tasks. A task must represent a simple practical action. (Simple case) Initial State GOAL Task１ Carry A to Ground Task2 Carry B to Ground Task3 Put C on B A B C B C A A B C A B C A C B Task3 Put A on C
231. Method （＝Resolving a big task into some small tasks） Make a drink （Task） Deliver a drink （Task） Take a drink to a buddies （Task）
232. Method （＝Resolving a big task into some small tasks） mix branch Collect all colors Branch Collect red and blue branch Collect green and red Pickup Blue Red Green mix Green Green mix Red Blue Red Blue No order task Total order task Partial Order Task There are all color jewels in a field There are green and red jewels in a field There are red and blue jewels in a field Primitive task Make a drink （Task） Precondition
233. Method （＝Resolving a big task into some small tasks） Make a drink （Task） Deliver a drink （Task） Take a drink to a buddies （Task）
234. Method Method Put that in car Bring a car Go to a battle field Put a drink In the car Attach a label Put it in Treasure box Total Order Task Has a car Primitive task Deliver a drink （Task） Precondition Put a drink In the car A car is near Partial Order Task
235. By iteration of applying methods , Task Network is generated. Mix Blue Red Blue START Bring a Car Go to a battl e field GOAL Put it in the car Attach a label Put it in Treasure box
236. Killzone 2’s AI Alex Champandard, Tim Verweij, Remco Straatman, "Killzone 2 Multiplayer Bots", http://files.aigamedev.com/coverage/GAIC09_Killzone2Bots_StraatmanChampandard.pdf
237. Organization in KILLZONE 2’s AI team Commander AI Squad Leader AI Squad Leader AI Squad Leader AI Order a tactic “Defend”,”Progress” Report of the tactics (Success/Failure) (Feedback) Order a target positon Order a target Request a re-order Request a re-order Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI
238. Agent Architecture of a member (Individual AI) Orders Messages Daemon World State HTN Planner Plan Task Execution Controller Input Intelligence Stimuli Percepti on Threats
239. Agent Architecture of a Squad leader Strategy Orders Member Messages daemon World State HTN Planner Plan Task Execution Individual Orders Intelligence
240. Bullet Rocket fire Rocket To Humanoid Within range Not to Humanoid or turret Within a turret’s range Attack（Task） load fire end load fire end
241. … Progress None Defend a base （Task） Delete member’s memory Gather new information of buddies Begin a sequence Progress Go and Stop Defend
242. Behave （Task） Crouching Select Plan Medic start Broadcast Care End Car Defend Target fix Use a tool Walk care Move Stop Continue Begin Medic Not chage Change a tool Apply a method Apply a method Apply a method Apply a method
243. https://www.youtube.com/watch?v=GgtIZNJOuMA
244. Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive
245. The Sims（EA, Maxis, 2000） • The game is to observe a society of agents Ken Forbus, “Simulation and Modeling: Under the hood of The Sims” (NorthWerstern Univ) http://www.cs.northwestern.edu/%7Eforbus/c95-gd/lectures/The_Sims_Under_the_Hood_files/frame.htm Richard Evans, Modeling Individual Personalities in The Sims 3, GDC 2010 http://www.gdcvault.com/play/1012450/Modeling-Individual-Personalities-in-The
246. The Sims 「Motif Engine」 • AI Character Engine Ken Forbus, “Simulation and Modeling: Under the hood of The Sims” (NorthWerstern University) http://www.cs.northwestern.edu/%7Eforbus/c95-gd/lectures/The_Sims_Under_the_Hood_files/frame.htm Data - Needs - Personality - Skills - Relationships Sloppy - Neat Shy - Outgoing Serious - Playful Lazy - Active Mean - Nice Physical - Hunger - Comfort - Hygiene - Bladder Mental - Energy - Fun - Social - Room Motive Engine Cooking Mechanical Logic Body Etc.
247. Utility-based Agent • Principle = Maximize a modd Hunger +20 Comfort -12 Hygiene -30 Bladder -75 Energy +80 Fun +40 Social +10 Room - 60 Mood +18 - Urinate (+40 Bladder) - Clean (+30 Room) - Unclog (+40 Room) Mood +26 - Take Bath (+40 Hygiene) (+30 Comfort) - Clean (+20 Room) Mood +20 Bathtub Toilet Ken Forbus, “Simulation and Modeling: Under the hood of The Sims” (NorthWerstern大学、講義資料) http://www.cs.northwestern.edu/%7Eforbus/c95-gd/lectures/The_Sims_Under_the_Hood_files/frame.htm
248. Mood calculation Mood = W_Hunger(X_Hunger) * X_Hunger + W_Engergy(X_Energy) * X_Energy + … -100 0 100 -100 0 100 -100 0 100 -100 0 100 -100 0 100 -100 0 100 W_Hunger W_Energy W_Comfort W_Fun W_Hygiene W_Social W_Bladder W_Room -100 0 100 -100 0 100 Ken Forbus, “Simulation and Modeling: Under the hood of The Sims” (NorthWerstern大学、講義資料) http://www.cs.northwestern.edu/%7Eforbus/c95-gd/lectures/The_Sims_Under_the_Hood_files/frame.htm
249. Utility W_Hunger X_Hunger W_Hunger(-80) -80 60 W_Hunger(60)
250. Utility comparison and The law of diminishing marginal utility Hunger degree at -80 ＝ W_Hunger(-80)*(-80) Hunger degree at 60 ＝ W_Hunger(60)*(60) Δ ＝ W_Hunger(60)*(60) - W_Hunger(-80)*(-80) Utility for hunger W_Hunger X_Hunger W_Hunger(-80) -80 60 W_Hunger(60) 90 W_Hunger(90)
251. The law of diminishing marginal utility Δ（-80 → 60）＝W_Hunger(60)*(60) - W_Hunger(-80)*(-80) Δ (60→90） ＝W_Hunger(90)*(90) - W_Hunger(60)*(60) Δ（-80 → 60） is much larger than Δ(60→90） Utility for hunger W_Hunger X_Hunger W_Hunger(-80) -80 60 W_Hunger(60) 90 W_Hunger(90)
252. Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI
253. SPATIAL AI Chapter 4
254. PATH-FINDING Chapter 4-2
255. レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
256. Navigation Data Waypoint・Graph （Networked Graph with points） Navigation Mesh（Networked Graph with polygons） Walk-able フリー素材屋Hoshino http://www.s-hoshino.com/
257. Pathfinding Dragon Age pathfinding program put to the test （University of Alberta） https://www.youtube.com/watch?v=l7YQ5_Nbifo Daivuk RTS - Pathfinding A* https://www.youtube.com/watch?v=95aHGzzNCY8
258. （Mesh） Cost : 0.5 Visibility: 1.0 Surface： earth （Mesh） Cost: 0.8 Visibility: 0.7 Surface： pond （Object） Movable : to (1.0,0.8) Able to lift up: false Able to ride on： false Hardness： 0.9 Weight： 0.4 （Object） Action： knock down the lever Effect： Open the door （Object） Door Link between two nodes Unified Format of Location based Information = Common Ground
259. Player’s predicted route（called Golden Path） M. Jack , M. Vehkala, “Spaces in the Sandbox: Tactical Awareness in Open World Games,” GDC 2013, https://www.gdcvault.com/play/1018136/Spaces-in-the-Sandbox-Tactical
260. ゴールデンパスに沿った位置検索技術 Goal (guessed By Meta-AI) Player Golden Path Meta-AI finds the best point to put an enemy along the golden path.
261. Character AI Spatial AI Meta-AI ③ ordering to move and talk to a player ④ Path finding Pathfinding to a point near by player ⑤ run and face to a player ① Searching a point to talk to a player ② return the point around a player
262. TACTICAL POINT SYSTEM Chapter 4-3
263. History of Tactical Point System • Tactical Position Picking Killzone (2005, Guerrilla) in Program • TPS (Tactical Point System) CRYENGINE (2010, CRYTEK) Tool & Runtime System • EQS(Environment Query System) UNREAL ENGINE 4 （2014, Epic games）Tool & Runtime System • PQS (Point Query System) FINAL FANTASY XV (2016, SQUARE ENIX)
264. Point Query System A system to find a best positon - for a character’s ability - in a terrain - in real-time
265. Point Query System principle Point distribution （Generation） distributing point around the objective （例）grid、circle Filtering Removing points not adjust for a purpose by a conditon (1) ． ． ． Evaluation Evaluation for remained points and pick up one points with best score Filtering Removing points not adjust for a purpose by a conditon (2) Filtering Removing points not adjust for a purpose by a conditon (N)
266. PQS (Point Query System) A bowman finds the best point (i) Game situation (ii) Generating points around it (iii) Filtering points with bad terrain (iv) Filtering points where it’s arrow can not reach (v) Filtering points around buddies (vi) Picking up one point with highest terrain
267. Enemies Buddies Player Character Rock Mountation Rock 海 Hole
276. Terrain Analysis by TPS (Tactical Point System) M. Jack , M. Vehkala, “Spaces in the Sandbox: Tactical Awareness in Open World Games,” GDC 2013, https://www.gdcvault.com/play/1018136/Spaces-in-the-Sandbox-Tactical
277. M. Jack , M. Vehkala, “Spaces in the Sandbox: Tactical Awareness in Open World Games,” GDC 2013, https://www.gdcvault.com/play/1018136/Spaces-in-the-Sandbox-Tactical Terrain Analysis by TPS (Tactical Point System)
278. Character AI Spatial AI Meta-AI ⑤ Running the point in the front of a player ② Pathfinding to a player’s goal point ③ Searching the best point in the front of a player by PQS ⑥⑦Path-finding Finding a apath to a destination point ⑧running in the front of a player and having a conversation ④ the point in the front of a player ①Guessing player’s goal point
279. PQS finds the point in front of Player Character
280. PQS finds the point in front of Player Character
281. PQS finds the point in front of Player Character
282. PQS finds the point in front of Player Character
283. CASE 4: BUDDY NPC running in the front of a player • Objective: Making a buddy NPC running in the front of a player character (In many games, NPC only follows a player character) • Problem: Buddy NPCs must finds the position in the front of a play character intermittently • Solution: MCS-AI cooperative model finds the best positon to run in the front of a player character by using position query system. (1) Meta-AI Guesses a player’s destination point (2) Meta-AI asks Spatial AI to returns a path to the player’s destination point. (3) Meta-AI asks Spatial AI to get a point in the front of a player character. (4) Spatial AI returns the best point in the front of a player character by using PQS to Meta-AI. (5) Meta AI askes the Character AI to go to the point. (6) The Character AI asks Spatial AI to return a path to the point. (7) Spatial AI returns the path to the point. (8) The Character AI runs to the point in the front of a player character and has conversation with it. • Result: It seems that a buddy NPC guesses the player destination and runs in the front of a player character
284. AMBIENT AI Chapter 9 (NON-PLAYER CHARACTER IN THE TOWN) Not Just Planning: STRIPs for Ambient NPC Interactions in Final Fantasy XV by Hendrik Skubch, Square Enix (nucl.ai Conference 2015) https://archives.nucl.ai/recording/not-just-planning-strips-for-ambient-npc-interactions-in-final-fantasy-xv/
285. Smart Environmental Actors Smart Object Agent Smart Point Smart Space Agent Agent Control Control Control ｄoor Handle Space Object Point
286. DATA LOGGING AND VISUALIZATION Chapter 10
287. 物理的都市空間 スマートスペース メタバース
288. Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI
289. 年 企業 テーマ Open 2003 Microsoft 「Teo Feng」における強化学習 2005 Microsoft 「Forzamotor Sports」における強化学習 2013 DeepMind A tariのゲームをDQNで学習 2015 DeepMind 囲碁 AlphaGO 2017 AnyLogic 倉庫・機械などモデルのシミュレーション Microsoft 「パックマン」多報酬学習 Hybrid Reward Architecture for Reinforcement Learning 2019 Google 「サッカーシミュレーター」による強化学習の研究 〇 DeepMind 「Capture the flag」によるディープラーニング学習 Microsoft 「AirSim」ドローンシミュレーター 〇 Nvidia 「ドライビングシミュレーター」 Mircrosoft 「TextWorld」アドベンチャーゲームを題材に言語学習 〇 facebook 「CraftAssist」マインクラフト内で会話研究 〇 CarMelon カーネギーメロン大学「MineRL」マインクラフトを使ったAIコンテスト 〇 facebook 「LIGHT」ファンタジーワールドを構築してクラウドワーカーで会話研究 〇 OpenAI 「Dota2」OpenAIによる「OpenAIFive」 〇 DeepMind 「StarCraft2」AlphaStar 〇 DeepMind 「Capture the Flag」QuakeIII エンジン 〇 2020 Nvidia GameGAN「ディープラーニングによるパックマンの目コピー」 〇 DeepMind 「Agent57」AtariのほとんどのゲームをDQN+LSTMなどで学習 〇 OpenAI 「HIDE AND SEEK」かくれんぼを用いたマルチエージェントのカリキュラム学習 〇
290. Dota2 eSports OpenAI Five: Dota Gameplay https://www.youtube.com/watch?v=UZHTNBMAfAA 解説：『Dota 2』における人間側のチャンピオンチームとAIチームの戦い https://alienwarezone.jp/post/2316
291. [参考論文] 「OpenAI」については以下を参考にしました。 OpenAI Five https://openai.com/projects/five/ Christopher Berner, et al.,“Dota 2 with Large Scale Deep Reinforcement Learning” https://arxiv.org/abs/1912.06680
292. StarCraft~StarCraft2 （DeepMind, 2019）
293. StarCraftのAI • Santiago Ontañon, Gabriel Synnaeve, Alberto Uriarte, Florian Richoux, David Churchill, et al.. • “A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft”. IEEE Transactions on Computational Intelligence and AI in games, IEEE Computational Intelligence Society, 2013, 5(4), pp.1-19. hal- 00871001 • https://hal.archives-ouvertes.fr/hal-00871001
294. StarCraftの7つのアーキテクチャ (2010-2020)
295. StarCraft~StarCraft2における 人工知能 （DeepMind, 2019）
296. [参考文献] SC2LEについては以下の文献を参考にしました。 また図 7.8は以下の文献から引用しました。 Oriol Vinyals, et al., “StarCraft II: A New Challenge for Reinforcement Learning”, https://arxiv.org/abs/1708.04782 PySC2 - StarCraft II Learning Environment https://github.com/deepmind/pysc2
297. StarCraft II API StarCraft II Binary PySC2 Agent アクション select_rect(p1, p2) or build_supply(p3) or … 観察 Resource Possible action Construction order Screen （Game Information） Minimap Reward -1/0/+1 SC2LE
298. 評価値 Value Network Baseline features アクション・タイプ ディレイ ユニット選択 命令発行 ターゲット選択 Residual MLP MLP MLP Pointer Network Attention D 分散表現 MLP 分散表現 MLP 分散表現 MLP Embedding MLP コア Deep LSTM スカラー エンコーダー MLP エンティティ エンコーダー トランス フォーマー 空間 エンコーダー ResNet ゲーム パラメーター群 エンティティ ミニマップ
299. https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii
300. DeepMind社「Capture the flag」 (2019)
301. シミュレーション 現実 機械学習 （ディープ ラーニン グ） https://deepmind.com/blog/article/capture-the-flag-science
302. π Game Screen Game Screen Game Screen Game Screen Game Screen Sampled vector Action Inner Reward ｗ Winning or Lose Judgemnet Policy Game Point Slow RNN Fast RNN Xt 𝑄𝑡 𝑄𝑡+1
303. 赤フラグを青チーム陣地に 持ち帰る青エージェント
304. https://deepmind.com/blog/article/capture-the-flag-science
305. OpenAI「HIDE AND SEEK」 (2019年)
308. Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI
309. レベル Character AI Enemy character Player Data Collection Spatial AI Meta-AI Order Ask & Report Control the game Support query query Brain MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
310. City Control AI Road obeserbing AI Area Control AI Observing and Controlling AI For building Observing and Controlling AI For roads Observing and Controlling AI For Park Observing and Controlling AI For Human Command Report Command Report 監視 制御 Huma n Drone Robot Digital Avatar Human 監視 制御 監視 制御 監視 制御 監視 制御 協調 協調 Coope ration Coopera tion 監視 制御 監視 制御 監視 制御 City
311. City Control AI Road obserbing AI Area Control AI Observing and Controlling AI For building Observing and Controlling AI For roads Observing and Controlling AI For Park Observing and Controlling AI For Human Command Report Command Report 監視 制御 Huma n Drone Robot Digital Avatar Human 監視 制御 監視 制御 監視 制御 監視 制御 協調 協調 Coope ration Coopera tion 監視 制御 監視 制御 監視 制御 City MetaAI Character AI Character AI Character AI Spatial AI
312. City Environmental Actor Network AI in Smart City （Meta-AI, Character AI, Spatial AI） AI System for Smart City City（Material, Structure, Network）
313. Latour’s Actor Network Theory • Actor is a subject and component in a network. • It acts autonomously and has influence to the network. • Every device is an actor in a city network. https://jimproctor.us/envs/mapping-actors-processes/
314. City Actor Network City (Physical) City AI
315. Environment World of a city Ordering World Model World Simulation Sensors in a city Actors in a city Spatial AI Recognition Decision Making Agents in a city Meta-AI （City AI） Character AI
316. Simulation by using Tokyo City 3D Models “PLATEAU”
317. Future Vision: AI-Driven Smart City Patrol by Drones delivery by a mobile robot Shopping supported by a robot Screens: Smart = Human Living and cooperation with Robots. Meta-AI will provide public services by using robots and drones.
318. The screenshot is on Unity screen shot with PLATEAU. Look of a 3D model ofa city “Shinagawa” road Pythonでの数値シミュ レーション同様、この道 路1つ1つにサービスの水 準を設定し、市民が通る と効用が得られるような 画面を作る。 Simulation by using a city 3D model data “PLATEAU” Tatsuya Ishimasa, Youichiro Miyake, Meta-AI driven Smart City, 64th conference of Infrastructure planning and management. 12/2021
319. Real World Real World Sensor Effector Digital Twin Prediction By decision simulating Game Engine Information Route
320. Physical Real World Smart space Metaverse Spatial AI connects real world and metaverse
321. Real World Real World Sensor Effector Prediction By decision simulating Game Engine Information Route