Game AI in Game Industry - Overview of Industrial Game AI – Youichiro Miyake @miyayou [email protected] 5th International Game AI summer school 2023.6.29 https://www.facebook.com/youichiro.miyake https://miyayou.com
Youichiro Miyake Career highlights - AI Technical Advisor, “FINAL FANTASY XIV” in 2012 - Lead AI Architect, “FINAL FANTASY XV” in 2016 - AI Technical Director, “KINGDOM HEARTS III” in 2019 - QA Automation AI Technical Advisor, “FINAL FANTASY VII REMAKE” in 2020 - Director, Digital Games Research Association (DiGRA) JAPAN - Director, The Japanese Society for Artificial Intelligence Awards - 2008 Funai Award, Digital Contents Symposium 8th - 2012 Award for Programming/Development Environment Excellence, CEDEC - 2012 Youth Encouragement Award, DiGRA JAPAN - 2018 Book Award, CEDEC - 2021 Japanese Society of AI, Best Paper Award • SQEX AI Division, General Manager 20 members SQEX AI & Arts Alchemy, CTO 10members
AI Deep Neural Network 2D/3D Navigation Character System (GOAP, Emotional) Meta AI Natural Langage Programming Animation Generation Game AI (planning) Procedural Contents Generation
Game AI Useful Link Lists by Youichiro Miyake • History of Game AI In Game Industry • https://www.ai-gakkai.or.jp/resource/my-bookmark/my- bookmark_vol37-no6/ • Fundamental Knowledge of Game AI in Game Industry • https://www.ai-gakkai.or.jp/resource/my-bookmark/my- bookmark_vol32-no4/
My English lecture on YouTube • GDC 2018 • Eos is Alive: The AI Systems of Final Fantasy XV • https://www.youtube.com/watch?v=ygNRNru1B_s • Replaying Japan • Keynote presentation: Youichiro Miyake • https://www.youtube.com/watch?v=9pIgpktZW3o
FFXV in Game AI Pro (Free download) • http://www.gameaipro.com/ • A Character Decision-Making System for FINAL FANTASY XV by Combining Behavior Trees and State Machines • Youichiro Miyake, Youji Shirakami, Kazuya Shimokawa, Kousuke Namiki, Tomoki Komatsu, Joudan Tatsuhiro, Prasert Prasertvithyakarn, and Takanori Yokoyama • http://www.gameaipro.com/GameAIPro3/GameAIPro3_Chapter11_A_Character_Decision- Making_System_for_FINAL_FANTASY_XV_by_Combining_Behavior_Trees_and_State_Machines .pdf • Logging Visualization in FINAL FANTASY XV • Matthew W. Johnson, Fabien Gravot, Shintaro Minamino, Ingimar Hólm Guðmundsson, Hendrik Skubch, and Youichiro Miyake • http://www.gameaipro.com/GameAIPro3/GameAIPro3_Chapter03_Logging_Visualization_in_FI NAL_FANTASY_XV.pdf • Predictive Animation Control Using Simulations and Fitted Models • Ingimar Hólm Guðmundsson, Hendrik Skubch, Fabien Gravot, and Youichiro Miyake • http://www.gameaipro.com/GameAIPro3/GameAIPro3_Chapter16_Predictive_Animation_Contr ol_Using_Simulations_and_Fitted_Models.pdf
FFXIV in Game AI Pro (Free download) • http://www.gameaipro.com/ • Precomputed Pathfinding for Large and Detailed Worlds on MMO Servers • Fabien Gravot, Takanori Yokoyama, Youichiro Miyake • http://www.gameaipro.com/GameAIPro/GameAIPro_Chapter20_Precomputed_Pathfinding_for _Large_and_Detailed_Worlds_on_MMO_Servers.pdf • Game Balancing AI • Game Balancing using Genetic Algorithms to Generate Player Agents • Kazuko Manabe and Youichiro Miyake • http://www.gameaipro.com/GameAIProOnlineEdition2021/GameAIProOnlineEdition2021_Chap ter17_Game_Balancing_using_Genetic_Algorithms_to_Generate_Player_Agents.pdf
GDC talk in 2018, 2022 • Eos is Alive: The AI Systems of 'Final Fantasy XV‘ (2018) • https://www.gdcvault.com/play/1025115/Eos-is-Alive-The-AI • AI Summit: Meta-Character-Spatial AI Dynamic Cooperative Model in Digital Game AI (2022) • https://www.gdcvault.com/play/1027641/AI-Summit-Meta- Character-Spatial
How to design AI ? Player Character Rock Ground Pond For example, how to design AI in this situation ? Ａ B （Example） If the player is in A, B goes around in B. If the player is in B, B attacks A.
How to design AI ? Player Character Rock Ground Pond For example, how to design AI in this situation ? Ａ B （Example） If the player is in A, B goes around in B. If the player is in B, B attack A. But this statement is not the AI’s idea. A game designer writes these ideas. = They are orders to AI. Classical AI = Scripted AI
Script From Scripted AI to Autonomous AI Game Designer’s idea Game Designer’s idea Knowledge Thiniking Scripted AI Autonomous AI No longer a puppet（scripted AI）. A character should be an autonomous AI which thinks by itself. Knowledge and thinking should be given to an AI.
Player Character Rock Ground Pond How to design AI ? Ａ B Terrain data （Way Points） Step 1 Let AI understand terrain ＝ Give terrain data to AI AI can guess it relative position to a player by using terrain data.
Navigation AI Thinking about positon by using terrain data Player Character Rock Ground Pond How to design AI ? Ａ B Terrain data （Way Points） Step 2 Let AI know its position by using terrain data. AI can guess its relative position to a player by using terrain data.
Navigation AI How to design AI ? Thinking by using terrain data Networked graph search Terrain data （Way Points） Thinking by using Terrain data Calculate the shortest path
Navigation AI How to design AI ? Thinking by using terrain data Networked graph search Terrain data （Way Points） Thinking by using Terrain data Calculate the shortest path
Navigation AI How to design AI ? Thinking by using terrain data Networked graph search Terrain data （Way Points） Thinking by using Terrain data Calculate the shortest path
Navigation AI How to design AI ? Make Networked Graph include LOS (Line of Sight) data. Terrain data （Way Points） Thinking by using Terrain data All points can be seen from any point in a group. All points can be seen from any point in a group.
Navigation AI How to design AI ? Make Networked Graph include LOS (Line of Sight) data. Terrain data （Way Points） Thinking by using Terrain data Any point of the group can not be seen any point in the group.
Thinking its positon by using terrain data Player Character Rock Ground Pond How to design AI ? Terrain data （Way Points） Step 3 Let AI think about timing when to attack. Decision Making
How to design AI ? Attack? Hide? Threaten? selection ＝ Decision Making Thinking Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making
How to design AI ? Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making Threaten (= Going to the positon where a player can see AI)
How to design AI ? Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making Hide（＝AI goes to the positon where a player can not see AI）
Thinking its positon by using terrain data Player Character Rock Ground Pond How to design AI ? Terrain data （Way Points） Step 1- 3 Let AI be intelligent. Decision Making
Making Autonomous AI Game Designer (Human) Terrain data（Way Points） Thinking based on Terrain data Thinking based on Object data The process to make AI be intelligent Step 1： Give knowledge Step2： Make AI think by using knowledge Step3 : Let AI do Decision Making by itself
Knowledge Thinking Making Autonomous AI Game Designer (Human) Intelligence ＝ Knowledge × Thinking Terrain data （Way Points） Thinking based on Terrain data Thinking based on Object data Object data Body data Thinking based on Body data Representing knowledge Thinking on knowledge
Intelligence Knowledge Thinking Making Autonomous AI Game Designer (Human) By adding knowledge and thinking, AI grows up to be more intelligent. Terrain data （Way Points） Thinking based on Terrain data Thinking based on Object data Object data Body data Thinking based on Body data Representing knowledge Thinking on knowledge
Intelligence Knowledge Thinking Making Autonomous AI Game Designer (Human) Terrain data （Way Points） Thinking based on Terrain data Thinking based on Object data Object data Body data Thinking based on Body data Representing knowledge Thinking on knowledge Intelligence a character has is called “Character AI”.
Classical Meta AI = Dynamic difficulty Adaptive AI 岩谷徹氏： ファンと一般のユーザーを満足させる方法のひとつは人工知能AIのような考え方です。プ レーヤースキルをプログラム側から判断して、難易度を調整していくというものです。これを私はセ ルフゲームコントロールシステムと呼んで10年以上前から開発に使っています。 － International Game Designers Panel － http://game.watch.impress.co.jp/docs/20050312/gdc_int.htm Toru Iwatani (PACMAN Creator) said : A way to satisfy game freak and game fun is AI . By judging player’s skill by program, difficulty level is dynamically adaptive to the player. That is called “Level Control System”, which had been used for 10 years.
Published Pac-Man specification papers Japanese Society of Artificial Intelligence,No.1, 2019. Anyone can download the article: https://jsai.ixsq.nii.ac.jp/ej/index.php?active_action=repository_view_main_item_detail&page_id=13&block_id=23&item_id=9670&item_no=1 Toru Iwatani, Mirei Takahashi, Youichiro Miyake, “Toru Iwatani, the PAC-MAN Creater, Interview about the Origin of Game AI “PAC-MAN”,” , Japanese Society of Artificial Intelligence, No.1 Vol.34, No.1 pp.86-99., 2019.
① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques
① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques
AI Monsters’ Behavior Policy: PAC-MAN • Wave Attack … 4 monsters attack to a player with all of them at once. • Two mode: Surrounding mode and dispersion mode
① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques
Monster Wave Attack Surrounding Dispersion AKA Go to the Pac-man’s position （８ｘ８dot） Go to the upper right of play field and move around it . PINK Go to 3 cell forward position to Pac-man’s position Go to the upper left of play field and move around it . CYAN Go to the symmetry positon to AKA’s position around Pac-man’s position. Go to the lower right of play field and move around it . ORANGE Within 130 dots from Pac-man, it has the same character of AKA, and Out of the 130 dots, it moves randomly. Go to the lower left of play field and mover around it . Wave Attack two modes and monsters’ behavior
① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques
① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques
• Not exact speech recognition • When SEAMAN cannot understand a user’s speech, he ignores it, and begins to lead the conversation by himself. • SEAMAN has some different pronunciations for one speech. It makes difference of meaning by pronouciations. SEAMAN Game AI techniques
Level Script Navigation AI Character AI メタAI 1995 2000 2005 2010 (3D Games) 1994 2005 ( 2010頃～ （Open World Game) Spatial AI 1980 PlayStation (1994) Xbox360 (2005) PlayStation3 (2006) Scripted AI 三宅陽一郎、水野勇太、里井大輝、 「メタAI」と「AI Director」の歴史的発展、日本デジタルゲーム学会（2020年、Vol.13, No.2） LS-Model Model LCN-AI Model MCS-AI Cooperative Model MCN-AI Model
レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
レベル Character AI Enemy character Player Data Collection Spatial AI Meta-AI Order Ask & Report Control the game Support query query Brain MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
AI in game, and AI outside of game Game AI （outside of game） Game AI （In game） Meta-AI Character AI Navigation AI Supporting AI QA-AI Auto-balancing AI Interface-AI Data Mining Simulation Game Visualization Biofeedback
レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
Meta AI History 1980 1990 1980 1990 2000 Classical Meta AI Modern Meta AI Development of Character AI (Autonomous AI ) Classical Meta AI to control difficulty of the game (Weaken enemies). Modern Meta AI to design game system dynamically.
AI Director (Meta AI) = Left 4 Dead Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html Example : Left 4 Dead
Modern Meta AI Meta AI dynamically changes a game Meta AI Allocation of enemies Spawning enemies Story Generation Procedural Terrain Generation User Experience
Left 4 Dead (Valve) AI Director controls NPC spawning distribution and game-play-pacing. Genre：Online Action Developer： Valve software Publisher : Valve software Hardware: XBox360 Year： 2006 Using nav-mesh for recognizing and controlling Real-time status of game. Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009 http://www.valvesoftware.com/publications.html
Left 4 Dead – AI Director Meta AI recognize game status in real-time Predicting player’s route and spawning monsters dynamically. Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009 http://www.valvesoftware.com/publications.html
AI Director decide when enemies are spawned. USER’S INTENSITY ACTUAL POPULATION DESIRED POPULATION Build Up …プレイヤーの緊張度が目標値を超えるまで 敵を出現させ続ける。 Sustain Peak … 緊張度のピークを3-5秒維持するために、 敵の数を維持する。 Peak Fade … 敵の数を最小限へ減少して行く。 Relax … プレイヤーたちが安全な領域へ行くまで、30-45秒間、 敵の出現を最小限に維持する。 AI Director makes users relax, and break relax repeatedly. Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009 http://www.valvesoftware.com/publications.html Meta AI(=AI Director) observes players’ action in real-time in Left 4 Dead
Adaptive Dramatic Dynamic Pacing [Basic Idea] (1) Meta AI populates many enemies, enough to get user’s intention up to the value (measured by input). (2) When user’s intention goes over the value, meta AI stops the population. (3) When user becomes relaxed, go to (1). Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
Meta AI recognize game status via Navigation-Mesh Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
Active Area Set Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html The area where meta-AI populate and extinguish agents is called AAS（= Active Area Set）
Active Area Set Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
Active Area Set Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
Flow Distance(predicting a path) Meta- AI predict a path - where they arrive - where is back - direction Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
Action to AAS Meta AI spawns and extinguishes enemies in Active Area Set. Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
Area seen from player In the Area where player can see, Meta-AI cannot spawn enemies Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
Area to spawn enemies FRONT BACK Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html Meta- AI spawns enemies in the area players cannot see.
Monster and Item appearance frequency High Low Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html Wandereｒs (High) Mobs(Middle) Special Infected (Middle) Bosses (Low) Weapon Caches (Low) Scavenge Items (Middle)
Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html (1) Decide number of enemies to populate by user’s escape route length. (2) Populate enemies along the route in a area around the player. (3) When a player goes out the area, the population stops, and enemies vanish. (4) In the state of relax or when the are can be seen from a player, all enemies are eliminated from the map. Algorithm of Meta AI
References (1) Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009. (2) Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html (3) 三宅 陽一郎, “メタAI”,「デジタルゲームの技術」 P.186-190, ソフトバンク クリエイティブ
Character AI Spatial AI Meta-AI ③ ordering to move and help a player ① find a point between a player and an enemy ⑤ Pathfinding Pathfinding to a goal point ④ Go to the point and fight with enemies ② return a point
L-system による街の自動生成 City Engine(central pictures) Yoav I H Parish, Pascal Müller http://www.centralpictures.com/ce/tp/paper.pdf http://www.centralpictures.com/ce/ George Kelly, Hugh McCabe, A Survey of Procedural Techniques for City Generation http://www.gamesitb.com/SurveyProcedural.pdf
Procedural Generation in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
Black Combination in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
Procedural Generation Map in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
Auto-analyzing the map in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
Start, Exit, objectives auto-distribution in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
Influence Map The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
Influence Map The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
Active Are Set The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
Meta AI （AI Director, Dynamic Adaptive Pacing) The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
メタＡＩ（自動適応ペーシング） Meta AI （AI Director, Dynamic Adaptive Pacing) The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8
Summary Classical Game System Modern Game System In classical game system, game contents are fixed after the development, but in modern game system, game contents are dynamically created.
Summary Classical Game System Modern Game System Meta AI dynamically creates a game play flow, and gives an order to character AI and a game event. The reason why meta AI action is simple is that character AI has become autonomous.
Referenced Papers (1) Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009. (2) Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html
Earth revival (Nuverse) • Finding the animation of character by analyzing the lines 800 within 800ns. • GPT3 • Sound Generation Nuverse lets AI do all the talking and moving of NPCs in Earth: Revival https://premortem.games/2023/03/22/nuverse-lets-ai-do-all- the-talking-and-moving-of-npcs-in-earth-revival/ ジェネレーティブAIでゲーム開発支援！ GDCで発表されたリア ルな事例や取り組みを紹介 https://www.famitsu.com/news/202303/24297009.html
ChatGPT and Stable Diffusion usage RPG: Tales of Syn • How AI-assisted RPG Tales of Syn utilizes Stable Diffusion and ChatGPT to create assets and dialogues • https://gameworldobserver.com/2023/03/06/tales-of-syn-ai- rpg-stable-diffusion-chatgpt-game • Tales of Syn • https://talesofsyn.com/ • https://twitter.com/_hackmans_/status/1624501399383072768
レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
FC SFC SS, PS PS2,GC,Xbox Xbox360, PS3, Wii DC （次世代） Hardware 時間軸 2005 1999 Character AI Complex AI AI and game world has become more complex Simple AI （スペースインベーダー、タイトー、1978年） （アサシンクリード、ゲームロフト、2007年）
Environment Intelligence Artificial Intelligence＝ dynamically makes an AI’s action in harmony with artificial environment. What is intelligence？ Body （Inner Structure） Input（Sensor） Output（Action）
Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module Agent Architecture
Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module Objective・ Phenomenon Agent Architecture
Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module Agent Architecture
World Sensor 知識 生成 Knowledg e Making 意思決定 Decision Making 運動 生成 Motion Making Information Flow 記憶 Principle of Learning Action Result Decision Making Body
World Sensor 知識 生成 Knowledg e Making 意思決定 Decision Making 運動 生成 Motion Making Information Flow 記憶 Principle of Learning Action Result Decision Making Body
Deep Learning in Ubisoft https://gdcvault.com/play/1025653/The-Alchemy-and-Science-of The Alchemy and Science of Machine Learning for Games Yves Jacquier （Ubisoft Montreal）
Deep Learning in Ubisoft https://gdcvault.com/play/1025653/The-Alchemy-and-Science-of The Alchemy and Science of Machine Learning for Games Yves Jacquier （Ubisoft Montreal）
https://gdcvault.com/play/1025653/The-Alchemy-and-Science-of 35:00- Deep Learning in Ubisoft The Alchemy and Science of Machine Learning for Games Yves Jacquier （Ubisoft Montreal）
Black & White （Lionhead,2000） Training of creature by learning http://www.youtube.com/watch?v=2t9ULyYGN-s http://www.lionhead.com/games/black-white/ 14:20-
LEARNING TO FIGHT T. Graepel, R. Herbrich, Julian Gold Published 2004 Computer Science https://www.microsoft.com/en-us/research/wp-content/uploads/2004/01/graehergol04.pdf
Early in the learning process … … after 15 minutes of learning Reward for decrease in Wulong Goth’s health Ralf Herbrich, Thore Graepel, Joaquin Quiñonero Candela Applied Games Group,Microsoft Research Cambridge "Forza, Halo, Xbox Live The Magic of Research in Microsoft Products" http://research.microsoft.com/en-us/projects/drivatar/ukstudentday.pptx
Early in the learning process … … after 15 minutes of learning Punishment for decrease in either player’s health Ralf Herbrich, Thore Graepel, Joaquin Quiñonero Candela Applied Games Group,Microsoft Research Cambridge "Forza, Halo, Xbox Live The Magic of Research in Microsoft Products" http://research.microsoft.com/en-us/projects/drivatar/ukstudentday.pptx
1998「Astronoka」 (PlayStation) Generating parameters of character specification by genetic algorithm. 三宅 陽一郎「ゲームAI技術20年の進化とこれから」 https://cedil.cesa.or.jp/cedil_sessions/view/1943 森川幸人「やってきたこと」よ Trap Jump Break Round about Creatures evolves to adapt the environment
Genetic Algorithm Make a group evolve in one direction First generation New generation（１００～） … Evolution by generation One generation produces a next generation
Genetic Algorithm 遺伝子 Gene Next Generation Parent ① Parent② Pick up two parents from base group Crossover two genes Produce new generation (selection) (crossover) (production) Present Generation The iteration makes a desired generation in the environment 遺伝子 Gene
Flow chart of GA A player makes traps Beginning of a day Trap battle begins Trap battle ends Evaluation Fitness value Order Delete the last two NPC Pick up two parents from NPCs with high fitness value Generate two new NPCs Generate new generation Get enough number of new NPC The day ends Generation exchange rate changes Mutation rate changes
４－② Simulation and evaluation Evaluation score is proportional to how long NPC go into the field avoiding and breaking traps. G Fitness＝Score＋Time*0.3＋Enjoy param*0.5＋Trap_Score＋Safety＋HP*0.5 Time for clear G C B A B C C B C C S G A B C Start Bonus =100 Goal Bonus =50 Bonus =30 Bonus =10 Bonus =0
GA improvement in Game system Average of fitness A curve of fitness a day Generations (1) To give a feeling of evolution to users, the game system evolves 5 generations a day by GA. (2) To make constant speed of evolution of one day, the game system controls the number of iterations of GA evolution.
City Conquest Paul Tozor Postmortem: Intelligence Engine Design Systems' City Conquest https://www.gamedeveloper.com/business/postmortem-intelligence-engine-design-systems-i-city-conquest-i-
1997「GANBARE MORIKAWAKUN 2GO」 (PLAYSTATION) Character Control by Neural Network 三宅 陽一郎「ゲームAI技術20年の進化とこれから」 https://cedil.cesa.or.jp/cedil_sessions/view/1943 森川幸人「やってきたこと」より引用 Action Player orders an action Player orders an action Reaction from an item Decision Making Learning from the result AI！
Color１ Color 2 Form Smell Sound Behavior … Take an action Knock Kick Take back Eat Jump Push おいしい 快い 不快 Impression from item Nourishing Enjoyable Damaged Fear Fine … Table Data … … … … Texture Neural Network with Backpropagation in game ゲーム中にBP学習される ニューラルネットワーク② Put it on the head
Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module
Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module
Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive
Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive
Rule Control ID 0 : IF …. THEN … ID 1 : IF …. THEN … ID 3 : IF …. THEN …. ID 4 : IF …. THEN … ID .. : ID 4 : IF …. THEN … Priority: 3 Priority: 1 Priority: 4 Priority: 5 Priority: 2 A constant priority value or dynamic priority is attached to each rule. The rule with highest priority is fired in all fired rules. The system is used by tactical thinking and character behavior. Priority: - Control Rule ＝ IF （ Action Condition） then （Action Command）
Rule Control ID 0 : IF …. THEN … ID 1 : IF …. THEN … ID 3 : IF …. THEN …. ID 4 : IF …. THEN … ID .. : ID 5 : IF …. THEN … Priority: 3 Priority: 1 Priority: 4 Priority: 5 Priority: 2 Thinking module picked up one rule from the game situation. Priority: - Control Thinking Module Rule ＝ IF （ Action Condition） then （Action Command）
Example http://gomibako.symphonic-net.com/vote.cgi?genre=act_b&rating=9 ID 0 : IF （an enemy found） THEN （escape） ID 1 : IF （cannot find any） THEN （random walk） ID 2 : IF （found item） THEN （get the item） ID 3 : IF （found door） THEN （go through it） ID 0 : IF （enemy is strong） THEN （Magic） ID 1 : IF （found many enemies） THEN （attack the weakest） ID 2 : IF （HP<20） THEN （spell refresh magic ） ID 3 : IF （found a hole） THEN （avoid it） Action game RPG
Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive
F.E.A.R Agent Architecture Genre：Horror FPS Developer： Monolith Production Publisher : SIERRA Hardware: Windows Year： 2004 Agent Architecture Considerations for Real-Time Planning in Games (AIIDE 2005) http://web.media.mit.edu/~jorkin/AIIDE05_Orkin_Planning.ppt Sensors Working Memory Planner Blackboard Navigation Animation / Movement Targeting Weapons World World
Chrome Hounds walk along the pat Search the position of base Find the path Go to a Base Attack the base Conquer a base Shoot Walk Stop Attack until the base is fallen Tactics Operator (Behavior) ハウンズ Strategy Hierarchical Goal-oriented decision making in Chrome Hounds 三宅陽一郎 「クロムハウンズにおける人工知能開発から見るゲームAIの展望」 https://cedil.cesa.or.jp/cedil_sessions/view/50
Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Decision Making is generally a very complex high-degree process. But for digital game there are 7 simple decision making algorithms. Simulation-based AI Utility-based AI 「(something)-based AI」means that an algorithm uses (something) as a unit. Task-based AI Decision Making
State Machine State State State State Machine（Finite State Machine） AI’s instruction is described in a state, and changes in the world and AI are described in a transition condition. A state machine has a loop structure but does not have feedback dynamics.
Hierarchical State Machine Two enemy characters are patrolling. One player comes to the room. AI character is described as Hierarchical Finite State Machine (HFSM).
(Example) Quake HFSM http://ai-depot.com/FiniteStateMachines/FSM-Practical.html Monster’s FSM Spawn Idle Attack Die Melee Melee Smash Left Right Finish spawning Lost goal Located goal Zero health Zero health
Watch Chase Chase Attack Attack Warning Command Ally Watch the exit Call an Ally Ally not found & Out of battle field Join with Ally Join with Ally Found Ally Lost enemy Found enemy Lost enemy Found enemy Patrol 10 sec. passed Hear a sound Ally Responds Transition Condition State Higher State Hierarchical State Machine
Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Decision Making is generally a very complex high-degree process. But for digital game there are 7 simple decision making algorithms. Simulation-based AI Utility-based AI 「(something)-based AI」means that an algorithm uses (something) as a unit. Task-based AI Decision Making
(1) Hierarchical Structure of Box (2) Each box has some behaviors (3) Each box has one selection-rule (random, sequential, priority-order…) (4) Finally, one behavior is selected at end-of-tree. (5) Decision Making Behavior-based AI Genre：SciFi-FPS Developer： BUNGIE Studio Publisher : Microsoft Hardware: Xbox, Windows, Mac Year： 2004 Behavior-Tree (has become very popular method .) Damian Isla (2005), Handling Complexity in the Halo 2 AI, GDC Proceedings., http://www.gamasutra.com/gdc2005/features/20050311/isla_01.shtml
root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule From the possible nodes in this layer, make the highest-priority node active.
root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule Execute possible nodes from this layer in order.
root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule Set one possible node from this layer to be executed randomly.
Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Decision Making is generally a very complex high-degree process. But for digital game there are 7 simple decision making algorithms. Simulation-based AI Utility-based AI 「(something)-based AI」means that an algorithm uses (something) as a unit. Task-based AI Decision Making
State machine compared to Behavior tree • State machine＝ Steady control step by step • Behavior tree ＝ Adapt behavior fluently We want to use both good points
AI Graph Hierarchical Layered Architecture State machine Behavior tree A node on graph structure can reference to graph structure. IDLE MOVE FIGHT MOVE ATTACK State machine Different types of graph system can be combined.
Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive
Task based AI A big task is formed by combining some small tasks. A task must represent a simple practical action. (Simple case) Initial State GOAL Task１ Carry A to Ground Task2 Carry B to Ground Task3 Put C on B A B C B C A A B C A B C A C B Task3 Put A on C
Method （＝Resolving a big task into some small tasks） mix branch Collect all colors Branch Collect red and blue branch Collect green and red Pickup Blue Red Green mix Green Green mix Red Blue Red Blue No order task Total order task Partial Order Task There are all color jewels in a field There are green and red jewels in a field There are red and blue jewels in a field Primitive task Make a drink （Task） Precondition
Method Method Put that in car Bring a car Go to a battle field Put a drink In the car Attach a label Put it in Treasure box Total Order Task Has a car Primitive task Deliver a drink （Task） Precondition Put a drink In the car A car is near Partial Order Task
By iteration of applying methods , Task Network is generated. Mix Blue Red Blue START Bring a Car Go to a battl e field GOAL Put it in the car Attach a label Put it in Treasure box
Killzone 2’s AI Alex Champandard, Tim Verweij, Remco Straatman, "Killzone 2 Multiplayer Bots", http://files.aigamedev.com/coverage/GAIC09_Killzone2Bots_StraatmanChampandard.pdf
Organization in KILLZONE 2’s AI team Commander AI Squad Leader AI Squad Leader AI Squad Leader AI Order a tactic “Defend”,”Progress” Report of the tactics (Success/Failure) (Feedback) Order a target positon Order a target Request a re-order Request a re-order Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI
Agent Architecture of a member (Individual AI) Orders Messages Daemon World State HTN Planner Plan Task Execution Controller Input Intelligence Stimuli Percepti on Threats
Behave （Task） Crouching Select Plan Medic start Broadcast Care End Car Defend Target fix Use a tool Walk care Move Stop Continue Begin Medic Not chage Change a tool Apply a method Apply a method Apply a method Apply a method
Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive
The Sims（EA, Maxis, 2000） • The game is to observe a society of agents Ken Forbus, “Simulation and Modeling: Under the hood of The Sims” (NorthWerstern Univ) http://www.cs.northwestern.edu/%7Eforbus/c95-gd/lectures/The_Sims_Under_the_Hood_files/frame.htm Richard Evans, Modeling Individual Personalities in The Sims 3, GDC 2010 http://www.gdcvault.com/play/1012450/Modeling-Individual-Personalities-in-The
The Sims 「Motif Engine」 • AI Character Engine Ken Forbus, “Simulation and Modeling: Under the hood of The Sims” (NorthWerstern University) http://www.cs.northwestern.edu/%7Eforbus/c95-gd/lectures/The_Sims_Under_the_Hood_files/frame.htm Data - Needs - Personality - Skills - Relationships Sloppy - Neat Shy - Outgoing Serious - Playful Lazy - Active Mean - Nice Physical - Hunger - Comfort - Hygiene - Bladder Mental - Energy - Fun - Social - Room Motive Engine Cooking Mechanical Logic Body Etc.
The law of diminishing marginal utility Δ（-80 → 60）＝W_Hunger(60)*(60) - W_Hunger(-80)*(-80) Δ (60→90） ＝W_Hunger(90)*(90) - W_Hunger(60)*(60) Δ（-80 → 60） is much larger than Δ(60→90） Utility for hunger W_Hunger X_Hunger W_Hunger(-80) -80 60 W_Hunger(60) 90 W_Hunger(90)
レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
Navigation Data Waypoint・Graph （Networked Graph with points） Navigation Mesh（Networked Graph with polygons） Walk-able フリー素材屋Hoshino http://www.s-hoshino.com/
Pathfinding Dragon Age pathfinding program put to the test （University of Alberta） https://www.youtube.com/watch?v=l7YQ5_Nbifo Daivuk RTS - Pathfinding A* https://www.youtube.com/watch?v=95aHGzzNCY8
（Mesh） Cost : 0.5 Visibility: 1.0 Surface： earth （Mesh） Cost: 0.8 Visibility: 0.7 Surface： pond （Object） Movable : to (1.0,0.8) Able to lift up: false Able to ride on： false Hardness： 0.9 Weight： 0.4 （Object） Action： knock down the lever Effect： Open the door （Object） Door Link between two nodes Unified Format of Location based Information = Common Ground
Player’s predicted route（called Golden Path） M. Jack , M. Vehkala, “Spaces in the Sandbox: Tactical Awareness in Open World Games,” GDC 2013, https://www.gdcvault.com/play/1018136/Spaces-in-the-Sandbox-Tactical
Character AI Spatial AI Meta-AI ③ ordering to move and talk to a player ④ Path finding Pathfinding to a point near by player ⑤ run and face to a player ① Searching a point to talk to a player ② return the point around a player
History of Tactical Point System • Tactical Position Picking Killzone (2005, Guerrilla) in Program • TPS (Tactical Point System) CRYENGINE (2010, CRYTEK) Tool & Runtime System • EQS(Environment Query System) UNREAL ENGINE 4 （2014, Epic games）Tool & Runtime System • PQS (Point Query System) FINAL FANTASY XV (2016, SQUARE ENIX)
Point Query System principle Point distribution （Generation） distributing point around the objective （例）grid、circle Filtering Removing points not adjust for a purpose by a conditon (1) ． ． ． Evaluation Evaluation for remained points and pick up one points with best score Filtering Removing points not adjust for a purpose by a conditon (2) Filtering Removing points not adjust for a purpose by a conditon (N)
PQS (Point Query System) A bowman finds the best point (i) Game situation (ii) Generating points around it (iii) Filtering points with bad terrain (iv) Filtering points where it’s arrow can not reach (v) Filtering points around buddies (vi) Picking up one point with highest terrain
Terrain Analysis by TPS (Tactical Point System) M. Jack , M. Vehkala, “Spaces in the Sandbox: Tactical Awareness in Open World Games,” GDC 2013, https://www.gdcvault.com/play/1018136/Spaces-in-the-Sandbox-Tactical
M. Jack , M. Vehkala, “Spaces in the Sandbox: Tactical Awareness in Open World Games,” GDC 2013, https://www.gdcvault.com/play/1018136/Spaces-in-the-Sandbox-Tactical Terrain Analysis by TPS (Tactical Point System)
Character AI Spatial AI Meta-AI ⑤ Running the point in the front of a player ② Pathfinding to a player’s goal point ③ Searching the best point in the front of a player by PQS ⑥⑦Path-finding Finding a apath to a destination point ⑧running in the front of a player and having a conversation ④ the point in the front of a player ①Guessing player’s goal point
CASE 4: BUDDY NPC running in the front of a player • Objective: Making a buddy NPC running in the front of a player character (In many games, NPC only follows a player character) • Problem: Buddy NPCs must finds the position in the front of a play character intermittently • Solution: MCS-AI cooperative model finds the best positon to run in the front of a player character by using position query system. (1) Meta-AI Guesses a player’s destination point (2) Meta-AI asks Spatial AI to returns a path to the player’s destination point. (3) Meta-AI asks Spatial AI to get a point in the front of a player character. (4) Spatial AI returns the best point in the front of a player character by using PQS to Meta-AI. (5) Meta AI askes the Character AI to go to the point. (6) The Character AI asks Spatial AI to return a path to the point. (7) Spatial AI returns the path to the point. (8) The Character AI runs to the point in the front of a player character and has conversation with it. • Result: It seems that a buddy NPC guesses the player destination and runs in the front of a player character
AMBIENT AI Chapter 9 (NON-PLAYER CHARACTER IN THE TOWN) Not Just Planning: STRIPs for Ambient NPC Interactions in Final Fantasy XV by Hendrik Skubch, Square Enix (nucl.ai Conference 2015) https://archives.nucl.ai/recording/not-just-planning-strips-for-ambient-npc-interactions-in-final-fantasy-xv/
[参考論文] 「OpenAI」については以下を参考にしました。 OpenAI Five https://openai.com/projects/five/ Christopher Berner, et al.,“Dota 2 with Large Scale Deep Reinforcement Learning” https://arxiv.org/abs/1912.06680
StarCraftのAI • Santiago Ontañon, Gabriel Synnaeve, Alberto Uriarte, Florian Richoux, David Churchill, et al.. • “A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft”. IEEE Transactions on Computational Intelligence and AI in games, IEEE Computational Intelligence Society, 2013, 5(4), pp.1-19. hal- 00871001 • https://hal.archives-ouvertes.fr/hal-00871001
[参考文献] SC2LEについては以下の文献を参考にしました。 また図 7.8は以下の文献から引用しました。 Oriol Vinyals, et al., “StarCraft II: A New Challenge for Reinforcement Learning”, https://arxiv.org/abs/1708.04782 PySC2 - StarCraft II Learning Environment https://github.com/deepmind/pysc2
StarCraft II API StarCraft II Binary PySC2 Agent アクション select_rect(p1, p2) or build_supply(p3) or … 観察 Resource Possible action Construction order Screen （Game Information） Minimap Reward -1/0/+1 SC2LE
π Game Screen Game Screen Game Screen Game Screen Game Screen Sampled vector Action Inner Reward ｗ Winning or Lose Judgemnet Policy Game Point Slow RNN Fast RNN Xt 𝑄𝑡 𝑄𝑡+1
レベル Character AI Enemy character Player Data Collection Spatial AI Meta-AI Order Ask & Report Control the game Support query query Brain MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis
City Control AI Road obeserbing AI Area Control AI Observing and Controlling AI For building Observing and Controlling AI For roads Observing and Controlling AI For Park Observing and Controlling AI For Human Command Report Command Report 監視 制御 Huma n Drone Robot Digital Avatar Human 監視 制御 監視 制御 監視 制御 監視 制御 協調 協調 Coope ration Coopera tion 監視 制御 監視 制御 監視 制御 City
City Control AI Road obserbing AI Area Control AI Observing and Controlling AI For building Observing and Controlling AI For roads Observing and Controlling AI For Park Observing and Controlling AI For Human Command Report Command Report 監視 制御 Huma n Drone Robot Digital Avatar Human 監視 制御 監視 制御 監視 制御 監視 制御 協調 協調 Coope ration Coopera tion 監視 制御 監視 制御 監視 制御 City MetaAI Character AI Character AI Character AI Spatial AI
Latour’s Actor Network Theory • Actor is a subject and component in a network. • It acts autonomously and has influence to the network. • Every device is an actor in a city network. https://jimproctor.us/envs/mapping-actors-processes/
Environment World of a city Ordering World Model World Simulation Sensors in a city Actors in a city Spatial AI Recognition Decision Making Agents in a city Meta-AI （City AI） Character AI
Future Vision: AI-Driven Smart City Patrol by Drones delivery by a mobile robot Shopping supported by a robot Screens: Smart = Human Living and cooperation with Robots. Meta-AI will provide public services by using robots and drones.
The screenshot is on Unity screen shot with PLATEAU. Look of a 3D model ofa city “Shinagawa” road Pythonでの数値シミュ レーション同様、この道 路1つ1つにサービスの水 準を設定し、市民が通る と効用が得られるような 画面を作る。 Simulation by using a city 3D model data “PLATEAU” Tatsuya Ishimasa, Youichiro Miyake, Meta-AI driven Smart City, 64th conference of Infrastructure planning and management. 12/2021
miyayou
signer
robinlee
moocres
cm_manaus
okuramasafumi
moneyforwarddev
katsuhisa91
kaityo256
kfujita
oripsolob
gr2m
hannesfritz
jmmastey
jonrohan
mojombo
jakevdp
jponch
philhawksworth
productmarketing
lemiorhan
reverentgeek
notwaldorf