Tweet
Tweet

Slide 1

Slide 1 text

Game AI in Game Industry - Overview of Industrial Game AI – Youichiro Miyake @miyayou y.m.4160@gmail.com 5th International Game AI summer school 2023.6.29 https://www.facebook.com/youichiro.miyake https://miyayou.com

Slide 2

Slide 2 text

Youichiro Miyake Career highlights - AI Technical Advisor, “FINAL FANTASY XIV” in 2012 - Lead AI Architect, “FINAL FANTASY XV” in 2016 - AI Technical Director, “KINGDOM HEARTS III” in 2019 - QA Automation AI Technical Advisor, “FINAL FANTASY VII REMAKE” in 2020 - Director, Digital Games Research Association (DiGRA) JAPAN - Director, The Japanese Society for Artificial Intelligence Awards - 2008 Funai Award, Digital Contents Symposium 8th - 2012 Award for Programming/Development Environment Excellence, CEDEC - 2012 Youth Encouragement Award, DiGRA JAPAN - 2018 Book Award, CEDEC - 2021 Japanese Society of AI, Best Paper Award • SQEX AI Division, General Manager 20 members SQEX AI & Arts Alchemy, CTO 10members

Slide 3

Slide 3 text

Who is Youichiro Miyake ? • https://scholar.google.com/citations?user=CXqiolQAAAAJ • https://researchmap.jp/youichiro_miyake?lang=en • https://miyayou.com/

Slide 4

Slide 4 text

My Works (2004-2023) AI for Game Titles Books

Slide 5

Slide 5 text

AI Deep Neural Network 2D/3D Navigation Character System (GOAP, Emotional) Meta AI Natural Langage Programming Animation Generation Game AI (planning) Procedural Contents Generation

Slide 6

Slide 6 text

My belief •AI can change game design. •AI will make the new future of game.

Slide 7

Slide 7 text

Game AI Useful Link Lists by Youichiro Miyake • History of Game AI In Game Industry • https://www.ai-gakkai.or.jp/resource/my-bookmark/my- bookmark_vol37-no6/ • Fundamental Knowledge of Game AI in Game Industry • https://www.ai-gakkai.or.jp/resource/my-bookmark/my- bookmark_vol32-no4/

Slide 8

Slide 8 text

My English lecture on YouTube • GDC 2018 • Eos is Alive: The AI Systems of Final Fantasy XV • https://www.youtube.com/watch?v=ygNRNru1B_s • Replaying Japan • Keynote presentation: Youichiro Miyake • https://www.youtube.com/watch?v=9pIgpktZW3o

Slide 9

Slide 9 text

FFXV in Game AI Pro (Free download) • http://www.gameaipro.com/ • A Character Decision-Making System for FINAL FANTASY XV by Combining Behavior Trees and State Machines • Youichiro Miyake, Youji Shirakami, Kazuya Shimokawa, Kousuke Namiki, Tomoki Komatsu, Joudan Tatsuhiro, Prasert Prasertvithyakarn, and Takanori Yokoyama • http://www.gameaipro.com/GameAIPro3/GameAIPro3_Chapter11_A_Character_Decision- Making_System_for_FINAL_FANTASY_XV_by_Combining_Behavior_Trees_and_State_Machines .pdf • Logging Visualization in FINAL FANTASY XV • Matthew W. Johnson, Fabien Gravot, Shintaro Minamino, Ingimar Hólm Guðmundsson, Hendrik Skubch, and Youichiro Miyake • http://www.gameaipro.com/GameAIPro3/GameAIPro3_Chapter03_Logging_Visualization_in_FI NAL_FANTASY_XV.pdf • Predictive Animation Control Using Simulations and Fitted Models • Ingimar Hólm Guðmundsson, Hendrik Skubch, Fabien Gravot, and Youichiro Miyake • http://www.gameaipro.com/GameAIPro3/GameAIPro3_Chapter16_Predictive_Animation_Contr ol_Using_Simulations_and_Fitted_Models.pdf

Slide 10

Slide 10 text

FFXIV in Game AI Pro (Free download) • http://www.gameaipro.com/ • Precomputed Pathfinding for Large and Detailed Worlds on MMO Servers • Fabien Gravot, Takanori Yokoyama, Youichiro Miyake • http://www.gameaipro.com/GameAIPro/GameAIPro_Chapter20_Precomputed_Pathfinding_for _Large_and_Detailed_Worlds_on_MMO_Servers.pdf • Game Balancing AI • Game Balancing using Genetic Algorithms to Generate Player Agents • Kazuko Manabe and Youichiro Miyake • http://www.gameaipro.com/GameAIProOnlineEdition2021/GameAIProOnlineEdition2021_Chap ter17_Game_Balancing_using_Genetic_Algorithms_to_Generate_Player_Agents.pdf

Slide 11

Slide 11 text

GDC talk in 2018, 2022 • Eos is Alive: The AI Systems of 'Final Fantasy XV‘ (2018) • https://www.gdcvault.com/play/1025115/Eos-is-Alive-The-AI • AI Summit: Meta-Character-Spatial AI Dynamic Cooperative Model in Digital Game AI (2022) • https://www.gdcvault.com/play/1027641/AI-Summit-Meta- Character-Spatial

Slide 12

Slide 12 text

INTRODUCTION Chapter 2

Slide 13

Slide 13 text

How to design AI ? Player Character Rock Ground Pond For example, how to design AI in this situation ?

Slide 14

Slide 14 text

How to design AI ? Player Character Rock Ground Pond For example, how to design AI in this situation ? Ａ B （Example） If the player is in A, B goes around in B. If the player is in B, B attacks A.

Slide 15

Slide 15 text

How to design AI ? Player Character Rock Ground Pond For example, how to design AI in this situation ? Ａ B （Example） If the player is in A, B goes around in B. If the player is in B, B attack A. But this statement is not the AI’s idea. A game designer writes these ideas. = They are orders to AI. Classical AI = Scripted AI

Slide 16

Slide 16 text

Script From Scripted AI to Autonomous AI Game Designer’s idea Game Designer’s idea Knowledge Thinking Scripted AI Autonomous AI

Slide 17

Slide 17 text

Script From Scripted AI to Autonomous AI Game Designer’s idea Game Designer’s idea Knowledge Thiniking Scripted AI Autonomous AI No longer a puppet（scripted AI）. A character should be an autonomous AI which thinks by itself. Knowledge and thinking should be given to an AI.

Slide 18

Slide 18 text

Player Character Rock Ground Pond How to design AI ? Ａ B Terrain data （Way Points） Step 1 Let AI understand terrain ＝ Give terrain data to AI AI can guess it relative position to a player by using terrain data.

Slide 19

Slide 19 text

Navigation AI Thinking about positon by using terrain data Player Character Rock Ground Pond How to design AI ? Ａ B Terrain data （Way Points） Step 2 Let AI know its position by using terrain data. AI can guess its relative position to a player by using terrain data.

Slide 20

Slide 20 text

Navigation AI How to design AI ? Thinking by using terrain data Networked graph search Terrain data （Way Points） Thinking by using Terrain data Calculate the shortest path

Slide 21

Slide 21 text

Navigation AI How to design AI ? Thinking by using terrain data Networked graph search Terrain data （Way Points） Thinking by using Terrain data Calculate the shortest path

Slide 22

Slide 22 text

Navigation AI How to design AI ? Thinking by using terrain data Networked graph search Terrain data （Way Points） Thinking by using Terrain data Calculate the shortest path

Slide 23

Slide 23 text

Navigation AI How to design AI ? Make Networked Graph include LOS (Line of Sight) data. Terrain data （Way Points） Thinking by using Terrain data All points can be seen from any point in a group. All points can be seen from any point in a group.

Slide 24

Slide 24 text

Navigation AI How to design AI ? Make Networked Graph include LOS (Line of Sight) data. Terrain data （Way Points） Thinking by using Terrain data Any point of the group can not be seen any point in the group.

Slide 25

Slide 25 text

Navigation AI How to design AI ? An AI can be seen by the player. Terrain data （Way Points） Thinking by using Terrain data

Slide 26

Slide 26 text

How to design AI ? The player can not see the AI. Navigation AI Terrain data （Way Points） Thinking by using Terrain data

Slide 27

Slide 27 text

How to design AI ? The player can see the AI. Navigation AI Terrain data （Way Points） Thinking by using Terrain data

Slide 28

Slide 28 text

How to design AI ? The player can not see the AI. Navigation AI Terrain data （Way Points） Thinking by using Terrain data

Slide 29

Slide 29 text

How to design AI ? The AI will move so the player can see it. Navigation AI Terrain data （Way Points） Thinking by using Terrain data

Slide 30

Slide 30 text

How to design AI ? The AI will move so the player can not see it. Navigation AI Terrain data （Way Points） Thinking by using Terrain data

Slide 31

Slide 31 text

Thinking its positon by using terrain data Player Character Rock Ground Pond How to design AI ? Terrain data （Way Points） Step 3 Let AI think about timing when to attack. Decision Making

Slide 32

Slide 32 text

How to design AI ? Attack? Hide? Threaten? selection ＝ Decision Making Thinking Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making

Slide 33

Slide 33 text

How to design AI ? Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making Attack

Slide 34

Slide 34 text

How to design AI ? Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making Threaten (= Going to the positon where a player can see AI)

Slide 35

Slide 35 text

How to design AI ? Navigation AI Terrain data （Way Points） Thinking by using Terrain data Decision Making Hide（＝AI goes to the positon where a player can not see AI）

Slide 36

Slide 36 text

Thinking its positon by using terrain data Player Character Rock Ground Pond How to design AI ? Terrain data （Way Points） Step 1- 3 Let AI be intelligent. Decision Making

Slide 37

Slide 37 text

Making Autonomous AI Game Designer (Human) Terrain data（Way Points） Thinking based on Terrain data Thinking based on Object data

Slide 38

Slide 38 text

Making Autonomous AI Game Designer (Human) Terrain data（Way Points） Thinking based on Terrain data Thinking based on Object data The process to make AI be intelligent Step 1： Give knowledge Step2： Make AI think by using knowledge Step3 : Let AI do Decision Making by itself

Slide 39

Slide 39 text

Knowledge Thinking Making Autonomous AI Game Designer (Human) Intelligence ＝ Knowledge × Thinking Terrain data （Way Points） Thinking based on Terrain data Thinking based on Object data Object data Body data Thinking based on Body data Representing knowledge Thinking on knowledge

Slide 40

Slide 40 text

Intelligence Knowledge Thinking Making Autonomous AI Game Designer (Human) By adding knowledge and thinking, AI grows up to be more intelligent. Terrain data （Way Points） Thinking based on Terrain data Thinking based on Object data Object data Body data Thinking based on Body data Representing knowledge Thinking on knowledge

Slide 41

Slide 41 text

Intelligence Knowledge Thinking Making Autonomous AI Game Designer (Human) Terrain data （Way Points） Thinking based on Terrain data Thinking based on Object data Object data Body data Thinking based on Body data Representing knowledge Thinking on knowledge Intelligence a character has is called “Character AI”.

Slide 42

Slide 42 text

References GAME AI PRO 3 (2017/6) http://www.gameaipro.com/

Slide 43

Slide 43 text

FFXV AI Book is published in Japanese in 2019 • I want to publish in English. • Now finding a publisher.

Slide 44

Slide 44 text

History of Game AI in Japan

Slide 45

Slide 45 text

Classical Meta AI = Dynamic difficulty Adaptive AI 岩谷徹氏： ファンと一般のユーザーを満足させる方法のひとつは人工知能AIのような考え方です。プ レーヤースキルをプログラム側から判断して、難易度を調整していくというものです。これを私はセ ルフゲームコントロールシステムと呼んで10年以上前から開発に使っています。 － International Game Designers Panel － http://game.watch.impress.co.jp/docs/20050312/gdc_int.htm Toru Iwatani (PACMAN Creator) said : A way to satisfy game freak and game fun is AI . By judging player’s skill by program, difficulty level is dynamically adaptive to the player. That is called “Level Control System”, which had been used for 10 years.

Slide 46

Slide 46 text

Published Pac-Man specification papers Japanese Society of Artificial Intelligence,No.1, 2019. Anyone can download the article: https://jsai.ixsq.nii.ac.jp/ej/index.php?active_action=repository_view_main_item_detail&page_id=13&block_id=23&item_id=9670&item_no=1 Toru Iwatani, Mirei Takahashi, Youichiro Miyake, “Toru Iwatani, the PAC-MAN Creater, Interview about the Origin of Game AI “PAC-MAN”,” , Japanese Society of Artificial Intelligence, No.1 Vol.34, No.1 pp.86-99., 2019.

Slide 47

Slide 47 text

① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques

Slide 48

Slide 48 text

① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques

Slide 49

Slide 49 text

AI Monsters’ Behavior Policy: PAC-MAN • Wave Attack … 4 monsters attack to a player with all of them at once. • Two mode: Surrounding mode and dispersion mode

Slide 50

Slide 50 text

Monster’s two mode: surrounding and dispersion Surrounding mode Dispersion mode

Slide 51

Slide 51 text

7s 20s 7s 20s 5s 20s 5s 5s 20s 5s 20s 5s 27 34 54 59 7 25 30 50 55 5 79 84 7s 20s 7s 20s 5s 27 34 54 59 7 Monster dispersion Monster Attack A B C, D (sec) (sec) (sec) 3 types of Wave Attack time table

Slide 52

Slide 52 text

① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques

Slide 53

Slide 53 text

Monster Wave Attack Surrounding Dispersion AKA Go to the Pac-man’s position （８ｘ８dot） Go to the upper right of play field and move around it . PINK Go to 3 cell forward position to Pac-man’s position Go to the upper left of play field and move around it . CYAN Go to the symmetry positon to AKA’s position around Pac-man’s position. Go to the lower right of play field and move around it . ORANGE Within 130 dots from Pac-man, it has the same character of AKA, and Out of the 130 dots, it moves randomly. Go to the lower left of play field and mover around it . Wave Attack two modes and monsters’ behavior

Slide 54

Slide 54 text

Surrounding around a PAC-MAN

Slide 55

Slide 55 text

① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques

Slide 56

Slide 56 text

2 ３ ４ monsters A B C Number of Monsters to be spawn 244 30 90 ３ ４ monsters 50 ４monsters (Count of eaten foods)

Slide 57

Slide 57 text

① Monsters’ Wave attack two modes ② Monsters’ characteristic behaviors and indirect cooperation ④ Speed control for each level ③ Spawning timing control PAC-MAN Game AI techniques

Slide 58

Slide 58 text

SPEED PATTERN SPEED A Ｂ Ｃ Ｄ 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 ① イジケ ワープ ② ① イジケ ワープ ② ① イジケ ワープ ② ① イジケ ワープ ②

Slide 59

Slide 59 text

「SEAMAN」（1999, Dream Cast） Main game designer：Yutaka Saito （At the time：VIVARIUM） Interview (2017)

Slide 60

Slide 60 text

https://jsai.ixsq.nii.ac.jp/ej/?action=repository_action_common_download&item_id=8633&item_no=1&attribute_id=22&file_no=1

Slide 61

Slide 61 text

Mr.Saito speakes about the techniques of natural language conversation of SEAMAN.

Slide 62

Slide 62 text

• Not exact speech recognition • When SEAMAN cannot understand a user’s speech, he ignores it, and begins to lead the conversation by himself. • SEAMAN has some different pronunciations for one speech. It makes difference of meaning by pronouciations. SEAMAN Game AI techniques

Slide 63

Slide 63 text

Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI

Slide 64

Slide 64 text

OVERVIEW Chapter 1

Slide 65

Slide 65 text

OVERVIEW 1.1

Slide 66

Slide 66 text

Intelligent Game System Game System Intelligent Game System Transition in 1995 when 3D game begins.

Slide 67

Slide 67 text

What is digital game ? • Interactive Digital Space • The space becomes structured • AI becomes a module in a whole system

Slide 68

Slide 68 text

Level Script Navigation AI Character AI メタAI 1995 2000 2005 2010 (3D Games) 1994 2005 ( 2010頃～ （Open World Game) Spatial AI 1980 PlayStation (1994) Xbox360 (2005) PlayStation3 (2006) Scripted AI 三宅陽一郎、水野勇太、里井大輝、 「メタAI」と「AI Director」の歴史的発展、日本デジタルゲーム学会（2020年、Vol.13, No.2） LS-Model Model LCN-AI Model MCS-AI Cooperative Model MCN-AI Model

Slide 69

Slide 69 text

レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis

Slide 70

Slide 70 text

レベル Character AI Enemy character Player Data Collection Spatial AI Meta-AI Order Ask & Report Control the game Support query query Brain MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis

Slide 71

Slide 71 text

AI in game, and AI outside of game Game AI （outside of game） Game AI （In game） Meta-AI Character AI Navigation AI Supporting AI QA-AI Auto-balancing AI Interface-AI Data Mining Simulation Game Visualization Biofeedback

Slide 72

Slide 72 text

Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI

Slide 73

Slide 73 text

レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis

Slide 74

Slide 74 text

Meta AI History 1980 1990 1980 1990 2000 Classical Meta AI Modern Meta AI Development of Character AI (Autonomous AI ) Classical Meta AI to control difficulty of the game (Weaken enemies). Modern Meta AI to design game system dynamically.

Slide 75

Slide 75 text

(Example） Xevious（Namco、1983） Enemy Table Rewinding Enemy 0 Enemy １ Enemy 2 Enemt 3 Enemy 4 Enemy 5 https://www.youtube.com/watch?v=mbSHIFUUDLg

Slide 76

Slide 76 text

Monster’s two mode: surrounding and dispersion Surrounding mode Dispersion mode

Slide 77

Slide 77 text

AI Director (Meta AI) = Left 4 Dead Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html Example : Left 4 Dead

Slide 78

Slide 78 text

Modern Meta AI Meta AI dynamically changes a game Meta AI Allocation of enemies Spawning enemies Story Generation Procedural Terrain Generation User Experience

Slide 79

Slide 79 text

Left 4 Dead (Valve) AI Director controls NPC spawning distribution and game-play-pacing.  Genre：Online Action  Developer： Valve software  Publisher : Valve software  Hardware: XBox360  Year： 2006 Using nav-mesh for recognizing and controlling Real-time status of game. Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009 http://www.valvesoftware.com/publications.html

Slide 80

Slide 80 text

Left 4 Dead – AI Director Meta AI recognize game status in real-time Predicting player’s route and spawning monsters dynamically. Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009 http://www.valvesoftware.com/publications.html

Slide 81

Slide 81 text

AI Director decide when enemies are spawned. USER’S INTENSITY ACTUAL POPULATION DESIRED POPULATION Build Up …プレイヤーの緊張度が目標値を超えるまで 敵を出現させ続ける。 Sustain Peak … 緊張度のピークを3-5秒維持するために、 敵の数を維持する。 Peak Fade … 敵の数を最小限へ減少して行く。 Relax … プレイヤーたちが安全な領域へ行くまで、30-45秒間、 敵の出現を最小限に維持する。 AI Director makes users relax, and break relax repeatedly. Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009 http://www.valvesoftware.com/publications.html Meta AI(=AI Director) observes players’ action in real-time in Left 4 Dead

Slide 82

Slide 82 text

Adaptive Dramatic Dynamic Pacing [Basic Idea] (1) Meta AI populates many enemies, enough to get user’s intention up to the value (measured by input). (2) When user’s intention goes over the value, meta AI stops the population. (3) When user becomes relaxed, go to (1). Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html

Slide 83

Slide 83 text

Meta AI recognize game status via Navigation-Mesh Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html

Slide 84

Slide 84 text

Active Area Set Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html The area where meta-AI populate and extinguish agents is called AAS（= Active Area Set）

Slide 85

Slide 85 text

Active Area Set Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html

Slide 86

Slide 86 text

Active Area Set Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html

Slide 87

Slide 87 text

Flow Distance(predicting a path) Meta- AI predict a path - where they arrive - where is back - direction Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html

Slide 88

Slide 88 text

Action to AAS Meta AI spawns and extinguishes enemies in Active Area Set. Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html

Slide 89

Slide 89 text

Area seen from player In the Area where player can see, Meta-AI cannot spawn enemies Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html

Slide 90

Slide 90 text

Area to spawn enemies FRONT BACK Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html Meta- AI spawns enemies in the area players cannot see.

Slide 91

Slide 91 text

Monster and Item appearance frequency High Low Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html Wandereｒs (High) Mobs(Middle) Special Infected (Middle) Bosses (Low) Weapon Caches (Low) Scavenge Items (Middle)

Slide 92

Slide 92 text

Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html (1) Decide number of enemies to populate by user’s escape route length. (2) Populate enemies along the route in a area around the player. (3) When a player goes out the area, the population stops, and enemies vanish. (4) In the state of relax or when the are can be seen from a player, all enemies are eliminated from the map. Algorithm of Meta AI

Slide 93

Slide 93 text

References (1) Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009. (2) Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html (3) 三宅 陽一郎, “メタAI”,「デジタルゲームの技術」 P.186-190, ソフトバンク クリエイティブ

Slide 94

Slide 94 text

Character AI Spatial AI Meta-AI ③ ordering to move and help a player ① find a point between a player and an enemy ⑤ Pathfinding Pathfinding to a goal point ④ Go to the point and fight with enemies ② return a point

Slide 95

Slide 95 text

Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI

Slide 96

Slide 96 text

META-AI X PROCEDURAL CONTENTS GENERATION Chapter 3

Slide 97

Slide 97 text

PCG Game AI AI PCG （PCG, Procedural Contents Generation ）

Slide 98

Slide 98 text

Rogue (1980) Rect[0] Rect[0] Rect[1] Rect[0] Rect[1] Rect[2] Rect[3] http://racanhack.sourceforge.jp/rhdoc/intromaze.html

Slide 99

Slide 99 text

Terrain Generation from Brown Motion ロバート・ブラウン博士によって、１８２７年に発見された現象。 微粒が媒質（液体）の中で行う不規則な運動。 アインシュタイン博士によって、熱運動する媒質の不規則な 衝突によって引き起こされると説明された。 http://ja.wikipedia.org/wiki/%E3%83%96%E3%83%A9%E3%82%A6%E3%83%B3%E9%81%8B%E5%8B%95 Kazunori Miyata 「Procedural Contents Geneation」（CEDEC 2008）

Slide 100

Slide 100 text

Terrain Generation in Fractal (1987) http://ja.wikipedia.org/wiki/%E3%83%96%E3%83%A9%E3%82%A6%E3%83%B3%E9%81%8B%E5%8B%95 http://www.kenmusgrave.com

Slide 101

Slide 101 text

Terrain Generation in Fractal(1987) http://ja.wikipedia.org/wiki/%E3%83%96%E3%83%A9%E3%82%A6%E3%83%B3%E9%81%8B%E5%8B%95 http://www.kenmusgrave.com

Slide 102

Slide 102 text

NO MAN’S SKY (Hello Games, 2016) http://www.no-mans-sky.com/ 宇宙、星系、太陽系、惑星を自動生成する。

Slide 103

Slide 103 text

FarCry2 (Dunia Engine ) デモ 草原自動生成 時間システム 樹木自動生成 動的天候システム 動的天候システム http://www.farcry2-hq.com/downloads,18,dunia-engine-nr1.htm

Slide 104

Slide 104 text

Far Cry 2 Dunia Engine - Growing Vegetation (Far Cry HQ) https://www.youtube.com/watch?v=FI3oR6vqn1Q

Slide 105

Slide 105 text

Terrain Generation by Voronoi and Noise Jacob Olsen, Realtime Procedural Terrain Generation http://oddlabs.com/download/terrain_generation.pdf ２次元中点変位法 ボロノイ図 ＋ ＝

Slide 106

Slide 106 text

Ken Musgrave http://www.kenmusgrave.com/

Slide 107

Slide 107 text

Terragen(Planetside Software) Landscape Geneation http://www.planetside.co.uk/terragen/

Slide 108

Slide 108 text

3D Terrain Genetation in Age of Empires III 西川善司, 「3DゲームファンのためのAGE OF EMPIRESエンジン講座(後編)こだわりの影生成と算術合成 されるディテール、次回作はXbox2?」, GAME Watch, 2005

Slide 109

Slide 109 text

L-system による街の自動生成 City Engine(central pictures) Yoav I H Parish, Pascal Müller http://www.centralpictures.com/ce/tp/paper.pdf http://www.centralpictures.com/ce/ George Kelly, Hugh McCabe, A Survey of Procedural Techniques for City Generation http://www.gamesitb.com/SurveyProcedural.pdf

Slide 110

Slide 110 text

https://www.youtube.com/watch?v=NfizT369g60

Slide 111

Slide 111 text

https://www.youtube.com/watch?v=FzoY062kY1s

Slide 112

Slide 112 text

Terraon Generation from Americas （Eidos） https://www.eidosmontreal.com/news/worldgen-painting-the-world-one-layer-at-a-time/

Slide 113

Slide 113 text

VITRUVIO CityEngine Plugin for Unreal Engine https://esri.github.io/cityengine/vitruvio

Slide 114

Slide 114 text

SimCity 4 Engine, Glassbox https://www.youtube.com/watch?v=tKOgo7EFl_w

Slide 115

Slide 115 text

Procedural Generation in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8

Slide 116

Slide 116 text

Black Combination in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8

Slide 117

Slide 117 text

Procedural Generation Map in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8

Slide 118

Slide 118 text

Auto-analyzing the map in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8

Slide 119

Slide 119 text

Start, Exit, objectives auto-distribution in WarFrame The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8

Slide 120

Slide 120 text

Influence Map The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8

Slide 121

Slide 121 text

Influence Map The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8

Slide 122

Slide 122 text

Active Are Set The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8

Slide 123

Slide 123 text

Meta AI （AI Director, Dynamic Adaptive Pacing) The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8

Slide 124

Slide 124 text

メタＡＩ（自動適応ペーシング） Meta AI （AI Director, Dynamic Adaptive Pacing) The Living AI in Warframe's Procedural Space Ships Dan Brewer, Digital Extremes http://aigamedev.com/open/coverage/vienna14-report/#session8

Slide 125

Slide 125 text

Summary Classical Game System Modern Game System In classical game system, game contents are fixed after the development, but in modern game system, game contents are dynamically created.

Slide 126

Slide 126 text

Summary Classical Game System Modern Game System Meta AI dynamically creates a game play flow, and gives an order to character AI and a game event. The reason why meta AI action is simple is that character AI has become autonomous.

Slide 127

Slide 127 text

Referenced Papers (1) Michael Booth, "Replayable Cooperative Game Design: Left 4 Dead," Game Developer's Conference, March 2009. (2) Michael Booth, "The AI Systems of Left 4 Dead," Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford. http://www.valvesoftware.com/publications.html

Slide 128

Slide 128 text

Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847

Slide 129

Slide 129 text

Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847

Slide 130

Slide 130 text

Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847

Slide 131

Slide 131 text

CoG 2021: Adversarial Reinforcement Learning for Procedural Content Generation SEED – Electronic Arts https://www.youtube.com/watch?v=kNj0qcc6Fpg

Slide 132

Slide 132 text

Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847

Slide 133

Slide 133 text

Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847

Slide 134

Slide 134 text

Adversarial Reinforcement Learning for Procedural Content Generation Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, Konrad Tollmar https://arxiv.org/abs/2103.04847

Slide 135

Slide 135 text

NUVERS uses GPT3 in Animation Selection ［GDC 2023］GPT-3でNPCの会話を生成。Nuverseが「アース：リバイバル」でチャレンジする，新テクノロ ジーによるゲーム開発の“新たな一歩” https://www.4gamer.net/games/651/G065105/20230325007/

Slide 136

Slide 136 text

Earth revival (Nuverse) • Finding the animation of character by analyzing the lines 800 within 800ns. • GPT3 • Sound Generation Nuverse lets AI do all the talking and moving of NPCs in Earth: Revival https://premortem.games/2023/03/22/nuverse-lets-ai-do-all- the-talking-and-moving-of-npcs-in-earth-revival/ ジェネレーティブAIでゲーム開発支援！ GDCで発表されたリア ルな事例や取り組みを紹介 https://www.famitsu.com/news/202303/24297009.html

Slide 137

Slide 137 text

ChatGPT and Stable Diffusion usage RPG: Tales of Syn • How AI-assisted RPG Tales of Syn utilizes Stable Diffusion and ChatGPT to create assets and dialogues • https://gameworldobserver.com/2023/03/06/tales-of-syn-ai- rpg-stable-diffusion-chatgpt-game • Tales of Syn • https://talesofsyn.com/ • https://twitter.com/_hackmans_/status/1624501399383072768

Slide 138

Slide 138 text

ChatGPT と Stable Diffusion を用いた RPG: Tales of Syn • https://twitter.com/_hackmans_/status/1624501399383072768 • https://talesofsyn.com/

Slide 139

Slide 139 text

Tales of Syn Game - NPC Chat GPT3 Prototype (Tales of Syn) https://www.youtube.com/watch?v=ejw6OI4_lJw

Slide 140

Slide 140 text

Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI

Slide 141

Slide 141 text

CHARACTER AI Chapter 4

Slide 142

Slide 142 text

レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis

Slide 143

Slide 143 text

FC SFC SS, PS PS2,GC,Xbox Xbox360, PS3, Wii DC （次世代） Hardware 時間軸 2005 1999 Character AI Complex AI AI and game world has become more complex Simple AI （スペースインベーダー、タイトー、1978年） （アサシンクリード、ゲームロフト、2007年）

Slide 144

Slide 144 text

(Ex) Space Invader (1978) プレイヤーの動きに関係なく、決められた動きをする （スペースインベーダー、タイトー、1978年）

Slide 145

Slide 145 text

（Ex） Prince of Persia （プリンスオブペルシャ、1989年）

Slide 146

Slide 146 text

Halo (Bungie) http://halo.bungie.net/inside/publications.aspx#pub15068

Slide 147

Slide 147 text

Environment Intelligence Artificial Intelligence＝ dynamically makes an AI’s action in harmony with artificial environment. What is intelligence？ Body （Inner Structure） Input（Sensor） Output（Action）

Slide 148

Slide 148 text

Intelligence World Environment World Effector・Body Sensor・ Body Agent Architecture Agent Architecture

Slide 149

Slide 149 text

Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module Agent Architecture

Slide 150

Slide 150 text

Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module Objective・ Phenomenon Agent Architecture

Slide 151

Slide 151 text

Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module Agent Architecture

Slide 152

Slide 152 text

World Sensor 知識 生成 Knowledg e Making 意思決定 Decision Making 運動 生成 Motion Making Information Flow 記憶 Principle of Learning Action Result Decision Making Body

Slide 153

Slide 153 text

World Sensor 知識 生成 Knowledg e Making 意思決定 Decision Making 運動 生成 Motion Making Information Flow 記憶 Principle of Learning Action Result Decision Making Body

Slide 154

Slide 154 text

Deep Learning in Ubisoft https://gdcvault.com/play/1025653/The-Alchemy-and-Science-of The Alchemy and Science of Machine Learning for Games Yves Jacquier （Ubisoft Montreal）

Slide 155

Slide 155 text

Deep Learning in Ubisoft https://gdcvault.com/play/1025653/The-Alchemy-and-Science-of The Alchemy and Science of Machine Learning for Games Yves Jacquier （Ubisoft Montreal）

Slide 156

Slide 156 text

https://gdcvault.com/play/1025653/The-Alchemy-and-Science-of 35:00- Deep Learning in Ubisoft The Alchemy and Science of Machine Learning for Games Yves Jacquier （Ubisoft Montreal）

Slide 157

Slide 157 text

Belief – Desire – Intention モデル Desire (Perceptrons) Opinions (Decision Trees) Beliefs (Attribute List) Richard Evans, “Varieties of Learning”, 11.2, AI Programming Wisdom Low Energy Source =0.2 Weight =0.8 Value = Source*Weight = 0.16 Tasty Food Source =0.4 Weight =0.2 Value = Source*Weight = 0.08 Unhappines s Source =0.7 Weight =0.2 Value = Source*Weight = 0.14 ∑ 0.16+0.08+0.14 Threshold （0～1の値に 変換） hunger Perceptron in Black & White （Lionhead,2000）

Slide 158

Slide 158 text

Black & White （Lionhead,2000） Training of creature by learning http://www.youtube.com/watch?v=2t9ULyYGN-s http://www.lionhead.com/games/black-white/ 14:20-

Slide 159

Slide 159 text

Reinforcement Learning (Ex) Fighting Game Kick Punc h Magic R_0 : Reward = Damage http://piposozai.blog76.fc2.com/ http://dear-croa.d.dooo.jp/download/illust.html

Slide 160

Slide 160 text

LEARNING TO FIGHT T. Graepel, R. Herbrich, Julian Gold Published 2004 Computer Science https://www.microsoft.com/en-us/research/wp-content/uploads/2004/01/graehergol04.pdf

Slide 161

Slide 161 text

3 ft Q-Table THROW KICK STAND 1ft / GROUND 2ft / GROUND 3ft / GROUND 4ft / GROUND 5ft / GROUND 6ft / GROUND 1ft / KNOCKED 2ft / KNOCKED 3ft / KNOCKED 4ft / KNOCKED 5ft / KNOCKED 6ft / KNOCKED actions game states 13.2 10.2 -1.3 3.2 6.0 4.0 +10.0 Ralf Herbrich, Thore Graepel, Joaquin Quiñonero Candela Applied Games Group,Microsoft Research Cambridge "Forza, Halo, Xbox Live The Magic of Research in Microsoft Products" http://research.microsoft.com/en-us/projects/drivatar/ukstudentday.pptx

Slide 162

Slide 162 text

Early in the learning process … … after 15 minutes of learning Reward for decrease in Wulong Goth’s health Ralf Herbrich, Thore Graepel, Joaquin Quiñonero Candela Applied Games Group,Microsoft Research Cambridge "Forza, Halo, Xbox Live The Magic of Research in Microsoft Products" http://research.microsoft.com/en-us/projects/drivatar/ukstudentday.pptx

Slide 163

Slide 163 text

Early in the learning process … … after 15 minutes of learning Punishment for decrease in either player’s health Ralf Herbrich, Thore Graepel, Joaquin Quiñonero Candela Applied Games Group,Microsoft Research Cambridge "Forza, Halo, Xbox Live The Magic of Research in Microsoft Products" http://research.microsoft.com/en-us/projects/drivatar/ukstudentday.pptx

Slide 164

Slide 164 text

GENETIC ALGORITHM

Slide 165

Slide 165 text

「Astronoka」（ENIX, 1998） https://www.jp.square-enix.com/game/detail/astronoka/ Evolving by Genetic Algorithm

Slide 166

Slide 166 text

1998「Astronoka」 (PlayStation) Generating parameters of character specification by genetic algorithm. 三宅 陽一郎「ゲームAI技術20年の進化とこれから」 https://cedil.cesa.or.jp/cedil_sessions/view/1943 森川幸人「やってきたこと」よ Trap Jump Break Round about Creatures evolves to adapt the environment

Slide 167

Slide 167 text

No content

Slide 168

Slide 168 text

Genetic Algorithm Make a group evolve in one direction First generation New generation（１００～） … Evolution by generation One generation produces a next generation

Slide 169

Slide 169 text

Genetic Algorithm 遺伝子 Gene Next Generation Parent ① Parent② Pick up two parents from base group Crossover two genes Produce new generation (selection) (crossover) (production) Present Generation The iteration makes a desired generation in the environment 遺伝子 Gene

Slide 170

Slide 170 text

Flow chart of GA A player makes traps Beginning of a day Trap battle begins Trap battle ends Evaluation Fitness value Order Delete the last two NPC Pick up two parents from NPCs with high fitness value Generate two new NPCs Generate new generation Get enough number of new NPC The day ends Generation exchange rate changes Mutation rate changes

Slide 171

Slide 171 text

４－② Simulation and evaluation Evaluation score is proportional to how long NPC go into the field avoiding and breaking traps. G Fitness＝Score＋Time*0.3＋Enjoy param*0.5＋Trap_Score＋Safety＋HP*0.5 Time for clear G C B A B C C B C C S G A B C Start Bonus =100 Goal Bonus =50 Bonus =30 Bonus =10 Bonus =0

Slide 172

Slide 172 text

GA improvement in Game system Average of fitness A curve of fitness a day Generations (1) To give a feeling of evolution to users, the game system evolves 5 generations a day by GA. (2) To make constant speed of evolution of one day, the game system controls the number of iterations of GA evolution.

Slide 173

Slide 173 text

Neural Networks in Supreme Commander 2 (GDC 2012) Michael Robbins (Gas Powered Games) http://www.gdcvault.com/play/1015406/Off-the-Beaten-Path-Non http://www.gdcvault.com/play/1015667/Off-the-Beaten-Path-Non

Slide 174

Slide 174 text

Neural Networks in Supreme Commander 2 (GDC 2012) Michael Robbins (Gas Powered Games) http://www.gdcvault.com/play/1015406/Off-the-Beaten-Path-Non http://www.gdcvault.com/play/1015667/Off-the-Beaten-Path-Non Neural Network Application

Slide 175

Slide 175 text

Neural Networks in Supreme Commander 2 (GDC 2012) Michael Robbins (Gas Powered Games) http://www.gdcvault.com/play/1015406/Off-the-Beaten-Path-Non http://www.gdcvault.com/play/1015667/Off-the-Beaten-Path-Non

Slide 176

Slide 176 text

City Conquest Paul Tozor Postmortem: Intelligence Engine Design Systems' City Conquest https://www.gamedeveloper.com/business/postmortem-intelligence-engine-design-systems-i-city-conquest-i-

Slide 177

Slide 177 text

Build = 1,11,46,Mine,8 Build = 1,11,48,Mine,16 Build = 1,11,47,Skyc,24 Build = 1,14,42,Turr,28 Build = 1,14,45,GrSl,100808 Build = 1,14,40,Turr,171184 Build = 1,9,47,Mine, 52 Build = 1,9,49,Mine, 60 Build = 1,12,44,RktL,100836 Build = 1,10,49,Drop,100812 Build = 1,13,45,RktL,100816 Build = 1,17,44,RktL,100820 Build = 1,10,45,Skyc,100824 Upgrade = 0,13,45,RktL,100820 Build = 0,11,46,Mine,8 Build = 0,11,48,Mine,16 Build = 0,11,47,Skyc,24 Build = 0,14,42,Turr,28 Build = 0,14,40,Turr,171184 Build = 0,9,47,Mine 52 Build = 0,9,47,Mine 60 Build = 0,15,45,GrSl,100808 Build = 0,10.,49,Drop,100812 Build = 0,13,45,RktL,100816 Build = 0,17,44,RktL,100820 Build = 0,10,45,Skyc,100824 Upgrade = 0,13,45,RktL,100816 Build = 0,17,49,Drop,100832 Paul Tozor Postmortem: Intelligence Engine Design Systems' City Conquest https://www.gamedeveloper.com/business/postmortem-intelligence-engine-design-systems-i-city-conquest-i-

Slide 178

Slide 178 text

TT L_1 TT L2 TT L3 TT L4 TT L1 TT L2 TT L3 TT L4 Parent１ Parent ２ TT L_1 TT L2 TT L3 TT L4 TT L_1 TT L2 TT L3 TT L4 Child １ Child ２

Slide 179

Slide 179 text

Typical Agents in Digital Games (1)Morikawa-kun (SCE、1997年） (2)Astronoka (ENIX、1998年） (3)The Sims（EA, Maxis, 2000）

Slide 180

Slide 180 text

1997「GANBARE MORIKAWAKUN 2GO」 (PLAYSTATION) Character Control by Neural Network 三宅 陽一郎「ゲームAI技術20年の進化とこれから」 https://cedil.cesa.or.jp/cedil_sessions/view/1943 森川幸人「やってきたこと」より引用 Action Player orders an action Player orders an action Reaction from an item Decision Making Learning from the result AI！

Slide 181

Slide 181 text

Impression Impression Impression Impression https://www.youtube.com/watch?v=CFnCd38yvD8 Article https://morikatron.ai/2022/03/gameai_history_04/

Slide 182

Slide 182 text

Color１ Color 2 Form Smell Sound Behavior … Take an action Knock Kick Take back Eat Jump Push おいしい 快い 不快 Impression from item Nourishing Enjoyable Damaged Fear Fine … Table Data … … … … Texture Neural Network with Backpropagation in game ゲーム中にBP学習される ニューラルネットワーク② Put it on the head

Slide 183

Slide 183 text

Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI

Slide 184

Slide 184 text

CHARACTER BEHAVIOR LEARNING Chapter 3-2

Slide 185

Slide 185 text

Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module

Slide 186

Slide 186 text

Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI

Slide 187

Slide 187 text

DECISION MAKING ALGORITHM Chapter 3-3

Slide 188

Slide 188 text

Intelligence World Environment World Effector・Body Sensor・ Body Memory Recog- nition Working Memory Small Memory Information Processing Abstraction of Information Decision making Body Control Effector・Body Motion Making Motion synthesis process body module Motion Synthesis Decision making Module Decision making Module Decision making Module

Slide 189

Slide 189 text

Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive

Slide 190

Slide 190 text

Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive

Slide 191

Slide 191 text

Rule Control ID 0 : IF …. THEN … ID 1 : IF …. THEN … ID 3 : IF …. THEN …. ID 4 : IF …. THEN … ID .. : ID 4 : IF …. THEN … Priority: 3 Priority: 1 Priority: 4 Priority: 5 Priority: 2 A constant priority value or dynamic priority is attached to each rule. The rule with highest priority is fired in all fired rules. The system is used by tactical thinking and character behavior. Priority: - Control Rule ＝ IF （ Action Condition） then （Action Command）

Slide 192

Slide 192 text

Rule Control ID 0 : IF …. THEN … ID 1 : IF …. THEN … ID 3 : IF …. THEN …. ID 4 : IF …. THEN … ID .. : ID 5 : IF …. THEN … Priority: 3 Priority: 1 Priority: 4 Priority: 5 Priority: 2 Thinking module picked up one rule from the game situation. Priority: - Control Thinking Module Rule ＝ IF （ Action Condition） then （Action Command）

Slide 193

Slide 193 text

Example http://gomibako.symphonic-net.com/vote.cgi?genre=act_b&rating=9 ID 0 : IF （an enemy found） THEN （escape） ID 1 : IF （cannot find any） THEN （random walk） ID 2 : IF （found item） THEN （get the item） ID 3 : IF （found door） THEN （go through it） ID 0 : IF （enemy is strong） THEN （Magic） ID 1 : IF （found many enemies） THEN （attack the weakest） ID 2 : IF （HP<20） THEN （spell refresh magic ） ID 3 : IF （found a hole） THEN （avoid it） Action game RPG

Slide 194

Slide 194 text

Decision Making Rule-based AI  Genre：Simulation  Developer： Maxis  Publisher : EA  Hardware: PC, PS3  Year： 2009 [GDC2010] Richard Evans, "Modeling Individual Personalities in The Sims 3" http://www.gdcvault.com

Slide 195

Slide 195 text

Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive

Slide 196

Slide 196 text

F.E.A.R Agent Architecture  Genre：Horror FPS  Developer： Monolith Production  Publisher : SIERRA  Hardware: Windows  Year： 2004 Agent Architecture Considerations for Real-Time Planning in Games (AIIDE 2005) http://web.media.mit.edu/~jorkin/AIIDE05_Orkin_Planning.ppt Sensors Working Memory Planner Blackboard Navigation Animation / Movement Targeting Weapons World World

Slide 197

Slide 197 text

Planning by Chaining Example kSymbol_ TargetIsDead kSymbol_ TargetIsDead Attack kSymbol_ WeaponLoaded kSymbol_ WeaponLoaded Reload kSymbol_ WeaponArmed kSymbol_ WeaponArmed Pick up a weapon None Planner Planning Action Pools

Slide 198

Slide 198 text

Replaning AI in F.E.A.R. Jeff Orkins, Three States and a Plan: The AI of FEAR http://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear.pdf

Slide 199

Slide 199 text

Hierarchical Goal-oriented decision making in Chrome Hounds Iteration: Decomposing a goal to smaller goals Goal Goal Goal Goal Goal Goal Goal Goal Goal 三宅陽一郎 「クロムハウンズにおける人工知能開発から見るゲームAIの展望」 https://cedil.cesa.or.jp/cedil_sessions/view/50

Slide 200

Slide 200 text

Chrome Hounds walk along the pat Search the position of base Find the path Go to a Base Attack the base Conquer a base Shoot Walk Stop Attack until the base is fallen Tactics Operator (Behavior) ハウンズ Strategy Hierarchical Goal-oriented decision making in Chrome Hounds 三宅陽一郎 「クロムハウンズにおける人工知能開発から見るゲームAIの展望」 https://cedil.cesa.or.jp/cedil_sessions/view/50

Slide 201

Slide 201 text

Conquer_Combas_TeamAI.avi https://www.nicovideo.jp/watch/so14536895 https://www.nicovideo.jp/watch/so14536912? d=09f8b28b-b460- 4c8d-ad6b-d5d1f2917925 Hierarchical Goal-oriented decision making in Chrome Hounds Hierarchical Goal-oriented decision making in Chrome Hounds https://www.nicovideo.jp/watch/so14536895 https://www.nicovideo.jp/watch/so14536912 https://www.nicovideo.jp/watch/so14536698 三宅陽一郎 「クロムハウンズにおける人工知能開発から見るゲームAIの展望」 https://cedil.cesa.or.jp/cedil_sessions/view/50

Slide 202

Slide 202 text

Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Decision Making is generally a very complex high-degree process. But for digital game there are 7 simple decision making algorithms. Simulation-based AI Utility-based AI 「(something)-based AI」means that an algorithm uses (something) as a unit. Task-based AI Decision Making

Slide 203

Slide 203 text

State Machine State State State State Machine（Finite State Machine） AI’s instruction is described in a state, and changes in the world and AI are described in a transition condition. A state machine has a loop structure but does not have feedback dynamics.

Slide 204

Slide 204 text

Hierarchical State Machine Two enemy characters are patrolling. One player comes to the room. AI character is described as Hierarchical Finite State Machine (HFSM).

Slide 205

Slide 205 text

(Example) Quake HFSM http://ai-depot.com/FiniteStateMachines/FSM-Practical.html Monster’s FSM Spawn Idle Attack Die Melee Melee Smash Left Right Finish spawning Lost goal Located goal Zero health Zero health

Slide 206

Slide 206 text

Quake (id Software) https://www.youtube.com/watch?v=LpEygM6ZkfI&t=350s

Slide 207

Slide 207 text

Watch Chase Chase Attack Attack Warning Command Ally Watch the exit Call an Ally Ally not found & Out of battle field Join with Ally Join with Ally Found Ally Lost enemy Found enemy Lost enemy Found enemy Patrol 10 sec. passed Hear a sound Ally Responds Transition Condition State Higher State Hierarchical State Machine

Slide 208

Slide 208 text

Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Decision Making is generally a very complex high-degree process. But for digital game there are 7 simple decision making algorithms. Simulation-based AI Utility-based AI 「(something)-based AI」means that an algorithm uses (something) as a unit. Task-based AI Decision Making

Slide 209

Slide 209 text

Behavior-based AI Behavior A behavior represent not an animation detail but a physical action.

Slide 210

Slide 210 text

Behavior-based AI Behavior Behavior Decision-Making Behavior A behavior-based AI constructs its thinking by using some behaviors.

Slide 211

Slide 211 text

(1) Hierarchical Structure of Box (2) Each box has some behaviors (3) Each box has one selection-rule (random, sequential, priority-order…) (4) Finally, one behavior is selected at end-of-tree. (5) Decision Making Behavior-based AI  Genre：SciFi-FPS  Developer： BUNGIE Studio  Publisher : Microsoft  Hardware: Xbox, Windows, Mac  Year： 2004 Behavior-Tree (has become very popular method .) Damian Isla (2005), Handling Complexity in the Halo 2 AI, GDC Proceedings., http://www.gamasutra.com/gdc2005/features/20050311/isla_01.shtml

Slide 212

Slide 212 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 213

Slide 213 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 214

Slide 214 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 215

Slide 215 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule From the possible nodes in this layer, make the highest-priority node active.

Slide 216

Slide 216 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 217

Slide 217 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule Execute possible nodes from this layer in order.

Slide 218

Slide 218 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 219

Slide 219 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 220

Slide 220 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 221

Slide 221 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 222

Slide 222 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule Set one possible node from this layer to be executed randomly.

Slide 223

Slide 223 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 224

Slide 224 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 225

Slide 225 text

root Battle Retreat Idle Attack Hide Escape Block Stop Recover Trap Sleep Heal Arrow Sword Hind in wood Dig Hide behind object Magic Ice type Wind type Priority Priority Sequence Sequence Random Priority Random Priority Random Behavior （Leaf node） Layer Layer Selection Rule Selection Rule

Slide 226

Slide 226 text

Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Decision Making is generally a very complex high-degree process. But for digital game there are 7 simple decision making algorithms. Simulation-based AI Utility-based AI 「(something)-based AI」means that an algorithm uses (something) as a unit. Task-based AI Decision Making

Slide 227

Slide 227 text

State machine compared to Behavior tree • State machine＝ Steady control step by step • Behavior tree ＝ Adapt behavior fluently We want to use both good points

Slide 228

Slide 228 text

AI Graph Hierarchical Layered Architecture State machine Behavior tree A node on graph structure can reference to graph structure. IDLE MOVE FIGHT MOVE ATTACK State machine Different types of graph system can be combined.

Slide 229

Slide 229 text

Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive

Slide 230

Slide 230 text

Task based AI A big task is formed by combining some small tasks. A task must represent a simple practical action. (Simple case) Initial State GOAL Task１ Carry A to Ground Task2 Carry B to Ground Task3 Put C on B A B C B C A A B C A B C A C B Task3 Put A on C

Slide 231

Slide 231 text

Method （＝Resolving a big task into some small tasks） Make a drink （Task） Deliver a drink （Task） Take a drink to a buddies （Task）

Slide 232

Slide 232 text

Method （＝Resolving a big task into some small tasks） mix branch Collect all colors Branch Collect red and blue branch Collect green and red Pickup Blue Red Green mix Green Green mix Red Blue Red Blue No order task Total order task Partial Order Task There are all color jewels in a field There are green and red jewels in a field There are red and blue jewels in a field Primitive task Make a drink （Task） Precondition

Slide 233

Slide 233 text

Method （＝Resolving a big task into some small tasks） Make a drink （Task） Deliver a drink （Task） Take a drink to a buddies （Task）

Slide 234

Slide 234 text

Method Method Put that in car Bring a car Go to a battle field Put a drink In the car Attach a label Put it in Treasure box Total Order Task Has a car Primitive task Deliver a drink （Task） Precondition Put a drink In the car A car is near Partial Order Task

Slide 235

Slide 235 text

By iteration of applying methods , Task Network is generated. Mix Blue Red Blue START Bring a Car Go to a battl e field GOAL Put it in the car Attach a label Put it in Treasure box

Slide 236

Slide 236 text

Killzone 2’s AI Alex Champandard, Tim Verweij, Remco Straatman, "Killzone 2 Multiplayer Bots", http://files.aigamedev.com/coverage/GAIC09_Killzone2Bots_StraatmanChampandard.pdf

Slide 237

Slide 237 text

Organization in KILLZONE 2’s AI team Commander AI Squad Leader AI Squad Leader AI Squad Leader AI Order a tactic “Defend”,”Progress” Report of the tactics (Success/Failure) (Feedback) Order a target positon Order a target Request a re-order Request a re-order Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI Individual AI

Slide 238

Slide 238 text

Agent Architecture of a member (Individual AI) Orders Messages Daemon World State HTN Planner Plan Task Execution Controller Input Intelligence Stimuli Percepti on Threats

Slide 239

Slide 239 text

Agent Architecture of a Squad leader Strategy Orders Member Messages daemon World State HTN Planner Plan Task Execution Individual Orders Intelligence

Slide 240

Slide 240 text

Bullet Rocket fire Rocket To Humanoid Within range Not to Humanoid or turret Within a turret’s range Attack（Task） load fire end load fire end

Slide 241

Slide 241 text

… Progress None Defend a base （Task） Delete member’s memory Gather new information of buddies Begin a sequence Progress Go and Stop Defend

Slide 242

Slide 242 text

Behave （Task） Crouching Select Plan Medic start Broadcast Care End Car Defend Target fix Use a tool Walk care Move Stop Continue Begin Medic Not chage Change a tool Apply a method Apply a method Apply a method Apply a method

Slide 243

Slide 243 text

https://www.youtube.com/watch?v=GgtIZNJOuMA

Slide 244

Slide 244 text

Decision Making Model State-based AI Goal-based AI Rule-based AI Behavior-based AI Usually decision-Making is a very complex process. But in artificial intelligence, there are some simple basic styles. Simulation-based AI Utility-based AI Task-based AI Decision Making Reactive Non-Reactive

Slide 245

Slide 245 text

The Sims（EA, Maxis, 2000） • The game is to observe a society of agents Ken Forbus, “Simulation and Modeling: Under the hood of The Sims” (NorthWerstern Univ) http://www.cs.northwestern.edu/%7Eforbus/c95-gd/lectures/The_Sims_Under_the_Hood_files/frame.htm Richard Evans, Modeling Individual Personalities in The Sims 3, GDC 2010 http://www.gdcvault.com/play/1012450/Modeling-Individual-Personalities-in-The

Slide 246

Slide 246 text

The Sims 「Motif Engine」 • AI Character Engine Ken Forbus, “Simulation and Modeling: Under the hood of The Sims” (NorthWerstern University) http://www.cs.northwestern.edu/%7Eforbus/c95-gd/lectures/The_Sims_Under_the_Hood_files/frame.htm Data - Needs - Personality - Skills - Relationships Sloppy - Neat Shy - Outgoing Serious - Playful Lazy - Active Mean - Nice Physical - Hunger - Comfort - Hygiene - Bladder Mental - Energy - Fun - Social - Room Motive Engine Cooking Mechanical Logic Body Etc.

Slide 247

Slide 247 text

Utility-based Agent • Principle = Maximize a modd Hunger +20 Comfort -12 Hygiene -30 Bladder -75 Energy +80 Fun +40 Social +10 Room - 60 Mood +18 - Urinate (+40 Bladder) - Clean (+30 Room) - Unclog (+40 Room) Mood +26 - Take Bath (+40 Hygiene) (+30 Comfort) - Clean (+20 Room) Mood +20 Bathtub Toilet Ken Forbus, “Simulation and Modeling: Under the hood of The Sims” (NorthWerstern大学、講義資料) http://www.cs.northwestern.edu/%7Eforbus/c95-gd/lectures/The_Sims_Under_the_Hood_files/frame.htm

Slide 248

Slide 248 text

Mood calculation Mood = W_Hunger(X_Hunger) * X_Hunger + W_Engergy(X_Energy) * X_Energy + … -100 0 100 -100 0 100 -100 0 100 -100 0 100 -100 0 100 -100 0 100 W_Hunger W_Energy W_Comfort W_Fun W_Hygiene W_Social W_Bladder W_Room -100 0 100 -100 0 100 Ken Forbus, “Simulation and Modeling: Under the hood of The Sims” (NorthWerstern大学、講義資料) http://www.cs.northwestern.edu/%7Eforbus/c95-gd/lectures/The_Sims_Under_the_Hood_files/frame.htm

Slide 249

Slide 249 text

Utility W_Hunger X_Hunger W_Hunger(-80) -80 60 W_Hunger(60)

Slide 250

Slide 250 text

Utility comparison and The law of diminishing marginal utility Hunger degree at -80 ＝ W_Hunger(-80)*(-80) Hunger degree at 60 ＝ W_Hunger(60)*(60) Δ ＝ W_Hunger(60)*(60) - W_Hunger(-80)*(-80) Utility for hunger W_Hunger X_Hunger W_Hunger(-80) -80 60 W_Hunger(60) 90 W_Hunger(90)

Slide 251

Slide 251 text

The law of diminishing marginal utility Δ（-80 → 60）＝W_Hunger(60)*(60) - W_Hunger(-80)*(-80) Δ (60→90） ＝W_Hunger(90)*(90) - W_Hunger(60)*(60) Δ（-80 → 60） is much larger than Δ(60→90） Utility for hunger W_Hunger X_Hunger W_Hunger(-80) -80 60 W_Hunger(60) 90 W_Hunger(90)

Slide 252

Slide 252 text

Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI

Slide 253

Slide 253 text

SPATIAL AI Chapter 4

Slide 254

Slide 254 text

PATH-FINDING Chapter 4-2

Slide 255

Slide 255 text

レベル Character AI Enemy character Player 情報獲得 Spatial AI Meta-AI Order Ask & Report ゲーム全体をコントロール Support query query 頭脳として機能 MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis

Slide 256

Slide 256 text

Navigation Data Waypoint・Graph （Networked Graph with points） Navigation Mesh（Networked Graph with polygons） Walk-able フリー素材屋Hoshino http://www.s-hoshino.com/

Slide 257

Slide 257 text

Pathfinding Dragon Age pathfinding program put to the test （University of Alberta） https://www.youtube.com/watch?v=l7YQ5_Nbifo Daivuk RTS - Pathfinding A* https://www.youtube.com/watch?v=95aHGzzNCY8

Slide 258

Slide 258 text

（Mesh） Cost : 0.5 Visibility: 1.0 Surface： earth （Mesh） Cost: 0.8 Visibility: 0.7 Surface： pond （Object） Movable : to (1.0,0.8) Able to lift up: false Able to ride on： false Hardness： 0.9 Weight： 0.4 （Object） Action： knock down the lever Effect： Open the door （Object） Door Link between two nodes Unified Format of Location based Information = Common Ground

Slide 259

Slide 259 text

Player’s predicted route（called Golden Path） M. Jack , M. Vehkala, “Spaces in the Sandbox: Tactical Awareness in Open World Games,” GDC 2013, https://www.gdcvault.com/play/1018136/Spaces-in-the-Sandbox-Tactical

Slide 260

Slide 260 text

ゴールデンパスに沿った位置検索技術 Goal (guessed By Meta-AI) Player Golden Path Meta-AI finds the best point to put an enemy along the golden path.

Slide 261

Slide 261 text

Character AI Spatial AI Meta-AI ③ ordering to move and talk to a player ④ Path finding Pathfinding to a point near by player ⑤ run and face to a player ① Searching a point to talk to a player ② return the point around a player

Slide 262

Slide 262 text

TACTICAL POINT SYSTEM Chapter 4-3

Slide 263

Slide 263 text

History of Tactical Point System • Tactical Position Picking Killzone (2005, Guerrilla) in Program • TPS (Tactical Point System) CRYENGINE (2010, CRYTEK) Tool & Runtime System • EQS(Environment Query System) UNREAL ENGINE 4 （2014, Epic games）Tool & Runtime System • PQS (Point Query System) FINAL FANTASY XV (2016, SQUARE ENIX)

Slide 264

Slide 264 text

Point Query System A system to find a best positon - for a character’s ability - in a terrain - in real-time

Slide 265

Slide 265 text

Point Query System principle Point distribution （Generation） distributing point around the objective （例）grid、circle Filtering Removing points not adjust for a purpose by a conditon (1) ． ． ． Evaluation Evaluation for remained points and pick up one points with best score Filtering Removing points not adjust for a purpose by a conditon (2) Filtering Removing points not adjust for a purpose by a conditon (N)

Slide 266

Slide 266 text

PQS (Point Query System) A bowman finds the best point (i) Game situation (ii) Generating points around it (iii) Filtering points with bad terrain (iv) Filtering points where it’s arrow can not reach (v) Filtering points around buddies (vi) Picking up one point with highest terrain

Slide 267

Slide 267 text

Enemies Buddies Player Character Rock Mountation Rock 海 Hole

Slide 268

Slide 268 text

No content

Slide 269

Slide 269 text

No content

Slide 270

Slide 270 text

No content

Slide 271

Slide 271 text

No content

Slide 272

Slide 272 text

No content

Slide 273

Slide 273 text

No content

Slide 274

Slide 274 text

No content

Slide 275

Slide 275 text

No content

Slide 276

Slide 276 text

Terrain Analysis by TPS (Tactical Point System) M. Jack , M. Vehkala, “Spaces in the Sandbox: Tactical Awareness in Open World Games,” GDC 2013, https://www.gdcvault.com/play/1018136/Spaces-in-the-Sandbox-Tactical

Slide 277

Slide 277 text

M. Jack , M. Vehkala, “Spaces in the Sandbox: Tactical Awareness in Open World Games,” GDC 2013, https://www.gdcvault.com/play/1018136/Spaces-in-the-Sandbox-Tactical Terrain Analysis by TPS (Tactical Point System)

Slide 278

Slide 278 text

Character AI Spatial AI Meta-AI ⑤ Running the point in the front of a player ② Pathfinding to a player’s goal point ③ Searching the best point in the front of a player by PQS ⑥⑦Path-finding Finding a apath to a destination point ⑧running in the front of a player and having a conversation ④ the point in the front of a player ①Guessing player’s goal point

Slide 279

Slide 279 text

PQS finds the point in front of Player Character

Slide 280

Slide 280 text

PQS finds the point in front of Player Character

Slide 281

Slide 281 text

PQS finds the point in front of Player Character

Slide 282

Slide 282 text

PQS finds the point in front of Player Character

Slide 283

Slide 283 text

CASE 4: BUDDY NPC running in the front of a player • Objective: Making a buddy NPC running in the front of a player character (In many games, NPC only follows a player character) • Problem: Buddy NPCs must finds the position in the front of a play character intermittently • Solution: MCS-AI cooperative model finds the best positon to run in the front of a player character by using position query system. (1) Meta-AI Guesses a player’s destination point (2) Meta-AI asks Spatial AI to returns a path to the player’s destination point. (3) Meta-AI asks Spatial AI to get a point in the front of a player character. (4) Spatial AI returns the best point in the front of a player character by using PQS to Meta-AI. (5) Meta AI askes the Character AI to go to the point. (6) The Character AI asks Spatial AI to return a path to the point. (7) Spatial AI returns the path to the point. (8) The Character AI runs to the point in the front of a player character and has conversation with it. • Result: It seems that a buddy NPC guesses the player destination and runs in the front of a player character

Slide 284

Slide 284 text

AMBIENT AI Chapter 9 (NON-PLAYER CHARACTER IN THE TOWN) Not Just Planning: STRIPs for Ambient NPC Interactions in Final Fantasy XV by Hendrik Skubch, Square Enix (nucl.ai Conference 2015) https://archives.nucl.ai/recording/not-just-planning-strips-for-ambient-npc-interactions-in-final-fantasy-xv/

Slide 285

Slide 285 text

Smart Environmental Actors Smart Object Agent Smart Point Smart Space Agent Agent Control Control Control ｄoor Handle Space Object Point

Slide 286

Slide 286 text

DATA LOGGING AND VISUALIZATION Chapter 10

Slide 287

Slide 287 text

物理的都市空間 スマートスペース メタバース

Slide 288

Slide 288 text

Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI

Slide 289

Slide 289 text

年 企業 テーマ Open 2003 Microsoft 「Teo Feng」における強化学習 2005 Microsoft 「Forzamotor Sports」における強化学習 2013 DeepMind A tariのゲームをDQNで学習 2015 DeepMind 囲碁 AlphaGO 2017 AnyLogic 倉庫・機械などモデルのシミュレーション Microsoft 「パックマン」多報酬学習 Hybrid Reward Architecture for Reinforcement Learning 2019 Google 「サッカーシミュレーター」による強化学習の研究 〇 DeepMind 「Capture the flag」によるディープラーニング学習 Microsoft 「AirSim」ドローンシミュレーター 〇 Nvidia 「ドライビングシミュレーター」 Mircrosoft 「TextWorld」アドベンチャーゲームを題材に言語学習 〇 facebook 「CraftAssist」マインクラフト内で会話研究 〇 CarMelon カーネギーメロン大学「MineRL」マインクラフトを使ったAIコンテスト 〇 facebook 「LIGHT」ファンタジーワールドを構築してクラウドワーカーで会話研究 〇 OpenAI 「Dota2」OpenAIによる「OpenAIFive」 〇 DeepMind 「StarCraft2」AlphaStar 〇 DeepMind 「Capture the Flag」QuakeIII エンジン 〇 2020 Nvidia GameGAN「ディープラーニングによるパックマンの目コピー」 〇 DeepMind 「Agent57」AtariのほとんどのゲームをDQN+LSTMなどで学習 〇 OpenAI 「HIDE AND SEEK」かくれんぼを用いたマルチエージェントのカリキュラム学習 〇

Slide 290

Slide 290 text

Dota2 eSports OpenAI Five: Dota Gameplay https://www.youtube.com/watch?v=UZHTNBMAfAA 解説：『Dota 2』における人間側のチャンピオンチームとAIチームの戦い https://alienwarezone.jp/post/2316

Slide 291

Slide 291 text

[参考論文] 「OpenAI」については以下を参考にしました。 OpenAI Five https://openai.com/projects/five/ Christopher Berner, et al.,“Dota 2 with Large Scale Deep Reinforcement Learning” https://arxiv.org/abs/1912.06680

Slide 292

Slide 292 text

StarCraft~StarCraft2 （DeepMind, 2019）

Slide 293

Slide 293 text

StarCraftのAI • Santiago Ontañon, Gabriel Synnaeve, Alberto Uriarte, Florian Richoux, David Churchill, et al.. • “A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft”. IEEE Transactions on Computational Intelligence and AI in games, IEEE Computational Intelligence Society, 2013, 5(4), pp.1-19. hal- 00871001 • https://hal.archives-ouvertes.fr/hal-00871001

Slide 294

Slide 294 text

StarCraftの7つのアーキテクチャ (2010-2020)

Slide 295

Slide 295 text

StarCraft~StarCraft2における 人工知能 （DeepMind, 2019）

Slide 296

Slide 296 text

[参考文献] SC2LEについては以下の文献を参考にしました。 また図 7.8は以下の文献から引用しました。 Oriol Vinyals, et al., “StarCraft II: A New Challenge for Reinforcement Learning”, https://arxiv.org/abs/1708.04782 PySC2 - StarCraft II Learning Environment https://github.com/deepmind/pysc2

Slide 297

Slide 297 text

StarCraft II API StarCraft II Binary PySC2 Agent アクション select_rect(p1, p2) or build_supply(p3) or … 観察 Resource Possible action Construction order Screen （Game Information） Minimap Reward -1/0/+1 SC2LE

Slide 298

Slide 298 text

評価値 Value Network Baseline features アクション・タイプ ディレイ ユニット選択 命令発行 ターゲット選択 Residual MLP MLP MLP Pointer Network Attention D 分散表現 MLP 分散表現 MLP 分散表現 MLP Embedding MLP コア Deep LSTM スカラー エンコーダー MLP エンティティ エンコーダー トランス フォーマー 空間 エンコーダー ResNet ゲーム パラメーター群 エンティティ ミニマップ

Slide 299

Slide 299 text

https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii

Slide 300

Slide 300 text

DeepMind社「Capture the flag」 (2019)

Slide 301

Slide 301 text

シミュレーション 現実 機械学習 （ディープ ラーニン グ） https://deepmind.com/blog/article/capture-the-flag-science

Slide 302

Slide 302 text

π Game Screen Game Screen Game Screen Game Screen Game Screen Sampled vector Action Inner Reward ｗ Winning or Lose Judgemnet Policy Game Point Slow RNN Fast RNN Xt 𝑄𝑡 𝑄𝑡+1

Slide 303

Slide 303 text

赤フラグを青チーム陣地に 持ち帰る青エージェント

Slide 304

Slide 304 text

https://deepmind.com/blog/article/capture-the-flag-science

Slide 305

Slide 305 text

OpenAI「HIDE AND SEEK」 (2019年)

Slide 306

Slide 306 text

No content

Slide 307

Slide 307 text

No content

Slide 308

Slide 308 text

Contents • Chapter 1. Overview • Chapter 2. Meta-AI • Chapter 3. Meta-AI x ProceDural contents generation • Chapter 4. Character AI • Chapter 5. Character Behavior Learning • Chapter 6. Decision Making Algorithm • Chapter 7. Spatial AI • Chapter 8. Deep Learning • Chapter 9. Future of Game AI

Slide 309

Slide 309 text

レベル Character AI Enemy character Player Data Collection Spatial AI Meta-AI Order Ask & Report Control the game Support query query Brain MCS-AI dynamic co- operative model Dynamic allocation of enemies Observing level in real-time Direction for agents Making progress of game Autonomous thinking Cooperation Team AI Preparing data to make Meta-AI and Character AI recognize the level Managing object representation Managing Navigation data Path-finding / Tactical point analysis

Slide 310

Slide 310 text

City Control AI Road obeserbing AI Area Control AI Observing and Controlling AI For building Observing and Controlling AI For roads Observing and Controlling AI For Park Observing and Controlling AI For Human Command Report Command Report 監視 制御 Huma n Drone Robot Digital Avatar Human 監視 制御 監視 制御 監視 制御 監視 制御 協調 協調 Coope ration Coopera tion 監視 制御 監視 制御 監視 制御 City

Slide 311

Slide 311 text

City Control AI Road obserbing AI Area Control AI Observing and Controlling AI For building Observing and Controlling AI For roads Observing and Controlling AI For Park Observing and Controlling AI For Human Command Report Command Report 監視 制御 Huma n Drone Robot Digital Avatar Human 監視 制御 監視 制御 監視 制御 監視 制御 協調 協調 Coope ration Coopera tion 監視 制御 監視 制御 監視 制御 City MetaAI Character AI Character AI Character AI Spatial AI

Slide 312

Slide 312 text

City Environmental Actor Network AI in Smart City （Meta-AI, Character AI, Spatial AI） AI System for Smart City City（Material, Structure, Network）

Slide 313

Slide 313 text

Latour’s Actor Network Theory • Actor is a subject and component in a network. • It acts autonomously and has influence to the network. • Every device is an actor in a city network. https://jimproctor.us/envs/mapping-actors-processes/

Slide 314

Slide 314 text

City Actor Network City (Physical) City AI

Slide 315

Slide 315 text

Environment World of a city Ordering World Model World Simulation Sensors in a city Actors in a city Spatial AI Recognition Decision Making Agents in a city Meta-AI （City AI） Character AI

Slide 316

Slide 316 text

Simulation by using Tokyo City 3D Models “PLATEAU”

Slide 317

Slide 317 text

Future Vision: AI-Driven Smart City Patrol by Drones delivery by a mobile robot Shopping supported by a robot Screens: Smart = Human Living and cooperation with Robots. Meta-AI will provide public services by using robots and drones.

Slide 318

Slide 318 text

The screenshot is on Unity screen shot with PLATEAU. Look of a 3D model ofa city “Shinagawa” road Pythonでの数値シミュ レーション同様、この道 路1つ1つにサービスの水 準を設定し、市民が通る と効用が得られるような 画面を作る。 Simulation by using a city 3D model data “PLATEAU” Tatsuya Ishimasa, Youichiro Miyake, Meta-AI driven Smart City, 64th conference of Infrastructure planning and management. 12/2021

Slide 319

Slide 319 text

Real World Real World Sensor Effector Digital Twin Prediction By decision simulating Game Engine Information Route

Slide 320

Slide 320 text

Physical Real World Smart space Metaverse Spatial AI connects real world and metaverse

Slide 321

Slide 321 text

Real World Real World Sensor Effector Prediction By decision simulating Game Engine Information Route