Page. 1 Yusuke Hayashi || 2024.10.26 Y u s u k e H a y a s h i | | 2 0 2 4 . 1 0 . 2 6 P a g e . 0 On the Collective Predictive Coding Hypothesis and the Phase Transition Phenomena in Multi-Agent Systems Japanese Association for Philosophy of Science: 27th October 2024 Workshop Yusuke Hayashi (AI Alignment Network)

Page. 2 Yusuke Hayashi || 2024.10.26 Hidden states FEP agent Actions Active Inference World model Collective Predictive Coding as Active Inference ① Is it possible to describe collective phenomena using a single-agent model? Collective Predictive Coding What I talk about today is… and the phase transition phenomena in multi-agent systems. Sensory inputs/Rewards

Page. 3 Yusuke Hayashi || 2024.10.26 Sensory inputs/Rewards Hidden states FEP agent Actions Active Inference World model Collective Predictive Coding as Active Inference ① Is it possible to describe collective phenomena using a single-agent model? Collective Predictive Coding ② The answer is yes. Deep learning integrates signals generated by individual neurons to manifest a cohesive function as a whole. What I talk about today is… and the phase transition phenomena in multi-agent systems.

Page. 4 Yusuke Hayashi || 2024.10.26 Collective Predictive Coding as Active Inference What I talk about today is… and the phase transition phenomena in multi-agent systems. update update Small CPC Models Agent 1 Agent 2 Sample Size Deep Learning Neuronal Population 1 Neuronal Population 2 Large CPC Models update update Agent Population1 Agent Population2

Page. 5 Yusuke Hayashi || 2024.10.26 What is Active Inference? Collective Predictive Coding as Active Inference FEP agent (dog) ① Perceptual inference

Page. 6 Yusuke Hayashi || 2024.10.26 What is Active Inference? Collective Predictive Coding as Active Inference FEP agent (dog) ① Perceptual inference External world (the dog owner)

Page. 7 Yusuke Hayashi || 2024.10.26 What is Active Inference? Collective Predictive Coding as Active Inference Hidden states FEP agent (dog) ① Perceptual inference External world (the dog owner)

Page. 8 Yusuke Hayashi || 2024.10.26 What is Active Inference? Collective Predictive Coding as Active Inference Hidden states FEP agent (dog) ① Perceptual inference External world (the dog owner) Sensory inputs/ Rewards

Page. 9 Yusuke Hayashi || 2024.10.26 What is Active Inference? Collective Predictive Coding as Active Inference Hidden states Sensory inputs/ Rewards FEP agent (dog) ① Perceptual inference External world (the dog owner)

Page. 10 Yusuke Hayashi || 2024.10.26 What is Active Inference? Collective Predictive Coding as Active Inference Hidden states Sensory inputs/ Rewards FEP agent (dog) ① Perceptual inference Internal representations External world (the dog owner)

Page. 11 Yusuke Hayashi || 2024.10.26 What is Active Inference? Collective Predictive Coding as Active Inference Hidden states Sensory inputs/ Rewards Perceptual inference FEP agent (dog) ① Perceptual inference Internal representations External world (the dog owner) Parameters

Page. 12 Yusuke Hayashi || 2024.10.26 Internal representations What is Active Inference? Collective Predictive Coding as Active Inference Hidden states Sensory inputs/ Rewards Perceptual inference FEP agent (dog) Action policies ② Active inference External world (the dog owner) Parameters

Page. 13 Yusuke Hayashi || 2024.10.26 What is Active Inference? Perceptual inference Internal representations Sensory inputs/ Rewards Action policies Collective Predictive Coding as Active Inference Hidden states Active inference FEP agent (dog) ② Active inference External world (the dog owner) Parameters Parameters

Page. 14 Yusuke Hayashi || 2024.10.26 What is Active Inference? Perceptual inference Internal representations Sensory inputs/ Rewards Action policies Collective Predictive Coding as Active Inference Hidden states Parameters Parameters Active inference External world (the dog owner) FEP agent (dog) ③ Learning / Parameter optimization Parameters Learning

Page. 15 Yusuke Hayashi || 2024.10.26 What is Active Inference? Collective Predictive Coding as Active Inference ① Perc eptual infere nce The hidden states of the external world are inferred from sensory inputs based on a generative model. In this process, the internal representation that minimizes free energy is selected. ② A ctive in feren ce Through actions, a FEP agent can intervene in the external environment to obtain the desired sensory input. This is known as active inference, where actions are chosen to minimize free energy. ③ Learning / Parameter optimization Alongside perception and action, the generative model itself is also updated. This update is similarly determined to minimize free energy. Free energy/Objective function Expected free energy Calculate the expected value for sequential data. The minimization of free energy and expected free energy drives perception, action, and learning.

Page. 16 Yusuke Hayashi || 2024.10.26 What is Collective Predictive Coding? Collective Predictive Coding as Active Inference Free energy/Objective function Collective free energy/Objective function Note that the collective regularization term cannot be expressed as a sum of terms for individual agents. The global/collective representation acts as an interactive force or bond that unites the entire collective through interactions among individual agents.

Page. 17 Yusuke Hayashi || 2024.10.26 The phase transition phenomena in multi-agent systems We will now finally move on to the latter topic… Sensory inputs/Rewards Hidden states Collective FEP agent Actions Active Inference 🜂 ⇶ ⁂ Ω 🜂 ↺ 〠？ FEP agent 1 (robot dog①) FEP agent 2 (robot dog②) 🜂 ⇶ ⁂ Ω 🜂 ↺ 〠？ FEP agent 3 (robot dog③) FEP agent 4 (robot dog④) Global/Collective representation Collective free energy/Objective function

Page. 18 Yusuke Hayashi || 2024.10.26 The expansion of neurnal population complicates the loss landscape The phase transition phenomena in multi-agent systems (Very small) Neural Networks Neuron 1 Neuron 2 Deep Learning Size of Neuronal population/Community size Complex… The expansion of neuronal population complicates the loss landscape in parameter space, ultimately resulting in a vast number of singularities.

Page. 19 Yusuke Hayashi || 2024.10.26 The expansion of neurnal population complicates the loss landscape The phase transition phenomena in multi-agent systems (Very small) Neural Networks Neuron 1 Neuron 2 Deep Learning Size of Neuronal population/Community size Complex… The expansion of neuronal population complicates the loss landscape in parameter space, ultimately resulting in a vast number of singularities.

Page. 20 Yusuke Hayashi || 2024.10.26 The expansion of community size complicates the loss landscape The phase transition phenomena in multi-agent systems (Very small) Scientific community Size of Community Complex… The expansion of community size complicates the loss landscape in parameter space, ultimately resulting in a vast number of singularities. Large Scientific community

Page. 21 Yusuke Hayashi || 2024.10.26 The phase transition phenomena in multi-agent systems update update Small CPC Models Agent 1 Agent 2 Sample Size Deep Learning Neuronal Population 1 Neuronal Population 2 With the updating of the posterior distribution of parameters, the parameters of the CPC model are discontinuously updated, jumping from one singularity to another. Phase transitions, discontinuous shifts in the paradigm. Large CPC Models update update Agent Population1 Agent Population2

Page. 22 Yusuke Hayashi || 2024.10.26 Summary (Very small) Scientific community Complex… The occurrence of phase transitions, represented by discontinuous shifts in the dominant paradigm within a community. Large Scientific community Size of Community