Simulation Gap

Presented at the Swedish Congress of Philosophy, june 2026

---

Active Inference (AIF) is a comprehensive, normative framework that offers a unified perspective on how living organisms, including humans, interact with their environment. Since it offers a novel way to explain the neural and cognitive processes underlying sentient behavior, it has gained a lot of traction in fields such as neuroscience, cognitive science, and psychology. In essence, it describes organisms as inference engines that minimize prediction error through an internal generative model.

But where do phenomenal experiences fit into all this? While AIF is currently the most structurally ambitious framework linking action, inference, and cognition, it faces an explanatory "Simulation Gap": It characterizes mental states as probabilistic descriptions like "precision-weighted beliefs" without accounting for the first-person nature of phenomenal experiences. Current accounts often identify conscious content with "higher-level posterior beliefs", yet these remain abstract mathematical descriptions rather than explanations of phenomenology. This opens up a gap to the phenomenologically structured experience of consciousness, and a comprehensive theory would need a clear explanation of how this gap is to be bridged.

I argue that this Simulation Gap arises because AIF characterizes beliefs at the level of probabilistic state description without specifying their representational format. One way to bridge this gap is to understand conscious experience as policy-indexed, egocentric generative simulations, thereby giving phenomenal structure to otherwise abstract belief states. That would provide AIF the first-person perspective that objective state descriptions lack and turn AIF into a full-fledged explanation of brain, mind, and consciousness.

Transcript

1. Simulation Gap Andreas Chatzopoulos, 2026 UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
2. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE AIF
3. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE Limitations of AIF Simulation Gap Way forward Outline
4. Brain as a prediction machine UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE PREDICTION PREDICTION ERROR (DISCREPANCY) SENSORY DATA
5. Generative model Used to simulate the hidden causes of sensory data. Probabilistic model, constantly asking: "what world must exist to explain these sensations?" UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
6. Two paths to inference UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE When prediction fails, we change the model. We update our beliefs to better match incoming sensory evidence. Perception When prediction fails, we adjust the world to make our sensations match the predictions. Action
7. Policies Policy = Sequence of actions, a plan for how to manipulate the environment. Policy selection = The selection of a specific action plan. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
8. The Active Inference loop UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
9. AIF and phenomenal experiences UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE Not explicitly treated in most AIF accounts. Only briefly mentioned as tied to policy-conditioned beliefs "what would I expect to experience if I acted this way". Also tied to policy selection that determines which beliefs are afforded high precision and thus come to dominate. The contents of consciousness are the beliefs that "win" this competition under a selected policy.
10. AIF and phenomenal experiences UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE ➔ No exhaustive explanation of a first-person view or phenomenal experiences. ➔ Some accounts talk about the content of consciousness, but does not explain how the experiences of this content are generated by the brain.
11. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE With AIF as a base, would it possible to develop and extended version that would be able to explain these hard-to-explain features?
12. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE First step: Find philosophical theories about the mind that would be compatible with AIF.
13. USEFUL PHILOSOPHICAL THEORIES UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
14. Arises because AIF specifies the generative model and its beliefs at the level of probabilistic state description, without specifying any representational format other than probability distributions. Simulation Gap UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
15. What kind of representational and biological format would allow probabilistic, action- conditioned inference to appear as an immersive, egocentric world of experience? Representational format? UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
16. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE Use a hypothetical model as a first step toward an explanation?
17. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE Focusing on the visual aspect of simulations: Could we find a model that has a similar output as our experienced inner simulations?
18. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE How could we construct such a model? If we were to build an artificial system with a similar visual simulation as output, how would it be implemented? Better yet; can we find an already existing example of how something similar to this might be achieved?
19. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
20. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
21. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
22. Example usage Simulated word in which virtual AI agents can be trained. The agents can interact with the simulated world and learn without access to the real world. UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
23. ...an agent used Genie to generate internal worlds? ...the genie world generation system was an internal part of the agent? What if... UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
24. By watching a lot of video game content, the model has learned action-like latent variables that explain transitions between frames. Given two frames, there must be some cause for whatever changed between them. ➔ It can infer what action will cause which changes in the generated world. Latent Action Model UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE move left frame 1 frame 2
25. ➔ In AIF, actions represents possible ways the sensory stream can be influenced. ➔ In AIF, selected action depends on the most preferred policy, and policy selection means that the system selects among possible action sequences. Difference: In Genie, the sequence is provided by a human user. Functional comparison holds: Different action sequences generate different possible worlds. Actions UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE ➔ Actions corresponding to changes between frames are encoded in latent action space (move left", "move right" etc.) ➔ A user chooses a discrete latent action, and the dynamics model generates the next frame. ➔ Capable of selecting a sequence of latent actions that conditions the future trajectory of the generated world. Genie Compatibility
26. Based on all seen video frames and the learned action outcomes, the model has learned to predict what will happen next in the generated world based on what action is taken. ➔ It can predict how the world unfolds over time. Dynamics Model UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
27. ➔ In AIF, perceptions and actions are encoded in an internal model of how sensory states unfold. Answers the question: "What sensory impression will I get if I perform this action?" Both systems involve counterfactual predictions that represent possible next states. They both have a model of "what would happen if..." World UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE ➔ Action-Controlled world model. ➔ Future video frames depend on action-like input. Answers the question: "Given this current visual state, what should happen next if this action is taken?" Genie Compatibility
28. ➔ The video tokenizer compresses video frames into discrete tokens. ➔ The dynamics model predicts future tokens, and the tokenizer’s decoder turns predicted tokens back into image space. Video tokenizer UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
29. That is, the world model does not operate directly on finished images; it uses an intermediate representational format that can then be predicted over and rendered back into visible scenes. Video tokenizer UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
30. ➔ AIF tells us that perception and action involve probabilistic inference, performed by a generative model. ➔ Genie shows a concrete implementation in which a generative model can use an intermediate representational format to produce a world-like output. ➔ The move from probabilistic prediction to world-like presentation seems to require an representational that works similar to the one found in Genie. That would help us close the Simulation Gap. AIF and Genie UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
31. ➔ AIF does not tell us any details about how immersive simulations are generated from probabilistic models. ➔ We can find philosophical theories that are aligned with the concept of inner simulations, but these require those simulations to be immersive and fully rendered. ➔ Genie gives us a proof-of-concept that could perhaps serve as a model: An action-conditioned generative model that seems to be aligned with AIF and produces fully rendered, explorable scenes as output. Summary UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
32. ➔ If we want to extend AIF into a theory that approaches an explanation of the visual immersive part of phenomenal consciousness, we need to specify not only the probabilistic logic of inference but also the implemented format in which probabilistic predictions become a egocentric, immersive world. ➔ Genie gives us an artificial case in which latent action, temporal prediction, and generative rendering combine into an immersive, explorable world that can perhaps serve as a model for this. Conclusion UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE
33. Thank You! UNIVERSITY OF GOTHENBURG | COLLEGIUM OF COGNITIVE SCIENCE andreas.chatzopoulos@ait.gu.se cognitivescience.se SpeakerDeck: