Emergent Language Reading Group #1 “A Survey on Emergent Language” Section 1-2 Naoto Yoshida Kyoto University Symbol Emergence System Lab. 2024-10-16 

Short Introduction of Emergent Language Reading Group ● A Survey on Emergent Language, 2024 ArXiv ○ Jannik Peters, Constantin Waubert de Puiseau, Hasan Tercan, Arya Gopikrishnan, Gustavo Adolpho Lucas De Carvalho, Christian Bitter, Tobias Meisen ● Comprehensive overview of the emergent communication/language (EmCom) community ● Personal Motivation ○ I want to get an overview of the whole field, outside the lab. 

Main content 

Progress of Emergent Language / Communication ● Research on “Emergent Language / Communication” ● Classical studies: ○ “coordination game” by David Lewis (game theory) ○ Lewis, D.K.: Convention: A Philosophical Study, 1st edn. Harvard Univ. Press, Cambridge, Mass. (1969) ● focused on specific features of the language. ● Recent Advances in Machine Learning ○ Multi-agent reinforcement learning (MARL) ○ agent-agent communication / agent-human communication ○ using Natural Language (NL) style fashion ● Reinforcement Learning (RL) in EC/EL ○ By using RL, we may be able to gain insight into the evolutionary process of NL. ○ Different from NL processing, which is based on statistical understanding and usage, and does not assume the evolution of language or the use of text alone. CLASSICAL simple signaling game hand-crafted simulation (supervised learning, non-situated agents) AFTER DEEP RL Multi-agent reinforcement learning (MARL) from wikipedia Section 1: Introduction 

The (rough) Current State of the Field of EL/EC ● Numerous different methods and metrics are already established. ● but they are complex to structure and important issues remain regarding the analysis and comparison of achieved results. ● Taxonomy for EL that encompasses key concepts and terminology of the field. ● Summary of established and recent metrics. ● Summary of EL research: highlight the achievement. ● Comprehensive and systematic literature review: PRISMA (Page et al. 2021) ● Previous surveys of EL focused only on a subgroup of characteristics or specific parts of EL. ○ specific learning settings, methodological summaries, more general overview. Section 1: Introduction 

2.1 Communication ● Communication ○ Transfer of exchange of signals, which can be interpreted to form some information. ● Signals ○ intended: (deliberate) utterances ○ unintended: uncontrolled bodily reactions deliberate utterance un-deliberate utterance (?) uncontrolled bodily reactions Section 2: Background / 2.1 Communication 

Forms of Communication ● Different forms of Communication ○ Intrapersonal ■ example: Internal talk (内言) ○ Interpersonal ■ Communication between entities that mutually influence each other ■ has a social character and might only relate to a goal or task of one of the participants (?) ○ Group ■ differs only in the number of entities involved and the communication goal ■ more formal and focuses on a common goal or group task ○ Public ■ example: lecture ○ Mass ■ example: blog entries, Youtube? Section 2: Background / 2.1 Communication Intrapersonal Interpersonal Group Public not covered: implicit communication 

Example: Interpersonal Communication Section 2: Background / 2.1 Communication 

2.2 Natural Language (NL) ● NL is a prime example of a versatile and comprehensive form of communication designed to convey meaning. ● The origins and evolution of natural language remain a mystery. ○ It is commonly agreed upon that a very intricate evolutionary process was involved. Section 2: Background / 2.2 Natural Language https://www.history.com/news/humans-evolution-neanderthals-denisovans ○ Related fields ■ Neurolinguistics (neuroscience): the human brain most likely developed specific areas and functionalities specifically for more complex language based communication ■ Linguistics: this evolution can be seen in language development itself, which is a constantly ongoing process ■ Emergent communication (AI): the research of suitable model structures for the processing of language, while concurrently developing and evaluating language. Language as a Dynamic Phenomenon 

Communication Model Section 2: Background / 2.2 Natural Language Semiotic cycle (Bleys 2015, Eecke & Beuls 2020) ● Actual communication process via NL is generally easier to conceptualize ● Speaker and Listener ● Speaker ○ produces an utterance based on the meaning to be conveyed. ○ “meaning” results from the combined conceptualization of the speaker’s goal and model of the world. ● Listener ○ receives the utterance and comprehends it to derive a meaning. ○ Not a direct copy of the initial one by the speaker but it still refers to the shared world. ○ The interpretation of the meaning, which the listener’s world model informs, leads to some action by the listener. (game theoretic / MARL view) ● Shared World ○ shared world and the respective world models of speaker and listener that function as grounding for the information exchange via language. meaning = internal representation (?, imo) 

Major Levels of Linguistic Structure Section 2: Background / 2.2 Natural Language ● The semiotic cycle puts the utterance as an externalized information carrier into focus. ● Utterance is external and available for analysis. ○ Other components are internalized and thus difficult to define and measure. ● Levels of linguistic structure ○ Phonetics (音声学) ■ includes the entire bandwidth of the chosen, often continuous, language channel. ○ Phonology (音韻論) ■ the atomic building blocks of the spoken or written language, defined as phonemes (音素) or graphemes (書記素). ○ Morphology (形態論): ■ create and assign meaning by making words, in linguistics called lexemes (語彙素). ○ Syntax (統語論): ■ concerns the structure of sentences. grammars, word order. ○ Semantics (意味論): ■ the literal meaning of language constructions is of interest. ○ Pragmatics (語用論): ■ focuses on how context contributes to the meaning. Lexemes (single root word) big-bigger-biggest chile-children go-goes-went-gone-going from wikipedia Syntax tree (from wikipedia) 

What is Emergent Language? ● EL refers to a form of communication that develops among artificial agents through interaction, without being explicitly pre-programmed. ● bottom-up approach to (natural or artificial) language ● EL involves the agents creating, adapting, and refining linguistic structures and meanings to enhance their ability to exchange information effectively and efficiently ● It explores how linguistic elements such as syntax, semantics, and pragmatics can arise from the interaction of artificial agents and how these elements contribute to the agents’ performance and cooperation. Section 2: Background / 2.3 Emergent Language “Emergence” Language as a Dynamic Phenomenon Original Focus Language Origin Recent Shift (functional aspect) How to enable agent systems to benefit from a mechanism that helped humanity thrive and how to achieve communication capabilities as close as possible to NL. 

Difference in Comparison to NLP ● NLP (≒ LLMs) ○ MLP models learn language imitation via statistics, but they do not capture the functional aspects and the purpose of communication itself. ● EL/EC ○ EL uses language not as the sole objective but as a means to achieve something with meaning. ■ Cooperation / Task Performance ○ Accordingly, agents have to learn their own EL to enable functionality beyond simple statistical reproduction. ■ Existence of super-human language? (imo) ■ Similar logic to the deep reinforcement learning Section 2: Background / 2.3 Emergent Language “Emergence” 

Challenges of Emergent Language 1. Right incentives for language development is crucial. ○ Encouraging communication alone can lead to simple gibberish that helps with task completion but does not represent the intended natural language characteristics. 2. Question of how to measure successful communication. ○ Important to examine how agents use communication and the opportunity to send and receive information. ○ The measurability of language properties such as syntax, semantics, and pragmatics is also important for assessing the emergence of desirable language properties. Agent Agent 1: How to Motivate Agents? Utterance Speak? Listen? 2: How to measure successful communication? Difference?