Shiro Takagi @ IJCAI 2024 AI4Research Workshop Speculative Exploration on the Concept of Artificial Agents Conducting Autonomous Research

From Tool to Agent From Tool to Agent

Numerous Applications of AI in Academic Research AI for Science Applications to natural & social sciences, engineering, etc. Physics Informed ML, SciML, Symbolic Regression, Experimental Design, GFlowNet, etc. Automated Theorem Proving, AutoML, Scholarly Document Processing, Laboratory Automation, etc. [Wang et al. 2023] 3/31

From AI as a Tool for Science to AI as a Scientist Many AI for Science projects aim to create AI that assists human science → AI as a tool for human research I’m interested in machines that conduct research themselves →AI agent that conducts research (AI researcher) Ques tion Hyp othsis Verifi cation Report/ O bserve Report/ O bserve Verifi cation Hyp othsis Ques tion VS 4/31

What Will We Need to Create? What Will We Need to Create?

AI agent that conducts research 6/31

Definition of Science/(Academict) Research...? No universal definition... Activities employing the scientific method? Falsification?paradigm?research program? Systematic knowledge? Something requiring broader and deeper knowledge and specialized skills? Even scientists don’t have a consensus Few discussion on the common features across broader academic fields like science, mathematics, humanities, etc., or between theoretical and experimental research [Charmars 1999, Hepburn & Andersen 2021, Okasha 2002, Laudan 1987, Hansson 2021] [Hepburn & Andersen 2021] [Charmars 1999] [Hoyningen-Huene 2013] [Haack 2003] 7/31

Provisional definition In current research practices, merely doing what has already been done does not seem to be considered research all research seem to produce some formal knowledge 　→All research necessitates producing new knowledge? Theorems & proofs in math/theoretical research Algorithms and blueprints in engineering research Verification results in experimental research Text interpretations in the humanities Temporarily defining research as producing new knowledge and aiming to create AI that autonomously does this...? 8/31

What is Knowledge? What is Knowledge?

AI agent that produces new knowledge 10/31

Knowledge from the Perspective of Epistemology In epistemology, which is a branch of philosophy that studies knowledge, discussions on what constitutes knowledge have evolved from the assertion that “knowledge is justified true belief” to more refined perspectives e.g. knowledge is a belief formed through a reliable process, knowledge is a success through virtue, knowledge is... Acknowledging that it is imperfect, but as a first step, let us consider knowledge as justified true belief for simplicity →Generating knowledge is forming & justifying beliefs? [Steup & Neta 2024] 11/31

Knowledge for AI is not necessarily that for humans since knowledge is belief and belief depends on knowing subjects AI needs to understand and generate knowledge that is new to not AI itself but to humans AI should provide justifications that update not its own beliefs but human beliefs →Alignment problem between AI researcher and human AI needs to give justification autonomously Still no consensus on what is justification →No clear guideline for AI researcher design Implication from the Definition 12/31

AI agent that produces new knowledge AI agent that produces new knowledge for human society 13/31

Rsearch as Posing and Aswering Questions Rsearch as Posing and Aswering Questions

AI agent that produces new knowledge for human society 15/31

Characterizing Research by Focusing on the “Unknown” Premise: Knowledge must be new To produce knowledge, we must identify the unknown and decide to clarify it = question generation (1) →Research is an activity that generates questions and tries to answer them? Humans split answering questions into two steps: Since the answer is unknown, make a “prediction” of the answer = hypothesis generation (2) Since the answer is unknown, verify that it is truly the answer = hypothesis testing (3) →Human Research = (1) + (2) + (3) ? 16/31

Implication from the Characterization: Alignment Problem Again What is unknown depends on the knowing subject Question for AI isn’t necessarily that for humans The answer for the question may be apparent for AI Hypothesis for humans doesn’t need to be that for AI AI may not need to verify it because they know it Implications for AI researcher design: We may not need to create an AI that do things important in human research AI researcher may not need curiosity (for themselves) We may not need to study how to have AI generate hypotheses as humans do? 17/31

That Being Said... Much of what we don't know is still unknown to AI as well →After all, AI still may need to pose questions, generate hypotheses, and test hypotheses Actual research process is complex e.g. Humans repeatedly engage in trial and error to formulate new questions, generate hypotheses, and validate those hypotheses to generate a single hypothesis →In the end, it may come down to the general problem of how to create a general agent that effectively reduces uncertainty to achieve a goal 18/31

AI that Conducts AI Researh AI that Conducts AI Researh

Complete automation is challenging Huge impact for a society Automation of research in other fields Self- improve Discover problem Self-contained within computers Technical solutions for A I safety Why AI Research? 20/31

[Aschenbrenner 2024] Automated AI Research Would Trigger Intelligence Explosion 21/31

Primilinary Attempt Automated execution of whole process of the research from hypothesis generation to verification, while making the system generate hypothesis in free form [Takagi et al. 2023] 22/31

23/31

No content

Conclusion Conclusion

Take-Away Message We need more discussion about what research is There is no consensus on what research is and my discussion is based on provisional definition Alignment problem between human and AI researchers exists AI research automation would be the first priority 26/31

[Takagi 2023] 27/31

I am looking for comrades to develop an AI that can conduct research autonomously! Let’s work together! X account: @takagi_shiro 28/31

Thank you! Thank you!

References References

Wang et al. Scientific Discovery in the Age of Artificial Intelligence, 2023 Charmars, What is This Thing Called Science?, 1999 Hepburn & Andersen, Scientific Method: The Stanford Encyclopedia of Philosophy, 2021 Okasha, Philosophy of Science: A Very Short Introduction, 2002 Laudan, The Demise of the Demarcation Problem, 1983 Hansson, Science and Pseudo-Science: The Stanford Encyclopedia of Philosophy, 2021 Hoyningen-Huene, Systematicity: The Nature of Science, 2008 Haack, Defending Science: Within Reason, 2003 Steup & Neta, Epistemology: The Stanford Encyclopedia of Philosophy, 2024 Aschenbrenner 2024 Situational Awareness - The Decade Ahead Takagi et al. Towards Autonomous Hypothesis Verification via Language Models with Minimal Guidance, 2023 Baek et al, 2024, ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models Wang et al, 2023, SCIMON : Scientific Inspiration Machines Optimized for Novelty Gu & Krenn, 2024, Generation and human-expert evaluation of interesting research ideas using knowledge graphs and large language model Viswanathan et al, 2023, Prompt2Model: Generating Deployable Models from Natural Language Instructions Yang et al, 2024, AutoMMLab: Automatically Generating Deployable Models from Language Instructions for Computer Vision Tasks Qian et al, 2023, MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation Ilfargan et al, 2024, Autonomous LLM-Driven Research from Data to Human-Verifiable Research Papers Takagi, 2023, Speculative Exploration on the Concept of Artificial Agents Conducting Autonomous Research 31/31