Exploring the Bidirectional Relationship Between Artificial Intelligence and Neuroscience: Proceedings of a Workshop (2024)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2 The Bidirectionality of Neuroscience and Artificial Intelligence Highlights • In many ways, machine learning was inspired by the archi- tecture of the brain, from Rosenblatt’s perceptron model to convolutional neural networks. (Haas, McClelland, Sejnowski) • Cognitive science and neuroscience can inform artificial intel- ligence by revealing insights on natural intelligence, behavior, and brain function, which may necessitate interdisciplinary collaboration. (Patel, Pavlick, Sejnowski) • There is a symbiotic relationship between AI and neuroscience research: knowledge about brain architecture can inspire the design of intelligent machines, while AI can help neuroscien- tists generate theories and design experiments. (DiCarlo, Haas, McClelland, Patel, Pavlick, Sejnowski) • Today’s most powerful AI tools rely on algorithms that both resemble and differ from different than the human brain—these similarities and differences can be informative for both AI and neuroscience. (DiCarlo, McClelland, Patel, Pavlick, Sejnowski) • Simple models capturing the basic learning dynamics of neural networks can provide valuable insight into both the brain and artificial neural networks. (Patel) • Given the brain’s complexity, AI is becoming an essential neu- roscience data processing tool. Using multimodal and mul- 7 PREPUBLICATION COPY—Uncorrected Proofs 

8 THE BIDIRECTIONAL RELATIONSHIP BETWEEN AI AND NEUROSCIENCE tiscale data, such as digital twins, may help generate new insights that may not be accessible otherwise. (Jirsa) • To foster long-term collaboration between AI and neurosci- ence, governments can invest in digital infrastructure and inter- disciplinary research programs. (Jirsa, Pavlick) • Legislators crafting regulations pertaining to the research and use of AI may want to consider the unique relationship between AI and neuroscience. (Cohen) NOTE: This list is the rapporteurs’ summary of points made by the individual speakers identified, and the statements have not been endorsed or verified by the National Academies of Sciences, Engineering, and Medicine. They are not intended to reflect a consensus among workshop participants. Although AI was originally inspired by neuroscience, current AI models both resemble and differ from the human brain, and these similarities and differences are informative to neuroscience and the continued development of AI. Importantly, AI contributes critically to neuroscience research, both as an analytical tool and as a model of neural activity and cognition. Jona- than F. Cohen, professor of neuroscience and psychology at Princeton Uni- versity and codirector of the Princeton Neuroscience Institute, emphasized that as computational work in neuroscience and AI continues to strive for near-human intelligence, it is likely to “fall increasingly within the cross- hairs of regulatory concern.” While research needs to be ethical and safe, Cohen cautioned that the policies and guidelines should be developed to be pragmatic and sensible to not inhibit the development of these technolo- gies. As such, he hoped that the workshop would not only share the crucial contributions of neuroscience to AI but also communicate the importance of pursuing advanced AI for the fields of neuroscience and psychology. THE EFFECTS OF NEUROSCIENCE ON AI Ankit Patel, assistant professor in the Department of Neuroscience at the Baylor College of Medicine and in the Department of Electrical and Computer Engineering at Rice University, opened by remarking on the historical significance of this time, as AI starts to bring science fiction to life. He suggested that neuroscientists, as researchers trying to understand intelligence, should consider what their field can contribute to AI. According to Patel, though it may be counterintuitive, engineering pow- erful AI models no longer requires insight from neuroscience experiments— only more data and processing power. He outlined the main concerns of PREPUBLICATION COPY—Uncorrected Proofs 

BIDIRECTIONALITY OF NEUROSCIENCE AND ARTIFICIAL INTELLIGENCE 9 machine learning and AI engineers, including a model’s specific architec- ture, functions, and hyperparameters and the availability of diverse, high- quality training data. He referenced Rich Sutton’s 2019 essay “The Bitter Lesson,” which states that building human knowledge into AI agents can be limiting in the long run. Computational approaches tend to work better than brain-inspired approaches—a success Sutton describes as “tinged with a bit of bitterness” because it doesn’t center human understanding. Patel proposed that neuroscience experiments aiming to advance AI should pass a higher bar for funding, and grant reviewers should be skepti- cal of proposals claiming that an experiment will advance AI. He urged the neuroscience community to collectively channel efforts toward producing clinically relevant, human-centric knowledge such as deep brain stimula- tion for treatment-resistant depression or obsessive-compulsive disorder. Although theoretical and computational neuroscientists generally lack the training or incentives required to beat benchmarks in AI competitions, they are well-equipped to answer reductionist questions about natural intelligence. For instance, understanding the basic learning dynamics of simple neural networks will enable scientists to someday build algorithms for auditing or explaining the decisions of AI,1 or for tracing errors to their root cause. The basic learning dynamics of neural networks are still largely unknown, and Patel believes neuroscientists can fill in these knowledge gaps. He told the audience, “Imagine what awaits us inside of these black boxes.” Pat Churchland, professor emerita at the University of California San Diego, highlighted that the motor system and motor control might be an area where neuroscience can contribute to AI. In a follow-up to Church- land’s comment, Cohen asked what big questions in AI should be explored by experimental neuroscience. Patel and Ellie Pavlick, assistant professor of computer science at Brown University, both responded that better under- standing the inductive biases humans have evolved will help AI. Patel also suggested that building massive models of the brain called “responsomes,” mapping many different stimuli to the neural responses they elicit, could be used to pretrain AI in areas like vision. Pavlick added that identifying the inherent constraints of the brain, and asking whether those same con- straints exist in machines, will help researchers grasp the potential scope of AI moving forward. Another workshop participant, Josh Gordon, director of the National Institute of Mental Health and chief of the Integrative Neuroscience Section 1 The purpose of auditing AI is to assess the risks of its functionality and governance struc- ture and to recommend measures that can mitigate the identified risks. AI can be audited based on efficacy, robustness and safety, explainability, and bias. For more information, see https:// www.holisticai.com/blog/ai-auditing (accessed July 9, 2024). PREPUBLICATION COPY—Uncorrected Proofs 

10 THE BIDIRECTIONAL RELATIONSHIP BETWEEN AI AND NEUROSCIENCE at the National Institute of Neurological Disorders and Stroke at the time of the workshops, asked whether new findings about the brain’s dynamics are already informing AI. Patel responded that the specific architectural patterns neuroscientists have found in the brain rarely exist in AI, likely because AI didn’t have the same evolutionary or biophysical constraints. Still, he said AI researchers are intrigued by the brain’s ability to learn from fewer examples than current models and believes that studying this process could open the door for more efficient algorithms. THE ROLE OF AI IN COGNITIVE NEUROSCIENCE Pavlick discussed the role of large language models (LLMs) in cogni- tive science. She presented two roles of AI in cognitive neuroscience: as a predictive tool and as an explanatory model. Predictive tools map inputs like neuroimaging data to outputs and can be especially useful for clinical and mental health treatments. Explanatory models help researchers better understand how intelligence works. Pavlick proposed that, although they were not designed for this purpose, LLMs like ChatGPT can be used as explanatory models of cognition (Brown University, 2023. One argument against using LLMs as cognitive models, she said, is that they don’t implement any coherent theory, so their resemblance to human intelligence is difficult to interpret. Moreover, she said that critics argue that no one understands why LLMs work, so they can’t meaning- fully explain another poorly understood system like the human brain. But Pavlick believes that there is a “missed opportunity” if cognitive scientists and neuroscientists dismiss LLMs as irrelevant. LLMs are “objectively good” at simulating human behavior, she said: “There is a reason ChatGPT came on the scene and people woke up.” Despite their flaws, she asserted that it’s worth engaging with them seriously as scientists. Pavlick has found evidence that LLMs are more interpretable and brain-like than previously expected. For example, both the brain and LLMs appear to have modular, functionally specialized units that perform specific tasks (Merullo et al., 2023; Traylor et al., 2024). Pavlick described a “virtuous cycle” between generative AI and cogni- tive neuroscience, wherein cognitive neuroscience and AI inspire hypotheses about how humans behave, and neural and cognitive science in turn pro- vide experimental methods for testing those hypotheses (see Figure 2-1). If experiments reveal that an artificial neural network does a task differently than humans, it can inform new theories about human cognition (as has been the case in linguistics) or suggest ways of improving neural networks. Pavlick concluded by stating that this kind of work takes decades and requires interdisciplinary collaboration. She also highlighted the importance of making models open source so researchers can access model weights and PREPUBLICATION COPY—Uncorrected Proofs 

BIDIRECTIONALITY OF NEUROSCIENCE AND ARTIFICIAL INTELLIGENCE 11 Hypotheses about Forward-engineering, representations, structures, causal interventions mechanisms… (“lesions”) 🧠🧠 Analytical methods (relational similarity, Improved generalization, manifolds…) sample Hypotheses about representations, efficiency, structures, mechanisms… interpretability FIGURE 2-1 Virtuous cycle between generative AI and cognitive neuroscience. SOURCE: Presented by Ellie Pavlick on March 25, 2024. functions. During a later discussion, she emphasized that it isn’t productive to frame the contributions of AI to neuroscience, and vice versa, as trans- actional. Instead, she believes researchers should embrace the convergence of these research programs in interdisciplinary conversations. Sejnowski mentioned that for neuroscientists and engineers to help each other, there needs to be a culture where they can meet. DeepMind, an AI company founded by cognitive neuroscientist Demis Hassabis and acquired by Google in 2014, is one example. The BRAIN (Brain Research Through Advancing Innovative Neurotechnologies®) Initiative is another, he said, in part because the initiative’s U01 Team Science grants require applicants to build multidisciplinary teams. Walter Koroshetz, director of the National Institute of Neurological Disorders and Stroke, asked whether neuroscientists should be applying AI to existing data to iteratively improve models of brain function and whether they should be modeling AI outcomes themselves. Pavlick responded that there is already substantial interest in the latter, but it is occasionally overlooked because the field of AI is so new. Patel suggested that while he appreciates the value of small, clearly defined models, he isn’t sure how well smaller models would work as representations of larger AI models. THE IMPACT OF AI ON NEUROSCIENTIFIC DISCOVERIES As a brain and cognitive scientist, DiCarlo believes that the human mind is an emergent property of a machine (the brain) and can therefore be understood in engineering terms. The job of neuroscientists, he con- tinued, is to uncover the relationship between behavior, cognition, and biophysical interactions within the brain. According to DiCarlo, this goal is nearly identical to the goal of AI: to build machines and supporting theo- PREPUBLICATION COPY—Uncorrected Proofs 

12 THE BIDIRECTIONAL RELATIONSHIP BETWEEN AI AND NEUROSCIENCE ries that explain the mechanisms of human capabilities, especially human intelligence. He discussed the unique role of AI in neuroscience. AI-engineered systems, DiCarlo said, can serve as models that link multiple levels of neuroscientific study, from individual neurons to circuits to behavior and cognition. He used research on human visual intelligence as an example— decades of neuroanatomy work yielded conceptual models that map con- nections between areas in monkeys’ visual system (Rockland and Pandya, 1979; Felleman and Van Essen, 1991) but failed to explain how visual information was being processed. Around 2013, DiCarlo’s group used AI engineering tools to set unknown model parameters that would enable their models to succeed at visual object recognition tasks (Yamins et al., 2014). In time, this led to machine-executable multiscale models of the ventral visual processing stream, where these artificial models were func- tionally aligned with the brain. That is, models derived by this AI-forward route are now the best scientific hypotheses of the integrated set of brain mechanisms underlying a core component of human visual intelligence. He proposed that this case study can be generalized to other domains of neuroscience, including audition and language (see Figure 2-2 and Chapter 3 for examples). “This produces, in each domain, more capable AI systems that also turn out to be better multiscale neuroscientific models of brain function,” DiCarlo said. In visual neuroscience, studies have found that AI-derived models can be used to guide researchers to tiny, indiscernible changes within an image that, when shown to the brain, can dramatically modulate the response of high-level brain neurons in animal tests (Guo et al., 2022). This kind of knowledge, he speculated, could pave the way for new therapeutic treat- ments, where using models to precisely guide noninvasive modulation of neural circuit activity deep in the visual system could have beneficial human health impacts in areas such as anxiety and mood. He concluded by warn- ing that brain and cognitive science departments that do not embrace AI engineering may be left behind. Research institutions, he added, need access to high-performing models to do this kind of research. Cohen asked DiCarlo what, if any, criteria he applies when determin- ing whether an AI system could have neuroscience applications. DiCarlo responded that, unlike neuroscientists, engineers aren’t inherently motivated to model intelligence at multiple scales—producing the desired output from available input is the main goal. But the advantage of multiscale modeling, he said, is that it can guide the development of technologies intervening at each level (e.g., brain-machine interfaces at the neuron level or CRISPR [Clustered Regularly Interspaced Short Palindromic Repeat] interventions at the genetic level, to give two examples). If an AI tool has similar behav- ioral capabilities as the brain and makes the same patterns of mistakes, PREPUBLICATION COPY—Uncorrected Proofs 

The AI-forwardOF BIDIRECTIONALITY strategy behind this NEUROSCIENCE AND vision case example ARTIFICIAL is being INTELLIGENCE 13 successfully generalized to other brain and cognitive science domains Experimental discoveries Theory, principles, & measurements creation & synthesis NATURAL MACHINE EXECUTABLE MODELS AI Neuroscience, SCIENCES OF HUMAN [visual] CAPABILITIES ENGINEERING Computer science, Cognitive science, etc. Robotics, etc. Leading scientific hypotheses Deep architectures of the mechanisms of human and deep learning for [visual] brain function “AI” applications FIGURE 2-2 The AI-forward strategy behind visual system models is being success- fully generalized to other brain and cognitive science domains. SOURCE: Presented by Jim DiCarlo on March 25, 2024. it can be used to generate viable scientific hypotheses for neuroscience to explore and test those kinds of interventions. Deanna Barch, vice dean of research in arts and sciences, Couch Profes- sor of Psychiatry, and professor of radiology at Washington University in St. Louis, asked whether understanding cognitive and emotional processes is a prerequisite for creating accurate models of higher-order association regions in the brain. DiCarlo replied that by pushing a “simulation-forward approach,” one can learn how a given brain region works by first build- ing models that simulate the human cognitive processes that explain the mechanism. He suggested that by building machine systems that aim to achieve and align with human cognitive processes, adding neuroanatomical and neurophysiological measurements, and iterating, the resulting artificial models will increasingly align with human brain function at both the neural and behavioral levels. These scientific models will in turn unlock key goals of neuroscience in health and education, concluded DiCarlo. BRIDGING NEUROSCIENCE AND AI WITH MULTISCALE, MULTIMODAL DATA Viktor Jirsa, director of the Inserm Institut de Neurosciences des Sys- tèmes at Aix-Marseille University, chief science officer of EBRAINS, and lead investigator in the Virtual Brain Twin Project, discussed outcomes of the Human Brain Project. The key outcome of this project, he said, is EBRAINS, a European digital neuroscience infrastructure that aims to integrate neuroscience data across multiple scales (Schirner et al., 2022). According to Jirsa, one major challenge in neuroscience is that with neurodegeneracy, multiple mechanisms can lead to the same outcome (e.g., symptoms), which creates enormous variability across subjects. This vari- ability can cause challenges for clinical translation. His team is exploring a PREPUBLICATION COPY—Uncorrected Proofs 

14 THE BIDIRECTIONAL RELATIONSHIP BETWEEN AI AND NEUROSCIENCE potential solution—creating digital twins, or digital collections of multiscale data that can be used to simulate treatments to make predictions about individual patient outcomes (Amunts et al., 2024). One challenge Jirsa’s team is still addressing is how to map all the different forms of data they have onto the same brain reference space (see Figure 2-3). He mentioned epilepsy as one clinical case where this digital twin framework may be useful (Jirsa et al., 2023). Some treatment-resistant epilepsy patients have electrodes temporarily implanted into their brains to identify where their seizures are coming from. While the electrodes are implanted, Jirsa said, researchers can collect intracranial data from them. But by extrapolating a fuller picture of the brain from that person’s digital twin, one could use machine learning to make predictions about epilepto- genic zones beyond the reach of those electrodes. He presented humanoid robotic hand movement (Goebel et al., in preparation) as another clinical example of fruitful integration of data across modalities. Jirsa concluded by reiterating that fusing data, models, and tools across scales and modalities yields insight that would not be accessible otherwise. Diane Bovenkamp, vice president of scientific affairs at BrightFocus Foundation, asked how people should handle sex-based or cultural biases in algorithms. Jirsa responded that such biases come from biased training data and that they aren’t necessarily harmful if they are acknowledged and taken advantage of. For instance, he said that individual differences in characteristics like age and sex could be used to make more personalized predictions about health outcomes. FIGURE 2-3 Anchoring in the multiscale and multimodal atlas. SOURCE: Presented by Viktor Jirsa on March 25, 2024, from https://www.ebrains. eu/ (accessed July 10, 2024) and contributions by Timo Dickscheid. PREPUBLICATION COPY—Uncorrected Proofs 

BIDIRECTIONALITY OF NEUROSCIENCE AND ARTIFICIAL INTELLIGENCE 15 NAVIGATING THE INTERSECTION OF AI AND NEUROSCIENCE Jay McClelland, Lucie Stern Professor in the Social Sciences, director of the Center for Mind, Brain, Computation and Technology at Stanford University, and consulting research scientist at DeepMind, centered his remarks on the evolution of AI and the role of machine learning in cognitive neuroscience. He opened by clarifying that models are not the final word. Rather, he said, They’re tools for exploring the implications of ideas. When a model goes beyond what has been done before, it sharpens our observations. It allows us to see where our ideas succeed and where they fail. And those failures are the key moments for the advancement of the next question that we need to try to address. While today’s cognitive neuroscience models seem powerful, he said that there is still a lot we don’t understand. McClelland recounted the early days of parallel distributed processing, or PDP (Rumelhart, et al., 1986), when rather than attempting to recreate some biological learning rule, PDP relied on externally defined objective functions. Today’s generative pretrained transformer (GPT) models, like ChatGPT, he said, draw from some PDP ideas and add query-based attention models, which track where things occur in context and make them accessible when questioned. While GPT is a powerful tool, GPT-4 is not open, he said, preventing scientists from analyzing its inner workings or results. McClelland pre- sented one instance that reveals deep underlying differences in how GPT-4 and human brains think: the “reversal curse” (Berglund et al., 2023), in which a model will understand a relationship in one direction (e.g., “Tom Cruise’s mother is Mary Lee Pfeiffer”) but not the other (e.g., failing to identify Mary Lee Pfeiffer’s son as Tom Cruise). While GPT-4 should not be considered an “instantiation of anyone’s belief, at least about the human cognitive system,” studying models like it can inspire new testable hypoth- eses about the brain. McClelland highlighted the importance of investing in research and development in these fields moving forward. DISCUSSION In the panel discussion, Sean Hill, neuroscientist at the University of Toronto and inaugural scientific director of the Krembil Centre for Neu- roinformatics at the Centre for Addiction and Mental Health, asked what a multiscale neuroscience grand challenge for AI could be. Adding to that, Cohen asked what has held funders back from establishing “moonshot programs” for developing large AI models. DiCarlo suggested two possible directions for a moonshot in this field: building integrative models across PREPUBLICATION COPY—Uncorrected Proofs 

16 THE BIDIRECTIONAL RELATIONSHIP BETWEEN AI AND NEUROSCIENCE many areas of cognition at the neural network level or developing multi- scale models of a single sensory system that aim to align at a very precise mechanistic level. In Jirsa’s experience working with brain imaging data, he’s noticed that the data has evolved across multiple scales, and in order to handle the challenges that presents, researchers tend to keep dynamics around experimental variables as stable as possible. One of the biggest chal- lenges in connecting this data to real-world tasks, he said, will be analyzing data in the context of an ever-changing environment. Jirsa also said that there are ethical constraints around gathering enough training data but that the United States should consider creating digital infrastructure like EBRAINS to make existing data and models easier to share. Pavlick added that demand for physical infrastructure and per- sonnel will also pose a challenge. That said, she mentioned, “I don’t think you can take for granted that it’s actually a matter of getting enough GPUs [computer graphics processing units].” First, academics will need to hone in on a goal that all collaborators agree on, as industry leaders like OpenAI have. McClelland agreed and suggested that scientists work together to gain access to computational resources at the scale that corporations have. McClelland asked why industry leaders like Sam Altman, rather than academic leaders like Terry Sejnowski, are more likely to engage directly with Congress and other policymakers. DiCarlo said that most large-scale engineering efforts go on in industry settings, where there are more com- putational resources than academic researchers may have access to. He suggested that if these large-scale industry models were open and available, academic researchers hoping to apply them toward human health and edu- cation could get more involved. Panelists also briefly discussed AI regulation (for a more extensive discussion of AI regulation and policy, see Chapter 6). Historically, Patel said that regulators have feared being perceived as anti-innovation and have waited for crisis points to intervene. With AI, he thinks regulatory bodies should be more proactive. As a first step, he said, the government must decide what these regulatory bodies will look like—AI subdivisions within existing organizations or an entirely new group. Pavlick added that blanket regulation broadly banning systems, which could potentially be misused, would be too restrictive, and that building nuance into regulation will be key. She also believes that AI models should be regulated regardless of whether they are built in a neuroscience or computer science context. Sejnowski and Haas closed the session by reviewing the parallels between AI models and the brain. LLMs have revealed that “we don’t really understand what it means to understand,” Sejnowski said. In other words, models like ChatGPT are both highly effective and bear limited resemblance to the human brain, suggesting that “understanding” does not necessarily depend on the architecture of the brain as scientists currently know it. Haas PREPUBLICATION COPY—Uncorrected Proofs 

BIDIRECTIONALITY OF NEUROSCIENCE AND ARTIFICIAL INTELLIGENCE 17 agreed that when using AI to explain intelligence, a computational system can produce the same output as the human brain without using the same underlying mechanisms. “We’re just creating a mirror,” she said, “but not necessarily solving something that we can understand.” They proposed that studying the simplest possible conceptual models will help neuroscientists fill these gaps in knowledge and generate new theories. PREPUBLICATION COPY—Uncorrected Proofs 

PREPUBLICATION COPY—Uncorrected Proofs