Braintree: An agentic framework for in-silico vision neuroscience

Abstract

Vision neuroscience has many of the ingredients needed to conduct experiments entirely in-silico. We need AI systems that can reason over scientific knowledge, generative methods for creating precisely controlled stimuli, image-computable models that can predict neural responses with increasing accuracy, and tools that can analyze and interpret data. Here we take the first step toward making a vision neuroscience experiment end-to-end computable. We introduce Braintree, an agentic framework that can autonomously run multi-experiment studies directly with in-silico models of human visual cortex. Given a hypothesis, Braintree can reason over the current literature, design an experiment, build a stimulus set from algorithmically manipulated or synthesized stimuli, run the experiment on candidate brain encoding models, analyze the responses and then use the results to design even stronger tests. The entire loop runs autonomously and can accept human input at any stage. We evaluated Braintree on three increasingly open-ended scientific problems. First, Braintree reproduced published findings using their original stimuli and then asked whether the same effects could be replicated with entirely new stimulus sets that it constructed. Second, it designed new experiments to find stimulus dimensions that could drive two brain regions differently, even when those regions otherwise had similar response profiles. Third, it searched for experiments that could tell competing models of the same brain region apart. Experiments discovered using just two models also exposed differences across a much broader set of models, suggesting that Braintree had found more general dimensions along which brain models disagree. Braintree can augment the capabilities of a vision neuroscientist as a verifier for existing hypotheses, a test-bed for new ideas, and/or a tool for generating stimuli that can discriminate between competing hypotheses or models of the brain. Braintree shows how AI agents and brain models can work together to make in-silico vision neuroscience experiment computable, allowing vision scientists to explore many more experiment conditions and identify the most informative ones to test on the human brain.