A preliminary roadmap for AI-assisted science

How will language-based agents accelerate science in the next few years?

I am building a community to think about the ideas below. If you are excited about them, get in touch! Contact details at the bottom.

The goal of biology research is to cure disease, allow us to live long and healthy lives, feed the planet, and fight climate change. However, as I have written before, progress in biology today is too slow for us to accomplish most of these goals within our lifetimes. How can we make it go faster?

AI-powered agents offer the best opportunity we have had in decades to accelerate the pace of research in biology and throughout science. The impact of AI-powered agents on science will be at least as large as the genome, or significantly larger. In this essay, I will provide a preliminary answer to the question of what impact these agents will have on science, and what technical challenges we need to overcome to realize that impact.

AI agents will fully automate many in silico tasks.

Automating science is hard because science is about doing things for the first time. Robots can automate almost anything that needs to be done over and over again in exactly the same way, but things that are being done for the first time by definition are not being done over and over again. It is commonly held fallacy that biologists have a small set of protocols like PCR that are done over and over with parametric modifications; in reality, most biology research consists of coming up with the protocols in the first place, and new protocols are almost never amenable to automation by old robots. I have automated three wet lab protocols to date, and each time, it took getting the protocol to work by hand first, and then 3-4 months of adapting it to work with some custom-assembled robot.

Large Language Models are potentially revolutionary for science because they are capable of doing things they have never done before. Language-based agents can thus, in principle, execute many in silico scientific tasks themselves. It is fully conceivable that, in the near future, one will be able to give high level instructions to a Large Science Model (“please clone mScarlet into a pAAV CAG GFP backbone for me”) and have the model execute it in seconds. Even high-level analysis tasks could in principle be automated (“please research all the ways of doing ChIP-seq analysis, and implement the most promising approaches.”) In the coming years, we will be able to expand the in silico scientific workforce to a size that is limited only by compute budget.

AIs that do their own research will raise major questions for biosecurity and AI safety. Certainly, an AI that can develop a cloning protocol for mScarlet can in principle provide a cloning protocol for deadly toxins; and could in principle also work out how to circumvent controls on producing those toxins. It will be important to integrate brakes and safety features into new foundation models, and it will also be important to ensure that newly developed capabilities are deployed for surveillance (e.g. at the point of chemical/DNA/protein synthesis) before they are deployed for investigation.

AI agents will also assist human researchers in the lab.

Although they will not be able to do laboratory experiments themselves, in silico agents will also be able to directly support researchers in the lab, for example by helping researchers to take lab notes and by providing real-time tracking and feedback as necessary. An AI hooked up to laboratory sensors might be able to provide feedback about the differences in how two researchers execute a given protocol, for example. The data that agents gather about human experimentation will also greatly facilitate the automation of the hands-on work of wet labs once we have laboratory robots that are themselves zero-shot capable (i.e., that are capable of doing tasks they have never done before).

Much of the time that is wasted in science today comes from inefficient information transfer. We often spend 3-6 months doing an experiment or setting up an assay only to discover that it doesn’t work well, or isn’t right for our purpose. Usually, there is some researcher somewhere who could tell us in advance, but finding that person and getting in touch with them is usually hard. Having an AI assistant that has read the entire literature and that has assisted all of your colleagues with their experiments will be like having your colleagues in the room as you plan your experiment. Moreover, if that AI has access to detailed knowledge about what happened in your colleagues’ experiments, it may be able to provide detailed and actionable critiques about how to make the experiment work better.

AI agents for science will require precision planning, data-efficient learning, and dedicated tools.

To realize these agents, three primary technical challenges will need to be overcome. Firstly, we will need to create agents that can reliably predict the next action that needs to be taken to achieve a specific objective. Many AI shops are undoubtedly working on a solution to the general version of this problem, which will be necessary in order for an AI to do mundane tasks like booking flights reliably. However, the challenge for science is probably much harder than in other domains, because the scientific literature is sparse (there is not usually a ton of data on what other people have tried in a given circumstance), and because small details matter. (The difference between “add 1mL of HCl” and “add 1L of HCl” is critical, for example.) Thus, building robust agents for scientific next-step prediction will likely require new metrics for distinguishing correct steps from incorrect steps.

Secondly, data-efficient retrieval will be critical here. Once we have appropriate metrics, agents will need to be able to learn from their mistakes, both by instruction and observation. When an experiment goes wrong, you don’t get to see it go wrong 100 times; you only get to see it go wrong once. Memory capabilities for language-based agents are developing rapidly, and will be a key component in the science agents of the future.

Finally, we will need to give language models access to a suite of tools that are sufficient to implement the protocols they come up with. These tools may take the form of APIs, like those that are being developed today to allow language models to book hotels for you. However, as mentioned above, science is about doing things for the first time: it isn’t clear to me whether there is a set of tools that will exhaustively cover all of the things we may want an AI assistant to do. It will likely be necessary to allow language models to make their own tools, which may raise further safety questions.

We need integrated teams of AI researchers and scientists

Most AI researchers I talk to have little if any conception of how biologists spend their time or what challenges they face; and most biologists have little if any idea of what is possible or impossible using AI. AI researchers who are exposed to the nitty gritty details of how biology actually works often get scared away by the complexity, and biology researchers exposed to the details of how AI works often conclude that it cannot be trusted. We will need teams of hard-core AI researchers and hard-core scientists working together, with a rapid iteration cycle, in order to build tools that leverage the cutting edge and that actually add value for the scientists. Moreover, as Large Science Models are developed, it will be essential that they remain focused on specific, concrete tasks that are relevant to science, rather than on high-level generalities.

The hype levels right now are high, and most of the demos flying around on twitter today are brittle. It seems inevitable that the hype bubble will burst as people discover that language models cannot actually solve all of science within the coming year. But science 5 years from now will be vastly more productive than it is today. It is time now to begin working on what the laboratory of the future will look like: how we can make it as productive as possible, and how we can make sure that it remains safe and controllable in the years to come.

Contact: SAM dot RODRIQUES at gmail dot com.