Once a distant ambition for technology researchers, the prospect of artificial intelligence models teaching themselves autonomously to be more efficient and capable appears ever closer to reality.
As the technology advances, developers say it is approaching “recursive self-improvement,” or RSI, in which AI models find ways to improve themselves and build their successor. It could bring the promise of advances in science and medicine, tech company executives say, but also risks.
The uncertainty over where it all could lead is at the heart of growing fears about AI evading human control, and possible threats to humanity, which led several AI moguls to join last weekend in a call to slow down the technology’s pace of growth.
Anthropic this week detailed how its model Claude is helping the company to develop the next, more intelligent version of itself. Claude is now leading 26% of Anthropic’s model research and development, which the company said means it can complete most of a given task “end-to-end from a high-level prompt” while still being under human supervision. The models are not working completely autonomously — at least not yet.
Here are some key points about recursive self-improvement.
Leading AI companies have different definitions for recursive self-improvement. Some define it as when there is any feedback from AI on model improvement, while others define it as AI working toward that goal fully autonomously.
Autonomous recursive self-improvement essentially means AI that can improve itself designing the next version of the system, then the next version, and so on, said Anthony Aguirre, president and CEO of the nonprofit Future of Life Institute and a physics professor at the University of California, Santa Cruz.
“The really important thing here is that as AI is doing more of it, it gets faster, because AI operates just much, much more quickly than the humans do," he said.
The fear around RSI is based largely on a runaway superintelligence emerging from that process, said John Thickstun, an assistant professor of computer science at Cornell University who studies methods that control the behavior of AI models. But he said a more grounded view suggests a kind of recursive self-improvement has been going on in AI development for a while now.
"We have already, for years, been using these models in supportive roles for creating the next version of these models. So people use the past generation of models to write code for the AI systems that then create the next generation," he said.
For years, prominent AI researchers such as OpenAI co-founder Andrej Karpathy have experimented with trying to get AI models to train and improve new AI systems. Those efforts have brought minor improvements, but not big creative leaps, Thickstun said.
But AI companies today, Aguirre said, are much closer to pulling off those bigger leaps in improvement.
“You can see in these plots from Anthropic over time, more and more of research is being done by the AI and it’s becoming closer and closer to fully autonomous,” he said. “And the result of that success, ultimately is something that is, I think, extremely scary. I think this is probably the worst idea in the history of humanity to do this. And yes, they’re doing it.”
Anthropic's recent announcement provided the public — and other labs — with some insight into RSI progress, and it encouraged its competitors to share similar metrics. Still, the company has not expressly said how close it is to achieving fully autonomous model improvement.
ChatGPT maker OpenAI announced this month that it has developed an automated “research intern,” which it defines as a system that can carry out well-defined research tasks under human direction, including “tasks that would take a skilled researcher a few days.” The company has said it is moving forward with the goal of creating an automated AI “researcher” by March 2028.
The company said in that announcement that while RSI can help align models' actions with human values and intentions, that doesn't mean “rapid RSI is necessarily an outcome we should pursue.”
“Whether and how to proceed must depend on our ability to preserve human control and on informed democratic choices about the benefits and risks,” the company said in a blog post.
Elon Musk seems more eager to forge ahead. He said in March that for xAI's Grok models, “humans are gradually getting less and less in the loop” on model improvement and that “every successive model is built by the one before it,” but clarified that the process was not yet fully automated. That target might be reached by the end of this year, he added, “but not later" than 2027.
Microsoft and some other leading AI companies seem to be taking a different approach.
Mustafa Suleyman, the CEO of Microsoft AI, has said the company is moving toward “humanist superintelligence,” or advanced AI capabilities that are in service of people and humanity at large. Suleyman said in a 2025 essay that this would not mean “an unbounded and unlimited entity with high degrees of autonomy,” but rather AI that is “carefully calibrated, contextualized, within limits.”
A key challenge labs face — and have been facing essentially since the technology’s inception — is ensuring their safety measures advance alongside the models' capabilities.
Divisions have emerged in the tech industry over calls for a coordinated AI slowdown for safety, and not every major player in the AI space has specifically commented on their path forward with RSI.
Anthropic, which has been a leading voice in the calls for pacing, has said it would slow or temporarily pause its development work — assuming its global competitors also did so, and in a "verifiable manner.”
OpenAI explicitly said this month it does not yet know how to “safely get all the way to aligned, full RSI,” adding that the company “cannot assume that progress in alignment and safety will keep pace.” More capable systems can become harder to monitor, it continued, but pursuing RSI is still a goal it says it values because an “automated AI researcher can also be an automated safety or alignment researcher.”
FILE - Dario Amodei, CEO and co-founder of Anthropic, attends the annual meeting of the World Economic Forum in Davos, Switzerland, Jan. 23, 2025. (AP Photo/Markus Schreiber, File)
