Andriy Onufriyenko via Getty

‘I was seriously wrong’: Experts say superintelligent AI may arrive as soon as next year

AI is speeding up research, potentially creating a runaway loop.

Some forecasters who closely track AI developments have drastically revised predictions this month, saying that “artificial superintelligence” may arrive years earlier than they thought. 

Sentinel, a forecasting group, now expects superhuman AI to arrive around 2028, not 2031. The AI Futures Project, which last year laid out a scenario in which superintelligent AI might arrive in 2027, says development is proceeding faster than the scenario assumed. 

Both groups changed their predictions in response to a Nov. 5 report by METR, a nonprofit that evaluates AI risks. METR found that the speed at which AI models were improving within AI developer Anthropic had roughly doubled since late last year, which it attributed mostly to the extensive use of AI agents to help with research. 

Some experts think AI-driven research could start a runaway loop of AI improving itself, known as recursive self-improvement, a threat they say demands quick action.

“Long, deliberate processes are not going to cut it for the pace at which this is moving,” said Joe O’Brien, a researcher at the Institute for AI Policy and Strategy, a think tank. “This is something that is going to have to be addressed under the current administration, with the incoming Congress. And quickly.” 

‘A big update’

To groups that monitor the progress of AI development, the METR report arrived like a bomb.

“This level of acceleration due to AI is very unexpected,” the AI Futures Project wrote in a Nov. 10 update, adding that AI researchers better than the best human expert will “likely arrive at some point in 2027 (assuming no substantial deliberate slowdown).”

“I was seriously wrong,” wrote METR researcher Ryan Greenblatt, who was not involved in the report, Nov. 5 on X. “This report on AI R&D automation (RSI) is a big update towards ASI [artificial superintelligence] within 1-2 years.”

Not everyone is convinced. Ramez Naam, a computer scientist and futurist who founded the venture capital firm PlanetaryVC, wrote on X that METR’s conclusions are too hard to verify.

“The METR report on progress at Anthropic is missing too much,” Naam wrote. “I completely trust the METR team to go into this with integrity. But we all have different priors and sometimes pull very different conclusions out of the same data.”

The terms of the report’s publication gave METR access to Anthropic’s internal data, but allowed Anthropic to redact any of that data from the public report. This included removing what METR called “a majority of the evidence we found most informative.”

“We redacted a lot, and I accept that this makes it harder for others to check METR’s work,” wrote Sam McCandlish, an Anthropic co-founder, on X. “But we determined that sharing exactly what is driving the speedup isn’t the right thing to do, both because it’s commercially sensitive and because it risks handing out a roadmap for others to go faster. That would make the risks METR is worried about worse. That’s a tradeoff. I think we made the right call, though we’re continuing to assess the pros and cons here.” 

Officials with Anthropic, which raised $81 billion today in a record-breaking initial public offering, did not respond to a request for comment. 

METR declined to comment. 

Still, some of the leading figures in AI say that, missing data or not, this time is different. 

Four days after the METR report came out, Paul Christiano delivered the opening Hinton Lecture in Toronto, where Nobel Prize winner Geoffrey Hinton, one of the “godfathers of AI,” introduced him as “one of the most important researchers working on AI safety.” Christiano is a former head of safety at the federal Center for AI Standards and Innovation and AI Safety Institute, and helped develop reinforcement learning from human feedback, a widely used technique in today’s AI models, while a researcher at OpenAI. 

In his address, Christiano delivered a stark warning. 

“I think there is a high risk of an AI takeover within the next three years,” he said, because AI is speeding up research. “We are seeing what you might expect to see at the beginning of a process of acceleration, or shortly before you saw 10 years of progress compressed into a short period of time,” he said. 

Christiano predicted that the AI system that emerges from this process would surpass human skills in many, perhaps most, of the areas where current systems are weak. He said such an AI could control drones and construction robots, giving it both military hardware and the ability to build infrastructure without human help, or use its hacking skills to spread to servers around the world, preventing attempts to simply “pull the plug.” 

Today’s much less capable AI has already found vulnerabilities in every major operating system and web browser. 

If the pace of AI progress keeps accelerating, “Congress, the U.S. government, and humanity more generally, will probably end up completely disempowered by around the end of 2027. Or maybe early 2028,” said Daniel Kokotajlo, executive director of the AI Futures Project and a former researcher at OpenAI. 

Policy and politics

More than 1,300 AI company employees have signed a statement about the need to slow the pace of AI development and negotiate with China to slow as well. Congress and public opinion suggest a shift toward support for regulation. 

But the White House has resisted pressure to more closely regulate domestic AI, instead rebranding AI as “Super Intelligence” as part of an effort to boost public support for the technology. On Friday, the Pentagon renamed its Chief Digital and Artificial Intelligence Office the Super Intelligence Office. “As the Super Intelligence Office our mission is to harness the full power of sovereign Super Intelligence to ensure decisive military superiority across every domain,” the office said in a press release. It declined to comment further. 

Frank Cilluffo, director of Auburn University’s McCrary Institute for Cyber and Critical Infrastructure Security, said in an October interview with Defense One that policy is falling behind. 

“What we’re thinking about right now will be OBE [overtaken by events] in many ways,” he said. “I would normally say three to five years. Now the cycle is more like six to nine months at best.”

O’Brien, of the Institute for AI Policy and Strategy, said security risks such as the theft of U.S. AI models by major adversaries remain largely unaddressed. 

“We are on track to have systems that are incredibly powerful much faster than we are on track to have the security to prevent these systems from being stolen or sabotaged or misused,” he said. “To the extent that the defense community can dramatically accelerate efforts to develop data center security, model robustness, and various other precursors to safely deploying very powerful AI, now is the time.”