Gradual disempowerment caused by increasingly capable artificial intelligence could leave people dependent on systems they can no longer understand or control, researchers have warned.
The concern goes beyond the loss of jobs. As AI assistants and robots take on more cognitive and physical tasks, human skills may deteriorate, decision-making could be transferred to machines and citizens may lose influence over the institutions that govern them.
Jay Chooi, founder and chief executive of Robocurve, which tracks and measures AI-driven robotics capabilities, said he placed the risk “on the very top of my list”.
“It’s a civilization-scale problem,” Mr Chooi said, adding that many people felt powerless because addressing it would require “a whole-of-society approach”.
How gradual disempowerment could unfold
The term refers to a scenario in which AI steadily displaces human labour, assumes responsibility for important social decisions and weakens human creativity and cognition in the process.
AI models are expected by some forecasters to become better than people at a widening range of mental tasks, including advanced mathematics, scientific research, programming, writing, art and music. Developments in robotics could also allow machines to outperform people in physical work such as manufacturing, construction, mining and warehouse operations.
The economic pressure to adopt such systems could make human involvement difficult to preserve, even for employers who would prefer to keep their staff. A company that retained people while a rival used cheaper, faster AI systems could find itself unable to compete.
One illustration is a fictional hammer manufacturer whose human workforce makes the products and runs the business. If a competitor introduced robots able to produce hammers more quickly and AI agents able to manage the company with little supervision, the manufacturer might be forced to automate or risk collapse.
Applied across every industry, that process could mean fewer people practising the skills required to keep the economy operating. Over time, knowledge that was once widely held could disappear, while workers became dependent on AI to perform and explain complex tasks.
Margaret Mitchell, an AI research scientist at Hugging Face, said people risk losing abilities when they stop using them.
“If people aren’t actually practicing the skills that the agents are doing for them, they lose the ability to have that skill, and so now you become reliant,” she said. “What happens when something goes wrong? There’s no human you can point to who can fix it.”
Concerns over democracy and state power
The potential consequences extend beyond employment. Military decisions are already being influenced by AI, and the researchers warn that countries could face pressure to automate more of their decision-making if rivals gain an advantage by doing so.
Jan Kulveit, a co-author of the paper on gradual disempowerment and co-founder of ACS Research Group, said states depended on citizens for taxes, military service and the enforcement of laws.
“If, or when, existing states don’t need their citizens, they’re less likely to be democratic,” he said.
Mr Kulveit compared the possible future relationship between AI-run states and their populations with the position of oil-rich states, which he said tended to be authoritarian because they did not rely on most citizens to pay taxes or on their votes to remain in power.
In a society where people were no longer essential to production or defence, citizens could lose a significant source of leverage over those in authority.
Mr Chooi said the danger did not necessarily depend on an AI system rebelling or deliberately seeking control. “AI could literally take over the world just by continuing to do what it’s supposed to do, which is automating everything,” he said.
He added: “Organizations are all going to have a very strong economic pressure, and political pressure, to phase out humans and move work more and more into robots or AI.”
Keeping humans involved
The researchers behind the concept say gradual disempowerment could emerge without a deliberate plan by AI systems. Instead, it could result from the perceived economic, cultural and governmental benefits of automating more activities.
They argue that the first step should be to track how much work is being handed to AI and identify the particular occupations and tasks it can perform. But Ms Mitchell said monitoring alone would not prevent human skills from being lost.
Companies and public bodies could instead design systems that require people to remain involved at important stages. Ms Mitchell described this as “strategic friction”, in contrast to the technology industry’s preference for processes that are as fast and effortless as possible.
“There’s this sense that the less friction the better, but there’s been a push within the more ethical AI community that strategic friction can be good because it can draw people’s attention to critical issues that otherwise they might overlook,” she said.
Mr Kulveit said some areas should be protected from competitive pressure to remove people altogether. AI, he argued, should be built to complement human expertise by explaining its decisions rather than simply producing answers.
Used in that way, AI could help people learn while they worked, rather than making them passive approvers of decisions they no longer understood. The researchers said this would require a focus on how systems support users, not only on their performance in technical tests.
Mr Chooi said the danger could be harder to recognise than more dramatic scenarios involving autonomous weapons or engineered disease. But he maintained that the economic incentives were already straightforward: organisations would continue to adopt systems that made human labour less necessary.
