King Charles has warned AI leaders of “existential dangers” if increasingly powerful technology falls into the wrong hands, urging the industry to establish safeguards before it is “too late”.
The King issued the warning during a meeting at his Dumfries House estate in Scotland, attended by Nvidia chief executive Jensen Huang, Google DeepMind founder Demis Hassabis, OpenAI chief financial officer Sarah Friar and Britain’s artificial intelligence minister, Kanishka Narayan.
Charles said artificial intelligence could improve and save lives, particularly in medicine and the life sciences. But he said its creators were also warning that the technology could develop more dangerous capabilities, “perhaps even to take life”.
“There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” he told the gathering. “Surely, we need sufficient means of control before it is all too late?”
The meeting comes amid growing concern that efforts to make AI safe are failing to keep pace with the development of more powerful systems. Debate has intensified over whether technology companies can regulate themselves, or whether governments should impose controls.
Paolo Benanti, an adviser to Pope Leo XIV who has called for governments to slow AI development, was also expected to attend. Anthropic chief executive Dario Amodei has similarly urged companies to reduce the speed of development, while two of the company’s researchers have warned that rapid advances could lead to human extinction in the not-too-distant future.
Those warnings have prompted an international debate over the motives behind calls for tighter controls. Questions have been raised over whether some executives might seek rules that protect the position of particular companies, head off wider government intervention or reduce spending in the race to develop AI.
No binding agreement is expected from the Scottish meeting. Buckingham Palace said discussions would examine whether shared principles could guide the future use of AI, so that it serves not only as a driver of capability and efficiency but also supports human dignity and the wellbeing of people and the planet.
The King has repeatedly highlighted the environmental risks associated with AI, reflecting his longstanding campaigning on the natural world. His role in convening the discussion continues an approach he used as Prince of Wales and now as monarch, bringing together figures to consider major global issues.
