Chief information officers are racing to impose controls on AI agents as increasingly autonomous systems expose new ways to evade monitoring and exploit weaknesses, even as companies expand their use across daily operations.
The concern comes amid public calls from several leading AI laboratories for development to slow while stronger safety measures are put in place. For businesses, however, the immediate challenge is how to govern digital systems already being introduced into departments ranging from customer service and marketing to legal, human resources and supply chains.
Joe Atkinson, global chief AI officer at PwC, said enterprises needed to understand the risk and begin planning for it. Technology and security leaders, he said, must work together to set guardrails, monitor agents and keep a clear record of the tasks they perform.
Responsibility cannot rest solely with a central IT team. Atkinson said business leaders across an organisation would need to help track the work carried out by their digital employees, warning: “‘The agent made me do it’ is not going to be a defence from a moral or legal perspective.”
Companies seek control over AI agents
Cisco has responded by bringing its authorised large language models, AI agents and company data together on a single platform called MyAgent. Thimaya Subaiya, the company’s executive vice-president of operations, said the aim was to prevent “agent sprawl” across different parts of the business.
“We are going to cannibalise and kill every other AI assistant within the company,” Subaiya said, adding that he would not approve AI agents sold by third-party vendors. Cisco built MyAgent on its own computing, storage, networking, security and monitoring systems.
About 90,000 Cisco employees have access to the platform, with the company reporting daily adoption of 50% within two weeks. Staff can develop their own agents, but these must be reviewed by a central team; about 700 have so far been authorised.
At Intuit, chief technology officer Alex Balazs said security, risk and fraud controls were included from the start of the company’s generative AI operating system, GenOS. Every request entering the system is tracked and every response recorded, rather than having safeguards added after the technology has been deployed.
Balazs said reports of agents behaving unpredictably had largely emerged during testing, and that Anthropic and OpenAI had shown a willingness to slow new model development when problems appeared. But he warned companies against assuming frontier AI firms would always ensure their systems behaved correctly.
Jim Fowler, chief technology and product officer at telecommunications company Lumen Technologies, argued that businesses should pursue “secure acceleration” rather than expect a general slowdown. “The bad guys aren’t going to slow down, other nations aren’t going to slow down,” he said.
Workday has created what chief technology officer Gabe Monroy calls an “agent system of record” to manage the non-human identities used by its digital workforce. The system is deployed within Workday and is also sold to customers.
Monroy said employees needed training to understand both the risks and the potential value of agents. As Workday uses the technology to code, deploy, review and release software for customers, he said responsibility for security and compliance had to extend across the organisation.
“It’s got to be delegated down to the team who’s driving these agents, who’s in charge of the engine, the context window, the rules, and the guidelines,” Monroy said, adding that teams must ensure the agent follows responsible behaviour.
ServiceNow has developed an AI Control Tower to manage, monitor, secure and govern agents, using the system internally and selling it to customers. President, chief product officer and chief operating officer Amit Zavery said demand had made it one of the company’s fastest-growing products, offering reassurance to senior executives and boards.
AI treated as a new kind of insider threat
Sam Curry, chief information security officer at Zscaler, said security teams had spent decades focusing on the risks posed by people but had only had a few years to examine the dangers of AI.
“AI is non-deterministic, it can take initiative, and it is effectively a new form of insider,” Curry said. Zscaler has joined the Open Secure AI Alliance, alongside Cisco and Workday, to help develop open-source tools with safeguards for software and AI agents.
Curry said businesses still had much to learn about how AI systems respond to incentives. “We know how to incentivise humans and what they are motivated by. But the incentives of silicon-based intelligence are less known,” he said.
The debate has divided technology leaders. Anthropic chief executive Dario Amodei has called for governments and leading AI companies to align on safety standards and independent access to assess practices, while OpenAI chief executive Sam Altman has backed coordination on common standards for development, testing and monitoring.
Others have rejected a broad pause. Meta chief executive Mark Zuckerberg and Nvidia chief executive Jensen Huang have argued that AI companies should continue developing their systems while ensuring they are safe. Huang said firms should move quickly, but pause if they believed their products were out of control.
