AI agents are being deployed at a pace that most organisations’ governance structures were never designed to handle. Gartner projects the average large enterprise will run more than 150,000 AI agents by 2028. Each of those agents queries internal systems, processes sensitive data, and takes actions on behalf of users, often across multiple platforms simultaneously, and largely without a human reviewing each step.
The readiness gap is stark. Research by Evolvance shows 74% of organisations plan to adopt agentic AI within the next two years, yet only 21% currently have a mature governance model for it. That gap is not simply a compliance concern, it is an operational liability.
Over a third of organisations in the same research admit they could not shut down a rogue AI agent if one emerged. In any other technology context, deploying a system you cannot switch off would be considered unacceptable risk management.
Agents outside of inventory cannot be governed
Before you can govern an agent, you need to know it exists. That sounds obvious, yet only a quarter of organisations have a formal, enterprise-wide strategy for agent identity management.
In practice, agents are spun up by individual teams, shared credentials are reused, and central governance teams are left with incomplete inventories that go stale almost immediately. Agents are processing customer data, accessing internal APIs, and chaining actions across cloud environments with minimal human oversight of each individual step.
When something goes wrong, tracing what happened, and who owned the agent responsible, is extraordinarily difficult without structured registration and logging from the outset.
What ungoverned agents actually cost
The risks are not theoretical. In June 2025, a critical zero-click vulnerability (CVE-2025-32711, “EchoLeak”) in Microsoft 365 Copilot showed exactly what is at stake.
The flaw allowed attackers to automatically exfiltrate sensitive and proprietary information from M365 Copilot’s context, without the user’s awareness or any specific victim behaviour, by sending a single crafted email. Data accessible to Copilot, including Outlook email, OneDrive, SharePoint, and Teams chat history, could be silently extracted after bypassing multiple security layers.
The vulnerability was patched, but the incident exposed a broader truth: AI agents that access rich organisational data, without tight controls and audit trails, are high-value targets.
Beyond security incidents, the regulatory cost of inaction is rising. The EU AI Act will impose penalties of up to €35 million or 7% of global annual turnover for prohibited AI practices. Regulators are not waiting for organisations to catch up. The most significant governance challenge in 2026 is that most existing frameworks were designed for AI systems that assist human decision-making, not systems that make and execute decisions independently.
When an agent takes an action that causes harm, liability is genuinely unclear under current frameworks, which makes internal accountability structures all the more critical.
Governance needs to be structural, not reactive
Gartner predicts that by 2027, 40% of enterprises will demote or decommission autonomous AI agents due to governance gaps identified only after production incidents. That is a costly way to learn.
The organisations that treat agent governance as infrastructure, built into deployment from day one rather than bolted on afterwards, are the ones that will scale AI without the reputational, regulatory, and operational fallout that follows the rest.
That means automatic registration of every agent, structured logging of every interaction, clear ownership, and integration into existing risk and compliance workflows before an incident forces the issue.
Deeploy’s Agent Governance platform is built to make exactly that possible.
About the author: Maarten Stolk is the co-founder and CEO of Deeploy, the leading European AI governance platform, and the co-founder of AI agency Enjins and national AI hub The Stack in Amsterdam.
Sources: Gartner, Evolvance, Hackernews, Strata
Want more? Download our whitepaper.
We dive into this and much more in the complete whitepaper. Co-developed with 12+ industry partners, including Deloitte & BearingPoint.
The question isn’t whether you’ll need robust AI governance. It’s whether you’ll build it in time.
