# AI agent governance checklist > AI agent governance is the set of policies, controls and oversight processes an organisation uses to manage how autonomous AI agents are registered, what they can access, how they behave at runtime and how their actions are recorded, so that every agent stays accountable and auditable throughout its life. **Author:** Roeland Pater **Last reviewed:** 2 July 2026 **Source:** https://deeploy.ai/eu-ai-act-hub/articles/ai-agent-governance-checklist/ The agentic era comes with new challenges: AI agents are being deployed across enterprises faster than security and governance teams can register them. A prototype assistant becomes an embedded business workflow within days, inheriting identities, calling APIs and acting on live data before anyone formally reviews it.  This checklist gives CISOs, Heads of AI and similar roles a practical, vendor-neutral way to bring every agent under control, and to align that control with the EU AI Act. ## Download this checklist as PDF ## **Why AI agent governance is now a board-level concern** Traditional security controls were built for deterministic software and for models that generate content. Agents are different. They reason probabilistically, choose their own tools, and execute multi-step actions across SaaS, cloud and endpoint environments. The exposure sits at the action layer, not the model layer: the risk is less about what an agent says and more about what it does with the permissions it has inherited. Two numbers frame the scale of the problem. Deeploy estimates that by 2028 as many as 1.3 billion AI agents could be operating autonomously across cloud environments. At the same time, organisations already run more agents in production than they can account for, many created without a security review. That combination, high volume plus low visibility, is what security researchers now call AI agent sprawl. The regulatory picture is tightening in parallel. Under the EU AI Act, "we did not know it was running" is not a defence that survives an audit. Breaches of the prohibited-practice rules alone can attract fines of up to €35 million or 7% of total worldwide annual turnover, whichever is higher (Article 99). ### How to use this checklist Work through the ten sections below in order. The first seven cover the security and operational controls you need regardless of jurisdiction. Section 8 maps those controls to specific EU AI Act obligations and the current compliance deadlines. Sections 9 and 10 keep the programme stable as the agent ecosystem changes. Each box is written so it can be assigned to an owner and evidenced during an audit. ## **1. Discover every agent and populate inventory ** No downstream control is enforceable without a reliable inventory. Discovery is the foundation of AI agent governance, and it has to include the agents nobody told you about. - Catalogue every agent in production and in development, with its business purpose recorded. - Capture identity inheritance and data-access paths for each agent. - Include shadow AI: agents spun up inside teams without a security review. - Record the owner, the environment and the source platform (Claude, Microsoft Copilot Studio, GitHub Copilot, Salesforce, ServiceNow, home-grown, and others). - Make registration automatic rather than manual, so the inventory stays current instead of going stale the moment it is written. A living inventory that agents populate themselves at the start of each session removes the spreadsheet-and-follow-up cycle that leaves registers out of date quickly. ## **2. Assign ownership and identity to every agent** An agent without a named owner is an agent nobody is accountable for. Attribution is what turns autonomy into something a governance team can supervise. - Require a defined business owner and team for every registered agent. - Record agent name, description, version, source platform, endpoint and capabilities. - Reuse a single registration across sessions so all traces from one use case stay linked, rather than creating duplicate records. - Ensure that a renamed owner or corrected description updates the registration without breaking the audit trail. - Make every agent action traceable to a defined identity boundary. ## **3. Control access and permission inheritance** Agents inherit permissions from users, creators and service accounts. Without discipline here, a narrow initial scope quietly expands into broad, unbounded authority. - Define delegation explicitly. Agents should operate under clearly scoped authority, not broad inherited roles. - Apply least privilege across every connected system. - Constrain the use of shared credentials and service accounts, which raise escalation risk. - Review effective permissions when an agent gains a new integration, since operational influence grows with each one. ## **4. Trace and log every agent action** Registration proves an agent exists. Tracing proves what it did. A complete, searchable record is what supports investigations, oversight and audit evidence. - Capture structured traces for every registered agent: full inputs and outputs, interaction type, timestamp and session context. - Record environment, model and user context alongside each interaction. - Centralise traces so they can be searched, alerted on and audited from one place. - Retain logs for long enough to meet your sector's audit and incident-reporting requirements. Logging on its own is not governance. When an agent can trigger cross-system workflows in seconds, the record has to be complete enough to reconstruct exactly what happened after the fact. ## **5. Enforce guardrails at runtime** Static, point-in-time controls cannot protect systems that decide and act dynamically. Governance has to operate in real time because agents do. - Evaluate intent, permission scope and operational impact before an action reaches downstream systems. - Block unsafe actions before impact, rather than reporting on them afterwards. - Detect abnormal behaviour patterns and interrupt high-risk execution chains. - Apply consistent policy across every platform, so similar agents are held to the same standard. ## **6. Govern the integration and tool surface** Every connector an agent can reach is part of its attack surface. Integration risk is not static: it grows as teams add tools and use cases. - Map execution paths, tool access and cross-system dependencies for each agent. - Track APIs and external services the agent can invoke. - Review identity providers and service accounts the agent draws credentials from. - Govern the automation and orchestration layers that sit between agents and core systems. - Inventory and monitor emerging protocols such as the Model Context Protocol (MCP), which expand connectivity and introduce new surfaces to watch. Frameworks such as MITRE ATLAS flag these execution paths and tool-invocation points as primary attack vectors, which makes them a sensible reference when you scope this work. ## **7. Centralise oversight across platforms** Fragmentation is the operating reality as enterprises will run several agent platforms at once. When visibility is scattered across separate dashboards, oversight weakens. - Aggregate policy signals, action telemetry and identity mapping into a single view. - Correlate behaviour across environments, so a low-risk action in one system is not missed when it forms a high-impact chain across several. - Govern at the ecosystem level, evaluating identity, intent and behaviour independently of where an agent runs. - Build the control layer to outlast individual vendors, so a new agent framework does not force a governance reset. ## **8. Map agents to EU AI Act obligations** The EU AI Act (Regulation (EU) 2024/1689) does not treat "agents" as a separate category. An agent is governed according to the risk classification of the system it forms part of. For a CISO or Head of AI, the practical work is connecting the controls above to the specific articles an auditor will ask about. - Confirm no agent performs a prohibited practice under Article 5 (in force since 2 February 2025), including the added prohibitions on AI-generated non-consensual intimate imagery and child sexual abuse material. - For high-risk use cases, operate a risk management system (Article 9) and data governance controls (Article 10). - Ensure automatic record-keeping and logging (Article 12) covers agent actions. - Provide effective human oversight (Article 14) with clear escalation and intervention points. - Meet accuracy, robustness and cybersecurity requirements (Article 15). - If you are a deployer, meet Article 26 duties: human oversight, operational monitoring, log retention and incident reporting. - Register qualifying high-risk systems in the EU database (Article 71) before putting them into service. - Run post-market monitoring (Article 72) and report serious incidents within the statutory deadline. - Meet transparency duties for AI-generated content, including deep fakes (Article 50). ## **9. Govern the full agent lifecycle** Governance should begin at deployment, before dependencies form, and continue through every change until retirement. - Apply controls at deployment, not retroactively once an agent is already embedded in a revenue process. - Link every registered agent to risk assessments, approval workflows and controls, using the same framework as your models and applications. - Re-review agents when they gain integrations, change owners or shift purpose. - Retire agents cleanly, with the audit trail preserved. - Reconcile inventories after mergers and acquisitions, which routinely introduce unmanaged agents, and normalise policy across acquired platforms. ## **10. Measure whether governance is working** Governance maturity is demonstrated by a measurable fall in unmonitored autonomy over time, not by the existence of a policy document. - Track the share of agents formally inventoried versus discovered reactively. - Measure time to detect and contain unsafe agent behaviour. - Monitor how often integration scope expands without review. - Record the rate of policy exceptions requested by business units. - Check audit-trace completeness for agent-driven actions. ### **Where Deeploy fits** Sections 1, 2, 4 and 9 of this checklist describe exactly what Deeploy's[agent governance](https://deeploy.ai/product/agent-governance/) is built to do. Its MCP Server lets any MCP-compatible agent register itself at the start of a session, capturing identity, ownership, source platform and metadata directly in your AI registry.  From that point, the agent sends structured traces into Deeploy automatically, and can be linked to the same risk assessments, controls and compliance workflows you already use for models and applications.  The result is one source of truth for every agent, with registration, inventory and audit trails handled as a natural part of how the agents operate rather than as a separate manual process. **If you are building agent governance from the ground up, treat this checklist as an assessment first: score where your organisation stands on each section, then close the gaps in order, starting with discovery and identity. To see how automatic registration, tracing and audit-ready workflows work in practice,[book a demo with Deeploy](https://deeploy.ai/book-demo/).** ## **Download this checklist as PDF** ## Frequently asked questions ## Frequently asked questions ### How is agent governance different from traditional automation governance?  Traditional automation follows fixed logic with predictable execution paths. Agents reason probabilistically, adapt their behaviour and select tools dynamically. Governance therefore has to assess not only what an agent is configured to do, but how it decides to act as conditions change. ### Does the EU AI Act regulate AI agents specifically?  The Act does not name agents as a separate category. An agent is governed according to the risk classification of the system it belongs to. In practice, an agent operating in a high-risk use case inherits the Article 9 to 15 requirements, and its deployer inherits the Article 26 duties. ### Who should own AI agent governance in the enterprise?  It is cross-functional. The CISO or Head of AI usually sponsors it, while operational ownership spans security architecture for policy enforcement, identity governance for delegation boundaries, platform teams for integration oversight, and AI enablement teams for the deployment lifecycle. Clear accountability prevents supervision fragmenting across departments. ### What is AI agent governance? AI agent governance is the structured approach an organisation uses to define how autonomous AI agents operate, what they can access and how their actions are monitored and recorded. It covers registration, identity and access boundaries, runtime controls, audit trails and lifecycle management, so that agents remain accountable enterprise assets rather than unmanaged exposure. ### Does the EU AI Act cover AI agents? Yes. AI agents are not a separate category but fall under the existing definitions of AI systems and GPAI models depending on their architecture. If an agentic system classifies as high-risk, the full Chapter III obligations apply. The AI Office has indicated it is monitoring agentic developments closely and may issue further guidance.