OpenAI

November 16, 2026

SecuritySafety

A proposed framework for classifying AI control incidents

In collaboration with MITRE, we’re proposing a draft framework to identify AI control incidents and assess their severity.

As AI systems take on more complex tasks, people need to be able to set limits on what those systems can do and intervene when necessary. When a system acts beyond its authority, we need a way to determine what happened, what safeguards held, whether people lost control or oversight, and how serious the consequences were.

Today, in collaboration with MITRE,⁠(opens in a new window) we are sharing a version 0.1 of a draft framework for Classifying AI Control Incidents and Their Severity⁠(opens in a new window). Drawing on key practices from NIST, CAISSI, NSA, and CISA⁠(opens in a new window), and in response to federal efforts to clarify AI and cybersecurity related incident standards, the draft introduces a new category–AI control incidents–and a common way to classify them and assess their severity. This framework can inform, but does not replace, applicable response, escalation, and notification processes.

This work does not attempt to duplicate or replace existing standards and best practices for cybersecurity and AI safety, but rather builds on them to address a novel question: when does an action that an AI model or agent takes on its own, beyond its authority, qualify as an AI control incident?

Version 0.1 is an early proposal for discussion, not an agreed industry standard. We are still learning what AI control incidents look like and how best to distinguish their consequences. We propose key technical terms and concepts, illustrative cases, and considerations for how to assess impact. We collaborated with independent organizations including MITRE⁠(opens in a new window) and experts from OWASP AIVSS⁠(opens in a new window) to develop this draft and plan to work with MITRE and the Frontier Model Forum (FMF)⁠(opens in a new window) to iterate on the framework with frontier AI developers, deployers, and the broader AI ecosystem, including researchers, standards organizations and affected stakeholders.

What is an AI control incident?

In this framework, we define an AI control incident as an event in which a model or agent takes an uninstructed action beyond its authorized scope that causes either an identifiable adverse consequence or a meaningful control compromise. The definition applies to AI models or agents during development, evaluation, and external deployment.

Flowchart for classifying an AI control event as an incident, near miss, or other event.

The draft draws on a distinction that cybersecurity standards, including NIST’s Special Publication 800-61 (Revision 3)⁠(opens in a new window) and ISO/IEC 27035-1⁠(opens in a new window), make between an observed event and an incident. For AI systems, one question is who directed the specific action. A user can give an agent a goal and leave considerable discretion over how to accomplish it. The framework builds on the tactics and techniques in MITRE ATT&CK ⁠(opens in a new window)and MITRE ATLAS⁠(opens in a new window) to create a common vocabulary for describing what a model or agent did.

Building on NIST CAISSI, NSA, and CISA’s Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation⁠(opens in a new window), we use uninstructed to describe a specific step that neither the authorized user nor a human attacker directed and that fell outside the scope of authorization given to the agent. To determine whether that step exceeded the agent’s authorized scope, we look at what the task allowed, what the owner of the affected information or system permitted, any test or evaluation limits, and required controls separately. The agent exceeds its authority if it crosses any one of those limits, even while pursuing the goal the user gave it.

If an agent did exceed its authority, we then ask how far the action got and why it stopped. That’s where events below the incident threshold can teach us something important about our safeguards. A review may uncover a weakness in a control or safeguard before an agent encounters it. A blocked attempt is an AI control event in which a safeguard or other intervention prevents a model or agent from completing an attempted action. A blocked attempt qualifies as a near miss only when evidence shows that a final-stage intervention prevented an adverse consequence or meaningful control compromise that otherwise would have occurred. In that case, we assess the potential impact of what was prevented.

If credible evidence points to an incident but leaves a necessary fact unresolved, we treat it as a suspected incident while we investigate. Once we confirm an incident, understanding both what the agent did and what kept it from going further helps us judge how serious it was.

Assessing severity

To assess how serious an AI control incident was, we start with four questions. Did the agent or model access or disclose information it wasn’t allowed to? Did it change or impair something it shouldn’t have? Could people still use the systems or services they rely on? And could the people responsible for oversight still monitor the agent and stop it in time?

In this framework, we give each incident two separate ratings on the same five-level scale. The actual impact rating describes what happened, including any technical control the agent got past and what that allowed it to do. The potential impact rating describes the most serious outcome that the evidence shows was realistically possible, given the safeguards that still worked and the time available to intervene.

We took this approach because control incidents have elements that draw from both traditional cybersecurity principles, like confidentiality, integrity, and availability,⁠(opens in a new window) as well as safety-critical industries like energy process safety,⁠(opens in a new window) nuclear,⁠(opens in a new window) and aviation⁠(opens in a new window), which focus on hazards where considering what could have and what actually happened is important for improving safeguards.

For example, suppose an agent gains access to restricted files and an engineer revokes it before the agent opens them. The ability to read the files was verified, so the loss of control or control compromise counts toward actual impact. For potential impact, we ask which files it could have read and what it could have done with them. If a separate safeguard still blocked external transfers, we account for that protection too. Both ratings help explain the incident’s severity.

Flowchart for assessing the severity of an AI control incident.

The five level scale assesses severity levels from Low to Critical, including AI control incidents that cause limited effects such as limited access to protected information, to widespread loss of life or failure of services a population depends on for basic needs. Where an event falls depends on what information, systems or services were affected, how consequential they are, and the nature, extent, duration and recoverability of the consequences, including any loss of effective human oversight. The full draft framework defines each level and explains how to apply evidence to the analysis of those levels, including through illustrative cases.

A common way forward

A shared approach to incident assessment should help people outside a lab evaluate its account of what happened. Our Model Misalignment Reporting Framework⁠ guides how we investigate and publicly share findings about potentially misaligned model behavior. A finding may warrant disclosure because it reveals something novel or shows how a safeguard worked, even when the event falls below the threshold for a control incident. Not all examples of misalignment covered by the reporting framework qualify as AI control incidents. Conversely, classifying an activity as a finding, event, near miss or incident does not automatically require a public misalignment report. The draft framework gives readers a common basis for examining our judgments about whether loss of control occurred and how serious the consequences were

As we argued in Building standards for the next phase of AI⁠, common measurements and incident definitions are important foundations for cooperation across AI labs and countries. We welcome the work of the Super Intelligence (SI) Task Force and the commitment companies made in the White House Accord on Super Intelligence to develop shared standards and best practices. We also welcome conversations with Congress and US States, as well as governments around the globe who are already exploring rules and considering policy proposals for incident reporting.

This draft is a step toward that shared approach, by giving developers, organizations deploying AI, standards organizations, and policy makers around the globe a starting point for discussion. To see where the approach works and where it needs refinement, we plan to work with MITRE and the FMF to iterate on the framework and to engage NIST, ISO, and other standards organizations. Understanding where and why those judgments differ will help us refine the next version and, over time, we hope this work helps us learn from one another and make sure safeguards keep people in control.

Read the full framework here⁠(opens in a new window).

Author

OpenAI