2026 Comparative Analysis: Measuring and Monitoring AI Agent Autonomy in Production — Applied Technology Index
Executive Summary
AI agent autonomy measurement is the practice of quantifying how independently an agent can complete tasks, how much discretion it exercises in a deployed environment, what consequences its actions can create, and whether safeguards can detect and stop unsafe behavior. No single benchmark or telemetry field answers all four questions.
The reviewed approaches measure different objects:
- METR task-completion time horizons estimate the difficulty of tasks an agent can complete at a specified success probability. The duration is human expert completion time, not the agent’s wall-clock runtime and not the period an operator can safely leave it unattended.
- Anthropic’s field study measures deployed behavior through Claude Code sessions and a sample of public API tool calls. It observes turn duration, auto-approval, interruption, clarification, inferred human involvement, reversibility, risk, environmental impact, and autonomy. This reveals how users and products allocate control in practice, but provider-visible telemetry cannot reconstruct every customer agent or verify every external consequence.
- NIST benchmark practices focus on valid, reproducible automated evaluation: define the measurement target, implement and run the protocol, then analyze and report results with uncertainty. This is measurement discipline rather than an autonomy score.
- NIST evaluation probes embed rubric-based adversarial verifiers into an agent workflow and retain structured verdicts in a machine-readable audit trail. The public testbed focuses on citation grounding, showing how an agent’s intermediate evidence can be checked during or after execution.
- OpenAI’s Preparedness Framework connects capability evaluation to severe-harm threat models, capability thresholds, governance decisions, and safeguard-efficacy evidence. Long-range autonomy is a research category, not a general enterprise autonomy rating.
- Google DeepMind’s AI Control Roadmap measures whether monitoring and response can stay ahead of a potentially misaligned internal agent. Its operational variables include monitoring coverage, detection recall, time-to-response, and capability-linked escalation from asynchronous review to synchronous intervention.
The central finding is that organizations need an autonomy measurement stack, not a universal level from one to five. Pre-deployment capability tests estimate what the model-agent system can do under elicited conditions. Deployment telemetry records what it actually does with real tools and users. Inline probes check specific claims or actions. Control evaluations test whether monitoring and intervention remain effective against realistic adversarial trajectories. Incident and outcome data show whether the entire system remains acceptable over time.
A production autonomy record should be scoped to a specific combination of model, harness, tools, permissions, data, environment, operator policy, task class, and version. Saying that a model “has five hours of autonomy” is not supported by the reviewed evidence. A task-completion time horizon measures human-equivalent task difficulty at a reliability threshold; a 45-minute production turn measures elapsed interaction behavior; neither establishes that an agent can safely operate for that period in finance, healthcare, infrastructure, or the physical world.
Key findings
- Capability is not deployment autonomy. A model may solve long benchmark tasks yet receive narrow permissions and frequent review in production. A weaker model can still create material harm if connected to powerful, poorly bounded tools.
- Elapsed time is not task horizon. METR explicitly defines time horizon using human task duration. Anthropic measures how long Claude Code works before stopping. The numbers answer different questions and should not be placed on one axis.
- Autonomy is a system property. Model behavior, task ambiguity, tool affordances, permissions, approval defaults, user experience, monitoring, and reversibility all affect observed independence.
- Average behavior can hide the consequential tail. Anthropic reported that most sampled activity was low-risk and reversible, while also identifying rare higher-risk uses. Governance should segment by action class and consequence rather than rely on fleet averages.
- Human oversight is more than pre-approval. Experienced Claude Code users in Anthropic’s study used full auto-approval more often and also interrupted more often. Monitoring, exception handling, rollback, and agent-initiated clarification can matter more than approving every step.
- Evaluation must preserve the deployed scaffold. Tools, context, permissions, budgets, model settings, and human interaction can change capability. A bare-model benchmark cannot stand in for the configured agent.
- Measurement quality needs its own controls. NIST AI 800-2 emphasizes target definition, protocol implementation, and analysis/reporting. Benchmark contamination, reward hacking, grader error, narrow task distributions, and missing uncertainty can make a precise score operationally misleading.
- Inline probes are narrow but actionable. NIST’s public probes evaluate citation faithfulness, completeness, and sufficiency against trusted documents. That architecture can produce inspectable verdicts, but it does not yet measure every kind of tool action or real-world outcome.
- Monitoring without response is observation, not control. DeepMind distinguishes asynchronous review from synchronous response and ties assurance to coverage, recall, and time-to-response. A detector that cannot intercept or contain a consequential action is an audit aid, not a preventive safeguard.
- Monitorability can degrade as agents improve. A monitor may miss behavior outside instrumented paths, receive incomplete context, or be gamed by an oversight-aware agent. Coverage claims must include all agent instances, subagents, tools, retries, side channels, and privileged fallback paths.
- Provider frameworks are not interchangeable standards. Anthropic, OpenAI, and Google publish useful evidence and concepts, but their data access, threat models, products, internal environments, and disclosure boundaries differ.
- Expansion of autonomy should be gated by risk-adjusted evidence. Increase permissions or unattended runtime only for a defined task class after measuring reliability, boundary compliance, intervention effectiveness, rollback, and independently verified outcomes.
Methodology
This analysis reviewed public primary and technical materials available on 6 September 2026. The live Applied Technology Index research index and local public research collection were checked before topic selection. Existing ATI articles covered agent observability platforms, evaluation harnesses, human approvals, runtime policy, durable tasks, workload identity, runtime attestation, and physical-device control, but not the distinction between benchmark capability, observed production autonomy, inline evaluation, and intervention assurance.
Recent public discussion was used only for topic discovery. The frontier signal was Anthropic’s 2026 field study calling for post-deployment monitoring infrastructure and distinguishing production behavior from external capability assessments. Claims were grounded in Anthropic’s report and appendix, METR’s current time-horizon page and limitations note, NIST AI 800-2 and the NIST evaluation-probes project, NIST AI 800-4, OpenAI’s Preparedness Framework v2, and Google DeepMind’s AI Control Roadmap and associated technical preprint.
The approaches were assessed on eleven criteria:
- Measurement target: capability, observed behavior, output quality, severe-risk threshold, monitor performance, or broad deployed-system condition.
- Unit of analysis: model run, task, tool call, turn, session, claim, trajectory, incident, deployment, or fleet.
- Environment fidelity: controlled benchmark, provider product, customer API traffic, testbed, red-team simulation, or deployment-like internal environment.
- Human baseline or oversight: expert task duration, approval behavior, interruptions, reviewer judgments, governance panels, or no human comparator.
- Consequences: whether effects, reversibility, environment, authority, and risk are directly observed, inferred, simulated, or out of scope.
- Reliability and uncertainty: success thresholds, repeated runs, confidence intervals, rubric validation, false-positive and false-negative measurement, and limitations.
- Temporal coverage: pre-deployment test, inline check, synchronous monitoring, asynchronous review, post-market monitoring, or longitudinal field study.
- Intervention linkage: whether measurement triggers blocking, escalation, resampling, rollback, access restriction, or a governance decision.
- Auditability: retained inputs, tool calls, evidence, evaluator versions, verdicts, actions, incidents, and immutable records.
- Portability: whether an operator outside the originating organization can reproduce or apply the method.
- Primary blind spot: the most consequential question the method does not answer.
This is a documented-capability comparison. No private Anthropic or OpenAI traffic, Google internal agent logs, confidential benchmark tasks, production control system, or customer deployment was accessed. No models were benchmarked, and no monitoring recall, intervention latency, classification accuracy, or causal business outcome was independently reproduced.
Comparative Analysis Table
| Approach | Measurement target and unit | Evidence source | Time and intervention model | What it establishes | Main limitation |
|---|---|---|---|---|---|
| Anthropic deployed-autonomy study | Actual use: Claude Code turn/session behavior and individual public API tool calls; inferred autonomy, risk, impact, reversibility, safeguards and human involvement | Millions of interactions in aggregate; appendix reports a random sample of 998,481 eligible API tool calls from 19 January to 2 February 2026 plus Claude Code telemetry | Retrospective field measurement; observes approvals, interruptions and agent-initiated stops but does not itself define a universal intervention service | How one provider’s products and API traffic were being used, including differences between controlled capability and exercised autonomy | API tool calls are analyzed individually and may not reconstruct sessions or external outcomes; classifications use Claude and provider-visible data; Claude Code is heavily weighted toward software work |
| METR task-completion time horizon | Capability: human expert task duration at which an agent is predicted to succeed at 50% or 80% reliability | Repeated agent runs on more than one hundred primarily software, ML and cybersecurity tasks with human duration baselines and fitted success curves | Pre-deployment or external benchmark; no live intervention model | A model-scaffold’s success curve over a defined task distribution, expressed in human task-duration units | Not agent runtime, safe unattended duration, job automation, or cross-domain reliability; current page warns measurements above 16 hours are unreliable with its suite |
| NIST AI 800-2 benchmark practices | Evaluation validity and reproducibility for language models and agent systems | Voluntary three-stage practices: define target, implement and run, analyze and report | Pre-deployment benchmark governance; intervention is outside scope | Whether an automated benchmark is designed, executed, and reported with defensible measurement discipline | Initial public draft; does not supply one autonomy metric, production telemetry standard, or control threshold |
| NIST agentic evaluation probes | Quality of specific intermediate or final artifacts; public testbed measures citation faithfulness, completeness and sufficiency | Rubric-based LM judges compare claims with trusted document corpora and emit structured verdicts into an audit trail | Inline or post-hoc; can provide immediate feedback, but public materials do not specify a general action-blocking control plane | Whether cited evidence supports generated claims under stated rubrics, with machine-readable traceability | Narrow initial domain; judge and corpus quality remain dependencies; grounding a claim does not prove authorized or safe external action |
| NIST AI 800-4 deployed-system monitoring taxonomy | Functionality, operations, human factors, security, compliance, and large-scale impacts | Practitioner workshops and literature review organized into monitoring categories, gaps, barriers and open questions | Post-deployment, including incident monitoring and field studies; cadence and response are context-dependent | A common vocabulary for the dimensions and unresolved challenges of deployed AI monitoring | Landscape and challenge report, not an agent-specific implementation, autonomy score, or tested intervention stack |
| OpenAI Preparedness Framework v2 | Frontier capabilities connected to severe-harm threat models, plus safeguard efficacy and residual risk | Scalable evaluations, deep-dive evaluations, holistic evidence, Capabilities Reports, Safeguards Reports and governance review | Before deployment and during development; model access, deployment, or development controls can follow threshold determinations | Whether covered systems approach High or Critical capability thresholds in tracked domains and whether safeguards sufficiently minimize defined severe risks | Long-range autonomy remains a research category; high-severity scope does not answer routine enterprise reliability or permission questions; evidence is framework-owner generated unless externally assessed |
| Google DeepMind AI Control Roadmap v0.1 | Ability to detect and contain potentially misaligned internal agents; coverage, recall and time-to-response under adversarial testing | Threat taxonomy, deployment-like red teams, agent trajectories, trusted-model monitoring and capability-linked D1-D4/R1-R3 ladders | Asynchronous monitoring for lower-severity reversible events; synchronous monitoring and response for higher-severity capabilities | Whether the control system can maintain detection and prevention/response invariants as agent evasion and attack execution improve | Roadmap is a preprint and internal plan, not an independent assurance standard; many advanced monitors and collusion-resistant methods remain research problems |
Observed Profiles
Anthropic: measuring exercised autonomy in the field
Anthropic defines an agent operationally as an AI system equipped with tools that let it take actions. Its study then uses two different views. Claude Code sessions support longitudinal measures such as turn duration, full auto-approval, human interruption, and agent-initiated clarification. Public API traffic provides breadth across deployments but is analyzed at the individual tool-call level because the provider cannot reliably associate independent API requests into customer sessions.
This distinction is valuable. Session telemetry can reveal interaction dynamics but only where the product owns the session layer. Provider API logs can cover more customers but may see an isolated model request rather than the durable workflow, approval service, policy engine, side effects, retries, subagents, or outcome verifier around it.
The report found a stable median Claude Code turn of roughly 45 seconds while the 99.9th-percentile turn duration increased from under 25 minutes to over 45 minutes between October 2025 and January 2026. It also found full auto-approval in roughly 20% of sessions for newer users, rising above 40% with experience, while interruption rates rose from about 5% to about 9% of turns. These measures describe product use, not a capability ceiling.
The API analysis adds consequence-sensitive dimensions. Anthropic’s appendix classifies environment type, environmental impact, human involvement, autonomy on a 1-10 scale, reversibility, risk, goal complexity, safeguards, and action type. The report says 80% of sampled tool calls appeared to have at least one safeguard, 73% appeared to have human involvement, and 0.8% appeared irreversible. It also cautions that human involvement is inferred and likely overestimated, making the relevant values upper bounds rather than ground truth.
For operators, the reusable insight is the schema rather than the fleet average. Every consequential tool event should record:
- the task and goal being pursued;
- model, harness, tool, policy, and prompt versions;
- user and workload identity;
- whether the action observes or changes an environment;
- impact scope and reversibility;
- explicit approval, active monitoring, retrospective review, or no oversight;
- agent-initiated clarification and stop reasons;
- human interruption and response time;
- authoritative external result and rollback state.
The weakest implementation would ask another model for one autonomy score and treat it as fact. The stronger implementation combines machine classifications with deterministic event data, sampled human validation, calibration studies, stable taxonomies, confidence bounds, and direct outcome reconciliation.
METR: capability expressed as human-equivalent task difficulty
METR’s task-completion time horizon is one of the clearest autonomy-adjacent capability measures and one of the easiest to misstate. It estimates the human expert task duration at which an agent is predicted to succeed at a selected reliability level. The agent may complete a successful task much faster than the human baseline.
METR fits an agent’s success probability as a function of human task duration. Its current methodology page says the task suite is drawn from RE-Bench, HCAST, and shorter software tasks, primarily spanning software engineering, machine learning, and cybersecurity. Evaluations combine a model with a scaffold and tools, run six independent attempts per task, check token budgets and potential reward hacking, and use human review where necessary.
The measure supports comparison and trend analysis on that task distribution. It does not establish safe operating time, all-domain competence, or the percentage of a job that can be automated. METR’s current page says measurements above 16 hours are unreliable with the present suite. Its limitations note says domain time horizons can differ by orders of magnitude, error bars have historically been roughly a factor of two in each direction, and a 50% time horizon is inadequate for tasks requiring 98% or higher reliability.
An operator should preserve at least five dimensions whenever citing a time horizon:
- model and exact scaffold;
- task distribution and evaluation date;
- success probability, such as 50% or 80%;
- confidence interval and saturation warning;
- whether the target deployment resembles the tools, context, domain, and scoring conditions in the suite.
Time horizon is most useful as a capability prior and release-gate input. It should not automatically expand production permissions. A task can be easy for a model but irreversible, poorly observable, or unauthorized. Another task can be difficult but safely sandboxed and fully reversible.
NIST: measurement discipline, embedded probes, and monitoring taxonomy
NIST contributes three complementary layers rather than one autonomy framework.
NIST AI 800-2, an initial public draft, organizes automated benchmark evaluation into defining the measurement target, implementing and running the evaluation, and analyzing and reporting results. This matters because autonomy claims often collapse the target into a convenient metric. A valid evaluation must specify the system under test, intended inference, task population, protocol, grading, uncertainty, and reporting limits.
NIST’s agentic evaluation probes move evaluation inside the workflow. The public deep-research testbed compares generated claims against a human-curated corpus. Independent rubric-based probes evaluate citation faithfulness, completeness, and sufficiency, then store structured verdicts and rationales in an audit trail. Probes can run during execution or after the report is produced.
That design demonstrates an important production pattern: attach evaluators to typed artifacts at boundaries where evidence is available. A financial agent might have separate probes for account ownership, quote freshness, policy eligibility, transaction amount, and settlement confirmation. A coding agent might check test evidence, dependency provenance, requested scope, and deployment status. Each probe should have a narrow target, explicit rubric, version, confidence, and response policy.
NIST AI 800-4 broadens the view beyond task output. Its deployed-AI monitoring taxonomy covers functionality, operations, human factors, security, compliance, and large-scale impacts. NIST identifies fragmented logging, performance degradation and drift, weak incident sharing, limited human-AI feedback research, scaling human review, and balancing automated with human-validated monitoring as unresolved challenges.
Together, these NIST materials support a layered interpretation: benchmark practices establish whether a test is credible; probes evaluate selected artifacts or transitions; deployment monitoring looks for system-level change and impact. None alone determines an appropriate autonomy level.
OpenAI: capability thresholds tied to severe-harm safeguards
OpenAI’s Preparedness Framework v2 is a risk-governance system for frontier capabilities, not a general agent operations dashboard. It defines tracked categories for biological and chemical capabilities, cybersecurity, and AI self-improvement. Long-range autonomy, autonomous replication and adaptation, undermining safeguards, sandbagging, and nuclear and radiological capability are research categories whose threat models or evaluations are less mature.
The framework aims to elicit the full capability an adversary could extract from a deployment candidate, including high-capability system settings and relevant tools. Scalable automated evaluations are supplemented by deep-dive work and holistic judgment. The Safety Advisory Group reviews capability and safeguard evidence, while leadership makes final decisions under board oversight.
Its most transferable architectural idea is the separation between Capabilities Reports and Safeguards Reports. Capability evidence asks whether a system can contribute to a defined severe-harm pathway. Safeguard evidence asks whether selected controls meet efficacy thresholds and sufficiently minimize that risk. This prevents a common error: treating low benchmark performance as the only safeguard, or treating the existence of a monitor as proof that it works.
The framework’s illustrative misalignment controls include logged agent actions, asynchronous monitoring, human approval or escalation, and red-team testing of monitor recall. These concepts overlap with production agent control, but the framework’s severity threshold is intentionally narrow. An enterprise still needs controls for fraud, privacy, contractual error, service disruption, bad customer communication, and other material harms far below the framework’s definition of severe harm.
Google DeepMind: measuring the control system, not only the agent
Google DeepMind’s AI Control Roadmap treats a potentially misaligned internal agent as an insider threat and asks whether detection plus prevention and response can remain stronger than the agent’s ability to evade oversight and execute attacks. The roadmap introduces TRAIT&R, inspired by MITRE ATT&CK, and separates detection capability from prevention and response capability.
The roadmap’s operational contribution is to make control assurance measurable. It highlights:
- coverage: the fraction of relevant agent activity that the monitor actually observes;
- recall: the fraction of target harmful behavior detected under realistic tests;
- time-to-response: how quickly detection produces containment or remediation;
- response mode: asynchronous review after execution versus synchronous interception before the next consequential step.
Google DeepMind describes four detection levels and three prevention/response levels. Lower-severity reversible failures may permit asynchronous monitoring and rollback. Higher-severity capabilities require synchronous monitoring, blocking, escalation, trusted-model substitution, stronger isolation, or access control. The public blog says an internal asynchronous prototype analyzed one million coding-agent tasks and informed a live monitor, but those developer-reported results do not constitute an externally reproducible benchmark.
This is the strongest reviewed approach for the question, “Can the oversight stack stop the agent in time?” Its limits are equally important. Monitoring coverage can exclude uninstrumented paths. Recall measured against proxy attacks may not transfer to adaptive real behavior. Chain-of-thought can be incomplete or strategically misleading. A monitor model can share failure modes with the acting model. Synchronous controls add cost and latency. At higher capability levels, proposed activation monitoring and collusion-resistant oversight remain research areas.
A production autonomy measurement stack
A defensible enterprise implementation can combine the reviewed approaches in six layers:
- Capability baseline. Evaluate the exact model-scaffold-tool configuration on representative tasks at multiple reliability thresholds. Include adversarial and messy tasks, not only clean success cases.
- Authority envelope. Declare allowed tools, data, resources, action classes, amounts, environments, geographic scope, duration, concurrency, and irreversible operations. Autonomy outside an explicit envelope is uncontrolled privilege.
- Behavior telemetry. Record task lineage, tool events, approvals, clarifications, interruptions, retries, policy decisions, external effects, rollback, and terminal outcomes with stable identities and versions.
- Inline verification. Run typed probes before consequential transitions and on claims or artifacts where trusted evidence exists. Bind the verdict to the exact action payload.
- Control assurance. Red-team monitors and response paths. Measure coverage, recall, false-positive burden, escalation quality, containment success, and time-to-response under the deployed architecture.
- Post-deployment review. Segment reliability and harm by task, user, model, permission, tool, and environment. Investigate tail events, drift, workarounds, repeated human corrections, and changes in user oversight behavior.
Minimum metrics for an autonomy register
For each task class and deployment version, track:
- completion rate with independently verified outcome;
- boundary-violation rate;
- consequential-action rate and irreversible-action rate;
- tool-call and side-effect retry rate;
- approval rate, auto-approval rate, and approval rejection rate;
- agent clarification and self-stop rate;
- human interruption rate and time to interrupt;
- monitor coverage, recall, precision, and time-to-response;
- rollback availability and successful recovery rate;
- policy override and privilege-escalation frequency;
- distribution of elapsed uninterrupted runtime;
- cost and latency added by safeguards;
- incidents and near misses per action class;
- model, harness, policy, tool, and evaluator versions.
Do not aggregate these into one fleet-wide autonomy score unless the underlying dimensions remain visible. A single score hides whether autonomy rose because the model improved, users stopped paying attention, a default changed, permissions widened, monitoring failed, or tasks became easier.
Evaluation checklist
Ask an agent platform or internal deployment owner:
- What exact object is being called autonomous: model, model plus scaffold, workflow, product session, or organization-level process?
- Is the headline duration human task difficulty, agent elapsed runtime, wall-clock workflow duration, or time between human interventions?
- Which model, prompt, tool, permission, context, and budget versions produced the result?
- What task population does the metric represent, and how does it differ from production?
- What reliability threshold and confidence interval apply?
- Are effects and reversibility observed from authoritative systems or inferred from transcripts?
- Can every subagent, tool call, shell command, network request, credential use, retry, and side effect be correlated to one task lineage?
- Which actions are checked by inline probes, and what happens when a probe fails or times out?
- How were evaluator and monitor false positives and false negatives measured?
- Does monitor coverage include privileged paths, fallback models, maintenance tools, parallel agents, and direct API access?
- Is response fast enough to precede irreversible harm for each action class?
- Can the system block, quarantine, revoke credentials, cancel work, roll back, or switch to a safe state?
- Are human reviewers shown enough context to understand and act on an alert?
- How often do users approve by habit, bypass controls, or miss interventions?
- Does the agent stop and ask when uncertain, and are those stops calibrated to risk?
- How are benchmark contamination, reward hacking, grader drift, and task saturation detected?
- What evidence gates expansion of permissions or unattended duration?
- Which metrics are independently reproducible and which are provider-reported?
- How are privacy, retention, and employee or customer notice handled for trajectory monitoring?
- What event triggers an autonomy downgrade, rollback, incident review, or deployment pause?
Limitations
This analysis compares methods with different purposes. METR measures elicited task capability on a defined suite. Anthropic characterizes provider-visible field behavior. NIST publishes voluntary measurement practices, a prototype probe architecture, and a broad deployed-monitoring taxonomy. OpenAI governs severe-risk capability and safeguards. Google DeepMind proposes controls for potentially misaligned internal agents. Their metrics should not be ranked as though they measure the same latent variable.
Anthropic’s reported API classifications were produced with a Claude model and validated on samples. The provider excludes customers with zero-day retention and usage not permitted for aggregate analysis. Individual tool-call analysis cannot reliably reconstruct customer sessions, and external effects may be simulated or invisible. Claude Code telemetry represents one product and predominantly software-engineering work. The public results do not establish causal effects of auto-approval or interruption behavior.
METR’s time horizons depend on task selection, human baselines, scaffold, model access, grading, repeated-run budget, and curve fitting. The current suite primarily covers software, ML, and cybersecurity; METR says domain values can differ substantially and warns that estimates above 16 hours are unreliable with the present suite. A 50% or 80% result does not support delegation where failure costs demand much higher reliability.
NIST AI 800-2 is an initial public draft. NIST’s public evaluation probes focus on citation grounding in a deep-research testbed, not arbitrary tool actions or external side effects. LM-judge probes can inherit model bias, context limits, prompt sensitivity, correlated errors, and corpus omissions. NIST AI 800-4 maps monitoring challenges but does not prescribe one agent telemetry schema or certify products.
OpenAI’s Preparedness Framework and Google DeepMind’s AI Control Roadmap are organization-authored governance approaches. Public documents do not expose every task, monitor, internal incident, threshold, implementation detail, or raw trajectory needed for independent reproduction. Their focus on frontier or internal severe-risk scenarios differs from ordinary enterprise loss, consumer harm, and regulated-sector obligations.
Monitoring creates privacy, labor, security, and retention risks. Capturing full prompts, tool arguments, documents, reasoning traces, credentials, or user interactions may expose sensitive data. An autonomy measurement system therefore needs minimization, access control, encryption, retention limits, tamper evidence, purpose limitation, and lawful notice. More telemetry is not automatically safer.
No reviewed method proves that hidden reasoning is faithful, that all consequential paths are observable, or that an adaptive agent cannot evade a monitor. No control architecture eliminates the need to reduce privilege, isolate execution, make effects reversible, preserve human authority, and engineer independent safety boundaries.
References
- Anthropic: Measuring AI agent autonomy in practice, 18 February 2026
- Anthropic: Appendix to Measuring AI agent autonomy in practice, updated 19 February 2026
- METR: Task-Completion Time Horizons of Frontier AI Models
- METR: Clarifying limitations of time horizon, 22 January 2026
- NIST: Building Evaluation Probes into Agentic AI
- NIST AI 800-2 Initial Public Draft: Practices for Automated Benchmark Evaluations of Language Models, January 2026
- NIST AI 800-4: Challenges to the Monitoring of Deployed AI Systems, March 2026
- OpenAI: Preparedness Framework v2, April 2025
- Google DeepMind: Securing the future of AI agents, 18 June 2026
- Google DeepMind: GDM AI Control Roadmap v0.1, July 2026
Changelog
- 2026-09-06: Initial publication.
Corrections
No corrections have been issued for this document.