2026 Comparative Analysis: Admission Control, Rate Limiting, and Backpressure for AI Agents — Applied Technology Index
Executive Summary
Admission control for AI agents decides whether a model or tool operation may start now, should wait, must use a smaller allocation, or should be rejected. Rate limiting enforces capacity over a time window. Backpressure carries overload information upstream so planners, queues, and users reduce demand instead of multiplying retries. These are related controls, but they are not synonyms.
Agent workloads make ordinary API limits harder to manage. One user request can fan out into planner calls, parallel subagents, retrieval, tool search, code execution, validation, and retries. A single long-context request can consume more token capacity than hundreds of short calls. A 429 can mean a temporary request-rate constraint, token exhaustion, a burst or acceleration limit, a rolling spend-rate control, or a monthly cap that will not recover through exponential backoff.
This analysis compares five documented operating profiles:
- OpenAI API rate limits expose request, token, daily, image, audio, project, long-context, shared-model, and Batch queue constraints. Response headers can report limits, remaining capacity, reset timing, and
Retry-Afterfor temporary rate-limit errors. - Anthropic Claude API limits separate requests per minute, input tokens per minute, and output tokens per minute. They use a token-bucket model, distinguish organization and optional workspace controls, account for prompt-cache reads explicitly, publish a programmatic Rate Limits API, and document acceleration limits and spend-cap error differences.
- Google Gemini API limits evaluate requests per minute, input tokens per minute, requests per day, model-specific dimensions, and rolling spend-rate limits at the project level. Active limits are account-specific in AI Studio, and published capacity is not a guaranteed minimum.
- Amazon Bedrock quotas are AWS account, Region, endpoint, model, and operation dependent. Model inference is controlled by token quotas, while
bedrock-runtimeandbedrock-mantlehave separate per-model allocations. Service Quotas provides the account control plane rather than one universal public rate table. - Cloudflare AI Gateway controls add an operator-owned gateway layer before providers. Its documented rate limiter is request-count based, uses fixed or sliding windows, applies uniformly to a gateway, and rejects excess traffic with
429; a separate spend-limit feature supplies cost controls.
The central finding is that provider limits are supply constraints, not an agent scheduling policy. They protect shared infrastructure and accounts, but they generally do not know which tenant is entitled to capacity, which task is urgent, whether a branch is speculative, whether an external side effect is safe to retry, or how much budget remains for the parent workflow. Production systems need their own admission controller before provider calls.
A strong control plane reserves capacity across at least five dimensions: requests, estimated input tokens, maximum or predicted output tokens, concurrent work, and money. It assigns those reservations to a user, tenant, agent run, task class, model pool, and deadline. It then reconciles reservations against actual usage, propagates wait or reject decisions to the planner, and prevents automatic SDK retries from combining with application retries into a retry storm.
The safest response to overload is usually not immediate provider failover. A fallback model may have different context limits, tool behavior, safety properties, prices, and independent or shared quotas. The system should first classify the limit, honor provider reset guidance, reduce speculative parallelism, queue work within a deadline, and preserve idempotency. Route to another model only when the candidate is approved for the task and the operation can be repeated safely.
Key Findings
- The narrowest exhausted bucket wins. Available requests do not imply available tokens; available tokens do not imply spend headroom; project capacity does not imply a particular tenant should receive it.
- A
429is an envelope, not a diagnosis. OpenAI warns thatRetry-Afterdoes not make quota or billing errors retryable. Anthropic documents both temporary rate-limit429s and spend-cap429s withoutretry-after, with a structured error code for the latter. Gemini can use429 RESOURCE_EXHAUSTEDfor rolling spend-rate controls as well as rate limits. - Limit scope differs materially. OpenAI documents organization and project scopes, not user scope. Anthropic enforces organization limits and supports lower workspace limits. Gemini applies limits per project, not per API key. Bedrock quotas vary by AWS account, Region, endpoint, operation, and model. Cloudflare’s default limiter applies uniformly to a gateway.
- Token accounting is provider-specific. Anthropic separates input and output limits and excludes cache-read input tokens from ITPM for most current models. OpenAI exposes token-capacity headers and documents estimation behavior. Gemini names input TPM as a standard dimension. Bedrock documents model-specific token burndown rates that can consume quota faster than raw token counts suggest.
- Burst behavior cannot be inferred from a per-minute number. Anthropic explicitly notes that 60 RPM may be enforced as one request per second and uses continuously replenished token buckets. Cloudflare exposes fixed and sliding request windows. Provider-internal burst controls may reject traffic before a naïve minute counter predicts exhaustion.
- Failed retries consume scarce capacity. OpenAI states that unsuccessful requests contribute to per-minute limits. Layering an SDK’s automatic retry with gateway and workflow retry loops can amplify one overload event into many calls.
- Spend and throughput are separate control planes. A rate limit bounds traffic velocity; a spend limit bounds money. Both are required because one expensive long-context request and many cheap short requests stress different resources.
- Batch queues need independent admission. OpenAI counts pending Batch input tokens against a model queue limit. Gemini documents separate Batch limits including enqueued tokens per model. Anthropic limits Message Batches by endpoint RPM and processing-queue entries. Moving work to Batch changes the capacity pool; it does not make capacity unlimited.
- Prompt caching changes effective throughput, not authorization. Anthropic’s cache-aware ITPM can materially increase document-processing capacity, but cached context still requires tenant isolation, freshness controls, and spend accounting.
- Gateway limits can protect an account but still be unfair. A gateway-wide request limit can let one noisy tenant consume all admitted traffic unless a second scheduler enforces tenant, user, task, and priority allocations.
- Backpressure must reach the planner. If only the HTTP client sees a
429, the agent may keep creating subagents or alternative tool calls. The runtime needs a typed capacity result that can pause, narrow, defer, or cancel branches. - Capacity figures are not service-level guarantees. Anthropic describes limits as maximum allowed usage, not guaranteed minimums. Google similarly states that specified Gemini limits are not guaranteed and actual capacity may vary.
Methodology
This analysis reviewed primary technical documentation available on 29 August 2026. Current public discussion was used only to identify rate limiting, retries, and gateway backpressure as an active production-agent concern. Capability claims are grounded in provider documentation from OpenAI, Anthropic, Google, AWS, and Cloudflare.
The approaches were assessed on twelve criteria:
- Scope: organization, project, workspace, AWS account, Region, endpoint, model, gateway, user, or agent run.
- Meter: requests, input tokens, output tokens, total tokens, images, audio, queued tokens, concurrent jobs, or spend.
- Window: continuously replenished bucket, fixed window, sliding window, daily reset, rolling spend window, or monthly cap.
- Discovery: dashboard, account console, response headers, programmatic API, Service Quotas, or static configuration.
- Error semantics: status code, structured error detail, exceeded dimension, reset time, and retry guidance.
- Burst handling: short-interval enforcement, acceleration limits, request shaping, or gateway-window behavior.
- Caching interaction: whether cache reads, cache writes, or estimated input affect token capacity.
- Batch isolation: whether asynchronous work has separate queues, limits, or enqueued-token accounting.
- Cost control: provider spend cap, rolling spend-rate control, project or workspace budget, or gateway limit.
- Retry safety:
Retry-After, automatic SDK retries, jitter, bounded attempts, and permanent-error classification. - Agent suitability: tenant fairness, task priority, parent-child budget allocation, fan-out control, and side-effect safety.
- Portability: provider-specific headers and quotas versus an operator-owned admission envelope.
This is a documented-capability and architecture analysis. No production accounts were provisioned and no common burst, concurrency, token, cache, Batch, or failover workload was executed. Current account-specific quota values were not compared because model access, usage tier, geography, Region, endpoint, account history, and negotiated terms can change them.
The comparison intentionally includes four provider control planes and one gateway control plane. Cloudflare AI Gateway is not presented as a substitute for provider quotas. It represents the additional layer an operator can configure before requests reach one or more providers. Application queues, workflow engines, and Kubernetes autoscaling are discussed as architecture patterns rather than ranked products.
Comparative Analysis Table
| Approach | Scope and primary meters | Backpressure and observability | Cost and batch controls | Best fit | Main limitation |
|---|---|---|---|---|---|
| OpenAI API | Organization and project; model-dependent RPM, RPD, TPM, TPD, image and audio dimensions; some shared model-family pools and separate long-context limits | Temporary 429; optional Retry-After; headers expose request, token, project-token, remaining, and reset values; official SDKs automatically retry eligible errors | Approved monthly usage is separate from configurable spend limits; Batch has a pending-input-token queue limit per model | Applications that can schedule from explicit remaining/reset headers and separate interactive from Batch traffic | Limits are not user-level; shared pools, SDK retries, estimated token reservations, and account-specific limits require local normalization |
| Anthropic Claude API | Organization, model, and optionally workspace; RPM, ITPM, OTPM; continuously replenished token bucket; separate Managed Agents and Files limits | 429 identifies the exceeded limit with retry-after; detailed request/input/output headers; acceleration limits can reject sharp traffic increases; Rate Limits API reads configured controls | Monthly tier cap and operator-set spend limits are separate; Message Batches has its own RPM and processing-queue limits | Teams needing explicit input/output capacity, workspace partitioning, and cache-aware throughput planning | Maximum limits are not guaranteed capacity; cached-token rules, cloud routes, model classes, and spend-cap status semantics vary |
| Google Gemini API | Project, model, and tier; typically RPM, input TPM, RPD, with model-specific IPM or TPD dimensions | Exceeding any dimension returns a rate error; active values are viewed in AI Studio; RPD resets at midnight Pacific; published capacity may vary | Rolling ten-minute spend-rate limits can return 429 RESOURCE_EXHAUSTED; Batch has separate concurrency, storage, and enqueued-token limits | Google AI Studio and Gemini applications using project-level capacity and separate Batch pools | Project scope is coarser than agent or tenant scope; current limits are account-specific; static limits are not a capacity guarantee |
| Amazon Bedrock | AWS account and Region, with endpoint-, operation-, and model-dependent quotas; token usage controls inference; model-specific token burndown may apply | Quotas are inspected through Service Quotas and AWS reference pages; throttling must be correlated with model, Region, endpoint, and operation | Quota increases use AWS processes; provisioned or alternative inference arrangements may change capacity planning | AWS estates that need regional service-quota governance and model access through Bedrock | No single portable limit envelope; quota names, adjustability, defaults, endpoint pools, model multipliers, and Regions create substantial configuration complexity |
| Cloudflare AI Gateway | Gateway-wide request count over a configured interval; fixed or sliding window; applied uniformly to all requests for that gateway | Excess traffic returns 429 Too Many Requests and is not processed; configuration is available through dashboard or API | Spend limits are a separate gateway feature; rate controls can sit before multiple provider endpoints | Operator-owned coarse protection against runaway request volume, suspicious traffic, and bill shock | Request-count limiting does not model tokens, output expansion, task priority, tenant fairness, concurrency, or provider-specific remaining capacity by itself |
Observed Profiles
OpenAI: rich headers around organization and project pools
OpenAI documents multiple simultaneous rate dimensions: requests per minute and day, tokens per minute and day, images per minute, and audio-minute rates for some streaming audio models. A request is admitted only while every applicable dimension has room. A low-token workload can exhaust RPM; a small number of long-context calls can exhaust TPM first.
The scope matters for multi-tenant agents. OpenAI describes rate limits at organization and project level rather than user level. Some model families share a rate pool, so switching between models named in the same pool does not create new capacity. Long-context requests can also have a separate rate limit. An operator therefore needs a live model-pool map rather than assuming every model identifier is an independent bucket.
The response-header contract is comparatively useful for scheduling. OpenAI can return request and token limits, remaining values, and reset durations, plus project-token equivalents when a project-scoped constraint applies. Retry-After gives a minimum wait for a temporary rate-limit error when present. The documentation explicitly cautions that this header does not mean quota, billing, or other action-required errors become retryable.
Retry ownership is an operational hazard. Official OpenAI SDKs automatically retry eligible rate-limit failures and honor Retry-After. If an agent framework, workflow engine, service mesh, and gateway each add another loop, one logical attempt can expand multiplicatively. The application should designate one retry owner, inspect the SDK configuration, cap both attempts and elapsed time, and attach every attempt to the same logical operation ID.
OpenAI also states that unsuccessful requests consume per-minute capacity. Immediate repetition can deepen the overload. For agent systems, the correct response is to stop admitting new speculative branches, reserve a delayed retry if its deadline allows, and release or reassign capacity only after the scheduler records the outcome.
Token reservation deserves special treatment. OpenAI documents that rate-limit calculation can use the greater of the configured maximum output and an estimate based on request size. Overstated output ceilings can therefore reduce apparent throughput before those tokens are generated. Schedulers should choose realistic output bounds without setting them so low that valid results truncate.
Batch work is a separate pool with a queue measured by total pending input tokens for a model. That allows non-urgent work to avoid interactive request limits, but the queue itself must be governed. A parent agent should not convert every delayed task into Batch work without checking queue headroom, completion deadline, data-retention requirements, and whether the operation is valid in Batch.
Anthropic Claude: separate input and output buckets with cache-aware accounting
Anthropic distinguishes spend limits from rate limits and measures Messages API throughput using RPM, input tokens per minute, and output tokens per minute for each model class. It documents a token-bucket algorithm whose capacity replenishes continuously rather than resetting only at fixed boundaries. It also warns that a nominal per-minute rate may be enforced over shorter intervals, so a sudden synchronized burst can fail even when a minute-level average appears safe.
The input-output split gives an admission controller more information. A document-analysis workload can be input-bound while a generation-heavy workload is output-bound. A scheduler can pack both classes more efficiently than if it tracks only a combined token number, provided it maintains conservative output forecasts and reacts to the most restrictive header.
Prompt caching directly affects the input meter. For most documented Claude models, uncached input and cache-creation tokens count toward ITPM while cache-read tokens do not. Anthropic’s example shows that a 2 million ITPM limit with an 80 percent cache-hit rate can process 10 million total input tokens per minute because 8 million are cache reads. This is a provider-published accounting example, not a universal performance guarantee.
The distinction is still useful for architecture. Repeated system instructions, tool definitions, long documents, and conversation prefixes can consume much less rate capacity after a cache hit. Admission should reserve uncached-token capacity based on expected cache state, then reconcile against actual usage. A cache miss must not be allowed to oversubscribe the input bucket simply because the planner expected a hit.
Anthropic documents acceleration limits in addition to ordinary buckets. A sharp increase in organization traffic can produce 429 responses even when an application’s static understanding of a tier suggests room. Gradual ramp-up and consistent traffic patterns are therefore part of capacity engineering, especially after a launch, region shift, or queue replay.
Workspace limits provide a useful internal partition. An organization can set lower request, input-token, output-token, and spend constraints for non-default workspaces while organization-wide ceilings remain authoritative. This can protect production traffic from an evaluation or batch workspace. It is not a complete per-user scheduler, but it provides a stronger provider-native tenant boundary than a single organization pool.
Error classification is unusually important. Temporary rate-limit errors include retry-after. Reaching the tier’s monthly spend cap also returns 429 with a rate_limit_error, but Anthropic documents no retry-after and supplies error.details.error_code: enforced_spend_limit_reached; retrying will fail until access resumes or the limit changes. An operator-configured spend limit can instead return HTTP 400. Status code alone is therefore insufficient.
Managed Agents endpoints have separate organization-level limits for create and read operations, independent of Messages API limits. An agent control plane should not let aggressive session polling consume the same conceptual budget as useful model work, and it should track control-plane reads separately even when the provider already isolates those API pools.
Google Gemini: project quotas plus a rolling spend-rate constraint
The Gemini API commonly evaluates requests per minute, input tokens per minute, and requests per day. Any exhausted dimension can reject the request. Limits apply per project rather than per API key, so creating more keys does not create capacity or isolate tenants. Requests-per-day quotas reset at midnight Pacific time, while model-specific dimensions such as images per minute or tokens per day can also apply.
Google ties active limits to model, usage tier, account status, and other factors, and directs users to AI Studio for current values. The documentation states that specified limits are not guaranteed and actual capacity may vary. A scheduler should consequently treat configured maxima as ceilings and observed availability as a changing signal, not reserve one hundred percent of the published number for latency-sensitive traffic.
Gemini adds a cost-velocity dimension: spend-based rate limits are evaluated on a rolling ten-minute window for applicable paid tiers. A workload can stay under RPM and TPM while expensive requests exceed the spend rate. The documented response is 429 RESOURCE_EXHAUSTED, with guidance to wait, reduce expensive request rate, or request an increase.
That design demonstrates why token quotas are not cost budgets. Model price, modality, context, output length, and tool use can change spend per call. A model gateway should estimate dollars before admission and reconcile actual billed usage after completion, while keeping provider spend-rate state distinct from long-horizon tenant budgets.
Gemini’s Batch API has separate limits from interactive calls. Google documents concurrency, input-file size, storage, and enqueued tokens per model. For agents, the enqueued-token ledger should be allocated by priority and deadline. Otherwise a low-priority bulk run can occupy the entire queue before an urgent offline evaluation or document job arrives.
Amazon Bedrock: regional service quotas and model-specific token burndown
Amazon Bedrock expresses capacity through AWS Service Quotas and service documentation rather than one provider-wide header table. Quotas depend on AWS account, Region, model, operation, and inference endpoint. The main quota page states that model inference is controlled by token-usage quotas and that some models consume quota tokens at a higher rate.
Bedrock documents separate allocations for bedrock-runtime and bedrock-mantle, even when they reach the same underlying model. Operators must therefore tag every request with endpoint as well as model and Region. Aggregating all Bedrock traffic into one generic token graph can hide the constrained pool.
Token burndown is a particularly important normalization problem. For models with a burndown multiplier, quota consumption can exceed the raw input or output token count. Capacity planning should store provider-adjusted quota units beside actual model tokens and money. Otherwise a multi-provider gateway may believe two workloads are equivalent when one drains its Bedrock quota faster.
AWS notes that default quotas can change based on regional factors, payment history, fraud controls, or approved increase requests. Service Quotas should be polled or exported into configuration management, and deployment reviews should verify which quotas are adjustable, which values are current in each Region, and which model or endpoint a name actually covers.
A regional fallback is not automatically safe. Moving an agent request can change data residency, network latency, model availability, quota pool, and policy. Cross-Region or alternate-endpoint capacity should be declared in advance as an approved route with explicit identity, retention, and failure semantics rather than improvised after throttling.
Cloudflare AI Gateway: operator-controlled request windows before providers
Cloudflare AI Gateway documents a gateway-wide limiter defined as a number of requests over a period. Operators select a fixed or sliding window. Fixed windows allow boundary bursts: traffic immediately before and after a reset can exceed the apparent rate over a short interval. Sliding windows enforce the number observed across the trailing interval and therefore smooth that boundary behavior.
When the configured limit is exceeded, the gateway returns 429 Too Many Requests and does not process the request. The same default behavior applies uniformly to all requests for that gateway. This is useful as a coarse circuit breaker against runaway loops, suspicious activity, and request-volume bill shock.
Uniformity is also the limitation. Ten short classifier calls and ten million-token analyses each count as ten requests under a request-only rule. A high-priority production agent and an internal experiment compete equally if they share the same gateway. Teams need multiple gateways, dynamic routes, or an upstream admission service when policies vary by tenant, model, task, token forecast, or business priority.
Cloudflare documents spend limits separately. This separation is architecturally correct: request velocity and cost exposure are different. Neither control alone is enough. A complete operator layer should combine request windows, token forecasts, provider remaining-capacity signals, spend ceilings, concurrency, and per-run budgets.
The gateway should not blindly retry its own 429. A rejection before provider processing means the operation did not reach the model, but the parent task can still miss its deadline or be duplicated by another worker. The response should become a typed scheduling event with a not-before time, reason, and accountable quota owner.
Architecture and Evaluation Guidance
Use hierarchical reservations instead of reactive retries
Before a model operation starts, reserve capacity in this order:
- Business task: verify deadline, priority, tenant entitlement, risk class, and remaining workflow budget.
- Agent run: reserve maximum parallel branches, model calls, tool calls, and dollars for the parent and children.
- Gateway pool: check request window, estimated tokens, concurrency, spend velocity, and queue depth.
- Provider pool: select the exact organization or account, project or workspace, model family, endpoint, and Region.
- Operation: reserve estimated input, cache-write risk, predicted or maximum output, and retry allowance.
A reservation is not the same as actual usage. Reconcile when the call completes, fails before admission, is cancelled, or reaches a terminal unknown state. Expire abandoned reservations and preserve enough evidence to prevent a delayed response from releasing the same capacity twice.
Normalize provider signals without flattening their meaning
A portable capacity record should contain:
- provider, account or organization, project or workspace, Region, endpoint, model, and shared-pool ID;
- request limit, remaining count, reset or replenishment estimate, and observation time;
- input, output, combined, cached, image, audio, and provider-adjusted token units where available;
- rolling spend headroom, monthly spend headroom, and tenant or workflow budget;
- Batch queue entries and enqueued tokens;
- current concurrency, queue depth, and oldest wait time;
- provider error type, structured error code,
Retry-After, and request ID; - whether the error is temporary, action-required, permanent for this route, or ambiguous;
- source of truth: response header, provider API, console export, configured gateway rule, or local estimate.
Do not invent precision. A missing header is unknown capacity, not unlimited capacity. A provider maximum is a ceiling, not guaranteed availability. A rounded remaining-token value should remain marked as rounded.
Make backpressure a planner-visible protocol
Convert low-level errors into one of a small number of runtime decisions:
- admit: capacity is reserved and execution may start;
- queue: work may wait until a declared not-before time within its deadline;
- narrow: reduce subagent count, context, output ceiling, retrieval breadth, or optional verification passes;
- reroute: use a pre-approved model pool whose capability, policy, and quota are compatible;
- defer to batch: move non-interactive work into a separately admitted queue;
- reject: no safe path meets deadline, budget, or policy;
- operator action required: billing, quota increase, credentials, region access, or contract change is needed.
The model can be informed that resources are constrained, but it should not control the authoritative counters. Keep reservation and release logic outside the prompt and outside model-generated code.
Bound fan-out before it begins
A planner that can spawn children should receive a capacity lease, not direct access to the full organization pool. Divide the parent allocation among branches and return unused capacity. Enforce a maximum tree width, depth, active requests, and retry budget.
Speculative branches should be the first work shed under pressure. Required validation should not be silently removed to preserve drafting capacity. Label task stages as required, optional, or replaceable, then let the scheduler degrade in a policy-defined order.
Retry only when identity and side effects are safe
A retry policy should require all of the following:
- the error is classified as temporary;
- the wait fits the task deadline;
- provider and local retry budgets remain;
- the SDK is not already performing an equivalent retry;
- the operation is idempotent or carries a valid idempotency key;
- any tool side effect is known not to have completed, or reconciliation can determine its state;
- the same request will not violate a revoked permission or stale approval;
- jitter prevents synchronized replay.
Model inference without external tools may be repeatable but not identical. A second generation can choose different actions. If the first request may have created tool calls or external effects before the connection failed, treat the outcome as ambiguous and reconcile rather than regenerate blindly.
Separate fairness from utilization
A high-utilization scheduler can still starve small tenants. Use weighted fair queues or reserved shares for tenant and task classes, then allow unused reservations to be borrowed with revocable priority. Preserve capacity for health checks, approvals, cancellation, and incident response so overload does not block the controls needed to recover.
Track both provider utilization and business service levels. Useful metrics include admitted-task rate, queue delay, deadline misses, rejected tasks, tokens reserved versus consumed, cache-hit forecast error, retry amplification, 429 rate by cause, cost per accepted completion, tenant starvation time, and capacity lost to abandoned reservations.
Test a failure matrix, not only steady throughput
A production evaluation should cover:
- a synchronized launch burst below the published minute average;
- one tenant consuming most RPM while another consumes most TPM;
- a long-context cache miss when a hit was forecast;
- output expansion beyond the predicted token reservation;
- a provider
429with validRetry-After, without it, and with a spend-cap error code; - overlapping SDK, gateway, and workflow retry policies;
- shared model-family quotas during fallback;
- a Batch queue filled by low-priority work;
- region or endpoint failover with a separate quota pool;
- parent cancellation while children wait for capacity;
- provider headers that are absent, delayed, rounded, or inconsistent with local counters;
- a quota change while workers still hold reservations;
- an external side effect followed by a lost model response;
- one gateway-wide limiter shared by tenants with different entitlements;
- traffic ramp-up that triggers an acceleration control.
The strongest design is not the one that achieves the highest burst throughput. It is the one that preserves priority, policy, deadlines, and side-effect correctness while returning to stable operation without a retry storm.
Procurement checklist
Ask each provider or gateway vendor:
- What are all applicable request, token, image, audio, concurrency, queue, and spend dimensions?
- At which account, organization, project, workspace, key, Region, endpoint, model, or gateway scope is each dimension enforced?
- Which models share pools, and which alternate routes have genuinely independent capacity?
- How are cache reads, cache writes, maximum output, and provider-specific token multipliers counted?
- Are limits fixed windows, sliding windows, token buckets, rolling spend windows, daily resets, or undisclosed controls?
- Which values can be read programmatically, and how stale or rounded can they be?
- How are temporary throttles distinguished from monthly caps, billing failures, abuse controls, and account restrictions?
- Which SDK errors are retried automatically, with what attempts, timeout, and jitter?
- What Batch or asynchronous queues exist, and how are queued entries or tokens metered?
- Can lower limits be assigned to projects, workspaces, tenants, users, or service accounts?
- Are published limits guaranteed, best-effort maxima, or subject to dynamic capacity?
- What evidence identifies the exact pool charged by a request?
- How quickly do approved quota changes propagate to headers, APIs, and enforcement?
- What happens to streaming requests when output capacity becomes constrained?
- Can traces and usage records be exported for reconciliation and capacity forecasting?
Limitations
This analysis reflects public documentation available on 29 August 2026. Model names, usage tiers, spend caps, quota dimensions, header fields, account qualification rules, cloud routes, gateway features, and Batch limits can change. Organizations should inspect their own consoles and APIs before using any example value for capacity planning.
No common workload was run. The article does not measure actual burst tolerance, sustained throughput, throttling precision, header accuracy, queue delay, cache-hit behavior, retry recovery, regional availability, model latency, gateway overhead, or cost. It does not establish that one provider offers more usable capacity than another.
The compared scopes differ. OpenAI, Anthropic, Gemini, and Bedrock controls protect provider capacity and customer accounts. Cloudflare AI Gateway provides an operator-controlled intermediary rule. Application admission controllers and workflow schedulers remain necessary for tenant fairness, task priority, deadlines, parent-child budgets, and side-effect safety.
Public limits are not service-level commitments. Anthropic and Google explicitly describe limits as maximums rather than guaranteed minimum capacity. Private contracts, reserved or provisioned throughput, priority tiers, enterprise support, abuse controls, and regional conditions can change observed behavior.
Provider token units are not fully interchangeable. Tokenization, cache accounting, output reservation, multimodal units, long-context rules, and Bedrock token burndown can differ. Cross-provider routing requires task-level testing and normalized cost and capacity records rather than comparing raw TPM values alone.
A gateway cannot know whether an agent result is correct or whether a retry is safe without application context. Rate limiting does not replace authorization, policy enforcement, human approval, idempotency, transaction reconciliation, sandboxing, observability, evaluation, or incident response.
This analysis does not compare pricing or recommend a single provider. Spend controls are discussed as infrastructure semantics; actual costs depend on model, modality, region, service tier, cache behavior, tools, contracts, and workload shape.
References
- OpenAI API: Rate limits
- OpenAI API: Spend limits
- Anthropic Claude Platform: Rate limits
- Anthropic Claude Platform: Rate Limits API
- Google AI for Developers: Gemini API rate limits
- Amazon Bedrock: Quotas
- Amazon Bedrock: Quotas for the bedrock-runtime endpoint
- Amazon Bedrock: How tokens are counted
- Cloudflare AI Gateway: Rate limiting
- Cloudflare AI Gateway: Spend limits
Changelog
- 2026-08-29: Initial publication.
Corrections
No corrections have been issued for this document.