# OpenAI Rate Limits: TPM, RPM, Tiers, and Large Corpus Strategies

OpenAI rate limits restrict API consumption through Requests Per Minute, Requests Per Day, and Tokens Per Minute across account tiers. Attempting to analyze large document collections by pushing raw file text into prompt payloads quickly exhausts TPM allocations and triggers HTTP 429 errors. Decoupling storage from inference by indexing files in an intelligent workspace and querying them through remote MCP retrieval prevents quota exhaustion while maintaining complete document access.

Source: https://fast.io/resources/openai-rate-limits/
Author: [Derek Labian](https://fast.io/authors/derek-labian/)
Last reviewed: 2026-09-13

## How OpenAI Rate Limits Operate Across TPM, RPM, and RPD

An engineering team running document ingestion on an initial OpenAI account hits a hard ceiling of 30,000 tokens per minute and 500 requests per minute on flagship models, triggering HTTP 429 rate limit errors within seconds of processing a multi-file batch. Pushing raw document payloads directly into prompt contexts exhausts per-minute token buckets before a single pipeline finishes.

"OpenAI rate limits are restrictions imposed by OpenAI on developer API usage, measured in Requests Per Minute (RPM), Requests Per Day (RPD), and Tokens Per Minute (TPM) across designated billing tiers."

To manage infrastructure load and maintain service availability across developer organizations, OpenAI meters API access across distinct operational metrics:

* **Tokens Per Minute (TPM).** Limits the aggregate count of prompt input tokens and completion output tokens processed by a model within a rolling 60-second window. On streaming Chat Completions and Responses calls, the API reserves anticipated completion tokens against remaining TPM balance until generation completes.
* **Requests Per Minute (RPM).** Caps total HTTP calls dispatched to an endpoint within a rolling 60-second span, regardless of token payload size.
* **Requests Per Day (RPD) and Tokens Per Day (TPD).** Establish cumulative daily ceilings over a 24-hour cycle. Daily limits protect development environments from accidental infinite loops and runaway billing.
* **Images Per Minute (IPM) and Audio Minutes.** Enforce media-specific throughput limits on multimodal generation and transcription endpoints.
* **Vector Store Ingestion Limits.** Organizations using OpenAI vector store endpoints encounter a shared ceiling of 300 requests per minute across file creation and batch endpoints.

Rate limits apply at the organization and project levels. They are not assigned to individual API keys or human users. Generating multiple API keys within a project draws from the exact same shared allocation.

### The Rolling Token Bucket Algorithm

OpenAI calculates token and request rates using a continuous sliding window rather than a static clock reset. Headroom does not reset at the start of each calendar minute. Instead, token usage decays continuously as requests age past the 60-second boundary.

If an application submits 26,000 prompt tokens on an account with a 30,000 TPM limit, remaining balance drops immediately to 4,000 tokens. Submitting a second request with 5,000 tokens seconds later triggers an HTTP 429 error. Full capacity returns only after the initial payload completely ages out of the active accounting window.

### Shared Model Limits and Long-Context Thresholds

Rate limits vary by model, and several model families share a collective quota. When multiple model variants sit under a shared limit in account settings, calls to any model in that family consume the same shared pool.

High-capacity models supporting extended context windows (such as GPT-4o and reasoning models) also enforce separate, stricter rate limits for long-context requests. When a prompt crosses defined input token thresholds, the request routes through dedicated inference pools with tighter RPM and TPM ceilings.

### Understanding Rolling Token Windows

The continuous nature of rolling token windows requires client applications to space out batch operations. Scripts that loop through local files and dispatch consecutive requests will saturate an entry token bucket within seconds.

Developers must calculate token budgets prior to dispatching requests. Because the API meters input tokens plus anticipated output tokens, setting a high output ceiling parameter immediately reduces available headroom for concurrent requests during that minute.

### Scoping Quotas Across Projects and Workspaces

OpenAI allows organizations to partition quotas across projects. Organization administrators can establish project-scoped token limits to prevent experimental scripts from consuming production quota.

Project-level rate limits cannot exceed overall organization allocations, but they can enforce tighter ceilings. When a project hits its assigned quota, the API returns headers indicating that the project limit was reached, leaving remaining organization headroom intact for other operational workloads.

## How Account Tiers and Model Quotas Determine Capacity

OpenAI automatically assigns accounts to designated usage tiers based on cumulative payment history and account age. As an organization spends money on the API and settles invoices, OpenAI graduates the account to higher tiers. Each tier advancement increases rate limits and monthly spend ceilings across flagship and standard models.

The table below outlines the qualification criteria, monthly spend limits, and representative rate limits for flagship models across developer tiers:

| Tier | Qualification Criteria | Monthly Spend Limit | Flagship TPM (GPT-4o) | Flagship RPM | Batch Queue Limit |
|---|---|---|---|---|---|
| Tier 1 | Initial credit purchase | Entry monthly limit | 30,000 TPM | 500 RPM | 90,000 queued tokens |
| Tier 2 | Low cumulative spend | Standard monthly limit | 450,000 TPM | 5,000 RPM | 1,350,000 queued tokens |
| Tier 3 | Moderate cumulative spend | Expanded monthly limit | 800,000 TPM | 5,000 RPM | 2,400,000 queued tokens |
| Tier 4 | High cumulative spend | Elevated monthly limit | 2,000,000 TPM | 10,000 RPM | 6,000,000 queued tokens |
| Tier 5 | Enterprise cumulative spend | Maximum monthly limit | 10,000,000+ TPM | 10,000 RPM | 30,000,000+ queued tokens |

Tier progression depends on cumulative settled payments and account tenure. The entry tier activates with an initial credit purchase, while subsequent tiers unlock as cumulative billing reaches higher thresholds and the account satisfies minimum tenure requirements. The exact qualification criteria and corresponding rate limits for each tier are detailed in the comparison table above.

### Tracking Quotas via HTTP Response Headers

Every response from the OpenAI API includes HTTP headers reporting real-time quota status. Inspecting these headers allows production applications to throttle requests proactively:

* `x-ratelimit-limit-requests`: Maximum requests permitted per minute for the model.
* `x-ratelimit-limit-tokens`: Maximum tokens permitted per minute for the model.
* `x-ratelimit-remaining-requests`: Requests remaining in the current rolling window.
* `x-ratelimit-remaining-tokens`: Tokens remaining before hitting the rolling limit.
* `x-ratelimit-reset-requests`: Duration until the request rate limit resets to baseline.
* `x-ratelimit-reset-tokens`: Duration until the token limit resets to baseline.
* `x-ratelimit-limit-project-tokens`: Token ceiling for the specific project scope.
* `x-ratelimit-remaining-project-tokens`: Token headroom remaining in the active project scope.
* `Retry-After`: Returned on HTTP 429 and 503 responses, indicating the minimum seconds the client must wait before retrying.

### The Ramp-Rate Throttling Guard

Staying below total TPM allocations does not guarantee uninterrupted throughput. OpenAI also enforces an internal traffic ramp-rate restriction. If an application abruptly ramps traffic from zero to high volume within seconds, the API can return an HTTP 429 response with an error code of `slow_down`.

This throttling mechanism protects physical infrastructure from sharp traffic spikes. As an operational guideline, once an application crosses high input volume thresholds, spreading traffic increases over gradual intervals prevents sudden `slow_down` responses while allowing automated scaling systems to adapt.

### Monitoring Quota Headers in Production

Production API middleware should parse `x-ratelimit-remaining-tokens` and `x-ratelimit-reset-tokens` on every response. If remaining tokens approach depletion, the client should delay dispatching secondary background operations.

Recording these header metrics in telemetry dashboards provides early warning of approaching quota ceilings before users experience dropped requests or pipeline timeouts.

### Batch API Queue Limits and Asynchronous Processing

For non-real-time workloads, OpenAI provides a Batch API. The Batch API processes requests asynchronously within a 24-hour completion window, offering reduced token pricing compared to standard endpoints.

Batch requests do not count against real-time TPM or RPM rate limits. Instead, they are governed by separate Batch Queue Limits based on total queued input tokens, as shown in the tier table above. Once a batch job completes, its tokens clear from the queue limit.

## How to Diagnose HTTP 429 Errors and Implement Backoff

When an application exceeds rate limit thresholds, the API returns an HTTP 429 status code. However, HTTP 429 errors stem from three distinct root causes, and treating them identically leads to faulty recovery loops.

Developers must inspect both the HTTP status and the JSON response body to identify the specific error type:

1. **Rate Limit Exceeded (`rate_limit_error`, code `rate_limit_exceeded`).** The application sent more requests or tokens than permitted within the active rolling 60-second window. The response typically includes a `Retry-After` header.
2. **Traffic Spike (`rate_limit_error`, code `slow_down`).** Total consumption remained within the organization's absolute TPM and RPM ceilings, but request volume increased too rapidly. The solution is to pause for the duration specified in `Retry-After` and ramp subsequent requests gradually.
3. **Quota Depletion (`insufficient_quota`).** The organization has exhausted its prepaid balance, reached its monthly spend limit, or encountered a billing failure. Retrying an `insufficient_quota` error with backoff will never succeed. The request will fail repeatedly until an account administrator deposits funds or adjusts the spend cap in account settings.

Additionally, HTTP 503 errors (`service_unavailable_error`, code `server_is_overloaded`) reflect internal OpenAI server saturation rather than client rate limit breaches. These responses should be retried using exponential backoff with random jitter.

### Implementing Exponential Backoff with Jitter

When handling temporary rate limits, naive retries cause synchronized retries across concurrent worker processes, creating thundering herd conditions that trigger secondary 429 errors.

The standard pattern combines exponential backoff, randomized jitter, and inspection of the `Retry-After` header. The following Python example illustrates how to wrap OpenAI API calls with backoff logic:

```python
import time
import random
import openai
from openai import OpenAI

client = OpenAI()

def call_openai_with_backoff(prompt: str, model: str = "gpt-4o", max_retries: int = 5):
    delay = 1.0
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(model=model, messages=[{"role": "user", "content": prompt}])
        except openai.RateLimitError as err:
            if attempt == max_retries - 1:
                raise err
            retry_after = None
            if hasattr(err, "response") and err.response is not None:
                hdr = err.response.headers.get("retry-after")
                if hdr:
                    try:
                        retry_after = float(hdr)
                    except ValueError:
                        retry_after = None
            sleep_time = retry_after + random.uniform(0.1, 0.5) if retry_after is not None else delay * (1 + random.uniform(0, 0.5))
            delay *= 2.0
            time.sleep(sleep_time)
        except openai.APIConnectionError as conn_err:
            if attempt == max_retries - 1:
                raise conn_err
            time.sleep(delay)
            delay *= 2.0
    raise RuntimeError("Exceeded maximum retry attempts")
```

### The Physical Ceiling of Client-Side Throttling

While exponential backoff resolves transient concurrency spikes, it cannot overcome a mismatch between prompt payload size and tier limits.

Consider an engineering team analyzing twenty technical PDFs totaling 60,000 words (approximately 80,000 tokens). If that team operates on an entry tier account with a 30,000 TPM limit, sending those documents in a single prompt will fail with HTTP 429 every single time. No amount of exponential backoff, request delay, or Redis token queueing can squeeze an 80,000-token payload through a 30,000-token gate. The prompt itself exceeds the physical volume of the 60-second bucket.

When individual document payloads exceed your tier limits, client-side retry logic reaches a dead end. Teams must change their context architecture.

### Handling Streaming Request Errors

For applications that stream output tokens using Server-Sent Events, rate limit errors typically arrive on the initial HTTP handshake before streaming commences.

However, if an error occurs mid-stream due to backend instability or connection termination, client applications should avoid automatically replaying the entire request if partial content was already displayed to the end user. Mid-stream recovery requires tracking received token offsets to resume gracefully.

### Distinguishing Spend Caps from Rate Limits

Account administrators can set soft and hard monthly spend caps in the OpenAI billing dashboard. A hard spend cap acts like a rate limit by rejecting requests once reached, but it returns an `insufficient_quota` error code rather than a rolling rate limit error.

Automated pipelines should parse this error code specifically to avoid executing retry loops that burn unnecessary compute cycles on locked billing accounts.

## Why Bulk Document Ingestion Fails at Scale

When developers hit TPM limits, the default advice in community forums is simple: deposit more cash with OpenAI to advance across tiers. While purchasing higher tiers raises your ceiling, relying on tier upgrades to solve document analysis problems represents expensive architectural debt.

Dumping raw document text directly into prompt payloads introduces three major drawbacks:

* **Excessive Token Costs.** Ingesting tens of thousands of tokens of raw file text on every user question means paying for prompt tokens repeatedly, even when the answer resides in two short paragraphs.
* **Increased Latency.** Processing large prompt payloads increases Time to First Token (TTFT). Flagship models spend seconds processing massive context windows before beginning generation.
* **Context Degradation.** As context windows expand with irrelevant boilerplate, models suffer from attention dilution, often missing key clauses buried in the middle of long documents.

This challenge is not unique to OpenAI. In Claude Projects, for example, users hit a hard 50-file limit per project, which is the exact moment real users go looking for external persistence and shared workspaces. In the OpenAI API, developers encounter a different barrier: they run headfirst into 30,000 or 450,000 TPM ceilings and vector store ingestion rate limits (300 requests per minute per vector store). In both ecosystems, the underlying design flaw is identical: attempting to upload and attach entire raw files into the model's active reasoning context.

### The Decoupled Workspace Architecture

The sustainable solution decouples persistent document storage and indexing from the language model's immediate prompt context:

1. **Store your document corpus in an intelligent workspace.** Upload source documents into a shared [Fast.io workspace](/product/workspaces/). You can upload files directly or import entire folder structures from Google Drive, Dropbox, Box, or Microsoft OneDrive through [Cloud Import](/product/cloud-import/). Google Drive, Dropbox, Box, and OneDrive folders can also stay in sync on a schedule or on demand.
2. **Enable Intelligence Mode.** When Intelligence is active, Fast.io automatically indexes files on arrival. The platform extracts text from PDFs, Word documents, spreadsheets, presentations, and scanned pages, generating semantic vector embeddings and full-text search indexes without requiring a separate vector database.
3. **Connect via Remote Model Context Protocol (MCP).** Connect your OpenAI assistant, agent framework, or backend application to Fast.io's remote MCP server over Streamable HTTP at `https://mcp.fast.io/mcp/tools`. Developers can review [Fast.io storage for agents](/storage-for-agents/) for configuration details.
4. **Retrieve Targeted Excerpts Instead of Full Files.** When a user asks a question about the corpus, the model calls Fast.io's semantic search tools via MCP, retrieves only the top three to five relevant excerpts (totaling 1,200 to 1,500 tokens), and passes those focused snippets into the prompt.

Fast.io does not raise OpenAI's rate limits. Instead, it transforms your context architecture so your application uses a fraction of its allocated tokens, keeping requests well under entry tier limits.

```json
{
  "mcpServers": {
    "fastio": {
      "url": "https://mcp.fast.io/mcp/tools",
      "headers": {
        "Authorization": "Bearer YOUR_FASTIO_API_KEY"
      }
    }
  }
}
```

### Token Math: Bulk Context versus Targeted MCP Retrieval

To understand the operational impact of this shift, compare the token economics of analyzing an enterprise contract repository across 40 files:

| Metric | Direct Prompt Ingestion | Intelligent Workspace Retrieval via MCP |
|---|---|---|
| Ingested Document Volume | 40 files (120,000 words) | 40 files stored and indexed in Fast.io workspace |
| Tokens per Request | ~160,000 prompt tokens | ~1,500 prompt tokens (retrieved excerpts) |
| Entry Tier Feasibility (30k TPM) | Fails instantly (far exceeds 30k limit) | Operates comfortably (uses fraction of minute limit) |
| Max Concurrent Queries (Entry Tier) | 0 queries | 15 to 20 concurrent queries per minute |
| Model Latency (TTFT) | 4 to 8 seconds | Sub-second initial token delivery |

By switching from bulk document uploads to targeted workspace search, token consumption drops to a tiny fraction of the original volume. An engineering team on an entry tier can support active multi-user document queries without paying premium sums to force tier upgrades.

### Connecting MCP to OpenAI Python and TypeScript Apps

Developers building with OpenAI's official SDKs or frameworks like LangChain, AutoGen, and CrewAI can integrate Fast.io MCP tools directly into their agent loops.

The agent queries the workspace toolset using standard function calling. When a user asks about compliance policies, the agent queries the Fast.io index, receives the exact policy paragraphs with file citations, and generates an answer grounded in source documents.

### Avoiding Vector Store Ingestion Bottlenecks

Building custom RAG pipelines on OpenAI's Assistants API requires creating and managing vector stores, which enforce a rate limit of 300 requests per minute across endpoints.

Offloading document storage and embedding generation to an intelligent workspace removes these ingestion throttles. Files remain searchable immediately upon upload, eliminating vector store provisioning scripts and token chunking pipelines.

## Steps to Scale Production Workflows for Large Document Corpora

Scaling generative AI applications across extensive document collections requires combining targeted retrieval with specialized processing patterns. Beyond basic semantic search, production architectures incorporate structured extraction, asynchronous batching, and auditable storage.

### Structured Extraction via Metadata Views

When business requirements demand extracting structured tabular data across thousands of files (such as pulling renewal dates, counterparty names, and liability caps from contracts, or invoice totals and line items from receipts), running general chat completions over MCP search can still generate substantial API traffic.

Instead of processing documents one by one through conversational prompts, teams use [Metadata Views](/product/document-data-extraction/). Metadata Views turn unstructured document collections into structured, queryable tables:

* Users define target fields in natural language (such as Contract Value, Effective Date, Governing Law, or Status).
* The workspace automatically extracts these typed fields (Text, Integer, Decimal, Boolean, URL, JSON, Date) across PDFs, images, scanned pages, spreadsheets, and Word documents.
* Extracted data populates an interactive, filterable spreadsheet view.
* AI agents can query the extracted table directly via MCP without reading full document texts.

Extracting structured data at the workspace layer eliminates repetitive LLM token processing for recurring document audits.

### Combining Asynchronous Batches with Real-Time Retrieval

Production architectures separate user-facing conversational queries from background batch operations. Real-time customer queries should always route through targeted MCP workspace search to minimize latency and preserve TPM capacity.

Conversely, large-scale background operations (such as re-evaluating historical archives or scoring quarterly performance metrics) should route through OpenAI's Batch API. By pairing the Batch API with indexed workspace storage, organizations achieve maximum throughput without risking real-time rate limit starvation.

### Per-File Versioning and Multi-Agent Coordination

In active engineering environments, multiple autonomous agents and human team members collaborate on the same file collections. Fast.io tracks complete per-file version history on every document.

When an agent analyzes a technical brief, updates an analysis spreadsheet, or adds an executive summary, previous file versions remain fully preserved in the workspace audit log. If an automated script generates an incorrect revision, human operators can restore prior versions with one click. Once an agent finishes building a document collection, it can transfer workspace ownership directly to a human team member while retaining administrative permissions for ongoing maintenance.

### Managing Rate Limits Across Multi-Agent Swarms

When deploying multi-agent teams where specialized agents handle research, drafting, and verification concurrently, total request volume can surge unexpectedly.

Assigning distinct folder boundaries within a shared workspace prevents agents from generating duplicate search queries. Agents coordinate through shared documents rather than broadcasting high-volume conversational chatter over API connections.

### Account Setup and Subscription Options

Creating an account on Fast.io is free. Setting up active workspaces for teams and agents requires an organization on a paid subscription.

New organizations can start with a 14-day free trial, which requires a credit card. Plans are Starter at $9.99/mo, Business at $49.99/mo, and Enterprise at $199.99/mo on [Fast.io pricing](/pricing/). Subscriptions include seats, persistent storage, and workspace intelligence capabilities.

## Frequently asked questions

### How do OpenAI API rate limits work?

OpenAI API rate limits restrict how frequently you can call endpoints within specific time windows. They are measured primarily in Requests Per Minute (RPM) and Tokens Per Minute (TPM), alongside daily request and token ceilings (RPD and TPD). Rate limits are scoped to entire organizations and projects rather than individual API keys, and they apply via a continuous rolling window rather than resetting on calendar minutes.

### How can I increase my OpenAI rate limit tier?

OpenAI automatically increases your rate limit tier as your cumulative payment history grows. Higher tiers unlock as cumulative billing reaches specified thresholds and the account satisfies tenure requirements. Once payment settles and tenure requirements are met, tier upgrades apply automatically in your account settings.

### How do you prevent rate limit 429 errors when analyzing large documents with OpenAI?

To prevent 429 errors when analyzing large document collections, avoid pasting raw file text into prompt payloads. Instead, store and index your corpus in an intelligent workspace like Fast.io, connect your assistant using remote Model Context Protocol (MCP), and retrieve only relevant excerpts using semantic search. This reduces token consumption per request to a small fraction of the original volume, keeping queries well under TPM limits.

### What is the difference between RPM, RPD, and TPM?

RPM (Requests Per Minute) limits the number of HTTP requests you can send within a rolling 60-second window. TPM (Tokens Per Minute) limits the total volume of input prompt tokens and output completion tokens processed within that same 60-second window. RPD (Requests Per Day) enforces an upper ceiling on the total number of calls permitted across a 24-hour cycle.

### Does the OpenAI rate limit apply across all API keys in an organization?

Yes. Rate limits are defined at the organization and project levels. Creating multiple API keys under the same project draws from the exact same RPM and TPM allocation. To partition quotas, administrators can configure distinct project scopes with dedicated token limits inside organization settings.

### What should I do if a single document prompt exceeds my model TPM limit?

If a single prompt exceeds your TPM limit (such as an 80,000-token document submitted on a 30,000 TPM tier), exponential backoff will never succeed because the request itself is larger than the entire 60-second token window. You must either split the document into smaller chunks or index the file in an external workspace to retrieve focused excerpts via MCP.

### What is the difference between an HTTP 429 rate limit error and an insufficient quota error?

An HTTP 429 error with type rate_limit_error indicates that your application exceeded its rolling RPM, TPM, or ramp-rate limits, which can be resolved by waiting and retrying with backoff. An error with code insufficient_quota means your account has depleted its prepaid balance or hit its monthly spend ceiling, which requires adding billing funds before any request can succeed.

## Sources

- [OpenAI Documentation: Rate Limits](https://platform.openai.com/docs/guides/rate-limits): OpenAI rate limits are restrictions imposed by the API on developer access within a specified period of time, tracked across requests per minute and tokens per minute.
- [OpenAI Documentation: Rate Limits](https://platform.openai.com/docs/guides/rate-limits): When an application exceeds temporary request or token thresholds, the OpenAI API returns an HTTP 429 status code.

## About Fast.io

Fast.io provides shared workspaces where people and AI agents work on the same files, with built-in semantic search and citation-backed chat over what they hold. Agents reach it through a remote MCP server, a REST API at https://api.fast.io/current/, and a command line client published on npm as @vividengine/fastio-cli. MCP setup is at https://mcp.fast.io/docs: Claude and most MCP clients connect to https://mcp.fast.io/mcp/tools, ChatGPT to https://mcp.fast.io/mcp/operations, and coding agents to https://mcp.fast.io/mcp/code.