Together AI Rate Limits: 1 RPS Free Caps, Tier Quotas, and 429 Error Fixes

As of September 2026, Together AI enforces serverless rate limits starting at 1 request per second (60 RPM) on initial credit accounts and scales dynamically across tiers to 600+ requests per minute according to Together AI's official documentation. Understanding where these boundaries live is critical for developers moving from experimental scripts to multi-agent production systems.

Together AI rate limits restrict the rate of requests per second (RPS), requests per minute (RPM), and tokens per minute (TPM) that can be sent to Together's serverless inference endpoints. Rather than locking accounts into rigid, permanent tiers, the platform evaluates capacity on a dynamic basis.

Dynamic Per-Model Rate Limits

For serverless inference, Together AI applies dynamic per-model rate limits that scale with sustained traffic, rather than enforcing a single global quota across every hosted model. Your allocated throughput is determined by two independent factors:

1. Live Model Capacity: The real-time GPU cluster availability allocated to that specific model across Together AI's infrastructure.
2. Historical Sustained Usage: Your organization's recent volume of successful, steady requests on that model.

Because rate limits are tracked per model, calling meta-llama/Llama-3.3-70B-Instruct-Turbo draws against a separate capacity pool than invoking a smaller model like meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo. A rate limit reached on one endpoint does not lock your access to other models on the platform.

Core Throughput Metrics: RPS, RPM, and TPM

Together AI throttles client traffic along three dimensions:

• Requests Per Second (RPS): The instantaneous rate of HTTP calls dispatched to the API.
• Requests Per Minute (RPM): The cumulative number of completed requests over a rolling 60-second window.
• Tokens Per Minute (TPM): The combined sum of prompt tokens sent to the model and completion tokens generated by the model over a rolling 60-second window.

Access Tiers and Infrastructure Models

In older documentation and community discussions, accounts were categorized into static designations like Free Tier, Tier 1, and Tier 2. While Together AI has transitioned to dynamic per-model scaling, new accounts begin with an initial credit allocation and standard 1 RPS (60 RPM) caps. As teams fund their accounts and establish steady request volume, the dynamic engine expands request quotas toward 600 RPM and beyond.

• OpenRouter Rate Limits: 20 RPM Caps, 429 Errors, and Agent Storage WorkaroundsOpenRouter restricts free models to 20 requests per minute and 50 to 1,000 requests per day based on credit purchases,...
• Google AI Studio Rate Limits: Free Tier Quotas, TPM, and Handling 429 ErrorsGoogle AI Studio rate limits enforce operational caps across requests per minute, tokens per minute, and daily request...
• Azure OpenAI Rate Limits: TPM Quotas, PTU Scaling, and 429 Error ResolutionAzure OpenAI rate limits are regional and subscription-level constraints defined by Tokens Per Minute (TPM) and...
• Gemini API Rate Limits: Tier Quotas, 429 Handling, and Large-Payload WorkflowsGemini API rate limits govern requests per minute (RPM), tokens per minute (TPM), and requests per day (RPD) across...
• Groq API Rate Limits: LPU Tier Quotas, TPM Ceilings, and Document HandlingGroq rate limit policies govern API throughput across GroqCloud LPUs through concurrent requests, requests per minute,...
• LlamaIndex Rate Limits: Ingestion Batching, Embedding Quotas, and Offloaded IndexingLlamaIndex rate limits are API request and token bottlenecks triggered while parsing, chunking, and embedding large...

More on this subject: Agent Security and Governance (51 guides)

When an API request fails with HTTP status code 429 Too Many Requests, Together AI is signaling that your client has exceeded either its request rate or its token throughput envelope. Inspecting the JSON payload reveals the exact reason for the throttle.

The Two Dynamic Error Types

The platform distinguishes between request frequency limits and token volume limits using dedicated error classifications:

1. dynamic_request_limited: This error occurs when your application dispatches more HTTP calls per second or minute than your dynamic allocation allows.
2. dynamic_token_limited: This error occurs when your cumulative token processing (prompt input tokens plus generated output tokens) exceeds your dynamic per-minute token quota.

A representative 429 error payload appears as follows:

{
  "error": {
    "message": "Rate limit exceeded for meta-llama/Llama-3.3-70B-Instruct-Turbo",
    "type": "invalid_request_error",
    "param": null,
    "code": 429,
    "error_type": "dynamic_request_limited"
  }
}

Distinguishing between these two types dictates your remediation path. A dynamic_request_limited error requires pacing your client calls, while a dynamic_token_limited error requires reducing prompt size or splitting context.

Differentiating 429 Rate Limits from 503 Capacity Errors

When evaluating rate limits, developers frequently confuse client-side throttles with infrastructure outages. Together AI maintains a clear boundary between client-side throttles and platform-side capacity:

• HTTP 429 Too Many Requests: Indicates that your organization's traffic exceeded its dynamic quota. The platform is healthy, but your requests exceeded your current limit.
• HTTP 503 Service Unavailable: Indicates that requests sent at or below your dynamic rate could not be served because the underlying model cluster is overloaded. This represents platform capacity pressure rather than client usage violation.

When encountering a 503, retry after a short delay, as Together AI's auto-scaler routes traffic to additional nodes. When encountering a 429, you must actively back off or shrink payloads.

Credit Balances (HTTP 402) vs. Rate Limits (HTTP 429)

Rate limits restrict speed and volume, whereas credit balances govern payment. If your prepaid account balance drops to zero or goes negative, Together AI does not return a 429. Instead, the API returns HTTP 402 Payment Required. Adding funds restores access immediately without waiting for rate limit windows to reset.

Billing Policies for Rate-Limited Requests

The platform does not bill for requests that return an HTTP 429 error. Because 429 responses are rejected at the edge gateway before compute execution, no tokens are processed or generated. Similarly, requests failing with HTTP 503 errors incur no charge. For Batch API jobs, individual requests that fail with an error are omitted from billing, although completed requests in a partially executed batch remain billable.

Monitoring rate limits programmatically prevents surprise throttles. Together AI returns specific telemetry headers on API responses that your application can parse to dynamically regulate request cadence.

Rate Limit Response Headers

The serverless inference API delivers five primary telemetry headers:

• x-ratelimit-limit: The maximum number of requests per second allowed for the invoked model.
• x-ratelimit-remaining: The number of permitted requests remaining in the active per-second window.
• x-ratelimit-reset: The integer number of seconds remaining until your request quota resets.
• x-tokenlimit-limit: The maximum number of tokens per second permitted.
• x-tokenlimit-remaining: The number of tokens remaining in the active window.

You can inspect these headers directly using a curl request:

curl -i https://api.together.ai/v1/chat/completions \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    "messages": [{"role": "user", "content": "ping"}]
  }'

Implementing Exponential Backoff with Jitter

When a 429 error occurs, a client should parse x-ratelimit-reset and pause execution. If that header is absent, use exponential backoff with randomized jitter to prevent thundering herd spikes against the gateway.

import os
import random
import time
import requests

def call_together_chat(messages, model="meta-llama/Llama-3.3-70B-Instruct-Turbo", max_retries=5):
    url = "https://api.together.ai/v1/chat/completions"
    headers = {
        "Authorization": f"Bearer {os.environ.get('TOGETHER_API_KEY')}",
        "Content-Type": "application/json"
    }
    payload = {"model": model, "messages": messages}
    for attempt in range(max_retries):
        response = requests.post(url, headers=headers, json=payload)
        if response.status_code == 200:
            return response.json()
        if response.status_code == 429:
            error_data = response.json().get("error", {})
            error_type = error_data.get("error_type", "dynamic_request_limited")
            reset_header = response.headers.get("x-ratelimit-reset")
            if reset_header:
                wait_seconds = float(reset_header) + random.uniform(0.1, 0.5)
            else:
                wait_seconds = (2 ** attempt) + random.uniform(0.5, 1.5)
            print(f"Rate limited ({error_type}). Retrying in {wait_seconds:.2f}s...")
            time.sleep(wait_seconds)
            continue
        if response.status_code == 503:
            wait_seconds = (2 ** attempt) + random.uniform(1.0, 2.0)
            print(f"Platform capacity pressure. Retrying in {wait_seconds:.2f}s...")
            time.sleep(wait_seconds)
            continue
        response.raise_for_status()
    raise RuntimeError("Exceeded maximum retry attempts against Together AI API.")

Handling Streaming Response Errors

When streaming chat completions with Server-Sent Events (stream: true), the API sends an initial HTTP 200 header as soon as the connection opens. If token rate limits are triggered during generation, the connection cannot be converted to an HTTP 429. Instead, Together AI emits a terminal SSE chunk containing an error object. Streaming parsers must inspect chunk events for finish_reason: "error" rather than relying solely on initial HTTP status codes.

Most rate limit documentation focuses on human chat patterns, where a user submits one query every minute. In multi-agent systems, rate limit dynamics change completely. Autonomous agents run in rapid, continuous execution loops that can trigger rate limit backpressure within seconds.

The Agent Execution Multiplier

When autonomous frameworks such as CrewAI, LangGraph, AutoGen, Cline, and OpenClaw automate reasoning and tool execution, execution loops run continuously:

1. The agent analyzes an objective and calls a tool.
2. The runtime environment executes the tool and captures output.
3. The agent immediately appends the output to its context and dispatches the updated message history back to Together AI.
4. The cycle repeats dozens of times without human intervention.

When four subagents operate concurrently in a research pipeline, they can generate hundreds of API requests in a two-minute window. Even if your account qualifies for 600 RPM, rapid concurrent calls from parallel workers can easily trigger short-window RPS limits.

Context Bloat and Token Exhaustion

While request frequency presents one constraint, context bloat is the primary driver of agent rate limit failures. Developers building document analysis, legal review, or code intelligence agents often attach raw source files directly into system prompts.

Consider an agent analyzing three technical manuals or corporate policy documents. If the agent prepends the full text of those files into each execution step:

• Turn 1 transmits 60,000 prompt tokens.
• Turn 2 transmits the tool output plus the original 60,000 prompt tokens.
• Turn 3 transmits the growing conversation history plus the original 60,000 prompt tokens.

Within three turns, a single agent transmits hundreds of thousands of prompt tokens in under a minute. This causes immediate token-per-minute exhaustion, triggering dynamic_token_limited errors.

Furthermore, packing massive document payloads into prompts degrades reasoning accuracy and burns prepaid credits rapidly. Real-world platforms frequently hit capacity barriers, as project knowledge is limited by the context window, driving developers to search for cleaner external file handling patterns.

The sustainable architectural solution is decoupling document storage from prompt memory.

Optimize agent payloads and avoid rate limit throttles

Keep prompt context lean by offloading reference documents to a Fast.io workspace with built-in semantic search and MCP connectivity. Monthly plans start with a trial of up to 30 days (credit card required); annual plans have no trial.

Start 30-Day Trial

Instead of stuffing raw files into prompt payloads, production agent systems store reference materials in persistent external storage. The agent retains only immediate instructions in active prompt memory and retrieves relevant excerpts on demand.

Centralizing Reference Materials in Workspaces

A persistent workspace acts as shared memory for your team and connected agents. Rather than re-uploading documents on every API call, teams place reference corpora in a Fast.io workspace.

Fast.io supports direct file uploads alongside cloud imports from Google Drive, Dropbox, Box, and OneDrive. This allows engineering and operations teams to organize documentation into persistent, structured folder hierarchies without manual file transfers.

Intelligent Indexing and Hybrid Search

After documents reside in a Fast.io workspace, enabling Intelligence Mode activates automated background indexing. Files are indexed for hybrid search, combining exact keyword matching and semantic retrieval.

This architectural shift fundamentally alters token consumption:

• Without Persistent Workspaces: The agent injects entire 50-page documents into every prompt, transmitting 60,000 tokens per call and breaching Together AI's TPM limits.
• With Persistent Workspaces: The agent queries the workspace search endpoint, retrieves only the three paragraphs directly relevant to the current subtask, and passes those excerpts to Together AI.

Prompt payloads decrease from massive multi-page document dumps to concise reference excerpts. Reducing payload volume by an order of magnitude keeps token consumption well beneath Together AI's dynamic TPM thresholds, preventing dynamic_token_limited throttles.

Connecting Agents via Model Context Protocol (MCP)

Agents communicate with Fast.io workspaces using the Model Context Protocol. Fast.io hosts a remote MCP server accessible over Streamable HTTP at https://mcp.fast.io/mcp (or https://mcp.fast.io/mcp/key when using Bearer token authentication) and legacy SSE at https://mcp.fast.io/sse. Setup details are documented in the Fast.io agent storage guide.

Here is an example MCP configuration block for agent clients:

{
  "mcpServers": {
    "fastio": {
      "url": "https://mcp.fast.io/mcp/key",
      "headers": {
        "Authorization": "Bearer FASTIO_API_KEY"
      }
    }
  }
}

Through this consolidated MCP toolset, agents search indexed workspace files, read specific sections, save generated deliverables, and maintain folder organization.

Auditing, Versioning, and Ownership Transfer

When multiple agents write files to a shared workspace, version control is essential. Fast.io maintains complete per-file version history, allowing humans to inspect revisions, track diffs, and restore prior versions. An append-only audit log records every file read, write, and share event performed by people and agents.

When an autonomous agent builds out workspace assets for a human client or team lead, Fast.io supports ownership transfer. The agent creates the organization and workspace structure, populates the deliverables, and transfers primary ownership to a human colleague while retaining scoped administrative access.

Fast.io does not modify or override Together AI's serverless rate limits. What it provides is the storage and retrieval layer that prevents prompt bloat, enabling agents to complete extensive document tasks without hitting Together AI rate caps.

Monthly plans start with a trial of up to 30 days (credit card required); annual plans have no trial. Review complete plan tiers and subscribe on the Fastio pricing directory.

If your production workload legitimately requires higher throughput, Together AI provides several pathways to expand capacity across models.

1. Warm Up Dynamic Limits with Steady Traffic

The dynamic rate-limiting engine rewards sustained, predictable traffic. If you anticipate a traffic surge, ramp up your request volume gradually rather than sending sudden spikes. Steady, successful requests signal valid demand to the platform's predictive allocators, automatically increasing your dynamic RPS and TPM allowances over time.

2. Move Beyond Initial Credit Quotas

For new organizations, accounts operating on initial credit allocations are capped at 1 RPS (60 RPM). Adding a credit card and maintaining a positive prepaid balance moves your account into paid tiers, raising standard limits to 600 RPM. For higher enterprise limits, organizations can contact Together AI sales to establish custom tier agreements.

3. Adopt Provisioned Throughput for Committed SLAs

For production services that require guaranteed capacity on stock foundation models, Together AI provides Provisioned Throughput. Provisioned Throughput reserves dedicated token capacity (measured in tokens per minute) with formal service level agreements, shielding your application from serverless rate throttling.

4. Route Async Workloads to Batch Inference

If your tasks do not require immediate interactive streaming (such as bulk data categorization, offline evaluations, or synthetic data generation), use Together AI's Batch Inference API. Batch processing executes jobs asynchronously within 24 hours at substantial cost discounts, without consuming your real-time serverless rate limits.

5. Deploy Dedicated Model Inference (DMI)

For multi-agent deployments running at enterprise scale, or teams hosting custom fine-tuned weights, Dedicated Model Inference provides fully isolated GPU clusters. With dedicated instances, standard serverless rate limits are eliminated. Throughput is bounded purely by the raw compute capacity of your reserved hardware.