# AWS Bedrock Token Limits: Model Output Caps, Converse API Parameters, and Quotas

An AWS Bedrock token limit refers to the parameter-enforced boundaries on input prompt length and maximum generated response tokens during model invocation. While input context windows reach 1,000,000 tokens on Amazon Nova Premier and 200,000 tokens on Claude 3.5 Sonnet, single completions are capped between 2,048 and 10,000 tokens. Setting maxTokens in the Converse API and separating model limits from service quotas prevents runtime failures.

Source: https://fast.io/resources/aws-bedrock-token-limit/
Author: [Tom Langridge](https://fast.io/authors/tom-langridge/)
Last reviewed: 2026-10-12

## What Are AWS Bedrock Token Limits?

While foundation models on Amazon Bedrock accept prompts spanning hundreds of thousands of tokens, every model caps single-completion output generation at a fraction of its input capacity. For example, Claude 3.5 Sonnet on Bedrock accepts 200,000 input tokens but caps output generation at 8,192 tokens, and Amazon Nova Premier supports 1,000,000 input tokens while capping single completions at 10,000 tokens.

An AWS Bedrock token limit refers to the parameter-enforced boundaries on input prompt length and maximum generated response tokens during model invocation. In generative AI systems, a token is the basic unit of text processing, roughly equivalent to four characters or 0.75 words in English. When an application communicates with Amazon Bedrock, token limits operate across two separate layers: the architectural boundaries of the foundation model and the account-level throughput limits of your AWS environment, as detailed in the [AWS Bedrock endpoints and quotas documentation](https://docs.aws.amazon.com/general/latest/gr/bedrock.html).

At the model invocation layer, every request processes tokens in two distinct phases: prompt prefill and token generation. During prefill, the model ingests the system prompt, conversational history, and context documents simultaneously. During generation, the model predicts output tokens one by one in an autoregressive loop. Because generating tokens requires sequential matrix calculations and dedicated GPU key-value (KV) cache memory, foundation model providers enforce strict ceilings on how many tokens a model can emit in a single HTTP response.

Developers encounter token limits when their application prompts generate extensive structured JSON, write long-form reports, or execute iterative code transformations. If an application requests more output tokens than the model architecture permits, or if the prompt plus generated tokens exceeds context limits, the request either fails immediately with an HTTP 400 error or terminates prematurely mid-sentence.

### Input Context Windows versus Output Completion Ceilings

A primary concept in Amazon Bedrock is the operational asymmetry between input context windows and output completion caps. The input context window defines the maximum sequence length a model can evaluate in its attention mechanism. This sequence includes system instructions, user prompts, past conversational turns, tool schemas, and document attachments.

In contrast, the output limit dictates the maximum number of new tokens the model can produce in response to that input. In the Bedrock API, developers specify this ceiling using request parameters such as `maxTokens` in the unified Converse API, `max_tokens` for Anthropic Claude, `max_gen_len` for Meta Llama, or `max_new_tokens` for Amazon Nova.

Crucially, you cannot trade input capacity for output capacity. An application sending a brief 100-token prompt to Claude 3.5 Sonnet cannot request a 50,000-token completion simply because 199,900 tokens of input headroom remain empty. The model output cap remains fixed at 8,192 tokens regardless of prompt brevity. Understanding this distinction is essential when designing multi-step agent workflows that generate large codebases, synthetic datasets, or comprehensive technical documentation.

### How Upfront Token Reservations Affect Concurrent Requests

Amazon Bedrock employs an upfront token reservation mechanism to maintain service stability across multi-tenant GPU clusters. When a client application dispatches a request to the `InvokeModel` or `Converse` endpoints, the Bedrock runtime does not wait to see how many tokens the model actually generates. Instead, Bedrock inspects the input prompt length and adds the full value configured in your maximum output token parameter.

Bedrock immediately reserves that combined token total against your account Tokens Per Minute (TPM) quota for the duration of the API call. If your code submits a prompt of 40,000 tokens and sets `maxTokens` to 8,192, Bedrock temporarily locks 48,192 tokens against your regional allocation.

If several background workers or autonomous agents execute in parallel, these reservations stack quickly. Ten concurrent workers with those parameters lock nearly 500,000 tokens of quota capacity. If your regional quota is 400,000 TPM, subsequent requests fail immediately with an HTTP 429 `ThrottlingException`, even if the model returns concise 200-token answers. Setting realistic, tailored output caps rather than leaving parameters at their theoretical maximums is necessary for stable production throughput.

## Comparing Foundation Model Token Limits on AWS Bedrock

Amazon Bedrock hosts foundation models from multiple providers, including Anthropic, Amazon, Meta, Mistral AI, Cohere, and AI21 Labs. Each provider structures its model architecture differently, resulting in wide variations in context windows, output limits, and parameter naming conventions.

The following comparison table outlines the documented token boundaries across primary foundation models available in Amazon Bedrock as of October 2026. Developers can verify current parameters in the [AWS Bedrock Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html) documentation.

| Foundation Model | Provider | Input Context Window | Max Output Tokens | Native Invocation Parameter | Default Stop Reason |
|---|---|---|---|---|---|
| Amazon Nova Premier | Amazon | 1,000,000 tokens | 10,000 tokens | `max_new_tokens` | `max_tokens` / `end_turn` |
| Amazon Nova Pro | Amazon | 300,000 tokens | 10,000 tokens | `max_new_tokens` | `max_tokens` / `end_turn` |
| Amazon Nova Lite | Amazon | 300,000 tokens | 10,000 tokens | `max_new_tokens` | `max_tokens` / `end_turn` |
| Amazon Nova Micro | Amazon | 128,000 tokens | 10,000 tokens | `max_new_tokens` | `max_tokens` / `end_turn` |
| Anthropic Claude 3.5 Sonnet | Anthropic | 200,000 tokens | 8,192 tokens | `max_tokens` | `max_tokens` / `end_turn` |
| Anthropic Claude 3.5 Haiku | Anthropic | 200,000 tokens | 8,192 tokens | `max_tokens` | `max_tokens` / `end_turn` |
| Anthropic Claude 3 Opus | Anthropic | 200,000 tokens | 4,096 tokens | `max_tokens` | `max_tokens` / `end_turn` |
| Meta Llama 3.3 70B Instruct | Meta | 128,000 tokens | 2,048 tokens | `max_gen_len` | `length` / `stop` |
| Meta Llama 3.1 405B Instruct | Meta | 128,000 tokens | 2,048 tokens | `max_gen_len` | `length` / `stop` |
| Mistral Large (24.07) | Mistral AI | 128,000 tokens | 8,192 tokens | `max_tokens` | `length` / `stop` |
| Amazon Titan Text Express | Amazon | 8,192 tokens | 8,192 tokens | `maxTokenCount` | `LENGTH` / `FINISH` |
| Amazon Titan Text Lite | Amazon | 4,096 tokens | 4,096 tokens | `maxTokenCount` | `LENGTH` / `FINISH` |
| Cohere Command R+ | Cohere | 128,000 tokens | 4,096 tokens | `max_tokens` | `MAX_TOKENS` / `COMPLETE` |

As shown in the table, output caps range from 2,048 tokens on Meta Llama models to 10,000 tokens on the Amazon Nova family. Understanding these differences allows architects to select the right model for specific generative requirements.

### Anthropic Claude Model Limits on Bedrock

Anthropic Claude models represent some of the most widely deployed reasoning engines on Amazon Bedrock. Claude 3.5 Sonnet and Claude 3.5 Haiku provide an input context window of 200,000 tokens, with a maximum output limit of 8,192 tokens. Earlier generation models, such as Claude 3 Opus and Claude 3 Haiku, maintain the same 200,000-token input window but enforce a lower maximum completion ceiling of 4,096 tokens.

When invoking Claude models via Bedrock's native `InvokeModel` API, developers pass parameters inside a JSON body matching Anthropic's Messages schema:

```json
{
  "anthropic_version": "bedrock-2023-05-31",
  "max_tokens": 8192,
  "temperature": 0.7,
  "messages": [
    {
      "role": "user",
      "content": "Generate a detailed system design specification."
    }
  ]
}
```

If your application specifies a `max_tokens` value greater than 8,192 for Claude 3.5 Sonnet, the Bedrock runtime returns a `ValidationException` stating that the parameter exceeds the maximum allowed value. If the model exhausts the allocated `max_tokens` before completing its generation, the response JSON sets `stop_reason` to `"max_tokens"`.

### Amazon Nova Model Ceilings and Multimodal Context

Amazon Nova models introduce deep multimodal support alongside expanded context windows. Amazon Nova Premier supports an input context window of 1,000,000 tokens, capable of processing extensive documentation archives, complex code repositories, or lengthy video files. Amazon Nova Pro and Amazon Nova Lite offer 300,000-token input windows, while Amazon Nova Micro provides a 128,000-token text-only context window.

Across the entire Nova family (Premier, Pro, Lite, and Micro), Amazon Bedrock standardizes the maximum completion output ceiling at 10,000 tokens. This makes Nova models capable for tasks requiring long-form text generation, extensive tabular synthesis, or detailed software module creation.

In native invocation payloads, Nova models accept inference settings within an `inferenceConfig` object, using `max_new_tokens` to define output limits. When called through the Converse API, Nova models map `maxTokens` to this underlying 10,000-token hardware cap.

### Meta Llama and Mistral AI Output Boundaries

Meta Llama models on Bedrock, including Llama 3.1 (8B, 70B, 405B) and Llama 3.3 70B Instruct, provide a 128,000-token input context window. In native Bedrock invocations using `InvokeModel`, Meta models use the parameter name `max_gen_len`. The documented range for `max_gen_len` is 1 to 2,048 tokens, with a default value of 512 tokens.

Because 2,048 tokens corresponds to approximately 1,500 words, developers building applications that generate long documents with Llama models must account for this compact completion boundary. If a response exceeds `max_gen_len`, the model sets the stop reason to `"length"`.

Mistral AI models, such as Mistral Large, also support a 128,000-token input context window. Unlike Llama, Mistral Large permits a maximum generation ceiling of 8,192 tokens, configured using the `max_tokens` parameter in native payloads.

## How to Configure Converse API Parameters: Managing maxTokens and Stop Reasons

Before AWS introduced the Converse API, developers writing applications across multiple model providers had to maintain custom payload serialization logic for each provider. Claude required `max_tokens`, Meta required `max_gen_len`, Titan required `maxTokenCount`, and Nova required `max_new_tokens`. Architectural differences between models and contexts are explored further in our guide to [AWS Bedrock context window architecture](/resources/aws-bedrock-context-window/).

The Amazon Bedrock Converse API (`Converse` and `ConverseStream`) standardizes conversational model invocations under a unified request structure. It is exposed exclusively on the `bedrock-runtime` endpoint.

In the Converse API, response length is controlled by the `maxTokens` parameter, located inside the `inferenceConfig` object.

```json
{
  "modelId": "anthropic.claude-3-5-sonnet-20240620-v1:0",
  "messages": [
    {
      "role": "user",
      "content": [{"text": "Draft an architectural review of our payment service."}]
    }
  ],
  "inferenceConfig": {
    "maxTokens": 4096,
    "temperature": 0.5,
    "topP": 0.9,
    "stopSequences": []
  }
}
```

The `maxTokens` parameter is an optional integer with a minimum value of 1. If you omit `maxTokens` from `inferenceConfig`, Amazon Bedrock automatically defaults to the maximum allowed output token limit for the specific model targeted by the request.

### Python Implementation with Boto3 and Usage Metrics

The following Python example demonstrates how to invoke a foundation model using the Converse API in `boto3`, specify `maxTokens`, and extract token usage metrics from the response.

```python
import boto3
from botocore.exceptions import ClientError

client = boto3.client("bedrock-runtime", region_name="us-east-1")
model_id = "anthropic.claude-3-5-sonnet-20240620-v1:0"
prompt_text = "Provide a comprehensive technical audit checklist for cloud migration."

try:
    response = client.converse(
        modelId=model_id,
        messages=[{"role": "user", "content": [{"text": prompt_text}]}],
        inferenceConfig={"maxTokens": 2048, "temperature": 0.3, "topP": 0.9},
    )
    # extract generated response text
    output_message = response["output"]["message"]["content"][0]["text"]
    stop_reason = response["stopReason"]
    # inspect detailed token consumption
    usage = response["usage"]
    input_tokens = usage["inputTokens"]
    output_tokens = usage["outputTokens"]
    total_tokens = usage["totalTokens"]
    print(f"Generated text: {output_message[:150]}...")
    print(f"Stop Reason: {stop_reason}")
    print(f"Tokens consumed: Input={input_tokens}, Output={output_tokens}, Total={total_tokens}")
    if stop_reason == "max_tokens":
        print("Warning: Response reached maxTokens limit and was truncated.")
except ClientError as e:
    error_code = e.response["Error"]["Code"]
    error_message = e.response["Error"]["Message"]
    print(f"Bedrock invocation failed [{error_code}]: {error_message}")
```

In this script, `response["usage"]` returns exact token metrics. Logging these values enables precise unit-cost tracking and allows monitoring systems to detect when generated responses consistently hit the `maxTokens` ceiling.

### Handling Truncation in ConverseStream Responses

When building interactive applications, developers typically implement `ConverseStream` to stream text deltas to end users with minimal Time-To-First-Token (TTFT). Monitoring token caps in a streaming context requires listening for specific stream events.

During a `ConverseStream` session, Bedrock emits a sequence of event blocks:
1. `messageStart`: Indicates message role (assistant).
2. `contentBlockDelta`: Emits incremental text chunks through `event["contentBlockDelta"]["delta"]["text"]`.
3. `messageStop`: Emits the terminal status of the message, including `stopReason`.
4. `metadata`: Emits the cumulative `usage` block containing final `inputTokens`, `outputTokens`, and latency metrics.

If the model hits the `maxTokens` cap during streaming, text generation halts immediately. The `messageStop` event delivers `event["messageStop"]["stopReason"] = "max_tokens"`. Client applications should detect this event and gracefully notify the user or initiate an automated continuation request.

## How Inference Token Limits Differ from Bedrock Service Quotas

A frequent source of confusion among cloud engineers is conflating Bedrock model inference token limits with AWS Service Quotas. While both concepts involve tokens, they govern completely different layers of the infrastructure, produce different error codes, and require distinct remediation strategies.

Inference token limits are architectural properties of foundation models. They govern the mathematical boundaries of a single model invocation: the context window and the `maxTokens` output cap. These limits cannot be raised by filing an AWS support ticket. For example, AWS Support cannot increase Claude 3.5 Sonnet's 8,192 output token limit to 15,000 tokens because that cap is baked into the model weights and Bedrock runtime design.

In contrast, Bedrock Service Quotas are account-level throughput rate limits enforced per AWS region. They dictate how much aggregate traffic your AWS account can send to Bedrock services across all concurrent users and applications. Quota adjustments can be managed directly in the [AWS Service Quotas console](https://console.aws.amazon.com/servicequotas/).

According to the AWS General Reference on Amazon Bedrock endpoints and quotas, AWS Bedrock enforces cross-region service quotas on tokens per minute, allocating 800,000 tokens per minute for Anthropic Claude 3.5 Sonnet across most supported regions. This quota considers the combined sum of input and output tokens across all calls to `Converse`, `ConverseStream`, `InvokeModel`, and `InvokeModelWithResponseStream`.

### Tokens Per Minute and Requests Per Minute Mechanics

AWS Service Quotas for Bedrock are primarily metered in two dimensions:
- Tokens Per Minute (TPM): The aggregate volume of input tokens submitted plus output tokens generated across all requests in a rolling 60-second window.
- Requests Per Minute (RPM): The total number of API calls initiated within a rolling 60-second window.

Bedrock tracks quotas separately for on-demand regional invocations, cross-region inference profiles, and provisioned throughput. For example, your account might have a default quota of 250 requests per minute and 400,000 tokens per minute for Claude 3.5 Sonnet in `us-east-1`.

When an application exceeds either threshold, the Bedrock runtime rejects the request with an HTTP 429 status code and a `ThrottlingException`. This error indicates that your account has saturated its allotted bandwidth on AWS infrastructure. It does not mean the model failed to process the prompt, nor does it indicate that your prompt exceeded the model's context window.

### Comparing Failure Modes and Remediation Paths

Understanding the differences between operational bottlenecks simplifies incident triage. The following comparison highlights how inference caps differ from service quotas:

| Operational Dimension | Inference Token Limit (`maxTokens`) | Service Quota (TPM / RPM) |
|---|---|---|
| Governing Layer | Foundation model architecture and runtime | AWS account throughput and regional infrastructure |
| Scope | A single API invocation | Aggregate traffic across all invocations in an account |
| Primary Error Signature | `stopReason: "max_tokens"` or HTTP 400 `ValidationException` | HTTP 429 `ThrottlingException` |
| Modifiable by User | Yes, down to 1 token within model limits | Yes, via AWS Service Quotas console |
| Modifiable by AWS Support | No (hard model architectural constraint) | Yes, quota increase requests are standard procedure |
| Primary Fix | Paging prompts, continuation loops, or concise generation | Exponential backoff, cross-region profiles, or quota increase |

If you receive an HTTP 400 `ValidationException` stating that your input prompt exceeded the context window, you must reduce the prompt length or switch to a model with a larger context window, such as Amazon Nova Premier. If you receive an HTTP 429 `ThrottlingException`, you should open the AWS Service Quotas console, search for Amazon Bedrock, locate the quota for Claude 3.5 Sonnet cross-region TPM, and submit an increase request.

## How to Architect Around Token Limits with Context Offloading

When engineering teams build complex autonomous agents or enterprise research tools, they often hit the practical limits of in-prompt data. As documented in Anthropic's help center on uploading files, Claude project uploads accept an unlimited number of files up to 30MB each, but total content must fit within Claude's context window. Whether working in web interfaces or building programmatic pipelines on AWS Bedrock, the practical ceiling of any AI session is the context window.

Attempting to solve file-intensive workflows by stuffing dozens of PDFs, spreadsheets, or source files directly into Bedrock prompt payloads creates severe operational challenges. First, large input prompts increase Time-To-First-Token latency because the model must process the entire sequence before generating its first word. Second, transmitting hundreds of thousands of raw text tokens across every turn exhausts account TPM quotas quickly. Third, models experience attention dilution when navigating massive unindexed prompts, frequently overlooking critical details placed in the middle of long texts.

To build resilient, cost-effective agent systems, engineering teams decouple persistent file storage from active model context windows.

### Continuation Patterns for Output Truncation

When an application legitimately requires a completion longer than a model's output cap (for example, generating a 15,000-token software specification with Claude 3.5 Sonnet), developers implement programmatic continuation loops.

When a call to `Converse` returns `stopReason == "max_tokens"`, the application preserves the generated text and immediately initiates a follow-up turn. The application appends the previous assistant completion to the `messages` array and sends a user message prompting the model to resume:

```python
# continuation turn pattern
messages.append({"role": "assistant", "content": [{"text": partial_output}]})
messages.append({"role": "user", "content": [{"text": "Continue generating precisely from where you stopped."}]})
continuation_response = client.converse(
    modelId=model_id,
    messages=messages,
    inferenceConfig={"maxTokens": 4096}
)
```

By stringing consecutive turns together, applications assemble long outputs while keeping each individual completion safely within Bedrock parameter ceilings.

### Decoupling Large Document Corpora with Fast.io Workspaces

While continuation loops solve output truncation, handling massive reference libraries requires external storage architecture. Rather than streaming large files directly into prompt tokens, production architectures store files in external workspaces and expose them to models via structured retrieval.

Traditional approaches rely on raw object stores like Amazon S3 paired with custom embedding pipelines, vector databases, and chunking scripts. While functional, building and maintaining custom retrieval infrastructure introduces substantial engineering maintenance. Local file storage offers fast local reads but fails when multiple agents or human reviewers require shared, synchronized access.

The large-corpus path on Fast.io provides a purpose-built workspace platform for agentic teams. Teams upload their document collections into [Fast.io workspaces](/product/workspaces/), or sync them directly from Google Drive, Dropbox, Box, or OneDrive. When Intelligence Mode is enabled on the workspace, Fast.io indexes incoming files for hybrid search (combining semantic meaning, full-text search, and metadata matching). Learn more about [Fast.io AI capabilities](/product/ai/) for workspace indexing.

Instead of attaching raw documents to Bedrock prompts, an AI assistant or agent connects to the workspace through the remote Model Context Protocol (MCP) server at `https://mcp.fast.io/mcp/tools`. For implementation details, consult the [Fast.io MCP documentation](https://mcp.fast.io/docs). Using MCP tools, the agent queries the indexed workspace dynamically, retrieving only the exact paragraphs, metrics, or legal clauses necessary to answer the prompt.

This architecture keeps Bedrock input prompts compact, reduces throttling risks, and leaves full reasoning headroom inside the model context window. Workspaces include per-file version history, a detailed activity log tracking human and agent actions, and role-based permissions.

| Plan Tier | Monthly Subscription | Workspace and Team Scale | Included AI Credits |
|---|---|---|---|
| Starter | $9.99/mo | 3 seats, 250 GB capacity | 100,000 credits |
| Business | $49.99/mo | 10 seats, 5 TB capacity | 600,000 credits |
| Enterprise | $199.99/mo | 30 seats, 25 TB capacity | 3,000,000 credits |

Monthly plans start with a 14-day free trial, which requires a credit card. See [pricing](/pricing/) for plan options.

## Frequently asked questions

### What is the token limit for AWS Bedrock?

AWS Bedrock enforces limits across two levels: foundation model inference boundaries and account service quotas. Model input context windows range from 4,096 tokens on Titan Text Lite to 200,000 tokens on Claude 3.5 Sonnet and 1,000,000 tokens on Amazon Nova Premier. Model output completion limits are capped between 2,048 tokens on Meta Llama and 10,000 tokens on Amazon Nova models. In addition, AWS enforces regional account quotas measured in Tokens Per Minute (TPM), such as 800,000 cross-region TPM for Claude 3.5 Sonnet.

### How do you increase the token limit in AWS Bedrock?

You can request an increase for account-level Service Quotas (Tokens Per Minute and Requests Per Minute) via the AWS Service Quotas console or AWS Support. However, you cannot increase a foundation model's architectural token limits, such as Claude 3.5 Sonnet's 8,192 maximum output token cap or Nova Premier's 10,000 output cap. Those ceilings are permanent design constraints of the model weights and runtime architecture.

### What is the maxTokens parameter in Bedrock Converse API?

The maxTokens parameter is an optional integer field inside the inferenceConfig object of the Bedrock Converse API. It defines the maximum number of new tokens the model can generate in a single response. It accepts a minimum value of 1. If omitted, Amazon Bedrock automatically defaults maxTokens to the targeted foundation model's maximum supported output ceiling.

### What happens when an AWS Bedrock response reaches maxTokens?

When a generation hits the configured maxTokens limit before completing naturally, generation stops immediately, and Bedrock sets the stopReason field in the response payload to 'max_tokens' (or 'length' in native Meta Llama calls). The returned content may end mid-sentence. Applications can detect this status and initiate a follow-up continuation prompt to resume generation.

### How does AWS Bedrock bill for tokens?

Amazon Bedrock bills on-demand inference based on the number of input tokens processed and output tokens generated, priced per 1,000 or per 1,000,000 tokens. Output tokens carry a higher price point than input tokens due to the computational intensity of autoregressive generation. For models supporting prompt caching, such as Anthropic Claude and Amazon Nova, cached input tokens are billed at a substantial discount compared to standard input rates.

## Sources

- [AWS General Reference: Amazon Bedrock endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/bedrock.html): According to the AWS General Reference on Amazon Bedrock endpoints and quotas, AWS Bedrock enforces cross-region service quotas on tokens per minute, allocating 800,000 tokens per minute for Anthropic Claude 3.5 Sonnet across most supported regions.
- [Anthropic Help Center: Upload files to Claude](https://support.claude.com/en/articles/8241126-upload-files-to-claude): As documented in Anthropic's help center on uploading files, Claude project uploads accept an unlimited number of files up to 30MB each, but total content must fit within Claude's context window.

## About Fast.io

Fast.io provides shared workspaces where people and AI agents work on the same files, with built-in semantic search and citation-backed chat over what they hold. Agents reach it through a remote MCP server, a REST API at https://api.fast.io/current/, and a command line client published on npm as @vividengine/fastio-cli. MCP setup is at https://mcp.fast.io/docs: Claude and most MCP clients connect to https://mcp.fast.io/mcp/tools, ChatGPT to https://mcp.fast.io/mcp/operations, and coding agents to https://mcp.fast.io/mcp/code.