ChromaDB Max Batch Size: Ingestion Limits and Batching Patterns

ChromaDB max batch size is the maximum number of items (embeddings, documents, and metadatas) that can be inserted into a Chroma collection in a single .add() or .upsert() call, governed by client.get_max_batch_size().

When developers attempt to ingest large datasets into ChromaDB in a single operation, execution frequently halts with an unhandled exception: ValueError: Batch size [X] exceeds maximum batch size 41666 or ValueError: Number of items to add exceeds max batch size. This error is not an arbitrary client throttling rule. It is a protective boundary enforced by the Chroma client to prevent memory crashes, socket timeouts, and query failures in the underlying storage engine.

+-------------------------------------------------------------------------+
|                        Application Ingestion Loop                       |
|           (Reads 50,000 documents from disk, S3, or API)                |
+-------------------------------------------------------------------------+
                                     |
                       collection.add(ids, docs, ...)
                                     |
                                     v
+-------------------------------------------------------------------------+
|                           Chroma Client Layer                           |
|       Validates batch length against client.get_max_batch_size()        |
+-------------------------------------------------------------------------+
         |                                                 |
  (Batch <= Limit)                                  (Batch > Limit)
         |                                                 |
         v                                                 v
  Chunks translated to SQL                          Raises ValueError:
  parameters for SQLite insert                      "Batch size exceeds
         |                                          maximum batch size"
         v
+-------------------------------------------------------------------------+
|                          SQLite Storage Engine                          |
|             Bound by SQLITE_MAX_VARIABLE_NUMBER parameter               |
+-------------------------------------------------------------------------+

The Transition from max_batch_size to get_max_batch_size

In earlier versions of Chroma, developers checked the batch ceiling by inspecting the client.max_batch_size property. In Chroma version 0.5.1, the project removed this property and replaced it with the client.get_max_batch_size() method. The maintainers made this change because determining the batch ceiling can require a network round trip or storage engine handshake on initial execution. Turning the property into an explicit method call signals to developers that querying the limit is a potentially blocking operation.

As outlined in the Chroma collection documentation, records can include document strings, pre-computed embeddings, metadata objects, and custom IDs. The returned batch size reflects the maximum number of records that the client configuration can process in one call:

Understanding which deployment mode you run determines whether your primary constraint is local SQLite parameter limits, HTTP transport latency, or managed multi-tenant API quotas.

• How to Manage LangChain Token Limits: Memory, Pruning, and MCP WorkspacesUnderstanding the LangChain token limit helps developers prevent context overflow errors in complex LLM pipelines. This...
• Mistral Token Limits: API Parameters, Generation Ceilings, and Workspace ArchitectureA Mistral token limit specifies the maximum number of tokens accepted in an API request payload or emitted during a...
• AWS Strands Agents: Build AI Agents with Amazon's Agent FrameworkAWS Strands Agents is Amazon's open-source Python framework for building AI agents that use tools, maintain memory, and...
• How to Build an AI Agent for Customer ServiceAI customer service agents now resolve 55-70% of tier-1 support tickets without human help, at roughly one-tenth the...
• How to Build an AI Agent Deep Research WorkflowA deep research workflow is an agentic pipeline that decomposes complex questions, searches multiple sources,...
• How to Optimize Token Costs for AI AgentsRunning autonomous AI agents gets expensive. Production sessions can cost $10-100+ if you aren't careful.

More on this subject: AI Agents: General Guides (172 guides)

Chroma relies on SQLite for metadata storage, document records, and collection coordination. Every record added to a collection contains multiple attributes: a unique string identifier, document text, embedding vectors or pointers, metadata dictionaries, and timestamps. When Chroma writes records to disk, it constructs parameterized SQL statements using question mark (?) placeholders to prevent SQL injection and format values correctly.

SQLite establishes a strict ceiling on the number of host parameter placeholders permitted in a single SQL statement. According to official SQLite documentation, the maximum value of a host parameter number is SQLITE_MAX_VARIABLE_NUMBER, which defaults to 999 for SQLite versions prior to 3.32.0 (2020-05-22) or 32766 for SQLite versions after 3.32.0. Detailed specifications can be verified directly in the official SQLite limits documentation.

The Mathematics Behind the 5,461 and 41,666 Limits

When Chroma executes a batch insert across multiple database columns, each row requires multiple parameter placeholders in the compiled SQL statement.

1. The 5,461 limit on standard SQLite: In an internal insert statement binding 6 parameters per record (such as id, document, embedding, metadata, creation timestamp, and sequence key), a single SQL statement reaches the SQLite 3.32.0 variable ceiling at exactly: 32,766 / 6 = 5,461 records. If your operating system uses a default build of SQLite, calling client.get_max_batch_size() or running batch inserts on a collection encounters this 5,461-item boundary.

2. The 41,666 limit on Chroma custom builds: To support higher ingestion throughput, Chroma's bundled binaries and container environments compile SQLite with a custom SQLITE_MAX_VARIABLE_NUMBER setting of 250,000. Dividing 250,000 by 6 parameters per row yields: 250,000 / 6 = 41,666.67 records. Chroma rounds this figure down to 41,666, which is the exact number developers observe in standard Docker images and Python wheels.

3. The 166 limit on legacy Linux distributions: Older Linux installations (such as Debian Buster or older enterprise distributions) package Python linked against SQLite versions older than 3.32.0. In those environments, SQLITE_MAX_VARIABLE_NUMBER remains at 999. Dividing 999 by 6 variables produces a limit of only 166 records per batch.

### install pysqlite3 if host sqlite is below version 3.35
pip install pysqlite3

In your application entry point, override the standard library SQLite module before importing Chroma:

__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')

import chromadb
client = chromadb.PersistentClient(path="./my_local_db")
print(f"Max batch size: {client.get_max_batch_size()}")

This configuration ensures your Python runtime uses a modern SQLite engine with the higher variable limit, avoiding unexpected crashes on legacy distributions.

Tutorials often demonstrate Chroma ingestion by adding three or four hardcoded documents in a single call. While that works for quick evaluation, deploying ingestion pipelines against thousands of documents requires systematic chunking. Even if client.get_max_batch_size() reports 41,666 items, sending thousands of documents in a single call is an operational anti-pattern.

Why 500 to 1,000 Items Is the Production Sweet Spot

In production environments, practical batch sizes should remain between 500 and 1,000 items rather than pushing the theoretical ceiling:

• Memory footprint: If Chroma generates embeddings locally using its default sentence-transformers model, computing embeddings for large text chunks in one pass allocates multiple gigabytes of RAM, often triggering an operating system Out-Of-Memory (OOM) kill.
• Network payload size: In client-server deployments, a batch of tens of thousands of records containing text documents and high-dimensional float vectors creates massive JSON payloads. Most HTTP reverse proxies, API gateways, and load balancers reject oversized requests with HTTP status 413 Request Entity Too Large.
• Transaction blast radius: Chroma writes batches within database transactions. If an individual record in a large batch contains malformed metadata or a corrupted character, the entire batch fails, forcing the pipeline to re-process and re-embed all preceding items.

Production Batching Generator Pattern

The following Python pattern dynamically inspects the client limit, applies a conservative production ceiling, and processes data using generator slices:

import time
from typing import Generator
import chromadb

def chunk_payloads(
    ids: list[str],
    documents: list[str],
    metadatas: list[dict],
    batch_size: int
) -> Generator[dict, None, None]:
    """Yield synchronized slices of IDs, documents, and metadata."""
    total_records = len(ids)
    for start_idx in range(0, total_records, batch_size):
        end_idx = min(start_idx + batch_size, total_records)
        yield {
            "ids": ids[start_idx:end_idx],
            "documents": documents[start_idx:end_idx],
            "metadatas": metadatas[start_idx:end_idx],
        }

def ingest_collection_safely(
    collection,
    client,
    ids: list[str],
    documents: list[str],
    metadatas: list[dict],
    max_chunk_size: int = 1000
) -> None:
    system_limit = client.get_max_batch_size()
    effective_batch_size = min(system_limit, max_chunk_size)
    print(f"Ingesting {len(ids)} items using batch size: {effective_batch_size}")
    for batch in chunk_payloads(ids, documents, metadatas, effective_batch_size):
        retries = 3
        backoff_delay = 1.0
        while retries > 0:
            try:
                collection.upsert(
                    ids=batch["ids"],
                    documents=batch["documents"],
                    metadatas=batch["metadatas"]
                )
                break
            except Exception as err:
                retries -= 1
                if retries == 0:
                    print(f"Failed batch starting with ID {batch['ids'][0]}: {err}")
                    raise err
                print(f"Transient error: {err}. Retrying in {backoff_delay}s...")
                time.sleep(backoff_delay)
                backoff_delay *= 2.0

Using .upsert() rather than .add() provides idempotency. If a network interruption occurs halfway through a multi-batch import, re-running the script updates modified records without raising duplicate key errors.

Index Large Document Collections Without ChromaDB Batch Size Limits

Upload or sync files to a Fast.io workspace and enable Intelligence Mode for automatic semantic indexing, hybrid search, and MCP agent access without managing batch limits. Monthly plans start with a 14-day free trial.

Start 14-Day Trial

Beyond the primary ValueError on batch size, vector ingestion pipelines face several recurring failure modes when scaling to production workloads.

Memory Exhaustion During Synchronous Embedding Generation

When you pass strings to the documents argument without supplying pre-computed vectors, Chroma invokes its embedding function automatically. If you pass thousands of documents at once, Chroma attempts to embed the entire batch in memory before writing to disk.

To prevent memory spikes:

1. Pre-compute embeddings in a separate worker process using dedicated micro-batches.
2. Supply both documents and embeddings explicitly to collection.add() or collection.upsert(). When vectors are passed explicitly, Chroma skips embedding calculation and writes the values directly to storage.

### pass pre-computed embeddings to avoid local model inference overhead
collection.add(
    ids=["doc_1", "doc_2"],
    documents=["First text chunk", "Second text chunk"],
    embeddings=[[0.021, -0.443, 0.129], [0.091, 0.112, -0.321]],
    metadatas=[{"source": "wiki"}, {"source": "wiki"}]
)

Duplicate IDs Within a Single Batch

While Chroma's .add() method ignores records if their IDs already exist in the database, submitting duplicate IDs within the same batch throws an immediate DuplicateIDError. Before chunking your data, verify that your ID generator produces unique keys across the entire array. Using content hashes (such as SHA-256 of the document text and source path) prevents duplicate keys from entering the batch pipeline.

Chroma Cloud Quotas and Write Ceilings

If you migrate from self-hosted Chroma to Chroma Cloud, local batching assumptions will break. Chroma Cloud enforces multi-tenant stability through strict quotas:

• Maximum records per write: 300 records per operation.
• Maximum document bytes: 16,384 bytes per document.
• Maximum metadata value size: 8,182 bytes per key.
• Maximum number of record metadata keys: 32 keys per record.

Code written for a local instance sending thousands of items per batch will fail on Chroma Cloud with HTTP 400 Bad Request or HTTP 429 Too Many Requests. When writing cross-environment code, inspect whether you connect to a cloud endpoint and adjust chunk size to 300 or lower.

SQLite Write Lock Contention

In PersistentClient mode, SQLite uses file-level locking during writes. If multiple ingestion processes or background threads attempt to call .add() concurrently on the same directory, workers encounter sqlite3.OperationalError: database is locked. For high-throughput concurrent ingestion, run Chroma in client-server mode (HttpClient), which queues write transactions through a single server daemon.

Managing custom vector databases, batch chunking scripts, SQLite parameter compilation flags, and lock contention is necessary when building proprietary search engines. However, for engineering teams that need AI agents and human teammates to query business documents, maintaining dedicated vector infrastructure introduces significant operational drag.

Every custom ingestion pipeline requires monitoring: handling schema migrations, managing embedding model versions, retrying failed batches, and maintaining synchronization when source files update. When files change in external storage, developers must write synchronization scripts to delete stale vectors and re-embed modified passages.

Storage and Retrieval Alternatives

Engineering teams generally evaluate three patterns for document knowledge:

1. Local storage and raw vector databases: Running ChromaDB locally is fast for single-developer prototyping. However, it isolates files on one machine, provides no multi-user access control, and requires custom batching scripts to avoid SQLite variable limits.
2. Cloud object storage (S3, Google Cloud Storage) with external RAG: Storing files in raw object storage scales storage capacity, but files remain unindexed blobs. Making documents searchable requires building an external pipeline: file ingestion triggers, parsing workers, chunking scripts, embedding APIs, and an external vector database.
3. Intelligent Workspaces on Fast.io: Fast.io persistent storage provides shared, org-owned cloud workspaces where files are automatically indexed for search and retrieval once Intelligence Mode is enabled.

+-------------------------------------------------------------------------+
|                         Team Document Sources                           |
|       (PDFs, Word Docs, Presentations, Spreadsheets, Scans)             |
+-------------------------------------------------------------------------+
                                     |
                       Upload, URL Import, or Cloud Sync
                                     |
                                     v
+-------------------------------------------------------------------------+
|                       Fast.io Shared Workspace                          |
|  - Stored in persistent, org-owned workspace boundaries                 |
|  - Full version history keeps every document revision auditable         |
|  - Granular permissions at org, workspace, folder, and file level       |
+-------------------------------------------------------------------------+
                                     |
                             Intelligence Mode
                                     |
                                     v
+-------------------------------------------------------------------------+
|                      Automatic Ingestion & Search                       |
|  - Automatic document parsing, text chunking, and semantic indexing     |
|  - Hybrid search: full-text keyword + semantic vector + metadata        |
|  - Structured extraction via Metadata Views (natural language schemas)  |
+-------------------------------------------------------------------------+
                                     |
                   Streamable HTTP Remote MCP Endpoint
                                     |
                                     v
+-------------------------------------------------------------------------+
|                         Autonomous AI Agents                            |
|             (Claude, Cursor, Codex, OpenClaw, Custom Code)              |
+-------------------------------------------------------------------------+

How Fast.io Replaces Manual Batch Ingestion

Instead of writing batch generators and tuning SQLite parameters, teams add documents directly to a Fast.io workspace. Fast.io indexes incoming files automatically:

• Automatic indexing: When files are uploaded or synced, Intelligence Mode parses document structure, generates semantic representations, and makes content queryable through hybrid search (combining full-text keyword matching, semantic vectors, and metadata filtering).
• Metadata Views: For structured extraction, Metadata Views allow teams to describe target fields in natural language. The system extracts typed values (dates, counterparties, totals, status flags) into a live, sortable spreadsheet without custom OCR rules or template training.
• Remote MCP access: Autonomous agents connect to the workspace using the Fast.io remote Model Context Protocol (MCP) server over Streamable HTTP at https://mcp.fast.io/mcp/tools (for general clients like Claude) or https://mcp.fast.io/mcp/code (for coding agents like Claude Code, Cursor, and Copilot). For configuration details and setup commands, see the Fast.io MCP documentation. Agents search workspace files and retrieve cited answers directly without handling local vector indexes.
• Cloud Sync: Bring existing archives into workspaces using Cloud Sync for Google Drive, Dropbox, Box, and OneDrive on a schedule or on demand.
• Collaboration and auditability: Workspaces include real-time Collaborative Notes for shared human-agent planning, an append-only audit log, per-file version history, and file leases for coordinating concurrent writes.

Creating an account is free; doing real work requires an organization on a paid subscription. Monthly plans start with a 14-day free trial that requires a credit card. Subscriptions are structured transparently on Fast.io pricing: Starter costs $9.99/mo (includes 3 seats, 250 GB storage, 5 workspaces, and 100,000 credits monthly), Business costs $49.99/mo (includes 10 seats, 5 TB storage, 50 workspaces, and 600,000 credits monthly), and Enterprise costs $199.99/mo (includes 30 seats, 25 TB storage, 200 workspaces, and 3,000,000 credits monthly). Annual plan billing is $99, $499, and $1,999 annually. Credits meter AI work. Storage and seats come included with each subscription plan.