Milvus Limits and Quotas: RPC Sizes, Dimensions, and Storage Architecture

Embedding pipelines break when developers treat a vector database as an object store. Attempting to ingest dense embeddings alongside multi-megabyte raw documents into Milvus triggers gRPC buffer rejections and stalls query nodes during segment compaction. Production stability requires treating Milvus strictly as a high-dimensional index while routing source documents to dedicated workspace storage.

Milvus system limits define the operational boundaries of the vector database, including a 64 MB maximum RPC request size, 32,768 vector dimensions, and a 65,535 character maximum on VARCHAR fields. These system constraints exist to protect distributed cluster nodes from memory saturation, thread starvation, and runaway coordinator synchronization overhead.

Milvus structures its distributed processing across four distinct layers:

1. Proxy Layer: Stateless frontend nodes that receive client requests over gRPC or REST, validate collection schemas, serialize network buffers, and fan out queries to query nodes.
2. Coordinator Layer: Management services including RootCoord, DataCoord, and QueryCoord that manage data definitions, allocate segment IDs, track cluster topology, and record schema states inside etcd.
3. Worker Nodes: Stateful execution engines divided into DataNodes (which consume log broker messages and build raw data segments) and QueryNodes (which load indexed vector segments into memory to execute nearest-neighbor searches).
4. Storage Substrate: Distributed object storage for immutable historical segments, message brokers for WAL ingestion, and etcd for metadata state.

When client applications push against default operational limits, bottlenecks manifest at specific points within this pipeline. A request exceeding message bounds fails at the proxy before reaching worker nodes. Conversely, an oversized collection count burdens etcd and RootCoord, degrading cluster-wide scheduling performance.

Understanding these boundaries allows engineering teams to construct retrieval architectures that scale predictably without crashing cluster components during traffic spikes.

• How to Handle Grok Token Limits and Process Large FilesThe Grok token limit restricts context capacity to 131,072 tokens on Grok 2, 500,000 tokens on Grok 4.5 and Grok 4.6,...
• Git LFS Limits: File Caps, Bandwidth Quotas, and Storage WorkaroundsGit LFS limits on GitHub restrict single large file pointers to 2 GB on Free and Pro accounts, 4 GB on Team accounts,...
• Groq Token Limits: LPU Quotas, Max Tokens, and Retrieval ArchitectureA Groq token limit defines the maximum allowable input prompt tokens and completion tokens processed by Language...
• AWS Bedrock Token Limits: Model Output Caps, Converse API Parameters, and QuotasAn AWS Bedrock token limit refers to the parameter-enforced boundaries on input prompt length and maximum generated...
• Can ChatGPT Access OneDrive? Workflows, Admin Limits & Fast StorageChatGPT can access OneDrive files through its cloud storage connector, but personal accounts remain unsupported while...
• ChatGPT Canvas Limits: File Sizes, Document Length, and WorkaroundsChatGPT Canvas limits documents to practical thresholds of roughly 4,000 lines of code or 25,000 words, running in an...

More on this subject: AI Agents: General Guides (172 guides)

Client applications communicate with the Milvus proxy layer via gRPC protocol buffers. Milvus limits the maximum payload of an individual RPC request to 64MB by default. If a client attempts to insert a batch of vectors, run a massive multi-vector query, or submit a scalar filter expression that exceeds 67,108,864 bytes, the gRPC framing layer rejects the call immediately with a RESOURCE_EXHAUSTED status code.

This limit applies uniformly across insert, search, query, and upsert operations. While modifying the proxy.maxMessageSize setting in milvus.yaml can raise this threshold, increasing gRPC frame buffers introduces severe production risks. Large frames trigger memory spikes on proxy nodes, disrupt Go garbage collection routines, and monopolize network worker threads.

Calculating Safe Ingestion Batch Sizes

Determining safe batch sizes requires calculating the raw byte volume of vector embeddings combined with schema scalar fields.

A standard dense vector consists of 32-bit floating-point values (4 bytes per dimension). Consider an embedding model producing 1,536 dimensions, such as standard OpenAI or Cohere representations:

Vector Size = 1,536 dimensions * 4 bytes = 6,144 bytes (~6 KB)

If each entity also carries 1 KB of scalar metadata (such as entity identifiers, timestamps, category tags, and chunk offsets), the total entity footprint reaches approximately 7.1 KB:

Total Entity Size = 6,144 bytes + 1,024 bytes = 7,168 bytes (~7 KB)

At 7.1 KB per record, a batch of 2,000 records produces a network payload of roughly 14.3 MB. This stays well below the 64 MB ceiling, leaves ample room for protocol buffer framing overhead, and prevents network socket timeouts:

from pymilvus import Collection, MilvusException

def insert_entities_in_batches(
    collection: Collection, 
    entities: list[dict], 
    batch_size: int = 2000
) -> int:
    total_inserted = 0
    for i in range(0, len(entities), batch_size):
        chunk = entities[i : i + batch_size]
        try:
            mutation_result = collection.insert(chunk)
            total_inserted += mutation_result.insert_count
        except MilvusException as e:
            ### Handle transport failures or payload rejections
            raise RuntimeError(f"Ingestion failed at index {i}: {e.message}")
    return total_inserted

Operating with batch sizes between 1,000 and 3,000 entities balances network round trips against worker serialization overhead. It maintains steady throughput without causing memory pressure on proxy instances.

Bulk Ingestion Paths for Historical Data

When migrating historical datasets comprising hundreds of millions of embeddings, streaming inserts through gRPC proxies becomes an inefficient path. Repeated gRPC serialization consumes high CPU cycles across proxy nodes and generates thousands of small segment fragments that demand aggressive background compaction.

For large-scale data initialization, Milvus provides a bulk insertion utility that reads pre-formatted Parquet, JSON, or NumPy files directly from object storage. The DataCoord assigns import tasks directly to DataNodes, which pull the data files from object storage, build sealed segments, and write index files without passing payload bytes through client-facing gRPC connections. Bypassing the proxy RPC layer eliminates message size restrictions entirely during bulk ingestion.

Vector indexes are mathematical structures tuned for specific dimensional geometries. Milvus supports dense vectors spanning 32,768 dimensions in its default configuration, managed by the proxy.maxDimension parameter.

While the database accommodates high dimensions for specialized biological or chemical sequence representations, production text embedding models operate between 384 and 3,072 dimensions. Increasing dimensionality impacts memory consumption linearly for raw vector storage and superlinearly for graph-based vector indexes like HNSW (Hierarchical Navigable Small World).

Memory Footprint Across Dimensional Scales

An HNSW index constructs a multi-layer proximity graph where each vector maintains connections to neighbouring nodes. Memory consumption is dictated by vector dimensions, the number of graph links per node (M), and search expansion width (efConstruction):

Index Memory ≈ Num_Vectors * (Dimensions * 4 bytes + M * 8 bytes)

For 1,000,000 vectors at 1,536 dimensions with M = 16, raw vectors require substantial RAM, and HNSW graph links require additional memory on QueryNodes. Doubling the vector dimension doubles the working memory required for the same entity count on query nodes. If developers attempt to index extreme dimensionalities, memory requirements balloon rapidly, forcing query nodes into memory protection states and slowing search latencies.

Multi-Tenant Isolation: Collections Versus Partition Keys

A single Milvus instance allows up to 65,536 collections before reaching architectural limits. While this ceiling appears high, treating collections as isolated tenant containers causes acute infrastructure bottlenecks long before reaching the theoretical maximum.

Each collection in Milvus requires independent etcd metadata tracking, dedicated segment allocations, and isolated coordinator scheduling threads. When a cluster creates several thousand collections:

1. etcd Saturation: RootCoord stores schema definitions, segment maps, and partition catalogs in etcd. Tens of thousands of collections bloat the etcd key space, causing heartbeat timeouts and slow leader elections.
2. QueryNode Overhead: QueryNodes load collection segments into memory independently. Managing thousands of microscopic collections fragments memory pools and stalls query scheduling.
3. Compaction Inefficiency: DataNodes spend disproportionate CPU cycles compacting tiny segments across idle collections rather than optimizing active data.

The production pattern for multi-tenancy avoids creating separate collections per customer. Instead, architectures deploy a unified collection equipped with Partition Keys:

from pymilvus import CollectionSchema, FieldSchema, DataType

### Define unified collection with partition key
fields = [
    FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=True),
    FieldSchema(name="org_id", dtype=DataType.VARCHAR, max_length=64, is_partition_key=True),
    FieldSchema(name="document_id", dtype=DataType.VARCHAR, max_length=64),
    FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=1536)
]
schema = CollectionSchema(fields=fields, description="Multi-tenant knowledge base")

When an entity carries is_partition_key=True, Milvus hashes the org_id value and routes entities to one of 1,024 physical partitions internally. Searches targeting a specific org_id execute partition pruning automatically, scanning only the relevant partition without collection management overhead.

Managing VARCHAR Limits and Scalar Filtering Overheads

Milvus supports 64 fields per collection schema and caps VARCHAR fields at 65,535 characters. Storing extensive text bodies inside VARCHAR fields is a common design flaw in retrieval systems.

While VARCHAR fields allow scalar filtering expressions (such as doc_type == 'legal' and created_at > 1700000000), loading lengthy text strings into QueryNodes degrades search throughput. Scalar fields must be retained in memory alongside vector graphs. Retaining raw text inside VARCHAR columns multiplies the memory footprint, which reduces the cache space available for vector similarity search and increases node provisioning requirements.

Decouple Document Storage From Milvus Embeddings

Connect AI agents to persistent Fast.io workspaces through our remote MCP server. Store raw document corpuses and large payloads while indexing embeddings in Milvus. Every organization starts with a 14-day free trial, which requires a credit card.

Start 14-Day Trial

Most vector database guides outline configuration flags without addressing the primary architectural challenge: vector databases are purpose-built for vector indexing, not for storing unstructured files. Attempting to force full source documents into Milvus VARCHAR fields or dynamic JSON attributes creates brittle ingestion pipelines and inflates infrastructure costs.

Decoupling source files from vector embeddings resolves this constraint. In a decoupled architecture, data storage divides into two specialized tiers:

1. The Vector Index Tier (Milvus): Stores only high-dimensional embeddings, numeric entity identifiers, source file references, chunk positional offsets, and minimal filterable tags (such as organization IDs or document categories).
2. The Persistent Workspace Tier: Stores authoritative raw documents, multi-page PDFs, technical manuals, contracts, presentation decks, and structured metadata views.

Raw Documents, PDFs, and Media
             │
             ▼
┌───────────────────────────┐      Text Chunks     ┌───────────────────────────┐
│ Persistent File Storage   │ ───────────────────► │ Embedding Generator       │
│ (Fast.io Workspaces)      │                      └─────────────┬─────────────┘
└─────────────┬─────────────┘                                    │
              │                                                  ▼ Embeddings + IDs
              │ File ID & Chunk Position           ┌───────────────────────────┐
              └──────────────────────────────────► │ Vector Index (Milvus)     │
                                                   └───────────────────────────┘

Retrieval and Hydration Sequence

Separating vector indexes from unstructured files transforms the retrieval lifecycle into a fast, predictable two-stage pipeline:

1. Semantic Search: The client application converts user input into a vector embedding and queries Milvus. Milvus searches its high-dimensional graph, returning the top-k nearest neighbor entity IDs, similarity distances, and associated file_id pointers.
2. Document Hydration: The client application uses the retrieved file_id and chunk offsets to fetch the precise document text, surrounding context, or full source file from the workspace tier.

This separation keeps Milvus segments compact. A 1,000,000-entity collection containing only 1,536-dimensional vectors and 64-byte file pointers requires modest RAM on query nodes. If those same records contained lengthy text chunks inside VARCHAR fields, memory requirements would multiply, driving up cloud infrastructure expenditures.

Decoupled storage allows data teams to update vector embeddings independently of source files. When upgrading an embedding model from 768 to 1,536 dimensions, engineers re-process files stored in the persistent workspace without risking data loss or managing complex database migrations.

In modern AI agent architectures, vector similarity search represents only one component of the retrieval pipeline. Autonomous agents and human teams require shared, persistent workspaces where source files remain versioned, auditable, and accessible across multiple processing stages.

Fast.io provides dedicated cloud workspaces designed for agentic teams and automated pipelines. While Milvus handles fast mathematical vector search, Fast.io serves as the persistent unstructured file tier. Instead of storing large document text inside Milvus VARCHAR fields, applications store the authoritative documents inside Fast.io workspaces and pass file references to vector records.

Unified MCP Connectivity for AI Agents

Agents interact directly with Fast.io workspaces through the remote Model Context Protocol server. The MCP server runs over Streamable HTTP: Claude apps and general MCP clients connect at https://mcp.fast.io/mcp/tools, while coding agents such as Claude Code and Cursor connect at https://mcp.fast.io/mcp/code. Interactive clients sign in with OAuth in the browser and carry no API key, while headless code sends an Authorization: Bearer <api key> header on the connection. Setup steps are documented at https://mcp.fast.io/docs. Developers can inspect technical onboarding instructions at https://fast.io/llms.txt and review developer tool documentation in the Fast.io storage guide.

Through a consolidated MCP toolset, AI assistants such as Claude, Cursor, and automated coding agents can:

• Read and write source documents directly within shared workspaces.
• Search files using Fast.io Intelligence Mode, which auto-indexes documents for hybrid semantic and full-text keyword retrieval.
• Extract structured fields across documents into Metadata Views without writing manual regex rules or maintaining brittle OCR templates.
• Track changes through per-file version history, allowing agents and human reviewers to inspect earlier document iterations and restore prior drafts.
• Exchange deliverables with external partners through branded shares that support custom expiration dates and recipient access controls.

Implementation Pattern: Hybrid Milvus and Fast.io Retrieval

Developers building agent workflows connect to Milvus for vector candidate generation and to Fast.io for document context retrieval. This Python snippet illustrates the coordination pattern:

import requests
from pymilvus import Collection

def retrieve_and_hydrate(
    milvus_collection: Collection,
    query_vector: list[float],
    fastio_workspace_id: str,
    fastio_api_token: str,
    top_k: int = 3
) -> list[dict]:
    ### Stage 1: Vector similarity search in Milvus
    search_params = {"metric_type": "COSINE", "params": {"ef": 64}}
    results = milvus_collection.search(
        data=[query_vector],
        anns_field="embedding",
        param=search_params,
        limit=top_k,
        output_fields=["file_id", "chunk_index"]
    )
    
    hydrated_chunks = []
    
    ### Stage 2: Hydrate context from Fast.io MCP endpoint
    mcp_endpoint = "https://mcp.fast.io/mcp/tools"
    headers = {
        "Authorization": f"Bearer {fastio_api_token}",
        "Content-Type": "application/json"
    }
    
    for hit in results[0]:
        file_id = hit.entity.get("file_id")
        chunk_idx = hit.entity.get("chunk_index")
        
        mcp_payload = {
            "jsonrpc": "2.0",
            "method": "tools/call",
            "params": {
                "name": "storage",
                "arguments": {
                    "action": "content",
                    "workspace_id": fastio_workspace_id,
                    "file_id": file_id
                }
            },
            "id": chunk_idx
        }
        
        response = requests.post(mcp_endpoint, json=mcp_payload, headers=headers)
        if response.status_code == 200:
            file_data = response.json().get("result", {})
            hydrated_chunks.append({
                "score": hit.distance,
                "file_id": file_id,
                "content": file_data.get("content", ""),
                "version": file_data.get("version", 1)
            })
            
    return hydrated_chunks

This pattern isolates Milvus from file payload limits while granting agents instant access to full document contexts, version records, and metadata schemas. For teams managing diverse agent storage needs, detailed architectural guides are available at /storage-for-agents/. Review plan configurations and deployment options on the pricing page.

Operating Milvus reliably in production requires active monitoring of system watermarks and proactive quota configuration. When data nodes or query nodes approach memory saturation, Milvus initiates rate-limiting or force-denial mechanisms to protect cluster state.

Memory Protection Watermarks

Milvus query nodes allocate memory for index caching, execution engine scratch space, and search worker buffers.

In milvus.yaml, administrators configure memory protection boundaries under quotaAndLimits:

quotaAndLimits:
  enabled: true
  limitWriting:
    memProtection:
      enabled: true
      dataNodeMemoryLowWaterLevel: 0.85
      dataNodeMemoryHighWaterLevel: 0.95
      queryNodeMemoryLowWaterLevel: 0.85
      queryNodeMemoryHighWaterLevel: 0.90
    diskProtection:
      enabled: true
      diskQuota: 524288 # MB

When QueryNode memory usage crosses the low water level threshold, the proxy throttles incoming insert and search requests. If memory usage reaches the high water mark, the cluster rejects write operations outright to prevent out-of-memory terminations.

Production Observability Checklist

Platform engineers should track four critical metrics across Prometheus and Grafana dashboards:

1. gRPC Buffer Sizing (milvus_grpc_message_size_bytes): Monitors distribution of request payload sizes. Frequent payload size warnings in metrics indicate that client batch sizes should be adjusted downward.
2. QueryNode Memory Utilization (milvus_querynode_memory_usage_ratio): Tracks memory consumption against node limits. Sustained high memory utilization signals that cluster capacity requires horizontal scaling by adding query nodes.
3. Collection Metadata Count (milvus_rootcoord_collection_num): Tracks active collections within etcd. Sudden spikes toward thousands of collections indicate anti-pattern usage where applications are creating tenant collections dynamically.
4. Segment Compaction Latencies (milvus_datacoord_compact_latency_seconds): Measures how long DataNodes spend merging small segments. Spikes in compaction duration signal that client inserts are submitting excessively small batches.