Ollama Memory Limit: VRAM Allocation, OOM Fixes, and Workspace RAG

An Ollama memory limit is the operational ceiling on system RAM and GPU VRAM allocated for model weights, KV cache slots, and concurrent request buffers during local inference. When developers run open-weights language models on local workstations or private servers, memory management dictates whether generation succeeds with low latency or abruptly crashes with an out-of-memory fault.

Context KV caches scale linearly with sequence length and batch slots, frequently causing CUDA OOM on GPUs with under 16 GB VRAM. Many developer guides confuse token context limits with physical RAM and VRAM memory limits. A token context limit, configured through the num_ctx parameter or OLLAMA_CONTEXT_LENGTH, represents sequence length capacity. In contrast, physical memory usage represents the exact hardware memory in gigabytes required to hold active weights, attention states, and runtime tensors. A model configured for a large token context does not merely read longer text; it reserves dedicated physical memory that can easily exceed host capacity.

This architectural reality mirrors constraints encountered across cloud platforms. In official Anthropic documentation, a Claude chat conversation accepts up to 20 files at up to 500MB each, while a project accepts files up to 30MB each with no fixed file count limit, constrained by the context window. Claude Projects has no fixed file-count cap, so any fixed project file count is false. The practical ceiling on a project is the context window, and reaching it is when teams with a large corpus look for another path. On local runtimes like Ollama, the ceiling is even more immediate: memory is bound by physical VRAM and system RAM.

Ollama allocates memory across three primary operational tiers:

1. Static Model Weights: The base memory required to load model layers into GPU VRAM or system RAM. For a model quantized at 4-bit precision (such as Q4_K_M), memory demand scales with parameter count: approximately 4.9 GB for an 8B model, 9.0 GB for a 14B model, 20.0 GB for a 32B model, and 43.0 GB for a 70B model. These weights occupy static memory for as long as the model stays active.

2. Dynamic Key-Value (KV) Cache: The attention state memory allocated for active prompt tokens and generated responses. Modern transformer architectures store Key and Value vectors for previous tokens to avoid recomputing self-attention at each generation step. The memory required for the KV cache scales with the configured context length, number of layers, key-value heads, head dimension, and numerical precision. On models using Grouped-Query Attention (GQA), such as Llama 3.1 8B, the KV cache requires approximately 128 KiB per token in 16-bit floating-point precision, consuming roughly 4.0 GB of VRAM at 32,768 tokens and 8.0 GB at 64,000 tokens.

3. Compute Buffers and Runtime Overhead: Ephemeral memory allocated by the CUDA runtime, scratchpad buffers for intermediate activations, and memory reserved by the host operating system or display manager. Running a local desktop environment alongside Ollama often consumes 1.0 to 2.0 GB of VRAM before Ollama loads its first tensor.

The table below illustrates total VRAM requirements across common model sizes and context lengths, combining static weights with unquantized 16-bit KV cache allocations:

When total required memory exceeds physical GPU VRAM, Ollama does not immediately terminate. Instead, its scheduler offloads partial model layers to system RAM using CPU inference. While this mechanism prevents a hard crash, moving tensors across the host PCIe bus reduces inference throughput from dozens of tokens per second to a crawl, stalling agent workflows.

• Llama 3.1 Context Window: 128K Tokens, RoPE Scaling, and Hardware NeedsThe Llama 3.1 context window is 128,000 tokens across all model sizes (8B, 70B, and 405B), enabled by an adjusted RoPE...
• AWS Bedrock Context Window: Token Limits, Model Capacities, and Memory ArchitectureThe AWS Bedrock context window defines the maximum sequence of input and output tokens a hosted foundation model...
• ChatGPT Memory Full: How to Fix, Clear Space, and Externalize KnowledgeWhen ChatGPT memory reaches full capacity, the model stops recording new personal facts and project context across...
• Gemini 2.0 Flash Context Window: 1M Architecture and RAG Best PracticesThe Gemini 2.0 Flash context window spans 1,048,576 input tokens and an 8,192-token output ceiling. While ingesting...
• Google Drive RAG: How to Build Retrieval-Augmented Generation Over Drive FilesGoogle Drive RAG is an architecture where AI agents retrieve relevant document chunks from Google Drive storage using...
• Connecting AI Agents to Google Workspace: Native Tools, API Limits, and Remote MCP StorageA Google Workspace agent coordinates tasks across Google Drive, Docs, and Sheets, but external agents face steep...

More on this subject: Agent Memory and Storage (310 guides)

Resolving Ollama memory pressure requires proactive configuration of model residency, concurrency ceilings, memory-efficient attention, and external retrieval architecture. Applying this four-step checklist stabilizes local inference on consumer and server hardware:

1. Tune OLLAMA_KEEP_ALIVE: Ollama automatically unloads idle models after 5 minutes by default via OLLAMA_KEEP_ALIVE. If you query a model and stop, it occupies memory for 300 seconds. Setting this duration lower frees memory promptly for other applications, while setting it to 0 unloads the model immediately after completion.

2. Cap OLLAMA_MAX_LOADED_MODELS to 1: By default, Ollama attempts to load multiple models into memory when requested, assuming sufficient capacity. Capping this parameter to 1 ensures that requesting a new model unloads the previous model using a least-recently-used policy, preventing multi-model VRAM exhaustion.

3. Enable Flash Attention (OLLAMA_FLASH_ATTENTION=1): Enabling Flash Attention activates optimized memory access kernels in the underlying inference engine. Flash Attention avoids materializing intermediate attention matrices in VRAM, reducing memory consumption during prompt evaluation and long-context processing.

4. Externalize Document Collections to an Indexed MCP Workspace: Rather than expanding local context windows to 64,000 or 128,000 tokens to process entire documentation sets, store reference files in an external workspace. Connecting local models to indexed workspaces via Model Context Protocol (MCP) retrieves targeted context passages on demand, keeping local memory footprint predictable.

Applying Server Configuration Across Environments

Depending on your operating system, configure these environment variables before starting the Ollama service:

Linux Deployments with systemd: To persist settings across reboots, edit the service override file:

sudo systemctl edit ollama.service

Insert the environment configuration under the service unit block:

[Service=unit]
Environment="OLLAMA_KEEP_ALIVE=2m"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_GPU_OVERHEAD=1073741824"

Save the file and restart the systemd daemon:

sudo systemctl daemon-reload
sudo systemctl restart ollama

macOS Desktop Installations: On macOS, GUI applications do not inherit shell environment variables declared in shell profile scripts. Register the parameters with launchctl before launching the application:

launchctl setenv OLLAMA_KEEP_ALIVE "2m"
launchctl setenv OLLAMA_MAX_LOADED_MODELS "1"
launchctl setenv OLLAMA_FLASH_ATTENTION "1"

Restart the Ollama desktop app from the Applications folder or menu bar to apply the changes.

Docker and Container Deployments: When running Ollama inside Docker, pass the environment flags directly in your run command or compose specification:

docker run -d \
  --gpus all \
  -e OLLAMA_KEEP_ALIVE=2m \
  -e OLLAMA_MAX_LOADED_MODELS=1 \
  -e OLLAMA_FLASH_ATTENTION=1 \
  -v ollama_data:/root/.ollama \
  -p 11434:11434 \
  --name ollama \
  ollama/ollama:latest

Setting these baseline variables establishes strict memory boundaries, preventing background tasks from accumulating unreleased VRAM.

When Ollama exhausts host memory, the failure manifests as either a CUDA driver error or an operating system process termination. Understanding the diagnostic signals distinguishes between GPU-specific memory exhaustion and system-wide memory collapse.

CUDA Out of Memory (CUDA OOM)

A CUDA OOM error occurs when the Ollama runner requests a VRAM allocation from the NVIDIA GPU driver that exceeds available video memory. This failure typically appears in server logs as:

cuda error: out of memory
ggml_cuda_init: failed to allocate CUDA buffer

This error emerges during two distinct runtime phases:

• Initial Model Loading: The runner calculates layer offload requirements based on model weight tensors. If the calculated requirement leaves no room for CUDA runtime contexts or display buffers, loading fails.
• Context Generation Expansion: The model loads successfully, but a user prompt containing thousands of tokens forces the KV cache to allocate additional VRAM slots mid-stream. If the GPU runs out of unfragmented blocks, inference crashes immediately.

To prevent initial loading crashes caused by desktop environment overhead, set OLLAMA_GPU_OVERHEAD. This variable instructs Ollama to reserve a specific number of bytes per GPU exclusively for non-Ollama processes:

export OLLAMA_GPU_OVERHEAD=1073741824

Linux Kernel OOM Killer and Exit Code 137

Exit code 137 indicates that the operating system terminated the Ollama process with SIGKILL (Signal 9). The exit code calculation is 128 plus signal number 9, totaling 137.

When Ollama runs models that exceed physical GPU VRAM, it offloads remaining layers and memory buffers to system RAM. If system RAM and swap space become completely full, the Linux kernel Out-Of-Memory (OOM) killer activates. To protect operating system stability, the kernel selects and terminates the highest memory-consuming user-space process, which is almost always the Ollama runner daemon.

To verify whether the OOM killer terminated Ollama, inspect kernel diagnostic messages and systemd service logs:

sudo dmesg -T | grep -i oom
journalctl -u ollama --no-pager -n 50

If the logs show that the kernel killed the process due to memory exhaustion, system RAM was depleted by CPU layer offloading or concurrent model loads.

In Docker environments on macOS or Windows, exit code 137 frequently stems from virtual machine memory caps. Docker Desktop runs inside a lightweight Linux VM with a fixed RAM ceiling. If you run a model requiring 6 GB of RAM inside a container limited to 4 GB, the VM kernel terminates the container immediately. Adjust the memory allocation in Docker Desktop resource settings to provide adequate headroom for your models.

Operational Telemetry with ollama ps

To inspect running models, VRAM usage, and active execution targets in real time, run:

ollama ps

The output reveals active memory distribution:

NAME               ID              SIZE      PROCESSOR        UNTIL              CONTEXT
llama3.1:8b        8eeb52625293    4.9 GB    100% GPU         2 minutes from now 4096

Inspect the PROCESSOR column closely:

• Pure GPU execution: All transformer layers reside in video memory, delivering optimal token throughput.
• Partial GPU and CPU split: Model weights and the KV cache exceeded physical VRAM, forcing Ollama to split layers across PCIe. This indicates that current context or model parameters are too large for your GPU.
• Pure CPU execution: Ollama could not access GPU drivers or VRAM was insufficient for even the base layers, resulting in slow CPU inference.

In addition, review API response metrics when integrating with automated pipelines. Ollama responses return prompt_eval_count and prompt_eval_duration. If prompt_eval_count is lower than the token count of your input prompt, Ollama truncated input text to fit within the active num_ctx ceiling.

Stop Ollama Out of Memory Errors with Remote MCP Workspaces

Store reference files, project documentation, and large datasets in Fast.io workspaces with automated hybrid indexing, connecting local Ollama agents through remote Model Context Protocol tools. Monthly plans start with a 14-day free trial, which requires a credit card.

Start 14-Day Trial

Autonomous agent systems and complex AI workflows rarely rely on a single model. An automated pipeline might use a lightweight routing model to classify user intent, a specialized embedding model to calculate vector similarities, and a larger conversational model to generate final responses. Running these models sequentially or concurrently creates severe memory volatility if residency lifecycles are unmanaged.

Understanding Concurrency Multipliers

When multiple requests hit Ollama simultaneously, memory demand multiplies across two dimensions:

1. Parallel Request Slots (OLLAMA_NUM_PARALLEL): Ollama supports processing concurrent prompts by creating parallel evaluation slots. However, each parallel slot maintains its own separate Key-Value cache. If an 8B model requires 4.0 GB of VRAM for a single 32,768-token KV cache, handling 4 parallel requests at that context length demands 16.0 GB of VRAM just for the attention cache, easily causing sudden OOM crashes. Restrict concurrency on memory-limited hardware:

export OLLAMA_NUM_PARALLEL=1

2. Loaded Model Count (OLLAMA_MAX_LOADED_MODELS): If an agent executes an embedding model (nomic-embed-text) to index a document and immediately calls llama3.1:8b to answer a question, Ollama retains the embedding model in VRAM while loading the generation model. On an 8 GB or 12 GB GPU, both models cannot coexist in VRAM simultaneously.

Setting OLLAMA_MAX_LOADED_MODELS=1 enforces an eviction sequence: Ollama automatically unloads the embedding model before allocating memory for the generation model. While model switching introduces a brief loading delay, it eliminates out-of-memory crashes.

Controlling Lifecycles via API Requests

Rather than relying purely on global server timers, applications can manage model residency dynamically on a per-request basis. The Ollama REST API supports the keep_alive parameter across /api/generate, /api/chat, and /api/embed.

For transient operations, such as generating an embedding or routing a single classification, command Ollama to unload the model immediately:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Classify the following query category: account billing",
  "keep_alive": 0,
  "options": {
    "num_ctx": 2048
  }
}'

Setting "keep_alive": 0 instructs Ollama to evict model weights from VRAM as soon as the response finishes streaming, leaving the entire GPU memory pool free for subsequent tasks.

Conversely, for sustained conversational workflows, pass "keep_alive": "30m" to prevent model reloads between user turns:

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1:8b",
  "messages": [
    {"role": "user", "content": "Analyze our system architecture design."}
  ],
  "keep_alive": "30m",
  "options": {
    "num_ctx": 8192
  }
}'

Baking Memory Constraints into Custom Modelfiles

For team environments where scripts and interactive users share an Ollama server, creating custom Modelfiles guarantees reproducible memory allocations. Create a file named Modelfile:

FROM llama3.1:8b
PARAMETER num_ctx 8192
PARAMETER num_gpu 28
PARAMETER temperature 0.2

Build and tag the constrained model:

ollama create llama3.1-constrained -f Modelfile

Running ollama run llama3.1-constrained guarantees that the model respects these memory boundaries regardless of client request headers, preventing runaway token allocations from exhausting host VRAM.

Stuffing large document libraries, codebase repositories, and architectural specifications directly into local LLM context creates severe operational bottlenecks. As context windows expand, Time-to-First-Token latency degrades, power consumption spikes, and the risk of attention dilution increases.

In official Ollama documentation for context length, tasks that require large context such as web search, agents, and coding tools should be set to at least 64,000 tokens when hardware allows. However, running a 64,000-token context window on an 8B model requires substantial dedicated VRAM, leaving negligible headroom on consumer hardware and causing immediate crashes on larger models.

Engineering teams building agentic workflows evaluate three primary architectural paths for processing large document collections:

• Raw Prompt Stuffing: Concatenating full text into prompts. This approach consumes extensive KV cache memory, causes CUDA OOM crashes, triggers slow CPU offloading, and risks silent prompt truncation.
• Self-Hosted Local Vector Databases: Operating vector databases such as Chroma, Qdrant, or Milvus alongside Ollama. While this pattern reduces context size, it forces teams to maintain chunking pipelines, run local embedding servers, manage disk indexes, and handle index synchronization.
• Intelligent Cloud Workspaces via Model Context Protocol (MCP): Storing files in organized cloud workspaces that index content automatically and expose standardized search tools to local models over Streamable HTTP.

Fast.io workspaces provide an intelligent workspace layer specifically built for teams using storage for agents. Instead of attaching large files to chat prompts or overloading local VRAM with extended token contexts, documents are stored in shared, persistent workspaces:

• Automated Indexing with Intelligence Mode: Once Intelligence Mode is enabled on a workspace, all uploaded files are automatically indexed for hybrid search, combining full-text keyword indexing, semantic meaning, and metadata retrieval without requiring external vector databases.
• Streamable HTTP MCP Integration: Local models, coding assistants, and agent runtimes connect to Fast.io through remote Model Context Protocol endpoints. Clients connect with OAuth or scoped tokens at https://mcp.fast.io/mcp/tools (for general assistants) or https://mcp.fast.io/mcp/code (for coding agents). Detailed connection procedures and endpoint configurations are available in the Fast.io MCP documentation.
• Multi-Source Synchronization: Cloud Sync connects Google Drive, Dropbox, Box and OneDrive folders to a workspace, one-way or two-way, on a schedule or on demand. Reference documents update in the workspace automatically without manual file transfers.
• Structured Data Extraction: For workflows requiring tabular analysis across contracts, financial reports, or technical specifications, Metadata Views extract typed schema fields directly from unstructured documents without manual data entry.
• Governance and Auditability: Every workspace maintains per-file version history and a detailed activity log of changes, tracking file uploads, modifications, and deletions across human members and autonomous agents.

The Hybrid Architecture in Practice

In this hybrid architecture, an agent running locally on Ollama maintains a lean, responsive context length of 4,096 to 8,192 tokens. Model weights and the compact KV cache fit entirely within physical GPU VRAM, ensuring fast inference speeds.

When a user asks a question about an extensive reference corpus, the agent does not attempt to ingest entire PDF manuals into its context window. Instead, the agent executes an MCP search call against the Fast.io workspace. The workspace performs hybrid semantic and keyword retrieval across the indexed library and returns only the relevant passages with source citations.

The local model synthesizes the answer from the retrieved excerpts, preserving local GPU VRAM, preventing out-of-memory terminations, and maintaining peak interactive responsiveness.