# Ollama Memory Limit: VRAM Allocation, OOM Fixes, and Workspace RAG

An Ollama memory limit defines the operational boundary for GPU VRAM and system RAM during local model inference. Misconfigured context windows and multi-model concurrency frequently trigger CUDA out-of-memory errors or exit code 137 process kills. Setting OLLAMA_KEEP_ALIVE, capping OLLAMA_MAX_LOADED_MODELS, enabling flash attention, and externalizing large document collections to remote MCP workspaces keeps local models responsive under tight hardware budgets.

Source: https://fast.io/resources/ollama-memory-limit/
Author: [Derek Labian](https://fast.io/authors/derek-labian/)
Last reviewed: 2026-11-06

## What Is an Ollama Memory Limit and How Does It Allocate VRAM?

An Ollama memory limit is the operational ceiling on system RAM and GPU VRAM allocated for model weights, KV cache slots, and concurrent request buffers during local inference. When developers run open-weights language models on local workstations or private servers, memory management dictates whether generation succeeds with low latency or abruptly crashes with an out-of-memory fault.

Context KV caches scale linearly with sequence length and batch slots, frequently causing CUDA OOM on GPUs with under `16 GB` VRAM. Many developer guides confuse token context limits with physical RAM and VRAM memory limits. A token context limit, configured through the `num_ctx` parameter or `OLLAMA_CONTEXT_LENGTH`, represents sequence length capacity. In contrast, physical memory usage represents the exact hardware memory in gigabytes required to hold active weights, attention states, and runtime tensors. A model configured for a large token context does not merely read longer text; it reserves dedicated physical memory that can easily exceed host capacity.

This architectural reality mirrors constraints encountered across cloud platforms. In official Anthropic documentation, a Claude chat conversation accepts up to 20 files at up to 500MB each, while a project accepts files up to 30MB each with no fixed file count limit, constrained by the context window. Claude Projects has no fixed file-count cap, so any fixed project file count is false. The practical ceiling on a project is the context window, and reaching it is when teams with a large corpus look for another path. On local runtimes like Ollama, the ceiling is even more immediate: memory is bound by physical VRAM and system RAM.

Ollama allocates memory across three primary operational tiers:

1. **Static Model Weights:** The base memory required to load model layers into GPU VRAM or system RAM. For a model quantized at `4-bit` precision (such as `Q4_K_M`), memory demand scales with parameter count: approximately `4.9 GB` for an `8B` model, `9.0 GB` for a `14B` model, `20.0 GB` for a `32B` model, and `43.0 GB` for a `70B` model. These weights occupy static memory for as long as the model stays active.

2. **Dynamic Key-Value (KV) Cache:** The attention state memory allocated for active prompt tokens and generated responses. Modern transformer architectures store Key and Value vectors for previous tokens to avoid recomputing self-attention at each generation step. The memory required for the KV cache scales with the configured context length, number of layers, key-value heads, head dimension, and numerical precision. On models using Grouped-Query Attention (GQA), such as Llama `3.1` `8B`, the KV cache requires approximately `128 KiB` per token in `16-bit` floating-point precision, consuming roughly `4.0 GB` of VRAM at `32,768` tokens and `8.0 GB` at `64,000` tokens.

3. **Compute Buffers and Runtime Overhead:** Ephemeral memory allocated by the CUDA runtime, scratchpad buffers for intermediate activations, and memory reserved by the host operating system or display manager. Running a local desktop environment alongside Ollama often consumes `1.0` to `2.0 GB` of VRAM before Ollama loads its first tensor.

The table below illustrates total VRAM requirements across common model sizes and context lengths, combining static weights with unquantized 16-bit KV cache allocations:

| Model Scale | Quantization | 4,096 Context | 16,384 Context | 32,768 Context | 64,000 Context | Minimum Recommended VRAM |
|---|---|---|---|---|---|---|
| 8B (Llama 3.1) | Q4_K_M | 5.5 GB | 7.0 GB | 9.0 GB | 12.9 GB | 12 GB VRAM (RTX 3060 / 4070) |
| 14B (Qwen 2.5) | Q4_K_M | 9.8 GB | 11.5 GB | 13.8 GB | 18.2 GB | 16 GB VRAM (RTX 4080) |
| 32B (Qwen 2.5) | Q4_K_M | 20.2 GB | 22.8 GB | 26.2 GB | 32.5 GB | 24 GB VRAM (RTX 3090 / 4090) |
| 70B (Llama 3.3) | Q4_K_M | 43.5 GB | 47.8 GB | 53.5 GB | 64.0 GB | 48 GB+ VRAM (Dual RTX 3090 / Apple Studio) |

When total required memory exceeds physical GPU VRAM, Ollama does not immediately terminate. Instead, its scheduler offloads partial model layers to system RAM using CPU inference. While this mechanism prevents a hard crash, moving tensors across the host PCIe bus reduces inference throughput from dozens of tokens per second to a crawl, stalling agent workflows.

## Four-Step Checklist to Resolve Ollama Memory Limits and Prevent OOM

Resolving Ollama memory pressure requires proactive configuration of model residency, concurrency ceilings, memory-efficient attention, and external retrieval architecture. Applying this four-step checklist stabilizes local inference on consumer and server hardware:

1. **Tune `OLLAMA_KEEP_ALIVE`:** Ollama automatically unloads idle models after 5 minutes by default via `OLLAMA_KEEP_ALIVE`. If you query a model and stop, it occupies memory for 300 seconds. Setting this duration lower frees memory promptly for other applications, while setting it to `0` unloads the model immediately after completion.

2. **Cap `OLLAMA_MAX_LOADED_MODELS` to 1:** By default, Ollama attempts to load multiple models into memory when requested, assuming sufficient capacity. Capping this parameter to `1` ensures that requesting a new model unloads the previous model using a least-recently-used policy, preventing multi-model VRAM exhaustion.

3. **Enable Flash Attention (`OLLAMA_FLASH_ATTENTION=1`):** Enabling Flash Attention activates optimized memory access kernels in the underlying inference engine. Flash Attention avoids materializing intermediate attention matrices in VRAM, reducing memory consumption during prompt evaluation and long-context processing.

4. **Externalize Document Collections to an Indexed MCP Workspace:** Rather than expanding local context windows to `64,000` or `128,000` tokens to process entire documentation sets, store reference files in an external workspace. Connecting local models to indexed workspaces via Model Context Protocol (MCP) retrieves targeted context passages on demand, keeping local memory footprint predictable.

### Applying Server Configuration Across Environments

Depending on your operating system, configure these environment variables before starting the Ollama service:

**Linux Deployments with systemd:**
To persist settings across reboots, edit the service override file:

```bash
sudo systemctl edit ollama.service
```

Insert the environment configuration under the service unit block:

```text
[Service=unit]
Environment="OLLAMA_KEEP_ALIVE=2m"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_GPU_OVERHEAD=1073741824"
```

Save the file and restart the systemd daemon:

```bash
sudo systemctl daemon-reload
sudo systemctl restart ollama
```

**macOS Desktop Installations:**
On macOS, GUI applications do not inherit shell environment variables declared in shell profile scripts. Register the parameters with `launchctl` before launching the application:

```bash
launchctl setenv OLLAMA_KEEP_ALIVE "2m"
launchctl setenv OLLAMA_MAX_LOADED_MODELS "1"
launchctl setenv OLLAMA_FLASH_ATTENTION "1"
```

Restart the Ollama desktop app from the Applications folder or menu bar to apply the changes.

**Docker and Container Deployments:**
When running Ollama inside Docker, pass the environment flags directly in your run command or compose specification:

```bash
docker run -d \
  --gpus all \
  -e OLLAMA_KEEP_ALIVE=2m \
  -e OLLAMA_MAX_LOADED_MODELS=1 \
  -e OLLAMA_FLASH_ATTENTION=1 \
  -v ollama_data:/root/.ollama \
  -p 11434:11434 \
  --name ollama \
  ollama/ollama:latest
```

Setting these baseline variables establishes strict memory boundaries, preventing background tasks from accumulating unreleased VRAM.

## Diagnosing CUDA Out of Memory Errors and Exit Code 137

When Ollama exhausts host memory, the failure manifests as either a CUDA driver error or an operating system process termination. Understanding the diagnostic signals distinguishes between GPU-specific memory exhaustion and system-wide memory collapse.

### CUDA Out of Memory (CUDA OOM)

A CUDA OOM error occurs when the Ollama runner requests a VRAM allocation from the NVIDIA GPU driver that exceeds available video memory. This failure typically appears in server logs as:

```text
cuda error: out of memory
ggml_cuda_init: failed to allocate CUDA buffer
```

This error emerges during two distinct runtime phases:

* **Initial Model Loading:** The runner calculates layer offload requirements based on model weight tensors. If the calculated requirement leaves no room for CUDA runtime contexts or display buffers, loading fails.
* **Context Generation Expansion:** The model loads successfully, but a user prompt containing thousands of tokens forces the KV cache to allocate additional VRAM slots mid-stream. If the GPU runs out of unfragmented blocks, inference crashes immediately.

To prevent initial loading crashes caused by desktop environment overhead, set `OLLAMA_GPU_OVERHEAD`. This variable instructs Ollama to reserve a specific number of bytes per GPU exclusively for non-Ollama processes:

```bash
export OLLAMA_GPU_OVERHEAD=1073741824
```

### Linux Kernel OOM Killer and Exit Code 137

Exit code `137` indicates that the operating system terminated the Ollama process with `SIGKILL` (Signal 9). The exit code calculation is `128` plus signal number `9`, totaling `137`.

When Ollama runs models that exceed physical GPU VRAM, it offloads remaining layers and memory buffers to system RAM. If system RAM and swap space become completely full, the Linux kernel Out-Of-Memory (OOM) killer activates. To protect operating system stability, the kernel selects and terminates the highest memory-consuming user-space process, which is almost always the Ollama runner daemon.

To verify whether the OOM killer terminated Ollama, inspect kernel diagnostic messages and systemd service logs:

```bash
sudo dmesg -T | grep -i oom
journalctl -u ollama --no-pager -n 50
```

If the logs show that the kernel killed the process due to memory exhaustion, system RAM was depleted by CPU layer offloading or concurrent model loads.

In Docker environments on macOS or Windows, exit code `137` frequently stems from virtual machine memory caps. Docker Desktop runs inside a lightweight Linux VM with a fixed RAM ceiling. If you run a model requiring `6 GB` of RAM inside a container limited to `4 GB`, the VM kernel terminates the container immediately. Adjust the memory allocation in Docker Desktop resource settings to provide adequate headroom for your models.

### Operational Telemetry with ollama ps

To inspect running models, VRAM usage, and active execution targets in real time, run:

```bash
ollama ps
```

The output reveals active memory distribution:

```text
NAME               ID              SIZE      PROCESSOR        UNTIL              CONTEXT
llama3.1:8b        8eeb52625293    4.9 GB    100% GPU         2 minutes from now 4096
```

Inspect the `PROCESSOR` column closely:
* **Pure GPU execution:** All transformer layers reside in video memory, delivering optimal token throughput.
* **Partial GPU and CPU split:** Model weights and the KV cache exceeded physical VRAM, forcing Ollama to split layers across PCIe. This indicates that current context or model parameters are too large for your GPU.
* **Pure CPU execution:** Ollama could not access GPU drivers or VRAM was insufficient for even the base layers, resulting in slow CPU inference.

In addition, review API response metrics when integrating with automated pipelines. Ollama responses return `prompt_eval_count` and `prompt_eval_duration`. If `prompt_eval_count` is lower than the token count of your input prompt, Ollama truncated input text to fit within the active `num_ctx` ceiling.

## Managing Multi-Model Concurrency and Runtime Model Lifecycles

Autonomous agent systems and complex AI workflows rarely rely on a single model. An automated pipeline might use a lightweight routing model to classify user intent, a specialized embedding model to calculate vector similarities, and a larger conversational model to generate final responses. Running these models sequentially or concurrently creates severe memory volatility if residency lifecycles are unmanaged.

### Understanding Concurrency Multipliers

When multiple requests hit Ollama simultaneously, memory demand multiplies across two dimensions:

1. **Parallel Request Slots (`OLLAMA_NUM_PARALLEL`):** Ollama supports processing concurrent prompts by creating parallel evaluation slots. However, each parallel slot maintains its own separate Key-Value cache. If an `8B` model requires `4.0 GB` of VRAM for a single `32,768`-token KV cache, handling 4 parallel requests at that context length demands `16.0 GB` of VRAM just for the attention cache, easily causing sudden OOM crashes. Restrict concurrency on memory-limited hardware:

```bash
export OLLAMA_NUM_PARALLEL=1
```

2. **Loaded Model Count (`OLLAMA_MAX_LOADED_MODELS`):** If an agent executes an embedding model (`nomic-embed-text`) to index a document and immediately calls `llama3.1:8b` to answer a question, Ollama retains the embedding model in VRAM while loading the generation model. On an `8 GB` or `12 GB` GPU, both models cannot coexist in VRAM simultaneously.

Setting `OLLAMA_MAX_LOADED_MODELS=1` enforces an eviction sequence: Ollama automatically unloads the embedding model before allocating memory for the generation model. While model switching introduces a brief loading delay, it eliminates out-of-memory crashes.

### Controlling Lifecycles via API Requests

Rather than relying purely on global server timers, applications can manage model residency dynamically on a per-request basis. The Ollama REST API supports the `keep_alive` parameter across `/api/generate`, `/api/chat`, and `/api/embed`.

For transient operations, such as generating an embedding or routing a single classification, command Ollama to unload the model immediately:

```bash
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Classify the following query category: account billing",
  "keep_alive": 0,
  "options": {
    "num_ctx": 2048
  }
}'
```

Setting `"keep_alive": 0` instructs Ollama to evict model weights from VRAM as soon as the response finishes streaming, leaving the entire GPU memory pool free for subsequent tasks.

Conversely, for sustained conversational workflows, pass `"keep_alive": "30m"` to prevent model reloads between user turns:

```bash
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1:8b",
  "messages": [
    {"role": "user", "content": "Analyze our system architecture design."}
  ],
  "keep_alive": "30m",
  "options": {
    "num_ctx": 8192
  }
}'
```

### Baking Memory Constraints into Custom Modelfiles

For team environments where scripts and interactive users share an Ollama server, creating custom Modelfiles guarantees reproducible memory allocations. Create a file named `Modelfile`:

```dockerfile
FROM llama3.1:8b
PARAMETER num_ctx 8192
PARAMETER num_gpu 28
PARAMETER temperature 0.2
```

Build and tag the constrained model:

```bash
ollama create llama3.1-constrained -f Modelfile
```

Running `ollama run llama3.1-constrained` guarantees that the model respects these memory boundaries regardless of client request headers, preventing runaway token allocations from exhausting host VRAM.

## Offloading Document Corpora to Fast.io Workspaces via Model Context Protocol

Stuffing large document libraries, codebase repositories, and architectural specifications directly into local LLM context creates severe operational bottlenecks. As context windows expand, Time-to-First-Token latency degrades, power consumption spikes, and the risk of attention dilution increases.

In official Ollama documentation for context length, tasks that require large context such as web search, agents, and coding tools should be set to at least 64,000 tokens when hardware allows. However, running a `64,000`-token context window on an `8B` model requires substantial dedicated VRAM, leaving negligible headroom on consumer hardware and causing immediate crashes on larger models.

Engineering teams building agentic workflows evaluate three primary architectural paths for processing large document collections:

* **Raw Prompt Stuffing:** Concatenating full text into prompts. This approach consumes extensive KV cache memory, causes CUDA OOM crashes, triggers slow CPU offloading, and risks silent prompt truncation.
* **Self-Hosted Local Vector Databases:** Operating vector databases such as Chroma, Qdrant, or Milvus alongside Ollama. While this pattern reduces context size, it forces teams to maintain chunking pipelines, run local embedding servers, manage disk indexes, and handle index synchronization.
* **Intelligent Cloud Workspaces via Model Context Protocol (MCP):** Storing files in organized cloud workspaces that index content automatically and expose standardized search tools to local models over Streamable HTTP.

[Fast.io workspaces](/product/workspaces/) provide an intelligent workspace layer specifically built for teams using [storage for agents](/storage-for-agents/). Instead of attaching large files to chat prompts or overloading local VRAM with extended token contexts, documents are stored in shared, persistent workspaces:

* **Automated Indexing with Intelligence Mode:** Once Intelligence Mode is enabled on a workspace, all uploaded files are automatically indexed for hybrid search, combining full-text keyword indexing, semantic meaning, and metadata retrieval without requiring external vector databases.
* **Streamable HTTP MCP Integration:** Local models, coding assistants, and agent runtimes connect to Fast.io through remote Model Context Protocol endpoints. Clients connect with OAuth or scoped tokens at `https://mcp.fast.io/mcp/tools` (for general assistants) or `https://mcp.fast.io/mcp/code` (for coding agents). Detailed connection procedures and endpoint configurations are available in the [Fast.io MCP documentation](https://mcp.fast.io/docs).
* **Multi-Source Synchronization:** Cloud Sync connects Google Drive, Dropbox, Box and OneDrive folders to a workspace, one-way or two-way, on a schedule or on demand. Reference documents update in the workspace automatically without manual file transfers.
* **Structured Data Extraction:** For workflows requiring tabular analysis across contracts, financial reports, or technical specifications, [Metadata Views](/product/document-data-extraction/) extract typed schema fields directly from unstructured documents without manual data entry.
* **Governance and Auditability:** Every workspace maintains per-file version history and a detailed activity log of changes, tracking file uploads, modifications, and deletions across human members and autonomous agents.

### The Hybrid Architecture in Practice

In this hybrid architecture, an agent running locally on Ollama maintains a lean, responsive context length of `4,096` to `8,192` tokens. Model weights and the compact KV cache fit entirely within physical GPU VRAM, ensuring fast inference speeds.

When a user asks a question about an extensive reference corpus, the agent does not attempt to ingest entire PDF manuals into its context window. Instead, the agent executes an MCP search call against the Fast.io workspace. The workspace performs hybrid semantic and keyword retrieval across the indexed library and returns only the relevant passages with source citations.

The local model synthesizes the answer from the retrieved excerpts, preserving local GPU VRAM, preventing out-of-memory terminations, and maintaining peak interactive responsiveness.

## Frequently asked questions

### How do I limit Ollama RAM and VRAM usage?

You can limit Ollama memory usage by capping the context window using OLLAMA_CONTEXT_LENGTH or num_ctx, setting OLLAMA_MAX_LOADED_MODELS to 1, tuning OLLAMA_KEEP_ALIVE to unload idle models quickly, and reserving system memory using OLLAMA_GPU_OVERHEAD. For persistent constraints, define num_ctx and num_gpu in a custom Modelfile.

### Why does Ollama give an out of memory error or exit code 137?

Exit code 137 occurs when the operating system kernel Out-Of-Memory (OOM) killer terminates Ollama with SIGKILL because total physical RAM and swap are exhausted. A CUDA out-of-memory error happens when the GPU runs out of VRAM while allocating model weights or expanding the Key-Value (KV) cache for large context windows.

### How does OLLAMA_MAX_LOADED_MODELS affect memory?

OLLAMA_MAX_LOADED_MODELS controls how many distinct models can remain loaded in memory concurrently. Setting this to 1 ensures that loading a new model automatically evicts the least recently used model from VRAM, preventing multi-model memory saturation on GPUs with limited capacity.

### How do I make Ollama unload models immediately after inference?

You can configure Ollama to unload models immediately by setting the environment variable OLLAMA_KEEP_ALIVE=0 before starting the server, or by passing keep_alive: 0 in the JSON payload of individual API requests to /api/generate or /api/chat.

### What is the difference between an Ollama memory limit and context length?

Context length (num_ctx) is the sequence capacity measured in tokens that a model can process in a single prompt and response cycle. An Ollama memory limit refers to the physical RAM and VRAM in gigabytes consumed by static model weights, dynamic Key-Value caches, and CUDA execution buffers.

### How does offloading documents to Fast.io via MCP prevent local OOM crashes?

Instead of loading entire document libraries into local Ollama context windows, files are stored in a Fast.io workspace with Intelligence Mode enabled. Local models query the workspace over Model Context Protocol using tools described in the Fast.io MCP documentation, retrieving only relevant passages. This keeps local context under 8,192 tokens and prevents GPU VRAM exhaustion.

## Sources

- [Ollama Documentation: Context length](https://docs.ollama.com/context-length): Ollama default context length allocates 4,096 tokens under 24 GiB VRAM, 32,768 tokens from 24 to 48 GiB VRAM, and 256,000 tokens above 48 GiB VRAM, with official guidance recommending at least 64,000 tokens for agents.
- [Anthropic: Claude Help Center](https://support.claude.com/en/articles/8241126-upload-files-to-claude): A Claude chat conversation accepts up to 20 files at up to 500MB each, while a project accepts files up to 30MB each with no fixed file count limit, constrained by the context window.

## About Fast.io

Fast.io provides shared workspaces where people and AI agents work on the same files, with built-in semantic search and citation-backed chat over what they hold. Agents reach it through a remote MCP server, a REST API at https://api.fast.io/current/, and a command line client published on npm as @vividengine/fastio-cli. MCP setup is at https://mcp.fast.io/docs: Claude and most MCP clients connect to https://mcp.fast.io/mcp/tools, ChatGPT to https://mcp.fast.io/mcp/operations, and coding agents to https://mcp.fast.io/mcp/code.