# How to Keep an AI Assistant from Showing Restricted Files

Keeping an AI assistant from showing restricted files requires container-based retrieval boundaries where search is scoped to the workspaces a user belongs to rather than run across a flat tenant-wide pool. When AI search indexes an entire drive, vector queries can retrieve sensitive passages regardless of who is asking. Enforcing boundaries at the container level before search runs prevents leaks and keeps private files out of assistant prompts.

Source: https://fast.io/resources/how-to-keep-ai-assistant-from-showing-restricted-files/
Author: [Tom Langridge](https://fast.io/authors/tom-langridge/)
Last reviewed: 2026-11-10

## How Container Boundaries Keep AI Assistants from Showing Restricted Files

An AI assistant that indexes an entire company drive will answer questions using passages from restricted files, and neither the assistant nor the asking user will realize a boundary was crossed. The failure is not an adversarial prompt injection or a model hallucination; it is an architectural gap where vector similarity search runs without checking file access controls.

In traditional enterprise IT systems, access control lists sit at the operating system or storage layer. Operating systems verify identity tokens and permissions before granting read handles to files or directories. When an AI pipeline or enterprise search engine ingests documents, it breaks them into smaller text segments, passes those chunks through an embedding model, and stores the resulting numerical arrays inside a vector index. During that ingestion process, the source file system access controls disappear unless the pipeline deliberately reconstructs them.

Keeping an AI assistant from showing restricted files requires container-based retrieval boundaries where search is scoped to the workspaces a user belongs to rather than run across a flat tenant-wide pool. Without strict boundaries, the assistant operates with the elevated permissions of its ingestion account rather than the restricted permissions of the employee asking the question.

### The Failure of Prompt Guardrails and Context Redaction

Many teams attempt to patch document oversharing by adding guardrails to system prompts. A common approach instructs the model: 'You are an internal assistant. Do not answer questions about executive compensation, unannounced mergers, or confidential personnel records.' Another variation instructs the model to inspect document titles and redact text if the user does not have clearance.

These prompt-level restrictions fail under production conditions. Language models are probabilistic completion engines, not deterministic security firewalls. A user does not need sophisticated adversarial jailbreaks to bypass prompt instructions. Simple phrasing variations, role-playing scenarios, or indirect inquiries can persuade the model to summarize a document while omitting the forbidden keyword.

Once a sensitive text passage enters the model context window, it influences every subsequent attention layer. The assistant can leak confidential details indirectly through comparative statements, tone adjustments, or aggregate summaries. If an assistant knows the company financial forecast because the chunk sits in its context window, an employee asking about department hiring plans can receive an answer shaped by that hidden context. Security boundaries must exist at the retrieval layer before documents ever reach the context window.

### How Ingestion Identity Disconnects from Query Identity

The primary cause of AI document leaks is the architectural separation between the ingestion identity and the querying identity. To populate a search index, an organization connects a background connector to Google Drive, SharePoint, Box, or an internal file server. To index everything efficiently, administrators run the connector using a service account with broad administrative read permissions across the entire domain.

The connector crawls files, converts text chunks into vector embeddings, and writes them into a unified database. Later, when an entry-level employee opens an AI assistant to research customer onboarding guides, the search tool executes a vector similarity query against that unified database. The search engine evaluates cosine distance or dot-product similarity between the query embedding and the stored document embeddings.

The mathematical scoring function evaluates semantic proximity, not security tokens. If the employee asks what priorities were discussed in recent leadership planning sessions, the nearest neighbors in mathematical space might be raw strategy memos stored in an executive folder that the employee cannot open. Because the vector index was built by a privileged service account, the retriever pulls the memo chunks and hands them to the language model. The assistant answers truthfully with confidential corporate strategy.

## How Four Architectural Defenses Prevent Document Oversharing

To prevent an AI assistant from leaking restricted files, engineering teams must replace flat indexing with defense-in-depth retrieval controls. Relying on a single inspection layer leaves systems vulnerable to configuration drift and edge-case queries.

A reliable security posture combines four structural defenses:
* **Container Partitioning:** Documents live inside discrete workspace containers rather than one flat, organization-wide pool.
* **Query-Time Membership Filtering:** The retrieval engine evaluates caller identity and restricts vector similarity search strictly to containers where the user or agent holds verified membership.
* **Share-Isolated Intelligence:** External collaborations and project handoffs run through scoped shares where the assistant answers exclusively from the specific files in that share.
* **Restricted Workspace Join Settings:** Workspaces holding sensitive files enforce invite-only membership policies, preventing unauthorized organization members from self-enrolling into restricted context pools.

Enforcing these four controls keeps documents outside a user's access out of retrieval, out of prompts, and away from the model.

### Evaluating Retrieval Boundaries Across Architecture Layers

The following structured comparison illustrates where different security mechanisms operate and why container-level enforcement provides the most reliable boundary:

| Defense Mechanism | Enforcement Point | Prompt Injection Resistance | Latency Impact | Operational Overhead |
|---|---|---|---|---|
| Container Partitioning | Storage container boundary | High (out-of-scope files are not retrieved) | Low (no per-chunk filtering) | Low (per-workspace membership) |
| Chunk Metadata Pre-Filtering | Vector index query traversal | High (filtered before retrieval) | Moderate to high | High (continuous metadata sync) |
| Post-Retrieval ACL Filtering | Application middleware | High (filtered after retrieval) | High (wasted retrieval passes) | Moderate (query-time lookups) |
| System Prompt Guardrails | Language model context | None (easily bypassed) | Negligible | Low (fragile prompt text) |

Container partitioning eliminates the need to inspect every vector chunk dynamically during search traversal. Because search covers only the containers the user belongs to, files outside them are not part of the search.

### The Risks of Context Starvation in Post-Retrieval Filtering

When teams reject prompt guardrails, their first technical impulse is often post-retrieval filtering. In this pattern, the system runs an unrestricted similarity search across the entire corporate index to retrieve the top twenty chunks. Application middleware then checks each chunk source document against the user permissions, discarding any unauthorized segments before building the LLM prompt.

In production, post-retrieval filtering causes severe context starvation. If a user asks a question about an ongoing operational initiative, the unrestricted vector search might find that fifteen relevant chunks originate from restricted executive strategy documents. The middleware removes those fifteen chunks to prevent a data leak, leaving only five low-scoring, generic passages.

The language model receives inadequate context and produces an incomplete, hallucinated, or unhelpful answer, even though fifty legitimate, permitted documents existed in the database at slightly lower similarity scores. Expanding the initial retrieval limit to fifty or a hundred chunks to compensate spikes database compute costs and introduces noticeable query latency. Filtering must occur before or during retrieval, never afterward.

## How Container Boundaries Compare to Chunk Metadata Filtering

Two primary engineering patterns exist for enforcing access control in semantic search: chunk-level metadata filtering and container-level partitioning. Understanding their operational differences determines whether an enterprise AI deployment remains secure over time.

Chunk-level metadata filtering attaches access control lists directly to every vector record in the database. When an ingestion pipeline splits a document into chunks, it copies user and group IDs into each chunk metadata payload. Container-level partitioning organizes documents into distinct workspaces and shares, where permissions govern the entire container rather than individual chunks. Fastio uses this container-based model. Permissions are set per organization, workspace, and share, through member roles. A workspace has no folder-level or file-level permissions; every workspace member reaches the whole workspace according to their assigned role.

When an employee or AI agent queries Fast.io Search or Intelligence (semantic search, summaries, and chat with citations to files, pages, and snippets), answers draw only from the workspaces and shares the asking person or agent belongs to. A share AI answers only from that specific share files. The retrieval boundary is therefore the workspace or share: material only some people may see lives in its own workspace or share, and an agent is a member of exactly the workspaces it needs. Review [Fast.io intelligent workspaces](/product/workspaces/) to see how container boundaries organize project data.

### Why Permission Drift Breaks Chunk Metadata Models

Permission drift is the gradual desynchronization between an enterprise identity provider and the access rules stored inside a vector index. In an active organization, permissions are fluid: employees change teams, contractors finish assignments, and document confidentiality tiers update.

In a chunk-based access model, updating permissions requires updating the metadata fields of every vector associated with modified documents. If an administrator restricts access to a project directory containing ten thousand pages, background workers must update millions of vector metadata records. If synchronization queues stall or rate limits occur, the vector index drifts from the source permissions.

During that drift window, users who lost access to a project can still retrieve sensitive passages through AI search. In contrast, container-based models rely on workspace membership. Removing a user's workspace membership excludes that workspace from the user's retrieval boundary without needing to touch a single vector record.

### Why Chunk Metadata Filters Fragment Search Graphs

Beyond permission drift, chunk-level metadata filtering impairs vector search performance. High-dimensional vector search relies on indexing structures like Hierarchical Navigable Small World (HNSW) graphs. HNSW graphs navigate multi-dimensional geometric paths to find nearest neighbors quickly.

When a query applies strict chunk metadata filters, the vector engine must prune graph nodes during traversal. When a metadata filter excludes a large majority of documents from an index, the navigation graph becomes fragmented. The algorithm encounters dead ends where all neighboring nodes fail the permission check, forcing the engine to fall back to expensive brute-force scans.

Container-based retrieval needs no per-chunk permission filters, because scope is set by membership before search runs.

## How to Manage Ingestion, Cloud Sync, and External Storage Boundaries

Enterprise knowledge rarely starts inside a pristine AI repository. Teams maintain years of accumulated documentation across external cloud services like Google Drive, Dropbox, Box, and OneDrive. Connecting these legacy repositories to an AI assistant introduces severe data exposure risks if permissions are handled carelessly.

Cloud Sync connects Google Drive, Dropbox, Box, and OneDrive folders to a workspace, one-way or two-way, on a schedule or on demand, never continuous, live, or real-time. SharePoint document libraries connect through the OneDrive connector. Crucially, the source drive sharing settings do not carry over to Fast.io. Synced files inherit the permissions of the Fastio workspace where they land.

What an assistant or user can retrieve is decided by Fastio workspace membership rather than by the source system access list. This separation protects organizations from legacy permission sprawl. In typical cloud drives, documents accumulate years of accidental open links and orphaned group access. Resetting permissions at the workspace boundary ensures AI assistants only query clean, verified access containers.

### Why Source Drive Sharing Rules Should Not Control AI Retrieval

Attempting to mirror source drive sharing rules into an AI retrieval engine creates endless security liabilities. External cloud drives use complex inheritance hierarchies, nested folder permissions, and file-level exceptions. Synchronizing these multi-tiered rules into vector databases requires continuous background polling and complex relational mapping.

When an employee creates an open share link in Google Drive to collaborate with a temporary vendor, an AI search engine that mirrors drive permissions might treat that document as public across the entire company. If folder inheritance is broken in SharePoint, a background sync worker can easily misinterpret the effective access rights.

By decoupling source permissions from workspace retrieval, Fast.io provides deterministic access control. Documents placed in an executive workspace are queryable only by members of that executive workspace, regardless of how loosely they were shared in the originating drive. When structured data extraction is required alongside semantic search, teams can use [document data extraction](/product/document-data-extraction/) via Metadata Views to build queryable schemas across files without altering container security.

### Sharing Isolated Folders Through Branded Shares

When teams must collaborate with outside clients, legal counsel, or external contractors, inviting them into an internal workspace can expose broader project context. Fastio solves this using branded shares (Send, Receive, and Exchange).

A share functions as its own independent retrieval container. When an external partner opens a share, Intelligence Mode inside that share answers questions exclusively from the files placed in that share. The external user cannot query the broader workspace, cannot see other projects, and cannot access internal team discussions.

Shares can be durable or expiring, include per-recipient access controls, and disable file downloads to prevent sensitive files from being copied to unmanaged local devices. External guests open shares directly via web links without creating an account and without consuming an organization seat. Notes kept beside the files for people and agents co-edit live with labeled cursors, keeping project notes organized within the same secure container.

## How to Scope AI Agent Credentials and Audit Changes

Modern AI assistants do not merely answer chat questions in a browser; they operate as autonomous agents that read specifications, query knowledge bases, and write code. Connecting these assistants requires secure protocol interfaces that enforce least privilege.

The Model Context Protocol (MCP) provides an open standard for connecting AI clients to external workspaces. Fast.io hosts a remote MCP server over Streamable HTTP, requiring no local bridge packages or daemons. Assistants connect to dedicated endpoints based on client type:
* Claude desktop, mobile, web, and Claude Cowork connect to `https://mcp.fast.io/mcp/tools`.
* Coding agents such as Claude Code, Cursor, Cline, Devin, and VS Code agent mode connect to `https://mcp.fast.io/mcp/code`.
* ChatGPT and Codex connect through the Fastio plugin in the ChatGPT plugin directory, with `https://mcp.fast.io/mcp/operations` serving as an alternative custom MCP server.

Detailed setup steps for all environments are documented at [Fast.io MCP setup](https://mcp.fast.io/docs).

### Scoping MCP Tools and Credentials for Assistants

When connecting interactive AI assistants, users sign in with OAuth and encounter a Review Permissions screen. Users choose between Read Only or Read & Write access and explicitly select which organizations and workspaces the connection reaches. If an assistant is assigned Read Only access, tools that change data are not offered at all. There is no 403, no insufficient_scope error, and no step-up prompt during execution because write tools do not exist on the connection.

Headless agents, such as automated background processors or CI runners, authenticate using a scoped API key passed via an `Authorization: Bearer <api key>` header. API keys carry explicit scopes and act with the permissions of the user who created them; they have no role of their own.

When multiple agents and human team members collaborate inside a shared workspace, file updates can conflict. Fastio provides advisory per-file locks in workspace and share storage. Agents acquire and release locks through MCP tooling using `lock-acquire` and `lock-release` on the `storage_manage` tool, while inspecting lock status via `lock-status` on the `storage` tool. A lock expires automatically unless renewed, other members can see who holds it, and it never blocks reads, allowing semantic retrieval to continue uninterrupted.

To configure Claude Code for secure workspace access, developers add the HTTP endpoint directly:

```bash
claude mcp add --transport http fast-io https://mcp.fast.io/mcp/code
```

After adding the server, users sign in with OAuth in the browser to choose workspace permissions. To explore multi-agent configurations, teams can review [storage for agents](/storage-for-agents/).

### Auditing Access Adjustments with the Detailed Activity Log

Keeping an AI assistant from showing restricted files requires verifiable visibility into access changes. Fast.io maintains a detailed activity log that records state changes across the organization, capturing actions performed by people and AI agents alike.

The detailed activity log tracks file uploads, edits, moves, and deletions, alongside membership changes, permission adjustments, and share creation. It does not record read-only events such as views, previews, opens, or downloads. The log is designed to show who changed access and when, rather than tracking who viewed which file.

Administrators must also verify workspace join settings. In Fastio, workspace join settings default to letting any member of an organization join. When establishing workspaces for sensitive projects, administrators must restrict this setting to invite-only. Workspace roles include owner, admin, member, guest, and view, while organization roles include owner, admin, and member. When an agent builds an organization and transfers ownership to a person with a claim link, the detailed activity log records the administrative transition. An owner can also transfer an organization or a workspace directly to another member, and the previous owner becomes an admin.

Fastio runs on cloud infrastructure partners, including Google Cloud Platform and Cloudflare, that are certified to industry-leading security standards. Combining infrastructure encryption in transit and at rest with container-isolated retrieval provides a dependable defense against internal data leaks.

## Frequently asked questions

### How do I prevent an AI assistant from exposing private files?

Preventing an AI assistant from exposing private files requires container-based retrieval partitioning. Instead of indexing all company documents into a single flat vector database, divide files into isolated workspaces. The AI assistant should only have access to query the specific workspaces where the user or agent holds authorized membership.

### Why does my AI assistant show files users aren't allowed to see?

AI assistants typically show restricted files because the background ingestion connector used an administrative service account to crawl and index company files. The vector search engine then calculates semantic similarity without checking user permissions, retrieving sensitive passages and inserting them directly into the assistant prompt context.

### How do you enforce access control in enterprise AI search?

Enforce access control in enterprise AI search by applying permission boundaries at the retrieval layer before text reaches the language model. The most effective approach is container partitioning, where the search engine only queries document embeddings within workspaces the caller is authorized to access.

### Can prompt engineering prevent an AI assistant from leaking confidential documents?

Prompt instructions and system guardrails cannot reliably prevent document leaks. Language models are probabilistic completion engines, and users can bypass prompt instructions through phrasing variations, role-playing, or indirect queries. Access control must be enforced deterministically at the retrieval layer.

### Do file permissions from Google Drive or OneDrive transfer to an AI index?

In Fastio, files synced from Google Drive, Dropbox, Box, or OneDrive do not carry over source drive sharing settings. Synced files inherit the permissions of the Fastio workspace where they land, ensuring retrieval boundaries depend strictly on verified workspace membership rather than legacy cloud permissions.

### How does Fast.io prevent AI assistants from retrieving restricted files?

Fast.io enforces container-based retrieval boundaries. Permissions are set per organization, workspace, and share by member role. Search and Intelligence answer only from the workspaces and shares the asking person or agent belongs to, so files in other workspaces are not retrieved or included in assistant prompts.

## About Fast.io

Fast.io provides shared workspaces where people and AI agents work on the same files, with built-in semantic search and citation-backed chat over what they hold. Agents reach it through a remote MCP server, a REST API at https://api.fast.io/current/, and a command line client published on npm as @vividengine/fastio-cli. MCP setup is at https://mcp.fast.io/docs: Claude and most MCP clients connect to https://mcp.fast.io/mcp/tools, ChatGPT to https://mcp.fast.io/mcp/operations, and coding agents to https://mcp.fast.io/mcp/code.