How to Implement Semantic Search with Fast.io API

What Is Semantic Search in Fast.io?

Semantic search in Fast.io finds files by meaning, not exact keywords. For example, query "Q3 contracts with Acme" to retrieve relevant documents even without those words in filenames.

It relies on Intelligence Mode, which indexes files with vector embeddings for RAG. Queries go through AI chat endpoints that retrieve and cite sources. This improves retrieval accuracy for agent workflows.

Unlike keyword search, semantic search understands context. It combines full-text indexing with AI to handle synonyms, intent, and relationships.

Prerequisites and Account Setup

Start with a Fast.io agent account for 50 GB free storage and 5,000 monthly credits. No credit card needed.

Use the API base URL https://api.fast.io/current/. Authenticate with JWT or API keys via /current/user/auth/.

Code example for agent signup (adapt for your LLM):

curl -X POST https://api.fast.io/current/user/ \\
  -d "first_name=Agent" \\
  -d "last_name=Search" \\
  -d "email=agent@example.com" \\
  -d "password=securepass123"

Verify email, then create an organization:

curl -X POST https://api.fast.io/current/org/ \\
  -H "Authorization: Bearer $JWT_TOKEN" \\
  -d "name=My Agent Org" \\
  -d "domain=agent-search"

Create and Configure a Workspace

Workspaces hold files. Create one:

curl -X POST https://api.fast.io/current/org/{org_id}/workspace/ \\
  -H "Authorization: Bearer $JWT_TOKEN" \\
  -d "folder_name=semantic-search-ws" \\
  -d "name=Semantic Search Workspace"

Enable Intelligence Mode for auto-indexing:

1. POST /current/workspace/{workspace_id}/intelligence/ with enabled=true.

Files uploaded now get indexed automatically (ai_state: ready).

Check status: GET /current/workspace/{workspace_id}/storage/root/ for ai fields on nodes.

Upload Files for Indexing

Chunked uploads for large files. First initiate:

curl -X POST https://api.fast.io/current/workspace/{workspace_id}/upload/initiate/ \\
  -H "Authorization: Bearer $JWT_TOKEN" \\
  -d "filename=document.pdf" \\
  -d "size=10485760"

Upload chunks, complete with /upload/complete/.

Or import from URL: POST /current/workspace/{workspace_id}/upload/web/ with source_url.

Wait for ai_state: ready in storage details. Indexed files support semantic queries.

Execute Semantic Searches via API

Use chat_with_files for semantic search. Default scope is full workspace.

Create chat:

curl -X POST https://api.fast.io/current/ai/chat/ \\
  -H "Authorization: Bearer $JWT_TOKEN" \\
  -d "type=chat_with_files" \\
  -d "query_text=Find contracts mentioning indemnity from last quarter" \\
  -d "workspace_id={workspace_id}"

Poll message: GET /current/ai/message/{message_id}/.

Response includes citations with nodeId, page, snippet.

For scoped: folders_scope=nodeId:3 limits to subfolders.

Numbered API flow:

1. POST /ai/chat/ create.
2. GET /ai/message/{id}/ poll.
3. Parse citations for relevant files.

Scoped Searches and Permissions

Permissions are enforced automatically. Queries respect member roles.

Scope to folder: folders_scope={folder_node_id}:2.

Multi-file: files_scope={node1}:{version1},{node2}:{version2}.

For shares: Use context_type=share endpoints.

Edge case: Non-indexed files (ai_state != ready) ignored in RAG.

Test with small set first.

Best Practices and Troubleshooting

• Monitor credits: GET /org/{org_id}/billing/usage/.
• Poll activity: /activity/poll/{entity_id}.
• Handle errors: Check error.code like 1667 for limits.
• Scale: Use webhooks for events.

Common issues:

Integrate with MCP for 251 tools in agent frameworks.