How to Build an Online AI Detector API with Hermes Agent

A 2023 study by Stanford University researchers published in the journal Patterns revealed that seven popular AI writing detectors misclassified 61% of essays written by non-native English speakers as AI-generated [Liang et al. 2023]. This high false-positive rate exposes a critical flaw in commercial detection tools: they do not detect authorship, but rather evaluate predictability in sentence structures, which disproportionately penalizes writers with structured or formal grammar. For organizations scanning thousands of documents, relying on commercial software is not only unreliable but also financially prohibitive. Every document scanned incurs recurring SaaS fees, which can quickly drain budgets when scaled across university departments, publishing houses, or content agencies.

Standard detection systems rely on two primary metrics: perplexity, which measures the predictability of word sequences, and burstiness, which measures sentence length variation. Because AI writing models are trained to produce highly probable word transitions, their output exhibits low perplexity. However, formal human writing, especially by students writing in a second language, also exhibits low perplexity because it follows textbook grammar patterns. This overlap makes statistical classification prone to false positives, necessitating custom, fine-tuned detection models that developers can adjust to their specific domain.

To address these reliability issues and eliminate recurring SaaS fees, developers are turning to a self-hosted AI detector. By deploying a self-hosted AI detector online, teams retain complete control over the underlying detection models, avoid sharing sensitive text with third-party providers, and scale operations without per-document pricing. Running a self-hosted AI detector online requires two core pieces of infrastructure: an execution environment that handles the analysis, and a secure storage layer that records the output. The Nous Research Hermes Agent provides the execution environment, exposing a flexible gateway that runs locally or in containers, while Fast.io serves as the persistent data and audit trail layer.

Exposing a self-hosted AI detector online requires an API gateway to receive text scans from external applications. The Nous Research Hermes Agent includes a built-in API server that allows you to run the agent as a background service. This configuration exposes an OpenAI-compatible HTTP endpoint, allowing external web applications to programmatically evaluate text content for AI signatures through an active Hermes Agent node.

Step-by-step command line instructions to start the Hermes API server:

First, create or open your environment configuration file, typically located at ~/.hermes/.env, and add the following lines to enable the API gateway and configure your authorization token:

API_SERVER_ENABLED=true
API_SERVER_KEY="your-secret-api-key"

Next, launch the gateway process from your terminal:

hermes gateway

By default, the server listens at http://127.0.0.1:8642. You can test the connection by sending a POST request to http://localhost:8642/v1/chat/completions using the API key you defined in the environment.

Clients connect to the gateway using standard OpenAI libraries, replacing the base URL and API key with your self-hosted details. The stateless nature of the HTTP endpoints ensures that each detection request is processed independently, although optional session headers can track long-running verification pipelines. This makes the Hermes Agent gateway an ideal intermediary between frontend user interfaces and the back-end evaluation scripts.

When active, these API servers handle parallelized subagent tasks efficiently, distributing incoming text evaluation requests across multiple sandboxed environments without blocking the main event loop. This concurrency is critical when multiple users or systems submit documents for scanning simultaneously. However, because containerized execution environments are ephemeral, you must route the results to a persistent database or storage service. While local filesystems or standard bucket storage work, they lack version history, built-in search, and collaborative access. Fast.io provides a shared workspace where all agent outputs, text submissions, and audit logs are recorded and structured.

Securely store and audit self-hosted AI detector logs

Persist Hermes Agent text scans and report metadata in a shared workspace with automated indexing. Every organization begins with a 14-day free trial.

Start 14-Day Trial

To run the actual classification, the agent needs a tool or script that evaluates the text. While deep learning models on Hugging Face provide high accuracy, a fast and dependency-free method is compression-ratio classification, which measures how predictable a piece of text is. AI-generated text, being highly repetitive and structured, compresses much more than human-written text.

Below is a Python script that calculates the compression ratio of a text sample and outputs a classification report. Save this script as ai_detector.py in your project folder:

import sys
import zlib

def evaluate_text(text: str) -> dict:
    if len(text) < 100:
        return {"status": "error", "message": "Text sample too short"}
        
    original_bytes = text.encode("utf-8")
    compressed_bytes = zlib.compress(original_bytes, level=9)
    
    ratio = len(compressed_bytes) / len(original_bytes)
    
    is_ai = ratio < 0.42
    confidence = min(100.0, max(0.0, (0.60 - ratio) * 500.0)) if is_ai else min(100.0, max(0.0, (ratio - 0.35) * 200.0))
    
    return {
        "status": "success",
        "compressibility_ratio": round(ratio, 4),
        "classification": "AI-generated" if is_ai else "Human-written",
        "confidence_percentage": round(confidence, 2)
    }

if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("Usage: python ai_detector.py <text_to_analyze>")
        sys.exit(1)
        
    sample_text = sys.argv[1]
    result = evaluate_text(sample_text)
    print(result)

The Hermes Agent runs this script using its terminal capability when a request arrives through the gateway. If you need deeper linguistic analysis, the script can be modified to call a local Ollama instance running a specialized classification model like llama3 or a fine-tuned deberta-v3-large model. Because the script runs in the agent's sandboxed terminal environment, you do not need to install complex dependencies on your host system. The agent handles package installation, executes the validation script, and outputs the result as a structured JSON object.

To register this script as an agent skill, create a SKILL.md file in the agent's directory. This file describes the tool parameters and tells the agent to run python ai_detector.py when evaluating text files. The agent then reads the file content, passes it to the script, and formats the output for the API caller.

Running an online AI detector API in production requires a permanent database to store input texts, classification reports, and verification logs. Storing these files locally inside the agent's container is risky because container lifecycles are ephemeral; a restart deletes all historical data. While basic object storage like Amazon S3 or Google Drive solves the persistence problem, these platforms lack built-in tools to search, version, or query the generated files without building additional database layers.

Fast.io provides an intelligent workspace that serves as a persistent database and collaboration layer for your agent's files. By configuring the agent to write its reports to a Fast.io workspace, all verification logs survive container teardowns. Fast.io adds several unique advantages for agent workflows:

First, the platform features per-file version history. If an agent refines a report or updates a log file, the previous versions are preserved, creating an auditable chain of custody.

Second, Fast.io's Intelligence Mode automatically indexes uploaded files. This enables developers and team members to run semantic queries over historical reports (such as asking "summarize all reports flagged with high confidence yesterday") rather than writing complex SQL databases.

Third, you can use Metadata Views to turn your flat JSON reports into a structured, queryable database. Developers define extraction fields in plain English (such as AI probability score, classification category, and file author), and Fast.io extracts this data into a sortable spreadsheet. This structure differs from the semantic search in Intelligence Mode, offering a clean database interface directly over your files. You can read more about this feature on the Metadata Views page.

Setting up this storage is straightforward. The Starter plan is priced at $29/mo, the Business plan is $99/mo, and the Growth plan is $299/mo. Every organization begins with a 14-day free trial, which requires a credit card to activate, allowing you to test the integration fully before committing.

Once your self-hosted AI detector online generates a report, you need to share the findings securely with stakeholders. Exposing reports through public directories is a security risk, while sending them as email attachments is inefficient. Fast.io provides branded shares (Send, Receive, and Exchange workflows) that allow you to distribute reports safely.

Branded shares support download controls, expiration dates, and password protection, ensuring only authorized clients or educators access the results. Furthermore, the reports remain versioned; if the agent updates a file in the workspace, recipients automatically see the latest version while retaining access to the history.

For developers building detection systems for clients, the handoff process is simplified through ownership transfer. The developer sets up the API gateway, configures the storage connection, and creates the client workspace. Once the system is tested and running, the developer initiates an ownership transfer, handing the entire organization over to the client. The client adds their payment info to start their subscription, while the developer can retain administrator access to monitor the Hermes Agent node. This ensures a clean transition from staging to production while maintaining the security of the underlying files.

By hosting your own API and connecting it to an intelligent workspace, you build a private, cost-effective validation pipeline. This architecture secures your data, eliminates recurring SaaS fees, and provides your team with structured, searchable reports that can be safely shared with anyone.