How to Set Up Automated Code Review with Hermes Agent

In Qodo's analysis of one million pull requests, 17% contained high-severity issues, scored 9 or 10 on a 10-point scale, that were likely to cause production impact. These defects would have passed manual review under time pressure. The same study referenced research by Capers Jones across 12,000 software projects showing that even formal code reviews catch only 60-65% of hidden defects, with informal reviews landing below 50%.

The typical response is a SaaS code review bot. CodeRabbit, Qodo, Greptile, and GitHub Copilot Code Review all offer webhook-triggered analysis that runs when a PR is opened or updated. They work well for teams that want a managed solution and don't mind per-seat pricing.

Nous Research Hermes Agent takes a different path. Instead of a hosted webhook, you configure a cron job that polls for open PRs, fetches diffs through the GitHub CLI, and runs them through a skill you define. The agent runs on your own infrastructure (a laptop, a VPS, a Docker container) and delivers findings to whatever messaging channel you already use. There's no server to expose, no webhook endpoint to secure, and no per-seat billing. The tradeoff is setup time: you're building a review pipeline rather than installing an app.

That tradeoff makes sense when your team has specific conventions that generic review bots ignore, when you want review logic you can version-control alongside your code, or when you need reviews delivered to a channel that isn't GitHub.

Before building the review pipeline, you need three things running: Hermes Agent itself, the GitHub CLI authenticated to your repositories, and a messaging gateway if you want results delivered outside your terminal.

Hermes Agent is open source under the MIT license. Install it following the official setup guide. Once installed, start the gateway service:

hermes gateway install

The gateway manages communication between the agent runtime, messaging platforms, and scheduled jobs.

GitHub CLI handles all repository access. Authenticate it once:

gh auth login

Verify access by listing open PRs on a target repo:

gh pr list --repo your-org/your-repo --state open --limit 3

Messaging gateway (optional but recommended). Hermes Agent supports delivery to Telegram, Discord, Slack, WhatsApp, Signal, email, and Matrix. Configure your preferred channel through the gateway settings. If you skip this step, results save to ~/.hermes/cron/output/ as local files, which works fine for solo developers who check results manually.

A Hermes Agent skill is a markdown file that tells the agent how to approach a specific type of task. For code review, you create a file at ~/.hermes/skills/code-review/SKILL.md that defines what the agent should look for and how it should report findings.

The skill definition starts with frontmatter:

---
name: code-review
description: Review pull requests for bugs, security issues, and code quality
---

Below the frontmatter, write the review criteria in plain language. The agent interprets this as its instruction set for every review it runs. A practical skill definition covers five categories:

• Logic errors: null handling, off-by-one mistakes, boundary conditions, race conditions
• Security vulnerabilities: injection patterns, auth bypasses, exposed credentials, unsafe deserialization
• Performance issues: N+1 queries, unbounded loops, missing pagination, memory leaks
• Style violations: dead code, inconsistent naming, missing type annotations
• Test coverage: untested branches, missing edge case tests, test assertions that don't verify behavior

For each finding, the skill instructs the agent to report a File:Line location, a severity level, a description of the problem, and a suggested fix. The severity levels are Critical (blocks merge, likely production impact), Warning (should fix before merge, potential issues), and Suggestion (non-blocking improvements).

The Nous Research requesting-code-review skill in the official repository adds a fail-closed principle: if the agent can't parse its own review output, it defaults to failure rather than silently passing. That's worth adopting in your own skill definition.

Persist your agent's review history across sessions

Free 50 GB workspace with built-in search across every review artifact your Hermes Agent produces. No credit card, MCP-ready endpoint for reads and writes.

Start 14-Day Trial

Generic review criteria catch generic bugs. The real value of an automated reviewer is catching violations of conventions that exist only inside your team's collective knowledge: the ORM you mandate, the test framework you standardized on, the error handling pattern you adopted after an incident.

Hermes Agent's persistent memory system lets you store these conventions so the agent applies them across every review without being reminded. In a chat session with Hermes, state your conventions directly:

"In our backend, all endpoints require type annotations and Pydantic models. We enforce SQLAlchemy ORM exclusively, no raw SQL. Test files live in tests/ using pytest fixtures. Error responses must use our standard ErrorResponse schema."

The agent saves this to its memory and applies it in future sessions, including cron-triggered review jobs. Over time, you build up a reviewer that knows your codebase's rules the way a senior engineer does after six months on the team.

You can layer conventions incrementally. After a post-incident review reveals a new class of bug, tell the agent: "From now on, flag any database migration that adds a NOT NULL column without a default value." The memory accumulates, and each review gets more specific to your codebase.

This is where Hermes Agent diverges most from hosted review bots. CodeRabbit and Qodo offer custom rules, but they're configured through a web dashboard or YAML config. Hermes conventions are stated in natural language and retained through memory, which makes them easier to add but harder to audit. If auditability matters, consider periodically exporting your stored conventions and reviewing them as a team.

With the skill created and conventions loaded, the last step is scheduling the review job. Hermes Agent's built-in cron scheduler accepts standard cron expressions, interval shorthand, and one-shot delays.

The basic command structure:

hermes cron create "0 */2 * * *" \
  "Review all open PRs on my-org/my-repo. Use gh pr list to find open PRs, then gh pr diff for each one. Report findings with severity levels." \
  --name pr-review \
  --deliver telegram \
  --skill code-review

This creates a job that runs every two hours, uses the code-review skill, and delivers results to Telegram. The prompt must be self-contained because each cron run starts a fresh agent session with no memory of previous chat conversations (though stored conventions from the memory system persist).

Common schedule patterns:

• 0 */2 * * * polls every 2 hours, good for active repositories
• 0 9,13,17 * * 1-5 runs three times on weekdays (morning, lunch, end of day)
• 0 9 * * 1 runs weekly on Monday mornings for lower-volume repos

The --deliver flag routes output to your messaging platform. Options include telegram, discord, slack, email, and local. Local delivery writes results to ~/.hermes/cron/output/, useful for debugging the review job before routing it to a team channel.

Multi-repo monitoring works by listing multiple repositories in the prompt itself. The agent processes them sequentially within a single job, so you don't need separate cron entries for each repo.

For teams that want reviews posted directly as GitHub PR comments rather than delivered to a chat channel, the agent can use gh pr review with --comment or --request-changes flags. Include this instruction in your cron prompt: "For critical findings, post a review requesting changes. For warnings and suggestions, post a comment."

Cron job output disappears into a chat thread or a local file. For teams that need a historical record of reviews, routing artifacts to a shared workspace creates an auditable trail.

Local storage options include writing to a git-tracked directory or a shared drive. S3 buckets work for teams already on AWS. Google Drive and Dropbox offer browser-accessible storage with search.

Fastio adds a layer that's useful for agent workflows specifically. The free plan includes 50 GB of storage, 5 workspaces, and included credits, with no credit card required. You can connect Hermes Agent to a Fastio workspace through the MCP server and have the agent write each review to a file in a dedicated workspace. Because Fastio auto-indexes uploaded files when Intelligence Mode is enabled, you can later search across all historical reviews by meaning, not just filename. Ask "which PRs had critical security findings in the last month?" and get cited answers.

The ownership transfer feature matters here too. An agent can build out a workspace with review history, dashboards, and summary files, then transfer ownership to a team lead who manages access for the rest of the engineering team. The agent retains admin access for continued writes.

To connect Hermes Agent to Fastio, configure the MCP server endpoint at /storage-for-agents/ in your agent's tool configuration. The agent can then use Fastio's storage, AI, and workspace tools alongside its existing GitHub CLI access, creating a pipeline where PR diffs flow in from GitHub and review artifacts flow out to a searchable workspace.