How to Set Up a Hermes Agent Email Bot

Nous Research's Hermes Agent uses a messaging gateway architecture to connect to external platforms. Email is one adapter among several (Telegram, Discord, Slack, WhatsApp, Signal), but it has a unique advantage: IMAP and SMTP are universal protocols supported by every major provider. No proprietary bot API, no webhook registration, no OAuth dance.

The email adapter polls your IMAP inbox for unseen messages at a configurable interval (15 seconds by default). When a new message arrives, the adapter strips HTML, extracts plain text and any attachments, then passes everything to the agent as a conversational turn. The agent processes the message, generates a response, and sends a reply through SMTP with proper In-Reply-To and References headers to maintain threading.

This approach works with Gmail, Outlook, Yahoo, Fastmail, ProtonMail (via Bridge), iCloud, self-hosted Dovecot/Postfix, or any other provider that exposes IMAP and SMTP endpoints. You are not locked into a vendor's bot framework or limited by API rate tiers that change with pricing plans.

One important architectural detail: email falls back to non-streaming mode automatically. Unlike Telegram where the agent can stream tokens in real time through message edits, email responses arrive as complete messages. This matches how humans expect email to work, so it is a natural fit rather than a limitation.

Before configuring email, you need Hermes Agent installed and running. The quickstart guide covers installation, but the short version is: Hermes runs on your server (local machine, Docker, SSH, Modal, or Singularity) and stores configuration in ~/.hermes/.

Use a dedicated email account. This is not optional for production use. The agent stores the email password in ~/.hermes/.env and has full inbox access via IMAP. Using your personal email means the agent can read every message you have ever received. Create a new account specifically for the agent, something like agent@yourdomain.com or a fresh Gmail address.

Enable two-factor authentication and generate an app password. Most providers (Gmail, Outlook, iCloud) require app-specific passwords when third-party applications access IMAP. For Gmail specifically:

1. Enable 2-Step Verification on the Google account
2. Go to myaccount.google.com/apppasswords
3. Generate a new app password (select "Mail" and your device type)
4. Copy the 16-character string (strip the spaces Google adds for display)

Always configure an allowlist. Without EMAIL_ALLOWED_USERS, anyone who discovers your agent's email address can send it commands. Since Hermes agents have terminal access by default, an unsecured email gateway is a remote code execution vulnerability. Lock it down before going live.

Hermes provides two configuration paths: an interactive wizard and manual environment variable setup.

Interactive Setup

Run the gateway setup command:

hermes gateway setup

The wizard prompts for your email address, password, IMAP host, SMTP host, and allowed senders. It writes the configuration to ~/.hermes/.env automatically.

Manual Configuration

If you prefer to configure directly, add these variables to ~/.hermes/.env:

# Required
EMAIL_ADDRESS=agent@yourdomain.com
EMAIL_PASSWORD=your-app-password-here
EMAIL_IMAP_HOST=imap.gmail.com
EMAIL_SMTP_HOST=smtp.gmail.com

# Optional (shown with defaults)
EMAIL_IMAP_PORT=993
EMAIL_SMTP_PORT=587
EMAIL_POLL_INTERVAL=15
EMAIL_ALLOWED_USERS=you@example.com,teammate@example.com

Provider-Specific Settings

Gmail:

• IMAP: imap.gmail.com (port 993, SSL)
• SMTP: smtp.gmail.com (port 587, STARTTLS)
• Password: App password (16 characters, no spaces)

Outlook / Microsoft 365:

• IMAP: outlook.office365.com (port 993, SSL)
• SMTP: smtp.office365.com (port 587, STARTTLS)
• Password: App password from Microsoft account security settings

Fastmail:

• IMAP: imap.fastmail.com (port 993, SSL)
• SMTP: smtp.fastmail.com (port 587, STARTTLS)
• Password: App password generated in Fastmail settings

Yahoo:

• IMAP: imap.mail.yahoo.com (port 993, SSL)
• SMTP: smtp.mail.yahoo.com (port 587, STARTTLS)
• Password: App password from Yahoo account security

iCloud Mail:

• IMAP: imap.mail.me.com (port 993, SSL)
• SMTP: smtp.mail.me.com (port 587, STARTTLS)
• Password: App-specific password (requires 2FA enabled)

After saving the .env file, start the gateway:

hermes gateway

The gateway process connects to all configured platforms and begins polling. You should see a log confirming the email adapter connected successfully.

Persist your email bot's outputs in a shared workspace

Fast.io gives Hermes Agent 50GB of indexed, searchable storage. Upload processed attachments, extracted data, and generated reports where your whole team can find them. No credit card, no expiration.

Sign Up Free

Hermes implements a three-tier access control model for email:

Tier 1: Allowlist enforcement. Set EMAIL_ALLOWED_USERS to a comma-separated list of email addresses. Only messages from these senders are processed. Everything else is silently ignored. This is the recommended configuration for any agent with terminal access or sensitive skills.

EMAIL_ALLOWED_USERS=alice@company.com,bob@company.com,ops-team@company.com

Tier 2: Pairing code. If an unrecognized sender contacts the agent and you have not set EMAIL_ALLOW_ALL_USERS=true, the agent can prompt with a pairing code challenge. This is useful for agents that need to onboard new users dynamically while still gating initial access.

Tier 3: Unrestricted. Setting EMAIL_ALLOW_ALL_USERS=true accepts messages from any sender. Only use this for agents that have no dangerous capabilities, like a public FAQ responder with no file system access.

For most deployments, Tier 1 is the right choice. You can add a EMAIL_HOME_ADDRESS variable to designate a default delivery target for scheduled cron job outputs, keeping automated reports flowing to the right inbox without manual intervention.

Restricting to Specific Folders

The adapter reads from INBOX by default. For a noisy inbox shared with other services, you have two options: configure server-side filters to route agent-bound messages to a dedicated folder, or use email alias routing (e.g., agent+tasks@gmail.com) combined with a Gmail filter that labels and archives incoming messages for the agent to process from a specific label.

The email adapter processes attachments automatically when messages arrive:

Image attachments (JPEG, PNG, GIF, WebP) are cached locally and passed to the agent's vision capabilities. If your underlying model supports multimodal input (GPT-4o, Claude, Gemini), the agent can analyze charts, read handwritten notes, describe screenshots, or extract text from photos. Send an image of a whiteboard and ask the agent to transcribe the diagrams into structured notes.

Document attachments (PDF, ZIP, and other files) are available through the agent's file access tools. The agent can read PDF content, extract data from compressed archives, and process the results according to your instructions.

Skipping attachments: For agents that only need text conversations, add skip_attachments: true to your config.yaml to reduce processing overhead and local storage use.

Persisting Processed Outputs

One challenge with email-driven agent workflows is that processed outputs often live only in the agent's local file system. When the agent extracts data from a PDF attachment or generates a report based on an email thread, that output disappears if the session ends or the server restarts.

Fast.io workspaces solve this by giving agents persistent, shareable storage. Configure the Fast.io MCP server as a Hermes skill, and your email bot can upload processed documents, extracted data, and generated reports to a workspace. From there, teammates access the files through the web UI, search them with Intelligence Mode, or query them through chat. The free tier includes 50GB of storage, 5,000 credits per month, and five workspaces with no credit card required.

This pattern works well for email triage workflows: the agent processes incoming messages, extracts relevant data, uploads structured outputs to a shared workspace, and sends a summary reply to the sender. Humans review the organized workspace rather than digging through email threads.

Verifying the Connection

After starting the gateway, send a test email from an allowed address to your agent's inbox. Within 15 seconds (or your configured EMAIL_POLL_INTERVAL), you should receive a reply. If nothing happens:

1. Check the gateway logs for connection errors
2. Verify the IMAP host and port are correct for your provider
3. Confirm you are using an app password, not your regular account password
4. Ensure the sender address is in EMAIL_ALLOWED_USERS

Common Issues

Authentication failures: The most common cause is using a regular password instead of an app password, or including spaces in the 16-character Gmail app password. Strip all spaces before pasting into .env.

Connection timeouts: Some corporate firewalls block IMAP port 993 and SMTP port 587. Test connectivity with openssl s_client -connect imap.gmail.com:993 to confirm the port is reachable from your server.

Slow responses: The default 15-second polling interval means worst-case latency is 15 seconds plus processing time. For near-instant responses, reduce EMAIL_POLL_INTERVAL to 5 or even 3, but be aware that aggressive polling can trigger rate limits on some providers.

Threading breaks: Some email clients (particularly older Outlook versions) do not honor In-Reply-To headers consistently. If replies appear as new threads instead of continuing the conversation, the issue is on the receiving client side, not the agent. The agent correctly sets both In-Reply-To and References headers per RFC 2822.

Self-message loops: The adapter automatically filters messages sent from the agent's own address and common automated senders (noreply addresses, mailer-daemon). If you still see loops, check that EMAIL_ADDRESS exactly matches the From address in outgoing messages.

Reducing Polling Interval

For time-sensitive workflows, set a lower interval:

EMAIL_POLL_INTERVAL=5

Keep in mind that some providers enforce connection rate limits. Gmail allows up to 15 IMAP connections per account and roughly 2,500 IMAP operations per 10 minutes. A 5-second interval with light inbox traffic stays well within these bounds.

Alternative: AgentMail for Dedicated Inboxes

If you need the agent to have its own email identity rather than accessing an existing inbox, the AgentMail skill provides dedicated agent-owned addresses (like your-agent@agentmail.to). This separates the agent's correspondence from any human inbox and is useful for autonomous outreach, service signups that require email verification, or agent-to-agent communication.