Skip to main content

Email System Design

Every agent on the platform gets its own email address in the form [email protected]. This is not just a vanity feature -- email is one of the primary channels through which agents interact with the outside world. The system handles both inbound (receiving) and outbound (sending) email with a rules engine that gives workspace owners fine-grained control over what gets through.

Inbound Email Flow​

When someone sends an email to an agent, the message passes through several systems before reaching the agent:

  1. SendGrid Inbound Parse: SendGrid receives email for agents.xpress.ai and forwards the parsed content (headers, body, attachments) to the platform via webhook
  2. Cloudflare Email Worker: The Cloudflare Email Worker handles DNS-level MX routing before forwarding to the platform's webhook endpoint. It sits in front of SendGrid to manage domain-level email routing and DNS resolution, ensuring that inbound email for agents.xpress.ai is directed to the correct processing pipeline.
  3. Platform Webhook: The AgentEmailResource receives the parsed email and identifies the target agent by the local part of the To address
  4. Rules Engine: The email passes through the agent's rules (described below) to determine where it goes

Deduplication​

The platform tracks processed email IDs to prevent the same message from being handled twice. This matters because webhook delivery is at-least-once -- SendGrid may retry if the initial delivery times out, and without deduplication the agent would process the same email multiple times.

Attachment Handling​

Email attachments are stored on the platform when received. Attachments are subject to size limits and are forwarded to the agent along with the email body. Large attachments that exceed the size limit are rejected at the webhook level to prevent storage abuse.

Outbound Email Flow​

When an agent wants to send an email, the flow includes an optional human approval step:

  1. The agent calls the send_email tool with recipients, subject, and body
  2. The outbound rules engine checks whether the email requires approval
  3. If approval required: the email is placed in the outbox, and a notification is sent to the workspace owner. The human can approve or reject the email.
  4. If no approval required: the email is sent immediately via SendGrid
tip

Outbound approval is especially useful during the early days of deploying an agent. You can require approval for all outbound emails, review what the agent is sending, and then relax the rules once you trust its behavior. This is much safer than letting a newly deployed agent email your customers unsupervised.

Email Rules Engine​

Each agent has an ordered list of email rules. Rules are processed top-to-bottom, and the first matching rule determines what happens to the email. This is similar to how email filters work in Gmail or Outlook, but applied at the agent level.

Each rule has:

  • Conditions: Match on sender address, subject line, or other headers (supports exact match and pattern matching)
  • Action: What to do when the conditions match -- forward to agent queue, place in inbox for human review, move to trash, or require approval for outbound

If no rule matches, the default behavior is to forward the email to the agent's queue for processing.

Known Contacts​

The known contacts list is a curated set of approved senders. Depending on the agent's configuration:

  • Auto-approved: Emails from known contacts bypass the rules engine and go directly to the agent queue
  • Manually managed: The workspace owner adds and removes contacts as needed

This provides a simple allowlist mechanism -- if you know that [email protected] should always reach the agent, add them to known contacts and their emails skip the rules entirely.

Trash and Soft Delete​

Trashed emails are soft-deleted, meaning they are marked as deleted but remain in the database. This allows:

  • Workspace owners to review what was trashed and restore emails that were incorrectly filtered
  • Audit trails for compliance purposes
  • Recovery from overly aggressive rules during initial setup

Design Trade-Offs​

Why SendGrid instead of running our own SMTP server? Running an SMTP server requires managing IP reputation, SPF/DKIM/DMARC records, bounce handling, and spam filtering. SendGrid handles all of this. The cost is a per-email fee and a dependency on a third-party service, but the operational simplicity is worth it for a platform that is not primarily an email product.

Why a rules engine instead of just forwarding everything to the agent? Without rules, agents would need to process every spam email, phishing attempt, and newsletter that arrives at their address. The rules engine filters noise before it reaches the agent, saving LLM tokens and preventing agents from acting on malicious emails.

Why human approval for outbound? Agents can make mistakes -- sending confidential information to the wrong recipient, using inappropriate tone, or responding to phishing emails. Outbound approval gives humans a safety net, especially during the trust-building phase of a new agent deployment.

See Also​