Deploying Cyrus IMAP

Cyrus IMAP is a highly scalable, enterprise-grade email server developed by Carnegie Mellon University. Originally designed to handle the email infrastructure for large organizations, Cyrus IMAP offers robust performance, advanced features, and exceptional reliability. Unlike traditional UNIX mail servers that store each user’s mail in a single file, Cyrus uses a sophisticated database-backed architecture that provides better performance, scalability, and data integrity.

The server supports industry-standard protocols including IMAP4rev1, POP3, NNTP, and LMTP, while offering unique features like murder topology for distributed deployments, mailbox replication for high availability, shared folders with fine-grained access controls, server-side message filtering with Sieve, and efficient mail delivery with single-instance message storage. Cyrus IMAP is trusted by universities, corporations, and service providers worldwide to handle millions of mailboxes and billions of messages.

Why Deploy Cyrus IMAP on Klutch.sh?

Klutch.sh provides an excellent platform for hosting Cyrus IMAP with several key advantages:

• Simple Docker Deployment: Deploy your Dockerfile and Klutch.sh automatically handles the containerization and orchestration
• TCP Traffic Support: Direct TCP access on port 8000 for IMAP/POP3 protocols, with configurable internal port routing
• Persistent Storage: Attach volumes to store mailboxes, indexes, and configuration data with guaranteed durability
• Automatic HTTPS: Secure IMAP connections with automatic SSL certificate provisioning
• Resource Scalability: Scale CPU and memory resources as your mail infrastructure grows
• High Availability: Deploy multiple instances with replication for fault tolerance
• Cost Efficiency: Pay only for the resources you use without expensive enterprise licensing

Prerequisites

Before deploying Cyrus IMAP, ensure you have:

• A Klutch.sh account (sign up at klutch.sh)
• Git installed locally
• Basic understanding of email protocols (IMAP, SMTP, LMTP)
• OpenSSL for generating SSL certificates
• Domain name with DNS access for mail server configuration
• Basic knowledge of email authentication (SPF, DKIM, DMARC)

Understanding Cyrus IMAP’s Architecture

Cyrus IMAP employs a unique architecture that differs significantly from traditional mail servers:

Core Components

Master Process: The Cyrus master daemon manages all service processes, handling spawning, monitoring, and restart of services. It reads the cyrus.conf configuration file to determine which services to start and how to manage them.

IMAP Daemon: Handles IMAP protocol connections, allowing clients to access, search, and manipulate mail messages. Supports IMAP4rev1 with extensions like IDLE, SORT, THREAD, and NAMESPACE.

POP3 Daemon: Provides POP3 access for clients that prefer the simpler download-and-delete protocol over IMAP’s more sophisticated mailbox management.

LMTP Daemon: Local Mail Transfer Protocol daemon receives mail from your MTA (Postfix, Sendmail, etc.) and delivers it to user mailboxes. LMTP is more efficient than using a separate local delivery agent.

Database Backend: Cyrus stores mailbox metadata, subscriptions, and seen state in various databases:

• Skiplist: Fast, crash-resistant database format (default)
• Berkeley DB: Alternative database backend
• MySQL/PostgreSQL: External database support for larger deployments

Authentication Layer: Supports multiple authentication mechanisms through SASL (Simple Authentication and Security Layer):

• PLAIN, LOGIN for basic authentication
• DIGEST-MD5, CRAM-MD5 for challenge-response
• GSSAPI for Kerberos integration
• External authentication via PAM, LDAP, SQL

Storage Architecture: Messages are stored in a structured directory hierarchy with:

• Single-instance storage (identical messages stored once)
• Message files named by unique identifiers
• Index files for fast message retrieval
• Cache files for performance optimization
• Quota tracking at mailbox and partition levels

Data Flow

1. MTA receives email and forwards to Cyrus via LMTP
2. LMTP daemon authenticates delivery credentials
3. Sieve scripts process message (if configured) for filtering and sorting
4. Message is stored in the appropriate mailbox partition
5. Index and cache files are updated for fast retrieval
6. IMAP/POP3 clients connect and authenticate via SASL
7. Client requests are processed using the index database
8. Messages are delivered to client with efficient streaming

Mailbox Hierarchy

Cyrus uses a hierarchical namespace system:

• user.username: Personal mailboxes (e.g., user.john, user.john.Sent)
• shared.foldername: Shared folders accessible by multiple users
• Partitions: Physical storage locations for distributing mailbox data
• ACLs: Fine-grained access control lists for each mailbox

Installation and Setup

Step 1: Create the Dockerfile

Create a Dockerfile in your project root. We’ll build from Ubuntu LTS for stability:

FROM ubuntu:22.04

# Prevent interactive prompts during package installation
ENV DEBIAN_FRONTEND=noninteractive

# Install Cyrus IMAP and dependencies
RUN apt-get update && apt-get install -y \
    cyrus-imapd \
    cyrus-admin \
    cyrus-clients \
    cyrus-common \
    sasl2-bin \
    libsasl2-modules \
    libsasl2-modules-sql \
    ssl-cert \
    openssl \
    vim \
    net-tools \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Create necessary directories
RUN mkdir -p /var/lib/cyrus \
    /var/spool/cyrus/mail \
    /var/spool/cyrus/news \
    /var/spool/cyrus/db \
    /var/spool/imap \
    /run/cyrus/socket \
    && chown -R cyrus:mail /var/lib/cyrus /var/spool/cyrus /var/spool/imap /run/cyrus

# Copy configuration files
COPY imapd.conf /etc/imapd.conf
COPY cyrus.conf /etc/cyrus.conf
COPY sasl/smtpd.conf /etc/sasl2/smtpd.conf

# Generate SSL certificates (replace with your own in production)
RUN openssl req -new -x509 -days 365 -nodes \
    -out /etc/ssl/certs/cyrus-imapd.pem \
    -keyout /etc/ssl/private/cyrus-imapd.key \
    -subj "/C=US/ST=State/L=City/O=Organization/CN=mail.example.com"

# Set proper permissions for SSL certificates
RUN chown cyrus:mail /etc/ssl/certs/cyrus-imapd.pem /etc/ssl/private/cyrus-imapd.key \
    && chmod 640 /etc/ssl/private/cyrus-imapd.key

# Expose IMAP, IMAPS, POP3, POP3S, LMTP, and SIEVE ports
EXPOSE 143 993 110 995 24 4190

# Create startup script
COPY docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh

# Health check
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
    CMD cyradm --user cyrus --password admin localhost <<< "lm" || exit 1

ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]

Step 2: Create Configuration Files

Create imapd.conf for Cyrus IMAP configuration:

# Cyrus IMAP Configuration File
# Network configuration
servername: mail.example.com
admin: cyrus admin

# Directories
configdirectory: /var/lib/cyrus
partition-default: /var/spool/cyrus/mail
sievedir: /var/spool/sieve
proc_path: /run/cyrus/proc

# TLS/SSL configuration
tls_server_cert: /etc/ssl/certs/cyrus-imapd.pem
tls_server_key: /etc/ssl/private/cyrus-imapd.key
tls_client_ca_file: /etc/ssl/certs/ca-certificates.crt

# Authentication
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN DIGEST-MD5 CRAM-MD5
allowplaintext: yes
sasl_auto_transition: no

# Authorization
admins: cyrus admin
postmaster: postmaster

# Mailbox options
unixhierarchysep: yes
altnamespace: yes
hashimapspool: yes
sieveusehomedir: no
sieve_extensions: fileinto reject vacation imapflags notify include envelope body relational regex subaddress copy

# Performance tuning
lmtp_downcase_rcpt: yes
mboxname_lockpath: /run/cyrus/lock
duplicate_db_path: /var/lib/cyrus/deliver.db
statuscache_db_path: /var/lib/cyrus/statuscache.db
ptscache_db_path: /var/lib/cyrus/ptscache.db

# Quotas
quotawarn: 90
autocreate_quota: 100000

# Logging
syslog_prefix: cyrus

# Misc
lmtpsocket: /run/cyrus/socket/lmtp
idlesocket: /run/cyrus/socket/idle
notifysocket: /run/cyrus/socket/notify

# Database backends
annotation_db: skiplist
duplicate_db: skiplist
mboxkey_db: skiplist
mboxlist_db: skiplist
ptscache_db: skiplist
quota_db: quotalegacy
seenstate_db: skiplist
statuscache_db: skiplist
subscription_db: flat
tlscache_db: skiplist

# Single instance store
singleinstancestore: yes

Create cyrus.conf for service management:

# Cyrus IMAP Service Configuration
START {
  # Master process
  recover       cmd="ctl_cyrusdb -r"

  # Replication (uncomment for replicated setups)
  # mupdatepush cmd="ctl_mboxlist -m"
}

SERVICES {
  # IMAP services
  imap          cmd="imapd -U 30" listen="0.0.0.0:imap" prefork=5 maxchild=100
  imaps         cmd="imapd -s -U 30" listen="0.0.0.0:imaps" prefork=1 maxchild=100

  # POP3 services
  pop3          cmd="pop3d -U 30" listen="0.0.0.0:pop3" prefork=3 maxchild=50
  pop3s         cmd="pop3d -s -U 30" listen="0.0.0.0:pop3s" prefork=1 maxchild=50

  # LMTP service
  lmtp          cmd="lmtpd" listen="0.0.0.0:lmtp" prefork=1 maxchild=20

  # Sieve service
  sieve         cmd="timsieved" listen="0.0.0.0:sieve" prefork=0 maxchild=10

  # Notification daemon
  notify        cmd="notifyd" listen="/run/cyrus/socket/notify" proto="udp" prefork=1

  # HTTP services (for CalDAV/CardDAV if needed)
  # http        cmd="httpd" listen="8080" prefork=1 maxchild=10
}

EVENTS {
  # Checkpoint the database every 30 minutes
  checkpoint    cmd="ctl_cyrusdb -c" period=30

  # Delete expired messages daily
  delprune      cmd="cyr_expire -E 3" at=0400

  # Delete expired mailbox tombstones
  deleteprune   cmd="cyr_expire -D 30" at=0430

  # Compress old database files
  tlsprune      cmd="tls_prune" at=0400

  # Synchronize seen state (optional)
  # sync_client cmd="sync_client -r" period=30
}

Create sasl/smtpd.conf for SASL authentication:

# SASL configuration for Cyrus IMAP
pwcheck_method: saslauthd
mech_list: PLAIN LOGIN DIGEST-MD5 CRAM-MD5
saslauthd_path: /var/run/saslauthd/mux

Step 3: Create Docker Entrypoint Script

Create docker-entrypoint.sh:

#!/bin/bash
set -e

# Initialize Cyrus database if not exists
if [ ! -f /var/lib/cyrus/mailboxes.db ]; then
    echo "Initializing Cyrus IMAP database..."
    su - cyrus -s /bin/bash -c "ctl_cyrusdb -r"

    # Create default admin user
    echo "admin" | saslpasswd2 -p -c cyrus
    echo "admin" | saslpasswd2 -p -c admin
fi

# Start saslauthd
echo "Starting SASL authentication daemon..."
mkdir -p /var/run/saslauthd
saslauthd -a sasldb -m /var/run/saslauthd &

# Wait for saslauthd to start
sleep 2

# Start Cyrus IMAP master process
echo "Starting Cyrus IMAP server..."
exec /usr/lib/cyrus/bin/master -d

Step 4: Create Docker Compose for Local Development

Create docker-compose.yml for local testing:

version: '3.8'

services:
  cyrus-imap:
    build: .
    ports:
      - "143:143"    # IMAP
      - "993:993"    # IMAPS
      - "110:110"    # POP3
      - "995:995"    # POP3S
      - "24:24"      # LMTP
      - "4190:4190"  # Sieve
    volumes:
      - cyrus-mail:/var/spool/cyrus/mail
      - cyrus-config:/var/lib/cyrus
      - cyrus-sieve:/var/spool/sieve
    environment:
      - DEBIAN_FRONTEND=noninteractive
    restart: unless-stopped
    hostname: mail.example.com

volumes:
  cyrus-mail:
  cyrus-config:
  cyrus-sieve:

Step 5: Initialize Git Repository

git init
git add Dockerfile imapd.conf cyrus.conf docker-entrypoint.sh docker-compose.yml
git add sasl/
git commit -m "Initial Cyrus IMAP deployment configuration"

Step 6: Test Locally

Before deploying to Klutch.sh, test your configuration locally:

# Build and start the container
docker-compose up -d

# Check logs
docker-compose logs -f

# Test IMAP connection
telnet localhost 143

# Test authentication
docker-compose exec cyrus-imap cyradm --user cyrus --password admin localhost

Deploying to Klutch.sh

Step 1: Push to GitHub

Create a new repository on GitHub and push your code:

git remote add origin https://github.com/yourusername/cyrus-imap-klutch.git
git branch -M master
git push -u origin master

Step 2: Connect Repository to Klutch.sh

1. Navigate to klutch.sh/app
2. Click "New Project" and select "Import from GitHub"
3. Authorize Klutch.sh to access your GitHub repositories
4. Select your Cyrus IMAP repository from the list
5. Klutch.sh will automatically detect the Dockerfile in your repository

Step 3: Configure Traffic Settings

1. In the project settings, select **TCP** as the traffic type
2. Set the internal port to **143** for IMAP protocol
3. Your Cyrus IMAP server will be accessible on port **8000** externally
4. For secure connections (IMAPS), you can alternatively use internal port **993**

Note: Clients will connect to your-app.klutch.sh:8000, and traffic will be routed to the container’s internal IMAP port.

Step 4: Add Persistent Storage

Cyrus IMAP requires persistent volumes for storing mailbox data, configuration, and indexes:

1. In your project settings, navigate to the "Storage" section
2. Add a volume with mount path: `/var/spool/cyrus/mail` and size: `50GB` (for mailbox storage)
3. Add a volume with mount path: `/var/lib/cyrus` and size: `10GB` (for database and configuration)
4. Add a volume with mount path: `/var/spool/sieve` and size: `5GB` (for Sieve scripts)

Storage recommendations based on usage:

• Mailbox storage: Start with 50GB, scale to 500GB+ for large deployments
• Configuration/Database: 10GB is typically sufficient
• Sieve scripts: 5GB handles thousands of user scripts

Step 5: Configure Environment Variables

Add the following environment variables in your Klutch.sh dashboard:

• DEBIAN_FRONTEND: noninteractive
• CYRUS_ADMIN_PASSWORD: Your secure admin password (optional, can be set post-deployment)

Step 6: Deploy

1. Review your configuration settings
2. Click "Deploy" to start the deployment process
3. Klutch.sh will build your Docker image and deploy the container
4. Monitor the build logs for any errors
5. Once deployed, your mail server will be available at `your-app.klutch.sh:8000`

Getting Started with Cyrus IMAP

Initial Setup

After your first deployment, complete the initial setup:

1. Access the Container: Use the Klutch.sh console or SSH to access your container

2. Set Admin Password: Connect to the running container and set a secure admin password:

# Set password for cyrus admin user
echo "your-secure-password" | saslpasswd2 -p -c cyrus

3. Verify Service Status: Check that all services are running:

# View running Cyrus processes
ps aux | grep cyrus

# Check service logs
tail -f /var/log/syslog | grep cyrus

Creating Mailboxes

Use the cyradm command-line tool to manage mailboxes:

# Connect to Cyrus admin interface
cyradm --user cyrus --password your-secure-password localhost

# Create a user mailbox
cm user.john

# Create shared mailbox
cm shared.announcements

# List all mailboxes
lm

# Set quota (in kilobytes)
sq user.john 1048576

# List quotas
lq user.john

# Set ACL (permissions)
sam user.john john lrswipkxtecda

# Quit cyradm
quit

User Management

Create and manage email users:

# Create a new user with SASL authentication
echo "user-password" | saslpasswd2 -p -c john

# List all SASL users
sasldblistusers2

# Delete a user
saslpasswd2 -d john

# Change user password
echo "new-password" | saslpasswd2 -p -c john

Access Control Lists (ACLs)

Cyrus IMAP uses a sophisticated ACL system:

# ACL permissions:
# l - lookup (mailbox is visible)
# r - read (read messages)
# s - seen flag (mark messages as read)
# w - write (set flags except \Seen and \Deleted)
# i - insert (post mail, copy mail)
# p - post (send mail to submission address)
# k - create mailboxes (create child mailboxes)
# x - delete mailbox (delete mailbox)
# t - delete messages (mark for deletion)
# e - expunge (expunge deleted messages)
# c - create (create child mailboxes, deprecated)
# d - delete (delete messages and mailbox, deprecated)
# a - administer (set ACLs)

# Grant full access to user
sam user.john john lrswipkxtecda

# Grant read-only access to another user
sam user.john jane lr

# Grant access to shared folder
sam shared.announcements john lrs
sam shared.announcements jane lrs

# Remove user access
dam user.john jane

Configuring Mail Client

Configure your email client to connect to Cyrus IMAP:

IMAP Settings

Server: your-app.klutch.sh
Port: 8000 (or 993 for IMAPS if configured)
Username: john
Password: user-password
Security: STARTTLS (or SSL/TLS for IMAPS)
Authentication: Normal password (PLAIN/LOGIN)

POP3 Settings (if enabled)

Server: your-app.klutch.sh
Port: 8000 (mapped to internal 110 or 995)
Username: john
Password: user-password
Security: STARTTLS (or SSL/TLS)

Testing with Command Line

Test your Cyrus IMAP server using command-line tools:

# Test IMAP connection with OpenSSL
openssl s_client -connect your-app.klutch.sh:8000 -starttls imap

# Manual IMAP commands
A01 LOGIN john user-password
A02 LIST "" "*"
A03 SELECT INBOX
A04 FETCH 1:* (FLAGS)
A05 LOGOUT

# Test with telnet (non-TLS)
telnet your-app.klutch.sh 8000

Sieve Mail Filtering

Configure server-side mail filtering with Sieve:

1. Create Sieve Script: Create a file filter.sieve:

require ["fileinto", "reject", "envelope"];

# Filter spam to Junk folder
if header :contains "X-Spam-Flag" "YES" {
    fileinto "INBOX.Junk";
    stop;
}

# Filter mailing list messages
if header :contains "List-Id" "developers.example.com" {
    fileinto "INBOX.Lists.Developers";
    stop;
}

# Vacation auto-reply
if header :contains "subject" "urgent" {
    vacation :days 7 :subject "Out of Office"
        "I am currently out of office. For urgent matters, contact admin@example.com.";
}

# Default: keep in INBOX
keep;

2. Upload Sieve Script:

# Use sieveshell to manage scripts
sieveshell --user=john --authname=john your-app.klutch.sh

# Upload script
put filter.sieve

# Activate script
activate filter

# List scripts
list

# Quit
quit

3. Test Sieve Script:

# Use sieve-test to validate syntax
sieve-test filter.sieve test-email.txt

Production Best Practices

Security Hardening

1. Use Strong Passwords: Generate secure passwords for admin and user accounts:

# Generate random password
openssl rand -base64 32

# Set strong admin password
echo "$(openssl rand -base64 32)" | saslpasswd2 -p -c cyrus

2. SSL/TLS Configuration: Replace self-signed certificates with proper SSL certificates:

# Generate CSR for certificate authority
openssl req -new -key /etc/ssl/private/cyrus-imapd.key \
    -out /etc/ssl/certs/cyrus-imapd.csr \
    -subj "/C=US/ST=State/L=City/O=Organization/CN=mail.yourdomain.com"

# After obtaining signed certificate from CA, update imapd.conf

Update imapd.conf:

tls_server_cert: /etc/ssl/certs/your-signed-cert.pem
tls_server_key: /etc/ssl/private/your-private-key.key
tls_client_ca_file: /etc/ssl/certs/ca-bundle.crt

# Enforce TLS
allowplaintext: no
tls_cipher_list: HIGH:MEDIUM:!aNULL:!MD5:!RC4

3. Disable Plain Text Authentication (after SSL is configured):

allowplaintext: no
sasl_minimum_layer: 128

4. Implement Firewall Rules: Restrict access to specific IP ranges if possible

5. Regular Security Updates: Keep Cyrus IMAP and dependencies updated:

# Add to Dockerfile for regular updates
RUN apt-get update && apt-get upgrade -y cyrus-imapd

High Availability Setup

For production environments requiring high availability:

1. Enable Replication: Configure master-replica replication

On the master server, update imapd.conf:

sync_log: yes
sync_log_channels: squatter

Update cyrus.conf:

SERVICES {
    # Add sync service
    sync        cmd="sync_server" listen="0.0.0.0:2005" prefork=1
}

On the replica server:

sync_client -S your-master-server

2. Murder Topology: For large deployments, implement Cyrus Murder architecture with frontend and backend servers

3. Load Balancing: Deploy multiple instances behind a load balancer for distributing client connections

Backup Strategy

Implement comprehensive backup procedures:

1. Mailbox Backup: Regularly backup the mail spool:

#!/bin/bash
# Backup script
BACKUP_DIR="/backups/cyrus_$(date +%Y%m%d_%H%M%S)"
mkdir -p $BACKUP_DIR

# Backup mail data
tar -czf $BACKUP_DIR/mail.tar.gz /var/spool/cyrus/mail

# Backup configuration
tar -czf $BACKUP_DIR/config.tar.gz /var/lib/cyrus

# Backup Sieve scripts
tar -czf $BACKUP_DIR/sieve.tar.gz /var/spool/sieve

# Keep only last 30 days
find /backups -name "cyrus_*.tar.gz" -mtime +30 -delete

2. Database Backup: Backup Cyrus databases:

# Create checkpoint before backup
ctl_cyrusdb -c

# Backup database files
cp -r /var/lib/cyrus/*.db $BACKUP_DIR/

3. User Backup: Backup SASL user database:

cp /etc/sasldb2 $BACKUP_DIR/

4. Incremental Backups: Use sync_client for incremental backups to replica servers

Performance Optimization

1. Database Tuning: Choose appropriate database backends for your workload:

# For high-performance deployments, consider:
mboxlist_db: skiplist
annotation_db: skiplist
duplicate_db: berkeley
seenstate_db: skiplist
statuscache_db: berkeley

2. Caching Configuration: Enable and tune caching:

# Enable caching
cachesize: 262144

# Tune metadata cache
metapartition-default: /var/spool/cyrus/meta

3. Process Limits: Adjust cyrus.conf service limits based on load:

SERVICES {
    imap    cmd="imapd -U 30" listen="0.0.0.0:imap" prefork=10 maxchild=200
    lmtp    cmd="lmtpd" listen="0.0.0.0:lmtp" prefork=5 maxchild=50
}

4. Filesystem Optimization: Use appropriate filesystem options:

# Mount mail partition with noatime for better performance
# Add to /etc/fstab:
# /dev/sdb1 /var/spool/cyrus/mail ext4 defaults,noatime 0 2

5. Index Optimization: Schedule regular index maintenance:

# Add to cyrus.conf EVENTS
EVENTS {
    # Optimize indexes
    squatter    cmd="squatter -r user" period=720
}

Monitoring and Logging

1. Enable Detailed Logging: Configure comprehensive logging in imapd.conf:

# Logging configuration
syslog_prefix: cyrus
syslog_facility: LOCAL6

Configure syslog to capture Cyrus logs:

# Add to /etc/rsyslog.conf
local6.*    /var/log/cyrus/cyrus.log

2. Monitor Service Health: Set up monitoring scripts:

health-check.sh

#!/bin/bash
cyradm --user cyrus --password admin localhost <<EOF
lm
quit
EOF

if [ $? -eq 0 ]; then
    echo "Cyrus IMAP is healthy"
    exit 0
else
    echo "Cyrus IMAP health check failed"
    exit 1
fi

3. Monitor Disk Usage: Track mailbox storage consumption:

# Check quota usage
cyradm --user cyrus --password admin localhost <<< "lq user.*"

# Check disk usage
du -sh /var/spool/cyrus/mail/*

4. Performance Metrics: Monitor connection statistics:

# View active connections
netstat -an | grep :143 | wc -l

# Check master process status
ps aux | grep 'cyrus/master'

Resource Allocation

Recommended resources based on deployment size:

Small Deployment (10-50 users):

• Memory: 2GB - 4GB
• CPU: 1 - 2 vCPU
• Storage: 50GB - 100GB

Medium Deployment (50-500 users):

• Memory: 4GB - 8GB
• CPU: 2 - 4 vCPU
• Storage: 100GB - 500GB

Large Deployment (500+ users):

• Memory: 8GB - 32GB
• CPU: 4 - 16 vCPU
• Storage: 500GB - 5TB+
• Consider murder topology with multiple backend servers

Troubleshooting

Cannot Connect to IMAP Server

Symptoms: Connection refused or timeout when connecting to IMAP port

Solutions:

1. Verify Service is Running:

ps aux | grep imapd
ps aux | grep master

# Check if ports are listening
netstat -tlnp | grep -E '143|993'

2. Check Firewall Settings:

# Ensure TCP port 8000 is accessible from Klutch.sh
# Verify internal port 143 is properly mapped

3. Review Logs:

tail -f /var/log/syslog | grep cyrus
journalctl -u cyrus-imapd -f

4. Test Local Connection:

telnet localhost 143
# Should see: * OK Cyrus IMAP server ready

Authentication Failures

Symptoms: Login attempts fail with “authentication failed” error

Solutions:

1. Verify User Exists:

sasldblistusers2
# Check if user is in the database

2. Test Authentication:

# Test with testsaslauthd
testsaslauthd -u john -p password -s imap

# Check saslauthd is running
ps aux | grep saslauthd

3. Reset User Password:

echo "new-password" | saslpasswd2 -p -c john

4. Check SASL Configuration:

# Verify /etc/sasl2/smtpd.conf
cat /etc/sasl2/smtpd.conf

# Check SASL mechanisms
imtest -m PLAIN -a john localhost

5. Review Authentication Logs:

grep -i "badlogin" /var/log/syslog
grep -i "auth" /var/log/syslog

Mailbox Access Errors

Symptoms: Unable to access mailboxes, “permission denied” errors

Solutions:

1. Check Mailbox Exists:

cyradm --user cyrus --password admin localhost
> lm user.john

2. Verify ACLs:

# Check mailbox permissions
cyradm --user cyrus --password admin localhost
> lam user.john

3. Recreate Mailbox (if corrupted):

cyradm --user cyrus --password admin localhost
> dm user.john
> cm user.john
> sam user.john john lrswipkxtecda

4. Check Filesystem Permissions:

ls -la /var/spool/cyrus/mail/
chown -R cyrus:mail /var/spool/cyrus/mail

Database Corruption

Symptoms: Mailbox list errors, database inconsistencies

Solutions:

1. Reconstruct Mailboxes:

# Reconstruct all mailboxes
reconstruct -r

# Reconstruct specific user
reconstruct user.john

2. Recover Database:

# Stop Cyrus
systemctl stop cyrus-imapd

# Recover database
ctl_cyrusdb -r

# Start Cyrus
systemctl start cyrus-imapd

3. Repair Quota:

quota -f user.john

4. Rebuild Indexes:

# Rebuild search indexes
squatter -r user.john

Performance Issues

Symptoms: Slow IMAP responses, high server load

Solutions:

1. Check Resource Usage:

# Monitor CPU and memory
top -u cyrus
vmstat 1

# Check I/O wait
iostat -x 1

2. Optimize Databases:

# Checkpoint and clean up
ctl_cyrusdb -c

# Rebuild indexes
squatter -r user

3. Review Process Limits:

# Check current limits
ulimit -a

# Increase in cyrus.conf if needed
maxchild=200 (in SERVICES section)

4. Analyze Slow Queries:

# Enable IMAP command logging
# Add to imapd.conf:
# debug: 1

# Review detailed logs
tail -f /var/log/syslog | grep IMAP

5. Partition Mailboxes: Distribute mailboxes across multiple partitions:

cyradm --user cyrus --password admin localhost
> cm user.john partition2

SSL/TLS Certificate Errors

Symptoms: Certificate validation failures, SSL handshake errors

Solutions:

1. Verify Certificate Files:

# Check certificate validity
openssl x509 -in /etc/ssl/certs/cyrus-imapd.pem -text -noout

# Verify key matches certificate
openssl x509 -noout -modulus -in /etc/ssl/certs/cyrus-imapd.pem | openssl md5
openssl rsa -noout -modulus -in /etc/ssl/private/cyrus-imapd.key | openssl md5

2. Check Certificate Permissions:

chown cyrus:mail /etc/ssl/certs/cyrus-imapd.pem
chown cyrus:mail /etc/ssl/private/cyrus-imapd.key
chmod 644 /etc/ssl/certs/cyrus-imapd.pem
chmod 640 /etc/ssl/private/cyrus-imapd.key

3. Test SSL Connection:

openssl s_client -connect localhost:993 -showcerts

Advanced Configuration

Murder Topology for Large Deployments

For organizations with millions of mailboxes, implement Cyrus Murder architecture:

1. Architecture Overview:

   • Frontend Servers: Handle client connections, proxy to backends
   • Backend Servers: Store actual mailbox data
   • MUPDATE Master: Maintains mailbox location database

2. Frontend Configuration (imapd.conf):

serverinfo: frontend
proxy_authname: murder
proxy_password: secret
mupdate_server: mupdate.example.com
mupdate_username: murder
mupdate_authname: murder

3. Backend Configuration (imapd.conf):

serverinfo: backend
mupdate_server: mupdate.example.com
mupdate_username: murder
mupdate_authname: murder
proxyservers: murder

4. MUPDATE Server Configuration:

serverinfo: mupdate
mupdate_admins: murder admin

CalDAV and CardDAV Support

Enable calendar and contact synchronization:

1. Enable HTTP Service in cyrus.conf:

SERVICES {
    http    cmd="httpd" listen="8080" prefork=1 maxchild=10
}

2. Configure CalDAV in imapd.conf:

caldav_realm: example.com
caldav_allowscheduling: yes

3. Create Calendar Mailboxes:

cyradm --user cyrus --password admin localhost
> cm user.john.#calendars
> sam user.john.#calendars john lrswipkxtecda

LDAP Authentication Integration

Integrate with LDAP directory services:

1. Install LDAP Support:

RUN apt-get install -y libsasl2-modules-ldap

2. Configure SASL LDAP (/etc/saslauthd.conf):

ldap_servers: ldap://ldap.example.com
ldap_search_base: ou=users,dc=example,dc=com
ldap_filter: (uid=%u)
ldap_bind_dn: cn=admin,dc=example,dc=com
ldap_bind_pw: admin-password

3. Update imapd.conf:

sasl_pwcheck_method: saslauthd
sasl_saslauthd_path: /var/run/saslauthd/mux

4. Start saslauthd with LDAP:

saslauthd -a ldap -m /var/run/saslauthd

Quota Management

Implement advanced quota policies:

1. Per-User Quotas:

# Set quota in MB
cyradm --user cyrus --password admin localhost
> sq user.john 1024000

# Check quota usage
> lq user.john

2. Domain-Wide Quotas:

# Set default quota for new users in imapd.conf
autocreate_quota: 1048576

3. Quota Warning Scripts:

quota-warning.sh

#!/bin/bash
for user in $(cyradm --user cyrus --password admin localhost <<< "lq user.*" | grep OVER); do
    echo "User $user is over quota"
    # Send warning email
done

Message Archiving

Implement email archiving for compliance:

1. Enable Archive Partition in imapd.conf:

archive_enabled: yes
archive_days: 2555
partition-archive: /var/spool/cyrus/archive
archivepartition-default: archive

2. Create Archive Policies:

# Move messages older than 7 years to archive
cyr_expire -X 2555 -a

3. Archive Management:

# List archived messages
cyradm --user cyrus --password admin localhost
> lm user.john.Archive.*

Email Filtering with External Tools

Integrate Cyrus with spam filtering:

1. SpamAssassin Integration: Configure LMTP to use SpamAssassin

2. Create Sieve Rule to handle spam:

require ["fileinto", "imap4flags"];

if header :contains "X-Spam-Flag" "YES" {
    setflag "\\Seen";
    fileinto "INBOX.Junk";
}

3. ClamAV Antivirus: Scan incoming messages for viruses

Backup and Restore Procedures

Detailed backup/restore operations:

1. Full Backup:

full-backup.sh

#!/bin/bash
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_ROOT="/backups/$TIMESTAMP"

# Stop Cyrus temporarily for consistent backup
systemctl stop cyrus-imapd

# Backup all data
mkdir -p $BACKUP_ROOT
tar -czf $BACKUP_ROOT/mail.tar.gz /var/spool/cyrus/mail
tar -czf $BACKUP_ROOT/config.tar.gz /var/lib/cyrus
tar -czf $BACKUP_ROOT/sieve.tar.gz /var/spool/sieve
cp /etc/imapd.conf $BACKUP_ROOT/
cp /etc/cyrus.conf $BACKUP_ROOT/
cp /etc/sasldb2 $BACKUP_ROOT/

# Restart Cyrus
systemctl start cyrus-imapd

2. Incremental Backup:

# Use sync_client for incremental backups
sync_client -S replica-server -u user.john

3. Restore Mailbox:

# Extract backup
tar -xzf mail.tar.gz -C /

# Reconstruct mailbox
reconstruct -r user.john

# Fix permissions
chown -R cyrus:mail /var/spool/cyrus/mail

API and Automation

Automate Cyrus administration:

1. Create Admin Script:

#!/usr/bin/env python3
import subprocess

def create_user(username, password):
    """Create a new Cyrus IMAP user"""
    # Create mailbox
    subprocess.run([
        'cyradm', '--user', 'cyrus', '--password', 'admin', 'localhost'
    ], input=f'cm user.{username}\nsq user.{username} 1048576\nquit\n'.encode())

    # Create SASL user
    subprocess.run([
        'saslpasswd2', '-p', '-c', username
    ], input=password.encode())

def delete_user(username):
    """Delete a Cyrus IMAP user"""
    subprocess.run([
        'cyradm', '--user', 'cyrus', '--password', 'admin', 'localhost'
    ], input=f'dm user.{username}\nquit\n'.encode())

    subprocess.run(['saslpasswd2', '-d', username])

# Example usage
create_user('john', 'secure-password')

2. REST API Wrapper: Create a REST API for user management using Flask or FastAPI

Additional Resources

• Official Cyrus IMAP Website
• Cyrus IMAP Installation Guide
• Cyrus IMAP Man Pages
• Cyrus SASL Documentation
• Klutch.sh Documentation
• Persistent Storage Guide
• TCP Traffic Configuration

Conclusion

Cyrus IMAP provides a robust, scalable solution for organizations requiring enterprise-grade email infrastructure. By deploying on Klutch.sh, you gain the benefits of containerization, automatic SSL, and flexible resource scaling without the complexity of managing bare-metal servers or traditional VPS configurations.

The unique architecture of Cyrus IMAP—with its database-backed storage, single-instance message store, and sophisticated ACL system—makes it particularly well-suited for large deployments where performance, reliability, and advanced features are critical. Features like murder topology, replication, Sieve filtering, and CalDAV support position Cyrus as a comprehensive messaging platform that goes beyond simple email storage.

Whether you’re migrating from another IMAP server, building a new email infrastructure, or scaling an existing deployment, Cyrus IMAP on Klutch.sh provides the foundation for a secure, high-performance mail system. Start with the basic configuration outlined in this guide, then expand functionality through replication, distributed architectures, and integration with your existing authentication infrastructure.

Your mail data remains under your control, security is built-in from the ground up, and the open-source nature of Cyrus IMAP ensures transparency, customizability, and freedom from vendor lock-in. Deploy Cyrus IMAP today and take control of your organization’s email infrastructure.