Deploying HyperKitty

Introduction

HyperKitty is the web-based archiver component of GNU Mailman 3, designed to provide beautiful, searchable archives for mailing lists. Built with Django, HyperKitty replaces the older Pipermail archiver with a modern, feature-rich web interface that makes browsing and searching mailing list archives intuitive and efficient.

As part of the Mailman 3 suite, HyperKitty integrates seamlessly with Mailman Core (the mailing list engine) and Postorius (the web-based administration interface). Together, these components provide a complete mailing list management solution that rivals commercial offerings while remaining fully open source.

Key highlights of HyperKitty:

Modern Web Interface: Clean, responsive design that works across desktop and mobile devices
Full-Text Search: Powerful search capabilities across all archived messages
Thread Visualization: View email threads in an organized, easy-to-follow format
User Authentication: Integration with Django’s authentication system and SSO providers
Gravatar Support: Display user avatars for a more personalized experience
Tag System: Organize and categorize discussions with tags
Export Capabilities: Export threads and messages in various formats
REST API: Programmatic access to archive data
Statistics and Analytics: View posting statistics and trends over time

This guide walks through deploying HyperKitty on Klutch.sh as part of the Mailman 3 web interface stack using Docker.

Why Deploy HyperKitty on Klutch.sh

Deploying HyperKitty on Klutch.sh provides several advantages for managing your mailing list archives:

Simplified Deployment: Klutch.sh automatically detects your Dockerfile and builds the Mailman web components without complex orchestration. Push to GitHub, and your archiver deploys automatically.

Persistent Storage: Attach persistent volumes for your archive database and static files. Your message archives survive container restarts and redeployments without data loss.

HTTPS by Default: Klutch.sh provides automatic SSL certificates, ensuring secure access to your mailing list archives without manual certificate management.

GitHub Integration: Connect your configuration repository directly from GitHub. Updates to your Dockerfile trigger automatic redeployments.

Scalable Resources: Allocate CPU and memory based on your archive size and expected traffic. Start small and scale up as your mailing lists grow.

Environment Variable Management: Securely store sensitive configuration like database credentials and API keys through Klutch.sh’s environment variable system.

Custom Domains: Assign a custom domain to your HyperKitty instance for a professional, branded mailing list experience.

Prerequisites

Before deploying HyperKitty on Klutch.sh, ensure you have:

A Klutch.sh account
A GitHub account with a repository for your HyperKitty configuration
Basic familiarity with Docker and containerization concepts
A Mailman Core instance running (can be deployed separately on Klutch.sh)
A PostgreSQL database (recommended for production)

Understanding HyperKitty Architecture

HyperKitty is part of the Mailman 3 web stack and works in conjunction with several components:

Django Framework: HyperKitty is built on Django, providing a robust foundation for the web application with built-in authentication, ORM, and admin capabilities.

Mailman Core Integration: HyperKitty receives archived messages from Mailman Core via its archiver API, storing them in its own database for fast retrieval and search.

Postorius Integration: While HyperKitty handles archiving, Postorius handles list administration. Both are typically deployed together in the mailman-web container.

Database Backend: HyperKitty stores archive data in a relational database (PostgreSQL recommended) separate from Mailman Core’s database.

Preparing Your Repository

To deploy HyperKitty on Klutch.sh, create a GitHub repository containing your Dockerfile and configuration.

Repository Structure

hyperkitty-deploy/
├── Dockerfile
├── settings_local.py
└── .dockerignore

Creating the Dockerfile

Create a Dockerfile in the root of your repository using the official Mailman web image:

FROM maxking/mailman-web:0.4

# Set environment variables
ENV SECRET_KEY=${SECRET_KEY}
ENV SERVE_FROM_DOMAIN=${SERVE_FROM_DOMAIN}
ENV HYPERKITTY_API_KEY=${HYPERKITTY_API_KEY}

# Database configuration
ENV DATABASE_URL=${DATABASE_URL}
ENV DATABASE_TYPE=${DATABASE_TYPE:-postgres}

# Mailman Core connection
ENV MAILMAN_REST_URL=${MAILMAN_REST_URL}
ENV MAILMAN_REST_USER=${MAILMAN_REST_USER}
ENV MAILMAN_REST_PASSWORD=${MAILMAN_REST_PASSWORD}

# Email configuration
ENV DEFAULT_FROM_EMAIL=${DEFAULT_FROM_EMAIL}
ENV EMAIL_HOST=${EMAIL_HOST}
ENV EMAIL_PORT=${EMAIL_PORT:-587}

# Create static files directory
RUN mkdir -p /opt/mailman-web-data/static

# Expose the web interface port
EXPOSE 8000

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
  CMD wget --no-verbose --tries=1 --spider http://localhost:8000/ || exit 1

Environment Variables Reference

HyperKitty requires several environment variables for proper operation:

Variable	Required	Default	Description
`SECRET_KEY`	Yes	-	Django’s secret key for signing cookies and tokens
`SERVE_FROM_DOMAIN`	Yes	-	The domain name from which Django will be served
`HYPERKITTY_API_KEY`	Yes	-	API key for Mailman Core to communicate with HyperKitty
`DATABASE_URL`	Yes	-	Full database connection URL
`DATABASE_TYPE`	No	postgres	Database type (postgres, mysql, sqlite)
`MAILMAN_REST_URL`	Yes	-	URL of the Mailman Core REST API
`MAILMAN_REST_USER`	Yes	-	Username for Mailman Core REST API
`MAILMAN_REST_PASSWORD`	Yes	-	Password for Mailman Core REST API
`MAILMAN_ADMIN_USER`	No	-	Default admin username to create
`MAILMAN_ADMIN_EMAIL`	No	-	Default admin email address

Deploying HyperKitty on Klutch.sh

Once your repository is prepared, follow these steps to deploy HyperKitty:

Generate Your Secret Keys

Before deployment, generate secure keys for Django and the HyperKitty API:

# Generate Django SECRET_KEY
openssl rand -base64 32

# Generate HYPERKITTY_API_KEY
openssl rand -hex 32

Save these keys securely for the environment variables configuration.

Push Your Repository to GitHub

Initialize your repository and push to GitHub:

git init
git add Dockerfile .dockerignore
git commit -m "Initial HyperKitty deployment configuration"
git remote add origin https://github.com/yourusername/hyperkitty-deploy.git
git push -u origin main

Create a New Project on Klutch.sh

Navigate to the Klutch.sh dashboard and create a new project. Give it a descriptive name like “mailman” or “mailing-lists”.

Create a New App

Within your project, create a new app. Connect your GitHub account if you haven’t already, then select the repository containing your HyperKitty Dockerfile.

Configure HTTP Traffic

HyperKitty serves its web interface over HTTP. In the deployment settings:

Select HTTP as the traffic type
Set the internal port to 8000

Set Environment Variables

In the environment variables section, add the following:

Variable	Value
`SECRET_KEY`	Your generated Django secret key
`SERVE_FROM_DOMAIN`	`your-app-name.klutch.sh`
`HYPERKITTY_API_KEY`	Your generated API key
`DATABASE_URL`	Your PostgreSQL connection URL
`MAILMAN_REST_URL`	URL to your Mailman Core instance
`MAILMAN_REST_USER`	Mailman Core REST API username
`MAILMAN_REST_PASSWORD`	Mailman Core REST API password

Attach Persistent Volumes

Add the following volumes for persistent storage:

Mount Path	Recommended Size	Purpose
`/opt/mailman-web-data`	10 GB	Web application data, static files, and uploads

Deploy Your Application

Click Deploy to start the build process. Klutch.sh will:

Detect your Dockerfile automatically
Build the container image
Attach the persistent volumes
Start the HyperKitty container
Provision an HTTPS certificate

Access HyperKitty

Once deployment completes, access your HyperKitty instance at https://your-app-name.klutch.sh. You can browse archives and search messages through the web interface.

Initial Configuration

Creating an Admin User

After deployment, create an administrator account:

Access your HyperKitty instance
If MAILMAN_ADMIN_USER and MAILMAN_ADMIN_EMAIL were set, an admin user is created automatically
Otherwise, use Django’s admin interface to create users

Connecting to Mailman Core

HyperKitty needs to be registered as an archiver in your Mailman Core configuration. In your Mailman Core’s mailman.cfg:

[archiver.hyperkitty]
class: mailman_hyperkitty.Archiver
enable: yes
configuration: /etc/mailman3/mailman-hyperkitty.cfg

Troubleshooting Common Issues

Archives Not Appearing

Symptoms: New messages are not showing up in the archives.

Solutions:

Verify the HYPERKITTY_API_KEY matches between HyperKitty and Mailman Core
Check that Mailman Core can reach HyperKitty’s archiver endpoint
Review logs for connection errors

Search Not Working

Symptoms: Full-text search returns no results.

Solutions:

Ensure the search index has been built
Check database connectivity
Verify sufficient disk space for the search index

Static Files Not Loading

Symptoms: CSS and JavaScript files return 404 errors.

Solutions:

Run Django’s collectstatic command
Verify the static files volume is mounted correctly
Check web server configuration

Additional Resources

Conclusion

Deploying HyperKitty on Klutch.sh gives you a modern, searchable mailing list archive system with automatic builds, persistent storage, and secure HTTPS access. Combined with Mailman Core and Postorius, you have a complete mailing list management solution that rivals commercial offerings.

Whether you’re running a small community mailing list or managing archives for a large organization, HyperKitty on Klutch.sh provides the foundation for professional, reliable mailing list archiving that you control.