Deploying kiwix-serve

Introduction

kiwix-serve is a lightweight web server for serving ZIM files, the compressed archive format used to distribute Wikipedia, TED talks, Stack Overflow, and thousands of other websites for offline access. Part of the Kiwix project, it enables organizations and individuals to provide access to educational content without internet connectivity.

Originally designed for schools and libraries in areas with limited internet access, kiwix-serve has become essential for offline knowledge distribution. The ZIM format efficiently compresses entire websites while maintaining full functionality, including search, images, and interactive elements.

Key highlights of kiwix-serve:

ZIM File Support: Serve any ZIM-formatted content archive
Full-Text Search: Fast, indexed search across served content
Multiple Libraries: Serve multiple ZIM files simultaneously
Lightweight: Minimal resource requirements for embedded devices
Modern UI: Clean, responsive interface for content browsing
OPDS Catalog: Supports the Open Publication Distribution System
Multi-Language: Serve content in any available language
Bandwidth Efficient: Compressed content reduces data usage
Library Management: Manage content through the web interface
100% Open Source: GPL licensed with active development

This guide walks through deploying kiwix-serve on Klutch.sh using Docker, adding content archives, and configuring access for users.

Why Deploy kiwix-serve on Klutch.sh

Deploying kiwix-serve on Klutch.sh provides several advantages:

Always-On Access: Your knowledge library is available 24/7 from anywhere.

Simplified Deployment: Klutch.sh handles container deployment without manual setup.

Persistent Storage: ZIM files and configurations survive restarts.

HTTPS by Default: Secure access to educational content.

Scalable Resources: Handle multiple concurrent users with proper resource allocation.

GitHub Integration: Track configuration changes in version control.

Custom Domains: Host your knowledge library on your own domain.

Prerequisites

Before deploying kiwix-serve on Klutch.sh, ensure you have:

A Klutch.sh account
A GitHub account with a repository for your configuration
ZIM files to serve (downloadable from the Kiwix library)
Basic familiarity with Docker
Sufficient storage for your ZIM files (Wikipedia can be 80+ GB)

Understanding kiwix-serve Architecture

kiwix-serve is a simple, focused application:

ZIM Files: Compressed website archives containing HTML, images, and metadata.

libzim: The core library for reading and searching ZIM files.

HTTP Server: Built-in web server for serving content.

Search Index: Full-text search across all served content.

OPDS Support: Catalog format for library applications.

ZIM File Sources

Preparing Your Repository

Create a GitHub repository for your kiwix-serve deployment.

Repository Structure

kiwix-deploy/
├── Dockerfile
├── library.xml
├── README.md
└── .dockerignore

Creating the Dockerfile

Create a Dockerfile in the root of your repository:

FROM ghcr.io/kiwix/kiwix-serve:latest

# Set environment variables
ENV PORT=${PORT:-8080}

# Create data directory for ZIM files
RUN mkdir -p /data

# Expose the web interface
EXPOSE 8080

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \
  CMD wget --no-verbose --tries=1 --spider http://localhost:8080/ || exit 1

# Default command to serve all ZIM files in /data
CMD ["kiwix-serve", "--port", "8080", "--library", "/data/library.xml"]

Alternative Dockerfile for Specific ZIM Files

For explicit content configuration:

FROM ghcr.io/kiwix/kiwix-serve:latest

# Create data directory
RUN mkdir -p /data

# Port configuration
ENV PORT=8080

# Number of threads for handling requests
ENV THREADS=${THREADS:-4}

# Expose port
EXPOSE 8080

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \
  CMD wget --no-verbose --tries=1 --spider http://localhost:8080/ || exit 1

# Serve specific ZIM files or all in directory
CMD ["sh", "-c", "kiwix-serve --port 8080 --threads ${THREADS} /data/*.zim"]

Environment Variables Reference

Variable	Required	Default	Description
`PORT`	No	8080	HTTP server port
`THREADS`	No	4	Number of request handling threads

Deploying kiwix-serve on Klutch.sh

Download ZIM Files

Obtain ZIM files from the Kiwix Library. Popular options include:

Wikipedia (various sizes and languages)
Stack Overflow
TED Talks
Project Gutenberg books

Push Your Repository to GitHub

Initialize and push your repository:

git init
git add Dockerfile .dockerignore README.md
git commit -m "Initial kiwix-serve deployment configuration"
git remote add origin https://github.com/yourusername/kiwix-deploy.git
git push -u origin main

Create a New Project on Klutch.sh

Navigate to the Klutch.sh dashboard and create a new project named “kiwix” or “offline-library”.

Create a New App

Within your project, create a new app. Connect your GitHub account and select the repository containing your Dockerfile.

Configure HTTP Traffic

In the deployment settings:

Select HTTP as the traffic type
Set the internal port to 8080

Attach Persistent Volumes

Add storage for your ZIM files:

Mount Path	Recommended Size	Purpose
`/data`	100+ GB	ZIM file storage (size depends on content)

Deploy Your Application

Click Deploy to start the build process.

Upload ZIM Files

After deployment, upload your ZIM files to the /data volume. Methods include:

Direct volume upload through Klutch.sh
Companion file manager application
Pre-populated volume during setup

Create Library XML (Optional)

For more control, create a library.xml file describing your content.

Access kiwix-serve

Once deployed with ZIM files, access your library at https://your-app-name.klutch.sh.

Managing Content

Adding New ZIM Files

To add content to your library:

Download the ZIM file from the Kiwix library
Upload to the /data directory
Restart the container to detect new files

Generating Library XML

Create a library manifest for multiple ZIM files:

kiwix-manage library.xml add /data/wikipedia_en.zim
kiwix-manage library.xml add /data/stackoverflow.zim

Removing Content

Remove unwanted ZIM files:

Delete the file from /data
Update library.xml if used
Restart the container

Popular ZIM Collections

Educational Content

Content	Size	Description
Wikipedia English	~90 GB	Complete English Wikipedia
Wikipedia Simple	~1 GB	Simple English for learners
TED Talks	~20 GB	Educational video collection
Khan Academy	~40 GB	Educational courses

Technical Documentation

Content	Size	Description
Stack Overflow	~50 GB	Programming Q&A archive
MDN Web Docs	~5 GB	Web development documentation
ArchWiki	~500 MB	Linux documentation

Medical Reference

Content	Size	Description
WikiMed	~1 GB	Medical Wikipedia articles
First Aid	~100 MB	Emergency medical reference

Optimizing Performance

Resource Allocation

CPU: Minimal for serving, more for search operations
Memory: ~100MB base, plus ~10MB per GB of ZIM files indexed
Storage: Depends entirely on chosen content

Caching Strategies

Use a CDN for frequently accessed content
Enable browser caching with appropriate headers
Consider multiple replicas for high traffic

Troubleshooting

Content Not Appearing

Verify ZIM files are in the correct directory
Check that files have .zim extension
Review container logs for parsing errors

Search Not Working

Ensure ZIM files include search indexes
Check for sufficient memory allocation
Verify libzim can access the files

Slow Performance

Allocate more CPU resources
Increase thread count
Consider using faster storage

Additional Resources

Conclusion

Deploying kiwix-serve on Klutch.sh gives you a powerful platform for hosting offline educational content. Whether you’re providing Wikipedia access to remote schools, distributing technical documentation, or building an offline knowledge archive, kiwix-serve on Klutch.sh offers reliable, always-on access to essential information.

With support for hundreds of content sources and efficient ZIM compression, you can serve vast amounts of knowledge while maintaining reasonable storage requirements.