Deploying Bitmagnet

Introduction

Bitmagnet is a self-hosted BitTorrent DHT indexer and search engine. It crawls the BitTorrent DHT network to discover and index torrents, providing a searchable database with a modern web interface and GraphQL API.

Built with Go for performance and efficiency, Bitmagnet can index millions of torrents while running on modest hardware. It’s designed for users who want a private, self-hosted torrent search experience.

Key highlights of Bitmagnet:

DHT Crawling: Automatically discovers torrents from the DHT network
Full-Text Search: Fast search across indexed torrents
Web Interface: Modern, responsive search UI
GraphQL API: Powerful query interface for integrations
Content Classification: Automatic categorization of content
Metadata Enrichment: Fetches additional metadata where available
Low Resource Usage: Efficient Go implementation
PostgreSQL Backend: Reliable database storage
Docker Ready: Easy containerized deployment
Real-Time Indexing: Continuous DHT discovery

This guide walks through deploying Bitmagnet on Klutch.sh using Docker.

Why Deploy Bitmagnet on Klutch.sh

Deploying Bitmagnet on Klutch.sh provides several advantages:

Private Search: Search torrents without relying on public trackers.

Persistent Index: Build your own searchable torrent database.

API Access: Integrate with other applications via GraphQL.

Always Running: 24/7 indexing builds a comprehensive database.

Scalable Resources: Allocate resources based on indexing needs.

Secure Access: HTTPS by default for your search interface.

Prerequisites

Before deploying Bitmagnet on Klutch.sh, ensure you have:

A Klutch.sh account
A GitHub account with a repository for your configuration
Basic understanding of Docker and containerization
PostgreSQL database (can be deployed on Klutch.sh)
Understanding of BitTorrent and DHT concepts

Understanding Bitmagnet Architecture

Bitmagnet consists of several components:

DHT Crawler: Connects to the BitTorrent DHT network and discovers info hashes.

Metadata Resolver: Fetches torrent metadata from peers.

Indexer: Processes and stores torrent information.

Web Server: Provides the search interface and API.

PostgreSQL: Stores the indexed data.

Preparing Your Repository

Create a GitHub repository for your Bitmagnet deployment.

Repository Structure

bitmagnet-deploy/
├── Dockerfile
├── config.yaml
└── .dockerignore

Creating the Dockerfile

Create a Dockerfile for Bitmagnet:

FROM ghcr.io/bitmagnet-io/bitmagnet:latest

# Environment variables
ENV POSTGRES_HOST=${POSTGRES_HOST}
ENV POSTGRES_PORT=${POSTGRES_PORT:-5432}
ENV POSTGRES_USER=${POSTGRES_USER}
ENV POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
ENV POSTGRES_DB=${POSTGRES_DB:-bitmagnet}

# DHT configuration
ENV DHT_ENABLED=true
ENV DHT_PORT=3334

# Web configuration
ENV WEB_PORT=3333

# Expose ports
EXPOSE 3333 3334/udp

# The base image includes the entrypoint

Configuration File

Create config.yaml for advanced settings:

postgres:
  host: ${POSTGRES_HOST}
  port: ${POSTGRES_PORT}
  user: ${POSTGRES_USER}
  password: ${POSTGRES_PASSWORD}
  database: ${POSTGRES_DB}

dht:
  enabled: true
  port: 3334
  # Crawling intensity (lower = more aggressive)
  crawl_rate_limit: 100

web:
  port: 3333
  # Enable GraphQL playground
  graphql_playground: true

indexer:
  # Enable content classification
  classify: true
  # Enable metadata enrichment
  enrich: true

Environment Variables Reference

Variable	Required	Default	Description
`POSTGRES_HOST`	Yes	-	PostgreSQL hostname
`POSTGRES_PORT`	No	5432	PostgreSQL port
`POSTGRES_USER`	Yes	-	PostgreSQL username
`POSTGRES_PASSWORD`	Yes	-	PostgreSQL password
`POSTGRES_DB`	No	bitmagnet	Database name
`DHT_ENABLED`	No	true	Enable DHT crawling
`WEB_PORT`	No	3333	Web interface port

Deploying Bitmagnet on Klutch.sh

Set Up PostgreSQL

Deploy a PostgreSQL database or use an existing one. Ensure it’s accessible from Klutch.sh.

Push Your Repository to GitHub

Initialize and push your configuration to GitHub.

Create a New Project on Klutch.sh

Navigate to the Klutch.sh dashboard and create a new project named “bitmagnet” or “torrent-search”.

Create a New App

Within your project, create a new app and connect your GitHub repository.

Configure HTTP Traffic

Set up HTTP for the web interface:

Select HTTP as the traffic type
Set the internal port to 3333

Note: DHT requires UDP on port 3334 for crawling.

Set Environment Variables

Configure the following:

Variable	Value
`POSTGRES_HOST`	Your PostgreSQL host
`POSTGRES_USER`	Database username
`POSTGRES_PASSWORD`	Database password
`POSTGRES_DB`	bitmagnet

Deploy Your Application

Click Deploy to build and start Bitmagnet.

Access the Web Interface

Once deployment completes, access Bitmagnet at https://your-app-name.klutch.sh.

Using Bitmagnet

Web Search

Search for torrents:

Navigate to the web interface
Enter search terms
Filter by category, size, or date
View torrent details

GraphQL API

Query the API for advanced searches:

query SearchTorrents {
  torrents(
    query: "linux"
    limit: 10
  ) {
    infoHash
    name
    size
    files {
      path
      size
    }
  }
}

Filtering Results

Use filters to refine searches:

Category: Movies, TV, Music, Software, etc.
Size Range: Minimum and maximum file sizes
Date Range: Filter by discovery date
File Count: Number of files in torrent

Monitoring Indexing

Statistics Dashboard

View indexing statistics:

Total torrents indexed
Indexing rate per hour
DHT connection status
Database size

Logs

Monitor crawler activity:

Check application logs in Klutch.sh dashboard
View DHT connection stats
Review metadata resolution success rate

Performance Tuning

Crawl Rate

Adjust crawling intensity:

Lower crawl_rate_limit for faster discovery
Higher values reduce resource usage
Balance based on your needs

Database Optimization

Optimize PostgreSQL:

Regular VACUUM operations
Appropriate indexes
Connection pooling

Resource Allocation

Scale resources based on index size:

More CPU for faster crawling
More RAM for search performance
More storage as index grows

Troubleshooting Common Issues

No Torrents Indexing

Solutions:

Verify DHT port is accessible (UDP 3334)
Check crawler is connected to DHT network
Review logs for connection errors

Search Not Returning Results

Solutions:

Wait for initial indexing (takes time)
Verify PostgreSQL connection
Check search syntax

High Database Growth

Solutions:

Implement retention policies
Add filters to limit what’s indexed
Increase storage allocation

Additional Resources

Conclusion

Deploying Bitmagnet on Klutch.sh gives you a private, self-hosted torrent search engine with DHT indexing capabilities. With its modern web interface, GraphQL API, and efficient Go implementation, Bitmagnet provides a powerful tool for discovering and searching BitTorrent content on your own infrastructure.