Skip to content
Klutch.sh Docs

Deploying a Bencher App

Bencher is a powerful, open-source continuous benchmarking platform designed to help development teams track and optimize application performance over time. Built for modern software development workflows, Bencher integrates seamlessly with your CI/CD pipelines to automatically run benchmarks, detect performance regressions, and maintain performance standards throughout your codebase. Whether you’re optimizing critical algorithms, monitoring web service latency, or tracking memory usage, Bencher provides the insights and tools to keep your application performant.

Why Bencher?

Bencher stands out in the performance monitoring landscape with its comprehensive features and developer-friendly approach:

  • Continuous Benchmarking: Automatically run and track benchmarks across every commit or pull request
  • Regression Detection: Intelligent detection of performance regressions with statistical analysis
  • Multi-Language Support: Works with Rust, Python, JavaScript, Java, C++, Go, and more
  • CI/CD Integration: Seamless integration with GitHub Actions, GitLab CI, and other CI platforms
  • Performance Tracking: Historical performance data and visualizations for trend analysis
  • Threshold Management: Set and enforce performance thresholds for critical metrics
  • Project Collaboration: Team-based access control and collaborative performance monitoring
  • Customizable Metrics: Track any benchmark metric that matters to your application
  • API-First Design: RESTful API for programmatic access and custom integrations
  • Web Dashboard: Beautiful, intuitive interface for exploring performance data
  • Open Source: Apache 2.0 licensed with active community support
  • Self-Hosted: Complete control over your performance data and infrastructure

Bencher is ideal for performance-conscious development teams, open-source projects, and organizations that need granular control over their benchmarking infrastructure. With persistent storage on Klutch.sh, your performance data is always safe, searchable, and accessible.

Prerequisites

Before deploying Bencher, ensure you have:

  • A Klutch.sh account
  • A GitHub repository with your Bencher deployment configuration
  • Basic familiarity with Docker and Git
  • (Optional) PostgreSQL database for production deployments

Deploying Bencher

  1. Create a New Project

    Log in to your Klutch.sh dashboard and create a new project for your Bencher deployment.

  2. Prepare Your Repository

    Create a GitHub repository with the following structure for your Bencher deployment:

    bencher-deploy/
    ├─ Dockerfile
    ├─ .env.example
    ├─ .gitignore
    └─ README.md

    Here’s a production-ready Dockerfile for Bencher:

    FROM rust:1.75-slim as builder
    

    WORKDIR /app
    

    # Install dependencies
    RUN apt-get update && apt-get install -y \
        git \
        curl \
        build-essential \
        && rm -rf /var/lib/apt/lists/*
    

    # Clone Bencher repository
    RUN git clone https://github.com/bencherdev/bencher.git .
    

    # Build Bencher
    RUN cargo build --release
    

    # Runtime stage
    FROM debian:bookworm-slim
    

    WORKDIR /app
    

    # Install runtime dependencies
    RUN apt-get update && apt-get install -y \
        ca-certificates \
        libssl3 \
        && rm -rf /var/lib/apt/lists/*
    

    # Copy binary from builder
    COPY --from=builder /app/target/release/bencher-cli /usr/local/bin/bencher
    COPY --from=builder /app/target/release/bencher-server /usr/local/bin/bencher-server
    

    # Create app directories
    RUN mkdir -p /app/data /app/logs
    

    # Expose port 61500 (default Bencher server port)
    EXPOSE 61500
    

    # Health check
    HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
      CMD curl -f http://localhost:61500/health || exit 1
    

    # Start Bencher server
    CMD ["bencher-server", "--listen", "0.0.0.0:61500"]

    Create a .env.example file for environment configuration:

    Terminal window
    # Bencher Configuration
    BENCHER_HOST=0.0.0.0
    BENCHER_PORT=61500
    BENCHER_LOG_LEVEL=info
    

    # Database Configuration
    DATABASE_URL=postgresql://bencher:bencher_password@localhost:5432/bencher
    DATABASE_MAX_CONNECTIONS=10
    

    # Security
    BENCHER_API_TOKEN_EXPIRATION=86400
    BENCHER_JWT_SECRET=your-secure-jwt-secret
    

    # Optional: GitHub Integration
    GITHUB_CLIENT_ID=your-github-client-id
    GITHUB_CLIENT_SECRET=your-github-client-secret

    Commit and push to your GitHub repository:

    Terminal window
    git init
    git add .
    git commit -m "Initial Bencher deployment"
    git remote add origin https://github.com/yourusername/bencher-deploy.git
    git push -u origin main

  3. Create a New App

    In the Klutch.sh dashboard:

    • Click “Create New App”
    • Select your GitHub repository containing the Dockerfile
    • Choose the branch (typically main or master)
    • Klutch.sh will automatically detect the Dockerfile in the root directory

  4. Configure Environment Variables

    Set up these environment variables in your Klutch.sh dashboard:

    VariableDescriptionExample
    BENCHER_HOSTServer host binding0.0.0.0
    BENCHER_PORTServer port61500
    BENCHER_LOG_LEVELLogging level (debug, info, warn, error)info
    DATABASE_URLPostgreSQL connection string (required for production)postgresql://user:pass@host:5432/bencher
    DATABASE_MAX_CONNECTIONSDatabase connection pool size10
    BENCHER_JWT_SECRETSecret key for JWT tokens (generate a secure random string)your-secure-jwt-secret
    BENCHER_API_TOKEN_EXPIRATIONAPI token expiration time in seconds86400

  5. Configure Persistent Storage

    Bencher requires persistent storage for benchmark data and databases. Add persistent volumes:

    Mount PathDescriptionRecommended Size
    /app/dataBenchmark data and artifacts50GB
    /app/logsApplication logs10GB

    In the Klutch.sh dashboard:

    • Navigate to your app settings
    • Go to the “Volumes” section
    • Click “Add Volume” for each mount path
    • Set mount paths and sizes as specified above

  6. Set Network Configuration

    Configure your app’s network settings:

    • Select traffic type: HTTP
    • Internal port: 61500 (the Bencher server port)
    • Klutch.sh will automatically handle HTTPS termination

  7. Deploy Your App

    • Review all settings
    • Click “Deploy”
    • Klutch.sh will build the Docker image and start your Bencher instance
    • Wait for the deployment to complete (typically 5-10 minutes for the initial build)

Initial Setup and Configuration

After deployment completes, access your Bencher instance:

Accessing the Bencher Dashboard

Navigate to your app’s URL: https://example-app.klutch.sh

You’ll see the Bencher dashboard where you can manage projects, view benchmarks, and configure integrations.

Creating Your First Project

  1. Log in to your Bencher instance
  2. Click “Create Project” or “New Project”
  3. Enter a project name (e.g., “My Application”)
  4. Add an optional description
  5. Configure project settings:
    • Public or private visibility
    • Repository URL (optional)
    • Performance thresholds
  6. Save the project

Setting Up Benchmark Tracking

After creating a project:

  1. Generate API Token

    • Go to project settings
    • Generate an API token for CI/CD integration
    • Save this token securely

  2. Configure Thresholds

    • Set performance thresholds for critical metrics
    • Define alert conditions and notification preferences
    • Configure regression detection sensitivity

  3. Add Team Members

    • Invite team members to your project
    • Set appropriate permission levels

Environment Variable Examples

Basic Bencher Configuration

Terminal window
BENCHER_HOST=0.0.0.0
BENCHER_PORT=61500
BENCHER_LOG_LEVEL=info
BENCHER_JWT_SECRET=your-secure-random-string-here
BENCHER_API_TOKEN_EXPIRATION=86400

PostgreSQL Configuration (Production)

For production deployments with PostgreSQL:

Terminal window
BENCHER_HOST=0.0.0.0
BENCHER_PORT=61500
BENCHER_LOG_LEVEL=info
DATABASE_URL=postgresql://bencher:secure_password@postgres-host:5432/bencher
DATABASE_MAX_CONNECTIONS=20
DATABASE_POOL_TIMEOUT=10
BENCHER_JWT_SECRET=your-secure-jwt-secret

Advanced Configuration

For fine-tuned control:

Terminal window
BENCHER_CORS_ALLOWED_ORIGINS=https://example.com
BENCHER_RATE_LIMIT_REQUESTS=1000
BENCHER_RATE_LIMIT_WINDOW=3600
BENCHER_BACKUP_ENABLED=true
BENCHER_BACKUP_INTERVAL=86400

Sample Code and Getting Started

Running Benchmarks with Bencher CLI

Terminal window
# List available projects
bencher project list


# Create a new benchmark result
bencher run --project my-app --branch main \
  --testbed localhost \
  --adapter rust_criterion \
  -- cargo bench


# Compare benchmark results
bencher run --project my-app --branch feature-x \
  --testbed localhost \
  --adapter rust_criterion \
  --threshold-check percentage:25% \
  -- cargo bench

Using the Bencher API

Terminal window
# Create a new project via API
curl -X POST https://example-app.klutch.sh/api/v0/projects \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Application",
    "description": "Performance tracking for my app",
    "visibility": "private"
  }'


# Get project details
curl https://example-app.klutch.sh/api/v0/projects/my-app \
  -H "Authorization: Bearer YOUR_API_TOKEN"


# Retrieve benchmark results
curl https://example-app.klutch.sh/api/v0/projects/my-app/results \
  -H "Authorization: Bearer YOUR_API_TOKEN"


# Create a benchmark report
curl -X POST https://example-app.klutch.sh/api/v0/projects/my-app/reports \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "branch": "main",
    "testbed": "production",
    "start_time": "2025-01-01T00:00:00Z",
    "end_time": "2025-01-31T23:59:59Z"
  }'

GitHub Actions Integration

Example workflow file for continuous benchmarking:

name: Bencher Benchmarks


on: [push, pull_request]


jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3


      - name: Install Rust
        uses: actions-rs/toolchain@v1
        with:
          toolchain: stable


      - name: Run Benchmarks
        run: |
          cargo bench --no-fail-fast


      - name: Upload to Bencher
        env:
          BENCHER_PROJECT: my-app
          BENCHER_API_TOKEN: ${{ secrets.BENCHER_API_TOKEN }}
          BENCHER_ADAPTER: rust_criterion
          BENCHER_HOST: https://example-app.klutch.sh
        run: |
          curl -X POST $BENCHER_HOST/api/v0/projects/$BENCHER_PROJECT/results \
            -H "Authorization: Bearer $BENCHER_API_TOKEN" \
            -H "Content-Type: application/json" \
            -d @benchmark-results.json

Python Benchmark Example

import requests
import json
from datetime import datetime


# Configuration
BENCHER_HOST = "https://example-app.klutch.sh"
PROJECT_ID = "my-app"
API_TOKEN = "your-api-token"


# Sample benchmark results
results = {
    "project": PROJECT_ID,
    "branch": "main",
    "testbed": "local",
    "adapter": "python_pytest",
    "results": [
        {
            "name": "test_algorithm_speed",
            "value": 42.5,
            "unit": "milliseconds"
        },
        {
            "name": "test_memory_usage",
            "value": 256.0,
            "unit": "megabytes"
        }
    ],
    "timestamp": datetime.utcnow().isoformat() + "Z"
}


# Upload results
headers = {
    "Authorization": f"Bearer {API_TOKEN}",
    "Content-Type": "application/json"
}


response = requests.post(
    f"{BENCHER_HOST}/api/v0/projects/{PROJECT_ID}/results",
    headers=headers,
    json=results
)


if response.status_code == 201:
    print("Benchmark results uploaded successfully")
else:
    print(f"Error uploading results: {response.status_code}")
    print(response.text)

Docker Compose for Local Development

For local testing before deploying to Klutch.sh:

version: '3.8'


services:
  bencher:
    build: .
    ports:
      - "61500:61500"
    environment:
      - BENCHER_HOST=0.0.0.0
      - BENCHER_PORT=61500
      - BENCHER_LOG_LEVEL=debug
      - DATABASE_URL=postgresql://bencher:bencher_password@postgres:5432/bencher
      - BENCHER_JWT_SECRET=dev-secret-key
    depends_on:
      postgres:
        condition: service_healthy
    volumes:
      - ./data:/app/data
      - ./logs:/app/logs
    restart: unless-stopped


  postgres:
    image: postgres:15-alpine
    environment:
      - POSTGRES_DB=bencher
      - POSTGRES_USER=bencher
      - POSTGRES_PASSWORD=bencher_password
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "bencher"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped


volumes:
  postgres_data:

Save as docker-compose.yml and run:

Terminal window
docker-compose up -d

Access Bencher at http://localhost:61500

Database Setup and Migration

Using PostgreSQL for Production

For production deployments with PostgreSQL:

  1. Deploy a PostgreSQL instance on Klutch.sh or use an external database
  2. Note the connection details (host, port, username, password, database name)
  3. Set the DATABASE_URL environment variable in Bencher
  4. Restart the application for changes to take effect

Database Initialization

On first deployment, Bencher automatically initializes the database schema:

Terminal window
# Manual database initialization (if needed)
bencher-server --database-url postgresql://user:pass@host:5432/bencher init

Backing Up Benchmark Data

Terminal window
# Backup PostgreSQL database
pg_dump -h postgres-host -U bencher -d bencher > bencher-backup.sql


# Restore from backup
psql -h postgres-host -U bencher -d bencher < bencher-backup.sql

Performance Tracking and Visualization

Creating Performance Reports

  1. Navigate to your project dashboard
  2. Click “Reports” or “Performance Reports”
  3. Select date range and metrics
  4. Configure visualization preferences:
    • Chart type (line, bar, scatter)
    • Aggregation method (average, median, P95, P99)
    • Comparison options
  5. Generate and export reports

Setting Up Alerts

Configure alerts for performance degradation:

  1. Go to project settings
  2. Set threshold values for each metric
  3. Configure notification channels:
    • Email notifications
    • Slack integration
    • Webhook endpoints
  4. Test alert triggers
Terminal window
# Get performance trend data for a metric
curl "https://example-app.klutch.sh/api/v0/projects/my-app/metrics/latency/trend" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -G \
  --data-urlencode "start_date=2025-01-01" \
  --data-urlencode "end_date=2025-01-31"

Scaling and Performance

Vertical Scaling

For large benchmark workloads:

  1. Navigate to your app settings in Klutch.sh
  2. Increase instance resources:
    • Minimum 2 CPU cores for production
    • Minimum 4GB RAM
  3. Increase persistent volume sizes as needed

Database Optimization

For high-volume benchmark data:

Terminal window
DATABASE_MAX_CONNECTIONS=50
DATABASE_POOL_TIMEOUT=30
DATABASE_CONNECTION_IDLE_TIMEOUT=300
DATABASE_STATEMENT_CACHE_SIZE=128

Caching Configuration

Enable caching for frequently accessed data:

Terminal window
BENCHER_CACHE_ENABLED=true
BENCHER_CACHE_TTL=3600
BENCHER_CACHE_MAX_SIZE=1000

Custom Domain Configuration

To use your own domain with Bencher:

  1. Navigate to your app’s “Domains” section in Klutch.sh
  2. Click “Add Custom Domain”
  3. Enter your domain (e.g., benchmarks.yourdomain.com)
  4. Configure your DNS provider with a CNAME record: 
    benchmarks.yourdomain.com → example-app.klutch.sh
  5. Klutch.sh automatically provisions SSL certificates

Security Best Practices

Authentication and Access Control

  • Use strong, unique BENCHER_JWT_SECRET for token generation
  • Regularly rotate API tokens
  • Implement role-based access control for team members
  • Enable two-factor authentication if available
  • Keep API tokens secure and never commit them to repositories

Network Security

  • Always use HTTPS (Klutch.sh provides this automatically)
  • Restrict database access to only your Bencher instance
  • Use strong database passwords for PostgreSQL
  • Monitor access logs for suspicious activity
  • Implement rate limiting to prevent abuse

Data Privacy

  • Your benchmark data stays on your server
  • No external API calls for data storage or analytics
  • Full control over data retention and deletion policies
  • Regular backups ensure data resilience
  • Encrypt sensitive data at rest and in transit

Troubleshooting

Common Issues and Solutions

Issue: Bencher server fails to start

Check the deployment logs in Klutch.sh dashboard. Common causes:

  • Missing or invalid BENCHER_JWT_SECRET environment variable
  • Database connection issues
  • Port 61500 already in use
  • Insufficient memory allocated

Issue: Benchmark results not uploading

Verify:

  • API token is valid and not expired
  • Project ID is correct
  • Benchmark data format is valid
  • Network connectivity to Bencher server

Issue: Database connection errors

Solutions:

  • Verify DATABASE_URL is correct and complete
  • Check that PostgreSQL instance is running and accessible
  • Ensure database credentials are accurate
  • Verify firewall rules allow connection to database

Issue: Slow benchmark result retrieval

Causes and solutions:

  • Increase instance resources (CPU, RAM)
  • Optimize database queries and indexes
  • Reduce historical data retention period
  • Enable caching for frequently accessed data

Issue: High memory usage

Solutions:

  • Increase instance RAM allocation
  • Reduce benchmark data retention period
  • Optimize PostgreSQL configuration
  • Monitor and limit concurrent API requests

Upgrading Bencher

To update Bencher to a newer version:

  1. Update your Dockerfile to use the latest Bencher commit: 
    RUN git clone https://github.com/bencherdev/bencher.git . && git pull origin main
  2. Commit and push to GitHub
  3. Klutch.sh will automatically rebuild with the latest version
  4. Test the updated version in a staging environment first

Advanced Configuration

GitHub OAuth Integration

Terminal window
GITHUB_OAUTH_ENABLED=true
GITHUB_CLIENT_ID=your-github-client-id
GITHUB_CLIENT_SECRET=your-github-client-secret
GITHUB_OAUTH_REDIRECT_URI=https://example-app.klutch.sh/auth/github/callback

Webhook Integrations

Configure webhooks for custom integrations:

Terminal window
BENCHER_WEBHOOK_ENABLED=true
BENCHER_WEBHOOK_URL=https://your-service.com/webhook
BENCHER_WEBHOOK_EVENTS=benchmark_complete,threshold_exceeded

Custom Metric Units

Define custom units for benchmark metrics:

Terminal window
BENCHER_CUSTOM_UNITS=requests_per_second,memory_percent,cache_hit_ratio

Additional Resources

Conclusion

Deploying Bencher on Klutch.sh provides you with a comprehensive, self-hosted continuous benchmarking platform that helps your team maintain and improve application performance. With persistent storage for benchmark data, intelligent regression detection, and beautiful performance visualizations, Bencher enables data-driven performance optimization. Klutch.sh’s managed infrastructure ensures your benchmarking system is always available, secure, and scalable.

Start monitoring your application’s performance today by deploying Bencher on Klutch.sh and gain the insights needed to optimize your codebase for peak performance.