Skip to content
Klutch.sh Docs

Deploying GitLab Runner

Introduction

GitLab Runner is a powerful open-source continuous integration and deployment agent that works seamlessly with GitLab CI/CD pipelines. It executes jobs defined in .gitlab-ci.yml files, enabling automated testing, building, and deployment workflows for your software projects.

GitLab Runner stands out for its:

  • Flexible Executor Support: Run jobs in Docker containers, shells, or virtual machines with multiple executor options
  • Scalable Architecture: Deploy multiple runners to handle concurrent jobs and distribute workload efficiently
  • Advanced Caching: Built-in cache mechanisms to speed up pipeline executions and reduce build times
  • Secure Token Authentication: Token-based registration ensures secure communication with GitLab instances
  • Real-Time Log Streaming: Live pipeline execution tracking with detailed logs and status updates
  • Tag-Based Job Routing: Use tags to route specific jobs to runners with required capabilities or resources
  • Concurrent Job Execution: Configure runners to execute multiple jobs simultaneously based on available resources
  • Self-Hosted Control: Complete ownership and control of your CI/CD infrastructure with privacy and compliance benefits

Deploying GitLab Runner on Klutch.sh provides you with a managed, scalable platform for running your CI/CD pipelines with support for Docker-based job execution, persistent storage for caches and artifacts, secure environment variables, and high availability. This comprehensive guide walks you through deploying GitLab Runner using a Dockerfile, including detailed installation steps, runner registration, persistent storage configuration, and production-ready best practices.

Prerequisites

Before you begin, ensure you have the following:

  • A Klutch.sh account
  • A GitHub account with a repository for your GitLab Runner project
  • A GitLab account (GitLab.com or self-hosted GitLab instance)
  • Docker installed locally for testing (optional but recommended)
  • GitLab Runner registration token from your GitLab project or instance
  • Basic understanding of Docker, CI/CD concepts, and GitLab CI/CD pipelines

Installation and Setup

    1. Create Your Project Directory

      First, create a new directory for your GitLab Runner deployment project:

      Terminal window
      mkdir gitlab-runner-klutch
      cd gitlab-runner-klutch
      git init

    2. Create the Dockerfile

      Create a Dockerfile in your project root directory. This will define your GitLab Runner container configuration. Klutch.sh automatically detects the Dockerfile in the root directory and uses it for deployment:

      FROM gitlab/gitlab-runner:latest
      

      # Set working directory
      WORKDIR /etc/gitlab-runner
      

      # Install additional dependencies for Docker-in-Docker support
      RUN apt-get update && apt-get install -y \
          curl \
          ca-certificates \
          && rm -rf /var/lib/apt/lists/*
      

      # Create necessary directories
      RUN mkdir -p /etc/gitlab-runner/config \
                   /etc/gitlab-runner/cache \
                   /etc/gitlab-runner/builds
      

      # Copy custom configuration if any
      COPY entrypoint.sh /entrypoint.sh
      RUN chmod +x /entrypoint.sh
      

      # Expose Prometheus metrics port (optional)
      EXPOSE 9252
      

      # Set default environment variables
      ENV RUNNER_EXECUTOR=docker
      ENV DOCKER_IMAGE=alpine:latest
      ENV RUNNER_REQUEST_CONCURRENCY=1
      

      # Health check
      HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
          CMD gitlab-runner verify --name runner-1 2>&1 | grep -q "is alive" || exit 1
      

      # Use custom entrypoint
      ENTRYPOINT ["/entrypoint.sh"]

      Note: This Dockerfile uses the official GitLab Runner image as a base, ensuring compatibility with the latest GitLab CI/CD features. The runner will be configured to use the Docker executor for running jobs in isolated containers.

    3. Create Custom Entrypoint Script

      Create an entrypoint.sh file to handle runner registration and startup:

      #!/bin/bash
      set -e
      

      # Function to register runner if not already registered
      register_runner() {
          if [ ! -f /etc/gitlab-runner/config/config.toml ]; then
              echo "Registering GitLab Runner..."
      

              gitlab-runner register \
                  --non-interactive \
                  --url "${GITLAB_URL}" \
                  --registration-token "${REGISTRATION_TOKEN}" \
                  --executor "${RUNNER_EXECUTOR}" \
                  --docker-image "${DOCKER_IMAGE}" \
                  --description "${RUNNER_DESCRIPTION:-gitlab-runner-klutch}" \
                  --tag-list "${RUNNER_TAGS:-docker,klutch}" \
                  --run-untagged="${RUN_UNTAGGED:-false}" \
                  --locked="${RUNNER_LOCKED:-false}" \
                  --access-level="${ACCESS_LEVEL:-not_protected}" \
                  --docker-privileged="${DOCKER_PRIVILEGED:-false}" \
                  --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
                  --docker-volumes "/cache" \
                  --config /etc/gitlab-runner/config/config.toml
      

              echo "GitLab Runner registered successfully!"
          else
              echo "Runner already registered, skipping registration..."
          fi
      }
      

      # Register runner if registration token is provided
      if [ -n "${REGISTRATION_TOKEN}" ]; then
          register_runner
      else
          echo "Warning: REGISTRATION_TOKEN not set, skipping automatic registration"
          echo "You can register manually using: gitlab-runner register"
      fi
      

      # Start GitLab Runner
      echo "Starting GitLab Runner..."
      exec gitlab-runner run \
          --config /etc/gitlab-runner/config/config.toml \
          --working-directory /etc/gitlab-runner/builds \
          --user gitlab-runner

      This script automatically registers the runner on first startup if a registration token is provided, and starts the runner service.

    4. Advanced Dockerfile with Docker-in-Docker Support

      For running Docker commands within your CI/CD jobs, use this advanced configuration:

      FROM gitlab/gitlab-runner:latest
      

      # Install Docker CLI and additional tools
      RUN apt-get update && apt-get install -y \
          apt-transport-https \
          ca-certificates \
          curl \
          gnupg \
          lsb-release \
          git \
          jq \
          && curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg \
          && echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null \
          && apt-get update \
          && apt-get install -y docker-ce-cli \
          && rm -rf /var/lib/apt/lists/*
      

      # Set working directory
      WORKDIR /etc/gitlab-runner
      

      # Create necessary directories with proper permissions
      RUN mkdir -p /etc/gitlab-runner/config \
                   /etc/gitlab-runner/cache \
                   /etc/gitlab-runner/builds \
                   /etc/gitlab-runner/certs \
          && chown -R gitlab-runner:gitlab-runner /etc/gitlab-runner
      

      # Copy entrypoint script
      COPY entrypoint.sh /entrypoint.sh
      RUN chmod +x /entrypoint.sh
      

      # Expose Prometheus metrics port
      EXPOSE 9252
      

      # Environment variables
      ENV RUNNER_EXECUTOR=docker
      ENV DOCKER_IMAGE=alpine:latest
      ENV RUNNER_REQUEST_CONCURRENCY=1
      ENV CI_SERVER_URL=""
      ENV REGISTRATION_TOKEN=""
      ENV RUNNER_NAME=gitlab-runner-klutch
      ENV RUNNER_TAGS=docker,klutch
      

      # Health check
      HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
          CMD gitlab-runner verify 2>&1 | grep -q "is alive" || exit 1
      

      # Use custom entrypoint
      ENTRYPOINT ["/entrypoint.sh"]

    5. Create Configuration Template

      Create a config.toml.template file for advanced runner configurations:

      concurrent = ${CONCURRENT_JOBS:-4}
      check_interval = 0
      log_level = "${LOG_LEVEL:-info}"
      

      [session_server]
        session_timeout = 1800
      

      [[runners]]
        name = "${RUNNER_NAME:-gitlab-runner-klutch}"
        url = "${GITLAB_URL}"
        token = "${RUNNER_TOKEN}"
        executor = "docker"
        limit = ${RUNNER_LIMIT:-0}
        request_concurrency = ${RUNNER_REQUEST_CONCURRENCY:-1}
      

        [runners.custom_build_dir]
          enabled = true
      

        [runners.cache]
          Type = "local"
          Path = "/etc/gitlab-runner/cache"
          Shared = true
      

          [runners.cache.local]
            MaxUploadedArchiveSize = 0
      

        [runners.docker]
          tls_verify = false
          image = "${DOCKER_IMAGE:-alpine:latest}"
          privileged = ${DOCKER_PRIVILEGED:-false}
          disable_entrypoint_overwrite = false
          oom_kill_disable = false
          disable_cache = false
          volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
          shm_size = 0
          pull_policy = ["if-not-present"]

    6. Create .gitignore File

      Create a .gitignore file to exclude sensitive and build files:

      # Runner configuration (contains tokens)
      config.toml
      

      # Cache and build directories
      cache/
      builds/
      

      # Environment variables
      .env
      .env.local
      

      # Logs
      *.log
      

      # IDE
      .vscode/
      .idea/

    7. Create README.md

      Create a README.md file with basic instructions:

      # GitLab Runner on Klutch.sh
      

      This repository contains the configuration for deploying GitLab Runner on Klutch.sh.
      

      ## Quick Start
      

      1. Set up environment variables in Klutch.sh dashboard
      2. Deploy the application
      3. Runner will automatically register and start processing jobs
      

      ## Required Environment Variables
      

      - `GITLAB_URL`: Your GitLab instance URL (e.g., https://gitlab.com)
      - `REGISTRATION_TOKEN`: Runner registration token from GitLab
      

      See the full documentation for more details.

    8. Initialize Git and Push to GitHub

      Initialize your repository and push it to GitHub:

      Terminal window
      git add .
      git commit -m "Initial GitLab Runner setup for Klutch.sh"
      git branch -M main
      git remote add origin https://github.com/yourusername/gitlab-runner-klutch.git
      git push -u origin main

Deploying on Klutch.sh

    1. Connect Your GitHub Repository

      • Navigate to klutch.sh/app
      • Click “New Project” or select an existing project
      • Click “New App” and select “Import from GitHub”
      • Authorize Klutch.sh to access your GitHub account if you haven’t already
      • Select your GitLab Runner repository from the list

    2. Configure Build Settings

      Since Klutch.sh automatically detects the Dockerfile in your repository root, the build configuration is handled automatically. The platform uses Nixpacks as the build system, but when a Dockerfile is present, it takes precedence.

    3. Configure the Internal Port

      GitLab Runner primarily communicates outbound to GitLab, but if you want to expose Prometheus metrics:

      • In the app configuration, set the Internal Port to 9252 (for metrics endpoint)
      • Select HTTP as the traffic type
      • This is optional; the runner will work without exposing this port

    4. Set Up Persistent Storage

      GitLab Runner requires persistent storage for configuration, cache, and build artifacts:

      In your app settings, add the following volume mounts:

      • Mount Path: /etc/gitlab-runner/config | Size: 1 GB (for runner configuration)
      • Mount Path: /etc/gitlab-runner/cache | Size: 20 GB (for job caches)
      • Mount Path: /etc/gitlab-runner/builds | Size: 10 GB (for job workspaces)

      Note: Adjust the cache and builds volume sizes based on your pipeline requirements. Projects with large artifacts or many concurrent jobs may require more storage.

    5. Obtain GitLab Runner Registration Token

      Before configuring environment variables, you need to obtain a runner registration token from GitLab:

      For Project-Specific Runner:

      1. Navigate to your GitLab project
      2. Go to Settings → CI/CD
      3. Expand the Runners section
      4. Find the Project runners section
      5. Click “New project runner”
      6. Configure runner settings and tags
      7. Copy the registration token displayed

      For Group Runner:

      1. Navigate to your GitLab group
      2. Go to Settings → CI/CD
      3. Expand the Runners section
      4. Click “New group runner”
      5. Configure runner settings and tags
      6. Copy the registration token displayed

      For Instance Runner (Self-Hosted GitLab):

      1. Navigate to Admin Area → CI/CD → Runners
      2. Click “New instance runner”
      3. Configure runner settings
      4. Copy the registration token displayed

    6. Configure Environment Variables

      Add the following environment variables in your Klutch.sh app settings. Mark sensitive values as secret:

      Required Configuration:

      GITLAB_URL=https://gitlab.com
      REGISTRATION_TOKEN=<mark-as-secret>

      Runner Configuration:

      RUNNER_DESCRIPTION=gitlab-runner-klutch
      RUNNER_TAGS=docker,klutch,linux
      RUNNER_EXECUTOR=docker
      DOCKER_IMAGE=alpine:latest
      RUN_UNTAGGED=false
      RUNNER_LOCKED=false
      ACCESS_LEVEL=not_protected

      Concurrency Settings:

      CONCURRENT_JOBS=4
      RUNNER_REQUEST_CONCURRENCY=1
      RUNNER_LIMIT=0

      Docker Executor Settings:

      DOCKER_PRIVILEGED=false
      DOCKER_PULL_POLICY=if-not-present

      Optional - Advanced Settings:

      LOG_LEVEL=info
      METRICS_SERVER=:9252
      SENTRY_DSN=<mark-as-secret>

      Note: If you’re using self-hosted GitLab, replace https://gitlab.com with your GitLab instance URL. The REGISTRATION_TOKEN is sensitive and should always be marked as secret.

    7. Deploy the Application

      • Review all configurations
      • Click “Deploy” to start the deployment
      • Klutch.sh will build the Docker image from your Dockerfile and deploy it
      • Monitor the build logs to ensure everything completes successfully

    8. Verify the Deployment

      Once the deployment is complete:

      • Check the application logs in Klutch.sh dashboard for successful registration message
      • Navigate to your GitLab project’s Settings → CI/CD → Runners
      • Verify that your runner appears in the list with a green status indicator
      • The runner should show as “online” and ready to accept jobs
      • Trigger a test pipeline to verify the runner picks up and executes jobs

Runner Registration Process

GitLab Runner must be registered with your GitLab instance before it can execute jobs. The registration process associates the runner with your GitLab project, group, or instance.

Automatic Registration

The provided entrypoint.sh script automatically registers the runner on first startup when the following environment variables are set:

  • GITLAB_URL: Your GitLab instance URL
  • REGISTRATION_TOKEN: The registration token from GitLab

The registration includes:

  • Runner Description: Human-readable name for the runner
  • Tags: Labels for routing specific jobs to this runner
  • Executor Type: How jobs are executed (Docker, shell, etc.)
  • Docker Configuration: Default Docker image and settings
  • Concurrent Jobs: Number of jobs that can run simultaneously

Manual Registration

If you need to register the runner manually or update registration settings:

  1. Access the container shell through Klutch.sh dashboard

  2. Run the registration command:

    Terminal window
    gitlab-runner register \
        --non-interactive \
        --url "https://gitlab.com" \
        --registration-token "YOUR_TOKEN" \
        --executor "docker" \
        --docker-image "alpine:latest" \
        --description "gitlab-runner-klutch" \
        --tag-list "docker,klutch" \
        --run-untagged="false" \
        --locked="false"

  3. The runner configuration will be saved to /etc/gitlab-runner/config/config.toml

Verifying Runner Registration

Check runner status:

Terminal window
gitlab-runner verify

List registered runners:

Terminal window
gitlab-runner list

Configuring GitLab CI/CD Pipelines

To use your deployed GitLab Runner, you need to configure your .gitlab-ci.yml file to use the runner’s tags.

Basic Pipeline Example

Create a .gitlab-ci.yml file in your GitLab project:

# Define stages
stages:
  - build
  - test
  - deploy


# Build stage
build-job:
  stage: build
  tags:
    - docker
    - klutch
  script:
    - echo "Building the application..."
    - npm install
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 hour


# Test stage
test-job:
  stage: test
  tags:
    - docker
    - klutch
  script:
    - echo "Running tests..."
    - npm test
  dependencies:
    - build-job


# Deploy stage
deploy-job:
  stage: deploy
  tags:
    - docker
    - klutch
  script:
    - echo "Deploying application..."
    - ./deploy.sh
  only:
    - main
  dependencies:
    - build-job

Advanced Pipeline with Docker

For pipelines that need to build Docker images:

stages:
  - build
  - test
  - publish


variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""


build-docker-image:
  stage: build
  tags:
    - docker
    - klutch
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA


run-tests:
  stage: test
  tags:
    - docker
    - klutch
  image: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  script:
    - npm test
    - npm run test:integration


publish-release:
  stage: publish
  tags:
    - docker
    - klutch
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:latest
  only:
    - main

Using Caching

Leverage GitLab Runner’s cache to speed up builds:

cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - node_modules/
    - .npm/


build:
  tags:
    - docker
    - klutch
  script:
    - npm ci --cache .npm --prefer-offline
    - npm run build
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/
      - .npm/
    policy: pull-push

Environment Variables Reference

Required Environment Variables

VariableDescriptionExample
GITLAB_URLGitLab instance URLhttps://gitlab.com
REGISTRATION_TOKENRunner registration token from GitLabGR1348941...

Runner Configuration Variables

VariableDescriptionDefault
RUNNER_DESCRIPTIONHuman-readable runner namegitlab-runner-klutch
RUNNER_TAGSComma-separated list of tagsdocker,klutch
RUNNER_EXECUTORExecutor type (docker, shell, etc.)docker
DOCKER_IMAGEDefault Docker image for jobsalpine:latest
RUN_UNTAGGEDRun jobs without tagsfalse
RUNNER_LOCKEDLock runner to current projectfalse
ACCESS_LEVELRunner access levelnot_protected

Concurrency and Performance

VariableDescriptionDefault
CONCURRENT_JOBSMaximum concurrent jobs4
RUNNER_REQUEST_CONCURRENCYConcurrent job requests1
RUNNER_LIMITMaximum jobs to run (0 = unlimited)0

Docker Executor Settings

VariableDescriptionDefault
DOCKER_PRIVILEGEDEnable privileged modefalse
DOCKER_PULL_POLICYWhen to pull Docker imagesif-not-present
DOCKER_VOLUMESAdditional volumes to mount/var/run/docker.sock:/var/run/docker.sock

Optional Settings

VariableDescriptionDefault
LOG_LEVELLogging level (debug, info, warn, error)info
METRICS_SERVERPrometheus metrics endpoint:9252
SENTRY_DSNSentry error tracking DSN(empty)

Nixpacks Environment Variables

Since Klutch.sh automatically detects and uses the Dockerfile, these Nixpacks variables are only relevant if you’re deploying without a Dockerfile:

VariableDescriptionExample
NIXPACKS_BUILD_CMDCustom build commandecho "Build complete"
NIXPACKS_START_CMDCustom start command/entrypoint.sh

Note: When a Dockerfile is present, Nixpacks variables are not used as Docker takes precedence.

Persistent Storage Best Practices

Storage Layout

Organize your persistent storage for optimal performance and management:

/etc/gitlab-runner/
├── config/
│   └── config.toml       # Runner configuration (contains tokens)
├── cache/                # Job caches for faster builds
│   ├── runner-1/
│   └── runner-2/
├── builds/               # Job workspaces
│   ├── project-1/
│   └── project-2/
└── certs/                # TLS certificates (if needed)

Volume Size Recommendations

Based on project scale and pipeline complexity:

Small Projects (1-5 pipelines/day):

  • Config: 1 GB
  • Cache: 5 GB
  • Builds: 5 GB

Medium Projects (5-20 pipelines/day):

  • Config: 1 GB
  • Cache: 20 GB
  • Builds: 10 GB

Large Projects (20+ pipelines/day):

  • Config: 2 GB
  • Cache: 50 GB+
  • Builds: 20 GB

Cache Management

Configure cache cleanup to manage storage:

[[runners]]
  [runners.cache]
    Type = "local"
    Path = "/etc/gitlab-runner/cache"
    Shared = true
    MaxUploadedArchiveSize = 1073741824  # 1GB


  [runners.cache.local]
    MaxUploadedArchiveSize = 0

Implement periodic cache cleanup:

# Cleanup script
#!/bin/bash
CACHE_DIR=/etc/gitlab-runner/cache
RETENTION_DAYS=7


# Remove caches older than retention period
find $CACHE_DIR -type f -mtime +$RETENTION_DAYS -delete
find $CACHE_DIR -type d -empty -delete

Production Best Practices

Security

  1. Secure Registration Tokens: Always mark REGISTRATION_TOKEN as secret in Klutch.sh

  2. Principle of Least Privilege: Use project-specific runners instead of instance-wide runners when possible

  3. Disable Privileged Mode: Only enable DOCKER_PRIVILEGED=true if absolutely necessary for your pipelines

  4. Regular Updates: Keep GitLab Runner updated to the latest version

    FROM gitlab/gitlab-runner:latest

  5. Rotate Tokens: Periodically rotate runner registration tokens

  6. Network Security: Use private networks for communication between runners and GitLab

High Availability

  1. Multiple Runners: Deploy multiple runner instances for redundancy

  2. Health Monitoring: Use the built-in health check endpoint

    Terminal window
    curl http://example-app.klutch.sh:9252/metrics

  3. Automatic Restart: Klutch.sh automatically restarts failed containers

  4. Load Distribution: Use runner tags to distribute jobs across multiple runners

Performance Optimization

  1. Concurrent Jobs: Configure based on available resources

    CONCURRENT_JOBS=4

  2. Cache Dependencies: Use GitLab CI/CD cache to speed up builds

  3. Docker Image Caching: Set pull policy to if-not-present to avoid unnecessary pulls

  4. Resource Limits: Configure appropriate CPU and memory limits in Klutch.sh

  5. Local Docker Registry: Consider using a registry cache to speed up image pulls

Monitoring and Alerts

Monitor the following metrics:

  • Runner availability and uptime
  • Job queue length and wait times
  • Job success/failure rates
  • Cache hit rates
  • Storage usage trends
  • Container resource utilization

Access Prometheus metrics:

Terminal window
curl http://example-app.klutch.sh:9252/metrics

Key metrics to track:

  • gitlab_runner_jobs: Current jobs running
  • gitlab_runner_job_duration_seconds: Job execution times
  • gitlab_runner_errors_total: Error counts
  • gitlab_runner_concurrent_requests: Concurrent job requests

Troubleshooting

Common Issues

Issue: Runner fails to register

  • Solution: Verify GITLAB_URL and REGISTRATION_TOKEN are correct
  • Debug: Check application logs for registration errors
  • Verify: Ensure network connectivity to GitLab instance

Issue: Jobs not being picked up by runner

  • Solution: Check that job tags in .gitlab-ci.yml match runner tags
  • Debug: Verify runner is online in GitLab UI
  • Check: Ensure RUN_UNTAGGED is set to true if jobs have no tags

Issue: Docker-in-Docker not working

  • Solution: Verify Docker socket is mounted correctly in volumes
  • Alternative: Enable privileged mode with DOCKER_PRIVILEGED=true
  • Check: Ensure Docker service is available in job containers

Issue: Out of storage space

  • Solution: Implement cache cleanup or increase volume sizes
  • Debug: Check storage usage: df -h /etc/gitlab-runner
  • Clean: Manually remove old caches and builds

Issue: Jobs timeout or hang

  • Solution: Increase job timeout in .gitlab-ci.yml
  • Check: Review job logs for specific errors
  • Verify: Ensure sufficient resources (CPU/memory) are allocated

Issue: TLS certificate verification failed

  • Solution: Add custom certificates to /etc/gitlab-runner/certs/
  • Alternative: Set tls_verify = false in config.toml (not recommended for production)
  • Debug: Check GitLab instance SSL certificate validity

Debug Mode

Enable debug logging for troubleshooting:

LOG_LEVEL=debug

Verifying Runner Status

Check runner status from container:

Terminal window
# List registered runners
gitlab-runner list


# Verify runner connectivity
gitlab-runner verify


# Check runner status
gitlab-runner status

Accessing Runner Configuration

View runner configuration:

Terminal window
cat /etc/gitlab-runner/config/config.toml

Testing Runner Manually

Trigger a test job:

.gitlab-ci.yml
test-runner:
  tags:
    - docker
    - klutch
  script:
    - echo "Testing GitLab Runner on Klutch.sh"
    - gitlab-runner --version
    - docker --version

Advanced Configuration

Custom Docker Images

Configure runner to use custom Docker images with pre-installed dependencies:

[[runners]]
  [runners.docker]
    image = "your-registry.com/custom-image:latest"
    pull_policy = ["if-not-present"]

Docker Socket Binding

For jobs that need Docker access:

[[runners]]
  [runners.docker]
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]

Security Note: Mounting Docker socket gives jobs full Docker daemon access. Use with caution.

Resource Limits

Configure resource limits for job containers:

[[runners]]
  [runners.docker]
    memory = "2g"
    memory_swap = "2g"
    memory_reservation = "1g"
    cpus = "2"

Network Configuration

Configure custom networks for job isolation:

[[runners]]
  [runners.docker]
    network_mode = "bridge"
    dns = ["8.8.8.8", "8.8.4.4"]

Service Containers

Enable service containers for integration tests:

.gitlab-ci.yml
integration-test:
  tags:
    - docker
    - klutch
  services:
    - postgres:13
    - redis:6
  variables:
    POSTGRES_DB: testdb
    POSTGRES_USER: testuser
    POSTGRES_PASSWORD: testpass
  script:
    - npm run test:integration

Custom Executor Scripts

For advanced use cases, create custom executor scripts:

[[runners]]
  name = "custom-shell-runner"
  executor = "shell"
  shell = "bash"
  builds_dir = "/etc/gitlab-runner/builds"
  cache_dir = "/etc/gitlab-runner/cache"

Scaling GitLab Runners

Horizontal Scaling

Deploy multiple runner instances for increased capacity:

  1. Create Multiple Apps: Deploy separate GitLab Runner apps on Klutch.sh
  2. Use Shared Tags: Configure all runners with the same tags for load distribution
  3. Dedicated Runners: Assign specific runners to specific projects or job types

Autoscaling Considerations

While Klutch.sh doesn’t directly support runner autoscaling, you can:

  1. Deploy Multiple Static Runners: Create several runner instances for peak capacity
  2. Use Runner Limits: Configure RUNNER_LIMIT to control maximum jobs per runner
  3. Tag-Based Routing: Route resource-intensive jobs to dedicated runners

Runner Fleet Management

Manage multiple runners effectively:

# Runner naming convention
gitlab-runner-klutch-1  # General purpose
gitlab-runner-klutch-2  # General purpose
gitlab-runner-klutch-docker  # Docker builds
gitlab-runner-klutch-deploy  # Deployments only

Configure different concurrency levels:

  • General purpose: CONCURRENT_JOBS=4
  • Docker builds: CONCURRENT_JOBS=2 (resource intensive)
  • Deployments: CONCURRENT_JOBS=1 (serial execution)

Docker Compose for Local Development

For local testing before deploying to Klutch.sh, use this Docker Compose configuration:

version: '3.8'


services:
  gitlab-runner:
    build: .
    environment:
      - GITLAB_URL=https://gitlab.com
      - REGISTRATION_TOKEN=your-token-here
      - RUNNER_DESCRIPTION=gitlab-runner-local
      - RUNNER_TAGS=docker,local
      - RUNNER_EXECUTOR=docker
      - DOCKER_IMAGE=alpine:latest
      - CONCURRENT_JOBS=2
      - LOG_LEVEL=debug
    volumes:
      - ./config:/etc/gitlab-runner/config
      - ./cache:/etc/gitlab-runner/cache
      - ./builds:/etc/gitlab-runner/builds
      - /var/run/docker.sock:/var/run/docker.sock
    privileged: false
    restart: unless-stopped


volumes:
  runner-config:
  runner-cache:
  runner-builds:

Run locally with:

Terminal window
docker-compose up -d

Note: Docker Compose is only for local development. Klutch.sh does not support Docker Compose for deployments.

Migration from Other Platforms

From Jenkins

If you’re migrating from Jenkins:

  1. Convert Jenkinsfile stages to GitLab CI stages
  2. Replace Jenkins agents with GitLab Runner tags
  3. Migrate Jenkins credentials to GitLab CI/CD variables
  4. Update deployment scripts to use GitLab CI/CD syntax

From CircleCI

Converting CircleCI workflows:

  1. Map CircleCI orbs to GitLab CI templates or custom scripts
  2. Convert .circleci/config.yml to .gitlab-ci.yml
  3. Replace CircleCI contexts with GitLab groups and variables
  4. Update artifact storage and caching configurations

From Travis CI

Migrating from Travis CI:

  1. Convert .travis.yml to .gitlab-ci.yml format
  2. Replace Travis CI environment variables with GitLab CI/CD variables
  3. Update deployment provider configurations
  4. Migrate encrypted secrets to GitLab protected variables

Security Best Practices

Token Management

  1. Never Commit Tokens: Keep registration tokens out of version control
  2. Use Protected Variables: Mark sensitive variables as protected in GitLab
  3. Rotate Regularly: Change registration tokens periodically
  4. Scope Appropriately: Use project runners instead of shared runners for sensitive projects

Job Security

  1. Validate Input: Always validate external input in job scripts
  2. Use Protected Branches: Require protected branches for deployment jobs
  3. Limit Runner Access: Use tags to control which jobs run on which runners
  4. Audit Pipeline Logs: Regularly review pipeline logs for suspicious activity

Network Security

  1. Private Networking: Use private networks between runners and services
  2. Firewall Rules: Restrict runner network access to required services only
  3. TLS Everywhere: Use HTTPS for all GitLab communications
  4. VPN/Bastion: Consider VPN or bastion hosts for production runners

Container Security

  1. Non-Root User: Run jobs as non-root user when possible
  2. Read-Only Filesystems: Use read-only root filesystem where applicable
  3. Security Scanning: Integrate container security scanning in pipelines
  4. Minimal Images: Use minimal base images (Alpine, distroless)

Backup and Monitoring

Configuration Backup

Regularly backup runner configuration:

# Backup script
#!/bin/bash
BACKUP_DIR=/backups
DATE=$(date +%Y%m%d_%H%M%S)
CONFIG_FILE=/etc/gitlab-runner/config/config.toml


# Create backup
cp $CONFIG_FILE "${BACKUP_DIR}/config.toml.${DATE}"


# Keep only last 7 backups
ls -t ${BACKUP_DIR}/config.toml.* | tail -n +8 | xargs rm -f

Cache and Build Backup

Backup important caches and artifacts:

Terminal window
# Backup caches
tar -czf cache_backup_$(date +%Y%m%d).tar.gz /etc/gitlab-runner/cache


# Backup critical builds
tar -czf builds_backup_$(date +%Y%m%d).tar.gz /etc/gitlab-runner/builds

Monitoring Setup

Monitor runner health:

  1. Metrics Endpoint: Expose port 9252 for Prometheus metrics
  2. Log Aggregation: Send logs to external logging service
  3. Alerting: Set up alerts for runner downtime or job failures
  4. Dashboard: Create Grafana dashboard for runner metrics

Health Checks

Implement comprehensive health checks:

health-check.sh
#!/bin/bash
# Check runner service
if ! gitlab-runner verify &>/dev/null; then
    echo "Runner verification failed"
    exit 1
fi


# Check storage
CACHE_USAGE=$(df -h /etc/gitlab-runner/cache | awk 'NR==2 {print $5}' | sed 's/%//')
if [ $CACHE_USAGE -gt 90 ]; then
    echo "Cache storage critical: ${CACHE_USAGE}%"
    exit 1
fi


echo "Health check passed"
exit 0

Resources