Deploying GoSE

Introduction

GoSE (Go S3 Uploader) is a modern file uploader designed for scalability and simplicity. By leveraging S3-compatible storage backends, GoSE enables you to share files of virtually any size with features like chunk-based uploads, automatic deduplication, and seamless resumption of interrupted transfers.

Unlike traditional file sharing solutions that require complex databases and caching layers, GoSE only depends on an S3 storage backend, making it horizontally scalable and easy to deploy. The application bundles both frontend and backend components in a single binary or Docker image.

Key highlights of GoSE:

• Tera-Scale Uploads: Handle files of any size with chunk-based uploads
• Automatic Deduplication: Skip chunks that already exist, saving storage and bandwidth
• Resumable Uploads: Seamlessly resume interrupted uploads without data loss
• Direct S3 Transfers: Upload and download data goes directly to/from S3, not through GoSE
• S3 Compatible: Works with AWS S3, MinIO, Ceph RadosGW, and other S3-compatible storage
• Browser-Based Hashing: MD5 digest computed in browser for efficient deduplication
• Single Binary: No database or cache required - just S3
• Clean Web Interface: Simple drag-and-drop file sharing
• 100% Open Source: Licensed under Apache 2.0

This guide walks through deploying GoSE on Klutch.sh using Docker, configuring S3 storage, and setting up file sharing for your team or organization.

Why Deploy GoSE on Klutch.sh

Deploying GoSE on Klutch.sh provides several advantages for file sharing:

Simplified Deployment: Klutch.sh automatically detects your Dockerfile and builds GoSE without complex orchestration. Push to GitHub and your file uploader deploys automatically.

Minimal Infrastructure: GoSE only needs S3 storage - no database or cache required, keeping your deployment simple and maintainable.

HTTPS by Default: Klutch.sh provides automatic SSL certificates, ensuring secure uploads and downloads.

GitHub Integration: Connect your configuration repository directly from GitHub. Updates trigger automatic redeployments.

Scalable Resources: Allocate CPU and memory based on expected concurrent users.

Custom Domains: Assign a custom domain for branded file sharing.

Always-On Availability: Your file sharing platform remains accessible 24/7.

Prerequisites

Before deploying GoSE on Klutch.sh, ensure you have:

• A Klutch.sh account
• A GitHub account with a repository for your GoSE configuration
• An S3-compatible storage account (AWS S3, MinIO, Backblaze B2, etc.)
• S3 bucket created and credentials ready
• Basic familiarity with Docker and containerization concepts

Understanding GoSE Architecture

GoSE is designed for simplicity and scalability:

Single Binary: The application bundles frontend and backend in one executable, simplifying deployment.

S3-Only Storage: All files are stored directly in S3, eliminating the need for databases or local storage.

Chunk-Based Uploads: Files are split into chunks that are hashed and uploaded independently.

Browser-Side Hashing: MD5 digests are computed in the browser, enabling efficient deduplication before upload.

Direct S3 Access: Bulk data transfers happen directly between browser and S3, reducing load on GoSE.

Preparing Your Repository

To deploy GoSE on Klutch.sh, create a GitHub repository containing your Dockerfile and configuration.

Repository Structure

gose-deploy/
├── Dockerfile
├── README.md
└── .dockerignore

Creating the Dockerfile

Create a Dockerfile in the root of your repository:

FROM ghcr.io/stv0g/gose:latest

# Set environment variables for S3 connection
ENV GOSE_S3_ENDPOINT=${GOSE_S3_ENDPOINT}
ENV GOSE_S3_REGION=${GOSE_S3_REGION:-us-east-1}
ENV GOSE_S3_BUCKET=${GOSE_S3_BUCKET}
ENV GOSE_S3_ACCESS_KEY=${GOSE_S3_ACCESS_KEY}
ENV GOSE_S3_SECRET_KEY=${GOSE_S3_SECRET_KEY}

# Optional: Set base URL for share links
ENV GOSE_BASE_URL=${GOSE_BASE_URL}

# Expose the web interface port
EXPOSE 8080

# The base image includes the default entrypoint

Advanced Dockerfile with Custom Configuration

For more control over your deployment:

FROM ghcr.io/stv0g/gose:latest

# S3 Configuration
ENV GOSE_S3_ENDPOINT=${GOSE_S3_ENDPOINT}
ENV GOSE_S3_REGION=${GOSE_S3_REGION:-us-east-1}
ENV GOSE_S3_BUCKET=${GOSE_S3_BUCKET}
ENV GOSE_S3_ACCESS_KEY=${GOSE_S3_ACCESS_KEY}
ENV GOSE_S3_SECRET_KEY=${GOSE_S3_SECRET_KEY}
ENV GOSE_S3_PATH_STYLE=${GOSE_S3_PATH_STYLE:-false}

# Application Configuration
ENV GOSE_BASE_URL=${GOSE_BASE_URL}
ENV GOSE_LISTEN=${GOSE_LISTEN:-:8080}

# Upload Limits
ENV GOSE_MAX_UPLOAD_SIZE=${GOSE_MAX_UPLOAD_SIZE:-0}
ENV GOSE_CHUNK_SIZE=${GOSE_CHUNK_SIZE:-5242880}

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=10s --retries=3 \
  CMD wget --no-verbose --tries=1 --spider http://localhost:8080/ || exit 1

# Expose the web interface port
EXPOSE 8080

Creating the .dockerignore File

Create a .dockerignore file:

.git
.github
*.md
README.md
LICENSE
.gitignore
*.log
.DS_Store
.env
.env.local

Environment Variables Reference

Variable	Required	Default	Description
GOSE_S3_ENDPOINT	Yes	-	S3 endpoint URL (e.g., s3.amazonaws.com or MinIO URL)
GOSE_S3_REGION	No	us-east-1	S3 region
GOSE_S3_BUCKET	Yes	-	S3 bucket name for file storage
GOSE_S3_ACCESS_KEY	Yes	-	S3 access key ID
GOSE_S3_SECRET_KEY	Yes	-	S3 secret access key
GOSE_S3_PATH_STYLE	No	false	Use path-style URLs (required for MinIO)
GOSE_BASE_URL	No	-	Base URL for generated share links
GOSE_LISTEN	No	:8080	Address and port to listen on
GOSE_MAX_UPLOAD_SIZE	No	0	Maximum upload size in bytes (0 = unlimited)
GOSE_CHUNK_SIZE	No	5242880	Chunk size in bytes (default 5MB)

Deploying GoSE on Klutch.sh

Once your repository is prepared, follow these steps to deploy GoSE:

Set Up S3 Storage

Before deployment, configure your S3-compatible storage:

For AWS S3:

1. Create an S3 bucket
2. Create an IAM user with S3 access
3. Note the access key, secret key, and bucket name

For MinIO or other S3-compatible storage:

1. Create a bucket
2. Create access credentials
3. Note the endpoint URL, access key, secret key, and bucket name

Push Your Repository to GitHub

Initialize your repository and push to GitHub:

git init
git add Dockerfile .dockerignore README.md
git commit -m "Initial GoSE deployment configuration"
git remote add origin https://github.com/yourusername/gose-deploy.git
git push -u origin main

Create a New Project on Klutch.sh

Navigate to the Klutch.sh dashboard and create a new project. Give it a descriptive name like “gose” or “file-share”.

Create a New App

Within your project, create a new app. Connect your GitHub account if you haven’t already, then select the repository containing your GoSE Dockerfile.

Configure HTTP Traffic

GoSE serves its web interface over HTTP. In the deployment settings:

• Select HTTP as the traffic type
• Set the internal port to 8080

Set Environment Variables

In the environment variables section, add your S3 configuration:

Variable	Value
GOSE_S3_ENDPOINT	Your S3 endpoint (e.g., s3.amazonaws.com)
GOSE_S3_REGION	Your S3 region (e.g., us-east-1)
GOSE_S3_BUCKET	Your bucket name
GOSE_S3_ACCESS_KEY	Your S3 access key
GOSE_S3_SECRET_KEY	Your S3 secret key
GOSE_BASE_URL	https://your-app-name.klutch.sh
GOSE_S3_PATH_STYLE	true (if using MinIO)

Deploy Your Application

Click Deploy to start the build process. Klutch.sh will:

• Detect your Dockerfile automatically
• Build the container image
• Start the GoSE container
• Provision an HTTPS certificate

Access GoSE

Once deployment completes, access your GoSE instance at https://your-app-name.klutch.sh. You can immediately start uploading and sharing files.

Using GoSE

Uploading Files

To upload files:

1. Navigate to your GoSE instance
2. Drag and drop files onto the upload area, or click to browse
3. GoSE automatically chunks and uploads your files
4. Once complete, copy the share link

Sharing Files

Share links are generated automatically after upload:

• Links point directly to S3 for download
• No authentication required for download (depends on S3 bucket policy)
• Files remain available until deleted from S3

Resuming Uploads

If an upload is interrupted:

1. Return to GoSE
2. Select the same file again
3. GoSE detects existing chunks and resumes from where it left off

Deduplication

GoSE automatically deduplicates at the chunk level:

• Identical chunks are only stored once
• Uploading similar files is faster
• Storage costs are reduced

S3 Provider Configuration

AWS S3

GOSE_S3_ENDPOINT=s3.amazonaws.com
GOSE_S3_REGION=us-east-1
GOSE_S3_PATH_STYLE=false

MinIO

GOSE_S3_ENDPOINT=minio.example.com
GOSE_S3_REGION=us-east-1
GOSE_S3_PATH_STYLE=true

Backblaze B2

GOSE_S3_ENDPOINT=s3.us-west-002.backblazeb2.com
GOSE_S3_REGION=us-west-002
GOSE_S3_PATH_STYLE=false

Ceph RadosGW

GOSE_S3_ENDPOINT=rgw.example.com
GOSE_S3_REGION=default
GOSE_S3_PATH_STYLE=true

Security Considerations

S3 Bucket Security

• Bucket Policies: Configure appropriate bucket policies for your use case
• Public Access: Consider whether files should be publicly downloadable
• Encryption: Enable server-side encryption in S3
• Versioning: Consider enabling versioning for file recovery

Access Control

GoSE doesn’t include built-in authentication. Consider:

• Reverse Proxy Auth: Add authentication via Nginx or similar
• Private Buckets: Use signed URLs for controlled access
• IP Restrictions: Limit upload access to specific networks

Credential Security

• Store S3 credentials in Klutch.sh environment variables
• Never commit credentials to your repository
• Rotate credentials regularly
• Use IAM roles with minimal permissions

Troubleshooting Common Issues

Upload Failures

Symptoms: Files fail to upload or stall.

Solutions:

• Verify S3 credentials are correct
• Check S3 bucket exists and is accessible
• Verify CORS is configured correctly on the S3 bucket
• Check browser console for errors
• Ensure sufficient S3 permissions

Share Links Not Working

Symptoms: Download links are broken.

Solutions:

• Verify GOSE_BASE_URL is set correctly
• Check S3 bucket policy allows read access
• Ensure S3 endpoint is publicly accessible
• Verify files were uploaded successfully

S3 Connection Errors

Symptoms: Cannot connect to S3 backend.

Solutions:

• Verify endpoint URL is correct
• Check region matches your bucket
• For MinIO, ensure GOSE_S3_PATH_STYLE=true
• Test S3 credentials with AWS CLI or similar tool

CORS Issues

Symptoms: Browser blocks S3 requests.

Solutions:

Configure CORS on your S3 bucket:

{
  "CORSRules": [
    {
      "AllowedOrigins": ["https://your-app-name.klutch.sh"],
      "AllowedMethods": ["GET", "PUT", "POST", "DELETE", "HEAD"],
      "AllowedHeaders": ["*"],
      "ExposeHeaders": ["ETag"],
      "MaxAgeSeconds": 3000
    }
  ]
}

Additional Resources

• GoSE GitHub Repository
• GoSE Codeberg Mirror
• AWS S3 Documentation
• MinIO Documentation
• Klutch.sh Deployments

Conclusion

Deploying GoSE on Klutch.sh gives you a powerful, scalable file sharing platform backed by S3 storage. The combination of GoSE’s efficient chunk-based uploads and Klutch.sh’s deployment simplicity means you can share files of any size without managing complex infrastructure.

With automatic deduplication, resumable uploads, and direct S3 transfers, GoSE provides enterprise-grade file sharing capabilities in a lightweight package. Whether you’re sharing large datasets with collaborators or providing file upload functionality for your application, GoSE on Klutch.sh delivers reliable, scalable file transfers.