Deploying a Celery App

Celery is a powerful, distributed task queue for Python, used to run background jobs and process tasks asynchronously. It supports multiple brokers (like Redis and RabbitMQ) and is widely adopted for scalable, reliable job processing in Python applications.

This guide explains how to deploy a Celery application to Klutch.sh, both with and without a Dockerfile. It also covers installation and provides sample code to get started.

Prerequisites

• Python 3.8+
• pip installed
• Git and GitHub account
• Klutch.sh account

---

Getting Started: Install Celery

1. Create a new directory for your app and set up a virtual environment:

   mkdir my-celery-app
   cd my-celery-app
   python3 -m venv venv
   source venv/bin/activate

2. Install Celery:

   pip install celery

3. Create a basic Celery app (tasks.py):

   from celery import Celery
   
   app = Celery('tasks', broker='redis://localhost:6379/0')
   
   @app.task
   def add(x, y):
       return x + y

4. Add a requirements.txt file:

   pip freeze > requirements.txt

5. Test locally:

   celery -A tasks worker --loglevel=info

---

Deploying Without a Dockerfile

1. Push your Celery app to a GitHub repository.
2. Log in to Klutch.sh.
3. Create a new project and give it a name.
4. Create a new app:
   • Select your Celery GitHub repository and branch
   • Choose region, compute, number of instances, and add any environment variables
5. Add a start command in your app settings:

   celery -A tasks worker --loglevel=info

6. Click “Create” to deploy. Klutch.sh will build and deploy your app automatically.

---

Deploying With a Dockerfile

1. Add a Dockerfile to your project root. Example:

   # Use official Python image
   FROM python:3.11-slim
   
   # Set working directory
   WORKDIR /app
   
   # Copy requirements and install dependencies
   COPY requirements.txt ./
   RUN pip install --no-cache-dir -r requirements.txt
   
   # Copy app source
   COPY . .
   
   # Start the Celery worker
   CMD ["celery", "-A", "tasks", "worker", "--loglevel=info"]

2. Push your code (with Dockerfile) to GitHub.
3. In Klutch.sh, follow the same steps to create a project and app, but select the Dockerfile option when prompted.
4. Set the service details and environment variables as needed.
5. Click “Create” to deploy. Klutch.sh will build your Docker image and deploy your app.

Note: Make sure your broker (e.g., Redis) is accessible from your Klutch.sh deployment.

---

Resources

• Klutch.sh Documentation
• Celery Documentation

---

Deploying to Klutch.sh is simple and flexible. Choose the method that best fits your workflow and project requirements.