Deployment

Standalone Worker

For traditional deployments, run one or more standalone worker processes. This is the most robust and scalable option.

# Run with default settings
python manage.py runworker

# Run with specific concurrency and queues
python manage.py runworker --concurrency 100 --queue high_priority --queue default

# Run with scheduler enabled
python manage.py runworker --scheduler

# Run a worker for a batch queue
python manage.py runworker --queue=batch_queue

Scaling

It's safe to run multiple standalone worker instances. Both backends use atomic operations to prevent multiple workers from picking up the same task:

• Postgres: SELECT ... FOR UPDATE SKIP LOCKED
• Valkey: BLMOVE

Each worker has a unique ID. If a worker process is terminated uncleanly, its in-process tasks will be abandoned. On startup, a worker will attempt to rescue any tasks that were previously abandoned by a worker with the same ID.

Health Checks (Kubernetes)

The worker can report its status by updating a file's modification time every 5 seconds. This is useful for Liveness Probes.

python manage.py runworker --health-check-file /tmp/worker_health

Kubernetes Liveness Probe:

livenessProbe:
  exec:
    command:
    - /bin/sh
    - -c
    - 'test -f /tmp/worker_health && [ $(($(date +%s) - $(stat -c %Y /tmp/worker_health))) -lt 15 ]'
  initialDelaySeconds: 10
  periodSeconds: 10

Embedded Worker (All-in-One)

For simpler deployments, run the worker inside your ASGI web server's event loop. This reduces the number of processes you need to manage.

1. Create an Embedded ASGI Entrypoint

# myproject/asgi_embedded.py
import os
from django.core.asgi import get_asgi_application
from django_vtasks.asgi import get_worker_application

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")

# Get the standard Django ASGI application
django_asgi_app = get_asgi_application()

# Wrap it with the worker application
application = get_worker_application(django_asgi_app)

2. Run with an ASGI Server

Use any ASGI-compliant server that supports the lifespan protocol:

# Example with Granian
granian --interface asgi myproject.asgi_embedded:application --host 0.0.0.0 --port 8000

Scaling Embedded Workers

You can run multiple instances of the embedded configuration. Each web server process has its own worker, and they coordinate through the shared backend (Postgres or Valkey).

Shutdown Behavior

The ASGI server owns process signals, so the embedded worker stops in two ways:

1. ASGI lifespan shutdown — the normal path: the server tells the app it is shutting down, and the worker drains its active tasks before the process exits.
2. Stop signal (SIGTERM/SIGINT) — a chained handler stops the worker and scheduler as soon as the process is told to stop, without replacing the server's own signal handler.

The second path matters on servers that deliver lifespan shutdown only after every in-flight HTTP request has drained (granian). There, a request that never completes — a client that stops sending mid-body is enough — would otherwise leave a "stopping" process consuming queue work indefinitely, competing with its own replacement on the shared queue. With the chained handler, the worker stops consuming within milliseconds of the signal regardless of what the HTTP side is doing. It is installed automatically; no configuration is needed.

For granian specifically, two settings are worth setting alongside this:

• GRANIAN_WORKERS_KILL_TIMEOUT (granian, disabled by default): without it, granian waits forever for a worker that cannot finish draining, and never escalates to SIGKILL. Set it (e.g. 60) so a wedged worker is force-killed instead of holding its memory next to its replacement until the OOM killer intervenes. Tasks lost to the SIGKILL are re-enqueued by task rescue on the next worker start.
• VTASKS_SHUTDOWN_CANCEL_GRACE (vtasks, disabled by default): opt-in refinement of the same situation. After the stop signal and worker drain, waits this many seconds and then cancels any asyncio tasks still pending, so the server's request drain can complete and the process exits cleanly rather than waiting to be SIGKILLed. This aborts any HTTP request still running when the grace expires — set it above your slowest legitimate request.

Metrics

Standalone Worker

Enable the metrics server:

python manage.py runworker --metrics-port 9100

Metrics are available at http://localhost:9100/.

Embedded Worker

When running in embedded mode, the worker shares the same process as your web server. All metrics are exposed via your application's standard metrics endpoint (e.g., /metrics provided by django-prometheus).

Do not use --metrics-port in embedded mode.

Memory Optimization

Reduce worker memory by removing unneeded INSTALLED_APPS:

# settings.py
if os.environ.get("VTASKS_IS_WORKER") == "true":
    INSTALLED_APPS = prune_installed_apps(INSTALLED_APPS)
    ROOT_URLCONF = "django_vtasks.empty_urls"  # Omit if tasks require "reverse"

Set VTASKS_IS_WORKER=true in your worker's environment variables.

Reliability

Valkey Reliable Queue Pattern

When using Valkey, django-vtasks implements the Reliable Queue Pattern:

1. Worker waits for a task
2. Task is atomically moved from q:default to processing:<worker_id> via BLMOVE
3. Task is processed and acknowledged (removed from processing list)

If a worker crashes (OOM kill, power failure), the task remains in its processing: list. On the next startup, the same worker (or a new one with the same ID) rescues the task and moves it back to the main queue.

Database Backend

The Database backend uses SELECT ... FOR UPDATE SKIP LOCKED for safe concurrent processing. Failed tasks are moved to a Dead Letter Queue for inspection.

Example Docker Compose

version: '3.8'

services:
  web:
    build: .
    command: granian --interface asgi myproject.asgi:application --host 0.0.0.0 --port 8000
    ports:
      - "8000:8000"
    depends_on:
      - db
      - valkey

  worker:
    build: .
    command: python manage.py runworker --scheduler --health-check-file /tmp/health
    environment:
      - VTASKS_CONCURRENCY=50
    depends_on:
      - db
      - valkey
    healthcheck:
      test: ["CMD", "sh", "-c", "test -f /tmp/health && [ $(($(date +%s) - $(stat -c %Y /tmp/health))) -lt 15 ]"]
      interval: 10s
      timeout: 5s
      retries: 3

  db:
    image: postgres:16
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: myapp
      POSTGRES_PASSWORD: secret

  valkey:
    image: valkey/valkey:7.2

Example Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vtasks-worker
spec:
  replicas: 3
  selector:
    matchLabels:
      app: vtasks-worker
  template:
    metadata:
      labels:
        app: vtasks-worker
    spec:
      containers:
      - name: worker
        image: myapp:latest
        command: ["python", "manage.py", "runworker", "--scheduler", "--health-check-file", "/tmp/health"]
        env:
        - name: VTASKS_CONCURRENCY
          value: "50"
        livenessProbe:
          exec:
            command:
            - /bin/sh
            - -c
            - 'test -f /tmp/health && [ $(($(date +%s) - $(stat -c %Y /tmp/health))) -lt 15 ]'
          initialDelaySeconds: 10
          periodSeconds: 10
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "512Mi"
            cpu: "500m"