n8n Coolify Deployment: Queue Mode, Worker Scaling, and Production Hardening for Heavy Workflows

n8n Coolify Deployment: Queue Mode, Worker Scaling, and Production Hardening for Heavy Workflows

The basic n8n Coolify deployment handles a lot. But when you're running dozens of concurrent workflows — AI pipelines, data syncs, webhook processors, scheduled batch jobs — the single-process default mode becomes a bottleneck. Queue mode separates workflow triggering from execution, lets you scale workers independently, and prevents heavy workflows from blocking fast ones. This guide covers n8n Coolify deployment at production scale: queue mode with Redis, multiple worker instances, security hardening, and monitoring that tells you what's actually happening inside your automation stack.

This guide builds on our existing n8n Coolify series. If you haven't deployed n8n on Coolify yet, start with our basic n8n Coolify deployment guide. For local development on Windows, see our WSL2 setup guide. For the full self-hosting picture, read our comprehensive self-hosting guide.

Prerequisites

A running Coolify instance with at least one connected server — see our basic deployment guide
n8n already deployed on Coolify with PostgreSQL — queue mode requires PostgreSQL, not SQLite
At least 2 vCPU and 4GB RAM on your server — queue mode with workers uses more memory than single-process mode
A Redis instance — you can deploy one via Coolify's database management or use a separate Redis container
Basic familiarity with Docker Compose and environment variables

Verify your current n8n deployment is healthy before making changes:

# SSH into your Coolify server and check n8n container health
docker ps | grep n8n
docker logs n8n --tail 20

# Confirm PostgreSQL is the active database (not SQLite)
docker exec n8n env | grep DB_TYPE
# Should return: DB_TYPE=postgresdb
# If empty or sqlite, switch to PostgreSQL before enabling queue mode

# Check current n8n version
docker exec n8n n8n --version

Understanding Queue Mode: Why It Matters

In n8n's default single-process mode, one Node.js process handles everything: receiving webhook triggers, scheduling cron jobs, executing workflow steps, managing the UI, and serving the API. Under load, long-running workflows block everything else. A 10-minute data processing job can prevent a 2-second Slack notification from firing on time.

How Queue Mode Changes the Architecture

Main process — handles the n8n UI, API, webhook reception, and scheduling. Enqueues workflow execution jobs to Redis. Never executes workflows itself.
Worker processes — pull execution jobs from the Redis queue and run them. Can run on the same server or different servers. Scale horizontally by adding more workers.
Redis — the job queue. Decouples triggering from execution. Workers pick up jobs as capacity allows rather than everything competing on one process.

When Queue Mode Is Worth It

Enable queue mode when:

You have workflows that take more than 30 seconds to execute
You receive webhook bursts (multiple triggers arriving simultaneously)
You have more than 5–10 active workflows running on schedules
You want to isolate workflow failures so they don't affect the n8n UI
You need to process workflows on a separate, more powerful server from the one running the UI

Deploying Redis for the Job Queue

Before enabling queue mode, deploy a Redis instance. In Coolify, go to New Resource → Database → Redis on the same server as n8n. Configure it with a strong password and keep it internal (not publicly accessible).

Redis Configuration for n8n Queue Mode

# If deploying Redis via Docker Compose alongside n8n (alternative to Coolify's managed Redis):
# Add this service to your n8n Compose config in Coolify:

  redis:
    image: redis:7-alpine
    container_name: n8n_redis
    restart: unless-stopped
    command: redis-server --requirepass ${REDIS_PASSWORD} --maxmemory 512mb --maxmemory-policy allkeys-lru
    volumes:
      - n8n_redis_data:/data
    networks:
      - n8n_network
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 3

# Add to volumes:
  n8n_redis_data:

# Verify Redis is reachable from the n8n container:
docker exec n8n redis-cli -h n8n_redis -a YOUR_REDIS_PASSWORD ping
# Should return: PONG

Enabling Queue Mode in Coolify

Updating the Main n8n Service Environment

In Coolify, go to your n8n application → Environment Variables. Add the queue mode configuration. These variables tell the main n8n process to act as a coordinator rather than a workflow executor:

# Environment variables to add/update in Coolify for the MAIN n8n instance:

# Enable queue mode
EXECUTIONS_MODE=queue

# Redis connection (use internal Docker network name if on same server)
QUEUE_BULL_REDIS_HOST=n8n_redis
QUEUE_BULL_REDIS_PORT=6379
QUEUE_BULL_REDIS_PASSWORD=your-strong-redis-password
QUEUE_BULL_REDIS_DB=0

# How long to keep completed execution data (seconds)
# 604800 = 7 days
QUEUE_BULL_REDIS_TIMEOUT_THRESHOLD=10000
EXECUTIONS_DATA_MAX_AGE=604800
EXECUTIONS_DATA_PRUNE=true
EXECUTIONS_DATA_PRUNE_MAX_COUNT=10000

# Health check for the queue
QUEUE_HEALTH_CHECK_ACTIVE=true
QUEUE_HEALTH_CHECK_PORT=5678

# Keep existing PostgreSQL settings:
# DB_TYPE=postgresdb
# DB_POSTGRESDB_HOST=...
# etc.

# Existing required settings stay the same:
# N8N_HOST=your-n8n-domain.com
# N8N_PROTOCOL=https
# WEBHOOK_URL=https://your-n8n-domain.com/
# N8N_ENCRYPTION_KEY=your-existing-key

Full Queue Mode Docker Compose Configuration

For a complete queue mode setup via Coolify's Docker Compose deployment, here's the full configuration with main process, worker, and Redis:

version: '3.8'

services:
  # Main n8n process — UI, API, webhooks, scheduling
  n8n:
    image: n8nio/n8n:latest
    container_name: n8n_main
    restart: unless-stopped
    ports:
      - "5678:5678"
    environment:
      # Queue mode
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - QUEUE_BULL_REDIS_PASSWORD=${REDIS_PASSWORD}
      # Database
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=${POSTGRES_HOST}
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
      - DB_POSTGRESDB_USER=${POSTGRES_USER}
      - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
      # Server config
      - N8N_HOST=${N8N_HOST}
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
      - WEBHOOK_URL=https://${N8N_HOST}/
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
      # Auth
      - N8N_BASIC_AUTH_ACTIVE=true
      - N8N_BASIC_AUTH_USER=${N8N_BASIC_AUTH_USER}
      - N8N_BASIC_AUTH_PASSWORD=${N8N_BASIC_AUTH_PASSWORD}
      # Execution settings
      - EXECUTIONS_DATA_PRUNE=true
      - EXECUTIONS_DATA_MAX_AGE=168
      - GENERIC_TIMEZONE=UTC
    volumes:
      - n8n_data:/home/node/.n8n
    depends_on:
      redis:
        condition: service_healthy
    networks:
      - n8n_network

  # Worker — pulls jobs from Redis queue and executes workflows
  n8n_worker:
    image: n8nio/n8n:latest
    container_name: n8n_worker
    restart: unless-stopped
    command: worker
    environment:
      # Same queue and DB config as main — workers must connect to the same backend
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - QUEUE_BULL_REDIS_PASSWORD=${REDIS_PASSWORD}
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=${POSTGRES_HOST}
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
      - DB_POSTGRESDB_USER=${POSTGRES_USER}
      - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
      - GENERIC_TIMEZONE=UTC
      # Worker concurrency — how many workflows this worker runs simultaneously
      - QUEUE_WORKER_CONCURRENCY=5
    volumes:
      - n8n_data:/home/node/.n8n
    depends_on:
      - n8n
      - redis
    networks:
      - n8n_network

  redis:
    image: redis:7-alpine
    container_name: n8n_redis
    restart: unless-stopped
    command: redis-server --requirepass ${REDIS_PASSWORD} --maxmemory 512mb --maxmemory-policy allkeys-lru
    volumes:
      - n8n_redis_data:/data
    networks:
      - n8n_network
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 3

volumes:
  n8n_data:
  n8n_redis_data:

networks:
  n8n_network:

Verifying Queue Mode Is Active

# Check main process is running in queue mode
docker logs n8n_main --tail 30 | grep -i queue
# Look for: Queue mode is active

# Check worker is connected and polling
docker logs n8n_worker --tail 30 | grep -iE '(worker|queue|connected|polling)'
# Look for: Worker is ready

# Verify Redis has the n8n queue keys:
docker exec n8n_redis redis-cli -a YOUR_REDIS_PASSWORD keys 'bull:jobs*'
# After running a workflow, you'll see: bull:jobs:wait, bull:jobs:active, etc.

# Monitor the queue in real time:
docker exec n8n_redis redis-cli -a YOUR_REDIS_PASSWORD \
  subscribe bull:jobs:completed bull:jobs:failed
# Executions appear here as they complete or fail

Scaling Workers in Coolify

Adding More Worker Instances

One of the key advantages of queue mode is horizontal scaling. When your single worker is saturated, add more. In Coolify, you can scale the worker service independently of the main n8n process:

# Scale workers via Docker Compose replicas in Coolify:
# Update the n8n_worker service in your Compose config:

  n8n_worker:
    image: n8nio/n8n:latest
    command: worker
    restart: unless-stopped
    deploy:
      replicas: 3    # Run 3 worker instances
      resources:
        limits:
          cpus: '1.0'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 512M
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_WORKER_CONCURRENCY=5    # 5 concurrent workflows per worker
      # ... rest of config

# With 3 workers × 5 concurrency = 15 workflows executing simultaneously
# Scale based on observed queue depth and CPU/memory usage

# Monitor queue depth and worker activity:
docker exec n8n_redis redis-cli -a YOUR_REDIS_PASSWORD \
  llen bull:jobs:wait
# Returns number of workflows waiting for a worker
# If consistently > 0, add more workers or increase concurrency

Dedicated Workers for Heavy Workflows

Some workflows are CPU-intensive (AI calls, image processing, large data transforms). Separate them from lightweight workflows using custom job queues to prevent heavy jobs from consuming all worker capacity:

# Deploy specialized workers with different queue settings:

  # Fast worker — handles quick workflows (webhooks, notifications)
  n8n_worker_fast:
    image: n8nio/n8n:latest
    command: worker --concurrency=10
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PASSWORD=${REDIS_PASSWORD}
      - QUEUE_WORKER_CONCURRENCY=10
      # Handle only the default queue
      - QUEUE_BULL_SETTINGS_PREFIX=n8n-default
      - DB_TYPE=postgresdb
      # ... db config
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
    deploy:
      replicas: 2

  # Heavy worker — handles long-running AI and data workflows
  n8n_worker_heavy:
    image: n8nio/n8n:latest
    command: worker --concurrency=2
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PASSWORD=${REDIS_PASSWORD}
      - QUEUE_WORKER_CONCURRENCY=2   # Low concurrency — these are expensive
      - DB_TYPE=postgresdb
      # ... db config
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
    deploy:
      replicas: 1
      resources:
        limits:
          cpus: '2.0'
          memory: 2G

Security Hardening for Production n8n on Coolify

Replacing Basic Auth with n8n's Built-In User Management

Basic auth is fine for initial setup, but n8n's proper user management system gives you individual accounts, roles, and audit trails. Migrate to it in production:

# In Coolify environment variables, replace basic auth with user management:

# Remove these:
# N8N_BASIC_AUTH_ACTIVE=true
# N8N_BASIC_AUTH_USER=...
# N8N_BASIC_AUTH_PASSWORD=...

# Add these instead:
N8N_USER_MANAGEMENT_DISABLED=false

# Configure SMTP for invitation emails and password reset:
N8N_EMAIL_MODE=smtp
N8N_SMTP_HOST=smtp.yourdomain.com
N8N_SMTP_PORT=587
[email protected]
N8N_SMTP_PASS=your-smtp-password
[email protected]
N8N_SMTP_SSL=false

# Set the instance owner on first access:
# Navigate to https://your-n8n-domain.com/setup
# Create the owner account — this disables the setup page permanently

# Restrict who can sign up (prevent open registration):
N8N_DISABLE_PRODUCTION_MAIN_PROCESS=false
# Note: In queue mode, only the main process serves the UI
# Workers don't expose any HTTP endpoints — they're inherently isolated

Webhook Security: Validating Inbound Requests

n8n webhook URLs are publicly reachable by default. Add validation to prevent unauthorized triggering of your workflows:

# In your n8n workflows, add validation at the top of webhook-triggered workflows:

# 1. Use the Header Auth node to validate a shared secret:
# Webhook node → Header Auth
# Header Name: X-Webhook-Secret
# Header Value: {{ $env.WEBHOOK_SECRET }}
# (Set WEBHOOK_SECRET in Coolify env vars)

# 2. Or validate in a Function node (first node after Webhook):
# const expectedToken = $env.WEBHOOK_SECRET;
# const receivedToken = $input.first().json.headers['x-webhook-secret'];
# if (receivedToken !== expectedToken) {
#   throw new Error('Unauthorized');
# }
# return $input.all();

# 3. For GitHub/Stripe/other services, use their signature validation:
# Most have specific n8n nodes that handle signature verification automatically

# Limit webhook access by IP in Nginx (if using an external reverse proxy):
# location /webhook/ {
#     allow YOUR_TRUSTED_IP;
#     deny all;
#     proxy_pass http://localhost:5678;
# }

Monitoring Queue Mode with Grafana and Prometheus

Queue mode adds operational complexity — you need visibility into job queue depth, worker health, execution success rates, and processing latency. n8n exposes metrics via a Prometheus endpoint in queue mode:

# Enable n8n's Prometheus metrics endpoint:
# Add to Coolify environment variables:
N8N_METRICS=true
N8N_METRICS_PREFIX=n8n_

# The metrics endpoint is available at:
# https://your-n8n-domain.com/metrics
# (Restricted to authenticated requests in recent versions)

# Add to your Prometheus scrape config:
  - job_name: 'n8n'
    scrape_interval: 30s
    static_configs:
      - targets: ['localhost:5678']
    metrics_path: '/metrics'
    # If n8n requires auth:
    basic_auth:
      username: your-n8n-user
      password: your-n8n-password

# Key n8n metrics to monitor:
# n8n_executions_total           — total workflow executions
# n8n_executions_success_total   — successful executions
# n8n_executions_error_total     — failed executions
# n8n_executions_waiting_total   — workflows waiting in queue
# n8n_executions_running_total   — actively executing workflows

# Useful PromQL queries for Grafana dashboards:

# Success rate over last hour:
rate(n8n_executions_success_total[1h]) /
  (rate(n8n_executions_success_total[1h]) + rate(n8n_executions_error_total[1h])) * 100

# Queue depth (should be 0 or near 0 in a healthy system):
n8n_executions_waiting_total

# Error rate — alert if > 5%:
rate(n8n_executions_error_total[5m]) / rate(n8n_executions_total[5m]) * 100 > 5

Redis Queue Monitoring

# Monitor Redis queue health directly:

# Jobs waiting to be picked up by a worker:
docker exec n8n_redis redis-cli -a YOUR_PASSWORD llen "bull:jobs:wait"

# Jobs currently being processed:
docker exec n8n_redis redis-cli -a YOUR_PASSWORD llen "bull:jobs:active"

# Failed jobs (need investigation):
docker exec n8n_redis redis-cli -a YOUR_PASSWORD zcount "bull:jobs:failed" -inf +inf

# Completed jobs (retained for inspection):
docker exec n8n_redis redis-cli -a YOUR_PASSWORD zcount "bull:jobs:completed" -inf +inf

# Redis memory usage:
docker exec n8n_redis redis-cli -a YOUR_PASSWORD info memory | grep used_memory_human

# Add Redis exporter for Prometheus:
docker run -d \
  --name redis-exporter \
  -p 9121:9121 \
  -e REDIS_ADDR=redis://n8n_redis:6379 \
  -e REDIS_PASSWORD=YOUR_PASSWORD \
  --network n8n_network \
  oliver006/redis_exporter:latest

Tips, Gotchas, and Troubleshooting

Workers Start But Don't Pick Up Jobs

# Check worker logs for connection errors:
docker logs n8n_worker --tail 50 | grep -iE '(error|redis|queue|connect)'

# Common causes:

# 1. Wrong Redis password in worker config
#    Verify it matches exactly (no trailing spaces):
docker exec n8n_worker env | grep REDIS_PASSWORD
docker exec n8n_redis redis-cli -a YOUR_PASSWORD ping

# 2. Worker and main process have different ENCRYPTION_KEY
#    Both MUST use the identical N8N_ENCRYPTION_KEY value
docker exec n8n_main env | grep N8N_ENCRYPTION_KEY
docker exec n8n_worker env | grep N8N_ENCRYPTION_KEY
# These must match exactly

# 3. Redis network not reachable from worker container
docker exec n8n_worker ping n8n_redis
# Should resolve — if not, check Docker network config in Compose

# 4. Main process not in queue mode (worker has nothing to connect to)
docker logs n8n_main | grep EXECUTIONS_MODE
# Should show: queue

Workflows Stuck in "Running" State

# Workflows stuck in running usually means the worker crashed mid-execution
# Check for orphaned jobs in Redis:
docker exec n8n_redis redis-cli -a YOUR_PASSWORD \
  zrange "bull:jobs:active" 0 -1

# If jobs are stuck, they'll stay there until the lock expires (usually 30s)
# Force cleanup by restarting the worker:
docker restart n8n_worker

# n8n will pick up orphaned jobs automatically after the lock expires
# To check stuck executions in the n8n database:
docker exec n8n_main n8n executionList --status running --limit 20

# Manually mark stuck executions as failed (use with caution):
docker exec n8n_main n8n executionRetry EXECUTION_ID
# Or via UI: Executions → find stuck execution → delete and re-trigger if needed

High Memory Usage in Workers

# Monitor worker memory usage:
docker stats n8n_worker --no-stream

# If memory grows unbounded, Node.js memory limit may be too high
# Add NODE_OPTIONS to cap heap size:
# In Coolify environment variables for the worker:
NODE_OPTIONS=--max-old-space-size=1024
# This caps the Node.js heap at 1GB — set to 75% of your memory limit

# Also ensure EXECUTIONS_DATA_PRUNE is set on both main and workers:
EXECUTIONS_DATA_PRUNE=true
EXECUTIONS_DATA_MAX_AGE=168   # 7 days in hours

# Check if a specific workflow is leaking memory:
# Run docker stats during the workflow's execution
# Compare memory before and after — steady growth = memory leak in workflow code
docker stats n8n_worker --format "table {{.Name}}\t{{.MemUsage}}" | head -5

Updating n8n in Queue Mode

# Zero-downtime update procedure for queue mode:

# 1. Stop workers FIRST (let running jobs complete naturally)
docker stop n8n_worker
# Or in Coolify: scale workers to 0 replicas

# 2. Wait for in-flight jobs to complete:
docker exec n8n_redis redis-cli -a YOUR_PASSWORD llen "bull:jobs:active"
# Wait until this returns 0

# 3. Update n8n image in Coolify and redeploy the main process
# Application → Deployments → Redeploy (after updating image tag)

# 4. Restart workers with the new image
docker compose pull n8n_worker
docker compose up -d n8n_worker

# 5. Verify workers reconnected:
docker logs n8n_worker --tail 10
# Look for: Worker is ready

# Check version after update:
docker exec n8n_main n8n --version

Pro Tips

Set QUEUE_WORKER_CONCURRENCY based on workflow type — I/O-bound workflows (API calls, database queries) can handle 10+ concurrency per worker. CPU-bound workflows (data transformation, image processing) should use 2–3. Mix them by using dedicated worker instances as described above.
Use n8n's execution history for debugging — in queue mode, execution history is stored in PostgreSQL rather than SQLite, making it more queryable. You can write SQL directly against the execution history table during incident investigation: SELECT workflow_id, status, started_at, stopped_at FROM execution_entity WHERE status = 'error' ORDER BY started_at DESC LIMIT 20;
Instrument your workflows with logging nodes — add Function nodes at key points in complex workflows that write to n8n's internal log. In queue mode, these logs appear in the worker container logs, making distributed debugging possible without the UI.
Keep the main process lightweight — the main process in queue mode handles UI and webhook reception. Don't co-locate it with resource-intensive services on a small server. The UI becoming slow during a workflow burst is a sign the main process is being starved.
Test your encryption key backup — in queue mode, all workers and the main process share the same N8N_ENCRYPTION_KEY. If this key is lost, all stored credentials (API keys, OAuth tokens, database passwords) become unrecoverable. Store it in Vaultwarden or another secure vault, separate from your Coolify server.

Wrapping Up

Queue mode transforms n8n from a capable automation tool into a production-grade workflow engine that scales with your needs. The basic n8n Coolify deployment covered in our getting started guide gives you a working foundation. Queue mode with independent workers, proper security hardening, and Prometheus monitoring gives you a platform you can actually rely on for business-critical automations.

The upgrade path is gradual: deploy Redis, enable queue mode on your existing Compose config in Coolify, deploy a single worker, and verify everything works before adding more workers or tuning concurrency. The operational investment pays off the first time a 20-minute AI workflow runs without blocking your Slack notification workflows from firing on time.

For building the actual workflows that run on this infrastructure, check out our guide on building AI automation workflows with self-hosted n8n.

Need a Production-Grade Automation Platform Built for Your Team?

Designing queue mode architecture, sizing workers for your specific workflow mix, integrating n8n with your existing infrastructure, and setting up proper monitoring and alerting for an automation stack that business processes depend on — the sysbrix team handles the infrastructure so your team can focus on building the workflows that matter.

Talk to Us →

in Guides

# Automation Coolify DevOps Self-Hosted n8n

Uptime Kuma Setup: API Automation, Multi-Location Monitoring, and Incident Management Workflows

Learn how to automate monitor creation via the Uptime Kuma API, build multi-location monitoring with distributed agents, integrate alerts into full incident workflows, and treat your uptime configuration as code that survives server rebuilds.