homesponsors ⭐
 
   
🔍

Cron jobs and schedulers with BullMQ

July 4, 2026

In-process cron (node-cron, @nestjs/schedule, OS crontab) runs inside one Node process. That is fine for a single instance, but it does not survive restarts gracefully, deduplicate across replicas, or share infrastructure with your other background jobs.

BullMQ stores queues and schedulers in Redis. Job Schedulers (BullMQ 5.16+) are the recommended way to enqueue recurring work on a cron pattern or fixed interval. The same workers that process one-off jobs also process scheduled ones, with retries, backoff, and concurrency you already get from BullMQ.

This post covers Job Schedulers in plain Node.js, operations and pitfalls, a NestJS setup with @nestjs/bullmq, and a runnable demo with a fast cron heartbeat and a daily cleanup cron.

Prerequisites

For the NestJS section: npm i @nestjs/bullmq bullmq

BullMQ 2.0+ does not require a separate QueueScheduler instance. Use the Job Scheduler API (upsertJobScheduler), not the deprecated repeat option on queue.add().

Mental model

PieceRole
QueueHolds jobs waiting to run
WorkerExecutes jobs
Job SchedulerFactory that enqueues jobs on a schedule
Scheduled jobA job instance produced by a scheduler

A scheduler id is stable across deploys. Calling upsertJobScheduler with the same id updates the schedule in place instead of creating duplicates.

Queue and worker

Share one Redis connection config between the queue and the worker:

import { Queue, Worker } from 'bullmq';
const connection = { host: 'localhost', port: 6379 };
const queue = new Queue('reports', { connection });
const worker = new Worker(
'reports',
async (job) => {
console.log(`[${job.name}]`, new Date().toISOString(), job.data);
},
{ connection },
);
worker.on('failed', (job, error) => {
console.error(job?.name, error.message);
});

Start the worker before or shortly after registering schedulers. If no worker is running, jobs accumulate in Redis until one picks them up.

Cron schedulers

BullMQ uses a 6-field cron expression (optional seconds). A fast pattern for demos and heartbeats:

await queue.upsertJobScheduler(
'report-heartbeat',
{ pattern: '*/10 * * * * *' },
{
name: 'heartbeat',
data: { source: 'scheduler' },
opts: {
attempts: 3,
backoff: { type: 'exponential', delay: 1000 },
removeOnComplete: 50,
},
},
);

Daily cleanup at 03:15 in a specific timezone:

await queue.upsertJobScheduler(
'daily-cleanup',
{ pattern: '0 15 3 * * *', tz: 'Europe/Berlin' },
{
name: 'cleanup',
data: { scope: 'stale-sessions' },
opts: { attempts: 3 },
},
);

Set tz when the job must fire at a local wall-clock time. For millisecond intervals instead of cron, use every (mutually exclusive with pattern).

Other useful repeat options:

OptionPurpose
limitMaximum number of iterations
immediatelyRun once now, then follow the schedule
startDate / endDateBound the scheduler to a time window

Register schedulers on startup

Keep scheduler registration in a dedicated bootstrap script or onModuleInit hook so deploys upsert the same ids:

// scheduler.js
import { Queue } from 'bullmq';
const connection = { host: 'localhost', port: 6379 };
const queue = new Queue('reports', { connection });
await queue.upsertJobScheduler(
'report-heartbeat',
{ pattern: '*/10 * * * * *' },
{ name: 'heartbeat', data: { source: 'scheduler' } },
);
await queue.upsertJobScheduler(
'daily-cleanup',
{ pattern: '0 15 3 * * *', tz: 'Europe/Berlin' },
{ name: 'cleanup', data: { scope: 'stale-sessions' } },
);
const schedulers = await queue.getJobSchedulers();
console.log(
'Active schedulers:',
schedulers.map((item) => ({ key: item.key, pattern: item.pattern })),
);
await queue.close();

To remove a scheduler:

await queue.removeJobScheduler('daily-cleanup');

Shut down cleanly on SIGINT / SIGTERM: await worker.close() and await queue.close().

NestJS with @nestjs/bullmq

NestJS wraps BullMQ queues and workers as providers. Register Redis once, register the queue, inject it into a service that upserts schedulers on startup, and process jobs in a @Processor class.

// app.module.ts
import { Module } from '@nestjs/common';
import { BullModule } from '@nestjs/bullmq';
import { ReportsProcessor } from './reports.processor';
import { ReportsSchedulerService } from './reports-scheduler.service';
@Module({
imports: [
BullModule.forRoot({
connection: { host: 'localhost', port: 6379 },
}),
BullModule.registerQueue({ name: 'reports' }),
],
providers: [ReportsProcessor, ReportsSchedulerService],
})
export class AppModule {}
// reports-scheduler.service.ts
import { Injectable, OnModuleInit } from '@nestjs/common';
import { InjectQueue } from '@nestjs/bullmq';
import { Queue } from 'bullmq';
@Injectable()
export class ReportsSchedulerService implements OnModuleInit {
constructor(@InjectQueue('reports') private readonly reportsQueue: Queue) {}
async onModuleInit() {
await this.reportsQueue.upsertJobScheduler(
'report-heartbeat',
{ pattern: '*/10 * * * * *' },
{ name: 'heartbeat', data: { source: 'nestjs' } },
);
await this.reportsQueue.upsertJobScheduler(
'daily-cleanup',
{ pattern: '0 15 3 * * *', tz: 'Europe/Berlin' },
{ name: 'cleanup', data: { scope: 'stale-sessions' } },
);
}
}
// reports.processor.ts
import { Processor, WorkerHost } from '@nestjs/bullmq';
import { Job } from 'bullmq';
@Processor('reports')
export class ReportsProcessor extends WorkerHost {
async process(job: Job): Promise<void> {
console.log(`[${job.name}]`, new Date().toISOString(), job.data);
}
}

ReportsSchedulerService runs when the Nest app boots, so schedulers are upserted on every deploy. ReportsProcessor is the worker; Nest registers it automatically unless you set manualRegistration on BullModule.forRoot.

@nestjs/schedule (@Cron()) is still a good fit for trivial timers inside one instance. Prefer BullMQ schedulers when you already use Redis queues, run multiple replicas, or need the same retry and observability model as the rest of your background jobs.

Pitfalls

  • Worker must be running - schedulers enqueue jobs; something must consume them.
  • Busy queues slip - BullMQ creates the next scheduled job when the previous one starts processing. Under load, ticks can be less frequent than every or the cron interval suggests.
  • pattern vs every - mutually exclusive; pick one per scheduler.
  • Timezone - omit tz and cron runs in the server default zone; set it explicitly for "9 AM local" jobs.
  • Legacy repeat on add() - deprecated from BullMQ 5.16; use upsertJobScheduler for new code.

When to use what

ApproachGood for
@nestjs/schedule / node-cronSingle instance, simple in-process timers
BullMQ Job SchedulersMulti-instance apps, shared Redis, retries with async jobs
External cron + HTTPFire-and-forget HTTP triggers without queue semantics
2023

Postgres and Redis containers with Docker Compose

February 26, 2023

For Compose basics, see the Docker Compose overview. This post covers Postgres and Redis images with UI tools.

Prerequisites

  • Docker Compose installed

Configuration

The following configuration spins up Postgres and Redis containers with UI tools (Pgweb and Redis Commander).

Connection strings for Postgres and Redis are redis://localhost:6379 and postgres://username:password@localhost:5435/database-name.

Pgweb and Redis Commander are available at http://localhost:8085 and http://localhost:8081 links.

# docker-compose.yml
services:
postgres:
image: postgres:alpine
environment:
POSTGRES_DB: database-name
POSTGRES_PASSWORD: password
POSTGRES_USER: username
ports:
- 5435:5432
restart: on-failure:3
pgweb:
image: sosedoff/pgweb
depends_on:
- postgres
environment:
PGWEB_DATABASE_URL: postgres://username:password@postgres:5432/database-name?sslmode=disable
ports:
- 8085:8081
restart: on-failure:3
redis:
image: redis:latest
command: redis-server
volumes:
- redis:/var/lib/redis
- redis-config:/usr/local/etc/redis/redis.conf
ports:
- 6379:6379
networks:
- redis-network
redis-commander:
image: rediscommander/redis-commander:latest
environment:
- REDIS_HOSTS=local:redis:6379
- HTTP_USER=root
- HTTP_PASSWORD=qwerty
ports:
- 8081:8081
networks:
- redis-network
depends_on:
- redis
volumes:
redis:
redis-config:
networks:
redis-network:
driver: bridge

Run the following command to spin up the containers.

docker compose up
2021

Redis as custom storage for NestJS rate limiter

September 14, 2021

A rate limiter is a standard protection technique against brute force and DDoS attacks. NestJS provides a module for it, and the default storage is in-memory. Custom storage, Redis in this case, should be injected inside ThrottlerModule configuration.

Configuration

The configuration should contain

  • TTL (time to live) in seconds
  • maximum number of requests within TTL
  • Redis URL
// app.module.ts
import { APP_GUARD } from '@nestjs/core';
import { ThrottlerGuard, ThrottlerModule } from '@nestjs/throttler';
import { ThrottlerStorageRedisService } from 'nestjs-throttler-storage-redis';
// ...
@Module({
imports: [
ThrottlerModule.forRootAsync({
inject: [CustomConfigService],
useFactory: (configService: CustomConfigService) => ({
ttl: configService.THROTTLER_TTL_SECONDS,
limit: configService.THROTTLER_LIMIT,
storage: new ThrottlerStorageRedisService(configService.REDIS_URL)
})
})
// ...
]
})
export class AppModule {}

API endpoints setup

Binding the throttler guard can be done in multiple ways.

  • guard is bound globally for every API endpoint.
// app.module
import { ThrottlerGuard } from '@nestjs/throttler';
// ...
@Module({
// ...
providers: [
{
provide: APP_GUARD,
useClass: ThrottlerGuard
}
]
})
export class AppModule {}
  • global guard is overridden for the specific API endpoint with the Throttle decorator.
import { Throttle } from '@nestjs/throttler';
// ...
@Controller('users')
export class UsersController {
@Throttle(USERS_THROTTLER_LIMIT, USERS_THROTTLER_TTL_SECONDS)
@Get()
async getUsers() {}
}
  • global guard is skipped for the specific API endpoint with the SkipThrottle decorator.
import { SkipThrottle } from '@nestjs/throttler';
// ...
@Controller('posts')
export class PostsController {
@SkipThrottle()
@Get()
async getPosts() {}
}

Postgres and Redis container services for e2e tests in Github actions

September 8, 2021

End-to-end tests should use a real database connection, and provisioning container service for the Postgres database can be automated using Github actions. The environment variable for the connection string for the newly created database can be set in the step for running e2e tests. The same goes for the Redis instance.

# ...
jobs:
build:
# Container must run in Linux-based operating systems
runs-on: ubuntu-latest
# Image from Docker hub
container: node:20.9.0-alpine3.17
# ...
strategy:
matrix:
# ...
database-name:
- e2e-testing-db
database-user:
- username
database-password:
- password
database-host:
- postgres
database-port:
- 5432
redis-host:
- redis
redis-port:
- 6379
services:
postgres:
image: postgres:latest
env:
POSTGRES_DB: ${{ matrix.database-name }}
POSTGRES_USER: ${{ matrix.database-user }}
POSTGRES_PASSWORD: ${{ matrix.database-password }}
ports:
- ${{ matrix.database-port }}:${{ matrix.database-port }}
# Set health checks to wait until postgres has started
options: --health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: redis
# Set health checks to wait until redis has started
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
# ...
- run: npm run test:e2e
env:
DATABASE_URL: postgres://${{ matrix.database-user }}:${{ matrix.database-password }}@${{ matrix.database-host }}:${{ matrix.database-port }}/${{ matrix.database-name }}
REDIS_URL: redis://${{ matrix.redis-host }}:${{ matrix.redis-port }}
# ...
2020