Backend cheatsheetBullMQ queuesSlow work, off the request

BullMQ
Toolbox

Queues, workers and every state a job passes through, then the options, limits and Redis settings that keep background work reliable once real traffic arrives.

Start with a queue See the job states Key terms in plain words

By Shree Kumar SharmaRedisNestJS7 min read

Key terms in plain words

New to job queues? Start here. Every word below shows up later on this page, and each one comes with an everyday comparison from a busy restaurant kitchen. Underlined words in the modules link back to these cards.

The building blocks What happens to a job Running workers well Chains and timetables Running it for real

26 terms in 5 groups. Hover an underlined word anywhere on the page for a quick reminder, or click it to jump here.

The building blocks

BullMQ works like a restaurant kitchen. Waiters pin order tickets on a rail, and cooks take them down one at a time and make the food.

Queue: Think of it as the ticket rail above the kitchen pass; A named waiting line for work. Your app puts tasks on it and they wait there until something is free to do them. In BullMQ a queue is just a name stored in Redis.; For example new Queue('power-ups') makes a queue called power-ups.
Job: Think of it as one order ticket pinned on that rail; A single piece of work waiting to be done, such as sending one email. It has a name, some data describing the task, and options that say how to handle it.; For example queue.add('magic-word', { gameId: 'g42' }) adds one job.
Producer: Think of it as the waiter who writes the ticket and pins it up; Any part of your program that adds jobs to a queue. It doesn't wait for the work to happen, so it can answer the user straight away.
Worker: Think of it as a cook who takes tickets off the rail and makes the dish; A program that pulls jobs from a queue and runs them. You can start more workers on more machines when the queue gets busy, without touching the rest of the app.
Redis: Think of it as the rail itself, where every ticket physically hangs; A very fast database that keeps its data in memory. BullMQ stores every queue and job in Redis, so producers and workers on different machines all see the same list.
Connection: Think of it as the hatch between the dining room and the kitchen; The open line between your program and Redis. Queues and workers each need one, passed in as the connection option.

What happens to a job

A ticket moves along the rail, gets cooked, and either goes out to the table or comes back to be tried again.

Job state: Think of it as where the ticket sits right now: waiting, being cooked, served or sent back; The stage a job is at. The main ones are waiting, delayed, prioritized, active, completed and failed. Counting jobs in each state tells you whether the queue is healthy.
Retry: Think of it as sending a burnt dish back to be cooked again; Running a job again after it fails. The attempts option sets how many tries a job gets in total before it is marked failed for good.
Backoff: Think of it as waiting a bit longer before each new try, in case the oven needs to heat up; A pause between retries. Exponential backoff doubles the pause every time, so a struggling service gets more and more breathing room instead of being hammered.; For example 2s, then 4s, then 8s, then 16s.
Delay and priority: Think of it as "cook this at 8pm" and "this table first" written on the ticket; A delay holds a job back until a set time has passed. Priority lets some jobs jump ahead of others; in BullMQ 1 is the highest, and leaving it out keeps plain first come, first served order.
Lock and stalled job: Think of it as a cook's name written on the ticket, so nobody else picks it up; While a worker runs a job it holds a lock on it. If the worker crashes or freezes, the lock runs out and the job is called stalled. BullMQ then hands it to another worker.
Job id: Think of it as a table number on the ticket, so the same order is never cooked twice; A name you choose for a job. Adding a second job with the same id does nothing, so a task that might be triggered twice still runs only once.; For example jobId: 'grievance-report:2026-09-30'
Removing finished jobs: Think of it as clearing old tickets off the spike at the end of the night; Finished jobs stay in Redis until you remove them. The removeOnComplete and removeOnFail options delete them after a set age or once there are too many.

Running workers well

How many dishes a cook handles at once, how fast the kitchen may go, and how to close up without dropping a plate.

Concurrency: Think of it as how many pans one cook keeps on the stove at the same time; The number of jobs one worker runs at once. Raise it for jobs that mostly wait on something else, such as a network call, because the worker sits idle during that wait anyway.
Rate limit: Think of it as a rule that only so many plates may leave the pass each minute; A cap on how many jobs start in a stretch of time, shared by every worker on the queue. Useful when an outside service only accepts so many calls a second.
Progress: Think of it as a cook calling out "half done" across the kitchen; A number a job reports while it runs, such as how much of a batch it has sent. Dashboards and listeners can show it live.
Event: Think of it as a bell that rings whenever a dish goes out or comes back; A notice that something happened, such as a job completing or failing. Your code can listen for it and react, for example by writing a log line.
Graceful shutdown: Think of it as taking no new orders at closing time, but finishing the ones already on the stove; When the app is told to stop, worker.close() stops it picking up new jobs and waits for running ones to finish. A deploy then never cuts a job in half.

Chains and timetables

Dishes that need other dishes first, and jobs that come round on a schedule.

Flow: Think of it as a main course that only goes out once its sides are ready; A parent job with child jobs. The children run first, each with its own retries, and the parent starts only once all of them have finished and can read their results.
Job scheduler: Think of it as the standing order for bread that arrives every morning; A rule that adds a job on a timer, such as every minute or 9am each day. Setting it up twice still leaves just one schedule, so every server can do it at start up safely.
Cron pattern: Think of it as a shorthand note on the kitchen calendar; Five fields that describe when something repeats: minute, hour, day of month, month and day of week. A star means "any".; For example 0 9 * * * means 9:00 every day.
Idempotent: Think of it as pressing the lift button twice still calls only one lift; Doing something twice gives the same result as doing it once. It makes retries and repeated start ups safe.

Running it for real

Wiring it into a NestJS app and setting up Redis so jobs don't go missing.

Dependency injection: Think of it as the head chef handing each cook the tools they need instead of each one fetching their own; Instead of a class creating the things it uses, the framework creates them once and passes them in. In NestJS, @InjectQueue hands a class its queue this way.
Provider: Think of it as a supplier on the kitchen's list that the chef can call on; Anything NestJS, a popular framework for Node.js server apps, can create and hand to other classes, such as a service or a queue. With @nestjs/bullmq your queues become providers too.
Eviction policy: Think of it as the rule for which tickets get thrown out when the rail is full; What Redis does when it runs out of memory. Most settings quietly delete old keys, which would delete jobs. For a queue it must be noeviction, which refuses new writes instead.
Metrics: Think of it as the count of tickets waiting at the pass during a rush; Numbers you watch over time, such as how many jobs are waiting or how long they take. A waiting count that keeps growing is the first sign the workers can't keep up.

Queue, worker, job

A queue is a name in Redis. Producers add jobs to it, workers take them out and run them. Nothing else is needed to start.

BasicsQueueWorkerRedis

Picture it: producers, Redis, workers

power-ups.tsTypeScript

const connection = new IORedis(process.env.REDIS_URL!, { maxRetriesPerRequest: null });

export const powerUps = new Queue('power-ups', { connection });
await powerUps.add('magic-word', { gameId: 'g42', userId: 'u_91' });

new Worker('power-ups', async (job) => {
  const hint = await words.findHint(job.data.gameId);
  return { hint }; // stored as job.returnvalue
}, { connection, concurrency: 10 });

Why it matters: the API answers at once, and slow work runs on workers you can scale on their own.

Queuenew Queue("power-ups")Adds jobs and inspects the queue. Cheap, use it anywhere.

Workernew Worker("power-ups", fn)Pulls and runs jobs. Holds a blocking Redis connection.

The life of a job

Every job moves through a small set of states. Knowing them is how you read the dashboard and debug a stuck queue.

StatesMust knowRetriesStalled

Picture it: every state a job can be in

Not drawn: a flow parent waits in waiting-children until its children finish, job.retry() sends a failed job back to waiting, and a job whose worker died goes back to waiting once its lock expires.

State	Means	Check with
waiting	Ready, no worker free yet	`getWaitingCount()`
delayed	Waiting for a time: a delay or a backoff	`getDelayedCount()`
prioritized	Ready, ordered by priority	`getPrioritizedCount()`
active	A worker holds its lock	`getActiveCount()`
completed	Returned a value	`getCompletedCount()`
failed	Threw and has no attempts left	`getFailed()`
waiting-children	A flow parent waiting for its children	`getWaitingChildrenCount()`

Adding jobs

Options decide when a job runs, how often it retries, whether duplicates are allowed and how long it is kept.

OptionsattemptsbackoffjobId

add-with-options.tsTypeScript

await emails.add('grievance-report', { date: '2026-09-30' }, {
  jobId: 'grievance-report:2026-09-30',    // same id twice: the second add is ignored
  attempts: 5,
  backoff: { type: 'exponential', delay: 2_000 },
  delay: 60_000,                            // not before one minute from now
  priority: 1,                              // 1 is the highest; omit for plain FIFO
  removeOnComplete: { age: 86_400, count: 1_000 },
  removeOnFail: { age: 7 * 86_400 },
});

await emails.addBulk(recipients.map((r) => ({ name: 'send', data: r }))); // one round trip

Why it matters: a stable jobId makes a nightly job safe to trigger twice, and removeOn options stop Redis from filling up.

How long does a failing job wait between tries? Exponential backoff doubles the delay each time. Run it to see the schedule for the job above.

backoff.jsJavaScript

const delay = 2000;
const attempts = 5;
const exponential = (made) => Math.round(Math.pow(2, made - 1) * delay);

let total = 0;
for (let made = 1; made < attempts; made++) {
  const wait = exponential(made);
  total += wait;
  console.log('after try ' + made + ' wait ' + wait / 1000 + 's');
}
console.log('gives up after ' + attempts + ' tries, ' + total / 1000 + 's of waiting');

TerminalOutput

$ node backoff.js
after try 1 wait 2s
after try 2 wait 4s
after try 3 wait 8s
after try 4 wait 16s
gives up after 5 tries, 30s of waiting

Why it matters: five attempts at a 2 second base spread retries over half a minute, enough to ride out a short outage.

Option	Default	Set it when
`attempts`	1	The work can fail for reasons that pass, like a timeout
`backoff`	none	With attempts; exponential for flaky services
`jobId`	random	The job must not run twice for the same thing
`delay`	0	Reminders, grace periods, debouncing
`removeOnComplete`	keep all	Always, or Redis grows forever

Workers in production

Concurrency, rate limits, progress and a clean shutdown. The four settings every worker needs before it goes live.

WorkersconcurrencylimiterGraceful shutdown

concurrency: 10

Jobs one worker runs at once. Raise it for IO bound work.

See the example

Throughput

limiter{ max, duration }

At most max jobs per duration across all workers of the queue.

See the example

Rate limit

updateProgress(50)

Report progress; QueueEvents and dashboards see it.

See the example

Feedback

close()

Stops taking jobs and waits for active ones to finish.

See the example

Shutdown

email.worker.tsTypeScript

const worker = new Worker('emails', async (job) => {
  const batch = chunk(job.data.recipients, 500);
  for (const [i, part] of batch.entries()) {
    await mailer.send(part);
    await job.updateProgress(Math.round(((i + 1) / batch.length) * 100));
  }
}, {
  connection,
  concurrency: 5,
  limiter: { max: 50, duration: 1_000 }, // the provider allows 50 calls a second
});

worker.on('failed', (job, err) => log.error({ jobId: job?.id, err }, 'email job failed'));
process.on('SIGTERM', async () => { await worker.close(); process.exit(0); });

Why it matters: a deploy never cuts a job in half, and the mail provider never sees more than it allows.

SIGTERMdeploy starts
worker.close()no new jobs taken
Active jobs finishup to your grace period
exit(0)the pod stops cleanly

Flows and schedules

Flows run a parent after its children. Job schedulers add a job on a timer or a cron pattern, once per schedule, whatever the number of pods.

AdvancedFlowProducerupsertJobSchedulerCron

Picture it: a parent waits for its children

campaign.flow.tsTypeScript

await new FlowProducer({ connection }).add({
  name: 'send-campaign', queueName: 'campaigns', data: { campaignId },
  children: [
    { name: 'render-template', queueName: 'templates', data: { campaignId } },
    { name: 'resolve-audience', queueName: 'audience', data: { campaignId } },
    { name: 'check-quota', queueName: 'quota', data: { campaignId } },
  ],
});
// in the parent worker: const results = await job.getChildrenValues();

Why it matters: each step retries on its own, and the send starts only when everything it needs is ready.

schedules.tsTypeScript

await reports.upsertJobScheduler('daily-grievance-report',
  { pattern: '0 9 * * *', tz: 'Asia/Kolkata' },   // 9am IST every day
  { name: 'grievance-report', data: { recipients: 'management' } });

await sync.upsertJobScheduler('provider-sync', { every: 60_000 }); // every minute

Why it matters: upsert is idempotent, so every pod can call it on boot and there is still one schedule.

BullMQ in NestJS

@nestjs/bullmq registers queues as providers and turns a class into a worker, with dependency injection everywhere.

Framework@nestjs/bullmqWorkerHost@InjectQueue

power-ups.module.tsTypeScript

@Module({
  imports: [
    BullModule.forRoot({ connection: { url: process.env.REDIS_URL, maxRetriesPerRequest: null } }),
    BullModule.registerQueue({ name: 'power-ups', defaultJobOptions: { attempts: 3, removeOnComplete: 1_000 } }),
  ],
  providers: [PowerUpsService, PowerUpsProcessor],
})
export class PowerUpsModule {}

@Injectable()
export class PowerUpsService {
  constructor(@InjectQueue('power-ups') private readonly queue: Queue) {}
  use(gameId: string, kind: 'magic-word' | 'highlight') { return this.queue.add(kind, { gameId }); }
}

@Processor('power-ups', { concurrency: 10 })
export class PowerUpsProcessor extends WorkerHost {
  async process(job: Job<{ gameId: string }>) {
    return job.name === 'magic-word' ? this.hints.magicWord(job.data.gameId) : this.hints.highlight(job.data.gameId);
  }

  @OnWorkerEvent('failed')
  onFailed(job: Job, err: Error) { this.log.error({ jobId: job.id, err }); }
}

Why it matters: queues are injected like any provider, and moving slow work behind them is what made the power up APIs about 35% faster.

Production checklist

Redis settings, connections and cleanup that decide whether a queue survives its first busy week.

OperationsnoevictionmaxRetriesPerRequestMetrics

Setting	Value	Why
Redis `maxmemory-policy`	`noeviction`	Any other policy can silently delete job keys
Worker `maxRetriesPerRequest`	`null`	Blocking commands must wait, not fail after retries
`removeOnComplete` and `removeOnFail`	an age or a count	Finished jobs otherwise stay in Redis forever
`lockDuration`	30000 ms default	Raise it for jobs that block the event loop for long
Graceful shutdown	`await worker.close()`	Deploys do not cut jobs in half
Metrics	waiting, active, failed, duration	A growing waiting count is the first sign of trouble

redis-cliTerminal

$ redis-cli CONFIG GET maxmemory-policy
1) "maxmemory-policy"
2) "noeviction"
# anything else here and jobs can vanish under memory pressure

Which one do I need?

Name the behaviour you need, then the option or class that gives it to you.

I want to	Use
Answer the API now, do the work later	`queue.add()` and a Worker
Retry flaky work	`attempts` with `backoff: exponential`
Never run the same job twice	A stable `jobId`
Stay under a provider’s rate limit	Worker `limiter: { max, duration }`
Run something every morning	`upsertJobScheduler` with a cron pattern
Start a step after several others	A flow with `FlowProducer`
Deploy without losing jobs	`await worker.close()` on SIGTERM