A Node.js endpoint that generates a two-thousand-row PDF export, sends a confirmation email, and pings an external webhook, all synchronously inside the HTTP handler, responds in eight seconds on a good day and times out when the email provider is slow. The visible symptom is an API that's "sometimes slow," but the real cause is mixing two responsibilities that have no business sharing a request-response cycle: answering the user quickly, and running work that can take time or fail for reasons entirely outside the API's control.

Why a queue, not just a fire-and-forget async

Kicking off the heavy work in the background with setImmediate or an unawaited promise fixes the latency symptom, but creates a worse problem: if the Node process restarts (a deploy, a crash, horizontal scaling) mid-processing, the job disappears without a trace, no retry, no visibility. A persistent queue solves this by splitting job production (fast, inside the HTTP handler) from job consumption (in a separate worker, with retry and persistence).

BullMQ relies on Redis as its persistence and coordination backend. Each job sits in a dedicated Redis structure until it's processed, which guarantees it survives a worker restart.

// queue.js
import { Queue } from 'bullmq';
import { connection } from './redis.js';

export const exportQueue = new Queue('export-report', { connection });
// api/routes/export.js
app.post('/reports/:id/export', async (req, res) => {
  const job = await exportQueue.add('generate-pdf', {
    reportId: req.params.id,
    userId: req.user.id,
  }, {
    attempts: 3,
    backoff: { type: 'exponential', delay: 5000 },
  });

  res.status(202).json({ jobId: job.id });
});

The HTTP handler responds in a few milliseconds with a 202 Accepted and a job id, not the final result. That's a change to the API contract the client has to accept: the frontend now needs to poll a status endpoint or listen on a WebSocket instead of waiting for a synchronous response.

The worker: isolating the slow work

// worker.js
import { Worker } from 'bullmq';
import { connection } from './redis.js';
import { generatePdf } from './services/pdf.js';

const worker = new Worker('export-report', async (job) => {
  const { reportId, userId } = job.data;
  const pdfBuffer = await generatePdf(reportId);
  await uploadToStorage(pdfBuffer, `reports/${reportId}.pdf`);
  await notifyUser(userId, reportId);
}, {
  connection,
  concurrency: 5,
});

worker.on('failed', (job, err) => {
  console.error(`Job ${job.id} failed after ${job.attemptsMade} attempts`, err);
});

The worker runs in a process separate from the HTTP server, which lets it scale independently (more workers when the queue grows, with no need to touch the API instance count) and keeps a spike of heavy jobs from eating the event loop that also has to answer HTTP requests.

The concurrency and shared-resource trap

The concurrency option controls how many jobs a worker processes in parallel. A common mistake is setting it too high without accounting for limited external resources: a PDF generation step that spins up a headless browser (Puppeteer) easily eats 200 to 400 MB of memory per instance. With concurrency: 20 on a worker capped at 1 GB of RAM, the process gets OOM-killed after a handful of concurrent jobs, and BullMQ marks those jobs as stuck (stalled) rather than failed, which makes the diagnosis harder.

The right setting is measured, not guessed: watch the worker's memory under real load, then set concurrency against whichever resource is the tightest constraint (memory, connections to a rate-limited external API, database connections).

const worker = new Worker('export-report', processJob, {
  connection,
  concurrency: 5,
  limiter: {
    max: 10,
    duration: 1000, // at most 10 jobs started per second
  },
});

limiter is a different knob from concurrency: it caps the rate at which jobs start, useful when the work calls an external API with a quota (an email provider capped at N requests per second, for instance), independent of how many jobs run in parallel.

Idempotence and retry: the silent trap

BullMQ automatically retries a job that throws, following the configured attempts/backoff strategy. That behavior is only correct if the work is idempotent: if generatePdf succeeded on the previous attempt but notifyUser threw, a full retry regenerates the PDF and can send the notification email twice.

async function processJob(job) {
  const { reportId, userId } = job.data;

  let pdfUrl = job.data.pdfUrl;
  if (!pdfUrl) {
    const pdfBuffer = await generatePdf(reportId);
    pdfUrl = await uploadToStorage(pdfBuffer, `reports/${reportId}.pdf`);
    await job.updateData({ ...job.data, pdfUrl });
  }

  await notifyUser(userId, reportId, pdfUrl);
}

Persisting the intermediate result (pdfUrl) in the job's data via updateData means a retry after notifyUser fails doesn't regenerate the PDF; it resumes exactly where the job left off. It's the same principle as crash recovery in a distributed transaction, scaled down to a single job.

Deduplication with jobId

When the producer can be called more than once for the same logical operation (a double click, a client-side retry), setting a deterministic jobId prevents duplication at the queue level itself:

await exportQueue.add('generate-pdf', { reportId }, {
  jobId: `export-${reportId}`,
});

BullMQ silently refuses to add a job whose id already exists in the queue (as long as it hasn't completed), which avoids processing the same request twice without needing extra application-level deduplication logic.

Observability: knowing what's happening inside the queue

An unsupervised queue turns into a black box where jobs fail silently. BullMQ exposes events (completed, failed, stalled) and an API to query queue state (getJobCounts), which Bull Board or a custom dashboard can surface:

const counts = await exportQueue.getJobCounts('waiting', 'active', 'failed', 'delayed');
// { waiting: 3, active: 2, failed: 1, delayed: 0 }

Watching the waiting count over time is also the most direct signal for whether more workers are needed: a queue that keeps growing, even with zero errors, means production throughput is outpacing consumption throughput.

Graceful shutdown: don't lose a job mid-flight

A worker that gets killed (a deploy, an autoscaler scaling down, a container restart) while actively processing a job leaves that job in a stalled state until BullMQ's stall-detection timeout decides to retry it elsewhere. That's survivable for an idempotent job, but it can mean minutes of apparent silence on a job that was actually most of the way done. Handling SIGTERM explicitly lets the worker finish in-flight jobs before shutting down, rather than being cut off:

const worker = new Worker('export-report', processJob, { connection, concurrency: 5 });

process.on('SIGTERM', async () => {
  console.log('Received SIGTERM, finishing active jobs before exit...');
  await worker.close();
  process.exit(0);
});

worker.close() stops the worker from picking up new jobs and waits for jobs currently in progress to finish before resolving. Without it, an orchestrator's default termination grace period (often just a few seconds in Kubernetes) can kill the process mid-job, which is a common and avoidable source of jobs stuck in active with no worker left to ever mark them completed or failed.

Scheduled and recurring jobs

Not every job in a queue comes from a user action. Recurring work (a nightly report, a subscription renewal check every hour) is a frequent reason teams end up with a parallel cron setup next to their queue, which then needs its own monitoring and its own failure handling. BullMQ supports repeatable jobs natively, which keeps this kind of scheduling inside the same observability and retry machinery as everything else in the queue:

await exportQueue.add('nightly-digest', {}, {
  repeat: { pattern: '0 2 * * *' }, // every day at 02:00
});

The repeat option accepts a cron pattern, and BullMQ handles scheduling the next occurrence itself, including after a worker restart, since the schedule state lives in Redis rather than in the worker process's memory. This matters for the same reason persistence matters for one-off jobs: a cron job implemented as a setInterval in the Node process loses its schedule on every restart, while a BullMQ repeatable job resumes exactly where it should.

When BullMQ isn't enough

BullMQ handles volumes in the range of tens of thousands of jobs a day comfortably, with processing latency in the range of seconds to minutes. For very high-throughput stream processing (continuous events, tens of thousands of messages per second) or exactly-once delivery guarantees across a multi-service distributed system, a dedicated messaging system (Kafka, RabbitMQ with transactional acknowledgments) becomes the better fit, because Redis as a queue backend doesn't offer the same partitioning and retention guarantees as an actual distributed message log.

Takeaway

Offloading slow work to a queue changes the API contract (an asynchronous response with a pollable status, not a synchronous result) and shifts reliability responsibility from the HTTP handler to the worker. concurrency has to be tuned against the tightest resource constraint, not guessed, and every job should be written as idempotent from the start, because automatic retry is a guarantee, not an option.