A customer complains they were charged twice for the same order. Digging into the logs turns up two calls to POST /webhooks/stripe carrying the exact same event.id, a few seconds apart. The handler had no protection against that: each call created an invoice and fired a confirmation email. This isn't an exotic bug, it's the documented, expected behavior of every serious payment provider (Stripe, PayPal, Adyen, Mollie...), and any webhook integration that doesn't handle it explicitly will eventually double-charge someone.
Why webhooks arrive more than once
Payment providers guarantee "at least once" delivery, never "exactly once". The mechanism is simple: the provider sends the event, waits for a 2xx response within a short window (often a few seconds), and if the response doesn't arrive in time - a network blip, the server restarting mid-request, a slow downstream database call - it considers the delivery failed and retries later, with exponential backoff that can stretch over hours or days.
The classic trap: your server did process the event and created the invoice, but the HTTP response got lost on the way back (a dropped connection, a load balancer closing the socket), or your handler simply ran past the provider's timeout without actually failing. The provider has no way to know whether you processed the event or not; it only sees the missing 2xx. So it retries. If your code isn't idempotent, it reprocesses the event from scratch.
Diagnosing: confirm it's really a duplicate
Before fixing anything, check the application logs and the provider's dashboard to confirm the event.id is identical across both calls:
# Search application logs for repeated occurrences of the same event.id
grep "evt_1Nk3x" /var/log/app/webhooks.log
2024-02-14T10:22:01Z webhook received event=evt_1Nk3x type=payment_intent.succeeded
2024-02-14T10:22:03Z invoice created order=4821 amount=4900
2024-02-14T10:22:41Z webhook received event=evt_1Nk3x type=payment_intent.succeeded
2024-02-14T10:22:43Z invoice created order=4821 amount=4900
Two invoice creations for the same event.id, 40 seconds apart: that's a retry, not a provider bug. If the event.id values differ but map to the same logical transaction, the problem is different (the application code generated two payment intents for one user action, often an unguarded double click on the frontend) and gets fixed elsewhere.
The fix: idempotency-key deduplication before any side effect
The principle is to check, atomically, whether the event has already been processed, before triggering any business side effect (invoice creation, email, account credit).
CREATE TABLE processed_webhook_events (
event_id VARCHAR(255) PRIMARY KEY,
event_type VARCHAR(100) NOT NULL,
processed_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
// webhook-handler.ts (NestJS / Node)
async function handleStripeWebhook(event: Stripe.Event, db: Pool) {
const client = await db.connect();
try {
await client.query('BEGIN');
// Atomic insert: if event_id already exists, ON CONFLICT does nothing
// and rowCount is 0, which tells us "already processed".
const result = await client.query(
`INSERT INTO processed_webhook_events (event_id, event_type)
VALUES ($1, $2)
ON CONFLICT (event_id) DO NOTHING`,
[event.id, event.type],
);
if (result.rowCount === 0) {
await client.query('ROLLBACK');
logger.info(`Event ${event.id} already processed, skipping`);
return;
}
// Business side effects, inside the same transaction when possible
if (event.type === 'payment_intent.succeeded') {
const intent = event.data.object as Stripe.PaymentIntent;
await createInvoiceForOrder(client, intent.metadata.order_id, intent.amount_received);
}
await client.query('COMMIT');
} catch (err) {
await client.query('ROLLBACK');
throw err;
} finally {
client.release();
}
}
The key detail is running the INSERT ... ON CONFLICT DO NOTHING inside the same transaction as the business side effects: if the invoice gets created but the commit fails (the server crashes right after), the processed_webhook_events row doesn't exist either, so the later retry correctly recreates the invoice. If the insert succeeds but the business logic fails afterward, the rollback undoes both, and the next retry processes everything cleanly. This avoids the inconsistent state of "event marked processed but invoice never created."
A detail that's easy to overlook: respond 2xx as fast as possible, including on a detected duplicate. Doing heavy business processing (a third-party API call, PDF generation) synchronously inside the webhook handler raises the risk of hitting the provider's timeout and therefore the number of retries. The robust pattern is to do deduplication and persistence synchronously, then hand the actual business processing off to a queue (BullMQ, SQS) that is itself idempotent and processed asynchronously.
Also verify the signature, not just the event.id
Deduplication alone doesn't protect against a forged request. Every provider signs its payloads with a shared secret, and that signature must be verified against the raw request body, before any JSON parsing happens:
import Stripe from 'stripe';
const sig = req.headers['stripe-signature'];
let event: Stripe.Event;
try {
// req.body must be the raw Buffer, not already-parsed JSON
event = stripe.webhooks.constructEvent(req.rawBody, sig, process.env.STRIPE_WEBHOOK_SECRET);
} catch (err) {
logger.warn(`Invalid signature: ${err.message}`);
return res.status(400).send('invalid signature');
}
A common mistake in Express/NestJS apps is letting a global JSON body-parser middleware transform the body before constructEvent can verify the signature against the exact bytes the provider sent; the webhook endpoint needs its own route using express.raw().
When the deduplication table isn't enough
The processed_webhook_events table solves duplication at the webhook event level, but it doesn't solve the case where two genuinely different events (two distinct payment_intent.succeeded events) map to the same business intent - for example a user submitting a payment form twice before the first attempt resolves. That case is handled upstream, at PaymentIntent creation time, with an idempotency key your application supplies explicitly:
await stripe.paymentIntents.create(
{ amount: 4900, currency: 'eur', customer: customerId },
{ idempotencyKey: `order_${orderId}_attempt_1` },
);
With that key, Stripe itself guarantees that repeated calls carrying the same key return the same PaymentIntent instead of creating a second one, even across client-side network retries. The two mechanisms (an idempotency key at creation time, deduplication on webhook receipt) are complementary and cover different failure points.
Out-of-order delivery and table growth
Two more edge cases tend to surface once the basic deduplication is in place. First, providers do not guarantee ordering: a charge.refunded event can, in rare cases, be delivered and processed before the payment_intent.succeeded event for the same payment if the two are dispatched close together and one retry overtakes the other. Keying state transitions purely on "the last event I received" is fragile; it's safer to make the handler check the current state of the underlying resource (fetch the payment intent's actual status from the provider's API, or compare timestamps) rather than trusting that events always arrive in the order they were generated.
Second, processed_webhook_events grows forever if nothing prunes it, and most providers don't retry past a bounded window (Stripe gives up after three days by default). A simple retention job removes the risk of an ever-growing table without reintroducing duplicate processing risk:
-- Run nightly: events older than the provider's retry window are safe to drop
DELETE FROM processed_webhook_events
WHERE processed_at < now() - interval '7 days';
Seven days here is a deliberate margin above the provider's own retry window, not an arbitrary number; check the specific provider's documented maximum retry duration before picking the cutoff.
Takeaway
A payment webhook will be delivered more than once; that's a protocol guarantee, not an anomaly. The handler must check, atomically and inside the same transaction as any side effect, whether the event.id has already been processed before creating an invoice or sending an email. Pair that with an idempotency key at payment-intent creation to cover client-side double submissions, and respond 2xx quickly by offloading heavy processing to an asynchronous, itself-idempotent queue.
