How to Build an Idempotent Webhook Processor That Skips Duplicate Events

Use the Webhook trigger's event-id dedup header together with a MongoDB lookup so that a webhook event is processed exactly once, even when the sender retries the same delivery.

What This Integration Does

Webhook senders such as Shopify retry deliveries whenever they do not receive a fast acknowledgement, or when their own infrastructure hiccups. That means the same order-created or order-paid event can arrive two, three, or more times. If your workflow blindly creates a record, charges a customer, or sends an email on every delivery, those retries turn into duplicate side effects. This tutorial builds a guard that records every event id you have already handled in a MongoDB collection and short-circuits any repeat, giving you exactly-once processing on top of an at-least-once delivery channel.

The workflow is started by a Webhook trigger in Spojit. Each incoming POST carries an event id that you expose through the trigger's dedup header setting. As soon as a run starts, you look that id up in MongoDB with the Connector node calling find-documents. If a matching record already exists, a Condition node sends the run down a "skip" branch that does nothing. If it is new, you insert the id (claiming it), run your real business logic, and finish. Because the claim row is written before the side effects run, a retry that lands while or after the first run completes finds the row and is skipped. Re-runs of the same event leave no extra state behind.

Prerequisites

A MongoDB connection added under Connections with read and write access to a database you can use for a dedup ledger collection (this tutorial uses a collection named processed_events).
A Shopify connection if you want the example business-logic step (any other connector works the same way; the dedup pattern is connector-agnostic).
A signing connection for the Webhook trigger so deliveries are verified by HMAC. For Shopify webhooks, add a connection using the Shopify signing scheme; see Setting Up a Webhook Trigger.
The sender configured to include a stable, per-event id in a header (Shopify sends X-Shopify-Webhook-Id, which stays constant across retries of the same event).

Step 1: Configure the Webhook trigger and turn on dedup

Add a Trigger node and set its Trigger Type to Webhook. Pick your signing connection so each delivery is HMAC-verified, then copy the generated workflow URL into your sender (in Shopify, this is the webhook endpoint URL). Enable the opt-in dedup option and point it at the header that carries the event id, for example X-Shopify-Webhook-Id. The trigger returns 202 with an executionId the moment it accepts a delivery, and exposes the parsed JSON body and the dedup id to downstream nodes. Reference the body as {{ trigger.body }} and the event id as {{ trigger.eventId }} in later steps.

Treat {{ trigger.eventId }} as the single source of truth for "which event is this." Do not derive your own id from the payload contents, because a retry can carry a byte-for-byte identical body and you want the sender's stable id, not a hash that could collide across genuinely different events.

Step 2: Look up the event id in MongoDB

Add a Connector node in Direct mode, choose your MongoDB connection, and select the find-documents tool. Set collection to processed_events and set filter to match the current event id. Use a limit of 1 because you only need to know whether any row exists.

collection: processed_events
filter: { "eventId": "{{ trigger.eventId }}" }
limit: 1

The tool returns {{ step.data.documents }}, an array of matched documents. An empty array means this is the first time you have seen the event; a non-empty array means it is a duplicate. Name this node's output variable something clear like lookup so you can reference {{ lookup.data.documents }} downstream.

Step 3: Branch on whether the event was already seen

Add a Condition node directly after the lookup. Configure the if-branch to test whether the lookup found nothing, for example checking that the length of {{ lookup.data.documents }} equals 0. The true branch is your "new event" path that does the real work; the false branch is the "duplicate" path that intentionally ends the run with no side effects.

Keeping the duplicate branch empty is the whole point: a retry should be a no-op. You can optionally drop a Send Email node on the duplicate branch during early testing so you can watch retries getting caught, then remove it before going live.

Step 4: Claim the event id before doing any work

On the true (new event) branch, add a Connector node in Direct mode using your MongoDB connection and the insert-documents tool. Insert a single ledger row that records the id and a timestamp. Set collection to processed_events and pass a one-element documents array.

collection: processed_events
documents: [
  {
    "eventId": "{{ trigger.eventId }}",
    "source": "shopify",
    "processedAt": "{{ trigger.eventId }}"
  }
]

Claim the id here, before the business logic in the next step, so that the ledger row exists as early as possible. To make the claim collision-proof under concurrent retries, create a unique index on the eventId field in your MongoDB collection ahead of time. With a unique index in place, if two retries somehow reach the insert at the same instant, only one insert succeeds and the other fails fast, which keeps your side effects single.

Step 5: Run the real business logic

Still on the new-event branch and after the claim, add the Connector node that does the actual work. For an order webhook you might use the Shopify connector in Direct mode with get-order to fetch full order details from the lightweight webhook payload, then continue to whatever downstream action you need (creating a record, notifying a channel, and so on). Map the order id from the webhook body, for example {{ trigger.body.id }}, into the get-order input.

Because this step only ever runs once per event id (Step 3 already filtered out repeats, and Step 4 claimed the id), everything after this point is safe to treat as a single, authoritative pass over the event. If your logic spans several nodes, you can group concurrent independent calls with a Parallel node without affecting the guarantee.

Step 6: Acknowledge and end the run

Webhook triggers in Spojit already return 202 with an executionId the instant the delivery is accepted, so the sender gets its acknowledgement immediately and does not need to wait for your business logic to finish. You do not need a Response node for an async webhook like this. Simply let both branches reach the end of the workflow: the duplicate branch ends having done nothing, and the new-event branch ends after the claim and the business logic. The next retry of the same event will start a fresh run, find the claimed id in processed_events, and take the skip branch.

Tips

Add a unique index on eventId in your MongoDB ledger collection. The find-then-insert pattern alone has a tiny window where two concurrent retries can both see "not found"; the unique index closes it by rejecting the second insert.
Store a human-readable timestamp and the source in each ledger row. It makes the collection easy to audit later and lets you expire old rows with a scheduled cleanup workflow.
Keep the duplicate branch genuinely empty in production. The cheapest possible response to a retry is to do nothing at all.
If you process several webhook types into one collection, namespace the id (for example store "shopify:{{ trigger.eventId }}") so ids from different senders never collide.

Common Pitfalls

Deriving the dedup key from payload contents instead of the sender's stable event id. Two distinct events can share identical bodies, and one event's retries should share one id, so always use {{ trigger.eventId }} fed by the dedup header.
Inserting the ledger row after the business logic. If the work runs first and the run is retried before the insert lands, you get a duplicate side effect. Always claim the id before doing anything that has an external effect.
Forgetting the unique index. Without it, the insert-documents step happily writes a second row under a race, and your Condition guard cannot help once both runs are already past the lookup.
Letting the ledger grow forever. A high-volume webhook fills processed_events quickly; schedule a periodic cleanup that deletes rows older than your retry window using the MongoDB connector's delete-documents tool.

Testing

Before pointing real traffic at it, test on a small scope. Send a single test delivery from your sender (or use the Run button with a sample body that includes the dedup header), confirm the new-event branch ran and that exactly one row appeared in processed_events via a quick find-documents check. Then replay the same delivery so it carries the same event id: the run should take the skip branch and add no new row and no new side effect. Watch both executions in the run history to confirm the duplicate was caught. Ask Miraxa, the intelligent layer across your automation, "Why did my last run take the skip branch?" if a delivery is handled differently than you expect, since it can read the run and explain which Condition path fired.