How to Poll a REST Endpoint on a Schedule and Detect Changes Since Last Run

Build a Spojit workflow that fetches a REST resource on a cron schedule, compares it against the last snapshot stored in MongoDB, and posts to Slack only when something actually changed.

What This Integration Does

Many systems expose a REST endpoint but never push you a notification when their data changes: a pricing feed, a public status page, a vendor catalog, a configuration document. Polling that endpoint on a fixed schedule is easy, but a naive poll either spams you on every run or forces you to read the payload yourself to see what moved. This Spojit workflow closes that gap. It pulls the resource on a cron, diffs the new payload against the previous snapshot it saved, and notifies Slack with a precise list of what was added, removed, or changed. On runs where nothing moved, it stays silent.

The run model is a Schedule trigger firing on a Unix cron in your timezone. Each run fetches the resource with the http connector, loads the prior snapshot from a mongodb collection, and runs the json connector diff tool to produce a structured difference list. A Condition node gates the rest of the workflow on whether the difference count is above zero. When there are differences, a slack message goes out and the new payload is written back to MongoDB as the latest snapshot, so the next run compares against it. The workflow is idempotent in the sense that re-running it with unchanged data produces no notification and no snapshot churn.

Prerequisites

A reachable REST endpoint that returns JSON. If it needs authentication, have the API key or token ready to pass as a header on the http request.
A MongoDB connection added in Spojit (Connections -> Add connection -> MongoDB), pointing at a database where you can create a small snapshots collection.
A Slack connection authorized for the workspace and channel you want to post to, with permission to send messages.
A stable identifier for the resource you are polling (for example vendor-pricing). This becomes the document key so multiple resources can share one snapshots collection.

Step 1: Add a Schedule trigger

Start a new workflow and set the Trigger node type to Schedule. Enter a 5-field Unix cron expression and an IANA timezone. To poll every 15 minutes during business hours on weekdays in Sydney, use cron */15 9-17 * * 1-5 with timezone Australia/Sydney. A single Schedule trigger can hold more than one schedule if you want different cadences. The trigger output is { scheduledAt }, which you can reference downstream as {{ scheduledAt }} for logging or the notification timestamp.

Step 2: Fetch the resource with the http connector

Add a Connector node in Direct mode, choose the http connector, and select the http-get tool. Set the request url to your endpoint. If the API requires authentication, add a headers object such as the example below. Map the parsed response body to an output variable named fetched so later steps can read {{ fetched.data }} (or whichever field your endpoint nests its payload under).

{
  "url": "https://api.example.com/v1/pricing",
  "headers": {
    "Authorization": "Bearer YOUR_TOKEN",
    "Accept": "application/json"
  }
}

Keep this in Direct mode: it is a single predictable call, so there is no need for an agent and no AI cost.

Step 3: Load the previous snapshot from MongoDB

Add a second Connector node in Direct mode, choose the mongodb connector, and select the find-documents tool. Point it at your snapshots database and collection (for example poll_snapshots), and filter on the resource key so you read back only this resource's last saved payload.

{
  "database": "automation",
  "collection": "poll_snapshots",
  "filter": { "resourceKey": "vendor-pricing" },
  "limit": 1
}

Map the result to an output variable named previous. On the very first run there is no document yet, so this returns an empty result. Handle that in the next step by diffing against an empty object, which makes every field look "added" and produces a first-run notification (or you can skip the alert on first run with an extra Condition if you prefer a silent seed).

Step 4: Diff the two payloads with the json connector

Add a Connector node in Direct mode, choose the json connector, and select the diff tool. Pass the stored snapshot as obj1 and the freshly fetched payload as obj2. If your find-documents result is an array, reference the first element's stored payload field; use an empty object as a fallback for the first run.

{
  "obj1": {{ previous.documents.0.payload }},
  "obj2": {{ fetched.data }}
}

Map the result to an output variable named changes. The diff tool returns { differences, count }, where each entry in differences has a path (the diverging field), a type of added, removed, or changed, plus oldValue and newValue. A count of 0 means the payloads are identical. If you want a clean before/after compare, run a Transform node first to strip volatile fields like response timestamps or request IDs that change on every fetch but are not meaningful changes.

Step 5: Gate on whether anything changed

Add a Condition node that checks whether {{ changes.count }} is greater than 0. Wire the true branch to the notify-and-save steps below. Leave the false branch empty so that quiet runs end immediately with no Slack message and no write. This is the core of "notify only when something actually changed," and it keeps both your Slack channel and your snapshots collection free of noise.

Step 6: Notify Slack with the difference list

On the true branch, add a Connector node in Direct mode, choose the slack connector, and select the send-message tool. Set the target channel and compose a text body that summarizes the change count and references the run time. You can include the raw differences for full detail.

{
  "channel": "#pricing-watch",
  "text": "Detected {{ changes.count }} change(s) in vendor pricing at {{ scheduledAt }}.\n```{{ changes.differences }}```"
}

If you would rather have Spojit phrase the differences into a readable human summary instead of dumping the raw array, replace this step with a Connector node in Agent mode over the slack connector and prompt it to summarize {{ changes.differences }} before posting. Agent mode uses Miraxa, the intelligent layer across your automation, to do the reasoning and costs AI credits, so reserve it for when the readability is worth it.

Step 7: Save the new payload as the latest snapshot

Still on the true branch, add a final Connector node in Direct mode, choose the mongodb connector, and select the update-documents tool with upsert enabled. Match on the same resourceKey and overwrite the stored payload with the freshly fetched data, so the next scheduled run diffs against this version.

{
  "database": "automation",
  "collection": "poll_snapshots",
  "filter": { "resourceKey": "vendor-pricing" },
  "update": {
    "$set": {
      "resourceKey": "vendor-pricing",
      "payload": {{ fetched.data }},
      "updatedAt": "{{ scheduledAt }}"
    }
  },
  "upsert": true
}

Because the save lives on the true branch, the snapshot only advances when a real change was detected and announced. That keeps the stored baseline aligned with the last thing you were notified about, so a change is never silently absorbed.

Tips

Store one document per resource keyed by resourceKey, then reuse the same workflow pattern for many endpoints by changing only the key, the URL, and the channel.
Use a Transform node before diff to drop fields that change every fetch (server timestamps, ephemeral tokens, pagination cursors) so they do not trigger false-positive notifications.
Keep the cron cadence honest about the upstream rate limit. Polling every minute against an endpoint that updates hourly wastes calls; align the schedule to how often the data realistically moves.
If the payload is large, store and diff only the section you care about (for example {{ fetched.data.items }}) rather than the whole envelope, to keep diffs readable and writes small.

Common Pitfalls

First-run flood. With no prior snapshot, every field diffs as added. Either accept one seed notification or add an early Condition that skips the alert when previous is empty and just writes the baseline.
Snapshot saved on every run. If you place the update-documents step outside the true branch, the baseline always advances and you will never see the change on the next poll. Keep the save on the true branch.
Timezone surprises. The Schedule trigger cron is evaluated in the IANA timezone you set. A 0 9 * * * with the wrong timezone fires at the wrong local hour, so set Australia/Sydney (or your zone) explicitly.
Schema drift breaks the diff. If the endpoint renames or restructures fields, the diff will report large churn. Normalize with a Transform step, or narrow the diff to the stable subtree you actually monitor.

Testing

Before turning on the schedule, point the workflow at a low-risk resource and use the Run button to fire it once manually. Confirm the http-get step returns the payload, the first find-documents returns empty, and the diff step yields a sensible difference list with a non-zero count. Verify the Slack message arrives and a snapshot document is written. Run it a second time with no upstream change and confirm the Condition false branch is taken: no Slack message, no write. Finally, edit the source data (or temporarily diff a hand-edited copy), run again, and confirm only the changed fields appear in the difference list before enabling the cron.