How to Sync FTP Contracts Into a Searchable Knowledge Collection on a Schedule

Build a Spojit workflow that runs every night, lists new contract files dropped on an FTP server, downloads each one, embeds it into a persistent Knowledge collection, and posts the indexed counts to Slack.

What This Integration Does

Legal and procurement teams often receive signed contracts as files dropped onto an FTP server by an external party or a document system. On their own those files are just bytes in a folder: nobody can ask "which vendor agreements renew in Q3?" without opening each one. This workflow turns that folder into a searchable archive. Each night it picks up whatever contract files have landed, embeds their text into a persistent Knowledge collection in Spojit, and leaves you with a long-lived, AI-queryable library that any other workflow in your workspace can read.

The run model is a nightly Schedule trigger. On each run the workflow lists the contracts directory on FTP, loops over the files, downloads each one as bytes, and embeds it into a named persistent collection with the file name as the document key. Because the collection is persistent, it accumulates across nights: every run adds to the same archive rather than starting fresh. The Knowledge node overwrites a document if a file of the same name was already embedded, so re-running a night (or re-sending the same file) updates the existing entry instead of duplicating it. At the end of the run a Slack message reports how many files were found and how many were indexed, so you have a daily heartbeat of the sync.

Prerequisites

An FTP connection in Spojit pointing at the server that holds your contracts. Add it under Connections -> Add connection and confirm it works (the FTP connector exposes a verify-connection tool you can run from a Connector node to test reachability).
The remote directory path where contract files are dropped, for example /contracts/incoming, and read access to it.
A persistent Knowledge collection to embed into. Create it under the Knowledge section of the sidebar with New Collection (name it something like contracts). The embedding model is fixed when the collection is created, so pick it now and keep it.
A Slack connection, and the channel name or ID where the nightly summary should post.
Knowing the file format your contracts arrive in (most commonly PDF or Word), since the Knowledge node needs a Document Type.

Step 1: Add a Schedule trigger for the nightly run

Add a Trigger node and set its type to Schedule. Enter a 5-field Unix cron expression and an IANA timezone so the run fires at a fixed local time. For example, to run at 2:00 AM every day in Sydney:

Cron:     0 2 * * *
Timezone: Australia/Sydney

A single Schedule trigger can hold multiple schedules if you ever want, for example, both a weekday and a weekend slot. The trigger output is { scheduledAt }, the timestamp of the fire, which you can reference later as {{ scheduledAt }} if you want it in the Slack summary.

Step 2: List the contracts directory on FTP

Add a Connector node in Direct mode, choose your FTP connection, and select the list-directory tool. Set the path field to the folder your contracts land in:

path: /contracts/incoming

Map the result to an output variable such as listing. The tool returns the directory contents under data.entries, where each entry has name, type (file or directory), size, and modifiedAt (an ISO 8601 timestamp). You will iterate over {{ listing.data.entries }} in the next step.

Step 3: Loop over the files

Add a Loop node set to ForEach and point it at the entries from Step 2:

{{ listing.data.entries }}

Inside the loop body each item is available as the current entry, for example {{ entry.name }} and {{ entry.type }}. If your incoming folder can contain subfolders or non-contract files, add a Condition node as the first node in the loop body that only continues when the entry is a file of the type you expect, for example when {{ entry.type }} equals file and {{ entry.name }} ends with .pdf. The download and embed steps below all live inside this loop body.

Step 4: Download each contract file

Inside the loop body, add a Connector node in Direct mode using the same FTP connection, and select the download-file tool. Build the full remote path from the directory and the current entry name, and set the encoding to base64 so binary files such as PDFs and Word documents come back intact:

path:     /contracts/incoming/{{ entry.name }}
encoding: base64

Map the result to a variable such as file. The tool returns the bytes under data.content (a base64 string when you requested base64 encoding), along with data.encoding and data.size (the decoded byte length). You will feed {{ file.data.content }} straight into the Knowledge node next.

Step 5: Embed the file into your persistent collection

Still inside the loop body, add a Knowledge node in Embed mode. Configure it to write into your long-lived archive:

Collection: pick your persistent collection (contracts), not Transient. This is what makes the archive accumulate across nights and stay queryable from other workflows.
File Name: use the current entry name so each document is keyed by its filename: {{ entry.name }}. If a document with that name already exists in the collection, embedding overwrites it, which is what keeps re-runs from creating duplicates.
Document Type: choose the type that matches your files, for example PDF or Word.
Document Input: reference the downloaded bytes: {{ file.data.content }}.
Output Variable: map to something like embedResult. The output reports the chunk count and metadata for the document that was just embedded.

Keep the embedding model consistent with whatever the collection was created with. Because the model is fixed at collection creation, you only embed here, never re-pick it.

Step 6: Count what was indexed and post a Slack summary

After the loop, you want a count of how many files were found versus how many were embedded. The simplest reliable count is the number of entries the directory listing returned. Add a Transform node (or a Connector node using the array utility's length tool) to compute the total from {{ listing.data.entries }}, and map it to a variable such as fileCount.

Then add a Connector node in Direct mode using your Slack connection and the send-message tool. Set the channel and a templated message, for example:

channel: #contracts-ops
text:    Nightly contract sync complete. Files found in /contracts/incoming: {{ fileCount }}. Indexed into the "contracts" collection at {{ scheduledAt }}.

This gives the team a single daily heartbeat in Slack confirming the run happened and how much was archived. If you want the alert only on failures instead of every night, see the related article on Slack failure alerts below.

Tips

Keep the embed step deterministic by using a Connector node in Direct mode for the FTP calls. There is no AI judgment needed to list and download files, so Direct mode avoids spending AI credits on plumbing. The Knowledge node handles the embedding itself.
Once the archive is built, any other workflow in your workspace can read it: add a Knowledge node in Query mode pointed at the contracts collection with a natural-language Prompt like "list vendor agreements that renew before September". Collections are workspace-scoped, so you build the index once and query it everywhere.
If you want a "new files only" sync rather than re-listing everything, move processed files after embedding with the FTP rename tool (for example into /contracts/archived), so the incoming folder only ever holds files you have not yet indexed.
Use Miraxa, the intelligent layer across your automation, to scaffold this quickly. From the Workflow Designer try a prompt like "Add a Loop over {{ listing.data.entries }}, and inside it a Connector node that downloads each file from FTP and a Knowledge node that embeds it into the contracts collection." Then fine-tune the field mappings in the properties panel.

Common Pitfalls

Forgetting base64 encoding on download. Contract files are binary. If you leave download-file on the default utf8 encoding, PDFs and Word docs corrupt. Always set encoding to base64 and feed data.content into the Knowledge node.
Mismatched Document Type. The Knowledge node parses based on the Document Type you select. Pointing a PDF at a Plain Text type, or vice versa, produces poor or empty chunks. If your folder mixes formats, branch on the file extension with a Condition node and set the correct type per branch.
Listing folders as if they were files. list-directory returns directories and symlinks too, each with a type. Filter to type equal to file before downloading, or your download step will error on a directory entry.
Switching embedding models. The collection's embedding model is fixed at creation. Always embed (and later query) the same collection with the same model. If you need a different model, create a new collection rather than mixing models in one.
Timezone drift on the schedule. Cron without a timezone, or with the wrong IANA zone, fires at an unexpected local hour. Always set both the cron expression and the timezone explicitly so "2 AM" means 2 AM where your team is.

Testing

Before turning on the nightly schedule, validate the logic on a small scope. Drop two or three sample contract files into a test folder on the FTP server and temporarily point the list-directory path at it. Run the workflow with the Run button (you can run a scheduled workflow on demand) and watch the execution: confirm the listing returns your test entries, the loop iterates once per file, each download returns a non-zero data.size, and each Knowledge embed reports a chunk count. Then open the contracts collection in the Knowledge sidebar and check the document table shows each file with a READY status and a sensible chunk count. Finally, add a temporary Knowledge Query node and ask a question you know the answer to from one of the test files to confirm retrieval works. Once that passes, point the path back at the real folder and enable the schedule.