How to Maintain a Persistent Technical Specs Library Queryable from Any Workflow

Build a long-lived engineering specification library in Spojit by ingesting documents from an FTP server into a persistent Knowledge collection on a schedule, then expose it through a reusable Subworkflow that any other workflow can call to ask plain-language questions.

What This Integration Does

Engineering teams keep specifications, datasheets, and tolerance documents in shared file systems where they are hard to search and easy to forget. This tutorial turns that pile of files into a queryable knowledge base. One workflow watches an FTP directory on a schedule, downloads any new specification documents, and embeds them into a persistent Knowledge collection that lives at the workspace level. A second, smaller workflow wraps a Knowledge query so that any of your other automations (an order workflow, an approval workflow, a Slack alert) can ask "what is the maximum operating temperature for part X" and get a grounded answer without re-implementing the lookup each time.

The ingestion workflow runs on a Schedule trigger, so it fires unattended on the cadence you choose and leaves durable state behind: every embedded document persists in the collection until you delete it. Because Embed mode overwrites a document when you reuse its file name, re-runs are safe and idempotent for files that have not changed. The query Subworkflow is stateless: it takes a question as input, runs a single Knowledge query against the collection, and returns the answer to its parent, so calling it a thousand times never mutates the library. Changes you make to the child workflow take effect immediately for every parent that calls it.

Prerequisites

An FTP connection in Spojit pointing at the server and directory where your specification files land. See the FTP connector docs for credential setup.
A persistent Knowledge collection created ahead of time. In the Knowledge section of the sidebar, choose New Collection, name it (for example technical-specs), and note that the embedding model is fixed at creation. Use the same model for every document and every query.
A Slack connection so the ingestion run can post a short summary of what it indexed. See the Slack connector docs.
Know which document formats you will ingest (PDF, Word, Excel, CSV, Markdown, and Plain Text are all supported by the Knowledge node) and roughly how large the files are. Single documents over a few megabytes embed more slowly.

Step 1: Start the ingestion workflow on a Schedule trigger

Create a new workflow named Specs Library Ingest. Add a Trigger node and set its type to Schedule. Enter a 5-field cron expression and an IANA timezone, for example 0 6 * * 1-5 with Australia/Sydney to run at 6am on weekdays. A single Schedule trigger can hold several schedules if you want both a morning and evening pass. The trigger output is {{ scheduledAt }}, which you can reference downstream if you want to stamp the run time into your Slack summary.

Step 2: List the FTP directory

Add a Connector node in Direct mode, choose your FTP connection, and select the list-directory tool. Set path to the remote folder holding your specs, for example /engineering/specs. The tool returns an array of entries under data.entries, each with name, type, size, and modifiedAt. Name the output variable listing so later steps can read {{ listing.data.entries }}. Direct mode is deliberate here: listing a directory is a single predictable call with no AI cost.

Step 3: Loop over the files and download each one

Add a Loop node in ForEach mode iterating over {{ listing.data.entries }}. Inside the loop body, add a Condition node to skip directories: continue only when {{ item.type }} equals file (and optionally only when {{ item.name }} ends with an extension you care about, such as .pdf). On the true branch, add a Connector node in Direct mode using FTP and the download-file tool. Set path to the full remote path:

/engineering/specs/{{ item.name }}

The download returns data.content (the file bytes as a string), data.encoding (utf8 or base64), and data.size. Name the output variable file so the next step can embed {{ file.data.content }}.

Step 4: Embed each document into the persistent collection

Still inside the loop body, after the download, add a Knowledge node in Embed mode. Configure it like this:

Collection: pick your persistent technical-specs collection (not Transient, since the library must outlive the run).
File Name: set it to {{ item.name }}. Reusing the original file name means a later run with an updated file overwrites the old version instead of creating a duplicate, which keeps the library clean and re-runs idempotent.
Document Type: choose the format that matches your files, for example PDF or Plain Text. If you ingest mixed formats, branch on the file extension and use a separate Embed node per type.
Document Input: {{ file.data.content }} from the download step.
Output Variable: name it embedResult. It carries the chunk count and metadata, handy for the summary.

Each embedded document persists in the collection until you delete it from the document table in the Knowledge section.

Step 5: Post an ingestion summary to Slack

After the loop, add a Connector node in Direct mode using your Slack connection and the send-message tool. Set the channel to your engineering channel (for example #eng-ops) and the text to a short report so the team knows the library refreshed:

Specs library refreshed at {{ scheduledAt }}. Indexed {{ listing.data.entries.length }} entries from /engineering/specs into the technical-specs collection.

This gives you a lightweight audit trail in chat without opening the run history every morning.

Step 6: Build the reusable query Subworkflow

Create a second workflow named Specs Library Query. Give it a Manual trigger so its request body becomes the input; callers will pass a question field. Add a Knowledge node in Query mode and configure it:

Collection: the same persistent technical-specs collection.
Prompt: {{ input.question }}, so the natural-language question flows straight through.
Model: an AI model for synthesis; this is where Miraxa, the intelligent layer across your automation, grounds its answer in the retrieved spec chunks.
Result Count: leave at the default of 5, or raise it for broad questions.
Response Schema: optional. Supply a small JSON schema (for example { "answer": "string", "sourceFile": "string" }) if callers need structured output instead of prose.
Output Variable: name it answer.

Because collections are workspace-scoped, this query can read everything the ingestion workflow embedded even though they are separate workflows.

Step 7: Call the library from any other workflow

In any workflow that needs spec data, add a Subworkflow node. Set Workflow to Specs Library Query and pass the question as the Input:

{ "question": "What is the rated load for bracket {{ order.partNumber }}?" }

The parent pauses, the child runs its Knowledge query, and its final output returns to the parent for downstream use as the subworkflow node's result. This is the payoff of the pattern: the lookup logic lives in exactly one place, and updating it (a different model, a larger result count) instantly applies to every caller. For more on this node, see the Subworkflow nodes guide.

Tips

Keep the same embedding model for embed and query of a collection. The model is fixed when you create the collection, so the ingestion and query workflows automatically stay consistent as long as they target the same collection.
Set the Schedule cron to off-peak hours. Embedding many documents takes time, and a 6am run finishes before the team starts asking questions.
Use the document table in the Knowledge section to spot files stuck in PROCESSING or to delete retired specs. Deletions are permanent, so prune deliberately.
Ask Miraxa to scaffold the loop for you with a prompt like "Add a Loop node over {{ listing.data.entries }}, and inside it an FTP download-file Connector node followed by a Knowledge Embed node into the technical-specs collection," then fine-tune the fields in the properties panel.

Common Pitfalls

Choosing Transient instead of a persistent collection in the Embed node. Transient collections are auto-cleaned at the end of the run, so the library would vanish before any other workflow could query it. Always pick the named persistent collection here.
Forgetting to filter directories in the loop. The list-directory output includes subfolders; without the Condition checking {{ item.type }} equals file, the download step will fail on directory entries.
Mismatched Document Type. Embedding a PDF while the node is set to Plain Text produces garbled chunks. Branch on file extension when your directory holds mixed formats.
Re-embedding unchanged files with a fresh file name each run. That creates duplicates and inflates query results. Always set File Name to the stable original name so overwrites keep the library tidy.

Testing

Before turning on the schedule, validate the pieces on a small scope. Point the FTP list-directory at a folder holding two or three spec files and run the ingestion workflow with the Run button, then check the document table in the Knowledge section to confirm the documents reach READY status. Next, open Specs Library Query, run it manually with a request body like { "question": "..." } tied to one of those documents, and confirm the answer output cites the right content. Only once both halves work end to end should you enable the Schedule trigger and wire the Subworkflow into your production workflows.