How to Use RAG to Answer Questions from Company Documents

Use Retrieval-Augmented Generation to get accurate answers from your business documents.

What This Integration Does

Retrieval-Augmented Generation (RAG) is the answer to AI's hallucination problem. Instead of letting a language model answer from training data of unknown provenance, you retrieve passages from documents you trust and force the model to ground its answer in those passages. The result is an AI that quotes your handbook, your contracts, and your runbooks - and politely says "I don't know" when the answer isn't there.

This workflow exposes a question-answering endpoint over a Knowledge collection of company documents. At run time, the Knowledge node retrieves the top-matching chunks for the user's question, an AI Agent composes a grounded answer with citations, and a Response node returns both the answer and the source list. The same pattern slots into Slack bots, internal admin tools, and downstream workflows that need to reason over your documents.

Prerequisites

A populated Knowledge collection - handbook, policies, runbooks, customer-success playbooks, whatever fits the use case.
A trigger surface: a Webhook Trigger for direct callers, or a Slack connection if you want a slash-command interface.
Access to the AI Agent.

Step 1: Trigger

Add a Trigger node. For a programmatic API, set the sub-type to Webhook and expect a JSON body with at least question and optionally scope (a tag filter, e.g. handbook or runbooks). For Slack, wire a slash command at the boundary and forward the text payload into a webhook trigger.

Step 2: Knowledge Node - Retrieve

Add a Knowledge node in query mode. Pass the user's question and any scope filter:

{
  "query": "{{ trigger.question }}",
  "topK": 6,
  "minScore": 0.7,
  "filters": {
    "scope": "{{ trigger.scope }}"
  }
}

The node returns matched chunks with their sourceId and similarity score. Save the response as retrieved.

Step 3: Condition - Bail Out If Retrieval Is Empty

Add a Condition node that checks retrieved.results.length > 0. On the false branch return a friendly "I don't have that in our documents" response via the Response node and stop. This is faster and cheaper than letting the AI Agent produce the same fallback, and it gives you a clean metric for content gaps.

Step 4: AI Agent - Compose a Grounded Answer

Add a Connector node calling the AI Agent. Pin it hard with the system prompt:

You are a grounded question-answering assistant. Answer ONLY using
the provided context. If the answer isn't in the context, reply:
"I don't have that information in our documents."

Include a "Sources" footer listing each cited document by name.
Quote sparingly and only when wording matters.

Context:
{{ retrieved.results }}

Question: {{ trigger.question }}

Set the temperature to 0.2 for stable answers. Pass retrieved.results as the only context source - do not include the question alone or you re-enable hallucination.

Step 5: Format Citations

Add a Transform node that combines the AI Agent's textual answer with a structured citation list pulled from retrieved.results. Each citation should include the sourceId, document title, and chunk score. Downstream consumers (Slack message, browser UI, audit log) can render the citations however they need.

Step 6: Respond and Log

For HTTP callers, return JSON via the Response node: { answer, citations, usage }. For Slack, wire the response through a slack send-message with the answer in the main body and citations as a formatted block. In parallel, write the question, the matched sourceId list, and the final answer into a small audit table (mongodb insert-documents) so you can review what's being asked and how well retrieval is performing.

Tips

Quote, don't paraphrase - tell the AI Agent to use exact wording for any factual claim. Paraphrased answers drift quickly under pressure.
Tag aggressively at index time - tags let you scope queries to a sub-corpus (handbook vs runbooks vs customer playbooks), which dramatically improves answer quality on focused questions.
Iterate on chunk size - if answers feel like they're missing context, your chunks may be too small. If retrieval matches feel diluted, they may be too large. Reflow and re-index until you find the sweet spot for your content.

Common Pitfalls

Letting the model "be helpful" - without an explicit refusal instruction, models invent answers when retrieval misses. Always include the "if not in context, say so" clause.
Stale index - RAG is only as good as the underlying documents. If your indexing workflow isn't re-running on document change, the AI will confidently quote last quarter's policy.
Mixing scopes - if your collection mixes customer-facing and internal-only content, a public-facing bot can leak the wrong material. Use tags and filter at query time.

Testing

Create a question set: five questions you can verify by hand against a single source document, and five questions that are deliberately not covered. Run all ten through the workflow. Confirm the first five return correct answers with the right citations, and the second five fall through the refusal path cleanly. Only then expose the endpoint to the wider team.

Learn More

Knowledge base documentation