Why Is My RAG Agent Not Answering From the Database? The 5 Most Common Causes (FAQ)

A RAG agent in n8n is supposed to draw answers exclusively from the connected knowledge database, but in practice it often does not: it ignores the vector store, invents content, or claims that a piece of information is not available even though it is in the index. The cause is almost never the language model itself, but the configuration between indexing and querying: mismatched embeddings, unfavorable chunking, a retriever filter that excludes too much or too little, or a system prompt that does not clearly mandate tool use. This article breaks down the five most common failure patterns from n8n community threads and the official documentation and names the practical fix for each. As of: July 2026.

Causes in Indexing and Querying

• Embeddings mismatch: If documents were processed with a different embedding model or a different vector dimension during indexing than the search query uses at query time, the similarity search no longer returns usable matches because the vector spaces do not correspond to each other. In n8n this typically happens when the insert workflow uses a different embeddings node than the retrieval workflow, for example OpenAI during indexing and a local Ollama model at query time. The fix: wire the same embeddings model firmly into both workflows and rebuild the entire index after every model change instead of merely appending to it.
• Chunking problems: Chunks that are too large mix several topics into one vector and dilute retrieval precision, while chunks that are too small tear context apart, so the answer stays semantically close but incomplete in content. A good starting point is usually 500 to 1,000 characters with noticeable overlap, and the Recursive Character Text Splitter in n8n orients itself on paragraphs and headings, which separates content more cleanly than a pure character boundary. The fix: vary chunk size and overlap experimentally and check retrieval quality against real user questions, not just sample texts.
• Missing or incorrect retriever filter: Metadata filters in the vector store node combine multiple conditions with AND by default, so a filter that is set too narrowly, for example a combination of category and date, excludes all documents even though matching content exists in the index. The n8n forum also reports that dynamic filters are less reliable in the agent's tool mode than in chain mode. The fix: first remove the filter logic completely to check basic retrieval, then gradually add filters back.

Causes in Prompt and Tool Connection

• Agent does not use the vector store tool: Especially smaller language models do not reliably call the retrieval tool if the system prompt does not explicitly mandate its use, and instead answer questions from their training knowledge. In the n8n community, an explicit system prompt has proven to be an effective fix: one that enforces tool use before every answer, allows answers only based on the tool results, and instructs the agent to openly state knowledge gaps instead of improvising.
• Tool response is discarded by the agent: Even when the retriever delivers correctly, the agent can ignore the answer if the tool output is heavily structured, for example with many paragraphs and line breaks from several documents at once. In such cases the model responds that the information is not available even though matching results exist. The fix: reduce the number of returned documents (top-K) and format the tool output in the retriever node more compactly so the agent clearly recognizes it as the basis for the answer within the context.

Anyone who wants such a pipeline not just repaired but set up cleanly from the ground up will find in the AI Agent Consulting by NordFlux an approach based on a fixed price and German data sovereignty as a core principle.

Frequently Asked Questions About RAG Agents in n8n

Why does my n8n agent ignore the vector store completely?

Usually a clear instruction in the system prompt to use the retrieval tool before every answer is missing. Smaller models rarely infer this behavior on their own and instead fall back on their general training knowledge, even when a suitable tool is connected.

How can I recognize an embeddings mismatch?

A typical sign is that the similarity search consistently returns low or arbitrary match scores even though the content being searched for is verifiably in the index. Check whether the insert and retrieval workflow use exactly the same embeddings node with the same model version, see the n8n documentation on vector stores.

How large should chunks be for RAG in n8n?

A good starting point is usually 500 to 1,000 characters with some overlap between chunks. However, the right size depends on the structure of the source documents and should be adjusted based on real test questions, not a fixed rule of thumb.

Do I need to reindex after changing the embeddings model?

Yes. Since different embedding models produce different vector spaces, old and new vectors cannot be meaningfully searched together. After a model change, the entire index should be rebuilt, as several threads in the n8n community confirm for similar failure patterns.