Building RAG with n8n: Company Knowledge as a Chatbot, Step by Step

Retrieval-Augmented Generation (RAG) connects a language model with your own company documents, so that a chatbot delivers answers based on real internal knowledge instead of relying only on the model's general training knowledge. In n8n, you build a RAG system in practice as two separate workflows: The first workflow loads documents, splits them into text sections, converts these into vectors via an embedding model, and stores them in a vector store node. The second workflow connects this vector store via an AI Agent node as a searchable tool, so that the chatbot looks up matching text passages itself for every request and incorporates them into its answer. As of: July 2026.

Which building blocks does a RAG workflow in n8n need?

According to the n8n documentation, a RAG workflow in n8n consists of four interacting components: Document Loader, Text Splitter, Embeddings, and Vector Store with Retriever. The Document Loader fetches the external source data, for example from a file, a webhook, or a database query. The Text Splitter breaks long documents into smaller sections, called chunks, so that the embedding model can process them meaningfully. Embeddings convert text into vectors, that is, numerical representations of meaning, whereby n8n, according to the documentation, exclusively supports text embeddings. Finally, the Retriever fetches the matching vectors from the database for a query and translates them back into usable text. Details on these components are described in the official documentation at Store and search data with vectors.

How do you build the first workflow to load documents into the vector store?

For ingestion, you add a vector store node with the "Insert Documents" operation, connect an embedding model, and add a Default Data Loader node for splitting into chunks.

• Chunk size: The n8n documentation recommends sections of 200 to 500 tokens for fine-grained search, supplemented with a certain overlap between the chunks, so that context is not lost at the boundaries.
• Splitter choice: Available options are the Character Text Splitter, which splits by character length, the Token Text Splitter, which splits by token count, and the Recursive Character Text Splitter, which n8n recommends for most use cases because it orients itself on Markdown, HTML, or code structures.
• Metadata: Optionally, chunks can be enriched with additional information such as source, document type, or date, which makes later filtering easier.
• Clear Store: An option in insert mode to clear the existing store before refilling it.

For quick testing, the Simple Vector Store (In-Memory) node is suitable, which holds vectors directly in n8n's working memory. According to the official node documentation this node is explicitly intended only for development.

How do you make the vector store searchable for an AI chatbot?

For retrieval, you connect the same vector store node in "Retrieve Documents (As Tool for AI Agent)" mode to an AI Agent node, which then triggers the search independently whenever it needs it for an answer. In this mode, you configure the Name and Description of the tool so that the agent understands when it should access it, as well as a limit for the number of returned chunks and optionally "Include Metadata" to supply source references. According to the documentation, it is important that you use the same embedding model for querying as for inserting the data, otherwise the vector spaces do not match and the search delivers poor results. Alternatively, there is direct querying via the "Get Many" operation, where you set a fixed search query yourself instead of leaving it to the agent. The exact process is described in Retrieve relevant context described. Anyone who does not want to maintain this setup themselves will find, in NordFlux's AI Agents service, an implementation with a fixed price and German data sovereignty.

Where are the limits of RAG with n8n?

RAG is not a miracle cure: The quality of the answers depends directly on the structure of the source documents, and the Simple Vector Store built into n8n is, according to the documentation, explicitly intended only for development, because data is lost on a restart and, according to the documentation, is visible instance-wide to all users, regardless of the individual workflow's permissions. For production use, you therefore need a persistent external vector store, for which n8n provides its own nodes, for example for PGVector or Azure AI Search. Poorly structured source documents, such as scanned PDFs without a real text layer or large, unstructured tables, lead to poorly cut chunks and thus to inaccurate or incomplete chatbot answers. Clean, well-structured source data is often more important for result quality than choosing a larger embedding model.

Frequently asked questions about RAG with n8n

What is the difference between the vector store as a tool and direct querying via Get Many?

With "Get Many" you give the vector store a fixed, hard-wired search query, while the "Retrieve Documents (As Tool for AI Agent)" mode leaves the decision to the agent. The agent itself checks, based on the Name and Description of the tool, whether and when a search in the vector store makes sense at all for the current user question. This makes the setup more flexible for a chatbot that should also answer general questions without document reference.

Which embedding model should you choose for a company knowledge chatbot?

According to the n8n documentation, smaller embedding models are faster and cheaper and are suitable for general documents, while larger models offer better semantic understanding for complex specialist topics. In any case, it is crucial that you use the same model when inserting the data and when querying it later, since different models produce different vector spaces.

Can you use n8n's Simple Vector Store in production?

No, according to the n8n documentation, the Simple Vector Store (In-Memory) node is explicitly intended only for development, because data is lost on a restart of the instance and, according to the documentation, is visible instance-wide to all users. For a company knowledge chatbot used in production, you need a persistent, externally hosted vector store.

How large should the text sections be for search?

For fine-grained search, the n8n documentation recommends chunk sizes between 200 and 500 tokens, combined with a certain overlap between adjacent sections, so that no context is lost at the chunk boundaries. As a splitter strategy, n8n suggests the Recursive Character Text Splitter for most use cases, because it orients itself on the natural structure of the document instead of splitting arbitrarily by character count.