How to build a RAG chatbot with n8n: vector store, embeddings, and tool integration explained step by step.
Retrieval-Augmented Generation (RAG) connects a language model with your own company documents, so that a chatbot delivers answers based on real internal knowledge instead of relying only on the model's general training knowledge. In n8n, you build a RAG system in practice as two separate workflows: The first workflow loads documents, splits them into text sections, converts these into vectors via an embedding model, and stores them in a vector store node. The second workflow connects this vector store via an AI Agent node as a searchable tool, so that the chatbot looks up matching text passages itself for every request and incorporates them into its answer. As of: July 2026.
According to the n8n documentation, a RAG workflow in n8n consists of four interacting components: Document Loader, Text Splitter, Embeddings, and Vector Store with Retriever. The Document Loader fetches the external source data, for example from a file, a webhook, or a database query. The Text Splitter breaks long documents into smaller sections, called chunks, so that the embedding model can process them meaningfully. Embeddings convert text into vectors, that is, numerical representations of meaning, whereby n8n, according to the documentation, exclusively supports text embeddings. Finally, the Retriever fetches the matching vectors from the database for a query and translates them back into usable text. Details on these components are described in the official documentation at Store and search data with vectors.
For ingestion, you add a vector store node with the "Insert Documents" operation, connect an embedding model, and add a Default Data Loader node for splitting into chunks.
For quick testing, the Simple Vector Store (In-Memory) node is suitable, which holds vectors directly in n8n's working memory. According to the official node documentation this node is explicitly intended only for development.
For retrieval, you connect the same vector store node in "Retrieve Documents (As Tool for AI Agent)" mode to an AI Agent node, which then triggers the search independently whenever it needs it for an answer. In this mode, you configure the Name and Description of the tool so that the agent understands when it should access it, as well as a limit for the number of returned chunks and optionally "Include Metadata" to supply source references. According to the documentation, it is important that you use the same embedding model for querying as for inserting the data, otherwise the vector spaces do not match and the search delivers poor results. Alternatively, there is direct querying via the "Get Many" operation, where you set a fixed search query yourself instead of leaving it to the agent. The exact process is described in Retrieve relevant context described. Anyone who does not want to maintain this setup themselves will find, in NordFlux's AI Agents service, an implementation with a fixed price and German data sovereignty.
RAG is not a miracle cure: The quality of the answers depends directly on the structure of the source documents, and the Simple Vector Store built into n8n is, according to the documentation, explicitly intended only for development, because data is lost on a restart and, according to the documentation, is visible instance-wide to all users, regardless of the individual workflow's permissions. For production use, you therefore need a persistent external vector store, for which n8n provides its own nodes, for example for PGVector or Azure AI Search. Poorly structured source documents, such as scanned PDFs without a real text layer or large, unstructured tables, lead to poorly cut chunks and thus to inaccurate or incomplete chatbot answers. Clean, well-structured source data is often more important for result quality than choosing a larger embedding model.
With "Get Many" you give the vector store a fixed, hard-wired search query, while the "Retrieve Documents (As Tool for AI Agent)" mode leaves the decision to the agent. The agent itself checks, based on the Name and Description of the tool, whether and when a search in the vector store makes sense at all for the current user question. This makes the setup more flexible for a chatbot that should also answer general questions without document reference.
According to the n8n documentation, smaller embedding models are faster and cheaper and are suitable for general documents, while larger models offer better semantic understanding for complex specialist topics. In any case, it is crucial that you use the same model when inserting the data and when querying it later, since different models produce different vector spaces.
No, according to the n8n documentation, the Simple Vector Store (In-Memory) node is explicitly intended only for development, because data is lost on a restart of the instance and, according to the documentation, is visible instance-wide to all users. For a company knowledge chatbot used in production, you need a persistent, externally hosted vector store.
For fine-grained search, the n8n documentation recommends chunk sizes between 200 and 500 tokens, combined with a certain overlap between adjacent sections, so that no context is lost at the chunk boundaries. As a splitter strategy, n8n suggests the Recursive Character Text Splitter for most use cases, because it orients itself on the natural structure of the document instead of splitting arbitrarily by character count.
Founder of NordFlux. Spent four years automating processes at enterprise scale at Dräger, and now brings that depth to the mid-market — pragmatic and with full data sovereignty.
Certifications
Why RAG chatbots in n8n take 16 seconds or more, and how vector store choice, chunk size, model choice, and caching reduce response time.
The 5 most common reasons why RAG agents in n8n ignore the vector store: embeddings, chunking, filters, prompt, and tool output at a glance.
Which components a production-ready n8n automation stack for SMEs needs: database, queue mode, vector store, and monitoring at a glance.
Setting up the vector store, embedding model, and chunking strategy correctly is only half the work; the other half is ongoing operation: current documents, stable answer quality, and a chatbot that doesn't buckle as more users join. NordFlux builds RAG chatbots on n8n, from architecture through to production operation, so your company knowledge stays reliably searchable. In an initial conversation, we look at your use case and show you where the limits of RAG apply in your situation.
