RAG Chatbot Too Slow? Optimizing Response Times (Thread: 16s+)

When a RAG chatbot in n8n takes 16 seconds or more to answer simple questions, this is rarely due to the language model alone, but usually to the combination of vector store choice, chunk size, model selection, and missing caching. In a widely discussed thread in the n8n forum a user reported response times of 16 to 18 seconds for a production RAG chatbot with an AI Agent node, while a leaner HTTP/code pipeline with the same data achieved 3 to 5 seconds. The following levers can usually be adjusted in existing n8n workflows without a complete rebuild. As of: July 2026.

Vector store choice: not every database is built for speed

n8n natively supports several vector stores: Pinecone, Qdrant, Supabase Vector Store, and PGVector as a Postgres extension, plus an in-memory store for prototypes. For response time, what matters is less the provider's name than the combination of indexing method, network latency to the database, and filtering capability during similarity search. In a documented community case with PGVector high response times and bloated token usage led users to switch on a trial basis to a dedicated vector store like Qdrant, because it is built for pure vector search with filtering options, while PGVector as an extension of a relational database brings additional overhead. Anyone who wants to keep PGVector for data sovereignty reasons should check whether the Postgres instance is located close to the n8n server, since part of the latency arises from network round trips between the agent node and the database.

Chunk size and overlap: smaller is often faster

Chunk size determines how much text per section goes into the vector store and how much context is sent to the language model with each request. n8n uses the Recursive Character Text Splitter with a default chunk size of 1000 characters and 200 characters of overlap. In the same PGVector case, smaller values resolved noticeable problems: the user reduced the chunk size from 1200 to 512 characters, the overlap from 200 to 100, and the insert batch size from 200 to 32, which noticeably reduced token usage and response time. The effect is plausible: smaller chunks mean less irrelevant text per match, resulting in shorter prompts and less processing time for the language model. The downside is fragmentation, related information can end up split across multiple chunks. A good starting point for many knowledge bases lies between 500 and 800 characters with 10 to 20 percent overlap, adjusted to the text structure of the source documents.

Model choice and agent architecture: where the seconds really come from

The actual cause of the 16 seconds in the referenced thread was less the model choice than the architecture: the AI Agent node typically performs two to four internal language model calls per request, including for tool selection and reasoning loops, before the actual answer is generated. A leaner pipeline made up of embedding, vector search, prompt construction, and a single language model call achieved 3 to 5 seconds in the same thread, because it skips exactly these intermediate steps. For simple retrieve-and-answer scenarios without real tool selection, the full agent mode is therefore often overkill. In addition, model choice directly affects response time: smaller, faster models generally deliver lower latency than larger models, though at the cost of answer quality on more complex questions. Anyone who genuinely needs full agent flexibility, for example because the chatbot must choose between multiple tools and data sources, can hardly avoid this overhead. For all other cases, it is worth switching to a deterministic retrieval chain.

Caching: not recalculating repeated requests every time

Caching was not explicitly discussed in the referenced forum thread, but it is one of the obvious levers once a RAG chatbot is running in production. The documents are embedded and written to the vector store only once, with each new request only the user's question needs to be re-vectorized, the embedding of the knowledge base itself should not be recalculated on every chat call. Recurring or very similar questions can additionally be intercepted with a simple caching step in the workflow, for example by storing the question and answer and returning them directly on repetition, without running through vector search and the language model again. This is especially worthwhile for FAQ-heavy use cases with recurring phrasing, for individual support requests the effect is smaller. Anyone wanting to systematically reduce latency should tackle caching only after the first three levers, since it does not fix structural problems in chunking or architecture, but only reduces how often they are run.

Anyone who does not want to work through these levers alone in an existing n8n workflow, or wants a chatbot with a clear architecture set up cleanly from the start, will find in NordFlux's AI agent development a point of contact with a fixed price and German data sovereignty. For n8n automation regardless of the chatbot topic, the n8n consulting is the right entry point.

Frequently asked questions about slow RAG chatbots in n8n

Where does the value of 16 seconds come from?

From a thread in the n8n forum, in which a user documented response times of 16 to 18 seconds for a production RAG chatbot with an AI Agent node and vector store connection, while a simpler pipeline with the same data achieved 3 to 5 seconds.

Is the AI Agent node fundamentally too slow for RAG?

No. It is built for scenarios with real tool selection and multi-step reasoning, and rightly makes its calls there. In a simple retrieve-and-answer flow without these requirements, however, it creates overhead that can be avoided with a leaner chain.

What chunk size makes sense for most knowledge bases?

A range of 500 to 800 characters with 10 to 20 percent overlap is a workable starting point for many use cases, but should be tested against your own document structure. Chunks that are too small can tear apart context, chunks that are too large bloat prompts and response time.

Is switching vector stores worth it for speed alone?

Not as a first step. First it is worth checking whether network latency, chunk size, or agent architecture is the actual cause. Switching vector stores is more effort than a configuration adjustment and should only follow once the other levers have been exhausted.