RAG Pipeline (Detailed)

A production RAG pipeline has many more components than the basic "retrieve and generate" description suggests. Here is a complete pipeline: **Ingestion phase**: (1) Document loading — parse PDFs, Word docs, web pages, databases. (2) Chunking — split documents into pieces. Chunk size matters: too small loses context, too large dilutes relevance. Common: 500-1000 tokens with 100-200 token overlap. (3) Embedding — convert each chunk to a vector using an embedding model. (4) Indexing — store vectors in a vector database (Pinecone, pgvector, Weaviate). **Query phase**: (1) Query embedding — convert the user's question to a vector. (2) Retrieval — find the top-K most similar chunks (typically K=5-20). (3) Reranking — use a cross-encoder model to re-score retrieved chunks by relevance (dramatically improves quality). (4) Context assembly — combine the top chunks into a prompt with the original question. (5) Generation — LLM generates an answer grounded in the retrieved context. (6) Post-processing — add citations, validate claims, format output. Common pitfalls: wrong chunk size, no reranking (top retrieval results are often not the most relevant), ignoring metadata (filter by date, source, category before vector search), and not evaluating quality systematically.