How to Build a Production RAG Pipeline in Python: Step-by-Step with LangChain & Pinecone

A production RAG pipeline turns documents into searchable evidence, retrieves the passages relevant to a question, and gives those passages to an LLM with instructions to answer from them. The reliable version includes metadata, access checks, evaluation, logging, and failure handling from the first design.

Map the production RAG pipeline before writing code

A RAG pipeline has four connected stages: ingest source files, split and enrich the text, create and store embeddings, then retrieve and generate an answer. The request path also needs a policy layer. It should decide which user can search which documents before the retrieved context reaches the model.

Keep the original document ID, source URL, title, date, tenant, and access labels with every chunk. Those fields make filtering possible and let the answer show useful citations. A vector score alone is not a permission check, and semantic similarity does not tell the application if a document is current.

For a Python team, LangChain can coordinate loaders, splitters, embedding models, retrievers, and model calls. Pinecone's current RAG tutorial follows the same broad flow, using document chunks, embeddings, a vector index, and an LLM-backed answer path in one working example.

Teams that need help connecting the application layer to retrieval can review AI agent development as a related implementation path. A production pipeline is often part of a larger assistant, not a standalone search box.

Set up a reproducible Python environment

Pin the libraries and model names used by the first evaluation. Package updates can change retriever behavior, response formats, or integration imports. Keep secrets in environment variables, and use separate indexes or namespaces for development and production data.

pip install langchain langchain-openai langchain-pinecone pinecone python-dotenv tiktoken

Use a configuration object for the embedding model, chat model, index name, namespace, chunk policy, and retrieval count. Record that configuration with each evaluation run. The question is not simply if the pipeline answers one prompt. It is if the same version can be inspected and compared after a data or dependency change.

Load, clean, and split documents

Start with the source formats your application actually owns. Extract text from PDFs, HTML, office files, or database records while preserving headings, tables, page references, and source links when they carry meaning. Remove navigation and repeated footer text before embedding it.

Chunking should keep enough context for a passage to stand alone. A fixed character count is a starting point, not a rule. Split at headings and paragraph boundaries where possible, then add a modest overlap when a definition or procedure crosses a boundary. Test several chunk sizes against real questions instead of adopting a number from a tutorial.

Attach metadata during ingestion. Useful fields include document ID, section heading, source date, product area, customer or tenant, and access group. Store a content hash so a changed source can update the right records without duplicating old chunks.

Create embeddings and index the chunks

An embedding model maps each chunk to a vector for similarity search. The vector database stores that representation alongside the chunk text and metadata. Use the same embedding model and dimensions for index creation and query encoding, and plan how a model change will create a new index or namespace.

from langchain_openai import OpenAIEmbeddings; from langchain_pinecone import PineconeVectorStore; embeddings = OpenAIEmbeddings(model='text-embedding-3-large'); store = PineconeVectorStore.from_documents(documents=chunks, embedding=embeddings, index_name='rag-production')

Pinecone's RAG chatbot tutorial shows the document-to-chunk-to-embedding flow and explains how the vector index supplies context for private-data questions. Use its current integration examples as a reference, then check the versions selected for your own project.

Retrieve context and generate an answer

At query time, normalize the user request only as much as the meaning permits. Apply authorization filters, retrieve more candidates than you plan to show, and consider a reranking step when the first similarity results contain near matches. The final prompt should tell the model to answer from the supplied context and to say when the context does not support an answer.

retriever = store.as_retriever(search_kwargs={'k': 5}); results = retriever.invoke(user_question); context = ' '.join(doc.page_content for doc in results); answer = llm.invoke(prompt.format(context=context, question=user_question))

Return source references with the answer. A useful citation identifies the document and the relevant section, not just the fact that a vector search ran. If no passage meets the relevance threshold, a safe no-answer path is more valuable than a confident guess.

Production checks that belong in the first release

• Retrieval tests: keep questions with known supporting documents and measure recall at the chosen retrieval count.
• Answer tests: score faithfulness, completeness, citation correctness, and refusal behavior when evidence is absent.
• Access tests: verify that users cannot retrieve chunks from another tenant, department, or permission group.
• Freshness tests: change a source document and confirm the old chunk is replaced, expired, or clearly marked.
• Operational tests: set timeouts and retries for vector, embedding, and LLM calls, then verify behavior during partial outages.

Log the question, retrieval configuration, document IDs, scores, model version, latency, and answer outcome without storing sensitive content unnecessarily. These records let the team distinguish a bad source, a retrieval miss, a prompt problem, and a model failure.

Fix failures in the right layer

If the correct document never appears, inspect extraction, chunking, metadata, embedding choice, filters, and query rewriting. If the correct passage appears but the answer ignores it, inspect prompt instructions, context ordering, token limits, and answer evaluation. Adding more chunks is not a universal fix. It can add noise and raise inference cost.

Deploy a small golden set before opening the pipeline to production traffic. Re-run it after changing the embedding model, chunk policy, retriever, prompt, or source corpus. A production RAG pipeline becomes dependable through measured iteration, clear provenance, and explicit access boundaries.

For teams that need a broader implementation partner, an AI developer for hire can help connect the retrieval service to application logic and deployment controls.