Haystack 2.0: English for Building RAG Pipelines

Haystack 2.0 by deepset reimagined the original framework as a composable, component-based architecture for building production RAG systems. If you are working with Haystack in a multinational team or following the English-language documentation and community forums, you will need a firm grasp of the framework’s specific vocabulary. This guide covers the key terms you will encounter when designing, building, and debugging Haystack pipelines.

Key Vocabulary

Pipeline — the central abstraction in Haystack 2.0, a directed acyclic graph of connected components that data flows through from input to output. Definition sentence: Rather than a linear sequence of steps, a Haystack Pipeline can branch, merge, and conditionally route data depending on intermediate results. Example: “We built a separate indexing pipeline and a querying pipeline, connecting them through a shared document store.”

Component — a self-contained, reusable processing unit with declared inputs and outputs that can be connected to other components within a pipeline. Definition sentence: Any Python class decorated with @component that exposes a run method becomes a valid Haystack component. Example: “We wrote a custom component to strip personally identifiable information from documents before they enter the document store.”

DocumentStore — the storage backend responsible for persisting documents and their vector embeddings, supporting retrieval queries against them. Definition sentence: Haystack supports multiple DocumentStore backends — including OpenSearch, Weaviate, Qdrant, and an in-memory store for local development. Example: “The QdrantDocumentStore was chosen because it supports named vectors, which we needed for our hybrid retrieval strategy.”

Retriever — a component that queries the DocumentStore to return the most relevant documents for a given query, using dense vector search, sparse keyword search, or a combination of both. Definition sentence: The retriever sits between the query input and the generator, narrowing a large document collection down to the most relevant context. Example: “We replaced the dense retriever with a hybrid retriever after noticing that exact product-code queries were performing poorly with embeddings alone.”

Generator — a component that wraps a language model and produces text output, typically receiving retrieved documents as context alongside the user’s question. Definition sentence: Haystack provides built-in generators for OpenAI, Anthropic, Cohere, and locally hosted models, all following the same interface. Example: “We swapped in a local HuggingFaceLocalGenerator for our on-premises deployment without changing any other part of the pipeline.”

Ranker — a component positioned after the retriever that reorders retrieved documents by relevance using a cross-encoder model, typically more accurate than the initial retrieval but more expensive to run. Definition sentence: Adding a ranker improves the quality of the context window passed to the generator, especially when the retriever returns a large candidate set. Example: “The ranker reduced the top-20 retrieval results down to the top-5 before they were passed to the generator, cutting prompt length and improving answer quality.”

Embedding model — a model that converts text into dense numerical vectors, enabling semantic similarity search in the document store. Definition sentence: The same embedding model must be used during indexing and at query time, otherwise the vector representations are not comparable. Example: “We benchmarked three embedding models on our domain-specific test set before settling on one that balanced accuracy and inference speed.”

Hybrid retrieval — a retrieval strategy that combines dense vector search with sparse keyword-based search (such as BM25), then merges and re-ranks the results to capture both semantic and lexical relevance. Definition sentence: Hybrid retrieval is particularly valuable for technical domains where exact terminology matters as much as conceptual similarity. Example: “Hybrid retrieval improved our recall on queries containing specific API method names, which the dense embeddings had been handling poorly.”

Useful Phrases

• “The indexing pipeline runs as a batch job every night; the querying pipeline is served as a FastAPI endpoint that handles live user requests.”
• “We wired the retriever’s output directly into the ranker’s input, then passed the ranker’s output to the prompt builder before the generator.”
• “If you change the embedding model, you need to re-index all your documents — the existing vectors in the store will be incompatible.”
• “We’re using the in-memory document store for local development and integration tests, then swapping in Qdrant for staging and production.”
• “The custom component implements the @component decorator and exposes a run method that returns a typed dictionary — Haystack validates the output schema at pipeline build time.”

Common Mistakes

Confusing “retriever” and “ranker”

Both components deal with selecting relevant documents, but they operate at different stages and with different mechanisms. A retriever performs a fast approximate search over the entire document store. A ranker performs a slower, more precise reordering of a small candidate set already returned by the retriever. Saying “the ranker searches the database” is incorrect — say “the ranker reorders the candidates returned by the retriever.”

Using “pipeline” loosely to mean “any process”

In everyday English, pipeline is used metaphorically for any sequence of steps. In Haystack, Pipeline has a specific technical meaning: it is an instantiated object with a graph structure, validated connections, and a run method. When discussing Haystack specifically, reserve the word pipeline for this technical object rather than using it as a general metaphor, to avoid confusion in team discussions.

Misusing “connect” versus “wire up”

Both phrases are used when linking components in Haystack, but connect is the more neutral, formal term (and the name of the actual method: pipeline.connect(...)). Wire up is idiomatic and informal, common in spoken conversation and informal documentation. In written technical specifications, prefer connect; in conversation and code-review chats, both are natural.

Haystack 2.0’s composable architecture reflects a broader trend in the industry towards modular, observable AI systems. Mastering this vocabulary not only helps you use the framework more effectively — it also equips you to describe your system’s design clearly to stakeholders, to write accurate technical documentation, and to contribute meaningfully to the open-source community around RAG development.

Navigating Feedback Loops – A Practical Approach

Let’s face it: technical communication in a professional setting can be fraught with awkwardness, especially when dealing with feedback. As developers, we’re constantly receiving critiques, proposing changes, and justifying our decisions. Understanding the nuances of requesting and responding to this kind of feedback—particularly in the context of complex systems like Haystack 2.0 – is crucial for effective collaboration and ultimately, successful project delivery. A key element often missed when learning technical vocabulary is how that vocabulary manifests in real-world communication scenarios. It’s not enough to know what a “retriever” is; you need to understand how to discuss it with your team, request refinements, or explain its role during a code review.

Consider this Slack conversation:

Alice: Hey @Bob, I’ve been working on improving the document retrieval in the RAG pipeline.  I'm experimenting with different embedding models and refining the query expansion – hoping to boost recall for our knowledge base articles.  Any initial thoughts?
Bob: Sounds good! Just make sure you’re logging enough detail about the embeddings used so we can track performance later. Also, how are you handling potential synonym issues in the query expansion? That's a common area where retrieval suffers.

Notice Bob’s use of phrases like “logging enough detail” and “handling potential synonym issues.” These aren’t just technical terms; they represent specific requests for clarification or action. A developer new to this vocabulary might simply say, “I’m using different embeddings,” without specifying how that information will be used or acknowledging the broader context of performance monitoring. Similarly, Bob isn’t just stating a concern—he’s prompting Alice to consider a critical aspect of the system. The goal is not merely to define terms but to understand how they drive interactions and shape the flow of communication around complex technical tasks.

Furthermore, when writing PR descriptions or participating in code reviews, precise language is paramount. Instead of saying “I updated the retriever,” you might write: “Implemented a new sentence transformer embedding model (allennlp/sentence-transformers) to improve semantic similarity matching within the retriever component, resulting in a 15% increase in precision during initial testing.” This level of detail demonstrates understanding and allows reviewers to assess the impact of your changes effectively.

Here’s an example demonstrating how you might use Haystack’s DocumentStore API when requesting a specific change:

from haystack import DocumentStore
import json

store = DocumentStore(model="all-MiniLM-L6-v2")  # Example embedding model

documents = [
    {"mid": "doc1", "text": "The quick brown fox jumps over the lazy dog."},
    {"mid": "doc2", "text": "This is another document about foxes and dogs."}
]

store.delete_all() # Clear existing data for testing
store.add(documents=documents)

# Requesting a change to how documents are indexed:
print(json.dumps({"action":"update_index", "mid":"doc1", "new_embedding": "some_updated_embedding" }))

This simple example illustrates the kind of precise language needed when discussing data manipulation and model updates within a RAG pipeline – vocabulary that moves beyond the basic definitions and into practical application. Remember, effective technical communication is about more than just knowing what something is; it’s about conveying how you’re using it and ensuring everyone understands its implications.