Evaluating Retrieval-Augmented Generation Systems
and Fine-Tuned Large Language Models

1.1What is Retrieval-Augmented Generation?

Retrieval-Augmented Generation (RAG) denotes an architectural pattern that extends the static knowledge base of a Large Language Model (LLM) with dynamic access to external document collections. Formally, the process is described as a three-stage pipeline:

Here $q$ denotes the query, $\mathcal{D}$ the document collection, $\mathcal{R}$ the retriever (e.g., a dense bi-encoder with cosine similarity), and $\mathcal{G}$ the generator (the LLM). The composition $\mathcal{G} \circ \mathcal{R}$ unifies parametric knowledge (in the model weights) with non-parametric knowledge (in the index) — a concept discussed in the literature as hybrid memory (Lewis et al., 2020).

1.2RAG vs. Fine-Tuning: A conceptual comparison

Retrieval metrics evaluate the quality of the component $\mathcal{R}$. They differ in whether they (a) consider pure set relations (precision, recall), (b) account for position (MRR), or (c) incorporate graded relevance (NDCG).

2.1Precision@K and Recall@K

Here $\mathcal{R}_K$ is the set of top-$K$ retrieved documents and $\mathcal{V}$ is the set of all relevant documents in the corpus (ground truth).

2.2Mean Reciprocal Rank (MRR)

$\operatorname{rank}_i$ denotes the position of the first relevant document for query $q_i$. MRR is especially meaningful when the user typically reads only the first result (search UIs, FAQ bots).

2.3Normalized Discounted Cumulative Gain (NDCG)

NDCG is the only one of the metrics presented here that admits graded relevance (e.g., $\operatorname{rel}_i \in \{0,1,2,3\}$): "not relevant", "marginal", "relevant", "highly relevant". The logarithm in the denominator implements position discounting: later hits contribute less to the overall score.

2.4Confusion matrix & precision-recall curve

A complete diagnosis of retrieval behavior requires examining the full trade-off curve, not just a single point:

The RAGAS framework is reference-free: it requires no human-annotated gold answers, instead using a "judge LLM" to operationalize the evaluation criteria. The four main metrics systematically cover both pipeline components:

3.1Faithfulness

Here $S(\hat y)$ decomposes the generated answer into atomic claims, each individually checked against the context $c$. A claim is considered "supported" if it is logically derivable from $c$.

3.2Answer Relevance

From the generated answer $\hat y$, $n$ synthetic questions $\tilde q_j$ are back-generated. The cosine similarity of their embeddings to the original query $q$ measures whether the answer addresses the right question.

3.3Context Recall

The fraction of atomic claims from the gold answer $y^*$ that is covered by the retrieved context $c$. Low CR → retriever delivers incomplete context → the generator cannot deliver a complete answer even with perfect faithfulness.

Classical text-generation metrics evaluate how well a generated hypothesis $\hat y$ matches one (or more) reference answer(s) $y^*$. They differ primarily in granularity (n-grams vs. tokens) and in representation (lexical vs. semantic).

4.1BLEU (Bilingual Evaluation Understudy)

BLEU geometrically averages n-gram precisions $p_n$ and multiplies by a brevity penalty (BP) that penalizes overly short hypotheses. Standard weights: $w_n = 1/N$ with $N=4$.

4.2ROUGE (Recall-Oriented Understudy for Gisting Evaluation)

ROUGE-N is recall-oriented (the denominator is based on the reference, not the hypothesis). ROUGE-L uses the longest common subsequence (LCS) and is robust to word-order changes.

4.3BERTScore

Instead of counting exact token matches, BERTScore computes the maximum cosine similarity of contextualized embeddings (e.g., from BERT or RoBERTa). This solves the fundamental problem of lexical metrics: semantic equivalence under lexical variation.

A single metric value on a test set is a point estimate with sampling variance. Without quantifying this uncertainty, statements like "system A is better than system B" are scientifically unfounded. This chapter introduces three canonical tools: confidence intervals, the bootstrap, and inter-annotator agreement.

5.1Confidence intervals for metrics

For a mean $\bar X$ over $n$ queries with sample standard deviation $s$, the central limit theorem gives approximately:

5.2Bootstrap resampling

For non-normal metrics (e.g., NDCG, BLEU), the non-parametric bootstrap procedure (Efron, 1979) is preferable. From the test set, $B$ (typically 1000) random samples are drawn with replacement; quantiles are then read directly from the distribution of the $B$ metric values.

5.3Inter-annotator agreement (Cohen's $\kappa$)

When relevance is annotated by humans, the reliability of the annotation itself must be checked. Cohen's kappa corrects observed agreement by the agreement expected by chance:

The following live demo shows the fundamental behavior of BLEU, ROUGE, and BERTScore on five classes of text variation. Choose a variation to see how the three metrics react in qualitatively different ways.

The choice of the "right" metric depends on application context. The table below offers pragmatic recommendations for typical engineering scenarios:

7.1Anti-patterns: common evaluation mistakes

The reference implementations below show the core logic. For production systems, use of established libraries is recommended (RAGAS, evaluate, bert_score, scikit-learn).

8.1Custom implementation of retrieval metrics

import numpy as np
from typing import List, Sequence

def precision_at_k(retrieved: Sequence[str], relevant: set, k: int) -> float:
    """Fraction of top-K hits that are relevant."""
    top_k = retrieved[:k]
    return len([d for d in top_k if d in relevant]) / k

def recall_at_k(retrieved: Sequence[str], relevant: set, k: int) -> float:
    """Fraction of all relevant items that appear in the top-K."""
    top_k = retrieved[:k]
    return len([d for d in top_k if d in relevant]) / len(relevant)

def reciprocal_rank(retrieved: Sequence[str], relevant: set) -> float:
    """1/rank of the first relevant hit (0 if none)."""
    for i, doc in enumerate(retrieved, 1):
        if doc in relevant:
            return 1.0 / i
    return 0.0

def mean_reciprocal_rank(all_retrieved, all_relevant) -> float:
    return float(np.mean([
        reciprocal_rank(r, rel) for r, rel in zip(all_retrieved, all_relevant)
    ]))

def ndcg_at_k(relevance_grades: Sequence[float], k: int) -> float:
    """NDCG with graded relevance."""
    def dcg(scores):
        return sum((2**s - 1) / np.log2(i + 2) for i, s in enumerate(scores[:k]))
    actual = dcg(relevance_grades)
    ideal  = dcg(sorted(relevance_grades, reverse=True))
    return actual / ideal if ideal > 0 else 0.0

8.2Bootstrap confidence interval

def bootstrap_ci(scores: np.ndarray, n_boot: int = 1000,
                 alpha: float = 0.05, seed: int = 42) -> tuple:
    """Non-parametric 95% CI via bootstrap resampling."""
    rng = np.random.default_rng(seed)
    n = len(scores)
    boot_means = np.empty(n_boot)
    for b in range(n_boot):
        idx = rng.integers(0, n, size=n)
        boot_means[b] = scores[idx].mean()
    lo = np.quantile(boot_means, alpha / 2)
    hi = np.quantile(boot_means, 1 - alpha / 2)
    return float(scores.mean()), float(lo), float(hi)

8.3RAGAS, BLEU, ROUGE, BERTScore (libraries)

# RAGAS
from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_recall, context_precision

results = evaluate(
    dataset=eval_ds,
    metrics=[faithfulness, answer_relevancy, context_recall, context_precision],
)

# BLEU / ROUGE
from evaluate import load
bleu  = load("bleu")
rouge = load("rouge")
bleu_score  = bleu.compute(predictions=hyps,  references=[[r] for r in refs])
rouge_score = rouge.compute(predictions=hyps, references=refs)

# BERTScore (English)
from bert_score import score
P, R, F1 = score(hyps, refs, lang="en", model_type="roberta-large")

Six multiple-choice questions for self-assessment. Click an answer to receive immediate feedback.

11.1Metric overview