Hallucination Mitigation Techniques

#hallucination mitigation #model reliability #data quality #adversarial training #uncertainty estimation #ai safety #trust in ai #model architecture #regularization #robustness

1. Definition and Types of Hallucination

Definition and Types of Hallucination

In generative AI systems, hallucination refers to the phenomenon where models generate outputs that are factually incorrect, nonsensical, or entirely fabricated, despite appearing coherent. Unlike human hallucinations, which are perceptual distortions, AI hallucinations stem from limitations in training data, model architecture, or inference mechanisms. These errors manifest differently depending on the model type and application domain.

Formal Definition

Let a generative model M produce an output y given input x. A hallucination occurs when:

$$ P(y|x) > \tau \quad \text{but} \quad \text{FactCheck}(y) = \text{False} $$

where τ is the model's confidence threshold and FactCheck is a ground-truth verification function. The divergence arises because models optimize for P(y|x) rather than factual accuracy.

Taxonomy of Hallucinations

1. Factual Hallucinations

Occur when models generate plausible but factually incorrect statements, particularly in knowledge-intensive tasks. For example, a language model might claim "The Eiffel Tower is located in Berlin" with high confidence. These often stem from:

Knowledge cutoff limitations in training data
Over-reliance on statistical patterns without semantic understanding
Confidence calibration failures in the probability distribution

2. Contextual Hallucinations

Arise when generated content contradicts established context, common in dialogue systems and summarization. A model might:

Invent fictitious references in academic writing
Introduce characters not present in source text for summarization
Generate contradictory statements within a single response

3. Input-Output Divergence

Occurs when outputs bear no logical relationship to inputs, frequently observed in:

Image generation (e.g., creating anatomically impossible structures)
Machine translation (inserting untranslated phrases)
Code generation (producing syntactically valid but semantically incorrect code)

4. Creative Hallucinations

Intentional deviations from source material that may be desirable in artistic applications but problematic elsewhere. Characterized by:

Plausible extrapolations beyond training data distribution
Novel combinations of learned concepts without grounding
Emergent metaphorical or symbolic representations

Quantitative Characterization

The hallucination rate H for a model can be formalized as:

$$ H = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}(\text{FactCheck}(y_i) = \text{False}) $$

where N is the number of samples and 𝕀 is the indicator function. Advanced variants incorporate severity weighting:

$$ H_w = \frac{\sum_{i=1}^{N} w_i \cdot \mathbb{I}(\text{FactCheck}(y_i) = \text{False})}{\sum_{i=1}^{N} w_i} $$

with weights w_i reflecting the consequence magnitude of each hallucination.

1.2 Causes and Triggers of Hallucination

Hallucinations in large language models (LLMs) emerge from complex interactions between model architecture, training data, and inference dynamics. Understanding these root causes is critical for developing effective mitigation strategies.

Architectural Limitations

The autoregressive nature of transformer-based models creates an inherent tendency for hallucination. At each decoding step, the model predicts the next token based on previously generated tokens, propagating any errors forward. This Markovian dependency can be formalized as:

$$ P(x_t|x_{

where h_t represents the hidden state at position t, and errors in h_t compound exponentially through the sequence. The attention mechanism's tendency to over-rely on local patterns rather than global consistency further exacerbates this issue.

Training Data Biases

Three key data-related factors contribute to hallucinations:

Factual inconsistencies in training corpora teach models that conflicting information can be equally valid
Over-representation of speculative content (e.g., academic papers with unverified hypotheses) creates false confidence in uncertain statements
Missing negation examples lead to poor calibration of model confidence scores

Quantitatively, this manifests as miscalibrated logits where the model assigns:

$$ \text{logit}(y_{\text{false}}) - \text{logit}(y_{\text{true}}) < \epsilon $$

even for clearly verifiable facts, where ε represents the decision boundary threshold.

Decoding Strategy Effects

Common text generation approaches introduce distinct hallucination patterns:

Strategy	Hallucination Mechanism	Empirical Rate
Greedy Decoding	Error accumulation from local optima	23-28%
Beam Search	Premature commitment to incorrect paths	18-22%
Nucleus Sampling	Over-exploration of low-probability tails	31-37%

The temperature parameter τ in sampling-based methods controls this behavior through:

$$ P_{\tau}(x) = \frac{\exp(\text{logit}(x)/\tau)}{\sum_{x'\in V} \exp(\text{logit}(x')/\tau)} $$

where high values of τ (>1.0) disproportionately amplify hallucination likelihood.

Knowledge Cutoff Effects

Static training cutoffs create temporal hallucinations where models generate outdated information with high confidence. The recency gap ΔT between training data and query time correlates with hallucination frequency as:

$$ H_{\text{temp}} \propto \frac{1}{1 + e^{-k\Delta T}} $$

where k is a domain-dependent decay constant (typically 0.3-0.7 for general knowledge).

Prompt-Induced Hallucinations

Certain query structures systematically increase hallucination risk:

Compound questions with multiple sub-queries (87% increase in hallucination rate)
Leading presuppositions containing false premises (62% increase)
Overly specific requests beyond model capability (3.2× baseline rate)

The syntactic complexity C of a prompt shows logarithmic correlation with hallucination probability:

$$ P_H = \alpha \ln(1 + \beta C) $$

where α and β are model-specific coefficients measurable through controlled probing.

Causes and Triggers of Hallucination – Hallucination Mitigation Techniques – Tutorial Diagram

Diagram Description: The diagram would show the error propagation mechanism in autoregressive decoding and how attention weights affect hallucination probability across tokens.

Impact on Model Reliability and Trust

Hallucinations in large language models (LLMs) directly undermine their reliability by generating outputs that are factually incorrect, misleading, or entirely fabricated. The presence of hallucinations erodes user trust, particularly in high-stakes applications such as medical diagnosis, legal advice, or financial forecasting. A model that frequently hallucinates cannot be deployed in critical environments without rigorous safeguards, as the consequences of incorrect outputs range from reputational damage to legal liability.

Quantifying Reliability Degradation

The reliability of an LLM can be quantified using metrics such as hallucination rate (H), defined as the proportion of generated outputs containing at least one hallucinated statement. For a dataset of N queries, the hallucination rate is computed as:

$$ H = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}(y_i \text{ contains hallucination}) $$

where 𝕀 is the indicator function and y_i is the model's response to the i-th query. A high H indicates poor reliability, necessitating mitigation strategies such as retrieval-augmented generation or confidence calibration.

Trust Erosion Mechanisms

Trust in LLMs is multidimensional, encompassing:

Competence trust – belief in the model's ability to perform tasks accurately.
Integrity trust – confidence that the model will not deliberately deceive.
Benevolence trust – expectation that the model acts in the user's best interest.

Hallucinations degrade all three dimensions. For instance, a single high-profile hallucination in a medical LLM can lead to widespread distrust, even if the majority of outputs are correct. This is exacerbated by the availability heuristic, where users disproportionately weigh memorable failures over consistent performance.

Case Study: Hallucinations in Clinical Decision Support

In a 2023 study, an LLM deployed for clinical note generation exhibited a hallucination rate of 12% for medication recommendations. Physicians, upon detecting these errors, reported a 40% reduction in trust, even after the model was fine-tuned to reduce hallucinations. This underscores the asymmetry of trust: once broken, trust is difficult to restore, requiring not only technical fixes but also transparent communication about mitigation efforts.

Mitigation Strategies to Preserve Trust

To maintain trust, models must incorporate:

Uncertainty quantification – explicitly signaling low-confidence outputs.
Provenance tracking – attributing generated facts to verifiable sources.
User feedback loops – allowing corrections to improve future outputs.

For example, a model might append confidence scores to each claim:

$$ \text{Confidence}(c) = \sigma \left( \frac{1}{K} \sum_{k=1}^{K} \log p(c | x, z_k) \right) $$

where σ is the sigmoid function, x is the input, and z_k are sampled latent variables. This transparency allows users to weigh the model's outputs appropriately.

2. Data Quality and Preprocessing Strategies

2.1 Data Quality and Preprocessing Strategies

Noise Reduction and Outlier Detection

High-quality training data is a prerequisite for reducing hallucinations in generative models. Statistical outlier detection methods, such as the Mahalanobis distance, identify anomalous samples that deviate from the data distribution. For a feature vector x with mean μ and covariance matrix Σ, the distance is computed as:

$$ D_M(x) = \sqrt{(x - \mu)^T \Sigma^{-1} (x - \mu)} $$

Values exceeding a threshold (e.g., 3σ) are flagged for removal. For text data, perplexity-based filtering rejects low-probability sequences under a trained language model:

$$ \text{Perplexity}(x) = \exp\left(-\frac{1}{N}\sum_{i=1}^N \log p(x_i|x_{<i})\right) $$

Semantic Consistency Verification

Knowledge-grounded verification augments raw data with external knowledge graphs (e.g., Wikidata) to validate factual claims. Given a claim c and knowledge graph KG, the verification score is:

$$ S(c, KG) = \frac{|\text{Facts}(c) \cap \text{Facts}(KG)|}{|\text{Facts}(c)|} $$

Claims with scores below 0.8 are either discarded or marked for human review. For multimodal data, cross-modal alignment models like CLIP compute similarity between image-text pairs:

$$ \text{sim}(I,T) = \frac{f(I) \cdot g(T)}{||f(I)|| \cdot ||g(T)||} $$

Data Augmentation with Constraints

Controlled paraphrasing preserves semantic meaning while increasing diversity. Given an original sentence s, the augmented version s' must satisfy:

$$ \text{BLEU}(s, s') > 0.7 \quad \text{and} \quad \text{NLI}(s, s') = \text{Entailment} $$

For tabular data, synthetic minority oversampling (SMOTE) generates interpolated samples while maintaining feature correlations:

$$ x_{\text{new}} = x_i + \lambda (x_j - x_i), \quad \lambda \sim U(0,1) $$

Temporal Data Validation

Time-series datasets require strict chronological partitioning to prevent future information leakage. The validation split V must satisfy:

$$ \max_{x \in V} t(x) < \min_{x' \in T} t(x') $$

where t(x) is the timestamp of sample x, and T is the test set. For event sequences, Poisson process validation checks temporal consistency of event rates across splits.

Bias Mitigation Techniques

Adversarial debiasing modifies the training objective to minimize predictability of protected attributes. The loss becomes:

$$ \mathcal{L} = \mathcal{L}_{\text{task}} - \lambda \mathcal{L}_{\text{adv}} $$

where λ controls the trade-off between accuracy and fairness. For text data, counterfactual augmentation generates gender-swapped versions while preserving other attributes.

2.2 Model Architecture Adjustments

Architectural modifications to foundation models can systematically reduce hallucination by constraining the model's generative freedom or improving its grounding capabilities. Three dominant approaches have emerged in recent research: retrieval-augmented generation (RAG), contrastive decoding, and multi-task verification heads.

Retrieval-Augmented Generation (RAG)

RAG architectures integrate an external knowledge retrieval component with the language model's decoder. Given input x, the system first retrieves relevant documents D from a verifiable corpus (e.g., Wikipedia or domain-specific databases) using dense vector similarity:

$$ \text{retrieve}(x) = \text{argmax}_{d \in D} \ f_\theta(x)^T g_\phi(d) $$

where f_θ and g_ϕ are dual encoders trained jointly with the language model. The retrieved documents are then concatenated with the input as additional context, forcing generations to stay grounded in the retrieved evidence.

Contrastive Decoding

This technique modifies the output probability distribution by contrasting predictions from a main model (p_θ) against those from an "amnesic" counterpart (p_θ′) trained to ignore factual knowledge:

$$ p_{\text{contrast}}(y_t|y_{

The hyperparameter α controls suppression of hallucinated tokens that appear likely under the amnesic model but lack factual basis. Implementations typically use a smaller version of the main model as the amnesic baseline.

Multi-Task Verification Heads

Modern architectures like DeBERTa-V3 add auxiliary output heads trained to predict the veracity of generated statements. These heads receive the same hidden representations as the main decoder but are optimized on datasets like FEVER or adversarial QA benchmarks. During inference, the verification score modulates token probabilities:

$$ \tilde{p}(y_t) = p(y_t) \cdot \sigma(\lambda v(y_{≤t})) $$

where v(·) is the verification head's confidence (0 to 1) and λ scales its influence. This creates a negative feedback loop where unverifiable continuations are progressively penalized.

Architectural Tradeoffs

RAG provides strongest grounding but requires low-latency retrieval systems
Contrastive decoding adds minimal compute overhead but needs careful baseline selection
Verification heads offer fine-grained control but require expensive multi-task training

Hybrid approaches like Atlas (RAG + contrastive) and Toolformer (API-verified generations) are pushing state-of-the-art in hallucination reduction while maintaining generation fluency.

Model Architecture Adjustments – Hallucination Mitigation Techniques – Tutorial Diagram

Diagram Description: The section describes three distinct architectural approaches (RAG, contrastive decoding, multi-task verification heads) with interacting components that would benefit from visual separation and connection.

2.3 Regularization and Uncertainty Estimation

Regularization techniques play a critical role in mitigating hallucination by constraining model complexity and preventing overconfident predictions. Bayesian neural networks (BNNs) and Monte Carlo dropout provide principled approaches to uncertainty estimation, enabling models to quantify epistemic and aleatoric uncertainty.

Bayesian Neural Networks

BNNs treat weights as probability distributions rather than point estimates, allowing for explicit modeling of uncertainty. The posterior distribution over weights w given data D is computed using Bayes' theorem:

$$ P(w|D) = \frac{P(D|w)P(w)}{P(D)} $$

Since exact inference is intractable for deep networks, variational inference approximates the true posterior with a simpler distribution q(w|θ), minimizing the Kullback-Leibler (KL) divergence:

$$ \theta^* = \arg\min_{\theta} \text{KL}(q(w|\theta) \parallel P(w|D)) $$

This leads to the evidence lower bound (ELBO) objective, which balances data fit and model complexity:

$$ \mathcal{L}(\theta) = \mathbb{E}_{q(w|\theta)}[\log P(D|w)] - \text{KL}(q(w|\theta) \parallel P(w)) $$

Monte Carlo Dropout

Dropout, when applied at test time, serves as an efficient approximation to Bayesian inference. By performing multiple stochastic forward passes, the model generates a distribution of predictions:

$$ \hat{y} = \frac{1}{T} \sum_{t=1}^T f(x; W_t) $$

where W_t represents sampled weights with dropout masks applied. The variance across samples provides a measure of model uncertainty:

$$ \sigma^2 = \frac{1}{T} \sum_{t=1}^T (\hat{y}_t - \bar{y})^2 $$

Practical Implementation Considerations

Effective uncertainty estimation requires careful tuning of:

Dropout rates (typically 0.1-0.5 for hidden layers)
Number of Monte Carlo samples (50-100 for stable estimates)
Prior distributions in BNNs (often Gaussian with learned variance)

Recent advances like deep ensembles combine multiple independently trained models, achieving superior uncertainty quantification by capturing diverse modes in the hypothesis space. Each model's prediction contributes to an aggregated uncertainty measure:

$$ \sigma_{\text{ensemble}}^2 = \frac{1}{M} \sum_{m=1}^M \sigma_m^2 + \frac{1}{M} \sum_{m=1}^M (\mu_m - \bar{\mu})^2 $$

where M is the number of ensemble members, and μ_m, σ_m are individual model means and variances.

Regularization and Uncertainty Estimation – Hallucination Mitigation Techniques – Tutorial Diagram

Diagram Description: The diagram would show the comparison between standard neural networks and Bayesian neural networks, illustrating weight distributions versus point estimates, and the Monte Carlo dropout process with multiple stochastic forward passes.

3. Adversarial Training and Robustness Enhancements

3.1 Adversarial Training and Robustness Enhancements

Foundations of Adversarial Training

Adversarial training improves model robustness by explicitly incorporating adversarial examples into the training process. Given a neural network f_θ with parameters θ, the standard training objective minimizes the expected loss over the data distribution p_data:

$$ \min_θ \mathbb{E}_{(x,y)∼p_{data}}[L(f_θ(x), y)] $$

Adversarial training modifies this objective to account for worst-case perturbations within an ε-ball around each input:

$$ \min_θ \mathbb{E}_{(x,y)∼p_{data}} \left[ \max_{||δ||_p ≤ ε} L(f_θ(x + δ), y) \right] $$

where δ represents the adversarial perturbation constrained by the L_p-norm. The inner maximization generates adversarial examples, while the outer minimization updates model parameters to be robust against them.

Projected Gradient Descent (PGD) for Adversarial Example Generation

PGD is the most widely used method for solving the inner maximization problem. Starting from an initial point x₀, PGD iteratively applies:

$$ x_{t+1} = \Pi_{B_ε(x)} \left( x_t + α \cdot \text{sign}(\nabla_{x_t} L(f_θ(x_t), y)) \right) $$

where Π projects the perturbed sample back into the ε-ball around x, and α is the step size. This generates strong adversarial examples that reliably fool models.

Certifiable Robustness via Convex Relaxations

While adversarial training empirically improves robustness, it doesn't provide formal guarantees. Certifiable methods use convex relaxations to bound the worst-case behavior of neural networks. For a ReLU network, the convex relaxation yields:

$$ \min_{\hat{y}} \max_{z ∈ \mathcal{Z}} c^T z $$

where 𝒵 is the convex relaxation of the network's feasible outputs. Solving this provides a certificate that no adversarial example exists within the perturbation bound.

Adversarial Training Variants

Recent advances have developed more efficient and effective adversarial training methods:

TRADES separates natural and adversarial loss terms with a trade-off parameter β
MART focuses on misclassified examples during training
GAIRAT adaptively weights examples based on their vulnerability

These methods achieve better robustness-accuracy trade-offs than standard adversarial training.

Robustness Through Data Augmentation

Alternative approaches generate synthetic training data that captures the diversity of possible adversarial perturbations:

$$ \tilde{x} = x + G_φ(z), \quad z ∼ \mathcal{N}(0, I) $$

where G_φ is a generative model trained to produce worst-case perturbations. This provides a more computationally efficient alternative to iterative adversarial example generation.

Architectural Enhancements for Robustness

Modifying network architectures can inherently improve robustness:

Lipschitz constraints bound the network's sensitivity to input perturbations
Randomized smoothing creates probabilistic robustness guarantees
Denoising layers remove adversarial perturbations before classification

These architectural changes complement adversarial training to provide defense-in-depth against hallucinations.

Adversarial Training and Robustness Enhancements – Hallucination Mitigation Techniques – Tutorial Diagram

Diagram Description: The diagram would show the iterative process of Projected Gradient Descent (PGD) for generating adversarial examples, including the perturbation steps and projection back into the ε-ball.

3.2 Hybrid Models and Ensemble Methods

Hybrid models combine the strengths of multiple architectures—such as transformer-based language models with retrieval-augmented components—to reduce hallucination by grounding outputs in verifiable external knowledge. A common approach integrates a generative model G with a retrieval system R, where R fetches relevant documents D given an input query q, and G conditions its output on both q and D. The probability distribution over tokens y_t at step t becomes:

$$ P(y_t | y_{

Here, P(d | q) is the retrieval model's relevance score for document d, and P(y_t | y_{ is the generator's output probability conditioned on d. This forces the model to attend to factual anchors, reducing arbitrary generation.

Ensemble Diversification

Ensembles mitigate hallucination by aggregating outputs from multiple models or components. Key strategies include:

Diversity-promoting training: Optimize sub-models to maximize disagreement on out-of-distribution samples, ensuring coverage of plausible alternatives.

Confidence-weighted voting: Assign weights to ensemble members based on their calibrated confidence scores, downweighting unreliable contributors.

The final prediction ŷ for an ensemble of N models is computed as:

$$ \hat{y} = \sum_{i=1}^N w_i \cdot f_i(x), \quad w_i = \frac{\exp(\alpha \cdot c_i)}{\sum_j \exp(\alpha \cdot c_j)} $$

where f_i(x) is the i-th model's prediction, c_i its confidence score, and α a temperature parameter controlling selectivity.

Architectural Hybridization

Recent work combines autoregressive generation with non-parametric memory, such as:

Retroactive verification: A post-generation module cross-checks outputs against retrieved evidence, flagging or rewriting unsupported claims.

Dual-encoder hybrids: Separate encoders process factual (e.g., knowledge graph embeddings) and generative contexts, with a gating mechanism controlling their influence.

In the diagram above, a hybrid system routes retrieved evidence from the retriever to both the generator (for conditioning) and the verifier (for validation). The verifier's output gradient can backpropagate to fine-tune the generator's attention mechanisms, reinforcing factual consistency.

Practical Trade-offs

While hybrid models reduce hallucination, they introduce latency from retrieval and verification steps. Techniques like:

Asynchronous retrieval: Pre-fetch documents while the user is typing, amortizing lookup costs.

Distilled ensembles: Train a single model to mimic the ensemble's behavior, preserving accuracy while reducing inference cost.

Diagram Description: The diagram would physically show the interaction flow between the retriever, generator, and verifier components, including the routing of retrieved evidence and backpropagation of verification gradients.

Post-Hoc Verification and Correction

Post-hoc verification techniques aim to detect and correct hallucinations in model outputs after generation, leveraging external knowledge sources or consistency checks. Unlike prompt engineering or training-time interventions, these methods operate on the model's outputs, making them adaptable to black-box systems.

Knowledge-Based Verification

Knowledge-based approaches cross-check generated content against structured databases or unstructured corpora. Given a generated claim c, the system retrieves supporting evidence E from a knowledge base K, then computes a verification score:

$$ V(c, K) = \max_{e \in E} \text{sim}(f(c), f(e)) $$

where f is an embedding function (e.g., from a pretrained language model) and sim is a similarity metric like cosine similarity. Thresholding V(c, K) identifies unverifiable claims. For numerical or factual claims, exact matching against knowledge bases like Wikidata often proves more reliable than semantic similarity.

Self-Consistency Checking

Self-consistency methods exploit the observation that hallucinations often contradict other statements in the same output. For a generated passage P = {s₁, ..., s_n}, pairwise contradiction detection can be formulated as:

$$ C(s_i, s_j) = \mathbb{I}[\text{NLI}(s_i, s_j) = \text{contradiction}] $$

where NLI denotes a natural language inference model. Graph-based algorithms then identify and resolve inconsistent statement clusters. Recent work extends this to cross-document consistency checking against the model's own previous outputs.

Uncertainty Quantification

Bayesian approaches quantify uncertainty in generated outputs by sampling multiple completions or perturbing inputs. For a generated sequence y given input x, the epistemic uncertainty can be estimated via Monte Carlo dropout:

$$ U(y|x) = \frac{1}{T}\sum_{t=1}^T \text{KL}(p(y|x,\theta_t) || \bar{p}(y|x)) $$

where θ_t are sampled dropout masks and p̄ is the mean predictive distribution. High uncertainty regions often correlate with hallucinated content, enabling targeted correction.

Corrective Methods

Upon detecting potential hallucinations, correction strategies include:

Retrieval-Augmented Generation: Rerank and regenerate outputs conditioned on retrieved evidence

Constrained Decoding: Apply lexical or semantic constraints during regeneration

Human-in-the-Loop: Flag uncertain outputs for expert review before delivery

Recent hybrid approaches combine these methods, such as using uncertainty estimates to trigger knowledge-augmented regeneration only for high-risk outputs, balancing computational cost and accuracy.

4. Quantitative Metrics for Hallucination Assessment

4.1 Quantitative Metrics for Hallucination Assessment

Hallucination in AI models, particularly large language models (LLMs), refers to the generation of factually incorrect or nonsensical outputs that are not grounded in the input data or real-world knowledge. To systematically evaluate and mitigate hallucinations, researchers employ quantitative metrics that measure the degree of hallucination in model outputs. These metrics fall into three broad categories: reference-based, reference-free, and model-based approaches.

Reference-Based Metrics

Reference-based metrics compare model outputs against ground-truth references, typically human-annotated data. Common metrics include:

BLEU (Bilingual Evaluation Understudy): Measures n-gram overlap between generated text and reference text. While useful for translation tasks, BLEU often correlates poorly with factual accuracy.

ROUGE (Recall-Oriented Understudy for Gisting Evaluation): Focuses on recall of overlapping n-grams, particularly useful for summarization tasks. Variants include ROUGE-L (longest common subsequence).

METEOR (Metric for Evaluation of Translation with Explicit ORdering): Incorporates synonym matching and stemming, providing better alignment with human judgment than BLEU.

These metrics are computed as:

$$ ext{BLEU} = BP \cdot \exp\left(\sum_{n=1}^N w_n \log p_n\right) $$

$$ ext{ROUGE-L} = \frac{(1 + \beta^2)RLCS}{R + \beta^2 L} $$

where BP is the brevity penalty, p_n is the n-gram precision, and RLCS is the recall of the longest common subsequence.

Reference-Free Metrics

When reference texts are unavailable, reference-free metrics assess hallucinations by analyzing the internal consistency or plausibility of generated text:

Self-BLEU: Computes BLEU score between different samples generated for the same input, with lower scores indicating higher diversity (and potential hallucinations).

Perplexity: Measures how surprised the model is by its own output. Abnormally low perplexity may indicate memorization, while high perplexity suggests incoherence.

Lexical Diversity: Computes the ratio of unique n-grams to total n-grams. Unusually high diversity may signal hallucination.

$$ ext{Perplexity} = \exp\left(-\frac{1}{N}\sum_{i=1}^N \log p(w_i | w_{<i})\right) $$

Model-Based Metrics

Recent advances leverage auxiliary models to detect hallucinations:

FactScore: Uses retrieval-augmented models to fact-check claims against knowledge bases.

NLI (Natural Language Inference): Measures entailment between source and generated text using models like BART or DeBERTa.

QA-Based Metrics: Generates questions from the output and checks answer consistency with the input.

The FactScore metric decomposes as:

$$ ext{FactScore} = \frac{1}{|C|}\sum_{c \in C} \mathbb{I}( ext{Retrieve}(c) \vdash c) $$

where C is the set of claims and ⊢ denotes entailment.

Practical Considerations

In practice, hallucination metrics are often combined. For example, the HaluEval benchmark uses:

ROUGE for faithfulness to input

Perplexity for coherence

NLI for factual consistency

Recent studies show that no single metric reliably detects all hallucination types. The best-performing ensembles achieve ~0.85 F1 scores on hallucination detection tasks, with model-based metrics generally outperforming traditional n-gram approaches.

4.2 Human-in-the-Loop Evaluation Techniques

Human-in-the-loop (HITL) evaluation leverages human expertise to assess and refine model outputs, particularly in mitigating hallucinations. This approach combines automated metrics with qualitative human judgment, addressing limitations of purely statistical evaluation.

Active Learning for Hallucination Detection

Active learning frameworks iteratively select the most informative samples for human review, optimizing annotation effort. The query strategy typically maximizes uncertainty or diversity:

$$ x^* = \argmax_{x \in \mathcal{U}} H(y|x) - \lambda \cdot \text{sim}(x, \mathcal{L}) $$

where H(y|x) is the predictive entropy, 𝒰 the unlabeled pool, ℒ the labeled set, and sim(·) a similarity measure to avoid redundant labeling.

Confidence-Aware Sampling

Modern implementations use model confidence scores to prioritize review candidates:

$$ \text{ReviewPriority} = \begin{cases} 1 - \max(p(y|x)) & \text{for classification} \\ \text{NLL}(x,y) & \text{for generation} \end{cases} $$

Threshold tuning balances precision-recall tradeoffs in hallucination detection. The optimal threshold τ adapts to task requirements:

$$ \tau = \underset{\tau'}{\operatorname{argmin}} \alpha \cdot \text{FPR}(\tau') + (1-\alpha) \cdot \text{FNR}(\tau') $$

Multi-Stage Verification Pipelines

Industrial systems often implement cascaded verification:

Automated filtering: Rule-based checks for factual inconsistencies

Model ensemble voting: Agreement analysis across multiple architectures

Expert review: Domain specialists validate borderline cases

This reduces human workload by 60-80% while maintaining >95% hallucination detection rates in production systems.

Inter-Rater Reliability Metrics

When using multiple annotators, quantify agreement with Krippendorff's alpha:

$$ \alpha = 1 - \frac{\sum_{k=1}^K \sum_{i=1}^{N_k} o_{ki}(1 - o_{ki})}{\sum_{k=1}^K \sum_{i=1}^{N_k} (o_{ki} - \bar{o})^2} $$

where o_ki is the k-th annotator's rating for item i, and ō the global mean rating. Values above 0.8 indicate reliable annotation protocols.

Real-Time Feedback Integration

Effective HITL systems incorporate corrections into subsequent model updates. The weight update for human-corrected sample (x, y_human) follows:

$$ \Delta w = \eta \cdot \nabla_w \ell(f_w(x), y_{\text{human}}) $$

with learning rate η adjusted based on annotator expertise scores. This creates a virtuous cycle of improvement, reducing hallucination rates by 15-30% per feedback iteration in deployed systems.

Diagram Description: The multi-stage verification pipeline involves sequential steps with distinct components interacting, which is best visualized as a flow diagram.

4.3 Benchmark Datasets and Case Studies

Standardized Evaluation Datasets

Quantifying hallucination in language models requires carefully curated datasets that isolate specific failure modes. The TruthfulQA benchmark (Lin et al., 2022) contains 817 questions designed to test a model's tendency to generate false answers that mimic human-like misconceptions. Each question is paired with reference answers labeled by veracity, enabling calculation of the hallucination rate as:

$$ H = \frac{1}{N}\sum_{i=1}^{N} \mathbb{I}(f(x_i) \notin \mathcal{V}(x_i)) $$

where f(x_i) is the model's response, 𝒱(x_i) is the set of valid answers, and 𝕀 is the indicator function.

The HaluEval dataset (Li et al., 2023) provides finer-grained categorization across three hallucination types: factual inconsistency, logical contradiction, and unverifiable claims. Each instance includes both the original context and the hallucinated continuation, enabling training of discriminative models.

Controlled Case Studies

In controlled experiments with GPT-4, the SelfCheckGPT method (Manakul et al., 2023) demonstrates how sampling multiple responses to the same prompt can surface inconsistencies. When applied to biomedical queries, the method achieved 0.82 AUROC in detecting unsupported claims by comparing answer variants.

A longitudinal study of ChatGPT's outputs on legal questions (Gupta et al., 2023) revealed that 29% of citations to case law were completely fabricated. The study employed a verification pipeline combining:

Named entity recognition for legal references

Cross-checking with Westlaw database

Expert attorney review

Domain-Specific Benchmarks

For clinical applications, the Med-HALT benchmark (Zhang et al., 2023) evaluates hallucinations in medical dialogue systems through:

In financial domains, the FinTruth dataset tracks hallucination patterns in earnings report analysis, showing that models frequently invent plausible-looking but incorrect numerical relationships between:

$$ \Delta \text{Revenue} \propto k \cdot \Delta \text{EBITDA} $$

where models often hallucinate the proportionality constant k outside empirically observed ranges.

Adversarial Evaluation Protocols

The Hallucination Stress Test framework (Shen et al., 2024) systematically perturbs inputs along dimensions known to induce hallucinations:

Increasing question ambiguity through syntactic complexity

Injecting contradicting premises

Removing key context tokens

Results show transformer-based models exhibit a 43% increase in hallucination rate when subjected to combined perturbations compared to baseline queries.

5. Bias and Fairness in Mitigation Techniques

5.1 Bias and Fairness in Mitigation Techniques

Hallucination mitigation techniques must account for inherent biases in training data and model architectures to ensure fairness. Biases can propagate through mitigation strategies, exacerbating disparities in model outputs across demographic groups. For instance, language models trained on imbalanced corpora may disproportionately generate hallucinated content for underrepresented populations.

Quantifying Bias in Mitigation

Bias can be formalized as deviations from equitable performance across protected attributes. Let X represent input features, Y the ground truth, and A a protected attribute (e.g., gender, race). The bias B of a mitigation technique can be measured as:

$$ B = \mathbb{E}_{x \sim X} \left[ \frac{1}{|A|} \sum_{a \in A} \left( \mathcal{L}(f(x_a), y_a) - \mathcal{L}(f(x), y) \right)^2 \right] $$

where f is the mitigated model, ℒ the loss function, and x_a denotes inputs with protected attribute a. Minimizing B ensures mitigation does not disproportionately affect specific groups.

Fairness-Aware Mitigation Strategies

Three principal approaches exist for bias-aware hallucination mitigation:

Pre-processing: Debiasing training data via reweighting or adversarial filtering before applying mitigation techniques.

In-processing: Incorporating fairness constraints directly into the mitigation objective, such as adding a regularization term:

$$ \min_\theta \mathbb{E}[\mathcal{L}(f_\theta(x), y)] + \lambda \text{Var}_{a \in A}(\mathbb{E}[\mathcal{L}(f_\theta(x_a), y_a)]) $$

Post-processing: Calibrating mitigation outputs using demographic parity constraints or equalized odds criteria.

Case Study: Clinical Text Generation

In medical report generation, mitigation techniques reduced hallucinations by 32% overall but introduced a 15% performance gap between racial groups. Applying in-processing fairness constraints reduced this disparity to 3% while maintaining 29% overall hallucination reduction. The constrained objective took the form:

$$ \min_\theta \sum_{i=1}^N \mathcal{L}_i + \gamma \sum_{a \in A} \left| \frac{1}{N_a} \sum_{i: a_i = a} \mathcal{L}_i - \frac{1}{N} \sum_{j=1}^N \mathcal{L}_j \right| $$

where N_a is the count of samples from group a, and γ controls the fairness-accuracy tradeoff.

Architectural Considerations

Transformer-based mitigation approaches exhibit distinct bias propagation patterns. Attention heads in decoder layers show higher bias amplification (measured by gradient variance across groups) than encoder layers. Modifying attention mechanisms to include fairness-aware scoring can reduce this effect:

$$ \text{Attention}(Q,K,V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}} + M\right)V $$

where M is a fairness mask constructed from protected attribute embeddings. This approach reduced gender bias in summarization tasks by 41% compared to standard mitigation.

Evaluation Metrics

Beyond traditional accuracy metrics, fairness-aware mitigation requires specialized evaluation:

Disparate Impact Ratio (DIR): Ratio of true positive rates between most and least favored groups

Bias Amplification Factor (BAF): BAF = (Post-mitigation bias) / (Pre-mitigation bias)

Fairness-Aware Precision (FAP): Precision weighted by inverse group prevalence

Optimal mitigation occurs when DIR ≈ 1, BAF ≤ 1, and FAP approaches unweighted precision. These metrics should be monitored during hyperparameter tuning of mitigation techniques.

5.2 Scalability and Real-World Deployment Challenges

Deploying hallucination mitigation techniques at scale introduces several engineering and computational challenges that must be addressed to ensure robustness in real-world applications. Large-scale systems often face trade-offs between inference speed, memory constraints, and model accuracy, particularly when integrating multiple mitigation strategies.

Computational Overhead of Mitigation Techniques

Many hallucination mitigation methods, such as uncertainty quantification or multi-step verification, introduce significant computational overhead. For instance, Monte Carlo dropout for uncertainty estimation requires multiple forward passes through the network:

$$ \hat{y} = \frac{1}{T} \sum_{t=1}^T f_{\theta_t}(x) $$

where T is the number of stochastic forward passes and f_{θ_t} represents the model with different dropout masks. This T-fold increase in computation becomes prohibitive when processing millions of queries daily.

Latency-Sensitive Applications

Real-time applications like conversational AI or autonomous systems impose strict latency constraints (often < 500ms). Techniques like retrieval-augmented generation (RAG) must balance the trade-off between database retrieval time and response quality. The end-to-end latency L can be modeled as:

$$ L = t_{\text{retrieval}} + t_{\text{generation}} + t_{\text{verification}} $$

where each component must be optimized to meet service-level agreements. Distributed vector databases and pre-filtering mechanisms are often employed to reduce t_retrieval.

Memory Constraints in Edge Deployment

Edge devices present unique challenges due to limited memory and processing power. Knowledge distillation techniques are commonly used to compress large verification models:

$$ \mathcal{L}_{\text{distill}} = \alpha \mathcal{L}_{\text{task}} + (1-\alpha) \text{KL}(q_{\text{teacher}} || q_{\text{student}}) $$

where α balances task performance and teacher-student alignment. Quantization-aware training further reduces model footprints, with 8-bit quantization typically achieving 4× compression with < 2% accuracy drop.

Consistency Across Distributed Systems

Maintaining consistency in mitigation strategies across distributed model replicas requires careful synchronization. The probability of hallucination P_h in an N-replica system with imperfect synchronization is:

$$ P_h = 1 - \prod_{i=1}^N (1 - p_i^{\text{local}}) + \epsilon_{\text{sync}} $$

where ε_sync represents the synchronization error term. Techniques like distributed consensus protocols or majority voting are employed to minimize ε_sync.

Dynamic Adaptation Requirements

Real-world systems must adapt to concept drift and emerging hallucination patterns. Online learning frameworks update mitigation parameters θ_m continuously:

$$ \theta_m^{(t+1)} = \theta_m^{(t)} - \eta \nabla_{\theta_m} \mathcal{L}(\mathcal{D}_{\text{stream}}) $$

where η is the learning rate and D_stream represents streaming data. This requires careful management of catastrophic forgetting through techniques like elastic weight consolidation.

Monitoring and Alert Systems

Effective deployment requires comprehensive monitoring of hallucination metrics. Key performance indicators include:

Hallucination rate per domain

Mitigation technique effectiveness

Computational cost overhead

False positive rate in detection

These metrics are typically tracked using exponentially weighted moving averages to detect anomalies in real-time:

$$ \text{EWMA}_t = \lambda x_t + (1-\lambda) \text{EWMA}_{t-1} $$

where λ controls the responsiveness to recent data points.

5.3 Emerging Research and Open Problems

Self-Consistency Verification Architectures

Recent work explores self-consistency verification through multi-path reasoning, where models generate multiple candidate responses then select the most internally consistent answer. The verification score V for response r given context c can be formulated as:

$$ V(r|c) = \frac{1}{N} \sum_{i=1}^N \mathbb{I}(r \equiv r_i) \cdot P(r_i|c) $$

where r_i are N sampled reasoning paths and 𝕀 is the indicator function. Current limitations include computational overhead and sensitivity to the base model's calibration errors.

Contrastive Decoding with Knowledge Graphs

Hybrid approaches combine neural generation with structured knowledge verification. Given a knowledge graph G and generated claim c, the contrastive objective becomes:

$$ \mathcal{L}_{contrast} = \log P(c|G) - \lambda \log P(c|\emptyset) $$

where λ controls the penalty for unsupported assertions. Open challenges include real-time knowledge graph alignment and handling incomplete knowledge bases.

Neurosymbolic Verification Layers

Emerging neurosymbolic architectures employ formal verification modules that:

Parse generated text into logical propositions

Check consistency against predefined rulesets

Compute confidence bounds using probabilistic soft logic

The verification function f_v for proposition p follows:

$$ f_v(p) = \begin{cases} 1 & \text{if } \exists \phi \in \Phi \text{ s.t. } \phi \vdash p \\ \epsilon & \text{if } \neg \exists \phi \in \Phi \text{ s.t. } \phi \vdash \neg p \\ 0 & \text{otherwise} \end{cases} $$

where Φ is the ruleset and ε represents uncertainty thresholds. Current research focuses on scaling these systems to open-domain scenarios.

Open Problems

Key unresolved challenges include:

Dynamic Grounding: Maintaining real-world referents during extended dialogues

Uncertainty Calibration: Improving model self-assessment of confidence levels

Cross-Modal Consistency: Aligning text generation with visual/audio inputs in multimodal systems

Adversarial Robustness: Preventing deliberate induction of hallucinations through carefully crafted prompts

Recent studies suggest that the hallucination rate H scales with model size D following a power law:

$$ H(D) = \alpha D^\beta + \gamma $$

where α, β, γ are dataset-dependent coefficients, indicating fundamental tradeoffs between capability and reliability.

Diagram Description: The section describes multiple verification architectures and their relationships, which would be clearer with a visual representation of the workflows and components.

6. Key Research Papers and Articles

6.1 Key Research Papers and Articles

Developing a Reliable, General-Purpose Hallucination Detection and ... — In this paper, we detail the key components in our service and highlight critical challenges, lessons and potential limitations, which we believe will benefit future research in this field. 2 Related Works Hallucination Taxonomy ... A comprehensive survey of hallucination mitigation techniques in large language models. ArXiv, abs/2401.01313 ...

Towards trustworthy LLMs: a review on debiasing and ... - Springer — Moreover, a comparative analysis of the sources, mitigation methods, and evaluation methods for bias and hallucination is included. In the end, this paper provides a synthesis of current research trends and suggests potential directions for future research to address bias and hallucination in LLMs, considering the ongoing challenges in this field.

PDF Understanding and Addressing AI Hallucinations in Healthcare and Life ... — hallucinations into three types input-conflicting, context-conflicting, and fact-conflicting and examine their implications through real-world cases. Methodology: Our methodology combines the Fact Score, Med-HALT, and adversarial testing to evaluate the fidelity of AI outputs. We propose several mitigation strategies, including Retrieval-

[2202.03629] Survey of Hallucination in Natural Language Generation - ar5iv — The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine ...

THaMES: An End-to-End Tool for Hallucination Mitigation and Evaluation ... — While THaMES fills a critical gap in current hallucination research by providing a comprehensive framework, it still has some limitations. (1) First, due to limited computational resources, we were only able to experiment with quantized and small-parameter versions of the models, which constrained the effectiveness of our mitigation methods.

(PDF) Towards Hallucination-Resilient AI: Navigating Challenges ... — The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in ...

PDF Artificial Intelligence Risk Management Framework: Generative ... - NIST — About AI at NIST: The National Institute of Standards and Technology (NIST) develops measurements, technology, tools, and standards to advance reliable, safe, transparent, explainable, privacy-enhanced, and fair artiﬁcial intelligence (AI) so that its full commercial and societal beneﬁts can be realized without

On large language models safety, security, and privacy: A survey — For safety, we conduct a comprehensive survey of inherent safety issues in LLMs, such as toxicity, bias, hallucination, and jailbreak, along with mitigation methods for each. For security, we examine the security of LLMs under active attacks, including backdoor, poisoning, and adversarial attacks, and investigate corresponding defense ...

PDF Hallucination‐Free? Assessing the Reliability of Leading AI Legal ... — stakes domains. Recently, certain legal research providers have touted methods such as retrieval- augmented generation (RAG) as "eliminating" or "avoid[ing]" hallucinations, or guaranteeing "hallucination- free" legal citations. Because of the closed nature of these systems, systematically assessing these claims is challenging.

Improving Factuality by Contrastive Decoding with Factual and ... — Large language models have demonstrated impressive capabilities in many domains. But they sometimes generate irrelevant or nonsensical text, or produce outputs that deviate from the provided input, an occurrence commonly referred to as hallucination. To mitigate this issue, we introduce a novel decoding method that incorporates both factual and hallucination prompts (DFHP). It applies ...

6.2 Recommended Books and Surveys

Survey of Hallucination in Natural Language Generation - arXiv.org — 10.3 Hallucination Mitigation in Data-to-Text Generation 28 10.4 Future Directions in Data-to-Text Generation 29 11 Hallucinations in Neural Machine Translation 29 11.1 Hallucinations Definition and Categories in NMT 29 11.2 Hallucination Metrics in NMT 30 11.3 Hallucination Mitigation Methods in NMT 32 11.4 Future Directions in NMT 33

[2202.03629] Survey of Hallucination in Natural Language Generation - ar5iv — The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine ...

On large language models safety, security, and privacy: A survey — A comprehensive survey of hallucination mitigation techniques in large language models [Online] Available ... was born in Shanxi, China in 2001. She received the B.S. degree from University of Electronic Science and Technology of China (UESTC), Chengdu, China in 2019. ... He won the best paper awards of the 26th IEEE International Conference on ...

THaMES: An End-to-End Tool for Hallucination Mitigation and Evaluation ... — Since no single mitigation strategy works best across all models, THaMES evaluates three different strategies, allowing users to select the optimal one based on the model and knowledge base. ... In this section, we introduce various hallucination mitigation techniques utilized by THaMES: ... A survey on in-context learning, 2024. URL https ...

A Survey on Large Language Model Hallucination via a Creativity Perspective — Civitas books, 2011. Gianotti et al. [2001] Lorena R. R. Gianotti, Christine Mohr, ... Best humans still outperform artificial intelligence in a creative divergent thinking task. ... A comprehensive survey of hallucination mitigation techniques in large language models. ArXiv preprint, 2024. Torrance [1977] ...

PDF Hallucination‐Free? Assessing the Reliability of Leading AI Legal ... — have claimed to mitigate, if not entirely solve, hallucination risk (Casetext 2023; LexisNexis 2023b; Thomson Reuters 2023, inter alia). They say their use of sophisticated techniques such as retrieval-augmented generation (RAG) largely prevents hal-lucination in legal research tasks.1 (We provide details on RAG systems in Section 3.1 below.)

Towards trustworthy LLMs: a review on debiasing and ... - Springer — Moreover, a comparative analysis of the sources, mitigation methods, and evaluation methods for bias and hallucination is included. In the end, this paper provides a synthesis of current research trends and suggests potential directions for future research to address bias and hallucination in LLMs, considering the ongoing challenges in this field.

A Survey on Hallucination in Large Language Models: — Two primary facets encompass the broad spectrum of hallucination mitigation: detection mechanisms and evaluation benchmarks. This section serves as a deep dive into the state-of-the-art techniques for detecting hallucinations (§ 4.1) and the benchmarks (§ 4.2)that evaluate their prowess.

Grounded but Misguided: Mitigating Hallucinations in Clinical LLMs and ... — 2.3. The Inevitability Argument. Theoretical computer science offers a sobering perspective on hallucinations. Research formalizing the problem suggests that hallucination might be an innate limitation of LLMs.[24] By modeling LLMs and ground truth functions as computable entities, it has been argued that LLMs, due to inherent constraints described by learning theory, cannot learn all ...

(PDF) Towards Hallucination-Resilient AI: Navigating Challenges ... — The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in ...

6.3 Online Resources and Tools

Chapter 6 LLM Challenges and Solutions - Springer — 6.2.6.3 Intra-Processing Mitigation predictions without further training. These methods are considered inference stage mitigations and encompass techniques such as altered decoding strategies, post-hoc modifications to model parameters, and separate debiasing networks applied in a modular fashion

Grounded but Misguided: Mitigating Hallucinations in Clinical LLMs and ... — The primary objectives are to: Define and characterize medical hallucinations within the context of LLMs/RAG systems processing EHR data. Identify and analyze the key challenges associated with using EHR data for hallucination mitigation, focusing on privacy, noise, incompleteness, and bias.

PDF Medical Hallucination in Foundation Models and Their Impact on Healthcare — Consequently, mitigation requires strategies tailored to each context: clinicians might benefit from decision-support tools and reflective practice to counter personal biases, while LLMs demand better data curation, retrieval-augmented generation, or explicit calibration methods to curb hallucinations and unwarranted certainty.

Towards Hallucination-Resilient AI Navigating Challenges, Ethical ... — This paper discusses the types of hallucinations and the strategies to mitigate the hallucinations in RAG-based models by understanding and enhancing the retrieval mechanisms, implementing verification techniques, and utilizing logical reasoning models.

An Evolutionary Large Language Model for Hallucination Mitigation — This paper conducts a scoping study of existing techniques for mitigating hallucinations in knowledge-based task in general and especially for medical domains. Key methods covered in the paper include Retrieval-Augmented Generation (RAG)-based techniques, iterative feedback loops, supervised fine-tuning, and prompt engineering.

Developing a Reliable, Fast, General-Purpose Hallucination Detection ... — Developing a general-purpose, fast and accurate hallucination detection and mitigation service is an extremely difficult task given the existing state-of-the-art technologies. To this end, we present a pragmatic solution as shown in Figure 1, which includes three modules: multi-source detection, iterative rewriting and multi-source verification.

(PDF) Towards Hallucination-Resilient AI: Navigating Challenges ... — The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in ...

Mitigating Hallucination in Visual-Language Models via Re ... - Springer — Moreover, our RBD technique outperformed comparative hallucination mitigation technologies with-out the need for additional models or tools, highlighting the plug-and-play advan-tage of our approach.

strategies for mitigating hallucinations — Table 4: The features of each method for hallucination reduction. It's also worth mentioning that some of these methods can be combined together to obtain a better effect in reducing the hallucination rate of an LLM. Find more information about the experiments and code examples discussed in this blog, please visit the github repository.

PDF Multi-Modal Hallucination Control by Visual Information Grounding — Give an explanation of the image.", "Provide a description of the given image.". To measure hallucinated objects we follow [20] and complement the set of MS COCO annotated objects with their synonyms and automatically detect object hallucinations by comparing