|
Variance-Guided Score Regularization for Hallucination Mitigation in Diffusion Models |
Soundness: 2: fair
Presentation: 3: good
Contribution: 2: fair
Rating: 4: marginally below the acceptance threshold
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
This paper provides a theoretical and practical framework for mitigating hallucinations in diffusion models. The authors start by offering a density-based formulation of hallucinations, showing that the Gaussian nature of the diffusion process inevitably assigns non-zero probability to regions outside the true data support. They derive an explicit lower bound on this off-support mass as a function of the score function’s Lipschitz constant and boundedness, establishing a link between score smoothness and hallucination probability.
To mitigate hallucinations, they propose Variance-Guided Score Regularization (VSR) — a training-time regularizer that penalizes low Jacobian norms of the score function, effectively tightening the lower bound on off-support probability. The method is architecture-agnostic and leverages variance information learned during denoising. Empirical results on both standard datasets (MNIST, Hands, Shapes) and two new benchmarks (Cards and ChessImages) demonstrate consistent reductions in hallucinations (15–25%) while maintaining fidelity and diversity.
1. The paper is the first to explicitly connect score smoothness in diffusion models to hallucination probability, using measure-theoretic reasoning and formal lower bounds.
2. The smoothness penalty is a mathematically motivated modification that directly influences the score Jacobian. Its integration with variance estimation (via I-DDPM) adds interpretability and practical feasibility.
3. The authors conduct experiments on various datasets, showing reduction in both hallucination rate (H%) and score error (∆S).
1. The theoretical foundation of VSR hinges on a simplified assumption that disjoint supports (gaps) exist in high-dimensional data manifolds. This assumption holds for synthetic or low-dimensional distributions (e.g., 1D/2D Gaussians, discrete combinatorial datasets like Cards/ChessImages), but it may easily break down for continuous, correlated natural image manifolds. For example, in the high-dimensional image space, distributions of semantically different classes such as cats vs. dogs, or cats vs. tigers, may overlap in the feature manifold as they share visual attributes like eyes, fur, and texture.
2. Lemma 1, lacking a formal proof and relying on the above assumption, is more a heuristic hypothesis than a mathematically valid lemma. As a result, the derived bounds may not apply to real-world generative tasks where class manifolds are continuous and overlapping.
3. The lower bound derived in Theorem 2 relies on strong regularity assumptions including L-Lipschitz continuity, bounded scores, and compact boundaries, which rarely hold for high-dimensional natural images. The bounds thus provide conceptual intuition but limited quantitative predictive power.
4. The proposed regularizer encourages larger Jacobians, but intuitively, large Jacobians can also cause instability or excessive curvature in score fields, potentially harming training stability if not carefully tuned. This tension between theory and practice is not thoroughly discussed.
1. Could the approach generalize to text-conditioned or multi-modal diffusion models, where hallucinations stem from semantic misalignment rather than density leakage?
2. Given that theorem 2 assumes compact boundaries, how does this extend to unbounded data manifolds like real image spaces?
3. Have the authors evaluated how the score smoothness affects mode coverage vs. fidelity trade-off in FID metrics? |
Fully AI-generated |
|
Variance-Guided Score Regularization for Hallucination Mitigation in Diffusion Models |
Soundness: 2: fair
Presentation: 2: fair
Contribution: 3: good
Rating: 2: reject
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
This paper proposes a theoretical and practical framework for understanding and mitigating hallucinations—implausible or off-support samples—generated by diffusion models. The authors first establish a density-based analysis showing that diffusion models inevitably assign nonzero probability mass, and then introduce Variance-Guided Score Regularization (VSR), a training-time regularizer to reduce hallucinations. Empirical results across synthetic and real datasets (e.g., Hands-11K, MNIST, Shapes, and new Cards and ChessImages benchmarks) demonstrate that VSR reduces hallucination rates.
1. Elegant and practical regularization: VSR is simple to implement and architecture-agnostic.
2. Valuable benchmark contribution: The paper conducted research on a comprehensive set of datasets. Besides, Cards and ChessImages datasets expand the empirical scope for systematic hallucination evaluation at scale.
1. The biggest weakness of the paper is that its theorems are the unsurprising consequences of the assumptions (P1-3, A4), and the reviewer believe it is not responsible for the actual effect of the regularizer. By assuming that a smooth function has positive values on the boundary of a region, it is natural that it has non-empty integral in the interior. All the quantities (Lipschitzness, norm bounds) are merely quantities in the assumption and the paper does not show the scale of these quantities in experiments, therefore we cannot conclude that whether the assumptions are playing a significant role in inducing hallucinations in practice.
Furthermore, the theorem highlights that in order to reduce hallucination, we may need to enlarge the sharpness of the learned function in gap regions. However, the regularizer proposed in the paper encourages sharpness in the non-gap regions (training data supports). Therefore the theorem is not accountable for how the regularizer work in practice. Furthermore, in the experiments the paper is comparing models with / without regularizer with different score errors, making the comparison ineffective as score error is an important confounder for generation quality and hallucination rates.
2. The paper is badly-written in terms of mathematical rigor and result clarity. A list of confusions are listed below:
- In definition 1 the definition of H is ambiguous and probability distribution should not appear in the set definition.
- In preliminary: missing expectation symbols in ground-truth score definition; the definition support is inconsistent as it is closed but {x:p(x)>0} is open.
- In lemma 1 what is "reverse transition"? What is $\lambda^d$ on line 168? What is $P^{hall}$？
- What is hallucination rate measured in the experiments and how can it be >1 (table 2)?
- Various typos in theorems and lemmas.
3. The paper discusses the distinction between generalized samples and memorized samples, which is quite irrelevant to its main theme.
Can you elaborate on the weaknesses above? |
Fully human-written |
|
Variance-Guided Score Regularization for Hallucination Mitigation in Diffusion Models |
Soundness: 2: fair
Presentation: 1: poor
Contribution: 2: fair
Rating: 2: reject
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
The paper studies hallucinations in diffusion models from a density-based perspective and argues that smooth (Lipschitz) score fields inevitably assign non-zero probability mass to “gap” regions outside the true data support. Building on this, the authors derive a pointwise lower bound that connects the local score bound (S) and Lipschitz constant (L) to off-support density. They then propose a Variance-Guided Score Regularization (VSR) term that encourages larger score Jacobians during training, aiming to reduce the theoretical lower bound of gap mass. Empirically, the paper reports hallucination reductions on synthetic 1D/2D Gaussians and several image datasets, and introduces two structured benchmarks—Cards and ChessImages.
1. Foundational problem. Hallucination in diffusion models is a low-level, system-level reliability issue; giving it a precise formalization and trying to tie it to the geometry/smoothness of the score field is valuable.
2. Theory + benchmarks. The paper provides an explicit lower-bound analysis (linking score smoothness to off-support mass) and contributes two large, structured evaluation suites (Cards, ChessImages) with efficient detectors, which can help standardize hallucination studies.
1. The introduction is a bit misleading. It mentions hallucinations in large T2I models like FLUX and Stable Diffusion 3.5 (L31–35, L39-42) and links them to fairness issues, but the proposed score regularization method can’t directly be applied to those models. These models are flow-matching based and don’t expose their variance heads. It would be good to clarify what kind of diffusion models this method can actually work on, and whether it’s limited to toy.
2. The jump from the theorem to the method isn’t convincing. Section 4 argues that smooth scores lead to nonzero mass in “gap” regions, but the proposed loss simply increases the Jacobian norm everywhere. There’s no argument that this increase happens only on the data manifold. In low-density regions the score and Jacobian can already be large, so this might even amplify off-support gradients—the opposite of what’s intended. The paper should either analyze this behavior region-wise or show empirically that larger Jacobians actually correlate with fewer hallucinations.
3. The paper is hard to follow, e.g., Tables 2 and 3 are confusing.
- Table 2 shows H% values above 100 (e.g., 521.73) even though it’s described as a percentage. I couldn’t find a clear explanation of what these numbers mean or how they’re computed in the paper.
- The same metric switches notation between H% and %H in Table 3.
4. Figure 4b is also hard to interpret. The plot suggests that when $\lambda=0.0$, the learned score is closer to the true score, and increasing $\lambda$ makes it worse. This looks opposite to the claim that the regularizer improves the score field. In L252, it seems the regularization strength is the weight of the proposed loss?
5. It’s not clear how the smoothness loss is balanced with the standard denoising loss over time. The score magnitude changes a lot with t, so applying the same weight $\rho$ everywhere might over-regularize some steps. The paper doesn’t show any per-t weighting or sensitivity analysis for $\rho$ and $\eta$. Some discussion on how to tune or schedule this term would make the method easier to reproduce and understand.
Please refer to Weaknesses section. |
Lightly AI-edited |
|
Variance-Guided Score Regularization for Hallucination Mitigation in Diffusion Models |
Soundness: 3: good
Presentation: 4: excellent
Contribution: 1: poor
Rating: 8: accept, good paper
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
This paper addresses the problem of hallucinations (implausible samples outside the true data distribution support) in Diffusion Models (DMs). The core contribution is providing a density-based perspective that links the smoothness of the learned score function to a nonzero probability mass in gap regions, which mathematically leads to a positive hallucination rate.
To mitigate this, the authors introduce Variance-Guided Score Regularization (VSR), an architecture-agnostic training-time regularizer (L_smooth) that explicitly controls the score Jacobian to increase local score curvature, thereby reducing the theoretical lower bound on this "gap mass". The practical instantiation of L_smooth$ is guided by the learned denoising variance, often learned in modern DMs.
The paper also proposes two challenging new benchmark datasets, Cards and ChessImages, which feature extremely large semantic class spaces (21 x 10^5 and ~ 10^44, respectively) and employ efficient, training-free validators for systematic hallucination evaluation.
Empirical results across synthetic, standard (Hands, MNIST, Shapes), and the new challenge datasets demonstrate that VSR consistently reduces hallucinations (up to ~25% on proposed benchmarks) while maintaining or improving fidelity (FID, C-FID) and diversity/novelty (FLD).
The paper presents a **highly original and significant** contribution by establishing a novel theoretical link between **score function smoothness** and the **positive lower bound on probability mass allocated to non-data regions** (gaps), thereby formalizing the root cause of hallucinations in diffusion models.
The derived quantitative lower bound (Theorem 2) directly motivates the proposed **Variance-Guided Score Regularization (VSR)**, an **architecture-agnostic** mitigation strategy that effectively increases local score curvature via the score Jacobian. This principle-driven approach demonstrates **high quality** in its execution, evidenced by robust empirical results showing a consistent reduction in hallucination rates ($\sim 15-25\%$) across synthetic and real-world datasets, while simultaneously improving fidelity and generating novel samples (FLD). Furthermore, the introduction of the **Cards** and **ChessImages** datasets is a major **significant contribution**, providing new, scalable benchmarks with extremely high combinatorial complexity ($\sim 10^{44}$ semantic classes for ChessImages) and automated validation, addressing a key limitation in systematic hallucination evaluation for generative models. Finally, the paper's **clarity** is excellent, formalizing definitions of hallucination and clearly structuring the theoretical derivation and experimental analysis.
* **$\mathcal{L}_{smooth}$ Implementation Detail:** The paper mentions that calculating the high-dimensional covariance (or score Jacobian) is intractable for images and they rely on the **I-DDPM implementation to learn diagonal covariance**. While they adopt this, the **exact, explicit formulation** of the $\mathcal{L}_{smooth}$ objective *in terms of the learned diagonal covariance* for the high-dimensional image case is **not explicitly provided** in the main paper (Equation 2 uses $||J_{\theta}(x_t, t)||^2$, which is the target, but not the implementable proxy). This implementation detail is crucial for reproducibility and for confirming the practical connection between the theory and the image-based experiments.
* **Generalization vs. Memorization (ChessImages):** The paper notes that $90\%$ of valid boards are novel/generalized in the ChessImages dataset. While VSR's improvement on the FLD metric is noted, the paper should explicitly state and analyze **whether the addition of $\mathcal{L}_{smooth}$ (VSR) changes the ratio of Memorized vs. Generalized samples** compared to the baseline. Simply maintaining fidelity and reducing total hallucinations isn't enough; confirming that VSR helps promote generalization is a valuable secondary claim that needs direct evidence in the main text. Table 7 hints at this but a dedicated analysis or discussion would strengthen the claim.
1. **Please provide the explicit final formula for the Variance-Guided Score Regularization $\mathcal{L}_{smooth}$ (Equation 2) as it is implemented for the high-dimensional image datasets.** Specifically, express the term $||J_{\theta}(x_t, t)||^2$ (the squared norm of the Jacobian of the score) in terms of the **learned denoising variance/diagonal covariance** that is used in the practical implementation.
2. **Does VSR $\mathcal{L}_{smooth}$ explicitly influence the ratio of generalized (novel) samples to memorized samples?** Using the ChessImages dataset analysis (Section 7.1), can you show how the ratio $ \frac{\text{Novel Boards}}{\text{Valid Boards}}$ changes when using the DDPM baseline versus DDPM + VSR? An explicit comparison is needed to strongly support the claim that VSR promotes the generation of novel, valid samples.
3. The paper uses a classifier threshold of 0.98 for MNIST to detect hallucinations. **Could the authors comment on the sensitivity of the final hallucination rate (H%) to this specific threshold value?** Was a sensitivity analysis performed? |
Fully AI-generated |