Robust Prior Update (RPU): Reducing Hallucination in Diffusion-Based Inverse Problem Solvers
By
[Submitted on 1 Jun 2026]
A weekday bagel. Dependable, satisfying, no fuss.
Summary
This paper introduces Robust Prior Update (RPU), a module for diffusion-based inverse problem solvers that addresses measurement-conditioned hallucination—where reconstructed images appear realistic but contain details inconsistent with the actual measurements. The authors analyze how hallucinated content enters through the prior update step before measurement correction is applied. RPU probes the local stability of the diffusion prior update, re-anchors the displacement at the current iterate, and leaves measurement updates unchanged. Evaluated on FFHQ and ImageNet datasets using DPS, RPU shows improvements in PSNR and LPIPS metrics across inpainting and deblurring tasks, with human faithfulness studies showing 91.9% blind non-tie majority preferences on FFHQ box inpainting.
Key quotes
· 5 pulledDiffusion-based inverse problem solvers can produce realistic reconstructions, but realism alone does not ensure that the recovered details are supported by the measurement.
Our analysis separates Bayes-rule-based diffusion inverse solvers into a prior update and a measurement-conditioning step, showing that hallucinated content can enter through the prior-side proposal before the measurement correction is applied.
We propose Robust Prior Update (RPU), a solver-level module that probes the local stability of the diffusion prior update, re-anchors the resulting displacement at the current iterate, and leaves the measurement update unchanged.
In human judgments, RPU receives 91.9% of blind non-tie majority preferences and 91.1% of ground-truth-assisted non-tie preferences on FFHQ box inpainting.
These results support a targeted claim: robustifying the prior update can improve instance faithfulness in diffusion inverse solvers, especially when the prior shapes weakly constrained content.
You might also wanna read
SHARE: Pose-Free 3D Gaussian Splatting Framework for Robust Scene Reconstruction
Researchers introduce SHARE, a pose-free 3D Gaussian splatting framework that addresses the challenge of inaccurate camera poses in real-wor
arxiv.org·6mo agoTiny-diffusion: A minimal implementation of probabilistic diffusion models
Computing Hessian Inverse Products for Deep Neural Networks to Speed Up Gradient Descent
This article presents a GitHub repository that demonstrates how to compute the inverse of the Hessian matrix for deep neural networks and mu
github.com·4mo ago
Terrain Diffusion: A Diffusion-Based Framework for Infinite, Real-Time Terrain Generation
Researchers introduce Terrain Diffusion, a new generative framework that combines the fidelity of diffusion models with the infinite scalabi
arxiv.org·5mo ago