Analyzing Loss Functions in Diffusion Bridge Samplers
By
badmonster
Toasted just enough. A reliable bake, gently seasoned.
Summary
Diffusion bridges in deep-learning methods for sampling from unnormalized distributions are analyzed, comparing the performance of Log Variance (LV) loss and reverse Kullback-Leibler (rKL) loss. The study shows that rKL loss with the log-derivative trick consistently outperforms LV loss, especially for diffusion bridges with learned diffusion coefficients.
Key quotes
· 3 pulledRecent works show that the Log Variance (LV) loss consistently outperforms the reverse Kullback-Leibler (rKL) loss when using the reparametrization trick to compute rKL-gradients.
From a practical perspective we find that rKL-LD requires significantly less hyperparameter optimization and yields more stable training behavior.
Experimental results with different types of diffusion bridges on challenging benchmarks show that samplers trained with the rKL-LD loss achieve better performance.
You might also wanna read
NVIDIA Open-Sources Cosmos 3 Foundation Model for Physical AI Reasoning and Action Generation
NVIDIA has released Cosmos 3, an open-source foundation model for Physical AI that integrates physical reasoning, world generation, and acti
developer.nvidia.com·56m ago
NEST Simulator Included in SPEC CPU 2026 Benchmark Suite for First Time
The Standard Performance Evaluation Corporation (SPEC) released its first updated benchmark suite since 2017 in May 2026, designed to measur
ebrains.eu·1h ago
NASA reveals plans for permanent lunar base spanning hundreds of square kilometres
NASA has revealed plans to build a permanent lunar base spanning hundreds of square kilometres as part of its Artemis programme. The initial
newscientist.com·1h ago
Study Shows Weight Decay During Pretraining Improves Language Model Adaptability After Fine-Tuning
This research paper investigates how weight decay during pretraining of large language models affects their downstream adaptability (plastic
arxiv.org·2h ago
New Quantum Oblivious Transfer Protocol Achieves Simulation Security with One-Way Functions
This paper presents a new simulation-secure quantum oblivious transfer (QOT) protocol built on one-way functions in the plain model. The pro
quantum-journal.org·2h ago
From Plant Pathogens to Genome Editors: The Biotechnology Journey of TALEs and TALENs
This article reviews how TALEs (transcription activator-like effectors), originally discovered as bacterial proteins that Xanthomonas pathog
doi.org·3h ago