Latest Insights into Diffusion Models part1(Towards AGI 2024) | by Monodeep Mukherjee | May, 2024

1. Diffusion-HMC: Parameter Inference with Diffusion Model pushed Hamiltonian Monte Carlo

Authors: Nayantara Mudur, Carolina Cuesta-Lazaro, Douglas P. Finkbeiner

Abstract: Diffusion generative fashions have excelled at varied image know-how and reconstruction duties all through fields. A a lot much less explored avenue is their utility to discriminative duties involving regression or classification points. The cornerstone of up to date cosmology is the facility to generate predictions for observed astrophysical fields from idea and constrain bodily fashions from observations using these predictions. This work makes use of a single diffusion generative model to deal with these interlinked targets — as a surrogate model or emulator for cold darkish matter density fields conditional on enter cosmological parameters, and as a parameter inference model that solves the inverse draw back of constraining the cosmological parameters of an enter topic. The model is able to emulate fields with summary statistics in keeping with these of the simulated aim distribution. We then leverage the approximate likelihood of the diffusion generative model to derive tight constraints on cosmology by using the Hamiltonian Monte Carlo methodology to sample the posterior on cosmological parameters for a given check out image. Lastly, we show that this parameter inference methodology is additional sturdy to the addition of noise than baseline parameter inference networks.

2. Consideration-Pushed Teaching-Free Effectivity Enhancement of Diffusion Fashions

Authors: Hongjie Wang, Difan Liu, Yan Kang, Yijun Li, Zhe Lin, Niraj K. Jha, Yuchen Liu

Abstract: Diffusion Fashions (DMs) have exhibited superior effectivity in producing high-quality and varied footage. However, this distinctive effectivity comes on the worth of pricey architectural design, considerably on account of consideration module intently utilized in predominant fashions. Current works primarily undertake a retraining course of to bolster DM effectivity. That’s computationally pricey and by no means very scalable. To this end, we introduce the Consideration-driven Teaching-free Atmosphere pleasant Diffusion Model (AT-EDM) framework that leverages consideration maps to hold out run-time pruning of redundant tokens, with out the need for any retraining. Notably, for single-denoising-step pruning, we develop a novel score algorithm, Generalized Weighted Internet web page Rank (G-WPR), to ascertain redundant tokens, and a similarity-based restoration methodology to revive tokens for the convolution operation. In addition to, we advise a Denoising-Steps-Acutely aware Pruning (DSAP) methodology to manage the pruning funds all through fully totally different denoising timesteps for greater know-how top quality. Intensive evaluations current that AT-EDM performs favorably in the direction of prior art work in relation to effectivity (e.g., 38.8% FLOPs saving and as a lot as 1.53x speed-up over Safe Diffusion XL) whereas sustaining virtually the equivalent FID and CLIP scores as the entire model. Problem webpage: https://atedm.github.io.