'alphas_prev' of the DDIM sampler here should start with 1. In the original implementation, 'alpha_prev' starts with 'alphacums[0]', which is close to but not 1.. and is not theoretically correct. 'alphacums_prev[0]' is 1. instead.

This will lead to noisy outputs of the diffusion model. I understand it has limited influence to the LDM framework, since generated latent variables are not directly visible. But I believe the model perform better if modified.