# Loss doesn't converge for latent diffusion model

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floss-doesnt-converge-for-latent-diffusion-model%2F56262%3E%5C
**Category:** 🧨 Diffusers\
**Created:** [September 25, 2023, 9:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floss-doesnt-converge-for-latent-diffusion-model%2F56262 "2023-09-25T09:34:22Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![NavneetSajwan](https://avatars.discourse-cdn.com/v4/letter/n/b5a626/32.png) [@NavneetSajwan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FNavneetSajwan)%5C
**Post date:** [September 25, 2023, 9:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floss-doesnt-converge-for-latent-diffusion-model%2F56262%2F1 "2023-09-25T09:34:22Z")

</div>

I have been trying to train a latent diffusion model, but the loss doesn’t seem to converge. I took inspiration from [this](https://huggingface.co/docs/diffusers/tutorials/basic_training) example and modified it to input latents instead of images.

Link to my kaggle notebook: [Latent diffusion for Monets[Multi-gpu, high res] | Kaggle](https://www.kaggle.com/code/navneetsajwan/latent-diffusion-for-monets-multi-gpu-high-res)  
Things i’ve tried so far:

- model architecture: Since the latents input the unet are much smaller in size comapred to actual images for which the unet was built for, the middle layers might have very small size feature maps. And this might hamper the model’s ability to learn. So , i decreased some layers, didn’t work. In fact average loss increased.