loss=NAN

[lq-blackcat]

loss=NAN

[haoningwu3639]

Well, in my mind, we have not encountered such a problem before.
In my opinion, I suggest you (1) use the pre-trained parameters in Stable Diffusion to initialize the new module; (2) try to use float32 training; (3) check whether inf or nan appears during the forward process.
The training process of the diffusion model is relatively stable, and nan situations usually do not occur. If it still cannot be solved, you can continue to communicate with us.

[lq-blackcat]

WARNING:main:Could not enable memory efficient attention. Make sure xformers is installed correctly and a GPU is available: No such operator xformers::efficient_attention_forward_generic - did you forget to build xformers with python setup.py develop?
The issue occurs when I use xformers==0.0.13, and when I use xformers==0.0.16, I encounter a problem with loss=nan.

[Verg-Avesta]

I believe this issue(No such operator xformers::efficient_attention_forward_generic) is due to incompatibilities between xformers and versions of other libraries, such as CUDA or diffusers. Please carefully compare the versions of each library in your environment with those we recommend to see if there are any discrepancies.