Forever loading module and connection error

[f1am3d]

Forever loading module and connection error

[monorimet]

several hours? this is a known driver issue that usually resolves in 8-20mins.
Can you share more info about hardware/versions so we can try to recreate this? Thanks

[f1am3d]

@monorimet
Yes. Latest AMD driver to date: 23.8.1

OS: Windows 11
GPU: Radeon 7900 XTX
CPI: AMD 7950X

[f1am3d]

Here are the logs, which appear after I press the button "Generate Image(s)":

Found device AMD Radeon RX 7900 XTX. Using target triple rdna3-7900-windows.
Using tuned models for stabilityai/stable-diffusion-2-1(fp16) on device vulkan://00000000-0300-0000-0000-000000000000.
No internet connection. Using the model already present in the tank.
Model artifacts for euler_scale_model_input_fp16 found at C:\Users\f1am3d\.local/shark_tank/...
No internet connection. Using the model already present in the tank.
Verifying that model artifacts were downloaded successfully to C:\Users\f1am3d\.local/shark_tank/euler_scale_model_input_fp16_torch\euler_scale_model_input_fp16_torch.mlir...
loading existing vmfb from: C:\Users\f1am3d\Downloads\stable-diffusion\shark\euler_scale_model_input_fp16.vmfb
WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\euler_scale_model_input_fp16.vmfb...
No internet connection. Using the model already present in the tank.
Model artifacts for euler_step_fp16 found at C:\Users\f1am3d\.local/shark_tank/...
No internet connection. Using the model already present in the tank.
Verifying that model artifacts were downloaded successfully to C:\Users\f1am3d\.local/shark_tank/euler_step_fp16_torch\euler_step_fp16_torch.mlir...
loading existing vmfb from: C:\Users\f1am3d\Downloads\stable-diffusion\shark\euler_step_fp16.vmfb
Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\euler_step_fp16.vmfb...
use_tuned? sharkify: True
_1_64_512_512_fp16_tuned_stable-diffusion-2-1-base
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
No internet connection. Using the model already present in the tank.
Model artifacts for clip_1_64_512_512_fp16_stable-diffusion-2-1-base_vulkan found at C:\Users\f1am3d\.local/shark_tank/...
No internet connection. Using the model already present in the tank.
Verifying that model artifacts were downloaded successfully to C:\Users\f1am3d\.local/shark_tank/clip_1_64_512_512_fp16_stable-diffusion-2-1-base_vulkan\clip_1_64_512_512_fp16_stable-diffusion-2-1-base_vulkan_torch.mlir...
loading existing vmfb from: C:\Users\f1am3d\Downloads\stable-diffusion\shark\clip_1_64_512_512_fp16_stable-diffusion-2-1-base_vulkan.vmfb
Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\clip_1_64_512_512_fp16_stable-diffusion-2-1-base_vulkan.vmfb...
stablediffusion/tuned/stablediffusion/v2_1base/unet/fp16/length_64/tuned is not present in the models database
download pipeline failed, falling back to import_mlir
Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\unet_1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan.vmfb...
::: Detailed report (took longer than 2.5s):
  +1.039266586303711ms: get_iree_runtime_config
  +3.540515899658203ms: mmap C:\Users\f1am3d\Downloads\stable-diffusion\shark\unet_1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan.vmfb
  +3.540515899658203ms: ireert.SystemContext created
  +12555.148601531982ms: module initialized
50it [00:03, 16.37it/s]
stablediffusion/tuned/stablediffusion/v2_1base/vae/fp16/length_77/tuned is not present in the models database
download pipeline failed, falling back to import_mlir
Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\vae_1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan.vmfb...

[monorimet]

This message happens to be the last one that prints before the generated image is returned. Does it ever generate an image? Also, would you mind trying another model to see if you experience the issue there as well?

[f1am3d]

This message happens to be the last one that prints before the generated image is returned. Does it ever generate an image? Also, would you mind trying another model to see if you experience the issue there as well?

Yes, it generates some mess.

I will try other models and will let you know.

[f1am3d]

@monorimet Okay, here are some results so far:

1. anything v3.0 (not working)
2. stable-diffustion-2-1-base (not working)
3. stable-diffusion-2-1 (not working)
4. open journey (working!)

Working log:

To create a public link, set `share=True` in `launch()`.
Found device AMD Radeon RX 7900 XTX. Using target triple rdna3-7900-windows.
Using tuned models for prompthero/openjourney(fp16) on device vulkan://00000000-0300-0000-0000-000000000000.
loading existing vmfb from: C:\Users\f1am3d\Downloads\stable-diffusion\shark\euler_scale_model_input_1_512_512_vulkan_fp16.vmfb
WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\euler_scale_model_input_1_512_512_vulkan_fp16.vmfb...
loading existing vmfb from: C:\Users\f1am3d\Downloads\stable-diffusion\shark\euler_step_1_512_512_vulkan_fp16.vmfb
Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\euler_step_1_512_512_vulkan_fp16.vmfb...
use_tuned? sharkify: True
_1_64_512_512_fp16_tuned_openjourney
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\clip_1_64_512_512_fp16_tuned_openjourney_vulkan.vmfb...
Downloading (…)ch_model.safetensors: 100%|████████████████████████████████████████| 3.44G/3.44G [02:20<00:00, 24.5MB/s]
huggingface_hub\file_download.py:133: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\f1am3d\.cache\huggingface\hub. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
mat1 and mat2 shapes cannot be multiplied (128x1024 and 768x320)
Retrying with a different base model configuration
Loading Winograd config file from  C:\Users\f1am3d\.local/shark_tank/configs\unet_winograd_vulkan.json
100%|█████████████████████████████████████████████████████████████████████████████████| 107/107 [00:00<00:00, 9.30kB/s]
Loading lowering config file from  C:\Users\f1am3d\.local/shark_tank/configs\unet_v1_4_64_fp16_vulkan_rdna3_512x512.json
100%|█████████████████████████████████████████████████████████████████████████████| 24.2k/24.2k [00:00<00:00, 43.6kB/s]
Applying tuned configs on unet_1_64_512_512_fp16_tuned_openjourney_vulkan
No vmfb found. Compiling and saving to C:\Users\f1am3d\Downloads\stable-diffusion\shark\unet_1_64_512_512_fp16_tuned_openjourney_vulkan.vmfb
Configuring for device:vulkan://00000000-0300-0000-0000-000000000000
Using target triple -iree-vulkan-target-triple=rdna3-7900-windows from command line args
Saved vmfb in C:\Users\f1am3d\Downloads\stable-diffusion\shark\unet_1_64_512_512_fp16_tuned_openjourney_vulkan.vmfb.
Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\unet_1_64_512_512_fp16_tuned_openjourney_vulkan.vmfb...
::: Detailed report (took longer than 2.5s):
  +2.9997825622558594ms: get_iree_runtime_config
  +13.107061386108398ms: mmap C:\Users\f1am3d\Downloads\stable-diffusion\shark\unet_1_64_512_512_fp16_tuned_openjourney_vulkan.vmfb
  +17.12322235107422ms: ireert.SystemContext created
  +12111.94920539856ms: module initialized
50it [00:03, 13.52it/s]
Downloading (…)main/vae/config.json: 100%|█████████████████████████████████████████████| 547/547 [00:00<00:00, 547kB/s]
Downloading (…)ch_model.safetensors: 100%|██████████████████████████████████████████| 335M/335M [00:35<00:00, 9.32MB/s]
Loading Winograd config file from  C:\Users\f1am3d\.local/shark_tank/configs\vae_winograd_vulkan.json
100%|███████████████████████████████████████████████████████████████████████████████| 33.0/33.0 [00:00<00:00, 3.28kB/s]
Loading lowering config file from  C:\Users\f1am3d\.local/shark_tank/configs\vae_fp16_vulkan_rdna3.json
100%|██████████████████████████████████████████████████████████████████████████████| 2.08k/2.08k [00:00<00:00, 193kB/s]
Applying tuned configs on vae_1_64_512_512_fp16_tuned_openjourney_vulkan
No vmfb found. Compiling and saving to C:\Users\f1am3d\Downloads\stable-diffusion\shark\vae_1_64_512_512_fp16_tuned_openjourney_vulkan.vmfb
Configuring for device:vulkan://00000000-0300-0000-0000-000000000000
Using target triple -iree-vulkan-target-triple=rdna3-7900-windows from command line args
Saved vmfb in C:\Users\f1am3d\Downloads\stable-diffusion\shark\vae_1_64_512_512_fp16_tuned_openjourney_vulkan.vmfb.
Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\vae_1_64_512_512_fp16_tuned_openjourney_vulkan.vmfb...
50it [00:02, 16.67it/s]

[f1am3d]

Okay, I figured this out. Now it simply writes the message

Loading module C:\Users\f1am3d\Downloads\stable-diffusion\shark\vae_1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan.vmfb...

But the module is actually loaded. It's confusing, since it doesn't output any message after that, like "done" or "completed".
Anyway, most of the models are working, however all basic stable diffusion models are producing weird results, as I mentioned here.