Comments (41)
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
- In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
- Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
- Run webui.bat - this should rebuild the virtual environment venv
- When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
- enter: venv\Scripts\activate.bat
- the command line should now have (venv) shown at the beginning.
- enter the following commands:
-
- python.exe -m pip install --upgrade pip
-
- python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
-
- python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
-
- python -m pip uninstall -y nvidia-cudnn-cu11
-
- venv\Scripts\deactivate.bat
- webui.bat
- Install the TensorRT extension using the Install from URL option
- Once installed, go to the Extensions >> Installed tab and Apply and Restart
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
- In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
- Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
- Run webui.bat - this should rebuild the virtual environment venv
- When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
- enter: venv\Scripts\activate.bat
- the command line should now have (venv) shown at the beginning.
- enter the following commands:
- python.exe -m pip install --upgrade pip
- python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
- python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
- python -m pip uninstall -y nvidia-cudnn-cu11
- venv\Scripts\deactivate.bat
- webui.bat
- Install the TensorRT extension using the Install from URL option
- Once installed, go to the Extensions >> Installed tab and Apply and Restart
Thanks for the detailed steps, much appreciated. It worked!!
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
- In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
- Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
- Run webui.bat - this should rebuild the virtual environment venv
- When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
- enter: venv\Scripts\activate.bat
- the command line should now have (venv) shown at the beginning.
- enter the following commands:
- python.exe -m pip install --upgrade pip
- python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
- python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
- python -m pip uninstall -y nvidia-cudnn-cu11
- venv\Scripts\deactivate.bat
- webui.bat
- Install the TensorRT extension using the Install from URL option
- Once installed, go to the Extensions >> Installed tab and Apply and Restart
Thanks for the detailed steps, much appreciated. It worked!!
I can't install tensorrt.It give me a error.
note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for tensorrt
My pip is the newest.I don't know what's happen.
Just run cmd in Administrator mode.
from stable-diffusion-webui-tensorrt.
If Nvidia is going to advertise this as a main feature of using their GPU, then they really need to take care of this. I'm reading 4 months worth of threads here all with the same issues everyone else seems to be hitting.
AT LEAST fix the documentation. It's lazy at best. Don't they have enough resources to hire someone to fix and maintain this?
Normally this type of troubleshooting crap wouldn't bother me, but this is from the company that I spent a lot stupid money on for their graphics card. This isn't just some Joe Blow repo, this is advertised on their snobby website, and at all over CSE 2024 That's what makes this so frustrating to me.
Fix the doc, if you think it's good enough, you're an elitist snob and should be ashamed of yourself.
┻━┻ ︵ \( °□° )/ ︵ ┻━┻
"How to use
Click on the “Generate Default Engines” button." oh is that? Should I just click the button? You know, the button. Ya dummies. just click it. Done.
from stable-diffusion-webui-tensorrt.
For me msi rtx 2080 ti gaming x trio
issue
the procedure entry point
?destroyTensorDescriptorEx@...
.....
...venv\Lib\Site-packages\nvidia\cudnn\bin\cudnn_adv_infer64_8.dll
how to fixed
close Stable Diffusion Webui
$> cd stable_diffusion_2080ti(This need change for your sd folder name)\venv\Scripts
$> cmd
$> activate.bat
$> python.exe -m pip install --upgrade pip
$> python -m pip uninstall -y nvidia-cudnn-cu11
open sd not gonna pop the error window again.🚀
from stable-diffusion-webui-tensorrt.
This appears during installation:
*** Error running install.py for extension S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT.
*** Command: "S:\stable-diffusion-webui\venv\Scripts\python.exe" "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py"
*** Error code: 1
*** stderr: Traceback (most recent call last):
*** File "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 3, in <module>
*** from importlib_metadata import version
*** ModuleNotFoundError: No module named 'importlib_metadata'
This happens when starting the web UI:
venv "S:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.8.0
Commit hash: bef51aed032c0aaa5cfd80445bc4cf0d85b408b5
*** Error running install.py for extension S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT.
*** Command: "S:\stable-diffusion-webui\venv\Scripts\python.exe" "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py"
*** Error code: 1
*** stderr: Traceback (most recent call last):
*** File "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 3, in <module>
*** from importlib_metadata import version
*** ModuleNotFoundError: No module named 'importlib_metadata'
Launching Web UI with arguments:
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
*** Error loading script: lora.py
Traceback (most recent call last):
File "S:\stable-diffusion-webui\modules\scripts.py", line 527, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "S:\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
module_spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\lora.py", line 7, in <module>
import onnx
ModuleNotFoundError: No module named 'onnx'
---
*** Error loading script: trt.py
Traceback (most recent call last):
File "S:\stable-diffusion-webui\modules\scripts.py", line 527, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "S:\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
module_spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 8, in <module>
from polygraphy.logger import G_LOGGER
ModuleNotFoundError: No module named 'polygraphy'
from stable-diffusion-webui-tensorrt.
polygraphy
see here: #286
from stable-diffusion-webui-tensorrt.
Ok, si I simply switched to the dev branch and it worked right away !
from stable-diffusion-webui-tensorrt.
*** Error loading script: trt.py
Traceback (most recent call last):
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in
import ui_trt
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in
from exporter import export_onnx, export_trt
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in
from utilities import Engine
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 22, in
import onnx_graphsurgeon as gs
ModuleNotFoundError: No module named 'onnx_graphsurgeon'
What is "onnx_graphsurgeon"? I installe CUDA 11.8... on a 4060 16gb
from stable-diffusion-webui-tensorrt.
Is there a fix for the portable version, this won't work!
from stable-diffusion-webui-tensorrt.
Oh wait, I may have an explaination
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\extensions\Hypernetwork-MonkeyPatch-Extension\patches\external_pr\sd_hijack_checkpoint.py", line 11, in ResBlock_forward
return checkpoint(self._forward, x, emb
)
EDIT: removing Hypernetwork-MonkeyPatch-Extension solved my problem
from stable-diffusion-webui-tensorrt.
For some reason I randomly started getting this error, probably after accutlating to the latest SD version.
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
* In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists * Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
* Run webui.bat - this should rebuild the virtual environment venv * When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
* enter: venv\Scripts\activate.bat * the command line should now have (venv) shown at the beginning. * enter the following commands: * * python.exe -m pip install --upgrade pip * * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir * * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir * * python -m pip uninstall -y nvidia-cudnn-cu11 * * venv\Scripts\deactivate.bat * webui.bat * Install the TensorRT extension using the Install from URL option * Once installed, go to the Extensions >> Installed tab and Apply and Restart
Greetings,
I found this because I've tried multiple times to install this TRT extension while on a Linux OS (Ubuntu-based distro) with zero success. Each time I try to install via URL or from the extensions master list, the extension installation seems to hang, with an endless timer. I navigated to the ...webui/extensions/
folder and see that there is a 39.5 MB folder with the TRT files in there. Though, when I try to run ./webui.sh
, the UI never loads - it just remains in the terminal, also looking as though it hangs.
My questions are: is this the same general issue as what is described in this issue thread? If so, can you provide the same guidance for Linux users? I don't believe that it's a problem with the venv
; it really just seems to be an incomplete install. The reason I don't think this is a problem with the venv
folder, is because I can delete the TRT extension folder and run the UI with no problem right after that deletion.
I am supposing that if I go into the .../venv/lib/python3.10/site-packages/
folder and ran the
**python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
**python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
**python -m pip uninstall -y nvidia-cudnn-cu11
commands, it could be fixed, but I am not sure at all. I also don't know the Linux equivalent to the Windows commands you mentioned before, i.e. enter: venv\Scripts\activate.bat
& venv\Scripts\deactivate.bat
.
Can you please help?
from stable-diffusion-webui-tensorrt.
If Nvidia is going to advertise this as a main feature of using their GPU, then they really need to take care of this. I'm reading 4 months worth of threads here all with the same issues everyone else seems to be hitting. AT LEAST fix the documentation. It's lazy at best. Don't they have enough resources to hire someone to fix and maintain this?
Normally this type of troubleshooting crap wouldn't bother me, but this is from the company that I spent a lot stupid money on for their graphics card. This isn't just some Joe Blow repo, this is advertised on their snobby website, and at all over CSE 2024 That's what makes this so frustrating to me.
Fix the doc, if you think it's good enough, you're an elitist snob and should be ashamed of yourself.
┻━┻ ︵ \( °□° )/ ︵ ┻━┻
"How to use Click on the “Generate Default Engines” button." oh is that? Should I just click the button? You know, the button. Ya dummies. just click it. Done.
+1
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
(...)
* Once installed, go to the Extensions >> Installed tab and Apply and Restart
Hi !
I've managed to get it to work with v1-5-pruned-emaonly.safetensors [6ce0161689], but when I try with sd_xl_base_1.0.safetensors [31e35c80fc], I get the following error (I created a profile for the sd_xl_base_1.0 tensors which I selected in the UI, and I'm using a shape that I generated in the profile) :
*** Error completing request *** Arguments: ('task(l8v3an5pj39id83)', 'one robot on a bike', '', [], 80, 'Euler a', 1, 1, 7, 768, 1024, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000001EBA30067D0>, 0, False, '', 0.8, 194968604, False, -1, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
` (...)
File "C:\Users\nicolas\ai\sd.webui\webui\extensions-builtin\Lora\networks.py", line 486, in network_MultiheadAttention_forward
return originals.MultiheadAttention_forward(self, *args, **kwargs)
File "C:\Users\nicolas\ai\sd.webui\webui\venv\lib\site-packages\torch\nn\modules\activation.py", line 1205, in forward
attn_output, attn_output_weights = F.multi_head_attention_forward(
File "C:\Users\nicolas\ai\sd.webui\webui\venv\lib\site-packages\torch\nn\functional.py", line 5373, in multi_head_attention_forward
attn_output = scaled_dot_product_attention(q, k, v, attn_mask, dropout_p, is_causal)
NameError: name 'scaled_dot_product_attention' is not defined`
actually, when I select the sdxl tensor in the UI, I get this error in the logs :
changing setting sd_model_checkpoint to sd_xl_base_1.0.safetensors [31e35c80fc]: NameError
(...)
File "C:\Users\nicolas\ai\sd.webui\webui\venv\lib\site-packages\torch\nn\functional.py", line 5373, in multi_head_attention_forward
attn_output = scaled_dot_product_attention(q, k, v, attn_mask, dropout_p, is_causal)
NameError: name 'scaled_dot_product_attention' is not defined
and the tensor isn't shown as selected. I have to select it twice for it to stick
I relaunched the webui.bat and now I'm getting this error with sdxl base :
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
- In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
- Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
- Run webui.bat - this should rebuild the virtual environment venv
- When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
- enter: venv\Scripts\activate.bat
- the command line should now have (venv) shown at the beginning.
- enter the following commands:
- python.exe -m pip install --upgrade pip
- python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
- python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
- python -m pip uninstall -y nvidia-cudnn-cu11
- venv\Scripts\deactivate.bat
- webui.bat
- Install the TensorRT extension using the Install from URL option
- Once installed, go to the Extensions >> Installed tab and Apply and Restart
Worked like a charm.
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
- In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
- Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
- Run webui.bat - this should rebuild the virtual environment venv
- When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
- enter: venv\Scripts\activate.bat
- the command line should now have (venv) shown at the beginning.
- enter the following commands:
- python.exe -m pip install --upgrade pip
- python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
- python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
- python -m pip uninstall -y nvidia-cudnn-cu11
- venv\Scripts\deactivate.bat
- webui.bat
- Install the TensorRT extension using the Install from URL option
- Once installed, go to the Extensions >> Installed tab and Apply and Restart
Thanks for the detailed steps, much appreciated. It worked!!
I can't install tensorrt.It give me a error.
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tensorrt
My pip is the newest.I don't know what's happen.
from stable-diffusion-webui-tensorrt.
Hi, my error is "different"
from stable-diffusion-webui-tensorrt.
Installing the latest nvidia drivers was the solution for me.
from stable-diffusion-webui-tensorrt.
HI racerx2oo3,
Thanks so much for this answer. I had a different "not found" problem: Every time I launched webAI-user.bat after installing TensorRt, I got a popup error message that begin: "The procedure entry point ?destroyTensorDescriptorEx..." This message would popup just when the UI was launching and I would have to click multiple times to get rid of it. I followed your directions, and the popup went away. Thanks!
Zaffer
from stable-diffusion-webui-tensorrt.
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
* In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists * Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
* Run webui.bat - this should rebuild the virtual environment venv * When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
* enter: venv\Scripts\activate.bat * the command line should now have (venv) shown at the beginning. * enter the following commands: * * python.exe -m pip install --upgrade pip * * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir * * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir * * python -m pip uninstall -y nvidia-cudnn-cu11 * * venv\Scripts\deactivate.bat * webui.bat * Install the TensorRT extension using the Install from URL option * Once installed, go to the Extensions >> Installed tab and Apply and Restart
there is no venv folder in portable version. I followed all steps but it does not install. Gets stuck in "onnx_graphsurgeon" step. Nvidia should fix it, this is a mess.
from stable-diffusion-webui-tensorrt.
from stable-diffusion-webui-tensorrt.
Closing duplicates + Seems to be resolved.
from stable-diffusion-webui-tensorrt.
I'm facing the same issue. I deleted the VENV dir and followed the proposed procedure, but the output remains the same
HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
Exporting v1-5-pruned-emaonly to TensorRT
{'sample': [(1, 4, 64, 64), (2, 4, 64, 64), (8, 4, 96, 96)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
No ONNX file found. Exporting ONNX...
Disabling attention optimization
INFO:root:Exporting to ONNX...
Exporting to ONNX...
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
ERROR:root:invalid unordered_map<K, T> key
invalid unordered_map<K, T> key
Traceback (most recent call last):
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 84, in export_onnx
torch.onnx.export(
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 506, in export
_export(
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 1548, in _export
graph, params_dict, torch_out = _model_to_graph(
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 1113, in _model_to_graph
graph, params, torch_out, module = _create_jit_graph(model, args)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 989, in _create_jit_graph
graph, torch_out = _trace_and_get_graph_from_model(model, args)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 893, in _trace_and_get_graph_from_model
trace_graph, torch_out, inputs_states = torch.jit._get_trace_graph(
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\jit\_trace.py", line 1268, in _get_trace_graph
outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\jit\_trace.py", line 127, in forward
graph, out = torch._C._create_graph_by_tracing(
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\jit\_trace.py", line 118, in wrapper
outs.append(self.inner(*trace_inputs))
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1488, in _slow_forward
result = self.forward(*input, **kwargs)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\modules\sd_unet.py", line 91, in UNetModel_forward
return ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui(self, x, timesteps, context, *args, **kwargs)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
h = module(h, emb, context)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1488, in _slow_forward
result = self.forward(*input, **kwargs)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 82, in forward
x = layer(x, emb)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1488, in _slow_forward
result = self.forward(*input, **kwargs)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\extensions\Hypernetwork-MonkeyPatch-Extension\patches\external_pr\sd_hijack_checkpoint.py", line 11, in ResBlock_forward
return checkpoint(self._forward, x, emb)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\utils\checkpoint.py", line 249, in checkpoint
return CheckpointFunction.apply(function, preserve, *args)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\function.py", line 506, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
RuntimeError: invalid unordered_map<K, T> key
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
result = await self.call_function(
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
result = context.run(func, *args)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 135, in export_unet_to_trt
export_onnx(
File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 129, in export_onnx
exit()
File "C:\ProgramData\anaconda3\lib\_sitebuiltins.py", line 26, in __call__
raise SystemExit(code)
SystemExit: None
INFO:httpx:HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 500 Internal Server Error"
HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 500 Internal Server Error"
INFO:httpx:HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
from stable-diffusion-webui-tensorrt.
Looks like some people need to add the line "pip install importlib_metadata" (#157 (comment))
These people may just install "importlib_metadata" and they may be able to use TensorRT normally.
The portable version can also be used (I am), the method is: #27 (comment)
(From Google Translate)
from stable-diffusion-webui-tensorrt.
This is my problem
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
- In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
- Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
- Run webui.bat - this should rebuild the virtual environment venv
- When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
- enter: venv\Scripts\activate.bat
- the command line should now have (venv) shown at the beginning.
- enter the following commands:
- python.exe -m pip install --upgrade pip
- python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
- python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
- python -m pip uninstall -y nvidia-cudnn-cu11
- venv\Scripts\deactivate.bat
- webui.bat
- Install the TensorRT extension using the Install from URL option
- Once installed, go to the Extensions >> Installed tab and Apply and Restart
I successfully fixed my tensorRT shit without deleting vent and my existing vent, I just entered like you said, ran the commands in order, launched webui cringing super hard waiting for the windows error prompt sound to trigger me, but nope, webui loaded, tensorRT tab visible, hypothetically ready? FYI I had no clue this WASNT using tensorRT, im so glad I decided to give the extension tab a look over like wtf. They should really warn you or auto detect if you have an RTX and prompt you that its possible.
from stable-diffusion-webui-tensorrt.
called it vent like 9 times, I meant venv, lol, Anyways thanks a trillion!!
from stable-diffusion-webui-tensorrt.
Responded to the wrong github issue so i had to delete my comment lol
from stable-diffusion-webui-tensorrt.
For those who can't apply this fix I have fully automated installer that includes fix shared here : https://www.patreon.com/posts/86307255
And here a full no paywall tutorial
Double Your Stable Diffusion Inference Speed with RTX Acceleration TensorRT: A Comprehensive Guide
from stable-diffusion-webui-tensorrt.
I followed all the steps and I still get this error message:
AssertionError: Extension directory already exists
The extension folder is empty. Not sure how it stays the directory exists. I even started from scratch and this still happened.
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).* In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists * Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
* Run webui.bat - this should rebuild the virtual environment venv * When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
* enter: venv\Scripts\activate.bat * the command line should now have (venv) shown at the beginning. * enter the following commands: * * python.exe -m pip install --upgrade pip * * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir * * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir * * python -m pip uninstall -y nvidia-cudnn-cu11 * * venv\Scripts\deactivate.bat * webui.bat * Install the TensorRT extension using the Install from URL option * Once installed, go to the Extensions >> Installed tab and Apply and Restart
Greetings,
I found this because I've tried multiple times to install this TRT extension while on a Linux OS (Ubuntu-based distro) with zero success. Each time I try to install via URL or from the extensions master list, the extension installation seems to hang, with an endless timer. I navigated to the
...webui/extensions/
folder and see that there is a 39.5 MB folder with the TRT files in there. Though, when I try to run./webui.sh
, the UI never loads - it just remains in the terminal, also looking as though it hangs.My questions are: is this the same general issue as what is described in this issue thread? If so, can you provide the same guidance for Linux users? I don't believe that it's a problem with the
venv
; it really just seems to be an incomplete install. The reason I don't think this is a problem with thevenv
folder, is because I can delete the TRT extension folder and run the UI with no problem right after that deletion.I am supposing that if I go into the
.../venv/lib/python3.10/site-packages/
folder and ran the**python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir **python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir **python -m pip uninstall -y nvidia-cudnn-cu11
commands, it could be fixed, but I am not sure at all. I also don't know the Linux equivalent to the Windows commands you mentioned before, i.e.
enter: venv\Scripts\activate.bat
&venv\Scripts\deactivate.bat
.Can you please help?
I'm also on Ubuntu, found the issue and created a fix #225
Basically if you're stuck:
- Delete stable-diffusion-webui/venv dir
- Delete existing (if existing) extensions/Stable-Diffusion-WebUI-TensorRT
- Rerun webui.sh to rebuild venv
source venv/bin/activate
in the stable-diffusion-webui dir MAKE ABSOLUTELY SURE THIS IS ENABLED
pip uninstall -y tensorrt
pip install -y nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4 --no-cache-dir
pip uninstall -y nvidia-cudnn-cu11
deactivate
- Proceed with normal installation of Nvidia TensorRT
--- alternative, not recommend unless you know what you're doing
Then you can either apply my fix 7c93523 to https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT/blob/main/install.py and run python install.py
or using the A1111 SD UI install my repo instead https://github.com/andrewtvuong/Stable-Diffusion-WebUI-TensorRT as a temporary work around. Once the main package is fixed, I would uninstall and install the main again.
from stable-diffusion-webui-tensorrt.
well for those who can't still make I made a full tutorial and also auto installers
Double Your Stable Diffusion Inference Speed with RTX Acceleration TensorRT: A Comprehensive Guide
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).* In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists * Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
* Run webui.bat - this should rebuild the virtual environment venv * When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
* enter: venv\Scripts\activate.bat * the command line should now have (venv) shown at the beginning. * enter the following commands: * * python.exe -m pip install --upgrade pip * * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir * * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir * * python -m pip uninstall -y nvidia-cudnn-cu11 * * venv\Scripts\deactivate.bat * webui.bat * Install the TensorRT extension using the Install from URL option * Once installed, go to the Extensions >> Installed tab and Apply and Restart
Greetings,
I found this because I've tried multiple times to install this TRT extension while on a Linux OS (Ubuntu-based distro) with zero success. Each time I try to install via URL or from the extensions master list, the extension installation seems to hang, with an endless timer. I navigated to the...webui/extensions/
folder and see that there is a 39.5 MB folder with the TRT files in there. Though, when I try to run./webui.sh
, the UI never loads - it just remains in the terminal, also looking as though it hangs.
My questions are: is this the same general issue as what is described in this issue thread? If so, can you provide the same guidance for Linux users? I don't believe that it's a problem with thevenv
; it really just seems to be an incomplete install. The reason I don't think this is a problem with thevenv
folder, is because I can delete the TRT extension folder and run the UI with no problem right after that deletion.
I am supposing that if I go into the.../venv/lib/python3.10/site-packages/
folder and ran the**python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir **python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir **python -m pip uninstall -y nvidia-cudnn-cu11
commands, it could be fixed, but I am not sure at all. I also don't know the Linux equivalent to the Windows commands you mentioned before, i.e.
enter: venv\Scripts\activate.bat
&venv\Scripts\deactivate.bat
.
Can you please help?I'm also on Ubuntu, found the issue and created a fix #225
Basically if you're stuck:
- Delete stable-diffusion-webui/venv dir
- Delete existing (if existing) extensions/Stable-Diffusion-WebUI-TensorRT
- Rerun webui.sh to rebuild venv
source venv/bin/activate
in the stable-diffusion-webui dir MAKE ABSOLUTELY SURE THIS IS ENABLEDpip uninstall -y tensorrt pip install -y nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4 --no-cache-dir pip uninstall -y nvidia-cudnn-cu11
deactivate
- Proceed with normal installation of Nvidia TensorRT
--- alternative, not recommend unless you know what you're doing
Then you can either apply my fix 7c93523 to https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT/blob/main/install.py and run
python install.py
or using the A1111 SD UI install my repo instead https://github.com/andrewtvuong/Stable-Diffusion-WebUI-TensorRT as a temporary work around. Once the main package is fixed, I would uninstall and install the main again.
Thank you!
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
- In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
- Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
- Run webui.bat - this should rebuild the virtual environment venv
- When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
- enter: venv\Scripts\activate.bat
- the command line should now have (venv) shown at the beginning.
- enter the following commands:
- python.exe -m pip install --upgrade pip
- python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
- python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
- python -m pip uninstall -y nvidia-cudnn-cu11
- venv\Scripts\deactivate.bat
- webui.bat
- Install the TensorRT extension using the Install from URL option
- Once installed, go to the Extensions >> Installed tab and Apply and Restart
I tried to run this procedure several times, but it doesn't work fro me, I'm still getting this error:
*** Error running install.py for extension F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT.
*** Command: "F:\Stable-Diffusion\webui\venv\Scripts\python.exe" "F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py"
*** Error code: 1
*** stderr: Traceback (most recent call last):
*** File "F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 3, in
*** from importlib_metadata import version
*** ModuleNotFoundError: No module named 'importlib_metadata'
from stable-diffusion-webui-tensorrt.
I followed the instructions and it worked. 100%. In my case, the fifth point of the instructions (venv\Scripts\deactivate.bat) did not work and/or was not found. I then skipped it (omitted it). Thanks for the precise instructions. Now I have the TensorRT tab in the UI.
I previously set the arguments to default (deleted and kept backup). I have no idea if it has any meaning or use, but I also backed up the venv folder before deleting it.
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
- In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
- Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
- Run webui.bat - this should rebuild the virtual environment venv
- When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
- enter: venv\Scripts\activate.bat
- the command line should now have (venv) shown at the beginning.
- enter the following commands:
- python.exe -m pip install --upgrade pip
- python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
- python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
- python -m pip uninstall -y nvidia-cudnn-cu11
- venv\Scripts\deactivate.bat
- webui.bat
- Install the TensorRT extension using the Install from URL option
- Once installed, go to the Extensions >> Installed tab and Apply and Restart
I use the [Stable-Diffusion-WebUI-TensorRT] branch master but get errors caused by dll error,but changed to controlnet_v2 branch get fix it
from stable-diffusion-webui-tensorrt.
I tried to run this procedure several times, but it doesn't work fro me, I'm still getting this error:
*** Error running install.py for extension F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT. *** Command: "F:\Stable-Diffusion\webui\venv\Scripts\python.exe" "F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py" *** Error code: 1 *** stderr: Traceback (most recent call last): *** File "F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 3, in *** from importlib_metadata import version *** ModuleNotFoundError: No module named 'importlib_metadata'
I have the same problem(
I've gone through the procedure several times, but with no success. Even on a freshly installed Automatic1111...
from stable-diffusion-webui-tensorrt.
My rude fix: #143 (comment)
from stable-diffusion-webui-tensorrt.
What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
* In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists * Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
* Run webui.bat - this should rebuild the virtual environment venv * When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
* enter: venv\Scripts\activate.bat * the command line should now have (venv) shown at the beginning. * enter the following commands: * * python.exe -m pip install --upgrade pip * * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir * * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir * * python -m pip uninstall -y nvidia-cudnn-cu11 * * venv\Scripts\deactivate.bat * webui.bat * Install the TensorRT extension using the Install from URL option * Once installed, go to the Extensions >> Installed tab and Apply and Restart
I stumbled upon this in 2024 and used the following commands in case the newer packages help today. Here are the commands I used:
python.exe -m pip install --upgrade pip
python -m pip install nvidia-cudnn-cu12==9.0.0.312 --no-cache-dir
python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.3.0.post12.dev1 --no-cache-dir
python -m pip uninstall -y nvidia-cudnn-cu12
venv\Scripts\deactivate.bat
If you use version 10 of nvidia-cudnn-cu12, you will get an error when trying to use TensorRT later, so this uses the latest version of 9.
You can also use the commands in the 2023 post above with the older packages. The older models worked, but I was getting the below errors after using the older packages, although TensorRT seemed to work:
[E] 1: [myelinCache.cpp::nvinfer1::builder::MyelinAutotunerCache::deserialize::62] Error Code 1: Myelin (Myelin error from unknown graph)
[E] 4: The timing cache will not be used!
I do receive "[E] 4: The timing cache will not be used!" regardless of what packages I use.
from stable-diffusion-webui-tensorrt.
Related Issues (20)
- Does tensorRT support direct use of apis, and what are the parameters? HOT 1
- how to generate json config file in the LoRA tab. HOT 1
- [branch: Controlnet_V2] ONNX IR model version conflict, how do i downgrade?
- [E] 3: getPluginCreator could not find plugin: InstanceNormalization_TRT version: 1 when building engine with latest TRT_VERSION="10.0.0b6"
- No config file found for You can generate it in the LoRA tab. HOT 3
- Advise to add promt token check in warning report
- [Feature Request] Add a Command Line Interface for the exporter process.
- AttributeError: 'tensorrt_bindings.tensorrt.ICudaEngine' object has no attribute 'get_profile_shape' HOT 2
- How to process a batch of images with ControlNet TensorRT in img2img ?
- Latest version of automatic1111 wont start with this extension installed. HOT 2
- AttributeError: 'NoneType' object has no attribute 'device_memory' HOT 12
- Error Code 3: API Usage Error (Parameter check failed
- Export LoRA to onnx depending on cyberrealistic_v31 can't fit to v1-5-pruned-emaonly_cnet_1846e6c3c239531e.trt
- TensorRT doesn't install on fresh A1111 install on Ubuntu, stuck at install after copy/paste de git link HOT 10
- TensorRT doesn't find Loras from subdirectories HOT 2
- TensorTR extension is building engine with wrong parameters
- may a name mistake in beta version (utilities.py)
- Engines not loading with text size larger than 75 HOT 1
- SD3 TensoRRT (sd3-clip-t5xxlfp8) works perfectly but cannot get SDXL ... HOT 1
- ONNX Export Error: Tensors on Different Devices (CPU and CUDA) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stable-diffusion-webui-tensorrt.