Document that “engines” (unets) must be generated for each combination of checkpoint & LoRA

[nvidia]

Document that “engines” (unets) must be generated for each combination of checkpoint & LoRA

[devjo]

If you specify <lora:some_model:0.6> on a checkpoint for which you also have a TensorRT compiled unet that you produced with some_model, then you'll effectively get <lora:some_model:1.6>.
The initial fusing of a lora weights with the checkpoints at TensorRT compilation (generation) time adds some_model at strength 1.0 already to the checkpoint weights, so keeping the <lora:...> expression in the prompt will make the same weights be multiplied again. This is Not what you want.

Just don't specify the <lora:...> bit when using the TensorRT compiled unets and you'll get the lora effect applied anyway, as if you had used the angle bracket syntax <lora:some_model:1.0>.

PS. If you want to fuse the lora with the checkpoint at a strength other than 1.0, a workaround for now is to fuse the lora with the checkpoint manually before doing the checkpoint -> tensorrt conversion. Example.

[FizzleDorf]

this should go a little bit further and explain this needs to be done every time you change LoRA weights as well.

[prusswan]

not sure if the Lora weights are properly applied. I am getting very different results using the TRT version of the Lora model, when compared to the normal version (e.g. <lora:some_model:0.6>).

[prusswan]

@devjo thanks for this clarification, hopefully this will go into the documentation

[bigmover]

If you specify <lora:some_model:0.6> on a checkpoint for which you also have a TensorRT compiled unet that you produced with some_model, then you'll effectively get <lora:some_model:1.6>. The initial fusing of a lora weights with the checkpoints at TensorRT compilation (generation) time adds some_model at strength 1.0 already to the checkpoint weights, so keeping the <lora:...> expression in the prompt will make the same weights be multiplied again. This is Not what you want.

Just don't specify the <lora:...> bit when using the TensorRT compiled unets and you'll get the lora effect applied anyway, as if you had used the angle bracket syntax <lora:some_model:1.0>.

PS. If you want to fuse the lora with the checkpoint at a strength other than 1.0, a workaround for now is to fuse the lora with the checkpoint manually before doing the checkpoint -> tensorrt conversion. Example.

I found the lora "compile" with model A can't be used for model B. Would you mind to share how you solve it?