[Bug]: When FP8 is enabled, Only one lora applied

[Bocchi-Chan2023]

[Bug]: When FP8 is enabled, Only one lora applied

[Bocchi-Chan2023]

ok, is this fine?
https://www.mediafire.com/file/0e4knd012wv7xg0/Lora.zip/file

[w-e-w]

good / bad news
I got a reply from KohakuBlueleaf (who implemented the whole FP8 stuff)
and apparently this is a known issue

good news because it's a know issue
bad news because theres not fix eta

[w-e-w]

I cannot reproduce the issue

my test
using xyz to generate 10 images
lora using prompt s/r: Lora A + Lora B, Lora B + Lora A, Lora A, Lora B, blank (no lora)
FP8: disable, enable

• since the order of lora dosen't matter, so Lora A + Lora B, Lora B + Lora A should produce the same image
• comparing blank with the two single lora example are diffrent so we can see the lora is working
• we can see the behavior is consistent with or without FP8

xyz output

[Bocchi-Chan2023]

I cannot reproduce the issue

my test using xyz to generate 10 images lora using prompt s/r: Lora A + Lora B, Lora B + Lora A, Lora A, Lora B, blank (no lora) FP8: disable, enable

• since the order of lora dosen't matter, so Lora A + Lora B, Lora B + Lora A should produce the same image
• comparing blank with the two single lora example are diffrent so we can see the lora is working
• we can see the behavior is consistent with or without FP8

Are you on Dev branch?
anyway I'll upload images to explain how "no longer applicable"

[w-e-w]

Are you on Dev branch?

yes of course

the use of the term "no longer applicable" is very strange
I think you're trying to say "no longer applied"

---

I eddited your post and reply to save space

I remember we did wrote in our instructions to "upload" this is sysinfo as opposed to post it in plain text

[Bocchi-Chan2023]

yeah that's correct! sorry, my english is too bad ;(

[Bocchi-Chan2023]

prompt: score 9, score 8 up, score 7 up, source anime BREAK 1girl, inari_one_(umamusume), handy_fan
negative: monochrome, greyscale, 3d, realistic, source pony, source furry, source cartoon,
base model: ponyxl v6
seed:2476755399
DPM++ 3M SDE Karras / 20steps CFG 7
used loras: https://we.tl/t-tMGd8AMWi3

My results are these. (Sorry for the simplicity. I will provide more information later)

[Bocchi-Chan2023]

The test above is with cache fp16 enabled. Even when disabled, the two LoRAs do not seem to be applied simultaneously. (I will post a comparison image later.)

[Bocchi-Chan2023]

I'm sorry! I forgot an important piece of information!
As shown below, FP8 appears to break several LoRAs. I have just looked into the details and found that a similar issue was published in January of this year.
#14777

score 9, score 8 up, score 7 up, source anime BREAK 1girl, inari_one_(umamusume), handy_fan
<lora:Inari_one_pony_v3:1>
<lora:pony_handy_fan_r:1>
Negative prompt: monochrome, greyscale, 3d, realistic, source pony, source furry, source cartoon,
Steps: 20, Sampler: DPM++ 3M SDE, Schedule type: Karras, CFG scale: 7, Seed: 2476755399, Size: 1024x1024, Model hash: 67ab2fd8ec, Model: ponyDiffusionV6XL_v6StartWithThisOne, FP8 weight: Enable, Cache FP16 weight for LoRA: True, Clip skip: 2, Hypertile VAE: True, Lora hashes: "Inari_one_pony_v3: 7c1e0c59bfe8, pony_handy_fan_r: 124e98b4a464", Emphasis: No norm, Version: v1.9.4-160-ge33bb8fe

Networks with errors: Inari_one_pony_v3 (32), pony_handy_fan_r (32)

[w-e-w]

FP8 is going to change the results, that is to be expected

since your claim is "only the last lora is applyed"
you have to show show that there is meaningful* difference between the results of lora A + lora B and lora B + lora A
otherwise your demonstration is meaningless

make the full demo as like what I did above

meaningful*: I suspect due to precision / rounding / optimization, the results may not be pixel perfect when the apply order changed, but the results should be fairly similar
if there is not a meaningful difference after the apply order is changed then the issue (if exists) is not "only the last lora is applyed"

in my demo xyz

• you can see that there is a clear difference between the output of no lora and the two single lora
• you can also see there is a clear difference between 2 lora results and the single lora resuls
• but these NO meaningful difference between the resuls when the 2 lora apply order is changed
• the above behavior is consistent with or without FP8
  with this information we can determine lora dose work in FP8 regardless of the apply order

whether or not that FP8 breaks certain lora it's not in consideration at the moment
as the issue we are trying to settle here is "only the last lora is applyed"

---

if the issues of "only the last lora is applyed" exist, then you should see that

• resuls with 2 lora will be diffrent
• the 2 lora result should look like the one lora result

if FP8 breaks some lora (break as in no effect) then you should see that

• one of the single lora result should look like the no lora result
• or 2 lora result should look like single lora result

---

the we transfer stuff don't work

[Bocchi-Chan2023]

I understand. I will create a detailed demo when I have time.
I just confirmed that the same image is output when only pony_handy_fan_r is applied and when both (lora:Inari_one_pony_v3:1 lora:pony_handy_fan_r:1) are applied at the same time

[w-e-w]

upload the 2 loras lora:Inari_one_pony_v3 lora:pony_handy_fan_r
see if you can compress them in zip file and upload it hear
if it's too large then use some other method
if you got them from the website like civilai or huggingface then post the links

[w-e-w]

still seems to work fine........
I am still not getting

Networks with errors: Inari_one_pony_v3 (32), pony_handy_fan_r (32)

are you sure that the same issue exist across AMD ROCm and NVIDIA

fp8 lora xyz

I'm asking others to have a look

[Bocchi-Chan2023]

Yes, I have confirmed that it is reproducible on nvidia as well. But yes, it is. I may have a problem with the model of GPU I have, I will buy a new nvidia GPU this month and try it.
(Now tested on RX6600, RTX3060)

[bigmover]

I cannot reproduce the issue

my test using xyz to generate 10 images lora using prompt s/r: Lora A + Lora B, Lora B + Lora A, Lora A, Lora B, blank (no lora) FP8: disable, enable

• since the order of lora dosen't matter, so Lora A + Lora B, Lora B + Lora A should produce the same image
• comparing blank with the two single lora example are diffrent so we can see the lora is working
• we can see the behavior is consistent with or without FP8

xyz output

Fp8 is enabled? How can I use the feature?