bbc-mc/sdweb-merge-board

Multi-step automation merge tool. Extension/Script for Stable Diffusion UI by AUTOMATIC1111 https://github.com/AUTOMATIC1111/stable-diffusion-webui

Hi, I'm bbc_mc (a.k.a bbc-mc on github)

sdweb-merge-board's Issues

Merges Fail, probably because of Subfolders

i'm using subfolders to organize my models and the merge board doesn't seems to work, I've tried only one lane and it still failed

idk, does not work

error: https://pastebin.com/eiT43nSZ

Filenames containing plus signs are not parsed correctly in the recipe tab

When selecting files with plus symbols in their name from the multi merge tab everything works as expected, but sending it to the recipe tab results in recipe that will fail to parse correctly, truncating the last terms from the string since it assumes + is not part of the file name.
If it is unclear, I can generate an example scenario.

[BUG] cant merge anything

Hi, I always have same issue since a couple of last updates :

Starting merge under settings below,
A: Base\EpicSpaceMachine.ckpt [4ff93fa8]
B: Temp\openjourneyAka_v1.ckpt [ddc6edf2]
C:
S: Weighted sum
M: 0.362
F: False
O: test
CF: ckpt
Try to use old 'run_modelmerger' params. 'Checkpoint format is forced to 'ckpt'
<class 'TypeError'>
run_modelmerger() missing 2 required positional arguments: 'checkpoint_format' and 'config_source'
All Multi-Merge process finished. 1 files.
Error

Windows
Firefox

Hash indexing

Is there any way to create a system so merge board knows which merges it has made before and the resulting hashes from those models, so it can then in the future when you run the merge again know to only merge steps that have changed?

Like if i have a merge that looks like

SD 1.5 + Anything 3.0 - SD 1.4
Waifu diffusion + Trinart2
__O1__ + __O2__

if i change the second step to Waifu Diffusion + Artstation Diffusion, it will still perform step 1 even though that merge might still exist in the model directory.

Would make testing new merges and iterating through different variations / weights WAY faster, without having to modify the recipe to point to preexisting merges and keep track of that for every test, or run the same thing but merge unnecessary steps.

New Auto1111 versions are using 256SHA Hash, which breaks merge board and causes other issues

Hello, I really like merge-board., and I appreciate that you invested your time to create something so useful! I use it as part of my work flow and rely on it a lot. A couple of days ago Auto1111 was updated to use SHA256 hash values.

This breaks merge-board and also means there is a disconnect (or will be for many) between the old traditional hash values people share, use online, and store in their .png metadata. The SHA256 hash value is not similar to the old hash values.

I wanted to bring this to you attention, maybe you can update merge board and improve upon it to be a link between the old and new S256SHA hash values?

There is code here that derives the old hash values: https://github.com/etherealxx/oneclick-showmodelhash

Here is the original thread on Auto1111, people were originally reporting it as a bug (myself included): AUTOMATIC1111/stable-diffusion-webui#6761

Here is my [Feature Request] for Auto1111: AUTOMATIC1111/stable-diffusion-webui#6818
I don't know if they will implement anything to bridge the old and new hash values.

Additionally, maybe this will save you some time, but the new hashes are stored in a file called "cache.json" in the root install of Auto1111. The hashes only populate when their representative model is loaded. This means Auto1111 doesn't compute all hash values in the model directory upon startup.

In my opinion, it did all the hash values very quickly, I have a few symbolic links on various drives and a lot of models and it maybe adds only a few seconds to the Auto1111 boot time.

Anyway, I hope you find this useful, and thanks again for making merge-board 🥇

Fix for recent A1111 updates (May 2023)

The addition of the option to save metadata in model merges is what broke merge board.

I have a hackish fix. Hopefully someone with more skill can implement a checkbox in the merge board UI to make it legit.

Add "save_metadata = True" in the run_merge function within merge board's recipe.py file and add save_metadata to the very end of the first extras.run_modelmerger() call. I found that setting the value to True was necessary to avoid issues when saving the model to .safetensors. If you only want to use .ckpt, you can set safe_metadata to False.

Here's my implementation. Starts at line 114 in recipe.py:

 try:
      discard_weights = ""
      save_metadata = True
      bake_in_vae = "None"
      # backward compatibility for change of run_model_merger
      # 2023/01/22
      # https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/112416d04171e4bee673f0adc9bd3aeba87ec71a
      # def run_modelmerger(id_task, primary_model_name, secondary_model_name, tertiary_model_name, interp_method, multiplier, save_as_half, custom_name, checkpoint_format, config_source, bake_in_vae, discard_weights):
      results = extras.run_modelmerger(id_task, self.A, self.B, self.C, self.S, self.M, self.F, self.O, self.CF, config_source, bake_in_vae, discard_weights, save_metadata)

error on first run after extension install

its a simple one - folder extensions/sdweb-merge-board/csv is not present on clean install of extension, so writing history file fails

creating folder manually solves the problem

i assume the issue is that git does not include empty folders in a repository, so when doing a checkout folder is not present
simply do a touch csv/.placeholder so there is a file there and do a commit - or solve it in code with path mkdir if not present

  File "/home/vlado/dev/automatic/extensions/sdweb-merge-board/scripts/multimerge/util/merge_history.py", line 48, in add_history
    with open(self.filepath, "w", newline="", encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/vlado/dev/automatic/extensions/sdweb-merge-board/csv/history.tsv'

[Question] How to change the directory of result file that merged

May I confirm how to change the directory of result file that merged? I want to set results to another folder to ensure materials are separated with results.
Example:
[ckpt file materials folder which auto loaded]:
[path]:/drive/material files

[another folder that use to save merged files]:
[path] /drive/merged files

Error loading/saving model file:

I installed it through the extension tab of A 1111, my models are in the model folder, some are nested in other folders for organization.
i fill out the merge board and im getting this error: Error loading/saving model file: for each row of the mergeboard i filled.

ValueError: not enough values to unpack (expected 87, got 86) when start webui

After I update the script, go an error when start webui

Error executing callback ui_tabs_callback for C:\stable-diffusion-webui\extensions\sdweb-merge-board\scripts\merge_board.py
Traceback (most recent call last):
  File "C:\stable-diffusion-webui\modules\script_callbacks.py", line 95, in ui_tabs_callback
    res += c.callback() or []
  File "C:\stable-diffusion-webui\extensions\sdweb-merge-board\scripts\merge_board.py", line 29, in on_ui_tabs
    btn_run_merge, btn_reload_checkpoints, btn_send_to_recipe, submit_result, btn_clear_merge_settings, \
ValueError: not enough values to unpack (expected 87, got 86)

Does not work on 1.6.0

Throws this error every time on 1.6.0

run_modelmerger() missing 3 required positional arguments: 'add_merge_recipe', 'copy_metadata_fields', and 'metadata_json'
Try to use old 'run_modelmerger' params.
run_modelmerger() missing 4 required positional arguments: 'save_metadata', 'add_merge_recipe', 'copy_metadata_fields', and 'metadata_json'
Try to use old 'run_modelmerger' params.
run_modelmerger() missing 5 required positional arguments: 'discard_weights', 'save_metadata', 'add_merge_recipe', 'copy_metadata_fields', and 'metadata_json'
Try to use old 'run_modelmerger' params.
run_modelmerger() missing 7 required positional arguments: 'config_source', 'bake_in_vae', 'discard_weights', 'save_metadata', 'add_merge_recipe', 'copy_metadata_fields', and 'metadata_json'
Try to use old 'run_modelmerger' params. 'config_source' is ignored
Try to use old 'run_modelmerger' params. 'Checkpoint format is forced to 'ckpt'
run_modelmerger() missing 8 required positional arguments: 'checkpoint_format', 'config_source', 'bake_in_vae', 'discard_weights', 'save_metadata', 'add_merge_recipe', 'copy_metadata_fields', and 'metadata_json'
run_modelmerger() missing 9 required positional arguments: 'custom_name', 'checkpoint_format', 'config_source', 'bake_in_vae', 'discard_weights', 'save_metadata', 'add_merge_recipe', 'copy_metadata_fields', and 'metadata_json'
All Multi-Merge process finished. 1 files.
  Error <TypeError>

New version of gradio breaks everything

I'm not sure if this is really gradio's fault, but even by reverting to a week-old commit, merge board seems to fail, and the only next step would presumably be to revert gradio, which is where I gave up investigating this problem.

Your other extension, merge block, does not have the same problem and works perfectly fine.

With gradio 3.16.2, recently updated in webui's requirements file.

The following error results when trying to parse a recipe:

Traceback (most recent call last):
  File "/nix/store/b526rlzxv4kldhi97xkp38dz9yz7gb4b-python3.10-gradio-3.16.2/lib/python3.10/site-packages/gradio/routes.py", line 337, in run_predict
    output = await app.get_blocks().process_api(
  File "/nix/store/b526rlzxv4kldhi97xkp38dz9yz7gb4b-python3.10-gradio-3.16.2/lib/python3.10/site-packages/gradio/blocks.py", line 1018, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "/nix/store/b526rlzxv4kldhi97xkp38dz9yz7gb4b-python3.10-gradio-3.16.2/lib/python3.10/site-packages/gradio/blocks.py", line 953, in postprocess_data
    assert isinstance(
AssertionError: <class 'gradio.layouts.Row'> Component with id 2364 not a valid output component

The following error results when trying to run a merge:

Start Merge processes. Total process num: 1 .
Starting merge under settings below,
  A: sd15.ckpt
  B: sd14.ckpt
  C:
  S: Weighted sum
  M: 0.5
  F: False
  O: helloworld
 CF: ckpt
Try to use old 'run_modelmerger' params. 'config_source' is ignored
Try to use old 'run_modelmerger' params. 'Checkpoint format is forced to 'ckpt'
<class 'TypeError'>
run_modelmerger() missing 4 required positional arguments: 'custom_name', 'checkpoint_format', 'config_source', and 'bake_in_vae'
All Multi-Merge process finished. 1 files.
  Error

[Feature Request] add progress when merge is running

like the subject line says, display progress in webui while merge is running
currently, it says nothing and allows to click merge button again while previous merge is still running

[Feature Request] Implement Merge Block Presets as interpolation method selections

Would be a really useful if I could select the UNET block merge presets from your other extension as options for inter-method.

Further, I'd really like to see an extension for the rebasin method of merging models, if that's doable?

Error: <class 'KeyError'>

Up-to-date with auto1111 (Commit hash: 194cbd065e4644e986889b78a5a949e075b610e8)
Up-to-date with merge board extension

When running the merge it keeps failing after the first merge with a keyerror...

All Multi-Merge process finished. 10 files.
Merge complete. Checkpoint saved as: [MergeTest-v1-0.ckpt]
Error: <class 'KeyError'>
Error: <class 'KeyError'>
Error: <class 'KeyError'>
Error: <class 'KeyError'>
Error: <class 'KeyError'>
Error: <class 'KeyError'>
Error: <class 'KeyError'>
Error: <class 'KeyError'>
Error: <class 'KeyError'>

If I merge one at time (Skip merge if same-name ckpt already exists), it works but have to delete each one in succession to make it to the end...

[Feature Request] Allow negative M values/ Add Subtract Difference

This can be used to remove content, or style overtraining from a model.
For example, NAI is anime styled but has much greater ability at doing NSFW content, meanwhile WD is SFW mostly and only adds anime/beautification.
If we remove WD from NAI we get a NAI-Tagset model, anything NAI was trained to make besides the anime style can now be injected into other models
For example, here's a recipe for injecting it into the Cafe model:

# Recipe 
NAI_AnimeStyleSurpressed = model + wd-v1-3-float16 + SD-v1.4, -0.35, ckpt
Cafe+NAITagging = cafe-instagram-unofficial-test-epoch-9-140k-images-fp32 + __O1__ + SD-v1.4, 0.5, ckpt

# Variables(System) 
__O1__

# Variables(User defined)

Merge Board Fails to start

I recently updated my A111's webui (01/28/2023) and I noticed this errror. I made sure that the extension was up to date and it still at an issue.

Error executing callback ui_tabs_callback for E:\Downloads\stable-diffusion-webui\extensions\sdweb-merge-board\scripts\merge_board.py
Traceback (most recent call last):
  File "E:\Downloads\stable-diffusion-webui\modules\script_callbacks.py", line 106, in ui_tabs_callback
    res += c.callback() or []
  File "E:\Downloads\stable-diffusion-webui\extensions\sdweb-merge-board\scripts\merge_board.py", line 29, in on_ui_tabs
    btn_run_merge, btn_reload_checkpoints, btn_send_to_recipe, submit_result, btn_clear_merge_settings, \
ValueError: not enough values to unpack (expected 88, got 86)

I love this extension because it makes mixing models so much easier.

[Feature Request] User-specifiable number of stages

Rather than limit to 10 stages, make that user-definable, perhaps by an 'add stage' button after the final stage.