Additional Parameters "Scheduler args" NOT Saved in Toml file AND NOT passed

[rafstahelin]

Additional Parameters "Scheduler args" NOT Saved in Toml file AND NOT passed

[bmaltais]

Ok, I can work with that to reproduce the issue and find a fix… hope it won’t lead to a rat’s nest if code change.

[rafstahelin]

ok, i did some digging. Seems like the problem is talked about here:
https://github.com/kohya-ss/sd-scripts/issues/601#issuecomment-1605241097

Seems like additional parameters needs to be passed as str rather than list in toml

I am now running these additional args in cli since they are ignored in toml

@bmaltais is there any way to adjust this?

[bmaltais]

Can you provide an example of that specific toml variable as what it is setting it as now... and what it should be set to?

I should be able to transform to the right format based on the example...

[rafstahelin]

Going to close this issue:

• My goal is to pass --lr_scheduler_type "CosineAnnealingLR" --lr_scheduler_args "T_max=250" through GUI
• BUT it appears this can only be passed through CLI because it is sd-scripts in origin
• Hence, there is no point to pass it through the GUI Additional Parameters

Thanks @bmaltais

[rafstahelin]

@bmaltais
let me rephrase. I'd like to pass:
--lr_scheduler_type "CosineAnnealingLR" --lr_scheduler_args "T_max=250" in GUI

But at the moment, it only works in CLI. If you place it in GUI Additional Parameters it wont work

See:

@DarkAlchy Just pass --lr_scheduler_type and --lr_scheduler_args to command line.

Most of schedulers in torch.optim.lr_scheduler can be imported directly.

For example:
accelerate launch "train_network.py" ... --lr_scheduler_type "CosineAnnealingLR" --lr_scheduler_args "T_max=100"

Here's the thread: https://github.com/kohya-ss/sd-scripts/issues/601#issuecomment-1602496423

Seems like these args need to be passed as string like in CLI, but toml is list.

---

I don't know if this is the answer, but not sure it was ever addressed:
https://github.com/kohya-ss/sd-scripts/issues/601#issuecomment-1605241097

@DarkAlchy Oh, my fault.

It seems that args.blahblah just passed str instead of a list through .toml.

We should pass a list through .toml:
lr_scheduler_args = ['T_max=100', ...]

Could you somehow run the arg in CLI (necessary) but place it in toml in a way that it passes to wandb at the same time?

Does this make sense.

2 results:

1. additional parameter scheduling works through GUI
2. . the parameter passes to wandb

[rafstahelin]

Ok, I can work with that to reproduce the issue and find a fix… hope it won’t lead to a rat’s nest if code change.

I know everyone using scheduling args through GUI and that want to pass to wandb will be appreciative

[bmaltais]

OK, It is now available in the dev branch:

I have added it to the ./test/config/dreambooth-AdamW8bit-toml.json test config so you can easilly see it in action.

The dropdown list allow you to type custom scheduler names as the list is very short. If you know of other choices I can easilly add them to the list.

[rafstahelin]

@bmaltais confirming, this is working perfectly. Thanks!