[question] Hyperparameter tuning - single process

[dlr-rm]

[question] Hyperparameter tuning - single process

[Miffyli]

I will let @araffin provide answers for this one, but I highly recommend fixing the limitation in your environment. Being able to run RL experiments/optimization in parallel is a must for doing proper results in reasonable time, unless your environment is very fast (above 10000 steps-per-second).

[NicolasHaeffner]

Thanks for your quick response.
That's what I suspected (feared).
I am trying to come up with something clever, but the simulation software I use is giving me a hard time - on that note: it is not very fast either.. more like 100 steps per second

[araffin]

Does anybody see any workaround for that?

What about distributed hyperparameter optimization?

See Readme and Optuna documentation.

[NicolasHaeffner]

Unfortunately, that's not really an option either, due to licensing limitations.
I am using this.
I can train the agent on one environment no problem, but as it tries to spawn more environments it breaks.

[Miffyli]

There is no other workaround for this sadly :(. You could run the hyperparameter optimization with one environment alone but this will take a very long time.

[NicolasHaeffner]

I have spent some more time trying to get the environment to parallelize and I've had some minor successes, so that I think it might be possible somehow. (The error messages bikesim is giving me are not really descriptive...)
Thank you both for your inputs, I really appreciate!