zishunyu/actor-critic-alignment

Implementation of ``Actor-Critic Alignment for Offline-to-Online Reinforcement Learning''

actor-critic-alignment's Issues

Error in Antmaze environments

When I run run_aca.py for antmaze, this error will occur:
AttributeError: module 'd3rlpy.preprocessing.reward_scalers' has no attribute 'ConstantShiftRewardScaler'

My conda environment satisfy all the requirements listed in requirementstxt, maybe the requirement for d3rlpy is wrong?

medium-replay pre-trained models

Thanks to @mikiya1213, it has been brought to my attention that the medium-replay models have incorrect model attribute naming, preventing the online fine-tuning from them. I've re-uploaded the medium-replay models, and have tested on my end.