clementperroud/rl-trading-agent

Profitable Reinforcement Learning Agent on Crypto Markets

rl-trading-agent's Issues

Multiple errors when running notebook

When running the notebook there are several errors, some of which seem like inconsistency between the Rainbow library version and this notebook. Some of the errors are easy to solve such as:

First error

training_envs = gym.vector.SyncVectorEnv([lambda: make_env("/notebooks/data/processed/training/*.pkl") for _ in range(5)])
validation_envs = gym.vector.SyncVectorEnv([lambda: make_env("/notebooks/data/processed/validation/*.pkl") for _ in range(5)])

These lines does not work because the manage_data.ipynb file doesn´t output to training or validation folder.

Second error

env.add_metric('Position Changes', lambda history : f"{ 100*np.sum(np.diff(history['position']) != 0)/len(history['position']):5.2f}%" )
env.add_metric('Max Drawdown', max_drawdown)

These lines throw the error: WARN: env.add_metric to get variables from other wrappers is deprecated and will be removed in v1.0, to get this variable you can do env.unwrapped.add_metric for environment variables or env.get_attr('add_metric') that will search the reminding wrappers.

Changing to:

env.unwrapped.add_metric('Position Changes', lambda history : f"{ 100*np.sum(np.diff(history['position']) != 0)/len(history['position']):5.2f}%" )
env.unwrapped.add_metric('Max Drawdown', max_drawdown)

Fixes it

Third error

'Rainbow' object has no attribute 'e_greedy_pick_actions_or_random'

I think this one is because the method was renamed to e_greedy_pick_actions on this commit

Fourth error

This one I haven´t found how to fix it
could not broadcast input array from shape (15,7) into shape (15,6)

when the following line is ran:
agent.store_replays(obs, actions, rewards, next_obs, dones, truncateds)

P.D. Thanks for the amazing project!

robust_scale is use future prices ?

def add_features(df):
df["feature_close"] = robust_scale(df["close"].pct_change())
df["feature_open"] = robust_scale(df["open"]/df["close"])
df["feature_high"] = robust_scale(df["high"]/df["close"])
df["feature_low"] = robust_scale(df["low"]/df["close"])
df["feature_volume"] = robust_scale(df["volume"] / df["volume"].rolling(7*24).max())
df.dropna(inplace= True) # Clean your data !
return df
I think robust_scale must use future info