albert-z-guo/deep-reinforcement-stock-trading

A light-weight deep reinforcement learning framework for portfolio management. This project explores the possibility of applying deep reinforcement learning algorithms to stock trading in a highly modular and scalable framework.

deep-reinforcement-stock-trading's Issues

why don't agent balance learn slow on DDPG

Hi Albert,

I am trying to learn a DDPG model, and I refer the code from your GitHub repository, but I always get a large gap between the reward and the real price.
Can you do me a favor?
I would like to know why the learning process of DDPG is very slow and the reward balance is so small?

Thanks for your reply in advanced!

Best regards,
CCC

Loss is in millions ...experience_replay()

Hello Albert,
Please pardon this note as an issue, new to github, not sure where else i can ask a question.

I am trying to run your DQN model, and i find that the loss value reported is very high, not sure how to read the loss...
I have printed the following in the training.py file -

print(e, loss, action_dict[action], reward, agent.balance, len(agent.inventory),len(agent.memory),agent.buffer_size)
logger.info('Episode {}\tLoss: {:.2f}\tAction: {}\tReward: {:.2f}\tBalance: {:.2f}\tNumber of Stocks: {}'.format(e, loss, action_dict[action], reward, agent.balance, len(agent.inventory)))

Could you please clarify about the loss ?

Thank You
Arjun

Why don't save critic model weight on DDPG?

Hi, Albert

Could you tell me why don't save critic model weight on DDPG ?

how to save DDQN model

Hi! Thank you for your last answer!
Recently I try to train the DDQN in your project,so I write" if model_name == 'DDQN':
agent.model.save('saved_models/DDQN_ep' + str(e) + '.h5')"

But when I try to run "evaluate" the project reminded me "SavedModel file does not exist ".
I found that in DDQN.py there should be a file named"DDQN_target.h5". So could you tell me how to write exact code to save the model?
Thank you very much!!!

Save DDQN

Hi! Thank you for your answering!
I know what you mean,but my problem is that I modify model file names there like
if model_name == 'DDQN':
agent.model.save('saved_models/DDQN_ep' + str(e) + '.h5')
but the evaluate.py needs two files,one is"DDQN_ep10.h5", another is "DDQN_ep10_target.h5". I don't know how to save the "DDQN_ep10_target.h5",I can only save "DDQN_ep10.h5".
So I want to know how to do it.
Hope you can answer me ~
Sincerely！

when running evaluate with DDPG always choose sell

when I was running "evaluate.py" with DDPG ,agent always choose "sell"(action = 2).Could you tell me how to solve?

Memory issue

Greetings,
this seems a robust project. Unfortunately, I had several crashes because of lack of memory and never could finish even a episode in the training
:(

Saved models are only working on ^DJI_2016 stock

I get error or session ends abruptly in google collab hence I am not able to train DQN on other stocks and hence I am using your saved models. Only able to train till 2 episodes with 1 year of stock data

Your DQN saved models are only producing profits on stock ^DJI_2016 and not on any other stocks ( not even on ^DJI_2010-2015 )

I have tried evaluating DQN on other stocks such as Alphabet Inc , Shanghai Composite etc still model produces only a straight line of portfolio value

I have tried changing initial portfolio value as well from 50k to 500k and 5k still to no avail .

What should I do ?

Please help !!

Thanks Raghav