B.A. in Linguistics
B.S. in Computer Science and Engineeringmiroblog / tf_deep_rl_trader Goto Github PK
View Code? Open in Web Editor NEWTrading Environment(OpenAI Gym) + PPO(TensorForce)
Trading Environment(OpenAI Gym) + PPO(TensorForce)
B.A. in Linguistics
B.S. in Computer Science and EngineeringHi, thank you again for your work.
I have tried to put the number of trades inside the episode_finished function, but i cant find a good solution to do it. This is what I tried :
def episode_finished(r):
reward = "%.6f" % (r.episode_rewards[-1])
print("Finished episode {ep} of {totalep} after {ts} timesteps (reward: {reward})".format(ep=r.episode, ts=r.episode_timestep,
reward=reward, totalep=episodes_number))
#print("with {l} long position and {s} short".format(l=long, s=short))
#print("with {l} long position and short".format(l=n_long))
#print("info:", info)
if np.mean(r.episode_rewards[-1]) > 0 :
r.agent.save_model(SAVE_DIR, append_timestep=False)
return True
first I want to thank you for your great share. It very rare to find trading reinforcement learning system with ppo.
I have an error when I run this code.
SInce i dont have talib installed i replaced this line :
import talib
by this
import ta as talib
I have then an error when i launch the code :
render() got an unexpected keyword argument 'close'
from core.py line 164 and 203
and closer line 67
how can i fix it?
Thank you,
Greg
Your test and train datasets are the same
I'm loving working with your project.
I was wondering if you might be able to help me with something. I'm wanting to adapt your model to be able to run over multiple datasets. I have lots of different .CSV files containing the 5 minute historical data of many different stocks, however each dataset file is not very big. Each one only contains one trading day's worth of historical data. I'm basically trying to implement your reinforcement learning model on a daytrading type basis.
So, how might one modify your model to be able to train on multiple different datasets at once, when each dataset is fairly small, depicting past stocks that were day traded on in the past? Could it somehow loop through every file in the train and test folders, or would it be best to compile it into one dataset, and somehow have the model recognize a different ticket symbol so that it doesn't try to learn a pattern of pricing if the prices for the different tickers are all over the place. Would love your feedback! Thanks!
I went deeper in the code but their is something i cant understand.
In the file TFTraderEnv.py
I found this line inside the OhlcvEnv() class:
self.shape = (self.window_size, self.n_features+4)
I cant find those 4 features you add just before the training.
Where are they in the code?
Thank you for your awesome work.
greg
Traceback (most recent call last):
File "ppo_trader.py", line 134, in
main()
File "ppo_trader.py", line 131, in main
test_runner.run(num_episodes=1, deterministic=True, testing=True, episode_finished=print_simple_log)
File "/home/kedarnath/anaconda3/envs/DeepRLTrader/lib/python3.6/site-packages/tensorforce/execution/runner.py", line 104, in run
state, terminal, step_reward = self.environment.execute(action=action)
File "/home/kedarnath/Desktop/tf_deep_rl_trader-master/tf_deep_rl_trader-master/env/gymWrapper.py", line 69, in execute
state, reward, terminal, _ = self.gym.step(action)
File "/home/kedarnath/Desktop/tf_deep_rl_trader-master/tf_deep_rl_trader-master/env/TFTraderEnv.py", line 82, in step
s, r, d, i = self._step(action)
File "/home/kedarnath/Desktop/tf_deep_rl_trader-master/tf_deep_rl_trader-master/env/TFTraderEnv.py", line 156, in _step
self.n_short))
FileNotFoundError: [Errno 2] No such file or directory: './info/ppo_1579793.4444944328_LS_0_0.info'
The latest version of pandas 1.0.3 does not contain the pandas.stats section. Will there be any improvements in github?
I can duplicate your result of episode 100 (1000000 -> 1586872.177567203 ), but I try to episode 200,300,500... , the result became worse , it make a loss .
My question is
1.How improve it , more samples , feature , episode , window_size or other?
2.How can I check the algorithmic is correct and bug free ?
2019-08-02 02:18:35.623360: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open s
hared object file: No such file or directory
2019-08-02 02:18:35.623434: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open s
hared object file: No such file or directory
2019-08-02 02:18:35.623505: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open sha
red object file: No such file or directory
2019-08-02 02:18:35.623560: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open s
hared object file: No such file or directory
2019-08-02 02:18:35.623624: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot op
en shared object file: No such file or directory
2019-08-02 02:18:35.623686: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot op
en shared object file: No such file or directory
2019-08-02 02:18:35.623750: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared ob
ject file: No such file or directory
2019-08-02 02:18:35.623768: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...
2019-08-02 02:18:35.624180: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-08-02 02:18:35.723627: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least o
ne NUMA node, so returning NUMA node zero
2019-08-02 02:18:35.724424: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x564b519a8e50 executing computations on platform CUDA. Devices:
2019-08-02 02:18:35.724468: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): Tesla V100-SXM2-16GB, Compute Capability 7.0
2019-08-02 02:18:35.727145: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200000000 Hz
2019-08-02 02:18:35.727827: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x564b514d92b0 executing computations on platform Host. Devices:
2019-08-02 02:18:35.727870: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): ,
2019-08-02 02:18:35.727997: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-08-02 02:18:35.728024: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]
2019-08-02 02:18:36.257449: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu
_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilat
ion_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
start episode ... XBTUSD_5m_16000_train.csv at 7963
Tick: 8000/ Portfolio (krw-won): 984243.3935784912
Long: 3/ Short: 7
Tick: 8100/ Portfolio (krw-won): 936529.4092928643
Long: 12/ Short: 21
Tick: 8200/ Portfolio (krw-won): 915573.1424759856
Long: 22/ Short: 35
Tick: 8300/ Portfolio (krw-won): 862624.4792269642
Long: 35/ Short: 45
Tick: 8400/ Portfolio (krw-won): 878349.9144196685
Long: 49/ Short: 52
Tick: 8500/ Portfolio (krw-won): 863488.3085565092
Long: 60/ Short: 61
Tick: 8600/ Portfolio (krw-won): 776054.9662000438
Long: 71/ Short: 72
Tick: 8700/ Portfolio (krw-won): 763382.4955556361
Long: 82/ Short: 83
Tick: 8800/ Portfolio (krw-won): 734898.3695485827
Long: 90/ Short: 92
Tick: 8900/ Portfolio (krw-won): 685093.6483718295
Long: 103/ Short: 98
Segmentation fault (core dumped)
Any idea?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.