rosdyana/going-deeper-with-convolutional-neural-network-for-stock-market-prediction

Repository for Going Deeper with Convolutional Neural Network for Stock Market Prediction

going-deeper-with-convolutional-neural-network-for-stock-market-prediction's Issues

Format String F not compatible for python 3.5

The file run_binary_preprocessing.py is using Format String F (Literal String Interpolation) that is available just 3.6+ and the project runs on 3.5.

trying to use it and breaks literally every step of the way

would it be possible to get a complete example? I keep running in all kind of issues, from keras warnings, over methods not found errors, missing requirements, etc.

example:

/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/venv/bin/python /home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/myDeepCNN.py -i dataset/20_50/2880.TW -e 50 -d 50 -b 8 -o outputresult.txt
2019-06-19 05:39:27.101687: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-06-19 05:39:27.126885: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2112000000 Hz
2019-06-19 05:39:27.127344: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x1c1c450 executing computations on platform Host. Devices:
2019-06-19 05:39:27.127362: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): ,
Using TensorFlow backend.
loading dataset
train size : 3751
train size : 331
/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/myDeepCNN.py:55: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(32, (3, 3), activation="relu", padding="same", kernel_initializer="glorot_uniform")
number of classes : 331
border_mode='same', activation='relu')(input_layer)
WARNING:tensorflow:From /home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/venv/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/myDeepCNN.py:61: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(48, (3, 3), activation="relu", padding="same", kernel_initializer="glorot_uniform")
activation='relu')(x)
WARNING:tensorflow:From /home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/venv/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use rate instead of keep_prob. Rate should be set to rate = 1 - keep_prob.
/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/myDeepCNN.py:68: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(64, (3, 3), activation="relu", padding="same", kernel_initializer="glorot_uniform")
activation='relu')(x)
/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/myDeepCNN.py:74: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(96, (3, 3), activation="relu", padding="same", kernel_initializer="glorot_uniform")
activation='relu')(x)
/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/myDeepCNN.py:84: UserWarning: Update your Dense call to the Keras 2 API: Dense(activation="relu", units=256)
x = Dense(output_dim=256, activation='relu')(x)
/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/myDeepCNN.py:88: UserWarning: Update your Dense call to the Keras 2 API: Dense(activation="softmax", units=2)
x = Dense(output_dim=2, activation='softmax')(x)
Traceback (most recent call last):
File "/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/myDeepCNN.py", line 195, in
main()
File "/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/myDeepCNN.py", line 137, in main
model.fit(X_train, Y_train, batch_size=batch_size, epochs=epochs)
File "/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/venv/lib/python3.7/site-packages/keras/engine/training.py", line 952, in fit
batch_size=batch_size)
File "/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/venv/lib/python3.7/site-packages/keras/engine/training.py", line 751, in _standardize_user_data
exception_prefix='input')
File "/home/wohlgemuth/workspace/Going-Deeper-with-Convolutional-Neural-Network-for-Stock-Market-Prediction/venv/lib/python3.7/site-packages/keras/engine/training_utils.py", line 128, in standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking input: expected input_1 to have 4 dimensions, but got array with shape (3751, 7500)

RandomForest Classifier results

Thanks for making this work open-source.
I have tried to replicate your results using Taiwanese datasets.
I have generated images with dimension size of 50 and a period size of 20.
I have done testing with a random forest classifier.
I have tried a different number of estimators to check the improvement of the result.
I get accuracy around 0.55 with the shared Taiwanese dataset.
I would like to know whether you have a guess about what I could be missing?
I would like to know also why there is such a big gap between my results using the exact same pipeline and the results mentioned in the paper.

There seems to be a mistake here!

I have been developing and testing models for the financial market for some time. An accuracy of more than 90% seems impossible to me for such simple graphic samples (with little data). I replicated your model and got to 53%. By the way, the same thing was described in the "RandomForest Classifier results" issue. I began to doubt the results of this work. Can you explain these results?

fail to convert to images

running the first step python run_binary_preprocessing.py 2880.TW 20 50 only generates one sample image, the folder "0" and "1" are empty.

tons of issues with keras2

could this please be updated to work with a somewhat recent keras version?

thanks

UserWarning: Update your Conv2D call to the Keras 2 API...

The way you create label is different from the description in your master thesis

Your code :

Your master thesis :