filippomb/time-series-classification-and-clustering-with-reservoir-computing

Library for implementing reservoir computing models (echo state networks) for multivariate time series classification and clustering.

time-series-classification-and-clustering-with-reservoir-computing's Issues

TensorPCA yields complex data type array which causes error in Ridge module

Hi @FilippoMB,

I noticed that for the dataset that I'm using, the result of tensorPCA yields a complex data type Numpy array. This in turn causes an error in the ridge module which says that it does not support complex data type. Specifically, error ValueError: Complex data not supported is generated at https://github.com/FilippoMB/Time-series-classification-and-clustering-with-Reservoir-Computing/blob/master/code/modules.py#L205

I don't face this issue when I use PCA with the same dataset.

I tried to print out the eigenvalue and eigenvector data type at https://github.com/FilippoMB/Time-series-classification-and-clustering-with-Reservoir-Computing/blob/master/code/tensorPCA.py#L33-L38 and both these vectors are of the data type, complex128, for the dataset I am using.

I tried Googling a bit and found some resources such as https://stackoverflow.com/questions/10420648/complex-eigen-values-in-pca-calculation and https://stackoverflow.com/questions/48695430/how-to-make-the-eigenvalues-and-eigenvectors-stay-real-instead-of-complex. From what I understood, due to some numerical error, the eigenvalues and eigenvectors can have a small imaginary value when linalg.eig is used. I'm not sure whether my understanding is correct.

Any thoughts on this?

ESN predicting constant values in classification

Hi,
I'm using the ESNClassification to classify a binary outcome from data which is input in the format (N,T,V). I've played around with the hyperparameters but no matter the combination, I get either only ones or only zeroes.

Here is the config set-up:

`config['seed'] = 1
np.random.seed(config['seed'])

Hyperarameters of the reservoir

config['n_internal_units'] = 100 # size of the reservoir
config['spectral_radius'] = 0.4 # largest eigenvalue of the reservoir
config['leak'] = 0.5 # amount of leakage in the reservoir state update (None or 1.0 --> no leakage)
config['connectivity'] = 0.1 # percentage of nonzero connections in the reservoir
config['input_scaling'] = 0.1 # scaling of the input weights
config['noise_level'] = 0.01 # noise in the reservoir state update
config['n_drop'] = 0 # transient states to be dropped
config['bidir'] = False # if True, use bidirectional reservoir
config['circ'] = False # use reservoir with circle topology

Dimensionality reduction hyperparameters

config['dimred_method'] ='tenpca' # options: {None (no dimensionality reduction), 'pca', 'tenpca'}
config['n_dim'] = 40 # number of resulting dimensions after the dimensionality reduction procedure

Type of MTS representation

config['mts_rep'] = 'last' # MTS representation: {'last', 'mean', 'output', 'reservoir'}
config['w_ridge_embedding'] = 5.0 # regularization parameter of the ridge regression

Type of readout

config['readout_type'] = 'mlp' # readout used for classification: {'lin', 'mlp', 'svm'}

Linear readout hyperparameters

config['w_ridge'] = 5.0 # regularization of the ridge regression readout

SVM readout hyperparameters

config['svm_gamma'] = 0.005 # bandwith of the RBF kernel
config['svm_C'] = 5.0 # regularization for SVM hyperplane

MLP readout hyperparameters

config['mlp_layout'] = (40,1) # neurons in each MLP layer
config['num_epochs'] = 20 # number of epochs
config['w_l2'] = 0.05 # weight of the L2 regularization
config['nonlinearity'] = 'relu' # type of activation function {'relu', 'tanh', 'logistic', 'identity'}`

And the format of the input for training:
print(X_tr.shape) print(X_te.shape) print(y_tr.shape) print(y_te.shape)
with output:
(7801, 200, 130) (1132, 200, 130) (7801, 1) (1132, 1)

when I train on X_tr and y_tr, the output are either all ones or all zeroes. Any guidance on what could be causing this error?
Cheers

application to meteorological forcasting variable

Hi,
I want to expériment your work on meteorological forcasting. I'm not professionnal coder and work on python and keras since 2 years (never directly on tensorflow)). I take a look to your code and I'm not sure what and where modify the code to change the classification method (MAE in lieu of cross_entropy).

I see the line 166 in MLP.py, my gess is to change this one to !? and remove the one hot enoder in the data preproscessing.

An other modification is the posibility to set the activate fonction to, exemple: 'relu' and the output to 'linear', if my comprehension is good, all output layer have the same activation function.

My goal is to use your esn to preprocessing my data and eventualy feed an more complex model.

my last question, can I use only only reservoir processing (with no bidirectionnal ) and get the process value to feed an other kind of model? If yes, what is the goot output var in the code?

Sorry for all this question, Im just a bit entthousiast about your work an I have so much idea to explore with it.

PS: I add 'softsign' activation to your code, it is a very effective one!
PPS: If you can just guide me I will try to integer my idea to your code and will summit you the modif for approbation if you dont have time.

Input weights shape

I'm facing a problem understanding the architecture of the reservoir if we look at it as a neural network. In here the shape of the input for the reservoir "current_input" in the code is (N, V), and the shape of the _input_weights is (internal_units, V).
And normally, the shape of the _input_weights should be (size of reservoir* size of the input).
Can I kindly ask why the size of the _input_weights is (internal_units, V) and not (internal_units, V*N)?

are you available for freelancing

@FilippoMB
are you interesting in freelancing, i have a task. What is your email?

Test dataset is used in training in the example? classification_example.py

The examples show it's using the full 640 MTS-"X" for training and then use 370 of them - "Xte" to test. Isn't Xte a subset of X?

Please add Code license

Hi thanks for providing the code in github. As I don't know other ways to reach you , I am raising this as an issue. But my query is can we use your code for testing and if we use it for any competitions.may be adding a license info will benefit people like me. Thank you