Giter Site home page Giter Site logo

deepsurv's Introduction

DeepSurv

DeepSurv implements a deep learning generalization of the Cox proportional hazards model using Theano and Lasagne.

DeepSurv has an advantage over traditional Cox regression because it does not require an a priori selection of covariates, but learns them adaptively.

DeepSurv can be used in numerous survival analysis applications. One medical application is provided: recommend_treatment, which provides treatment recommendations for a set of patient observations.

For more details, see full paper DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network.

For an updated implementation of the Cox loss function in PyTorch, please see Feature Selection using Stochastic Gates (STG) by Yamada et al..

Installation:

From source

Download a local copy of DeepSurv and install from the directory:

git clone https://github.com/jaredleekatzman/DeepSurv.git
cd DeepSurv
pip install .

Dependencies

Theano, Lasagne (bleeding edge version), lifelines, matplotlib (for visualization), tensorboard_logger, and all of their respective dependencies.

Running the tests

After installing, you can optionally run the test suite with

py.test

from the command line while in the repo's main directory.

Running Experiments

Experiments are run using Docker containers built off of the floydhub deep learning Docker images. DeepSurv can be run on either the CPU or the GPU with nvidia-docker.

All experiments are in the DeepSurv/experiments/ directory.

To run an experiment, define the experiment name as an environmental variable EXPRIMENTand run the docker-compose file. Further details are in the DeepSurv/experiments/ directory.

Training a Network

Training DeepSurv can be done in a few lines. First, all you need to do is prepare the datasets to have the following keys:

{ 
	'x': (n,d) observations (dtype = float32), 
 	't': (n) event times (dtype = float32),
 	'e': (n) event indicators (dtype = int32)
}

Then prepare a dictionary of hyper-parameters. And it takes only two lines to train a network:

network = deepsurv.DeepSurv(**hyperparams)
log = network.train(train_data, valid_data, n_epochs=500)

You can then evaluate its success on testing data:

network.get_concordance_index(**test_data)
>> 0.62269622730138632

If you have matplotlib installed, you can visualize the training and validation curves after training the network:

deepsurv.plot_log(log)

deepsurv's People

Contributors

jaredleekatzman avatar jmigual avatar leihuang avatar piyush01123 avatar vanamsterdam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

deepsurv's Issues

compute_hazards does not work for loaded models

I was trying to understand why the concordance index was inconsistent when calculating from a loaded model (see this comment here) and discovered this problem: model.X and model.partial_hazard are not correctly saved in save_model.

How to replicate:

  1. Run DeepSurv Example jupyter notebook
  2. Save the trained model with model.save_model('bestparams.json', weights_file='bestweights.h5')
  3. Load the model with model2 = deepsurv.deep_surv.load_model_from_json('bestparams.json', weights_fp='bestweights.h5')
  4. In one cell, run the following repeatedly. Notice that with the original model, partial_hazards is consistent.
compute_hazards = theano.function(inputs = [model.X],outputs = -model.partial_hazard)
partial_hazards= compute_hazards(x_train)
print(partial_hazards)
  1. In another cell, run the same code with the saved and then reloaded model. Notice that the value of partial_hazards changes with each run of the cell.
model2 = deepsurv.deep_surv.load_model_from_json('bestparams.json', weights_fp='bestweights.h5')
compute_hazards = theano.function(inputs = [model2.X],outputs = -model2.partial_hazard)
partial_hazards2 = compute_hazards(x_train2)
print(partial_hazards2)

This results in an inconsistent concordance index, since you must compute the hazards to get the concordance. I have tried to fix it myself but I don't think I understand Theano tensors well enough. Hope this can be addressed soon.

AttributeError for deepsurv

I am a newbie for survival analysis. I try to set up the docker and run the experiments (linear) and have got the following error

deepsurv_1  | ('Saving model parameters to output file', '/shared/results/modelslinear_model_selu_0.jsone7d7f147-c66a-4478-9375-171ba5c13379.h5')
deepsurv_1  | Traceback (most recent call last):
deepsurv_1  |   File "/scripts/deepsurv_run.py", line 165, in <module>
deepsurv_1  |     save_model(model, output_file)
deepsurv_1  |   File "/scripts/deepsurv_run.py", line 100, in save_model
deepsurv_1  |     model.save_weights(output_file)
deepsurv_1  | AttributeError: 'str' object has no attribute 'save_weights'

So what shall I do with this error?

Need advise and simple questions from a beginner

Hi All @jaredleekatzman @vanAmsterdam @dareneiri ,

I am a beginner for using python and really hope to apply this great package. Questions below may seem stupid but I am trying to find a way out...

  1. what is the valid_data for and why we need it? Commonly, I train a model using train set and test it using test data.
  2. If I don't use any docker file, can I run this package?
  3. In the deepsurv front page, it is mentioned that it takes only two lines to train a network, but I saw many people define their own parameters, is this necessary? In another word, without searching the best parameters, we can not get the best deepsurv model?
  4. Can I find some example code somewhere showing the process that the experiments used in the paper? Or anyone would like to share some example code about how to apply this package?

Much appreciate for any reply!

Xiaozhen

Run test data, got 0.48 c-index?!

Hi there,
I tested run the code but got a 0.48 c-index with the following code. Any idea why it could be the case? When printing out the log plot, the maximum likelihood is flat. Is there something going on with the loss function?

def generate_data(treatment_group = False):
np.random.seed(123)
sd = deepsurv.datasets.SimulatedData(5, num_features = 9,
treatment_group = treatment_group)
train_data = sd.generate_data(950)
valid_data = sd.generate_data(300)
test_data = sd.generate_data(300)

return train_data, valid_data, test_data

train,valid,test = generate_data(True)

hyperparams = {
'n_in': 10,
'learning_rate': 1e-5,
'hidden_layers_sizes': [10,10]
}

network = DeepSurv(**hyperparams)

log = network.train(train,valid,n_epochs=500,validation_frequency = 1)

risk = network.predict_risk(test['x'])

library error

Could you please let me know which version of dependencies you used, and the environment for this package.

I run on linux, using Pycharm5.0, python 2.7.11, theano 0.8.2, and got many exceptions of library code, for example "AttributeError: type object 'GCC_compiler' has no attribute 'compile_str' " in lazylinker_c.py.

And here is the error using centos:

image

can't open file '/scripts/deepsurv_run.py': [Errno 2] No such file or directory

I am new to Survival Analysis and I am trying to run the experiment using the dockerfiles but it shows the following error:

Step 5/5 : CMD [ "python", "-u", "/scripts/rsf_run.py", "/shared/data/linear_survival_data.h5", "--results_dir", "/shared/results/", "--num_trees", "100"]
---> Using cache
---> a8f230af9824
Successfully built a8f230af9824
Successfully tagged experiments_rsf:latest
Starting experiments_deepsurv_1 ... done
Starting experiments_cph_1 ... done
Starting experiments_rsf_1 ... done
Attaching to experiments_deepsurv_1, experiments_cph_1, experiments_rsf_1
deepsurv_1 | python: can't open file '/scripts/deepsurv_run.py': [Errno 2] No such file or directory
experiments_deepsurv_1 exited with code 2
rsf_1 | python: can't open file '/scripts/rsf_run.py': [Errno 2] No such file or directory
cph_1 | python: can't open file '/scripts/cph_run.py': [Errno 2] No such file or directory
experiments_rsf_1 exited with code 2
experiments_cph_1 exited with code 2

This happens with all the experiments. I downloaded the zip and my directory is \DeepSurv-master\DeepSurv-master\experiments and the scripts deepsurv_run.py, rsf_run.py and cph_run.py are in the script folder.

Would really appreciate any help. Thanks

question on deep_surv.py

On Line 166: uncensored_likelihood = risk.T - log_risk

Why is there a .T what does that mean?

plotting the survival function

Hi,
thank's for this great package!
I was wondering if there is a way to plot (according to the weights that the net acheived the survival function? (like in Lifelines package )

No module named 'deep_surv'

As I try to run DeepSurv Example.ipynb, I get the following error:

Traceback (most recent call last):
File "D:\Thesis\DeepSurv\deepsurv_eg.py", line 4, in
import deep_surv
ModuleNotFoundError: No module named 'deep_surv'
[Finished in 0.1s with exit code 1]

I have the updated versions of Theano and Lasagne installed.

deepsurv import error

Hi,

>>> import deepsurv
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/deepsurv/__init__.py", line 1, in <module>
    from .deep_surv import DeepSurv
  File "/usr/local/lib/python2.7/dist-packages/deepsurv/deep_surv.py", line 1, in <module>
    import lasagne
  File "/usr/local/lib/python2.7/dist-packages/lasagne/__init__.py", line 19, in <module>
    from . import layers
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/__init__.py", line 7, in <module>
    from .pool import *
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/pool.py", line 6, in <module>
    from theano.tensor.signal import downsample
ImportError: cannot import name downsample
>>> import theano
>>> theano.__version__
'0.9.0'

Ubuntu 16.04 LTS

Using data with tied event times

Hi Jared,

In your current package you only support datasets with the assumption that there are no tied event times. What do you recommend doing for datasets that have tied event times (say, because time is in discrete intervals). How difficult would it be to implement breslow's method, for example, to handle these tied event times? Or what about jittering the event time so tied events are made unique (if event happened at 300 minutes, and there are duplicates of 300 minutes, then make subseuent event times 300.01, 300.02, etc).

Let's also assume that we need to have all events and cannot just drop the event times that are tied.

Thanks!

py.test ModuleNotFoundError & linear Experiment ImportError

pip install . is success, but py.test has a ModuleNotFoundError. I use the method mentioned in #5 but still throws the error
qq20180727-135418 2x

And when I run the linear experiment by EXPERIMENT=linear docker-compose up --build, it throws ImportError,
e0866aa2-4dc4-4466-9c2f-f6e8c41e6ca3
my python-dateutil version is 2.7.2, and I tried both python2 & python3, it throws the same error

SGD not used for optimization?

Hi,

Thank you for sharing the code. I just have a quick question: Is stochastic gradient descent used for optimization here?

required Theano version

Is it absolutely necessary to have Theano 0.8.2 to properly run DeepSurv? Would DeepSurv fail to function if the Theano version is different from 0.8.2? Installing DeepSurv rolls back my Theano to an older, outdated version.

Issues in initialising and running

I encountered several issues when I tried to apply your model to my data.

First, when I imported your package, it immediately reported an error about the โ€œdatasets.pyโ€. After a simple change in init.py, it could work.

Second, when I had multiple columns of x in both training and validation data, it reported an error caused by โ€œout of indexโ€ in Pandas data frame. I havenโ€™t tried further because I think there might be some other issues. These issues are strange because it seems in your experiments mentioned in your paper, there is not thus issues at all.

Could you please let me know your thoughts about the issue I encountered?

Regards,
Xing

create new tag/release

Could you please tag a new version?
The latest stable release, DeepSurv-2.0.0 is not fully compatible with Python 3.

Survival Probability at Specific Time Points

Can deepsurv calculate probabilities of survival at specific time points? When deepsurv recommends Treatment A over Treatment B, I would like to know point estimates with 95% confidence interval at specific time points? It would be easier to understand if deepsurv can produce numbers as below:
Treatment A: 1-year survival 94.5% (95% CI, 90.1-98.9)
Treatment B: 1-year survival 85.3% (95% CI, 80.1-89.5)
If possible, could you provide a code for this?

Thank you for the package

Would love to see your examples. and could not find py.test - was it part of the package?
your base module for windows does not have it

All the best!

The negative log likelihood Not update over epochs

I generated nonlinear survival data using the SimulatedData class in your package, and only got 0.54 c-index for the test dataset. The negative log likelihood plot showed two separate straight lines for the training data and validation data. I think the loss function did not converge over epochs during my training process. I attached the code I used below. Could you please help me find the reason? Thank you very much!!

def generate_data(treatment_group = False):
np.random.seed(123)
sd = SimulatedData(5, num_features = 9,
treatment_group = treatment_group)
train_data = sd.generate_data(5000)
valid_data = sd.generate_data(2000)
test_data = sd.generate_data(2000)

return train_data, valid_data, test_data

train_data,valid_data,test_data=generate_data(True)
hyperparams = {
'n_in': 10,
'learning_rate': 1e-5,
'hidden_layers_sizes':[10,10]
}

network = DeepSurv(**hyperparams)
log =network.train(train_data, valid_data, n_epochs=500)
network.get_concordance_index(**test_data)
deepsurv.plot_log(log)

Train error need help!

I don't get my own data,so I make a data,like: dataset = {'x':np.array([[0.1,0.3],[0.4,0.5],[0.3,0.2],[0.8,0.7],[0.5,0.2]],dtype = np.float32),'e':np.array([0,1,1,0,1],dtype = np.int32),'t':np.array([0.2,0.3,0.4,0.8,0.9],dtype = np.float32)}

please tell me whether the data is right?

can you give me a file about the data??

my train code:

import deepsurv
import pandas as pd
import numpy as np
#dataset = pd.read_csv('/home/mw/mw/work/DeepSurv-master/test.csv')
dataset = {'x':np.array([[0.1,0.3],[0.4,0.5],[0.3,0.2],[0.8,0.7],[0.5,0.2]],dtype = np.float32),'e':np.array([0,1,1,0,1],dtype = np.int32),'t':np.array([0.2,0.3,0.4,0.8,0.9],dtype = np.float32)}
train_data = dataset
valid_data = dataset
network = deepsurv.DeepSurv(n_in =2,learning_rate = 1e-5)
log = network.train(train_data, valid_data, n_epochs=500)

Again,when I train the network I meet a problem:

Traceback (most recent call last):
File "test.py", line 14, in
log = network.train(train_data, valid_data, n_epochs=500,logger = None)
File "/home/mw/mw/work/DeepSurv-master/deepsurv/deep_surv.py", line 397, in train
update_fn = update_fn, **kwargs
File "/home/mw/mw/work/DeepSurv-master/deepsurv/deep_surv.py", line 242, in _get_train_valid_fn
learning_rate=learning_rate, **kwargs
File "/home/mw/mw/work/DeepSurv-master/deepsurv/deep_surv.py", line 208, in _get_loss_updates
if self.restored_update_params:
AttributeError: DeepSurv instance has no attribute 'restored_update_params'
here

so I comment the code,but another question happens:

Traceback (most recent call last):
File "test.py", line 14, in
log = network.train(train_data, valid_data, n_epochs=500,logger = None)
File "/home/mw/mw/work/DeepSurv-master/deepsurv/deep_surv.py", line 410, in train
logger.logValue('loss', loss, epoch)
AttributeError: 'NoneType' object has no attribute 'logValue'

I don't know whether the cause of my data or other question,Please give me some help ! Thank you very much first !

Save and reload the model?

Hi There:
I just wonder if the load model function is working properly. I can see the weights and updates are saved and reloaded properly, by when calling the saved model to predict on the same data when training the model, the c-index is significantly lower, looks like the the model is not properly specified.
It would be helpful to illustrate the correct way to save and reload a trained model.

Thanks.

error when using get_concordance_index()

Hello and thank you for sharing your great work.

I have trained a network :

network = deepsurv.DeepSurv(10,0.0001) log = network.train(d_train, d_valid, n_epochs=500)

[INFO] Training CoxMLP Finished Training with 500 iterations in 20.05s

It seems it was able to predict a risk vector :

network.predict_risk(d_valid['x'])

array([[ -5.24271558e+00], [ -1.13109659e+03], [ 4.15896825e+00], [ -7.84299117e+00], [ 1.62468140e+01], [ -3.04364481e+00], [ -3.50972074e+00], [ -2.26831663e+02], [ -3.07352921e+00], [ -1.57916871e+00], .....

but when using get_concordance_index() like this :

network.get_concordance_index(d_test['x'],d_test['t'],d_test['e'])

I get this error :

`---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
in ()
----> 1 network.get_concordance_index(d_test['x'],d_test['t'],d_test['e'])

/Library/anaconda/lib/python3.5/site-packages/deepsurv/deep_surv.py in get_concordance_index(self, x, t, e, **kwargs)
268 return concordance_index(t,
269 partial_hazards,
--> 270 e)
271
272 def train(self,

/Library/anaconda/lib/python3.5/site-packages/lifelines/utils/init.py in concordance_index(event_times, predicted_event_times, event_observed)
384 return _concordance_index(event_times,
385 predicted_event_times,
--> 386 event_observed)
387
388

/Library/anaconda/lib/python3.5/site-packages/lifelines/utils/init.py in _concordance_index(event_times, predicted_event_times, event_observed)
857 died_ix = next_ix
858 else:
--> 859 assert not (has_more_died or has_more_censored)
860 break
861

AssertionError:`

Have you any idea of how I could fix it ?

Thank you !

About other features

Hi @jaredleekatzman ,

In your deepsurv front page, you mentioned 'Training DeepSurv can be done in a few lines. First, all you need to do is prepare the datasets to have the following keys: x t and e' How about if my survival data has other features? Will the net model recognize it automatically or should I index them?

Many thanks!

Unused in y_true in loss function

Generally, I believe for a neural network, the loss function consists of comparing the output from the neural network (y_pred) with the already assigned labels/data (y_true). I realize for your lost function, you dod not use y_true, which is the known time events, how can you get the loss updated this way?

Negative log likelihood equation

Hello,

I've read your paper and when looking for the negative log likelihood equation and implementation it looks like they are not the same. In the paper there's equation 3 which is like this:

But looking at the code it seems that you are not using the risk set (R(t)) and it behaves like this (notice the absence of R(Ti) in the second summation):

Is there a reason for this?

Exact training and validation procedure

Hi,

thank you very much for this github! I was wondering if you could provide me with a little more detail on how exactly you achieved the c-index reported in the paper. You mention doing a hyperparameter search with 3-fold cross-validation. I would like to benchmark your publication against one that I am soon going to publish.

What did you do once the optimal hyperparameters were obtained? Did you still use a validation set for early stopping, or train until the mean early stopping epoch on the full training set? Or is the prediction an ensemble of the three models from the hyperparameter search?

If I understand your tutorials and code correctly, I cannot just use your docker files since I am modifying the code (adding a couple of line that require another hyperparameter).

Also in your paper, you describe the WHAS data set as consisting of 5 features, but the HDF5 data set when downloaded seems to have 6 features. Am I getting something wrong or is that a typo in the paper?

Also, did you simply use the default values in DeepSurv.train() for n_epochs, patience etc.? If not, where can I get those values from?

Thanks!
Philippe

viz.py plotting error

Hello,

When i use vix.py to plot log in the code below, I get next error

deepsurv.plot_log(log) network.get_concordance_index(**atest)

`File "C:...\deepsurv\viz.py", line 38, in plot_log
num_epochs = len(log[TRAIN_LOSS])
KeyError: 'loss'

network.get_concordance_index(**atest)
Out[38]: 0.65957446808510634
`

Any idea where this key error comes from ?

question about training

Thanks for the great work @jaredleekatzman, but I have a question about training of DeepSurv. Is there any "batch" split from train set when training or just a whole epoch of example are feed to the network directly?

No module name 'deepsurv_logger'

Hello,
I get the following error if I run py.test or try to import deepsurv. Any clues? Thanks,

===================================================================== ERRORS =====================================================================
____________________________________________________ ERROR collecting tests/test_deepsurv.py _____________________________________________________
ImportError while importing test module '/run/media/eminozkan/0B46-D280/Cancer/DeepSurv/tests/test_deepsurv.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test_deepsurv.py:4: in <module>
    import deepsurv
/home/eminozkan/.conda/envs/life/lib/python3.5/site-packages/deepsurv/__init__.py:1: in <module>
    from .deep_surv import DeepSurv
/home/eminozkan/.conda/envs/life/lib/python3.5/site-packages/deepsurv/deep_surv.py:14: in <module>
    from deepsurv_logger import DeepSurvLogger
E   ImportError: No module named 'deepsurv_logger'

Question about use of partial_hazards in get_concordance_index()

I'm looking at both lifelines and your method for getting concordance index. In DeepSurv, you have:

        return concordance_index(t,
            partial_hazards,
e)

where t is time, e is event indicator, and partial_hazards is -self.partial_hazard.

In lifelines utils/init.py:

    return _concordance_index(event_times,
                              predicted_event_times,
event_observed)

Where it differs in using predicted_event_times in place of partial_hazards.

Do you mind explaining the use of partial_hazards in place of predicted_event_times to calculate CI?

Also, is it incorrect to have risk values that are negative? Many of my risk values end up being negative when I do:
self.network.predict_risk(x)

Thanks!

Clarification on hidden_layers_sizes

My first question:
If I use the following code below, is deepsurv indeed running without any hidden layers?:

hyperparams = {
    'n_in': 25,
    'learning_rate': 0.0001,
    'hidden_layers_sizes': None, 
    'lr_decay': 0,
    'momentum': 0.9,
    'L2_reg': 0.0,
    'L1_reg': 0,
    'activation': 'rectify',
    'dropout': 0,
    'batch_norm': False,
    'standardize': False,
}
network = deepsurv.DeepSurv(**hyperparams)

I can train the model:
python log = network.train(train_data, valid_data, n_epochs = 1300)
This results in a good concordance value. I just want to verify what is being passed. What were your predefined ranges you used for Optunity?

My second question is when I specify hidden_layers_sizes:
When I run the model with hyper-parameters as above, but with hidden_layers_sizes specified,

hyperparams = {
   ...
    'hidden_layers_sizes': [14,2], 
   ...
    'activation': 'rectify',
   ...
}

Then I get the error:

/home/spark/anaconda2/envs/theano-fips/lib/python2.7/site-packages/lasagne/layers/dense.pyc in get_output_for(self, input, **kwargs)
    122         if self.b is not None:
    123             activation = activation + self.b
--> 124         return self.nonlinearity(activation)
    125 
    126 

TypeError: 'str' object is not callable

It's only when I comment out the activation does it model actually attempt to train. But the code should regardless be running rectify, correct?

Statistical Significance

Hi,

I'm a beginner here, and I had a couple questions about DeepSurv. When doing "normal" cox proportional hazards survival analysis, we can analyze the impact of each covariate on the survival outcome (with hazard ratios) and calculate p-values. Does DeepSurv have the same univariate capabilities or is it supposed to be used on ALL the covariates at once? If so, how do we know which covariates have a statistically significant effect on the survival?

Edit: When running DeepSurv with a certain number of epochs, running it over and over results in different concordance indices. They seem to vary drastically (0.42 to 0.15 to 0.55, etc.). Am I doing something wrong?

Thanks so much!

AttributeError: DeepSurv instance has no attribute 'restored_update_params'

I wanted to use the latest version of DeepSurv based on this repository (primarily to save and load my model), and not on pypi (which was updated June 2016). Can you confirm if the below error is a known issue? While I was able to run my model fine previously, I am no longer able to.

I run:
network = deepsurv.DeepSurv(15,0.0001)
Then:
log = network.train(dataset_TRAIN, dataset_VALID, n_epochs = 300)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-10-aa424000404e> in <module>()

----> 3 log =  network.train(dataset_TRAIN, dataset_VALID, n_epochs = 300)


/usr/local/bin/anaconda2/envs/virtual_env/lib/python2.7/site-packages/deepsurv/deep_surv.pyc in train(self, train_data, valid_data, n_epochs, validation_frequency, patience, improvement_threshold, patience_increase, logger, update_fn, **kwargs)
    394             learning_rate=lr,
    395             momentum = momentum,
--> 396             update_fn = update_fn, **kwargs
    397         )
    398 

/usr/local/bin/anaconda2/envs/virtual_env/lib/python2.7/site-packages/deepsurv/deep_surv.pyc in _get_train_valid_fn(self, L1_reg, L2_reg, learning_rate, **kwargs)
    240         loss, updates = self._get_loss_updates(
    241             L1_reg, L2_reg, deterministic = False,
--> 242             learning_rate=learning_rate, **kwargs
    243         )
    244         train_fn = theano.function(

/usr/local/bin/anaconda2/envs/virtual_env/lib/python2.7/site-packages/deepsurv/deep_surv.pyc in _get_loss_updates(self, L1_reg, L2_reg, update_fn, max_norm, deterministic, **kwargs)
    206 
    207         # If the model was loaded from file, reload params
--> 208         if self.restored_update_params:
    209             for p, value in zip(updates.keys(), self.restored_update_params):
    210                 p.set_value(value)

AttributeError: DeepSurv instance has no attribute 'restored_update_params'

Error with a hint that don't know

Hi All @jaredleekatzman ,

I got an error which I can not figure out why even after reading the hint.... Can anyone help?

In[44]: log = network.train(mypbc_Train,mypbc_Valid,n_epochs=50,validation_frequency=1)
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/IPython/core/interactiveshell.py", line 2882, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
log = network.train(mypbc_Train,mypbc_Valid,n_epochs=50,validation_frequency=1)
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 428, in train
loss = train_fn(x_train, e_train)
File "/usr/lib64/python2.7/site-packages/theano/compile/function_module.py", line 871, in call
storage_map=getattr(self.fn, 'storage_map', None))
File "/usr/lib64/python2.7/site-packages/theano/gof/link.py", line 314, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "/usr/lib64/python2.7/site-packages/theano/compile/function_module.py", line 859, in call
outputs = self.fn()
File "/usr/lib64/python2.7/site-packages/theano/gof/op.py", line 912, in rval
r = p(n, [x[0] for x in i], o)
File "/usr/lib64/python2.7/site-packages/theano/tensor/basic.py", line 5436, in perform
z[0] = numpy.asarray(numpy.dot(x, y))
ValueError: shapes (234,6) and (10,10) not aligned: 6 (dim 1) != 10 (dim 0)
Apply node that caused the error: dot(x, W)
Toposort index: 8
Inputs types: [TensorType(float32, matrix), TensorType(float64, matrix)]
Inputs shapes: [(234, 6), (10, 10)]
Inputs strides: [(24, 4), (80, 8)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{add,no_inplace}(dot.0, InplaceDimShuffle{x,0}.0), Elemwise{Composite{(i0 * (Abs(i1) + i2 + i3))}}[(0, 2)](TensorConstant{(1, 1) of 0.5}, Elemwise{add,no_inplace}.0, dot.0, InplaceDimShuffle{x,0}.0)]]

Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
File "", line 1, in
log = network.train(mypbc_Train,mypbc_Valid,n_epochs=50,validation_frequency=1)
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 416, in train
update_fn = update_fn, **kwargs
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 254, in _get_train_valid_fn
learning_rate=learning_rate, **kwargs
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 201, in _get_loss_updates
+ regularize_layer_params(self.network, l2) * L2_reg
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 163, in _negative_log_likelihood
risk = self.risk(deterministic)
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 563, in risk
deterministic = deterministic)
File "/usr/lib/python2.7/site-packages/lasagne/layers/helper.py", line 197, in get_output
all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
File "/usr/lib/python2.7/site-packages/lasagne/layers/dense.py", line 121, in get_output_for
activation = T.dot(input, self.W)

HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

Can anyone point out why this error happen? Many thanks!

Questions about hyper parameter selection

The hyper parameters used for the experiments are already set. May I ask how are the current hyper parameters selected? Are there any tips for selecting hyper parameters to get better experimental results (using the datasets included under experiments folder)?

loss goes to nan

I am working on the public datasets a this link: https://vincentarelbundock.github.io/Rdatasets/datasets.html
in particular I have tried "aids2" and "flchain".
Nevertheless in both cases log takes the following values:
{'best_params': None,
'best_params_idx': -1,
'best_valid_ci': 0.5,
'best_validation_loss': inf,
'train': [array(28340558372.044426),
array(nan),
array(nan),
array(nan),
array(nan)],
'train_ci': [0.5, 0.5, 0.5, 0.5, 0.5],
'valid': [array(nan), array(nan), array(nan), array(nan), array(nan)],
'valid_ci': [0.5, 0.5, 0.5, 0.5, 0.5]}
what can be done to solve this issue?

Run py.test problem

I following readme.txt file,when i run py.test ,i meet some problem.How deal with it?There, I copy these problems.

ccf@ccf-Lenovo-Product:~/CCF/DeepSurv-master$ py.test
============================= test session starts ==============================
platform linux2 -- Python 2.7.6, pytest-3.1.2, py-1.4.34, pluggy-0.4.0
rootdir: /home/ccf/CCF/DeepSurv-master, inifile:
collected 8 items

tests/test_deepsurv.py .F.FEEEE

==================================== ERRORS ====================================
________________ ERROR at setup of TestDeepSurvTrain.test_train ________________

self = <class test_deepsurv.TestDeepSurvTrain at 0x7f9bffc83120>

@classmethod
def setup_class(self):
    self.train, self.valid, self.test = generate_data(treatment_group=True)

    hyperparams = {
        'n_in': 10,
        'learning_rate': 1e-5,
        'hidden_layers_sizes': [10]
    }
    network = DeepSurv(**hyperparams)
    log = network.train(self.train, self.valid,
      n_epochs=10,validation_frequency=1)

tests/test_deepsurv.py:63:


deepsurv/deep_surv.py:397: in train
update_fn = update_fn, **kwargs
deepsurv/deep_surv.py:243: in _get_train_valid_fn
learning_rate=learning_rate, **kwargs


self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa3d5b90>, L1_reg = 0.0
L2_reg = 0.0, update_fn = <function nesterov_momentum at 0x7f9c04ebfaa0>
max_norm = None, deterministic = False
kwargs = {'learning_rate': <TensorType(float32, scalar)>, 'momentum': array(0.0, dtype=float32)}
loss = Elemwise{add,no_inplace}.0
updates = OrderedDict([(W, Elemwise{add,no_inplace}.0), (b, Elemwise{add,no_inplace}.0),...b,no_inplace}.0), (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])

def _get_loss_updates(self,
L1_reg = 0.0, L2_reg = 0.001,
update_fn = lasagne.updates.nesterov_momentum,
max_norm = None, deterministic = False,
**kwargs):
    """
        Returns Theano expressions for the network's loss function and parameter
            updates.

        Parameters:
            L1_reg: float for L1 weight regularization coefficient.
            L2_reg: float for L2 weight regularization coefficient.
            max_norm: If not None, constraints the norm of gradients to be less
                than max_norm.
            deterministic: True or False. Determines if the output of the network
                is calculated determinsitically.
            update_fn: lasagne update function.
                Default: Stochastic Gradient Descent with Nesterov momentum
            **kwargs: additional parameters to provide to update_fn.
                For example: momentum

        Returns:
            loss: Theano expression for a penalized negative log likelihood.
            updates: Theano expression to update the parameters using update_fn.
        """

    loss = (
        self._negative_log_likelihood(self.E, deterministic)
        + regularize_layer_params(self.network,l1) * L1_reg
        + regularize_layer_params(self.network, l2) * L2_reg
    )

    if max_norm:
        grads = T.grad(loss,self.params)
        scaled_grads = lasagne.updates.total_norm_constraint(grads, max_norm)
        updates = update_fn(
            grads, self.params, **kwargs
        )
        return loss, updates

    updates = update_fn(
            loss, self.params, **kwargs
        )

    # If the model was loaded from file, reload params
  if self.restored_update_params:

E AttributeError: DeepSurv instance has no attribute 'restored_update_params'

deepsurv/deep_surv.py:209: AttributeError
---------------------------- Captured stdout setup -----------------------------
[ 1. 0. 1. ..., 1. 0. 0.]
[ 1. 1. 1. ..., 1. 1. 1.]
[ 0. 1. 1. ..., 1. 0. 1.]
________ ERROR at setup of TestDeepSurvTrain.test_network_predict_risk _________

self = <class test_deepsurv.TestDeepSurvTrain at 0x7f9bffc83120>

@classmethod
def setup_class(self):
    self.train, self.valid, self.test = generate_data(treatment_group=True)

    hyperparams = {
        'n_in': 10,
        'learning_rate': 1e-5,
        'hidden_layers_sizes': [10]
    }
    network = DeepSurv(**hyperparams)
    log = network.train(self.train, self.valid,
      n_epochs=10,validation_frequency=1)

tests/test_deepsurv.py:63:


deepsurv/deep_surv.py:397: in train
update_fn = update_fn, **kwargs
deepsurv/deep_surv.py:243: in _get_train_valid_fn
learning_rate=learning_rate, **kwargs


self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa3d5b90>, L1_reg = 0.0
L2_reg = 0.0, update_fn = <function nesterov_momentum at 0x7f9c04ebfaa0>
max_norm = None, deterministic = False
kwargs = {'learning_rate': <TensorType(float32, scalar)>, 'momentum': array(0.0, dtype=float32)}
loss = Elemwise{add,no_inplace}.0
updates = OrderedDict([(W, Elemwise{add,no_inplace}.0), (b, Elemwise{add,no_inplace}.0),...b,no_inplace}.0), (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])

def _get_loss_updates(self,
L1_reg = 0.0, L2_reg = 0.001,
update_fn = lasagne.updates.nesterov_momentum,
max_norm = None, deterministic = False,
**kwargs):
    """
        Returns Theano expressions for the network's loss function and parameter
            updates.

        Parameters:
            L1_reg: float for L1 weight regularization coefficient.
            L2_reg: float for L2 weight regularization coefficient.
            max_norm: If not None, constraints the norm of gradients to be less
                than max_norm.
            deterministic: True or False. Determines if the output of the network
                is calculated determinsitically.
            update_fn: lasagne update function.
                Default: Stochastic Gradient Descent with Nesterov momentum
            **kwargs: additional parameters to provide to update_fn.
                For example: momentum

        Returns:
            loss: Theano expression for a penalized negative log likelihood.
            updates: Theano expression to update the parameters using update_fn.
        """

    loss = (
        self._negative_log_likelihood(self.E, deterministic)
        + regularize_layer_params(self.network,l1) * L1_reg
        + regularize_layer_params(self.network, l2) * L2_reg
    )

    if max_norm:
        grads = T.grad(loss,self.params)
        scaled_grads = lasagne.updates.total_norm_constraint(grads, max_norm)
        updates = update_fn(
            grads, self.params, **kwargs
        )
        return loss, updates

    updates = update_fn(
            loss, self.params, **kwargs
        )

    # If the model was loaded from file, reload params
  if self.restored_update_params:

E AttributeError: DeepSurv instance has no attribute 'restored_update_params'

deepsurv/deep_surv.py:209: AttributeError
________ ERROR at setup of TestDeepSurvTrain.test_get_concordance_index ________

self = <class test_deepsurv.TestDeepSurvTrain at 0x7f9bffc83120>

@classmethod
def setup_class(self):
    self.train, self.valid, self.test = generate_data(treatment_group=True)

    hyperparams = {
        'n_in': 10,
        'learning_rate': 1e-5,
        'hidden_layers_sizes': [10]
    }
    network = DeepSurv(**hyperparams)
    log = network.train(self.train, self.valid,
      n_epochs=10,validation_frequency=1)

tests/test_deepsurv.py:63:


deepsurv/deep_surv.py:397: in train
update_fn = update_fn, **kwargs
deepsurv/deep_surv.py:243: in _get_train_valid_fn
learning_rate=learning_rate, **kwargs


self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa3d5b90>, L1_reg = 0.0
L2_reg = 0.0, update_fn = <function nesterov_momentum at 0x7f9c04ebfaa0>
max_norm = None, deterministic = False
kwargs = {'learning_rate': <TensorType(float32, scalar)>, 'momentum': array(0.0, dtype=float32)}
loss = Elemwise{add,no_inplace}.0
updates = OrderedDict([(W, Elemwise{add,no_inplace}.0), (b, Elemwise{add,no_inplace}.0),...b,no_inplace}.0), (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])

def _get_loss_updates(self,
L1_reg = 0.0, L2_reg = 0.001,
update_fn = lasagne.updates.nesterov_momentum,
max_norm = None, deterministic = False,
**kwargs):
    """
        Returns Theano expressions for the network's loss function and parameter
            updates.

        Parameters:
            L1_reg: float for L1 weight regularization coefficient.
            L2_reg: float for L2 weight regularization coefficient.
            max_norm: If not None, constraints the norm of gradients to be less
                than max_norm.
            deterministic: True or False. Determines if the output of the network
                is calculated determinsitically.
            update_fn: lasagne update function.
                Default: Stochastic Gradient Descent with Nesterov momentum
            **kwargs: additional parameters to provide to update_fn.
                For example: momentum

        Returns:
            loss: Theano expression for a penalized negative log likelihood.
            updates: Theano expression to update the parameters using update_fn.
        """

    loss = (
        self._negative_log_likelihood(self.E, deterministic)
        + regularize_layer_params(self.network,l1) * L1_reg
        + regularize_layer_params(self.network, l2) * L2_reg
    )

    if max_norm:
        grads = T.grad(loss,self.params)
        scaled_grads = lasagne.updates.total_norm_constraint(grads, max_norm)
        updates = update_fn(
            grads, self.params, **kwargs
        )
        return loss, updates

    updates = update_fn(
            loss, self.params, **kwargs
        )

    # If the model was loaded from file, reload params
  if self.restored_update_params:

E AttributeError: DeepSurv instance has no attribute 'restored_update_params'

deepsurv/deep_surv.py:209: AttributeError
_________ ERROR at setup of TestDeepSurvTrain.test_recommend_treatment _________

self = <class test_deepsurv.TestDeepSurvTrain at 0x7f9bffc83120>

@classmethod
def setup_class(self):
    self.train, self.valid, self.test = generate_data(treatment_group=True)

    hyperparams = {
        'n_in': 10,
        'learning_rate': 1e-5,
        'hidden_layers_sizes': [10]
    }
    network = DeepSurv(**hyperparams)
    log = network.train(self.train, self.valid,
      n_epochs=10,validation_frequency=1)

tests/test_deepsurv.py:63:


deepsurv/deep_surv.py:397: in train
update_fn = update_fn, **kwargs
deepsurv/deep_surv.py:243: in _get_train_valid_fn
learning_rate=learning_rate, **kwargs


self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa3d5b90>, L1_reg = 0.0
L2_reg = 0.0, update_fn = <function nesterov_momentum at 0x7f9c04ebfaa0>
max_norm = None, deterministic = False
kwargs = {'learning_rate': <TensorType(float32, scalar)>, 'momentum': array(0.0, dtype=float32)}
loss = Elemwise{add,no_inplace}.0
updates = OrderedDict([(W, Elemwise{add,no_inplace}.0), (b, Elemwise{add,no_inplace}.0),...b,no_inplace}.0), (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])

def _get_loss_updates(self,
L1_reg = 0.0, L2_reg = 0.001,
update_fn = lasagne.updates.nesterov_momentum,
max_norm = None, deterministic = False,
**kwargs):
    """
        Returns Theano expressions for the network's loss function and parameter
            updates.

        Parameters:
            L1_reg: float for L1 weight regularization coefficient.
            L2_reg: float for L2 weight regularization coefficient.
            max_norm: If not None, constraints the norm of gradients to be less
                than max_norm.
            deterministic: True or False. Determines if the output of the network
                is calculated determinsitically.
            update_fn: lasagne update function.
                Default: Stochastic Gradient Descent with Nesterov momentum
            **kwargs: additional parameters to provide to update_fn.
                For example: momentum

        Returns:
            loss: Theano expression for a penalized negative log likelihood.
            updates: Theano expression to update the parameters using update_fn.
        """

    loss = (
        self._negative_log_likelihood(self.E, deterministic)
        + regularize_layer_params(self.network,l1) * L1_reg
        + regularize_layer_params(self.network, l2) * L2_reg
    )

    if max_norm:
        grads = T.grad(loss,self.params)
        scaled_grads = lasagne.updates.total_norm_constraint(grads, max_norm)
        updates = update_fn(
            grads, self.params, **kwargs
        )
        return loss, updates

    updates = update_fn(
            loss, self.params, **kwargs
        )

    # If the model was loaded from file, reload params
  if self.restored_update_params:

E AttributeError: DeepSurv instance has no attribute 'restored_update_params'

deepsurv/deep_surv.py:209: AttributeError
=================================== FAILURES ===================================
_____________ TestDeepSurvInit.test_deepsurv_initialize_batch_norm _____________

self = <test_deepsurv.TestDeepSurvInit instance at 0x7f9bfa41f488>

def test_deepsurv_initialize_batch_norm(self):
  network = DeepSurv(batch_norm = True, **self.hyperparams)

tests/test_deepsurv.py:39:


self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa41fc68>, n_in = 10
learning_rate = 1e-05, hidden_layers_sizes = [10, 10], lr_decay = 0.0
momentum = 0.9, L2_reg = 0.0, L1_reg = 0.0, activation = 'rectify'
dropout = None, batch_norm = True, standardize = False

def __init__(self, n_in,
learning_rate, hidden_layers_sizes = None,
lr_decay = 0.0, momentum = 0.9,
L2_reg = 0.0, L1_reg = 0.0,
activation = "rectify",
dropout = None,
batch_norm = False,
standardize = False,
):
    """
        This class implements and trains a DeepSurv model.

        Parameters:
            n_in: number of input nodes.
            learning_rate: learning rate for training.
            lr_decay: coefficient for Power learning rate decay.
            L2_reg: coefficient for L2 weight decay regularization. Used to help
                prevent the model from overfitting.
            L1_reg: coefficient for L1 weight decay regularization
            momentum: coefficient for momentum. Can be 0 or None to disable.
            hidden_layer_sizes: a list of integers to determine the size of
                each hidden layer.
            activation: a lasagne activation class.
                Default: lasagne.nonlinearities.rectify
            batch_norm: True or False. Include batch normalization layers.
            dropout: if not None or 0, the percentage of dropout to include
                after each hidden layer. Default: None
            standardize: True or False. Include standardization layer after
                input layer.
        """

    self.X = T.fmatrix('x')  # patients covariates
    self.E = T.ivector('e') # the observations vector

    # Default Standardization Values: mean = 0, std = 1
    self.offset = theano.shared(numpy.zeros(shape = n_in, dtype=numpy.float32))
    self.scale = theano.shared(numpy.ones(shape = n_in, dtype=numpy.float32))

    network = lasagne.layers.InputLayer(shape=(None,n_in),
        input_var = self.X)

    if standardize:
        network = lasagne.layers.standardize(network,self.offset,
                                            self.scale,
                                            shared_axes = 0)
    self.standardize = standardize

    if activation == 'rectify':
        activation_fn = lasagne.nonlinearities.rectify
    else:
        raise IllegalArgumentException("Unknown activation function: %s" % activation)

    # Construct Neural Network
    for n_layer in (hidden_layers_sizes or []):
        if activation_fn == lasagne.nonlinearities.rectify:
            W_init = lasagne.init.GlorotUniform()
        else:
            # TODO: implement other initializations
            W_init = lasagne.init.GlorotUniform()


        network = lasagne.layers.DenseLayer(
            network, num_units = n_layer,
            nonlinearity = activation_fn,
            W = W_init
        )

        if batch_norm:
          network = lasagne.layers.po(network)

E AttributeError: 'module' object has no attribute 'po'

deepsurv/deep_surv.py:86: AttributeError
_________ TestDeepSurvInit.test_deepsurv_initialize_standardize_layer __________

self = <test_deepsurv.TestDeepSurvInit instance at 0x7f9bfa37c9e0>

def test_deepsurv_initialize_standardize_layer(self):
  network = DeepSurv(standardize = True, **self.hyperparams)

tests/test_deepsurv.py:47:


self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa3f7b00>, n_in = 10
learning_rate = 1e-05, hidden_layers_sizes = [10, 10], lr_decay = 0.0
momentum = 0.9, L2_reg = 0.0, L1_reg = 0.0, activation = 'rectify'
dropout = None, batch_norm = False, standardize = True

def __init__(self, n_in,
learning_rate, hidden_layers_sizes = None,
lr_decay = 0.0, momentum = 0.9,
L2_reg = 0.0, L1_reg = 0.0,
activation = "rectify",
dropout = None,
batch_norm = False,
standardize = False,
):
    """
        This class implements and trains a DeepSurv model.

        Parameters:
            n_in: number of input nodes.
            learning_rate: learning rate for training.
            lr_decay: coefficient for Power learning rate decay.
            L2_reg: coefficient for L2 weight decay regularization. Used to help
                prevent the model from overfitting.
            L1_reg: coefficient for L1 weight decay regularization
            momentum: coefficient for momentum. Can be 0 or None to disable.
            hidden_layer_sizes: a list of integers to determine the size of
                each hidden layer.
            activation: a lasagne activation class.
                Default: lasagne.nonlinearities.rectify
            batch_norm: True or False. Include batch normalization layers.
            dropout: if not None or 0, the percentage of dropout to include
                after each hidden layer. Default: None
            standardize: True or False. Include standardization layer after
                input layer.
        """

    self.X = T.fmatrix('x')  # patients covariates
    self.E = T.ivector('e') # the observations vector

    # Default Standardization Values: mean = 0, std = 1
    self.offset = theano.shared(numpy.zeros(shape = n_in, dtype=numpy.float32))
    self.scale = theano.shared(numpy.ones(shape = n_in, dtype=numpy.float32))

    network = lasagne.layers.InputLayer(shape=(None,n_in),
        input_var = self.X)

    if standardize:
      network = lasagne.layers.standardize(network,self.offset,
                                            self.scale,
                                            shared_axes = 0)

E AttributeError: 'module' object has no attribute 'standardize'

deepsurv/deep_surv.py:60: AttributeError
================= 2 failed, 2 passed, 4 error in 5.42 seconds ==================

How to Use DeepSurv Exactly?

I am still quite new to Python, and am trying to apply my data to deepsurv. Would you mind sharing the exact code you wrote between the two paragraphs below? I appreciate your help.

Training DeepSurv can be done in a few lines. First, all you need to do is prepare the datasets to have the following keys:
{
'x': (n,d) observations (dtype = float32),
't': (n) event times (dtype = float32),
'e': (n) event indicators (dtype = int32)
}

Then prepare a dictionary of hyper-parameters. And it takes only two lines to train a network:
network = deepsurv.DeepSurv(**hyperparams)
log = network.train(train_data, valid_data, n_epochs=500)

run EXPERIMENT=gaussian and deepsurv has a c-index around 0.5

I am trying to run the gaussian experiments.
Before you fix issue #29, deepsurv works just fine and the c-index is similar to the results presented in your paper. However, after updating to the latest version, I run the experiment again for multiple times. I've got a c-index around 0.5 for simulated non-linear dataset.
The results are as follows:

deepsurv_1  | Test metrics: {'mse': 0.35340887, 'mse_bootstrap': {'confidence_interval': (0.3502257244549425, 0.35606869292634413), 'mean': 0.3531472}, 'c_index_bootstrap': {'confidence_interval': (0.5028328594569582, 0.507284618344698), 'mean': 0.5050587389008281}, 'c_index': 0.5044739296688565}

A complete output for the experiment is as follows:

Attaching to experiments_cph_1, experiments_rsf_1, experiments_deepsurv_1
cph_1       | /usr/local/lib/python2.7/dist-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
cph_1       |   from ._conv import register_converters as _register_converters
cph_1       | /usr/local/lib/python2.7/dist-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
cph_1       |   "downsample module has been moved to the theano.tensor.signal.pool module.")
cph_1       | ('Arguments:', Namespace(dataset='/shared/data/gaussian_survival_data.h5', experiment='gaussian', plot_error=False, results_dir='/shared/results/', treatment_idx=None))
cph_1       | Loading datasets: /shared/data/gaussian_survival_data.h5
cph_1       | Training CPH Model
cph_1       | /usr/local/lib/python2.7/dist-packages/pandas/core/computation/check.py:17: UserWarning: The installed version of numexpr 2.2.2 is not supported in pandas and will be not be used
cph_1       | The minimum supported version is 2.4.6
cph_1       | 
cph_1       |   ver=ver, min_ver=_MIN_NUMEXPR_VERSION), UserWarning)
deepsurv_1  | /usr/local/lib/python2.7/dist-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
deepsurv_1  |   from ._conv import register_converters as _register_converters
rsf_1       | /usr/local/lib/python2.7/dist-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
rsf_1       |   from ._conv import register_converters as _register_converters
cph_1       | n=4000, number of events=3644
cph_1       | 
cph_1       |         coef  exp(coef)  se(coef)          z         p  lower 0.95  upper 0.95   
cph_1       | 0 -1.253e-02  9.875e-01 1.493e-02 -8.395e-01 4.012e-01  -4.180e-02   1.673e-02   
cph_1       | 1  1.048e-02  1.011e+00 1.495e-02  7.011e-01 4.832e-01  -1.883e-02   3.979e-02   
cph_1       | 2 -1.368e-02  9.864e-01 1.666e-02 -8.213e-01 4.115e-01  -4.634e-02   1.898e-02   
cph_1       | 3  1.102e-02  1.011e+00 1.655e-02  6.657e-01 5.056e-01  -2.142e-02   4.345e-02   
cph_1       | 4  1.062e-02  1.011e+00 1.658e-02  6.409e-01 5.216e-01  -2.187e-02   4.312e-02   
cph_1       | 5  1.864e-02  1.019e+00 1.669e-02  1.117e+00 2.641e-01  -1.409e-02   5.137e-02   
cph_1       | 6 -1.871e-02  9.815e-01 1.659e-02 -1.128e+00 2.594e-01  -5.124e-02   1.382e-02   
cph_1       | 7 -4.042e-02  9.604e-01 1.650e-02 -2.449e+00 1.432e-02  -7.278e-02  -8.068e-03  *
cph_1       | 8  1.483e-02  1.015e+00 1.647e-02  9.003e-01 3.680e-01  -1.746e-02   4.711e-02   
cph_1       | 9 -9.909e-03  9.901e-01 1.658e-02 -5.976e-01 5.501e-01  -4.242e-02   2.260e-02   
cph_1       | ---
cph_1       | Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 
cph_1       | 
rsf_1       | /usr/local/lib/python2.7/dist-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
rsf_1       |   "downsample module has been moved to the theano.tensor.signal.pool module.")
cph_1       | Concordance = 0.518
cph_1       | Train Likelihood: -27435.781889126756
cph_1       | Valid metrics: {'mse': 1.3591188828245035, 'c_index': 0.48575868575868575}
deepsurv_1  | ('Arguments:', Namespace(dataset='/shared/data/gaussian_survival_data.h5', experiment='gaussian', model='/models/gaussian_model_0.json', num_epochs=1000, plot_error=False, results_dir='/shared/results/', treatment_idx=None, update_fn='sgd', weights=None))
deepsurv_1  | Loading datasets: /shared/data/gaussian_survival_data.h5
deepsurv_1  | Loading json model: {"L2_reg": 4.4249755859375, "dropout": 0.4013037109375, "learning_rate": 0.00031940075434571123, "lr_decay": 0.000317275390625, "momentum": 0.9363432617187499, "batch_norm": true, "standardize": false, "n_in": 10, "hidden_layers_sizes": [17, 17, 17]}
rsf_1       | ('Arguments:', Namespace(dataset='/shared/data/gaussian_survival_data.h5', num_trees=100, results_dir='/shared/results/', treatment_idx=None))
rsf_1       | Loading datasets: /shared/data/gaussian_survival_data.h5
rsf_1       | Training RSF Model
deepsurv_1  | WARNING (theano.tensor.blas): We did not found a dynamic library into the library_dir of the library we use for blas. If you use ATLAS, make sure to compile it with dynamics library.
deepsurv_1  | 2018-01-15 01:05:16,986 - Training step 0/1000    |                         | - loss: 20.7979 - ci: 0.5023
cph_1       | Test metrics: {'mse': 1.3619913074136019, 'mse_bootstrap': {'confidence_interval': (1.3535952237897249, 1.3733029348865466), 'mean': 1.3634490793381357}, 'c_index_bootstrap': {'confidence_interval': (0.5039806821447451, 0.5086592477851751), 'mean': 0.5063199649649601}, 'c_index': 0.5061738614293155}
cph_1       | Saving Visualizations
experiments_cph_1 exited with code 0
deepsurv_1  | 2018-01-15 01:05:53,062 - Training step 100/1000  |**                       | - loss: 14.3097 - ci: 0.4924
rsf_1       | ('Train/Valid C-Index:', 0.6317852601385541)
deepsurv_1  | 2018-01-15 01:06:23,748 - Training step 200/1000  |*****                    | - loss: 11.2494 - ci: 0.4930
rsf_1       | Valid metrics: {'c_index': 0.6398349125621853}
deepsurv_1  | 2018-01-15 01:06:49,638 - Training step 300/1000  |*******                  | - loss: 9.5788 - ci: 0.4932
deepsurv_1  | 2018-01-15 01:07:15,965 - Training step 400/1000  |**********               | - loss: 8.6446 - ci: 0.4932
deepsurv_1  | 2018-01-15 01:07:42,136 - Training step 500/1000  |************             | - loss: 8.1508 - ci: 0.4934
deepsurv_1  | 2018-01-15 01:08:07,600 - Training step 600/1000  |***************          | - loss: 7.8668 - ci: 0.4937
deepsurv_1  | 2018-01-15 01:08:33,813 - Training step 700/1000  |*****************        | - loss: 7.7152 - ci: 0.4942
deepsurv_1  | 2018-01-15 01:08:59,912 - Training step 800/1000  |********************     | - loss: 7.6311 - ci: 0.4943
deepsurv_1  | 2018-01-15 01:09:26,828 - Training step 900/1000  |**********************   | - loss: 7.5882 - ci: 0.4947
deepsurv_1  | 2018-01-15 01:09:53,080 - Finished Training with 1000 iterations in 276.83s
deepsurv_1  | Training metrics: {'mse': 0.3554571, 'c_index': 0.4953763858239832}
deepsurv_1  | Valid metrics: {'mse': 0.3554571, 'c_index': 0.4953763858239832}
deepsurv_1  | Test metrics: {'mse': 0.35340887, 'mse_bootstrap': {'confidence_interval': (0.3502257244549425, 0.35606869292634413), 'mean': 0.3531472}, 'c_index_bootstrap': {'confidence_interval': (0.5028328594569582, 0.507284618344698), 'mean': 0.5050587389008281}, 'c_index': 0.5044739296688565}
deepsurv_1  | Saving Visualizations
deepsurv_1  | ('Saving model parameters to output file', '/shared/results/modelsgaussian_model_0.jsoned3c1cc6-ec7b-41f8-b926-6b91799b2031.h5')
experiments_deepsurv_1 exited with code 0
rsf_1       | Test metrics: {'c_index_bootstrap': {'confidence_interval': (0.6425199977021452, 0.6463824127128943), 'mean': 0.6444512052075198}, 'c_index': 0.6446193727615209}
experiments_rsf_1 exited with code 0

Could you please help me figure out the issue?

Hyperparameter Tuning Problem: nvcc compiler not found on $PATH

Hey everyone!

Thanks for the great package first of all! Even with limited ML experience, the DeepSurv network is working just fine. However, I am unable to run the hyperparameter tuning job.

I implement the hyperparam_search on Windows 10 with a CUDA-compatible NVIDIA graphics card on the CPU. I get the following error message (nvcc is part of CUDA, which I installed prior to the hyperparameter search):

hp_search_1 | ERROR (theano.sandbox.cuda): nvcc compiler not found on $PATH. Check your nvcc installation and try again.

I included the nvcc in the path variable (both environment + system variables), but still it is not working. Practically all of the help I find online is concerned with Linux applications, but it's not relevant to the environment I am using and I don't have enough experience to try Linux myself.

  1. Has anyone succeeded in implementing the hyperparam_search on a Windows CPU with the NVIDIA CUDA framework? Did you encounter the same problem & how did you solve it?
  2. Alternatively, did anyone submit the hyperparameter job to a cloud service and could provide a notebook / instructions on how to proceed?

Thank you very much in advance for your help. My master thesis & I are eternally grateful for your support!

Tim

Training and Testing input size error

I tried to use the test_deepsurv.py script.
First we created a random data of (5000,9) covariates for training, and (2000,9) for testing. However,when we run the code using 'n_in'=10 in the network hyperparameters it works while it doesn't in the network.predict_risk(self.test['x'])
Would anyone help me to know the meaning of the 10th dimension. I get the following error when I set n_in=9

z[0] = numpy.asarray(numpy.dot(x, y))
ValueError: shapes (5000,10) and (9,10) not aligned: 10 (dim 1) != 9 (dim 0)
Apply node that caused the error: dot(x, W)
Toposort index: 5
Inputs types: [TensorType(float32, matrix), TensorType(float64, matrix)]
Inputs shapes: [(5000, 10), (9, 10)]
Inputs strides: [(40, 4), (80, 8)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{add,no_inplace}(dot.0, InplaceDimShuffle{x,0}.0), Elemwise{Composite{(i0 * (Abs(i1) + i2 + i3))}}[(0, 2)](TensorConstant{(1, 1) of 0.5}, Elemwise{add,no_inplace}.0, dot.0, InplaceDimShuffle{x,0}.0)]]

while if I set it to 10, it throws an exception in the risk prediction as follows:

ValueError: shapes (2000,9) and (10,10) not aligned: 9 (dim 1) != 10 (dim 0)
Apply node that caused the error: dot(x, W)
Toposort index: 2
Inputs types: [TensorType(float32, matrix), TensorType(float64, matrix)]
Inputs shapes: [(2000, 9), (10, 10)]
Inputs strides: [(36, 4), (80, 8)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{Composite{(i0 * (Abs((i1 + i2)) + i1 + i2))}}[(0, 1)](TensorConstant{(1, 1) of 0.5}, dot.0, InplaceDimShuffle{x,0}.0)]]

Thank you in advance

Parameters setting

Any suggestions for parameters setting. The performance of default setting is very poor

py.test problem

I have anaconda and created an environment wit python 2.7. Ubuntu 16.04
First, I installed theano 0.8.2 then when i tried pip install Lasagne=0.2.dev1 it couldnt be found.
Then, I went to the docuemntation of Lasagne 0.2 and found that it requires the latest theano which is 0.10.
Here's the link of lassagne because when i use pip install lasagne==0.2.dev1 I get

Could not find a version that satisfies the requirement lasagne==0.2 (from versions: 0.1)
No matching distribution found for lasagne==0.2

These are the packages that I have:

cycler                    0.10.0                    <pip>
deepsurv                  0.1.0                     <pip>
functools32               3.2.3.post2               <pip>
Lasagne                   0.2.dev1                  <pip>
lifelines                 0.9.2                     <pip>
matplotlib                2.0.2                     <pip>
numpy                     1.13.1                    <pip>
openssl                   1.0.2l                        0  
pandas                    0.20.3                    <pip>
pip                       9.0.1                    py27_1  
pyparsing                 2.2.0                     <pip>
python                    2.7.13                        0  
python-dateutil           2.6.1                     <pip>
pytz                      2017.2                    <pip>
readline                  6.2                           2  
scipy                     0.19.1                    <pip>
setuptools                27.2.0                   py27_0  
six                       1.10.0                    <pip>
sqlite                    3.13.0                        0  
subprocess32              3.2.7                     <pip>
Theano                    0.10.0b1                  <pip>
tk                        8.5.18                        0  
wheel                     0.29.0                   py27_0  
zlib                      1.2.8                         3

When I try to run py.test, I get the following error (I got the same when uninstalled theano and reinstalled the 0.8.2 version):

================================================== test session starts ===================================================
platform linux -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/sara/anaconda3/envs/theano_env9, inifile:
collected 0 items / 1 errors 

========================================================= ERRORS =========================================================
___________________________________________________ ERROR collecting  ____________________________________________________
../../lib/python3.6/site-packages/_pytest/config.py:325: in _getconftestmodules
    return self._path2confmods[path]
E   KeyError: local('/home/sara/anaconda3/envs/theano_env9/lib/python2.7/site-packages/lasagne')

During handling of the above exception, another exception occurred:
../../lib/python3.6/site-packages/_pytest/config.py:356: in _importconftest
    return self._conftestpath2mod[conftestpath]
E   KeyError: local('/home/sara/anaconda3/envs/theano_env9/lib/python2.7/site-packages/lasagne/conftest.py')

During handling of the above exception, another exception occurred:
lib/python2.7/site-packages/numpy/core/__init__.py:16: in <module>
    from . import multiarray
E   ImportError: /home/sara/anaconda3/envs/theano_env9/lib/python2.7/site-packages/numpy/core/multiarray.so: undefined symbol: _Py_ZeroStruct

During handling of the above exception, another exception occurred:
lib/python2.7/site-packages/lasagne/__init__.py:12: in <module>
    import theano
lib/python2.7/site-packages/theano/__init__.py:42: in <module>
    from theano.configdefaults import config
lib/python2.7/site-packages/theano/configdefaults.py:5: in <module>
    import numpy
lib/python2.7/site-packages/numpy/__init__.py:142: in <module>
    from . import add_newdocs
lib/python2.7/site-packages/numpy/add_newdocs.py:13: in <module>
    from numpy.lib import add_newdoc
lib/python2.7/site-packages/numpy/lib/__init__.py:8: in <module>
    from .type_check import *
lib/python2.7/site-packages/numpy/lib/type_check.py:11: in <module>
    import numpy.core.numeric as _nx
lib/python2.7/site-packages/numpy/core/__init__.py:26: in <module>
    raise ImportError(msg)
E   ImportError: 
E   Importing the multiarray numpy extension module failed.  Most
E   likely you are trying to import a failed build of numpy.
E   If you're working with a numpy git repo, try `git clean -xdf` (removes all
E   files not under version control).  Otherwise reinstall numpy.
E   
E   Original error was: /home/sara/anaconda3/envs/theano_env9/lib/python2.7/site-packages/numpy/core/multiarray.so: undefined symbol: _Py_ZeroStruct

During handling of the above exception, another exception occurred:
../../lib/python3.6/site-packages/_pytest/config.py:362: in _importconftest
    mod = conftestpath.pyimport()
../../lib/python3.6/site-packages/py/_path/local.py:662: in pyimport
    __import__(modname)
lib/python2.7/site-packages/lasagne/__init__.py:14: in <module>
    raise ImportError("Could not import Theano." + install_instr)
E   ImportError: Could not import Theano.
E   
E   Please make sure you install a recent enough version of Theano. Note that a
E   simple 'pip install theano' will usually give you a version that is too old
E   for Lasagne. See the installation docs for more details:
E   http://lasagne.readthedocs.org/en/latest/user/installation.html#theano

During handling of the above exception, another exception occurred:
../../lib/python3.6/site-packages/py/_path/common.py:367: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
../../lib/python3.6/site-packages/py/_path/common.py:416: in gen
    for p in self.gen(subdir):
../../lib/python3.6/site-packages/py/_path/common.py:416: in gen
    for p in self.gen(subdir):
../../lib/python3.6/site-packages/py/_path/common.py:416: in gen
    for p in self.gen(subdir):
../../lib/python3.6/site-packages/py/_path/common.py:405: in gen
    dirs = self.optsort([p for p in entries
../../lib/python3.6/site-packages/py/_path/common.py:406: in <listcomp>
    if p.check(dir=1) and (rec is None or rec(p))])
../../lib/python3.6/site-packages/_pytest/main.py:682: in _recurse
    ihook = self.gethookproxy(path)
../../lib/python3.6/site-packages/_pytest/main.py:587: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
../../lib/python3.6/site-packages/_pytest/config.py:339: in _getconftestmodules
    mod = self._importconftest(conftestpath)
../../lib/python3.6/site-packages/_pytest/config.py:364: in _importconftest
    raise ConftestImportFailure(conftestpath, sys.exc_info())
E   _pytest.config.ConftestImportFailure: ImportError("Could not import Theano.\n\nPlease make sure you install a recent enough version of Theano. Note that a\nsimple 'pip install theano' will usually give you a version that is too old\nfor Lasagne. See the installation docs for more details:\nhttp://lasagne.readthedocs.org/en/latest/user/installation.html#theano",)
E     File "/home/sara/anaconda3/envs/theano_env9/lib/python2.7/site-packages/lasagne/__init__.py", line 14, in <module>
E       raise ImportError("Could not import Theano." + install_instr)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================ 1 error in 0.87 seconds =================================================

about right censoring data

Hello!
I have some question about the data processing. How to deal with the right censoring data which e=0? Just drop them? In experiment,we can only use the data which e=1?
Looking forward to your reply.

deepsurv.plot_log(log) does not work

Followed example given on the homepage for visualizing the training and validation curves, but it gives a 'key error', claiming the log file generated from network.train() does not contain the key 'loss'.

hyper parameter selection error

I try to tune the hyper parameter for whas dataset and get the following error:

Attaching to hyperparamsearch_hp_search_1
hp_search_1  | /usr/local/lib/python2.7/dist-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
hp_search_1  |   from ._conv import register_converters as _register_converters
hp_search_1  | 2018-01-19 08:11:29,163 - __main__ - DEBUG - Parameters: Namespace(box='/box_constraints.0.json', dataset='whas', logdir='/shared/logs', num_epochs=500, num_evals=100, num_folds=3, update_fn='adam')
hp_search_1  | 2018-01-19 08:11:29,163 - __main__ - DEBUG - Loading dataset: whas
hp_search_1  | Traceback (most recent call last):
hp_search_1  |   File "/hyperparam_search.py", line 196, in <module>
hp_search_1  |     x, y, strata = load_dataset(args.dataset)
hp_search_1  |   File "/hyperparam_search.py", line 105, in load_dataset
hp_search_1  |     ds = utils.load_datasets(dataset)['train']
hp_search_1  |   File "/deepsurv/utils.py", line 17, in load_datasets
hp_search_1  |     with h5py.File(dataset_file, 'r') as fp:
hp_search_1  |   File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/files.py", line 271, in __init__
hp_search_1  |     fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
hp_search_1  |   File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/files.py", line 101, in make_fid
hp_search_1  |     fid = h5f.open(name, flags, fapl=fapl)
hp_search_1  |   File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip-nCYoKW-build/h5py/_objects.c:2840)
hp_search_1  |   File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip-nCYoKW-build/h5py/_objects.c:2798)
hp_search_1  |   File "h5py/h5f.pyx", line 78, in h5py.h5f.open (/tmp/pip-nCYoKW-build/h5py/h5f.c:2117)
hp_search_1  | IOError: Unable to open file (Unable to open file: name = 'whas', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)
hyperparamsearch_hp_search_1 exited with code 1

my docker file is as follows:

FROM floydhub/dl-docker:cpu

ADD . /tmp/pip
RUN pip install /tmp/pip/master.zip

RUN \
  echo "h5py==2.7.0\n\
        lifelines==0.9.4\n\
        logger==1.4\n\
        Optunity==1.1.1\n\
        tensorboard-logger==0.0.3\n\
        matplotlib==2.0.0" > /requirements.txt && \
  pip install -U pip --proxy=xxx && \
  pip install -U numpy --proxy=xxx && \
  pip install -r /requirements.txt --proxy=xxx

COPY . /

CMD [ "python", "-u", "/hyperparam_search.py", \
"/shared/logs", \
"whas", \
"/box_constraints.0.json", \
"100", \
"--update_fn", "adam", \
"--num_epochs", "500", \
"--num_fold", "3" ]

Can anyone tell me why I get the error?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.