zaccharieramzi/fastmri-reproducible-benchmark

Try several methods for MRI reconstruction on the fastmri dataset. Home to the XPDNet, runner-up of the 2020 fastMRI challenge.

Hi there 👋

I am Zaccharie Ramzi, an AI Research Scientist at Meta working on neural interfaces. Before that, I was a post-doc working with Gabriel Peyre at ENS Ulm on optimization and implicit deep learning. Even before that, I was a PhD student working in 3 institutions: Inria in the Parietal team, Neurospin in the METRIC team, and the CEA astrophysics department in the Cosmostat team.

The topic of my PhD was deep learning for MRI reconstruction. This made me interested in all things connected to:

• convex non-smooth optimization
• computer vision
• MRI and healthcare
• deep learning

I like to write TensorFlow (and now PyTorch and Jax a bit) code not only to write neural networks but also to build other related functionalities. Open source is definitely my motto.

fastmri-reproducible-benchmark's Issues

About Test Files Number in qualitative_validation_for_net.ipynb in spite of Contrast Selection

Hi everyone,

I want to reconstruct Fastmri Single Coil Knee Test Dataset with Unet. Because of this, i used testing mode in Zerofilled2DSequence and Mask2DSequence in qualitative_validation_for_net.ipynb like this.

test_path = '/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/'
test_gen_zero = ZeroFilled2DSequence(test_path, af=AF, norm=True, mode='testing',contrast=None, mask_seed=0)
test_gen_scaled = Masked2DSequence(test_path, mode='testing', af=AF, scale_factor=1e6, contrast=None, mask_seed=0)

len(test_gen_scaled)
50

But I know that full Test files in Fastmri Single Coil Knee Dataset are 108. I checked this as:

import os
path, dirs, files = next(os.walk("/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/"))
file_count = len(files)
print(file_count)
108

It is true. In Testing mode, there is a problem in spite of none selection of contrast in Zerofilled2DSequence and Mask2Dsequence.

But, in validation dataset, there is no problem of counting seen like this:

val_path = '/content/drive/My Drive/UNET_Colab/knee_singlecoil_val/singlecoil_val/'
val_gen_zero = ZeroFilled2DSequence(val_path, af=AF, norm=True, mode='validation')
val_gen_scaled = Masked2DSequence(val_path, mode='validation', af=AF, scale_factor=1e6)

len(val_gen_scaled)
199

It is true for validation dataset of FastMRI Single Coil Knee Dataset.

What is the problem in testing mode of Zerofilled2Dsequence and Masked2DSequence?(It is valid problem in commit bcd3fdd,too) I am really confused. Because of this number inconsitency, the reconstruction number of images is conflicted with the total number of slices of test dataset.

If you return, i will appreciate.

Best regards,

Problem too many files open

When training the pdnet, I hit an error after 24 iterations (~4 hours so not very reproducible) that says: OSError: [Errno 24] Too many open files.

I checked the number of processes created when fitting, it is very high for the second epoch (above 35 specified, double that).

I need to reproduce this to create a tf issue.

I think this has to do with sequences so trying right now with datasets and it looks ok, also the speed is equivalent so early try-outs were not representative of the real use case.

About helpers/nn_mri.py file and import lrelu

Hi everyone,

For the testing , i want to run qualitative_validation_for_net.ipynb. But there is an error in this line.

from fastmri_recon.helpers.nn_mri import lrelu

No module named 'fastmri_recon.helpers'

In fastmri_recon folder , there is no helpers folder and consequently there is no file named nn_mri.py in helpers folder.

Is there any problem while uploading the helpers folder and its contexts?

Best regards,

Subclassed basis for all the cross-domain models

Right now all the models have a lot in common but nothing is shared via a common basis.
The simplest and most debuggable way to do so, is to have a subclassed basis for all the cross-domain models, that will be readily transformable in any model, potentially with multi-coil. This will allow extensive comparison on different parameters, like type of data consistency, buffer size and number of alternation.
Things to take into account:

• buffer of values like in PD-net
• 2 types of data consistency (and therefore different forward/adjoint operators)
• specify how to begin/end the networks, i.e. with a K network or an I network.
• extension to multi-coil either in the operator, or at the end
• extension to non-uniform

Refactor datasets

Oasis and fastMRI sequences have a lot in common.

To ease as much as possible the integration of other databases, the common backbone of these sequences should be extracted, and the current classes re-implemented.

About ZeroFilled2DSequence and Masked2DSequence in Unet

Hi everyone,

I want to ask about class ZeroFilled2DSequence and Masked2DSequence in Fastmri_sequences.py. When i wanted to test the Fastmri single coil test and val dataset in Unet reconstruction in terms of both metrics and reconstructions, i realized that val_gen scaled and test_gen_scaled had 50 h5 files names in spite of being available 108 h5 file in the path folder. So the numbers of reconstruction of unet results and metrics results are inconsistent with number of test and val path . To see the difference, i looked at the filenames in test_gen_scaled, names are shown like this:

'/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/file1000055_v2.h5',
'/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/file1000093_v2.h5',
'/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/file1000128_v2.h5',

But in the test path, there are some names that are not available in test_gen_scaled. These italics(missing paths in test_gen_scaled) are illustrated like this:

/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/file1000022_v2.h5
/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/file1000036_v2.h5
'/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/file1000055_v2.h5,
/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/file1000056_v2.h5
/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/file1000063_v2.h5
'/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/file1000093_v2.h5',
'/content/drive/My Drive/UNET_Colab/knee_singlecoil_test_v2/singlecoil_test_v2/file1000128_v2.h5',

Why test_gen_scaled , test_gen_zero (these are shown in val datasets,too) are skipping 2 h5 files in filepath? This changes the results in metrics and numbers of reconstruction in unet.

If you return, i will appreciate.

Best regards,

Have a clear listing of the data requirements

About Saving Unet The Results in .h5 format

Hi,

I saved the model for Fastmri Public Leaderboard but always encountered an error as EvalException: file1001066_v2.h5 does not exist in your submission. It is available in that folder. How did you save and upload your model results that i really want to know?

Thank you for all your support,

Best Regards,

Correct submission results naming for brain data

I think contrarily to what's indicated, in the case of brain data, the '_v2' is not to be added in the submitted filename.

Get rid of pytorch in the codebase

Having pytorch in the codebase is confusing.
We should remove it to also lighten the requirements.

About fastmri_recon/config.py

Hi,

I want to ask you about fastmri_recon/config.py. In test folder, 108 h5(AF4=50, AF8=58) files exist. But in the main code, as you see there are 120 files .

import os

FASTMRI_DATA_DIR = os.environ.get('FASTMRI_DATA_DIR', '/media/Zaccharie/UHRes/')
OASIS_DATA_DIR = os.environ.get('OASIS_DATA_DIR', '/media/Zaccharie/UHRes/')
LOGS_DIR = os.environ.get('LOGS_DIR', './')
CHECKPOINTS_DIR = os.environ.get('CHECKPOINTS_DIR', './')

n_volumes_train = 973
n_volumes_val = 199
n_volumes_test = {
4: 60,
8: 60,
}

Am i wrong about the number of test files?

Thank you for your patience and support.

Best Regards,

Error when running all tests

When running tests separately there is no problem.
However, when running all of them, like in Travis, for the non cartesian tests, we see this error coming up:

E         TypeError: in user code:
E         
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/fastmri_recon/data/datasets/preprocessing.py:43 from_kspace_to_nc_kspace_and_traj  *
E                 nc_kspace = nufft(nfft_ob, orig_image[:, None, ..., 0], traj, image_size)
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/fastmri_recon/models/utils/fourier.py:144 nufft  *
E                 kspace = forward_op(image_adapted, ktraj)
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tfkbnufft/nufft/interp_functions.py:211 _map_body  *
E                 y_not_shifted = run_interp(tf.reshape(_x, (tf.shape(_x)[0], -1)), _tm, params)
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tfkbnufft/nufft/interp_functions.py:99 run_interp  *
E                 coef, arr_ind = calc_coef_and_indices(
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tfkbnufft/nufft/interp_functions.py:41 calc_coef_and_indices  *
E                 arr_ind = tf.zeros((M,), dtype=int_type)
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py:2677 wrapped  **
E                 tensor = fun(*args, **kwargs)
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py:2730 zeros
E                 shape = ops.convert_to_tensor(shape, dtype=dtypes.int32)
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tensorflow/python/framework/ops.py:1341 convert_to_tensor
E                 ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py:321 _constant_tensor_conversion_function
E                 return constant(v, dtype=dtype, name=name)
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py:262 constant
E                 allow_broadcast=True)
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py:300 _constant_impl
E                 allow_broadcast=allow_broadcast))
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py:451 make_tensor_proto
E                 _AssertCompatible(values, dtype)
E             /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py:331 _AssertCompatible
E                 (dtype.name, repr(mismatch), type(mismatch).__name__))
E         
E             TypeError: Expected int32, got None of type 'NoneType' instead.

I have no idea why this shows up, and how to debug it, because it doesn't happen when forcing eager mode obviously. It basically means that at some point the length of the trajectory is not defined in advance, which is not the case.

Correct alternating domain sequence

Right now for cross domain networks, you can only have really alternating sequences.
We should correct that by having the i_domain//2 replaced by 2 counters, one for each specific domain.

Error in PDNet running upon compiling

Hi,

I came here from Public LeaderBoard of FastMRI and I don't have a lot of experience in Tensorflow. While running PDNet, I am facing attribute error in precision_policy.loss_scale is None: AttributeError: 'Policy' object has no attribute 'loss_scale'.

Have you got this error, too? Furthermore,Have you reached that challenge result with these codes?

Differentiate between contrasts

For both the training and the results it would be important to see how the contrast allows for a finer reconstruction.

This doesn't mean we can't have models trained on both contrasts then fine-tuned.

Add journal paper reference

Since the MDPI paper has been published it should also be a choice of citation of this repo.

In addition, the notebooks used to generate the results should be updated with the new package organisation and linked in the README.

Binder configuration

For easier sharing, the project should be in a binder-friendly state.

The 2 main points are:

• have an exact configuration
• have a notebook showing the difference between zero-filled, classical reconstruction, and the cascading approach.

Add TensorboardImage callback

Inspire from work done in other project to include a tensorboard image callback.

Correct url in setup.py

The url indicates a github which is not this one.

Migrate code to TF 2.0 with keras API

To make use of the latest software available and use all the power of tf, we should migrate the code from keras to tf.keras with tf 2.0

In particular we will be able to make use of tensorflow/tensorflow#31409 to get rid of the temporary fftshifts.

Allow to select custom mask when instantiating a sequence

Right now the only mask we use for the sequences, is the fastMRI suggested mask, which is cartesian non VDS.

We should be able to have a custom mask function (with a default) when instantiating sequences.

We should probably add the VDS cartesian mask as well.

Issues when running kikinet_sep_approach_af4

Hi there,

I'm trying to reproduce the results of the paper Ramzi, Z.; Ciuciu, P.; Starck, J.-L. Benchmarking MRI Reconstruction Neural Networks on Large Public Datasets. Appl. Sci. 2020, 10, 1816. and adapt the models to the reconstruction of other types of MR images. According to the checkpoints provided, it seems the training of the kikinet used the script fastmri_recon/training_scripts/single_coil/kikinet_sep_approach_af4.py. So right now I'm trying to run this training script, but I encounter some problems.

I modified the scripts a little bit by setting the use_multiprocsessing as false, disabling the GPU, setting n_epochs as 1 and setting chkpt_cback to save the checkpoints only at the last epoch. I keep getting this warning:

W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.

Could someone explain what is going wrong here and how should I modify the script?

The whole script is presented as follows

import os                   # modified
import os.path as op
import time

from keras_tqdm import TQDMCallback
import tensorflow as tf
from tensorflow.keras.callbacks import TensorBoard, ModelCheckpoint, LearningRateScheduler

from fastmri_recon.data.sequences.fastmri_sequences import Masked2DSequence, KIKISequence
from fastmri_recon.models.functional_models.kiki_sep import kiki_sep_net
from fastmri_recon.models.utils.data_consistency import MultiplyScalar
from fastmri_recon.models.utils.non_linearities import lrelu


train_path = 'C:/Users/yy263/Desktop/knee_singlecoil_train/singlecoil_train_ori/singlecoil_train/'  # modified
val_path = 'C:/Users/yy263/Desktop/knee_singlecoil_train/singlecoil_val_ori/'                       # modified
test_path = 'C:/Users/yy263/Desktop/knee_singlecoil_train/singlecoil_test/'                         # modified

cuda_visible_devices = '-1'                                                                         # modified
os.environ["CUDA_VISIBLE_DEVICES"] = ','.join(cuda_visible_devices)                                 # modified

n_samples_train = 34742
n_samples_val = 7135

n_volumes_train = 973
n_volumes_val = 199


AF = 4
train_gen_last = Masked2DSequence(train_path, af=AF, inner_slices=8, rand=True, scale_factor=1e6)
val_gen_last = Masked2DSequence(val_path, af=AF, scale_factor=1e6)
train_gen_i = KIKISequence(train_path, af=AF, inner_slices=8, rand=True, scale_factor=1e6, space='I')
val_gen_i = KIKISequence(val_path, af=AF, scale_factor=1e6, space='I')
train_gen_k = KIKISequence(train_path, af=AF, inner_slices=8, rand=True, scale_factor=1e6, space='K')
val_gen_k = KIKISequence(val_path, af=AF, scale_factor=1e6, space='K')

run_params = {
    'n_convs': 25,
    'n_filters': 48,
    'noiseless': True,
    'lr': 1e-3,
    'activation': lrelu,
}
multiply_scalar = MultiplyScalar()
n_epochs = 1                                        # modified


def learning_rate_from_epoch(epoch):
    return 10**(-(epoch // (n_epochs/3)) - 3)


def train_model(model, space='K', n=1):
    print(model.summary(line_length=150))
    run_id = f'kikinet_sep_{space}{n}_af{AF}_{int(time.time())}'
    chkpt_path = f'checkpoints/{run_id}' + '-{epoch:02d}.hdf5'
    print(run_id)

    chkpt_cback = ModelCheckpoint(chkpt_path, period=n_epochs)              # modified
    log_dir = op.join('logs', run_id)
    tboard_cback = TensorBoard(
        profile_batch=0,
        log_dir=log_dir,
        histogram_freq=0,
        write_graph=True,
        write_images=False,
    )
    lrate_cback = LearningRateScheduler(learning_rate_from_epoch)
    tqdm_cb = TQDMCallback(metric_format="{name}: {value:e}")
    tqdm_cb.on_train_batch_begin = tqdm_cb.on_batch_begin
    tqdm_cb.on_train_batch_end = tqdm_cb.on_batch_end
    if space == 'K':
        train_gen = train_gen_k
        val_gen = val_gen_k
    elif space == 'I':
        if n == 2:
            train_gen = train_gen_last
            val_gen = val_gen_last
        elif n == 1:
            train_gen = train_gen_i
            val_gen = val_gen_i
    model.fit_generator(
        train_gen,
        steps_per_epoch=n_volumes_train,
        epochs=n_epochs,
        validation_data=val_gen,
        validation_steps=1,
        verbose=0,
        callbacks=[tqdm_cb, tboard_cback, chkpt_cback, lrate_cback,],
        # max_queue_size=35,
        use_multiprocessing=False,                  # modified
        workers=35,
        shuffle=True,
    )
    return model

# first K net training
model = kiki_sep_net(None, multiply_scalar, to_add='K', last=False, **run_params)
train_model(model, space='K', n=1)
model = kiki_sep_net(model, multiply_scalar, to_add='I', last=False, **run_params)
train_model(model, space='I', n=1)
model = kiki_sep_net(model, multiply_scalar, to_add='K', last=False, **run_params)
train_model(model, space='K', n=2)
model = kiki_sep_net(model, multiply_scalar, to_add='I', last=True, **run_params)
train_model(model, space='I', n=2)

And here is what I get

2020-11-21 14:05:43.618232: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2020-11-21 14:05:45.301825: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2020-11-21 14:05:45.482548: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2020-11-21 14:05:45.490774: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: RON-MJ09HCC4
2020-11-21 14:05:45.494004: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: RON-MJ09HCC4
2020-11-21 14:05:45.496516: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-11-21 14:05:45.520179: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x198d47b6560 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-11-21 14:05:45.524038: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Model: "model"
______________________________________________________________________________________________________________________________________________________
Layer (type)                                     Output Shape                     Param #           Connected to
======================================================================================================================================================
kspace_input (InputLayer)                        [(None, 640, None, 1)]           0
______________________________________________________________________________________________________________________________________________________
lambda (Lambda)                                  (None, 640, None, 1)             0                 kspace_input[0][0]
______________________________________________________________________________________________________________________________________________________
lambda_1 (Lambda)                                (None, 640, None, 1)             0                 kspace_input[0][0]
______________________________________________________________________________________________________________________________________________________
concatenate (Concatenate)                        (None, 640, None, 2)             0                 lambda[0][0]
                                                                                                    lambda_1[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d (Conv2D)                                  (None, 640, None, 48)            912               concatenate[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_1 (Conv2D)                                (None, 640, None, 48)            20784             conv2d[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_2 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_1[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_3 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_2[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_4 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_3[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_5 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_4[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_6 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_5[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_7 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_6[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_8 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_7[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_9 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_8[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_10 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_9[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_11 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_10[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_12 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_11[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_13 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_12[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_14 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_13[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_15 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_14[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_16 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_15[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_17 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_16[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_18 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_17[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_19 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_18[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_20 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_19[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_21 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_20[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_22 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_21[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_23 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_22[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_24 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_23[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_25 (Conv2D)                               (None, 640, None, 2)             98                conv2d_24[0][0]
______________________________________________________________________________________________________________________________________________________
mask_input (InputLayer)                          [(None, 640, None)]              0
______________________________________________________________________________________________________________________________________________________
lambda_2 (Lambda)                                (None, 640, None, 1)             0                 conv2d_25[0][0]
======================================================================================================================================================
Total params: 499,826
Trainable params: 499,826
Non-trainable params: 0
______________________________________________________________________________________________________________________________________________________
None
kikinet_sep_K1_af4_1605985545
WARNING:tensorflow:`period` argument is deprecated. Please use `save_freq` to specify the frequency in number of batches seen.
WARNING:tensorflow:From kikinet_sep_approach_af4_ori.py:94: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
Please use Model.fit, which supports generators.
Training:   0%|                                                                                                                                                                            | 0/50 [00:00<?, ?it/s]Traceback (most recent call last):                                                                                                                                                         | 0/973 [00:00<?, ?it/s]
  File "kikinet_sep_approach_af4_ori.py", line 100, in <module>
    train_model(model, space='K', n=1)
  File "kikinet_sep_approach_af4_ori.py", line 94, in train_model
    shuffle=False,
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\util\deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1479, in fit_generator
    initial_epoch=initial_epoch)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\keras\engine\training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\keras\engine\training.py", line 855, in fit
    callbacks.on_train_batch_end(step, logs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\keras\callbacks.py", line 390, in on_train_batch_end
    self._call_batch_hook(ModeKeys.TRAIN, 'end', batch, logs=logs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\keras\callbacks.py", line 298, in _call_batch_hook
    batch_hook(batch, logs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\keras_tqdm\tqdm_callback.py", line 117, in on_batch_end
    self.append_logs(logs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\keras_tqdm\tqdm_callback.py", line 136, in append_logs
    metrics = self.params['metrics']
KeyError: 'metrics'
2020-11-21 14:05:59.152617: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
         [[{{node PyFunc}}]]
Training:   0%|                                                                                                                                                                            | 0/50 [00:13<?, ?it/s]
Epoch: 0:   0%|                                                                                                                                                                           | 0/973 [00:13<?, ?it/s]

(fastMRI) C:\Users\yy263\Desktop\fastmri-reproducible-benchmark-master\fastmri-reproducible-benchmark-master>python kikinet_sep_approach_af4_ori.py
2020-11-21 14:07:31.782620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2020-11-21 14:07:33.440426: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2020-11-21 14:07:33.624847: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2020-11-21 14:07:33.635379: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: RON-MJ09HCC4
2020-11-21 14:07:33.645497: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: RON-MJ09HCC4
2020-11-21 14:07:33.651373: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-11-21 14:07:33.685297: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2532a87c4f0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-11-21 14:07:33.692565: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Model: "model"
______________________________________________________________________________________________________________________________________________________
Layer (type)                                     Output Shape                     Param #           Connected to
======================================================================================================================================================
kspace_input (InputLayer)                        [(None, 640, None, 1)]           0
______________________________________________________________________________________________________________________________________________________
lambda (Lambda)                                  (None, 640, None, 1)             0                 kspace_input[0][0]
______________________________________________________________________________________________________________________________________________________
lambda_1 (Lambda)                                (None, 640, None, 1)             0                 kspace_input[0][0]
______________________________________________________________________________________________________________________________________________________
concatenate (Concatenate)                        (None, 640, None, 2)             0                 lambda[0][0]
                                                                                                    lambda_1[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d (Conv2D)                                  (None, 640, None, 48)            912               concatenate[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_1 (Conv2D)                                (None, 640, None, 48)            20784             conv2d[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_2 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_1[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_3 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_2[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_4 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_3[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_5 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_4[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_6 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_5[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_7 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_6[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_8 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_7[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_9 (Conv2D)                                (None, 640, None, 48)            20784             conv2d_8[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_10 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_9[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_11 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_10[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_12 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_11[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_13 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_12[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_14 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_13[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_15 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_14[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_16 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_15[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_17 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_16[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_18 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_17[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_19 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_18[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_20 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_19[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_21 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_20[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_22 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_21[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_23 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_22[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_24 (Conv2D)                               (None, 640, None, 48)            20784             conv2d_23[0][0]
______________________________________________________________________________________________________________________________________________________
conv2d_25 (Conv2D)                               (None, 640, None, 2)             98                conv2d_24[0][0]
______________________________________________________________________________________________________________________________________________________
mask_input (InputLayer)                          [(None, 640, None)]              0
______________________________________________________________________________________________________________________________________________________
lambda_2 (Lambda)                                (None, 640, None, 1)             0                 conv2d_25[0][0]
======================================================================================================================================================
Total params: 499,826
Trainable params: 499,826
Non-trainable params: 0
______________________________________________________________________________________________________________________________________________________
None
kikinet_sep_K1_af4_1605985653
WARNING:tensorflow:`period` argument is deprecated. Please use `save_freq` to specify the frequency in number of batches seen.
WARNING:tensorflow:From kikinet_sep_approach_af4_ori.py:94: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
Please use Model.fit, which supports generators.
Training:   0%|                                                                                                                                                                            | 0/50 [00:00<?, ?it/s]Traceback (most recent call last):                                                                                                                                                         | 0/973 [00:00<?, ?it/s]
  File "kikinet_sep_approach_af4_ori.py", line 100, in <module>
    train_model(model, space='K', n=1)
  File "kikinet_sep_approach_af4_ori.py", line 94, in train_model
    shuffle=True,
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\util\deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1479, in fit_generator
    initial_epoch=initial_epoch)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\keras\engine\training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\keras\engine\training.py", line 855, in fit
    callbacks.on_train_batch_end(step, logs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\keras\callbacks.py", line 390, in on_train_batch_end
    self._call_batch_hook(ModeKeys.TRAIN, 'end', batch, logs=logs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\tensorflow\python\keras\callbacks.py", line 298, in _call_batch_hook
    batch_hook(batch, logs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\keras_tqdm\tqdm_callback.py", line 117, in on_batch_end
    self.append_logs(logs)
  File "C:\Users\yy263\Anaconda3\envs\fastMRI\lib\site-packages\keras_tqdm\tqdm_callback.py", line 136, in append_logs
    metrics = self.params['metrics']
KeyError: 'metrics'
2020-11-21 14:07:47.026364: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
         [[{{node PyFunc}}]]
Training:   0%|                                                                                                                                                                            | 0/50 [00:12<?, ?it/s]
Epoch: 0:   0%|                                                                                                                                                                           | 0/973 [00:12<?, ?it/s]

Add the link to the article in the README

Now that the article has been published on hal, the link needs to be in the README and ask for a citation if need be.

Publish the weights of the nets trained on fastMRI

This will allow people to more rapidly benchmark their new ideas against the current nets.

KeyError: 'metrics' in unet_approach training

Hi everyone,

Now, i encountered a training error. It is

Training: 0%
0/10 [00:00<?, ?it/s]
Epoch 0: 0%
0/5 [00:00<?, ?it/s]

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-8-bf021aacb644> in <module>
----> 1 model.fit(
      2     train_set,
      3     steps_per_epoch=5,
      4     epochs=n_epochs,
      5     validation_data=val_set,

~/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)
     64   def _method_wrapper(self, *args, **kwargs):
     65     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
---> 66       return method(self, *args, **kwargs)
     67 
     68     # Running inside `run_distribute_coordinator` already.

~/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
    853                 context.async_wait()
    854               logs = tmp_logs  # No error, now safe to assign to logs.
--> 855               callbacks.on_train_batch_end(step, logs)
    856         epoch_logs = copy.copy(logs)
    857 

~/.local/lib/python3.8/site-packages/tensorflow/python/keras/callbacks.py in on_train_batch_end(self, batch, logs)
    388     if self._should_call_train_batch_hooks:
    389       logs = self._process_logs(logs)
--> 390       self._call_batch_hook(ModeKeys.TRAIN, 'end', batch, logs=logs)
    391 
    392   def on_test_batch_begin(self, batch, logs=None):

~/.local/lib/python3.8/site-packages/tensorflow/python/keras/callbacks.py in _call_batch_hook(self, mode, hook, batch, logs)
    296     for callback in self.callbacks:
    297       batch_hook = getattr(callback, hook_name)
--> 298       batch_hook(batch, logs)
    299     self._delta_ts[hook_name].append(time.time() - t_before_callbacks)
    300 

~/.local/lib/python3.8/site-packages/keras_tqdm/tqdm_callback.py in on_batch_end(self, batch, logs)
    115         self.inner_count += update
    116         if self.inner_count < self.inner_total:
--> 117             self.append_logs(logs)
    118             metrics = self.format_metrics(self.running_logs)
    119             desc = self.inner_description_update.format(epoch=self.epoch, metrics=metrics)

~/.local/lib/python3.8/site-packages/keras_tqdm/tqdm_callback.py in append_logs(self, logs)
    134 
    135     def append_logs(self, logs):
--> 136         metrics = self.params['metrics']
    137         for metric, value in six.iteritems(logs):
    138             if metric in metrics:

KeyError: 'metrics'

Have anyone faced a problem like this? How did you solve this problem?

Thank you for your help, Stay safe

Best regards,

Correcting linter errors everywhere

Linter is supposed to help not to bother.

I should fix the linter errors everywhere.

They mostly come from tensorflow imports, and click function calls.

Maybe other people have done this before.

Parallel MRI

Include parallel MRI methods in the nets.

Adding travis ci shield

To reflect that this work is unit tested, a shield for travis ci could be added (indeed not everyone uses refined github).

The shield is: https://travis-ci.com/zaccharieramzi/fastmri-reproducible-benchmark.svg?branch=master

Get rid of the fit_generator to use the more modern fit

fit_generator is going to be deprecated soon, so we need to switch to fit (which is more convenient anyway).

(for the training of keras models).

Refactor data utils

The data utils are a lot of function with different names but all use the same idea behind, it should be refactored.

Use tf datasets instead of keras sequence

This can be done in the same PR as #15 .

Basically the idea is to have a way cleaner data pipeline, and gain x3 time on training.

H5pyDeprecationWarning: The default file mode will change to 'r' (read-only) in h5py 3.0

This warning appears when loading an hdf5 file. This can be easily fixed by just specifying the read mode when opening the file.

Build fake h5 datasets for unit tests in order not to rely on fastMRI data

Right now after multiplt sollicitations to the fastMRI contact, I don't know whether I can use some sample data in my repo to do my unit tests.

Therefore, I need to build a fake temporary dataset (which could be of reduced size btw) to perform my unit tests.

Refactor training/eval/inference scripts

The different scripts for training, evaluation and inference have a lot in common. We should refactor them.

At the same time, click APIs could be refactored as well.

about validation_for_net

Hi everyone,

I want to ask you a question about validation_for_net to calculate metrics (PSNR and SSIM). But As you know, the test and challenge dataset of FastMRI is zero filled default. (with aliasing artifact). But in validation_for_net , we are calculating the validation metrics instead of test dataset. All metrics calculation is evaluated in terms of zero filled reconstruction. Is there any step for calculation for the test dataset?

If you return, i will appreciate.

Best regards,

Remove unused checkpoints/logs from git tracking

Some logs/checkpoints are unused and we should remove them to make sure the repo in not cluttered.

Correct imports in experiments notebooks

The experiments notebook have leftover imports.

Refactor training scripts

The current training scripts do not offer a lot of flexibility.

We should shape them in the pdnet way.

Support for non-uniform fourier transform

The k-space data can sometimes be acquired in a non-uniform way in the k-space.

To account for that we need to change the fourier operator from a classical uniform one to a non uniform one.

This is now available through tfkbnufft.

In addition to integrating it, the corresponding sampling patterns (replacing masks) should also be integrated.

Maybe to start with, single coil would be more manageable and therefore we would need a bit of a revamp in the single coil architectures and their training.

Produce graph with PSNR of PD-net function of number of samples

This is something that a lot of people are curious about: what are the data requirements of the PD-net.

This is also interesting in another way: are using all the data we have to the fullest?

unet_approach_af4.py running error: No h5 files of given contrast None at path

Hi everyone,

I encountered an error while running.(the screenshot is attached ). Has anyone faced a problem like this and known how to overcome and solve this?

If you help, i will appreciate.

Include GANs

Candidate nets include:

• DAGAN
• Compressed sensing with meta-learning
• RefineGAN

Potential issues:

• dealing with different image sizes

Allow other activation functions in the nets

Right now, ReLU is hardcoded as the activation function for most nets.
We want to be able to use some other activation function.

Switch to tensorflow addons for tqdm progress bar

Since tensorflow/addons#708 has been resolved, we could use tensorflow addons to create the tqdm progress bar instead of using keras_tqdm which is having some issues with tf 2.x.

Correct KIKI-net

Currently the implemented kiki-net is actually ikik. Shouldn't make much of difference.

However, the training is supposed to be iterative, i.e. the i and k nets should be trained separately.

Moreover, to avoid dying ReLU, we should use LeakyReLU rather than ReLU (this is probably why the net is so bad at this point after some trials on a small part of the training set).

Weight decay is also important to fix the overfitting (I immediately saw it when training the first K-net), however, a first good step would just be to add L2 weight regularisation and in a second step use AdamWOptimizer that implements the weight decay correction noted in this paper.

Have submission scripts ready to work on TGCC

Basically the idea is to have scripts that execute the training scripts.

But also, there is a need to refactor the code so that the data, logs and checkpoints are correctly saved (probably with environment variables).

Error in Running The Cascadenet

Hello,

In qualitative_validation_for_net, I am always facing the error named as

TypeError: reco_net_from_test_file() takes 2 positional arguments but 4 were given 
(<ipython-input-15-d6d7947e4efa> in save_figure_for_params(reco_function, test_gen, name, **net_params)
   ---> 34         im_recos= reco_function(*test_gen[image_index], model))

in testing mode.

For this problem, I tried to modify the code but I unfortunately encountered another problem. There are the same problem in PD, Kiki Sep 16. Is there anyone who solved this problem?

Add back patch to work with fftshift in tf 2.0

Because tf 2.1 won't be installed on TGCC before a month, I need to put back the patch to work with fftshift (and inverse) in order to have the networks work in 2.0.

This way I can benefit from the multiple gpus of the TGCC to train and fine tune my networks.

Readme reedition

Remove Loraks from readme as do not want to maintain.

Also mention the problem mentionned here about the checkpoints folder.

Better validation notebooks

Currently the validation notebooks are in an unstable state.

They should be made fit for public use, that is with proper comments, figures and options.