mrgiovanni / modelsgenesis Goto Github PK
View Code? Open in Web Editor NEW[MICCAI 2019] [MEDIA 2020] Models Genesis
License: Other
[MICCAI 2019] [MEDIA 2020] Models Genesis
License: Other
Hi Zongwei,
Thanks for your great work. Your implemention is very clearly 👍
I'm trying to pretrain on mine own dataset, but i don't know how much training data been used when you pretrain (patch level). Could you tell me this detail ?
Thanks,
Jing
I want to try your code on 2D. Is the network I am training using UNET?
Hello @MrGiovanni, thanks a lot for your work! It is very cool and interesting.
I am working on image classification (disease positive vs disease negative) in CT of head and neck cancer. The problem is that, as you state, the genesis chest model was trained on the -1000 HU to 1000 HU, and my interesting range would be much less (approx -50 HU to 200 HU). My dataset size is rather limited (less than 600 patients) so that is why I think a pre-trained model is a good approach but I was wondering if you have any advice on this problem or any data normalization technique. Thanks a lot in advance,
Best,
Agustina
error happens with:
output = activate(final_convolution, axis=1)
cause its output is not a keras layer.
suggest to replace with this line:
output = keras.layers.Softmax(axis = 1)(final_convolution)
it works!!!
By the way, good job 纵苇! Your pretrained model improves my project performance.
Hello,
I’m using some image transformations of Models Genesis (local pixel shuffling, non-linear transformation, in- and out-painting) to train an autoencodeur. As I’m working with 2D images, I adapted the code available to process 2D images. The transformations appear to work successfully.
However, they’re really time-consuming : on average 20 seconds per batch, with a batch size of 128 2D images. Because of this, training is impossible as it is far too slow. I tried decreasing the probability of each transformation (from 0.5 to 0.25), decreasing the number of blocks in pixel shuffling and using less transformations, which helped, but the process remains too slow.
Did you face the same issue during training? If yes, how did you overcome this issue?
Thank you !
thanks for your excellent work! there are some question
Thanks for your excellent paper and repo,
As you mentioned in the infinite_generator_3D code, you used folds 7,8,9 as test data, did you compute your reported cpm (0.971 in the paper) just on these folds?
thanks a lot.
This is a wonderful work, and there are some questions about data perocessing.
Question 1: When preparing our own image data, we always scale the data between -1 and 1. And I see the author process the data into 0 and 1. Is it necessary to scale the input data to 0 and 1?
Question 2: The author provide a UNet3D based model, and the number of the model class is 1. If we want use the author's pretrained model with 4 class (or other class), is it worked by loading the network layer without "module.out_tr.final_conv.weight", and just fine-tune the model? Do we need fixing the model layer and with a two stage training?
Thanks and looking forward your reply!
Hello, I have two questions.
1.I would like to ask if I want to use your pre-training weight, do I have to use your model as my feature extraction model?
2.Can you provide a link to download your pre-training weights?
I don't find the implement of permute_dimensions() function in BilinearUpSampling.py
demo in file "ModelsGenesis/tree/master/pytorch"
self.out_glb_avg_pool = F.avg_pool3d(self.base_out, kernel_size=self.base_out.size()[2:]).view(self.base_out.size( [0],-1)
view(self.base_out.size( [0],-1) ???
Hi !
First, thank you for making your incredible work available. Your results are outstanding and the code is pretty easy to understand.
In order to use your work in another application that I am investigating I wanted first to replicate your results on the BraTs dataset.
I have downloaded and preprocessed the data : resizing the images to 64x64x64 and normalizing between 0 and 1.
However, when finetuning Genesis CT I don't manage to get anything close to your results.
Is there something I am missing ? The whole image has to be fed to the network or split in patches ? Also in the code example given in the keras folder num_classes is set to 2 for segmentation which I am not sure to understand as the convolution will create (2, 64, 64, 64) sized mask.
Thank you for your help,
Regards,
Camille
I tried to download the LUNA16 dataset from zenodo.org, there are 10 files and I checked the md5. Everything is correct, but the Archive manager prompts "An error occurred while loading the archive". Is there anything I've done wrong? I have installed 7-zip extension.
Such as mrbrains18 dataset
Hello! When looking at your Readme.MD you mention that the infinite_cube_generator.py generates N cubes from each patient, N is the scale correct?
However, when using your own 3D extraced cubes from luna, when opening the file from a single patient, there are a lot more than 32 cubes. For example bat_32_s_64x64x32_0.npy has dimensions of (2848, 64, 64, 32) which are 2848 cubes. Could you explain to me what scale represents then in your code and what you use scale for when generating cubes? Did you extract the cubes you provided with a much larger scale?
Also, could you please explain the difference between the purpose of input_rows and crop_rows?
It seems that the pretrained weight .h5 is for Keras. Could you please provide .pth pretrained weight for Pytorch?
Thank you
Hello, I want to use my own data to refine your trained model according to the instructions you gave on GitHub, but Training from scratch appears when it is running. What is the reason for this? Looking forward to your reply!
Tks for the great job. But from my side, i have serveral questions
Hello,
First thanks for your amazing work. I am training my own genesis network Using lung CT scans. The patches generated includes a reasonable amount of Lung part while when training with Genesis_Chest_CT, the output of the network will learn one single value for all pixels, any idea why this happened?
Thanks,
Tian
Hi Zongwei:
Thanks for the excellent job.
I met with some difficulties when trying to learn Models Genesis from my own unlabeled data.
Best
lzp
Hi!
Thank you for sharing this code!
I have a newbie question,
We are trying to run it with the weights: genesis_nnunet_luna16_006.model on the Task004_Hippocampus data,
but we get the print : "Training from scratch", meaning we are not using the weights.
I checked and the condition: (model_dict[key].shape == pretrained_dict[key].shape) is False.
It would be great if you are familiar with the reason and maybe can clarify it for us,
and tell us if what we are trying to do is even possible.
Thanks in advance!
hi, Giovanni, I just re-implemented the MG-2D on my dataset and met with some other bug for cpu explosion.
When I used MG for pre-training, I found that the utilization rate of the CPU would increase strangely, but the utilization rate of the GPU fluctuated greatly, and the buzzer of the machine PC would alarm soon. After several rounds of debugging, I found that it was this piece of code that caused the CPU utilization to sharply increase to 4000%+!!!!
ModelsGenesis/pytorch/utils.py
Line 45 in 1393252
Maybe this is a special case in my machine, and it has no reference value.
Thanks. :-)
Hi Zongwei,
Thanks for the release of the Pytorch version.
However, there is some difference between the weights of Keras and Pytorch.
The network utilizes the UpSampling3D
as the up convolution layer in the Keras version, therefore, there is no trainable weight in those layers.
In contrast, the Pytorch model adopts the ConvTranspose3d
, which contains trainable weights.
Which version is recommended for us if we would like to reproduce the excellent performance in your accepted paper?
Best,
Yu-Cheng
Hi Zongwei:
Thank you for the excellent work.
I have some questions and any help would be appreciated.
How should I determine the input patch size? I mean, is there any particularity about whether the input patch is large or small. If my target is lung tissue such as pulmonary blood vessels, should I take a smaller patch size?
I am working on the segmentation of pulmonary vessels. Some previous work tried the Gaussian pyramid for multi-scale processing, and then used a 5x5x5 patch size into a double-layer network. Here, since your model has multiple layers of downsampling, does this mean that my patch can be correspondingly larger than 5 and no longer require multi-scale preprocessing ?
Sincerely looking forward to your reply.
Best Wishes!
zlin
Hello
The results showed Models Genesis top any 2D approaches, including ImageNet models. So I‘d like to try your pretrained weight of 2D.
Hi,
I find your work very interesting. I have tried to open a request for the pretrained weights, but it seems the link is down.
Thanks so much for sharing the code , and I have some questions.
1.could you give more information about the version of packages(requirements.txt),I cant reconstruct
the same env with these info.
2. When I using tf=1.14 and keras=2.3.1 . There occurs an error "“Tensor” object hs no attribute "assign""
I think the role of skip-connection is rather disturbing in your paper where reconstruction tasks are usually performed, what do you think?
Using skip connection in reconstruction often transfers a lot of information intact in the initial layer. However, you put a strong distortion on input, probably which is why the cheating problem does not occur.
what do you think about it? Does skip-connection play an important role in your reconstruction task?
Thanks for your work. Is it possible to give pytorch version in the future?
when I load the pre-trained weights, I got the ----ValueError: axes don't match array
anyone has the same problem? it's the problem of the version of keras?
Hello Zongwei, first of all thank you for your work. It appears to be a great idea and it is very clearly explained.
I was wondering if a version of your work will be developed for PET modality ?
Do you have some advises to train a model for PET modality using your approach (especially for the patch size) ?
I am working on a dataset with 128x128x(N slices) PET images.
Do you think that patches of size 64x64x64 would help a network (VNet for example) to learn from 128x128x(N slices) PET images ?
For how many epochs did you train your models ? (I think I saw 10 000 in one of your answers ?, before any early stop) ?
I also want to use your VNet Chest-CT implementation.
Do you have any additional advice regarding the fine-tuning ? (fine-tuning only the last layers, cubes of size 64 ?)
Thank you very much for your answers !
Is there any h5 files which I can download? Or I must pre-train the model by myself.
If I do classification task.
in TargetNet
self.base_model(x)
self.base_out = self.base_model.out512
Why not modify UNet3D.py
if self.do_seg:
self.out_up_256 = self.up_tr256(self.out512,self.skip_out256)
self.out_up_128 = self.up_tr128(self.out_up_256, self.skip_out128)
self.out_up_64 = self.up_tr64(self.out_up_128, self.skip_out64)
self.out = self.out_tr(self.out_up_64)
return self.out
else:
return self.out512
Doing so will save a lot of memory and FLOPS.
Hi!
Do you have any pretrained models for 2D x-ray data you could possibly share?
Thanks in advance
Marian
The size of the cube in your pre-trained code is 64X64. But I want to use your pre-trained model to segmentation the nodule , but the DICOM image size is 512X512, what should I do?
Hi Zongwei,
This is a good work!
I have a question regarding the downstream tasks. Do you resize or rescale or resample the LUNA and MRI 3D volumes and their segmentation masks for the downstream tasks?
Thanks,
Ananya
Thank you for all the hard work put into this project and for making the pretrained models and code available for public use! Would it be possible to also provide the code and pretrained weights for the 2D models, in particular Genesis Chest CT 2D, as mentioned in the paper?
Hi,
can you please provide the test code which i can run for my own data to see the performance.
Thanks!
Hi Zongwei,
Thanks for sharing such a great work.
I have tried to reproduce the results shown in the Tab. 2 of the paper, however, I can't get the comparable scores.
Could you release the experimental setting (preprocessing, loss functions, optimizer ...) of those experiments (NCC, NCS, ECC, LCS, BMS)?
Thanks for the help.
Best,
Yu-Cheng
Hi,
Thank you for releasing your code, I very much appreciate the extensive experiments in your paper.
I'd like to compare your self-supervision training procedure against a different self-supervision task on some different 2D medical imaging data. Would you be able to release the code you used to transform your 2D data when making comparisons against ImageNet (just the transformation code would be enough for me, not necessarily the whole 2D training procedure)? I imagine the code is pretty similar to your 3D code, but it would be nice to get the right parameters from you.
Since training on 3D is rather resource intensive, do you think that 2D pretraining performance is a good indicator to compare among different self-supervision tasks?
Thank you for your amazing work. I am implementing your method on my own dataset. I have questions regarding the PyTorch version of in-painting and out-painting. It looks like you replace the pixels within the window with random values in both transformations. For example, in in-painting, you use:
x[:,
noise_x:noise_x+block_noise_size_x,
noise_y:noise_y+block_noise_size_y,
noise_z:noise_z+block_noise_size_z] = np.random.rand(block_noise_size_x,
block_noise_size_y,
block_noise_size_z, ) * 1.0
which assigns random values to the block, instead of:
x[:,
noise_x:noise_x+block_noise_size_x,
noise_y:noise_y+block_noise_size_y,
noise_z:noise_z+block_noise_size_z] = np.random.rand()
In this case the block is replaced with a single random value, which is the case according to your paper:
We then assign a random value to all pixels outside the window while retaining the original intensities for the pixels within. As for in-painting, we retain the original intensities outside the window and replace the intensity values of the inner pixels with a constant value.
This is also the case suggested by fig.1 in the paper. Which one is correct? Also I don't quite understand why the random noise is multiplied with 1.0 in both in-painting and out-painting. Since the np.random.rand function generates float numbers, the multiplication seems unnecessary.
I am recreating your article, can the network input be changed to 128128128 without changing other parameters?
Hi there,
I would really like to read some additional details about nnU-Net and models genesis. So far you seem to have taken the first place in Task03, but it is difficult so see whether that is a significant result (given that I am getting some variation when running the same training several times which will also translate into different test set performances). Overall, your submission on the Decathlon is below ours, indicating that the pretraining may not be beneficial on all tasks. This makes is difficult to really estimate the impact of your pre-training strategy.
Specifically, I would be interested in how much your pretrained models help in semi-supervised learning. Say you take all non-LiTS/Task03 (Task03 is essentially LiTS) datasets with livers in them (BCV Abdomen, KiTS, Pancreas (?), ...) and run models genesis on them for pretraining, how well does your pretrained nnU-Net perform when fine tuned on 10, 20, 50 etc LiTS cases for the LiTS task? Can you beat the nnU-Net baseline by a significant margin if you use all these additional datasets for pretraining?
Best,
Fabian
hello, how do you think if i add GAN model into this framework, will I have chance to get more discrimitive feature from this autoencoder?
Hello! So interested in your work.@MrGiovanni
I don't quite understand what the parameter “len_depth” is for in infinite_generator_3D.py.
Could you please tell me more about that?
for d in range(config.input_deps):
for i in range(config.input_rows):
for j in range(config.input_cols):
for k in range(config.len_depth):
if crop_window[i, j, d+k] >= config.HU_thred:
t_img[i, j, d] = crop_window[i, j, d+k]
d_img[i, j, d] = k
break
if k == config.len_depth-1:
d_img[i, j, d] = k
d_img = d_img.astype('float32')
d_img /= (config.len_depth - 1)
d_img = 1.0 - d_img
This is a wonderful work and has great impact on 3D medical imaging deep learning.
I am trying to fine-tune Models Genesis to classify HCC pathological biomark (negative OR positive) based on liver CT data.
I'm fairly new in this field and there is a question about data pre-processing. I have abdomen CT image datas (DICOM files) and HCC ROI segmentation result (3D nii file, which value is 0 or 1 to identify the ROI region ), i don't konw how to process them to put into the model? Merge them into one npy file?
Could you please list the network architecture of 3D Models Genesis? I plan to employ the encoder as a fixed feature extractor.
Thank you for your help,
Best Regards.
Shanhu
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.