Comments (10)
Yes that makes sense and yes - you don't have to specify patch_level in this case if you only have one level. Also it turns out the code didn't support passing an arbitrary csv file list (without defined seg/patch parameters). I just updated the code (so you might need to do a git pull or otherwise to fetch the update) so that this functionality is supported, i.e. now if you just pass a csv-file with just list of filenames, without parameter values, the parameter values should be automatically inferred from the preset you specified or default values.
from clam.
The patch_size are set differently to accomodate the fact that we're patching at different levels. The way to think about it is that suppose our goal to do analysis using 256 x 256 patches at 20X, that's the same region as a 512 x 512 patch at 40X, except it's lower resolution.
So if we don't have a level for 20x, we can patch at 40x using 512 x 512 and downsample it before running feature extraction.
create_patches_fp.py doesn't actually do any cropping/image reading - it only saves coordinates of patches. The actual reading/resizing of patches happens in extract_features_fp.py. So if your goal is to get features for classification, you should do what's suggested above (i.e. use different patch sizes when patching, and use appropriate --custom_downsample or equivalently --target_patch_size when running extract_features_fp.py)
from clam.
Hi, there is currently no support for extracting patches from multiple resolutions at once (that might be something to add in the future). If you'd like you can perform them individually by following the instructions and running 3 set of commands for the 3 different resolutions. as long as you know what downsamples those 3 magnifications correspond to in your WSI files.
from clam.
Hi,
Sorry for continuing on a closed thread.
I'm having a similar issue- I have a dataset with ~200 non-pyramidal slides of which 150 are imaged at 40x and the rest at 20x. Since they are at different resolutions, I'm trying to extract the patches at 20x resolution for all the WSIs for uniformity of patch magnifications.
You mentioned in the above comment to run different patch extraction commands for different resolutions. Could you please provide more details as to how I could achieve it?
I'm not quite sure how to alter the parameters for different magnifications.
Thank you very much!
from clam.
Hi, the simplest way I can think of, is to pre-generate csv files corresponding to slides scanned at different magnifications. i.e. have 1 csv file corresponding filenames that 40x slides, and pass it to the script via --process_list, with the appropriate patch_level, and custom_downsample (if necessary) and also do the same with another csv file corresponding 20x slides.
from clam.
Hi,
Thanks for the quick response!
As a follow-up, please let me know if I've understood it correctly.
- I would create 2 process_lists- one for all 40x WSIs and the other for all 20x ones.
- Next, for the 40x WSIs, I run the create_patches script with a patch_size of 512 and use a custom downsample of 2.
- For the 20x WSIs, I simply run the create_patches script with a patch_size of 256
As a result, I would have patches of 256x256 with a magnification of 20X for all WSIs if I'm correct.
The patch_level you mentioned in the previous comment is not needed as all the WSIs are non-pyramidal with only one level[0], right?
Thanks in advance!
from clam.
@fedshyvana et. al., thanks so much for this wonderful software!
I am wondering about different-magnification images too but have a slightly different case. We have been given 70 images (four-class subtyping) at these resolutions:
41.221663X n=45
63.062992X n=17
82.443327X n=8
Here are my questions:
- In order to analyze all images at the first level (~41X), we can do as you suggest above, breaking up the data into different CSV files, running the 41X images at patch_level=0 and the 82X images at patch_level=1. However, does it really make sense to also set the patch_size to different values as suggested above? I'd think that as long as the correct patch_levels are used, then patch_size should be set to the same value (say 256) for all images at all magnifications. (Perhaps setting patch_size to a different value only applied to @mayurmallya since they had to use the custom_downsample option instead of patch_level?)
- Regarding the 63X images, which do not have any native zoom levels in common with the rest of the images, I see the custom_downsample option has been removed from create_patches_fp.py (it's still present in create_patches.py). If it had been present then I would have assumed to set it to 1.529851 and to again keep the patch_size at 256. But I'm not sure what to do regardless. Would you recommend I use create_patches.py? If so, does setting only custom_downsample make sense?
- I also see custom_downsample in create_patches.py assumes it's an int. If I were to change this to float and allow arbitrary values, do you see any complications arising?
- All these points aside, given my dataset, are there any overall recommendations you would have to tackle this subtyping problem using CLAM?
Thanks very much in advance!
from clam.
Thanks very much for your fast response @fedshyvana!
Just to be clear, I do have pyramidal levels for all my images as opposed to @mayurmallya. My issue is with simultaneously setting patch_level
and patch_size
/step_size
.
Maybe the following settings will clarify my question, if you wouldn't mind please confirming that this is the best way to go given that (1) I have levels for all images and (2) no levels in my 63x images can match any in the 41x and 82x images. In the end I want features from all 70 images from 256 x 256 patches at 41x:
create_patches_fp.py --process_list filenames_of_41x.csv --patch_level 0 --patch_size 256 --step_size 256 --preset MY-PRESET --save_dir /path/to/all/patches
create_patches_fp.py --process_list filenames_of_82x.csv --patch_level 1 --patch_size 256 --step_size 256 --preset MY-PRESET --save_dir /path/to/all/patches
create_patches_fp.py --process_list filenames_of_63x.csv --patch_level 0 --patch_size 392 --step_size 392 --preset MY-PRESET --save_dir /path/to/all/patches
extract_features.py --target_patch_size 256 --data_h5_dir /path/to/all/patches --data_slide_dir /path/to/all/images --csv_path /path/to/all/images.csv
I want to confirm that the following line is not what I should use for the 82x images:
create_patches_fp.py --process_list filenames_of_82x.csv --patch_level 1 --patch_size 512 --step_size 512 --preset MY-PRESET --save_dir /path/to/all/patches
Thanks so much!
from clam.
@andrew-weisman yes that looks right to me.
from clam.
Thanks very much for confirming @fedshyvana!
from clam.
Related Issues (20)
- The problem of uniform magnification HOT 2
- Issue running main.py HOT 2
- Run your own dataset HOT 2
- Some WSI cannot be segmented HOT 1
- Issue running create_heatmaps.py HOT 2
- Failed to install using clam.yaml due to conflicts HOT 2
- Running eval.py HOT 4
- Threshold adjustment in preprocessing stage HOT 1
- `custom_downsample` specification flipped for heatmaps? HOT 3
- Seeking Advice: Low Accuracy in CLAM Training on CAMELYON16 Dataset HOT 1
- What data types can be processed HOT 4
- Some artifacts are identified as tissue regions HOT 2
- Only one GPU during training HOT 4
- Many Nan values in the extracted feature HOT 7
- When running create_heatmaps it appears to take up a lot of existing memory! Is this normal? HOT 1
- Extracting features problem with h5 files HOT 1
- Problems encountered during feature extraction HOT 1
- For binary sub-disease classification HOT 1
- About parameters in process_list.autogen.csv HOT 1
- The code really needs fixing HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clam.