What are opportunities for tuning the model?

[mahmoodlab]

What are opportunities for tuning the model?

[fedshyvana]

@tmabraham Besides the hyperparamters you mentioned, the github was updated just a couple days ago, which gives you more options for tuning. e.g. you can now try a single-attention branch version of clam (--model_type clam_sb) which I recommend as it should run slightly faster while giving a favorable performance, and B can be set via --B, etc. Similarly, I have added the option to disable instance-level clustering via --no_inst_cluster. See options under main.py.
Other things you could try is adjusting model architecture, for the problems I tried, I had limited data for training so the model is fairly light-weight with a couple of layers and under 1M parameters (model_clam.py). You can try adding more stack of layers to/enlarge both the model itself as well as the attention module.
Also this repo is specifically for weakly-supervised problems where only the slide-level labels are provided. If your dataset gives additional labels at the pixel/ROI-level, you might want to think about whether there are other methods/modifications you can use that would take advantage of these additional labels to get better performance.

@jjhbw Yes, you can in principle try using different pre-trained models in extract_features.py w/ very slight changes to the code. Just make sure that if your embedding dimension changes, you would also need to make the corresponding changes to the model.

[jjhbw]

I don't want to derail this discussion as it seems focussed on training hyperparameters, but i'm also eager to hear stories of people using different pretrained models for patch-level feature extraction (extract_features.py).

Intuitively, I would speculate that the ImageNet embeddings may leave some descriptive power on the table, as the ImageNet training images are very different from tissue slide images.

[tmabraham]

@fedshyvana Is the single-attention branch version from the paper? Or is this a new version of the model?

I have an older version of the repository which I have already made changes to for my custom task, is there a way I can go through the code and disable the mutual exlcusivity assumption?

Also, I only have slide-level labels on the test sert for the task I am looking at right now.