nadeemlab/impartial

Interactive deep learning whole-cell segmentation and thresholding using partial annotations

impartial's Issues

Stoping the server did not reset the user session

After the a couple to epochs in training, when trying to stop the server and start a new user session , the previous data is retrieved.

Plugin Error while running Inference

While running inference, the plugin gives a 504 error. Error pops up after Infer runs on a few images and then stops. Error is independent of the image. The behavior is random, but able to reproduce on .png format images on cloud server. Typically seeing this if dataset has more than 8-9 images. Error is only seen on Cloud server not on local deployment.

Unable to run multi-channel png format image via public endpoint

Training not starting on Dataset images with 3 channels or png format files with 3 channels.
Progress bar gets stuck initializing with not no error or warning to the user.
When pressing infer plugin gives an error.

demo for Fiji plugin

Hi!
If you could have a demo recording showing how to use Fiji plugin and MONAI Label server (especially the DeepEdit interactive annotation functionality), that would be great!

Plugin hangs on attempting to close

Close on Plugin not working. Fiji app/plugin needs to be 'force quit' everytime.

Deploy ImPartial as a service

After implementing the original ImPartial pipelines as a MONAI Label app and developing an ImageJ/Fiji plugin that interacts with this API, we now need define the AWS infrastructure that would allow multiple users to benefit from the system.

Some of the requirements are:

• The service is publicly available but with restricted access. The restrictions are not completely defined yet but some ideas are to limit the number of iteration to 3, which is equivalent to give the user access to train for around 300 epochs. Another restriction would be to limit the availability time on 2hrs per session.
• The user interacting with ImPartial would have access to a dedicated GPU resource within the restrictions mentioned above.
• The user will upload their dataset and submit annotations through the ImageJ plugin.
• Once the session is over, the user will be able to download the labels for the full dataset and the last checkpoint of the trained model.
• In exchange for this free resource, we (ImPartial) will store all of the uploaded dataset, submitted labels and trained model.

How to start from weak annotations in numpy arrays?

Hello,

I am interested in ImPartial, not sure if it will solve my problem. I found the documentation alittle bit difficult to follow, so I appreciate if there is a recommended workflow.

I have weak annotations make by color thresholding for bright field images. I am try to enhance these labels. Can I use them as input instead of scribbles? Also, can will I interact with the output after that?

Thanks in advance!

can't start monailabel server using `-a api`

Hi!
After I run:

cd impartial
monailabel start_server -a api -s ~/ImPartial/Data/Vectra_WC_2CH_tiff/

I get error:

[2022-11-22 09:48:21,068] [1509998] [MainThread] [INFO] (uvicorn.error:75) - Started server process [1509998]
[2022-11-22 09:48:21,068] [1509998] [MainThread] [INFO] (uvicorn.error:45) - Waiting for application startup.
[2022-11-22 09:48:21,069] [1509998] [MainThread] [INFO] (monailabel.interfaces.utils.app:38) - Initializing App from: /home/hju/ImPartial/impartial/api; studies: /home/hju/ImPartial/Data/Vectra_WC_2CH_tiff; conf: {}
[2022-11-22 09:48:21,092] [1509998] [MainThread] [INFO] (monailabel.utils.others.class_utils:37) - Subclass for MONAILabelApp Found: <class 'main.Impartial'>
[2022-11-22 09:48:21,105] [1509998] [MainThread] [ERROR] (uvicorn.error:119) - Traceback (most recent call last):
  File "/home/hju/anaconda3/envs/monailabel-impartial-env/lib/python3.9/site-packages/starlette/routing.py", line 635, in lifespan
    async with self.lifespan_context(app):
  File "/home/hju/anaconda3/envs/monailabel-impartial-env/lib/python3.9/site-packages/starlette/routing.py", line 530, in __aenter__
    await self._router.startup()
  File "/home/hju/anaconda3/envs/monailabel-impartial-env/lib/python3.9/site-packages/starlette/routing.py", line 612, in startup
    await handler()
  File "/home/hju/anaconda3/envs/monailabel-impartial-env/lib/python3.9/site-packages/monailabel/app.py", line 104, in startup_event
    instance = app_instance()
  File "/home/hju/anaconda3/envs/monailabel-impartial-env/lib/python3.9/site-packages/monailabel/interfaces/utils/app.py", line 51, in app_instance
    app = c(app_dir=app_dir, studies=studies, conf=conf)
  File "/home/hju/ImPartial/impartial/api/main.py", line 25, in __init__
    for c in get_class_names(lib.configs, "TaskConfig"):
  File "/home/hju/anaconda3/envs/monailabel-impartial-env/lib/python3.9/site-packages/monailabel/utils/others/class_utils.py", line 144, in get_class_names
    module = importlib.import_module("." + name, package=current_module_name)
  File "/home/hju/anaconda3/envs/monailabel-impartial-env/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/hju/ImPartial/impartial/api/lib/configs/impartial.py", line 5, in <module>
    import lib.infers
  File "/home/hju/ImPartial/impartial/api/lib/infers/__init__.py", line 1, in <module>
    from .impartial import Impartial
  File "/home/hju/ImPartial/impartial/api/lib/infers/impartial.py", line 25, in <module>
    from monailabel.interfaces.tasks.infer_v2 import InferType
ModuleNotFoundError: No module named 'monailabel.interfaces.tasks.infer_v2'

I pip installed monailabel and other packages in requirements.txt within a conda environment, as when I tried to follow the python virtual environment setup in the readme https://github.com/nadeemlab/ImPartial#monai-label, openslide-python can't be installed. And according to https://openslide.org/api/python/#installing , it seems to better install openslide using some package manger like Anaconda.

I've done some testing on the monailabel installation. E.g., I can start a monailabel server if using a monailabel sample pathology application:

monailabel start_server -a apps/pathology/ -s ~/ImPartial/Data/Vectra_WC_2CH_tiff/

Also, using monailabel radiology sample application together with 3DSlicer also works. Any ideas?

Error Loading images on ImageJ plugin when running on 10.0.3.117 server

Running the Monai server with DAPI dataset images. Connected and ran the latest imageJ plugin.
The plugin doesn't show any raw images in the drop down and giving following error.

WARNING] Ignoring unsupported output: dialog [org.nadeemlab.impartial.ImpartialDialog] Exception in thread "AWT-EventQueue-0" java.security.PrivilegedActionException: java.security.PrivilegedActionException: org.json.JSONException: JSONObject["id"] not found. at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74) at java.awt.EventQueue.dispatchEvent(EventQueue.java:730) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) Caused by: java.security.PrivilegedActionException: org.json.JSONException: JSONObject["id"] not found. at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84) at java.awt.EventQueue$4.run(EventQueue.java:733) at java.awt.EventQueue$4.run(EventQueue.java:731)

Evaluation of Impartial model performance within Monai:

Impartial is a semi-supervised deep learning method which does whole cell segmentation. With using minimal number of scribbles by an expert pathologist.
In order to evaluate the performance of the impartial models during and after training there are 2 ways:

1. Human expert in the loop feedback, where the user reviews the inference results from the model, identifies the erroneous cells and provide additional scribbles and start model training again. (This is current working of Impartial with the human-in-the-loop)

2. In order to quantitatively measure the model's performance, many times pathologists have "fully annotated" images along with unlabeled data. These full labelled test images can be used to track model performance. Hence, we wish to incorporate typical evaluation metrics into our Impartial-Monailabel framework.

   Currently, monai supports "image" (upload_image) and "label" (save_label) APIs, we utilize them to submit "image" and "scribbles" respectively. Essentially, we utilize "label" attribute to submit the "scribbles"

   • We were hoping if there was a way to submit a "ground truth" label OR another attribute like "scribble" be added? OR utilize "tag" parameter of "save_label" API?
     For example:

         "image2": {
               "image": {
                 "ext": ".tif",
                 "info": {
                   "name": "image2.tif"
                 }
               },
               "labels": {
                 "gt": {
                   "ext": ".zip",
                   "info": {
                     "name": "image2_gt.zip"
                   }
                 },
                 "scribble": {
                   "ext": ".zip",
                   "info": {
                     "name": "image2_scribble.zip"
                   }
                 }
        }

   • Add API like "save_ground_truth()"
   • Add API like Evaluate(image, ground_truth) or evaluate() ?

Training time optimization

As an interactive service, ImPartial should allow users to train the model over their datasets in an efficient way, ideally under 5 mins using a single GPU. This training time is measured using 100 epochs and ~4000 sample patches per iteration.

As of today, it takes around 15 mins with the configurations mentioned above on a 4 GPUs machine.

sample images for using Fiji plugin

Hi!
Could you point me to some sample digital pathology images to download so that I can try out the Fiji plugin? My own current data is in .svs format and can't be opened by Fiji.

Questions new user

Hi, thanks for this great model.
However I have some questions.

How do you provide multi class labels groundtruths? Do you use different grayscale level on a single image to represent the different labels or for each image of training one need a groundtruth image for each label (1 for nuclei and 1 for cytoplasm).

Which input image size does one need? i believe 512*512 but can it be bigger?

Why does one need to build the Fiji plugin locally and you don't provide directly the Fiji Plugin from Update site?

Thanks a lot for your help.

interactive annotation & install ImageJ plugin

Hi there!
Really appreciate you put much effort in building ImPartial!

I want to try out ImPartial on my digital pathology data, especially want to use the interactive annotation feature. Is this feature available now or is still under active development?

I'm following the Monai-label and Impartial Integration to set up everything. In Client - Install ImageJ plugin, the command git checkout feat/imagej-plugin outputs error: pathspec 'feat/imagej-plugin' did not match any file(s) known to git, and from git branch -a it seems this branch is not there.

Besides, I can't find the impartial_imagej-0.1.jar used in:

Copy the latest build imagej plugin from 'ImPartial/imagej-plugintarget/impartial_imagej-0.1.jar' to '/Applications/Fiji.app/plugins'