broadinstitute/segmentation_experiments

BSRP summer internship project attempting to streamline nucleus segmentation

segmentation_experiments

BSRP summer internship project attempting to streamline nucleus segmentation

segmentation_experiments's Issues

Add functionality to load image objects

Separate reusable code from the notebook

Specially for the color functionality, create a separate module and keep your functions out there.

Create function to load data

Reduce load_COLOR_data() functions to a single function

There are 4 functions that have pretty much identical code in the load_all_data module, except some names that could be parameterized. Can you reduce the code to a single function that receives these names as parameter and returns the expected data?

The four functions are:

• load_purp_data
• load_pink_data
• load_default_data
• load_bw_data

Crop Images

• Take 64x64x3 shaped crops of images
• Normalize dataset by taking the same number of crops across all image types

Reclassify images in CSV file

Reclassify image labels in the CSV file to reflect the color of the image

Optimize composites.py

• create tensor to hold entire mask collection (similar to a cube)
• combine all layers in a single numpy function (vs the current for loop)
• multiply cube by a vector (similar to i+1) to give each slice it's own color
• collapse the cube into a single matrix again (numpy.sum(axis='z'))

Create a notebook for unmixing colors in purple images

Data Augmentation (regression)

• Downscale images that are equal to a given bin to every bin below it
  • only upscale to no more than 10 bins above it
• Generate more crops for larger bins with less examples
• Plot distribution of sizes after all these transformations
  • ideally, sizes should be uniformly distributed

Composite Mask images

Composite of all nucleus masks of one image into a single matrix (labeled matrix in scikit-image).

Average size of nuclei in images

Given the masks of nuclei in an image, compute the average size of those masks.

• Separate this code in a new module called sizes.py
• Create a different notebook to test the functionality.

Prototype a segmentation network

We will start solving the segmentation problem with a fully convolutional network. It can be a U-Net with ResNet backbone or similar.

Probably not as part of this issue, but for future reference we may want to include the harmonic encoding for ground truth objects as well as the progressive shrinking of networks.

Reorganize modules in a package and rename notebooks

Create a directory called functions (for now) and move all the Python modules (.py files) there. Then make sure that all your notebooks run by updating the imports from import xxx to import functions.xxx. Perhaps you need to create an empty __init__.py file.

Rename your notebooks with a two digit number to keep track of the order in which they should be run. For instance: 01_preprocessing.ipynb and 02_tensorflow_experiment.ipynb, and so on.

Train CNN to classify microscopy images by color

Tensorflow Experimentation

• Restrict code to run on free GPU in dgx server
• Run experiment using mnist dataset on GPU

Implement an image rescaling function

The goal is to have a function that gets an image, the ground truth composite and the target average object size as parameters and returns a new image with a new composite both rescaled to a new size that approximates the requested average object size. It would look like this:

rescaled_image, rescaled_composite = FUNCTION(image, composite, target_size)

The calculations that the function needs to make are:

• Current average object size.
• Ratio of current size (cs) and target object size (ts) as follows: ratio=cs/ts
• Use the skimage.transform.rescale function to transform the image and the composite
• Return the result

Verify visually and numerically that the object size of the rescaled images is close to the target size.

Regression Function

• similar pipeline to color classification
• Load composite of masks for each image to get the ground truth
• Extend crop function to get crop of image and mask composite
• With these crops, compute average size of nucleus
• Do not need to keep composites for training, just need to create the label
• Make sure code is robust; can it handle a blank crop (no nuclei in image)
• Neural network should work the same
• Except the Dense layer output will only be one: we just need the size
• Use absolute error loss function (or square error)
• Metric is error, not accuracy

Verify Color Predictions

• Represent the color model's predictions in a graph
• Show how the model displays confidence for each prediction

Switch from TensorFlow to PyTorch

• Investigate how to transition from tf to PyTorch
• Recreate color classification experiment using PyTorch
• Create new notebook(s) for these experiments

OFA Integration

• Clone OFA repo
• Use a pre-built sub-net to analyze images
  ex) v100 to identify and elephant, note10 to do the same
• ignore losses, top1, top5; we only want to classify a single image
• run experiments in a new notebook