waliens/sldc

An open-source Python framework created for accelerating development of large image analysis workflows

sldc's Issues

The `window` method from the `Image` class do not take the offset into account

In my simple application, I use SLDC to segment a grayscale image based on a threshold.
Since my image is very large (1.5GB), I have tested the application on a window of the image using the window function.
At the end of the workflow, when I visualize the polygons on the image, they are located at the origin of the image and not at the position of the window.

Issue with latest shapely version 1.8.5.post1

Issue: shapely.geometry.shape returns empty polygons when called on a geo json object. Therefore all polygons returned by sldc are empty.

Temporary workaround: use shapely version 1.8.4

[IMPROV] Test case when objects are not dispatched

[BUG] Parallel dc not used properly

Dispatch classify step is parallelized if self._parallel_dc of SLDCWorkflow is False, and is not otherwise.

See line 195 of workflow.py:

        if self._parallel_dc:
            timing.start_dc()
            pred, proba, dispatch_indexes = self._dispatch_classifier.dispatch_classify_batch(image, polygons, timing)
            timing.end_dc()
        else:
            pred, proba, dispatch_indexes = self._dc_parallel(image, polygons, timing)

[Convenience] Implement *_batch methods

Implement *_batch methods in base classes instead of their single param counterparts:

• Polygon classifier: predict, predict_batch
• Dispatching rule: evaluate, evaluate_batch

Is it a breaking change ?

Deprecation Warning

Running my application with SLDC generates several deprecation warnings:
sldc/workflow.py:160: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray. tile_polygons = np.array([polygons for result in tiles_polygons for _, polygons in result])

sldc/merger.py:212: ShapelyDeprecationWarning: The 'cascaded_union()' function is deprecated. Use 'unary_union()' instead. polygon = cascaded_union(polygons).buffer(-dilation_dist, join_style=join)

sldc/merger.py:155: ShapelyDeprecationWarning: __len__ for multi-part geometries is deprecated and will be removed in Shapely 2.0. Check the length of the geoms property instead to get the number of parts of a multi-part geometry. return np.array(merged_polygons), np.array(merged_labels)

Test `test_sldc.FullWorkflow.testDetectCircleParallel` timeout

This test timeouts on UNIX when the whole test suite is launched with coverage run -m unittest discover . -v . It seems to stale during the findContour call in the locator.locate method.
When run without the other tests of the suite, the testDetectCircleParallel seems to work.

Add support for one shot dispatcher

Currently, with the rule based dispatching system, an object must sometimes be evaluated by several rules before being match. This is a problem when all the rules use the same evaluation procedure (such as a ML model) as it induces repeated computations

[WINDOWS] Logger outputs sometimes don't write the prefix

Logger should handle all newline types.

Failure example on windows:

[pid:007944][2017-01-12T16:24:14.691000][INFO ] SLDCWorkflow : start segment/locate.
[pid:007944][2017-01-12T16:24:17.856000][INFO ] SLDCWorkflow : end segment/locate.
SLDCWorkflow : 16 tile(s) processed in 3.16407038581 s.
SLDCWorkflow : 17 polygon(s) found on those tiles.
[pid:007944][2017-01-12T16:24:17.856000][INFO ] SLDCWorkflow : start merging
[pid:007944][2017-01-12T16:24:17.872000][INFO ] SLDCWorkflow : end merging.
SLDCWorkflow : 17 polygon(s) found.
SLDCWorkflow : executed in 0.0160387223857 s.
[pid:007944][2017-01-12T16:24:17.872000][INFO ] SLDCWorkflow : start dispatch/classify.
[pid:007944][2017-01-12T16:24:17.873000][INFO ] DispatcherClassifier: 17/17 polygons dispatched by rule 'catchall'.
[pid:007944][2017-01-12T16:24:17.873000][INFO ] DispatcherClassifier : end dispatching (17/17 polygons dispatched).
[pid:007944][2017-01-12T16:24:17.874000][INFO ] DispatcherClassifier : end classification.
[pid:007944][2017-01-12T16:24:17.874000][INFO ] SLDCWorkflow : end dispatch/classify.
SLDCWorkflow : executed in 0.00142972895989 s.

Create a binding with bioformat

https://pythonhosted.org/python-bioformats/

[Convenience] Remove Loggable from DispatchingRule superclasses

Leave choice to the user to implement loggable or not.

[Py3 support] Floating point division in batch_split

In util.py, the divisions yield floating point numbers in the batch_split function instead of integers:

    smaller_batch_size = (item_count / n_batches)
    bigger_batch_size = (item_count / n_batches) + 1