waliens / sldc Goto Github PK
View Code? Open in Web Editor NEWAn open-source Python framework created for accelerating development of large image analysis workflows
License: MIT License
An open-source Python framework created for accelerating development of large image analysis workflows
License: MIT License
In my simple application, I use SLDC to segment a grayscale image based on a threshold.
Since my image is very large (1.5GB), I have tested the application on a window of the image using the window
function.
At the end of the workflow, when I visualize the polygons on the image, they are located at the origin of the image and not at the position of the window.
Issue: shapely.geometry.shape returns empty polygons when called on a geo json object. Therefore all polygons returned by sldc are empty.
Temporary workaround: use shapely version 1.8.4
Dispatch classify step is parallelized if self._parallel_dc of SLDCWorkflow is False, and is not otherwise.
See line 195 of workflow.py
:
if self._parallel_dc:
timing.start_dc()
pred, proba, dispatch_indexes = self._dispatch_classifier.dispatch_classify_batch(image, polygons, timing)
timing.end_dc()
else:
pred, proba, dispatch_indexes = self._dc_parallel(image, polygons, timing)
Implement *_batch methods in base classes instead of their single param counterparts:
predict
, predict_batch
evaluate
, evaluate_batch
Is it a breaking change ?
Running my application with SLDC generates several deprecation warnings:
sldc/workflow.py:160: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray. tile_polygons = np.array([polygons for result in tiles_polygons for _, polygons in result])
sldc/merger.py:212: ShapelyDeprecationWarning: The 'cascaded_union()' function is deprecated. Use 'unary_union()' instead. polygon = cascaded_union(polygons).buffer(-dilation_dist, join_style=join)
sldc/merger.py:155: ShapelyDeprecationWarning: __len__ for multi-part geometries is deprecated and will be removed in Shapely 2.0. Check the length of the geoms property instead to get the number of parts of a multi-part geometry. return np.array(merged_polygons), np.array(merged_labels)
This test timeouts on UNIX when the whole test suite is launched with coverage run -m unittest discover . -v
. It seems to stale during the findContour call in the locator.locate method.
When run without the other tests of the suite, the testDetectCircleParallel seems to work.
Currently, with the rule based dispatching system, an object must sometimes be evaluated by several rules before being match. This is a problem when all the rules use the same evaluation procedure (such as a ML model) as it induces repeated computations
Logger should handle all newline types.
Failure example on windows:
[pid:007944][2017-01-12T16:24:14.691000][INFO ] SLDCWorkflow : start segment/locate.
[pid:007944][2017-01-12T16:24:17.856000][INFO ] SLDCWorkflow : end segment/locate.
SLDCWorkflow : 16 tile(s) processed in 3.16407038581 s.
SLDCWorkflow : 17 polygon(s) found on those tiles.
[pid:007944][2017-01-12T16:24:17.856000][INFO ] SLDCWorkflow : start merging
[pid:007944][2017-01-12T16:24:17.872000][INFO ] SLDCWorkflow : end merging.
SLDCWorkflow : 17 polygon(s) found.
SLDCWorkflow : executed in 0.0160387223857 s.
[pid:007944][2017-01-12T16:24:17.872000][INFO ] SLDCWorkflow : start dispatch/classify.
[pid:007944][2017-01-12T16:24:17.873000][INFO ] DispatcherClassifier: 17/17 polygons dispatched by rule 'catchall'.
[pid:007944][2017-01-12T16:24:17.873000][INFO ] DispatcherClassifier : end dispatching (17/17 polygons dispatched).
[pid:007944][2017-01-12T16:24:17.874000][INFO ] DispatcherClassifier : end classification.
[pid:007944][2017-01-12T16:24:17.874000][INFO ] SLDCWorkflow : end dispatch/classify.
SLDCWorkflow : executed in 0.00142972895989 s.
Leave choice to the user to implement loggable or not.
In util.py, the divisions yield floating point numbers in the batch_split function instead of integers:
smaller_batch_size = (item_count / n_batches)
bigger_batch_size = (item_count / n_batches) + 1
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.