jhennies / cebraem Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
cebraem's People
Contributors
Stargazers
Watchers
cebraem's Issues
CebraEM: Add a cluster setup json
The current system to set user MAIL and GROUP using environment variables is not intuitive.
A cluster setup json might be better.
However: where would this be sitting?
a) /home/$USER/.cebra_em/config_cluster.json
-> I like this one, because it could be used by all projects and is specific to the user even if multiple users are computing on the same project or using the same CebraEM repository
b) /$PROJECT_PATH/config/config_cluster.json
c) /$REPO_PATH/???/config_cluster.json
Reduce/Remove interactive elements
Many user interactions on the command line are in the way of the GUI approach from #48. We therefore decided to remove and reduce these.
This includes:
- Configuration prompting
- Q+Enter after snakemake runs
- Custom workflow locking
Rotate around mouse pointer when transposing axes
CTRL+E and CTRL+T (transposing axes) should rotate around mouse pointer if the mouse pointer is located over the respective labels layer.
This way navigation in the data would become much more convenient.
Missing install.bat for installation on Windows machines
PyPI Packaging
The project should be pip-installable and on PyPI
Clean-up / post-processing
Still missing the clean-up section:
- Connected components
- merge small objects
- apply single-type segmentations
Larger brush doesn't work in all dimensions
Computation of membrane prediction and supervoxels
The respective buttons are already there
Error when exiting field of extracted area during gt annotation in napari
As shown in the attached screenshot, there occurs an error at ground truth annotation in napari as soon as I exit the extracted area with the curser. (The error pops up in the panel on the lower right side and I pasted the traceback at the end of this issue)
the error is not crucial as it is possible to work around by trying to not exit the area, sometimes it is hard to merge supervoxels that are very close to the boundaries bit so far i managed. It would just make it a little easier if that was fixed.
I encountered that the issue seems to occur only for the left and the top boundary of the cube. If I exit on the right or the bottom, no error occurs.
IndexError Traceback (most recent call last)
File ~/miniconda3/envs/cebra-em-env02/lib/python3.9/site-packages/vispy/app/backends/_qt.py:532, in QtBaseCanvasBackend.mouseMoveEvent(self=<vispy.app.backends._qt.CanvasBackendDesktop object>, ev=<PyQt5.QtGui.QMouseEvent object>)
530 if self._vispy_canvas is None:
531 return
--> 532 self._vispy_mouse_move(
self = <vispy.app.backends._qt.CanvasBackendDesktop object at 0x7fef3ade78b0>
ev = <PyQt5.QtGui.QMouseEvent object at 0x7fef106a6c10>
533 native=ev,
534 pos=_get_event_xy(ev),
535 modifiers=self._modifiers(ev),
536 )
File ~/miniconda3/envs/cebra-em-env02/lib/python3.9/site-packages/vispy/app/base.py:216, in BaseCanvasBackend._vispy_mouse_move(self=<vispy.app.backends._qt.CanvasBackendDesktop object>, **kwargs={'button': 1, 'buttons': [], 'last_event': , 'last_mouse_press': None, 'modifiers': (<Key 'Control'>,), 'native': <PyQt5.QtGui.QMouseEvent object>, 'pos': (222, 578), 'press_event': })
213 else:
214 kwargs['button'] = self._vispy_mouse_data['press_event'].button
--> 216 ev = self._vispy_canvas.events.mouse_move(**kwargs)
self = <vispy.app.backends._qt.CanvasBackendDesktop object at 0x7fef3ade78b0>
self._vispy_canvas.events.mouse_move = <vispy.util.event.EventEmitter object at 0x7fef3ade8a90>
kwargs = {'native': <PyQt5.QtGui.QMouseEvent object at 0x7fef106a6c10>, 'pos': (222, 578), 'modifiers': (<Key 'Control'>,), 'buttons': [], 'press_event': <MouseEvent blocked=False button=1 buttons=[1] delta=[0. 0.] handled=True is_dragging=False last_event=MouseEvent modifiers=(<Key 'Control'>,) native=<PyQt5.QtGui.QMouseEvent object at 0x7fef106a6c10> pos=[406 613] press_event=None source=None sources=[] time=1695040930.2576623 type=mouse_press>, 'last_event': <MouseEvent blocked=False button=1 buttons=[1] delta=[0. 0.] handled=False is_dragging=True last_event=MouseEvent modifiers=(<Key 'Control'>,) native=<PyQt5.QtGui.QMouseEvent object at 0x7fef106a6c10> pos=[225 578] press_event=MouseEvent source=None sources=[] time=1695040930.6366537 type=mouse_move>, 'last_mouse_press': None, 'button': 1}
self._vispy_canvas.events = <vispy.util.event.EmitterGroup object at 0x7fef3ade89a0>
self._vispy_canvas = <VispyCanvas (PyQt5) at 0x7fef3ae26e20>
217 self._vispy_mouse_data['last_event'] = ev
218 return ev
File ~/miniconda3/envs/cebra-em-env02/lib/python3.9/site-packages/vispy/util/event.py:453, in EventEmitter.call(self=<vispy.util.event.EventEmitter object>, *args=(), **kwargs={'button': 1, 'buttons': [], 'last_event': , 'last_mouse_press': None, 'modifiers': (<Key 'Control'>,), 'native': <PyQt5.QtGui.QMouseEvent object>, 'pos': (222, 578), 'press_event': })
450 if self._emitting > 1:
451 raise RuntimeError('EventEmitter loop detected!')
--> 453 self._invoke_callback(cb, event)
event = <MouseEvent blocked=False button=1 buttons=[1] delta=[0. 0.] handled=False is_dragging=True last_event=MouseEvent modifiers=(<Key 'Control'>,) native=<PyQt5.QtGui.QMouseEvent object at 0x7fef106a6c10> pos=[222 578] press_event=MouseEvent source=None sources=[] time=1695040930.669733 type=mouse_move>
self = <vispy.util.event.EventEmitter object at 0x7fef3ade8a90>
cb = <bound method QtViewer.on_mouse_move of <napari._qt.qt_viewer.QtViewer object at 0x7fef3ae1a040>>
454 if event.blocked:
455 break
File ~/miniconda3/envs/cebra-em-env02/lib/python3.9/site-packages/vispy/util/event.py:471, in EventEmitter._invoke_callback(self=<vispy.util.event.EventEmitter object>, cb=<bound method QtViewer.on_mouse_move of <napari._qt.qt_viewer.QtViewer object>>, event=)
469 cb(event)
470 except Exception:
--> 471 _handle_exception(self.ignore_callback_errors,
self = <vispy.util.event.EventEmitter object at 0x7fef3ade8a90>
cb = <bound method QtViewer.on_mouse_move of <napari._qt.qt_viewer.QtViewer object at 0x7fef3ae1a040>>
event = <MouseEvent blocked=False button=1 buttons=[1] delta=[0. 0.] handled=False is_dragging=True last_event=MouseEvent modifiers=(<Key 'Control'>,) native=<PyQt5.QtGui.QMouseEvent object at 0x7fef106a6c10> pos=[222 578] press_event=MouseEvent source=None sources=[] time=1695040930.669733 type=mouse_move>
(cb, event) = (<bound method QtViewer.on_mouse_move of <napari._qt.qt_viewer.QtViewer object at 0x7fef3ae1a040>>, <MouseEvent blocked=False button=1 buttons=[1] delta=[0. 0.] handled=False is_dragging=True last_event=MouseEvent modifiers=(<Key 'Control'>,) native=<PyQt5.QtGui.QMouseEvent object at 0x7fef106a6c10> pos=[222 578] press_event=MouseEvent source=None sources=[] time=1695040930.669733 type=mouse_move>)
472 self.print_callback_errors,
473 self, cb_event=(cb, event))
File ~/miniconda3/envs/cebra-em-env02/lib/python3.9/site-packages/vispy/util/event.py:469, in EventEmitter._invoke_callback(self=<vispy.util.event.EventEmitter object>, cb=<bound method QtViewer.on_mouse_move of <napari._qt.qt_viewer.QtViewer object>>, event=)
467 def _invoke_callback(self, cb, event):
468 try:
--> 469 cb(event)
cb = <bound method QtViewer.on_mouse_move of <napari._qt.qt_viewer.QtViewer object at 0x7fef3ae1a040>>
event = <MouseEvent blocked=False button=1 buttons=[1] delta=[0. 0.] handled=False is_dragging=True last_event=MouseEvent modifiers=(<Key 'Control'>,) native=<PyQt5.QtGui.QMouseEvent object at 0x7fef106a6c10> pos=[222 578] press_event=MouseEvent source=None sources=[] time=1695040930.669733 type=mouse_move>
470 except Exception:
471 _handle_exception(self.ignore_callback_errors,
472 self.print_callback_errors,
473 self, cb_event=(cb, event))
File ~/miniconda3/envs/cebra-em-env02/lib/python3.9/site-packages/napari/_qt/qt_viewer.py:1198, in QtViewer.on_mouse_move(self=<napari._qt.qt_viewer.QtViewer object>, event=)
1190 def on_mouse_move(self, event):
1191 """Called whenever mouse moves over canvas.
1192
1193 Parameters
(...)
1196 The vispy event that triggered this method.
1197 """
-> 1198 self._process_mouse_event(mouse_move_callbacks, event)
event = <MouseEvent blocked=False button=1 buttons=[1] delta=[0. 0.] handled=False is_dragging=True last_event=MouseEvent modifiers=(<Key 'Control'>,) native=<PyQt5.QtGui.QMouseEvent object at 0x7fef106a6c10> pos=[222 578] press_event=MouseEvent source=None sources=[] time=1695040930.669733 type=mouse_move>
self = <napari._qt.qt_viewer.QtViewer object at 0x7fef3ae1a040>
File ~/miniconda3/envs/cebra-em-env02/lib/python3.9/site-packages/napari/_qt/qt_viewer.py:1147, in QtViewer._process_mouse_event(self=<napari._qt.qt_viewer.QtViewer object>, mouse_callbacks=, event=<ReadOnlyWrapper at 0x7feeedf042c0 for MouseEvent>)
1145 layer = self.viewer.layers.selection.active
1146 if layer is not None:
-> 1147 mouse_callbacks(layer, event)
event = <ReadOnlyWrapper at 0x7feeedf042c0 for MouseEvent at 0x7feeedfa0850>
layer = <Labels layer 'instances' at 0x7fef20edce50>
mouse_callbacks = <function mouse_move_callbacks at 0x7ff09684e1f0>
File ~/miniconda3/envs/cebra-em-env02/lib/python3.9/site-packages/napari/utils/interactions.py:173, in mouse_move_callbacks(obj=<Labels layer 'instances'>, event=<ReadOnlyWrapper at 0x7feeedf042c0 for MouseEvent>)
170 obj._persisted_mouse_event[gen].wrapped = event
171 try:
172 # try to advance the generator
--> 173 next(gen)
gen = <generator object CebraAnnWidget._cb_instance_segmentation..lyr_instances_onmousedrag at 0x7feee5e73f20>
174 except StopIteration:
175 # If done deleted the generator and stored event
176 del obj._mouse_drag_gen[func]
File ~/repos/CebraEM/cebra-ann/cebra_ann/_widget.py:927, in CebraAnnWidget._cb_instance_segmentation..lyr_instances_onmousedrag(layer=<Labels layer 'instances'>, event=<ReadOnlyWrapper at 0x7feee5e3d540 for ReadOnlyWrapper>)
925 this_labels = []
926 else:
--> 927 positions = get_disk_positions(
positions = <class 'numpy.ndarray'> (16, 3) int64
brush_size = 5
event = <ReadOnlyWrapper at 0x7feee5e3d540 for ReadOnlyWrapper at 0x7feeedff6500>
self = <cebra_ann._widget.CebraAnnWidget object at 0x7fef2131ee50>
self.viewer = Viewer(camera=Camera(center=(0.0, 166.29326638806666, 194.6071992360609), zoom=2.542248348945979, angles=(0.0, 0.0, 90.0), perspective=0.0, mouse_pan=True, mouse_zoom=True), cursor=Cursor(position=(140.0, 297.27969395370627, 235.12252599230868), scaled=True, size=5, style=<CursorStyle.CIRCLE: 'circle'>), dims=Dims(ndim=3, ndisplay=2, last_used=0, range=((0.0, 384.0, 1.0), (0.0, 384.0, 1.0), (0.0, 384.0, 1.0)), current_step=(140, 191, 191), order=(0, 1, 2), axis_labels=('0', '1', '2')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[<Image layer 'raw' at 0x7feeb1a0ad90>, <Image layer 'mem' at 0x7feeb1a0ad00>, <Labels layer 'sv' at 0x7fef106ba940>, <Labels layer 'pre_merge' at 0x7fef212b4a60>, <Labels layer 'instances' at 0x7fef20edce50>, <Labels layer 'semantics_nuc' at 0x7fef0ff58760>, <Labels layer 'semantics_cyto' at 0x7fef102f7b80>, <Labels layer 'semantics_bg' at 0x7fee0c21d040>], help='use <1> for activate the label eraser, use <2> for activate the paint brush, use <3> for activate the fill bucket, use <4> for pick mode', status={'layer_base': 'instances', 'source_type': '', 'plugin': '', 'coordinates': ' [140 297 235]'}, tooltip=Tooltip(visible=False, text=''), theme='dark', title='CebraANN - /home/rheinnec/scratch/cebra_em_wfs/fo/gt/gt004', mouse_over_canvas=True, mouse_move_callbacks=[], mouse_drag_callbacks=[], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[<function dims_scroll at 0x7ff06c542ee0>], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={'Control': <function CebraAnnWidget._add_key_bindings..ctrl_pressed at 0x7fef2132f280>, 'Control-E': <function CebraAnnWidget._add_key_bindings..ctrl_e_pressed at 0x7fef2132f310>})
np = <module 'numpy' from '/home/rheinnec/miniconda3/envs/cebra-em-env02/lib/python3.9/site-packages/numpy/init.py'>
layer = <Labels layer 'instances' at 0x7fef20edce50>
928 brush_size,
929 tuple((np.array(event.position) - self.viewer.layers['instances'].translate).astype(int)),
930 layer.data.shape,
931 self.viewer.dims.order
932 )
933 this_labels = np.unique([layer.data[tuple(pos)] for pos in positions])
935 for this_label in this_labels:
File ~/repos/CebraEM/cebra-ann/cebra_ann/funcs.py:38, in get_disk_positions(brush_size=5, center=(12, 83, 2), shape=<class 'numpy.ndarray'> (3,) int64, dims_order=(0, 1, 2))
31 z, y, x = np.mgrid[
32 np.s[int(center[0] - brush_size / 2):int(center[0] + brush_size / 2 + 1)] if z_axis != 0 else np.s_[center[0]: center[0] + 1],
33 np.s_[int(center[1] - brush_size / 2):int(center[1] + brush_size / 2 + 1)] if z_axis != 1 else np.s_[center[1]: center[1] + 1],
34 np.s_[int(center[2] - brush_size / 2):int(center[2] + brush_size / 2 + 1)] if z_axis != 2 else np.s_[center[2]: center[2] + 1]
35 ]
37 # Mask with the structuring element to get the list of positions
---> 38 pos = np.array([z[strel > 0], y[strel > 0], x[strel > 0]]).swapaxes(0, 1)
strel = <class 'numpy.ndarray'> (1, 6, 6) uint8
np = <module 'numpy' from '/home/rheinnec/miniconda3/envs/cebra-em-env02/lib/python3.9/site-packages/numpy/init.py'>
z = <class 'numpy.ndarray'> (1, 6, 5) int64
y = <class 'numpy.ndarray'> (1, 6, 5) int64
x = <class 'numpy.ndarray'> (1, 6, 5) int64
40 # Remove positions that are out of upper bounds
41 pos = pos[(pos < shape).all(axis=1), :]
IndexError: boolean index did not match indexed array along dimension 2; dimension is 5 but corresponding boolean dimension is 6
Project loading when a different project is already loaded doesn't remove current layers
All napari layers need to be removed before a new project can be loaded
Switch command line interfaces to click
This is the first part of the SSC contributions to the engineering the CebraEM package.
The idea is to move away from plain argparse towards click. Using click will allow us to create GUIs with zero overhead through the following project once it is mature: https://github.com/ssciwr/clickqt
installation for linux fails on pc without cuda
while installing on linux using the install.sh on pc without cuda there's an issue. This is printed as error:
Executing installation script
Detecting CUDA version
Traceback (most recent call last):
File "/Bianca/conda/conda_envs/cebra-em-env/bin/install_torch.py", line 7, in
exec(compile(f.read(), file, 'exec'))
File "/src/CebraEM/cebra-em-core/bin/install_torch.py", line 17, in
custom_pytorch()
File "/src/CebraEM/cebra-em-core/cebra_em_core/linux/cuda.py", line 79, in custom_pytorch
CUDA_NUMS = CUDA_VERSION.strip().split('.')
AttributeError: 'NoneType' object has no attribute 'strip'
Quick fix:
moving lines 79 and 80 in cuda.py inside the else statement just below.
Apply pre-commit with formatting/linting to the project
Python error in quantile_norm of module pulp
when executing:
cem-run supervoxels --cluster slurm -c 64 -g 12
the follwouing error occurs:
Traceback (most recent call last):
File "/home/rheinnec/miniconda3/envs/cebra1/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/rheinnec/miniconda3/envs/cebra1/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/rheinnec/miniconda3/envs/cebra1/lib/python3.9/site-packages/snakemake/main.py", line 4, in
main()
File "/home/rheinnec/miniconda3/envs/cebra1/lib/python3.9/site-packages/snakemake/init.py", line 2736, in main
parser = get_argument_parser()
File "/home/rheinnec/miniconda3/envs/cebra1/lib/python3.9/site-packages/snakemake/init.py", line 1607, in get_argument_parser
lp_solvers = pulp.list_solvers(onlyAvailable=True)
AttributeError: module 'pulp' has no attribute 'list_solvers'
according to: snakemake/snakemake#2607
the list_solvers was updated to listSolvers
so i changed the line 1607 of file:
"/home/rheinnec/miniconda3/envs/cebra1/lib/python3.9/site-packages/snakemake/init.py"
from:
lp_solvers = pulp.list_solvers(onlyAvailable=True)
into:
lp_solvers = pulp.listSolvers(onlyAvailable=True)
this seems to fix the problem
CebraEM: Table in MoBIE project misses anchors
When viewing a mask in MoBIE that has multiple objects, it would be nice to be able to navigate to each object using the table entries.
Currently all objects are anchored to (0, 0, 0).
This would require parsing all objects at project initialization and determining their center point within the dataset (could be done on a lower resolution level)
Python KeyError at final segmentation: Manually initialized 'segmentations' in config file
I ran into the following issue while running: run.py -t nuc_it00 (running nucleus segmentation after annotation of gt_cube)
The prompted Error is:
Traceback (most recent call last):
File "/home/rheinnec/miniconda3/envs/cebra-em-env/bin/run.py", line 7, in
exec(compile(f.read(), file, 'exec'))
File "/home/rheinnec/repos/CebraEM/cebra-em/bin/run.py", line 325, in
run(
File "/home/rheinnec/repos/CebraEM/cebra-em/bin/run.py", line 211, in run
if bm not in config_seg['segmentations'].keys():
KeyError: 'segmentations'
I saw that in config/config_nuc_it00.json there is no 'segmentations' entry, which I think the script is looking for.
I think actually this is only needed to check if it is already there and otherwise it will be created. So I included a dummy-segmentation entry in the config file (or just copied and renamed another dictonary entry):
"segmentations": {
"cpuxxxs": 1
}
After that the segmentation started and is currently running.
I am not sure if it is again on my side that there is no entry, but also creating another iteration for nuc (nuc_it01) with init_segmentation.py and linking the ground thruths with link_gt.py did not add an 'segmentations' entry to the config. Maybe i am missing another function call?
Best wishes!
Marco
CebraEM: Quantile normalization terribly inefficient when mask has objects with large bounding boxes
It might be quicker to just retrieve n = 1024(?) pixels randomly within each mask object.
The question being: does that yield the same normalization result?
Using down-sampled layers of the EM dataset would not yield the same result as down-sampling is usually performed with mean interpolation such that the pixel values will have a different grayscale distribution.
Supervoxel, instances and semantics layers should not be editable
Set these layers to editable = False by default.
For some reason using
viewer.layers[layer_name].editable = False
only makes the layer un-editable until one moves slices in the viewer. Then it's back to layer.editable = True
CebraANN: Clean-up broken
Error message when using connected components
Missing 64x64 image cubes after stiching of segmentation
After finishing a segmentation I run the call:
run.py -t stitch-cyto_it05 --param beta=0.5 --cluster slurm -c 128 -g 12
and the stiching is perfromed.
The stiched image contains areas without voxel where they actually should be according to the unstiched image.
These areas seem to occur randomly distributed over the masked areas and have a size of 64x64 pixel in xy
For some reason they only occur in the s0 image. The downsampled images s1, s2 etc appear as expected.
Error in rule quantile norm, KeyError: 'quantile_samples' at final segmentation
Hi!
I am trying to rerun a segmentation for which the ROI flag was initially not properly set (unit was not defined). For the first run I got no error messages, but upon retrying I get an "error in rule quantile" error message (independent of the ROI flag). It looks like the program is looking for a "quantile_samples" in the config_raw file, which is not there? I tried to manually initiate an empty entry, but it looks like it needs a number, in which case I am not sure what to put.. Is this an issue with rerunning the segmentation or something else on my side?
Best wishes,
Marianne
Bioconda packaging
Invalid cpu configuration on slurm cluster
When executing:
cem-run supervoxels --cluster slurm -c 64 -g 12
the first job: run_quantile_norm cannot be submitted to the cluster with the error message:
"job submission failed: Requested node configuration is not available" - or something vcery similar
I fixed it by changing the following file:
/cebra-em/cebra_em/snakefiles/run_main.smk
at line 153 from
resources:
cpus=os.cpus(), time_min=60, mem_mb=32000
into
resources:
cpus=64, time_min=60, mem_mb=33000
quantile_norm snakejob requires more than 32G memory
I just ran into "out_of_memory" issues at the first job submitted by run.py (quantile_norm)
I could not find a way to increase the memory limit in the config files. (maybe I am just too dumb to find it)
I came over it by changing it in the repository file:
CebraEM/cebra-em/cebra_em/snakefiles/run_main.smk
line 154, position 50
64000 didnt work out either,
500000 worked
If that does not make sense in any way, maybe something is wrong with my data.
I will follow up on this as soon as the follow up steps work out or not.
Best wishes to you!
Marco
include scaling function for post-processing
I discovered the following problem at the data post-processing of segmentations:
I will give a very precise example but i think the feature could be of advantage for other things as well.
To achieve segmentations of cell boundaries it made sense (at least for some cells) to do a segmentation of the background and use the negative to get the cell area.
As the background is segmented only in the mask area, the segmentation gives only the background area inside the mask. That means that on the full image I get the segmented area (>0) as well as the area of the cell (will be zero) but also the background area outside the mask will be zero.
To get over this, an overlay of the mask and the segmented background area can be used zo extract only the zeros of the cell.
The problem here is:
For datasets with non-isotropic input, cebra-em can perform a scaling and interpolates the "missing" slides for the input image and the mask. But these interpolations are not stored somewhere due to their size.
That brings up the problem when I want to overlay the mask and segmentations, the z-resolution differs and not every layer of the segmenattion is covered by the mask. As I dont want to lose the interpolations in the segmenattion, I need to scale the mask manually.
Of course i could implement this by myself, but the functionality is somewhere inside of the cebraEM functions and it would probably make sense to have it as a function exported to the namespace?
For manual corrections afterwards this would maybe also make sense for the raw EM image, which would require the same functionality.
Improve project handling
- Project close button
- Close/remove buttons for raw, mem and sv
- Lock computation of mem and sv if they were loaded from a path on disk
Non-unique supervoxels
Although input supervoxels should have unique IDs in the first place, it would be good to ensure that by connected component analysis as it breaks the workflow if two SVs have the same label
Stitching works only for one beta value per segmentation
I am trying to stitch two datasets of the same segmentation but with different beta values.
To stich, I am using:
run.py -t stitch-cyto_it05 --param beta=0.5 --cluster slurm -c 128 -g 12
This works as it should
If I now want to go also for beta 0.6 with the following call:
run.py -t stitch-cyto_it05 --param beta=0.6 --cluster slurm -c 128 -g 12
Snakemake does not start any jobs and gives me the following output:
CebraEM: Stitching required
Implement stitching step
Merge the three Python packages into one
We achieve the same goal of not necessarily requiring CUDA runtime installations by doing a similar approach to many existing packages e.g. https://spacy.io/usage : The CUDA runtime is installed separatedly and detected at runtime, allowing CPU-only installations.
Mobie project validation fails
With a new installation init_project ... -m ... runs into a json validation error (see below).
The json for the mask is validated against https://raw.githubusercontent.com/mobie/mobie.github.io/master/schema/view.schema.json which is apparently not compatible with the mobie version supported here.
The best solution would be to update the mobie version.
Traceback (most recent call last):
File "/home/hennies/miniconda3/envs/cebra-em-all-env/lib/python3.9/site-packages/mobie/validation/metadata.py", line 196, in validate_view_metadata
validate_with_schema(view, "view")
File "/home/hennies/miniconda3/envs/cebra-em-all-env/lib/python3.9/site-packages/mobie/validation/utils.py", line 56, in validate_with_schema
jsonschema.validate(instance=metadata, schema=schema)
File "/home/hennies/miniconda3/envs/cebra-em-all-env/lib/python3.9/site-packages/jsonschema/validators.py", line 1121, in validate
raise error
jsonschema.exceptions.ValidationError: Additional properties are not allowed ('tables' was unexpected)
Failed validating 'additionalProperties' in schema[1]['properties']['segmentationDisplay']:
{'additionalProperties': False,
'description': '',
'properties': {'additionalTables': {'$ref': '#/definitions/additionalTables'},
'boundaryThickness': {'description': 'Thickness of the '
'boundary masks. '
'Only used if '
'showAsBoundaries '
'is true.',
'type': 'number'},
'colorByColumn': {'$ref': '#/definitions/name',
'description': 'Name of table column '
'that is used for '
'coloring. By default '
"the 'label_id' "
'column is used.'},
'lut': {'$ref': '#/definitions/lut'},
'name': {'$ref': '#/definitions/name',
'description': 'Name of this segmentation '
'display.'},
'opacity': {'$ref': '#/definitions/opacity'},
'opacityNotSelected': {'$ref': '#/definitions/opacityNotSelected'},
'randomColorSeed': {'description': 'Random seed for '
'the random color '
'lut (e.g. glasbey) '
'to reproduce the '
'exact colors of '
'the view. '
'(Optional)',
'type': 'number'},
'resolution3dView': {'description': 'Resolution used '
'for the 3d '
'viewer, in '
'physical units. '
'Only relevant if '
"'showSelectedSegmentsIn3d' "
'is true. Will be '
'determined '
'automatically if '
'not specified.',
'items': {'exclusiveMinimum': 0,
'type': 'number'},
'maxItems': 3,
'minItems': 3,
'type': 'array'},
'scatterPlotAxes': {'$ref': '#/definitions/scatterPlotAxes'},
'selectedSegmentIds': {'description': 'List of '
'selected '
'segment ids, '
'each of the '
'form '
'sourceName;timePoint;label_id',
'items': {'pattern': '^[^; '
'\\/]+;\\d+;\\d+$',
'type': 'string'},
'type': 'array'},
'selectionColor': {'$ref': '#/definitions/selectionColor'},
'showAsBoundaries': {'description': 'Show boundary '
'mask instead of '
'segment masks. '
'Default is false.',
'type': 'boolean'},
'showScatterPlot': {'description': 'Whether to show '
'the scatter plot. '
'The default is '
"'false', i.e. if "
'this property is '
'not present the '
'scatter plot '
'should not be '
'shown. If it is '
"'true' then "
"'scatterPlotAxes' "
'must be given.',
'type': 'boolean'},
'showSelectedSegmentsIn3d': {'description': 'Whether '
'to show '
'the '
'selected '
'segments '
'in the 3d '
'viewer.',
'type': 'boolean'},
'showTable': {'description': 'Show the table GUI '
'element. Default is true '
'(if the display has a '
'table).',
'type': 'boolean'},
'sources': {'description': 'The segmentation sources '
'that are part of this '
'display group. Multiple '
'sources should be moved '
'apart spatially with '
'source transform(s), e.g. '
'grid, otherwise they will '
'not be correctly displayed '
'in the viewer.',
'items': {'$ref': '#/definitions/name'},
'minItems': 1,
'type': 'array'},
'valueLimits': {'$ref': '#/definitions/valueLimits'},
'visible': {'description': 'Are the sources of this '
'display visible? Default '
'is true.',
'type': 'boolean'}},
'required': ['opacity', 'lut', 'name', 'sources'],
'type': 'object'}
On instance['segmentationDisplay']:
{'lut': 'glasbey',
'name': 'em-mask',
'opacity': 0.5,
'sources': ['em-mask'],
'tables': ['default.tsv']}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/hennies/miniconda3/envs/cebra-em-all-env/bin/init_project.py", line 7, in <module>
exec(compile(f.read(), __file__, 'exec'))
File "/g/schwab/hennies/src/github/jhennies/CebraEM/cebra-em/bin/init_project.py", line 65, in <module>
init_project(
File "/g/schwab/hennies/src/github/jhennies/CebraEM/cebra-em-core/cebra_em_core/cebra_em_project.py", line 250, in init_project
init_mobie_dataset(project_path=project_path, verbose=verbose)
File "/g/schwab/hennies/src/github/jhennies/CebraEM/cebra-em-core/cebra_em_core/cebra_em_project.py", line 139, in init_mobie_dataset
init_mask(
File "/g/schwab/hennies/src/github/jhennies/CebraEM/cebra-em-core/cebra_em_core/dataset/mobie_utils.py", line 411, in init_mask
add_source_to_dataset(
File "/home/hennies/miniconda3/envs/cebra-em-all-env/lib/python3.9/site-packages/mobie/metadata/source_metadata.py", line 251, in add_source_to_dataset
validate_view_metadata(view)
File "/home/hennies/miniconda3/envs/cebra-em-all-env/lib/python3.9/site-packages/mobie/validation/metadata.py", line 199, in validate_view_metadata
assert_true(False, msg)
File "/home/hennies/miniconda3/envs/cebra-em-all-env/lib/python3.9/site-packages/mobie/validation/utils.py", line 77, in _assert_true
raise ValueError(msg)
ValueError: Additional properties are not allowed ('tables' was unexpected)
Failed validating 'additionalProperties' in schema[1]['properties']['segmentationDisplay']:
{'additionalProperties': False,
'description': '',
'properties': {'additionalTables': {'$ref': '#/definitions/additionalTables'},
'boundaryThickness': {'description': 'Thickness of the '
'boundary masks. '
'Only used if '
'showAsBoundaries '
'is true.',
'type': 'number'},
'colorByColumn': {'$ref': '#/definitions/name',
'description': 'Name of table column '
'that is used for '
'coloring. By default '
"the 'label_id' "
'column is used.'},
'lut': {'$ref': '#/definitions/lut'},
'name': {'$ref': '#/definitions/name',
'description': 'Name of this segmentation '
'display.'},
'opacity': {'$ref': '#/definitions/opacity'},
'opacityNotSelected': {'$ref': '#/definitions/opacityNotSelected'},
'randomColorSeed': {'description': 'Random seed for '
'the random color '
'lut (e.g. glasbey) '
'to reproduce the '
'exact colors of '
'the view. '
'(Optional)',
'type': 'number'},
'resolution3dView': {'description': 'Resolution used '
'for the 3d '
'viewer, in '
'physical units. '
'Only relevant if '
"'showSelectedSegmentsIn3d' "
'is true. Will be '
'determined '
'automatically if '
'not specified.',
'items': {'exclusiveMinimum': 0,
'type': 'number'},
'maxItems': 3,
'minItems': 3,
'type': 'array'},
'scatterPlotAxes': {'$ref': '#/definitions/scatterPlotAxes'},
'selectedSegmentIds': {'description': 'List of '
'selected '
'segment ids, '
'each of the '
'form '
'sourceName;timePoint;label_id',
'items': {'pattern': '^[^; '
'\\/]+;\\d+;\\d+$',
'type': 'string'},
'type': 'array'},
'selectionColor': {'$ref': '#/definitions/selectionColor'},
'showAsBoundaries': {'description': 'Show boundary '
'mask instead of '
'segment masks. '
'Default is false.',
'type': 'boolean'},
'showScatterPlot': {'description': 'Whether to show '
'the scatter plot. '
'The default is '
"'false', i.e. if "
'this property is '
'not present the '
'scatter plot '
'should not be '
'shown. If it is '
"'true' then "
"'scatterPlotAxes' "
'must be given.',
'type': 'boolean'},
'showSelectedSegmentsIn3d': {'description': 'Whether '
'to show '
'the '
'selected '
'segments '
'in the 3d '
'viewer.',
'type': 'boolean'},
'showTable': {'description': 'Show the table GUI '
'element. Default is true '
'(if the display has a '
'table).',
'type': 'boolean'},
'sources': {'description': 'The segmentation sources '
'that are part of this '
'display group. Multiple '
'sources should be moved '
'apart spatially with '
'source transform(s), e.g. '
'grid, otherwise they will '
'not be correctly displayed '
'in the viewer.',
'items': {'$ref': '#/definitions/name'},
'minItems': 1,
'type': 'array'},
'valueLimits': {'$ref': '#/definitions/valueLimits'},
'visible': {'description': 'Are the sources of this '
'display visible? Default '
'is true.',
'type': 'boolean'}},
'required': ['opacity', 'lut', 'name', 'sources'],
'type': 'object'}
On instance['segmentationDisplay']:
{'lut': 'glasbey',
'name': 'em-mask',
'opacity': 0.5,
'sources': ['em-mask'],
'tables': ['default.tsv']}
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.