bf777/mesonet

A python toolbox for locating and exporting brain regions from mouse brain images.

mesonet's Issues

Google Colab instructions not working: `ModuleNotFoundError: No module named 'mesonet'`

Hello,

I have been trying to get MesoNet up and running with the demo Google Colab notebook you provide: https://colab.research.google.com/github/bf777/MesoNet/blob/master/mesonet_demo_colab.ipynb

I am able to reach this cell:

MesoNet/mesonet_demo_colab.ipynb

Lines 378 to 438 in f574f12

	{
	"cell_type": "code",
	"metadata": {
	"id": "7tbS1rlK3Osl"
	},
	"source": [
	"# We need to make sure that DeepLabCut doesn't run with a GUI (which isn't\n",
	"# supported in Colab).\n",
	"os.environ[\"DLClight\"]=\"True\"\n",
	"\n",
	"# Import mesonet and define the configuration file for each pipeline\n",
	"import mesonet\n",
	"## 1. Atlas to brain\n",
	"# Atlas-to-brain warp with U-Net and DeepLabCut\n",
	"print('1. Atlas-to-brain warp with U-Net and DeepLabCut')\n",
	"config_file_atlas_brain = mesonet.config_project(input_file, output_file_atlas_brain, 'test', \n",
	" atlas_to_brain_align=True, use_voxelmorph=False, \n",
	" use_unet=True, use_dlc=True, sensory_match=False, \n",
	" mat_save=False, olfactory_check=True,\n",
	" config=dlc_config, model=model)\n",
	"\n",
	"## 2. Brain to atlas\n",
	"# Brain-to-atlas warp with DeepLabCut\n",
	"print('2. Brain-to-atlas warp with DeepLabCut')\n",
	"config_file_brain_atlas = mesonet.config_project(input_file, output_file_brain_atlas, 'test', \n",
	" atlas_to_brain_align=False, use_voxelmorph=False, \n",
	" use_unet=True, use_dlc=True, sensory_match=False, \n",
	" mat_save=False, olfactory_check=True, \n",
	" config=dlc_config, model=model)\n",
	"\n",
	"## 3. Atlas to brain + sensory\n",
	"# Atlas-to-brain warp with U-Net, DeepLabCut, and sensory maps\n",
	"print('3. Atlas-to-brain warp with U-Net, DeepLabCut, and sensory maps')\n",
	"config_file_sensory = mesonet.config_project(input_file_sensory_raw, output_file_sensory, 'test',\n",
	" atlas_to_brain_align=True, use_voxelmorph=False, \n",
	" use_unet=True, use_dlc=True, sensory_match=True, \n",
	" sensory_path=input_file_sensory_maps, mat_save=False, \n",
	" config=dlc_config, model=model)\n",
	"\n",
	"## 4. MBFM + U-Net\n",
	"# Motif-based functional maps (MBFMs) with atlas directly applied using U-Net\n",
	"print('4. Motif-based functional maps (MBFMs) with atlas directly applied using U-Net')\n",
	"config_file_MBFM_U_Net = mesonet.config_project(input_file_MBFM, output_file_MBFM_U_Net, 'test', \n",
	" atlas_to_brain_align=True, use_voxelmorph=False, \n",
	" use_unet=True, use_dlc=False, sensory_match=False, \n",
	" mat_save=False, mask_generate=False, \n",
	" config=dlc_config, model=u_net_only_model)\n",
	"\n",
	"## 5. VoxelMorph\n",
	"# Local deformation warp with VoxelMorph and DeepLabCut\n",
	"print('5. Local deformation warp with VoxelMorph and DeepLabCut')\n",
	"config_file_voxelmorph = mesonet.config_project(input_file_voxelmorph, output_file_voxelmorph, 'test', \n",
	" atlas_to_brain_align=False, use_voxelmorph=True, \n",
	" use_unet=True, use_dlc=True, sensory_match=False, mat_save=False, \n",
	" config=dlc_config, model=model, \n",
	" align_once=True, olfactory_check=True, \n",
	" voxelmorph_model=voxelmorph_model)"
	],
	"execution_count": null,
	"outputs": []
	},

, but am not able to execute it as I get the following errors:

ModuleNotFoundError                       Traceback (most recent call last)
[<ipython-input-69-cb137b84186f>](https://localhost:8080/#) in <cell line: 10>()
      8 # sys.path.append('/content/MesoNet')
      9 # sys.path.append('/content/MesoNet/mesonet')
---> 10 import mesonet

1 frames
[/content/MesoNet/mesonet/utils.py](https://localhost:8080/#) in <module>
     14 import cv2
     15 import numpy as np
---> 16 import neurite as ne
     17 import matplotlib.pyplot as plt
     18 import pathlib

ModuleNotFoundError: No module named 'neurite'

I noticed that the instructions say here and here that if mesonet cannot be imported, that one should re-run the following cells:

MesoNet/mesonet_demo_colab.ipynb

Lines 97 to 143 in f574f12

	{
	"cell_type": "code",
	"metadata": {
	"id": "tgL7qiElE8O8"
	},
	"source": [
	"# NOTE: Rerun this cell and the following two cells if you're getting an error when importing MesoNet\n",
	"%cd MesoNet/"
	],
	"execution_count": null,
	"outputs": []
	},
	{
	"cell_type": "code",
	"metadata": {
	"id": "Xcmr-pE1yR75"
	},
	"source": [
	"# Install MesoNet\n",
	"!python setup.py install\n",
	"\n",
	"# Reinstall OpenCV to address compatibility issue\n",
	"!pip install opencv-python==4.4.0.46\n",
	"\n",
	"# Reinstall h5py\n",
	"!pip install h5py==2.10.0\n",
	"\n",
	"# pip install mesonet"
	],
	"execution_count": null,
	"outputs": []
	},
	{
	"cell_type": "code",
	"metadata": {
	"id": "UlkSR2Y1DMt-"
	},
	"source": [
	"# Use tensorflow 1.x (supported by DeepLabCut)\n",
	"# %tensorflow_version 1.x\n",
	"import tensorflow\n",
	"import os\n",
	"print(tensorflow.__version__)"
	],
	"execution_count": null,
	"outputs": []
	},

I have done this, but am still unable to import mesonet. Would you be able to provide any insight into why this is happening and what I can do?

Thank you so much. Please let me know if I can provide any further information.

Missing config file

Hi,
I've tried to use the GUI on my conda deeplabcut environment. It launches correctly, I can select my input and output folders, but as soon as I try using the "Atlas to Brain" option, I get the following error message, even though I did change the paths on the config.yaml file to match my computer's. Do you know where this comes from, should I modify an existing file, or is something missing from my environment ?
Thanks in advance

KeyError: 'TrainingFraction'

Hi,

I'm able to get the GUI running, but when I click any of the quick start automated pipelines I end up with with the error: KeyError: 'TrainingFraction'. How can I fix this? Thank you!

Here is the full error:
/Users/name/MesoNet/mesonet/models/DongshengXiao_brain_bundary.hdf5
/Users/name/Test_Images
1
/Users/name/Test_Images/0.png
OpenCV: FFMPEG: tag 0x4745504d/'MPEG' is not supported with codec id 2 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
DLC config file path: /Users/name/MesoNet/mesonet/dlc/config.yaml
Exception in Tkinter callback
Traceback (most recent call last):
File "/Users/name/opt/anaconda3/envs/DEEPLABCUT/lib/python3.9/tkinter/init.py", line 1892, in call
return self.func(*args)
File "/Users/name/opt/anaconda3/envs/DEEPLABCUT/lib/python3.9/site-packages/mesonet/gui_test.py", line 578, in
command=lambda: self.EnterThread("atlas_to_brain"),
File "/Users/name/opt/anaconda3/envs/DEEPLABCUT/lib/python3.9/site-packages/mesonet/gui_test.py", line 1100, in EnterThread
target=self.PredictDLC(
File "/Users/name/opt/anaconda3/envs/DEEPLABCUT/lib/python3.9/site-packages/mesonet/gui_test.py", line 1365, in PredictDLC
DLCPredict(
File "/Users/name/opt/anaconda3/envs/DEEPLABCUT/lib/python3.9/site-packages/mesonet/dlc_predict.py", line 158, in DLCPredict
deeplabcut.analyze_videos(
File "/Users/name/opt/anaconda3/envs/DEEPLABCUT/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_videos.py", line 490, in analyze_videos
trainFraction = cfg["TrainingFraction"][trainingsetindex]
File "/Users/name/opt/anaconda3/envs/DEEPLABCUT/lib/python3.9/site-packages/ruamel/yaml/comments.py", line 851, in getitem
return ordereddict.getitem(self, key)
KeyError: 'TrainingFraction'

'model' directory missing

I've tried running the GUI using the mesonet.gui_start() method, but it returns a FileNotFoundError (see error immediately below)

In [10]: mesonet.gui_start()
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[10], line 1
----> 1 mesonet.gui_start()

File ~/Documents/MesoNet/mesonet/gui_start.py:24, in gui_start(gui_type, git_repo, config_file)
     11 """
     12 Starts the MesoNet GUI interface.
     13 
   (...)
     21 :return:
     22 """
     23 if gui_type == "test":
---> 24     gui_test.gui(git_repo, config_file)
     25 elif gui_type == "train":
     26     gui_train.gui()

File ~/Documents/MesoNet/mesonet/gui_test.py:1414, in gui(git_find, config_file)
   1413 def gui(git_find, config_file):
-> 1414     Gui(git_find, config_file).root.mainloop()

File ~/Documents/MesoNet/mesonet/gui_test.py:111, in Gui.__init__(self, git_repo, config_file)
    109 # Render model selector listbox
    110 self.modelSelect = []
--> 111 for file in os.listdir(self.model_top_dir):
    112     if fnmatch.fnmatch(file, "*.hdf5"):
    113         self.modelSelect.append(file)

FileNotFoundError: [Errno 2] No such file or directory: '/home/scott/Documents/MesoNet/mesonet/models'

I've also tried tried to run mesonet from the command line using mesonet.predict_regions(config_file), but that returns a similar error in finding files in the mesonet/models directory (see error immediately below).

In [11]: config_file = mesonet.config_project(input_file, output_file, 'test')
/home/scott/Documents/MesoNet/mesonet

In [12]: mesonet.predict_regions(config_file)
/home/scott/Documents/MesoNet/mesonet/models/unet_bundary.hdf5
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Cell In[12], line 1
----> 1 mesonet.predict_regions(config_file)

File ~/Documents/MesoNet/mesonet/predict_regions.py:153, in predict_regions(config_file)
    150 atlas_label_list = cfg["atlas_label_list"]
    151 original_label = cfg["original_label"]
--> 153 predictRegion(
    154     input_file,
    155     num_images,
    156     model,
    157     output,
    158     mat_save,
    159     threshold,
    160     mask_generate,
    161     git_repo_base,
    162     atlas_to_brain_align,
    163     dlc_pts,
    164     atlas_pts,
    165     olfactory_check,
    166     use_unet,
    167     plot_landmarks,
    168     align_once,
    169     atlas_label_list,
    170     region_labels,
    171     original_label,
    172 )

File ~/Documents/MesoNet/mesonet/predict_regions.py:83, in predictRegion(input_file, num_images, model, output, mat_save, threshold, mask_generate, git_repo_base, atlas_to_brain_align, dlc_pts, atlas_pts, olfactory_check, use_unet, plot_landmarks, atlas_label_list, align_once, region_labels, original_label)
     81     model_to_use = load_model(model_path)
     82 else:
---> 83     model_to_use = load_model(model)
     84 # Resizes and prepares images for prediction
     85 print(input_file)

File ~/anaconda3/envs/DLC/lib/python3.8/site-packages/keras/utils/traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     67     filtered_tb = _process_traceback_frames(e.__traceback__)
     68     # To get the full stack trace, call:
     69     # `tf.debugging.disable_traceback_filtering()`
---> 70     raise e.with_traceback(filtered_tb) from None
     71 finally:
     72     del filtered_tb

File ~/anaconda3/envs/DLC/lib/python3.8/site-packages/keras/saving/save.py:226, in load_model(filepath, custom_objects, compile, options)
    224 if isinstance(filepath_str, str):
    225     if not tf.io.gfile.exists(filepath_str):
--> 226         raise IOError(
    227             f"No file or directory found at {filepath_str}"
    228         )
    230     if tf.io.gfile.isdir(filepath_str):
    231         return saved_model_load.load(
    232             filepath_str, compile, options
    233         )

OSError: No file or directory found at /home/scott/Documents/MesoNet/mesonet/models/unet_bundary.hdf5

It seems like there's a problem with my install such that this /models directory is never created. Do you have any ideas about what's missing?

Also, just in case it's helpful, I'm running Ubuntu 22.04.1 LTS. And my Conda information is below. Thanks!

active environment : DLC
    active env location : /home/scott/anaconda3/envs/DLC
            shell level : 2
       user config file : /home/scott/.condarc
 populated config files : /home/scott/.condarc
          conda version : 22.9.0
    conda-build version : 3.22.0
         python version : 3.9.13.final.0
       virtual packages : __cuda=11.6=0
                          __linux=5.15.0=0
                          __glibc=2.35=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/scott/anaconda3  (writable)
      conda av data dir : /home/scott/anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/scott/anaconda3/pkgs
                          /home/scott/.conda/pkgs
       envs directories : /home/scott/anaconda3/envs
                          /home/scott/.conda/envs
               platform : linux-64
             user-agent : conda/22.9.0 requests/2.28.1 CPython/3.9.13 Linux/5.15.0-60-generic ubuntu/22.04.1 glibc/2.35
                UID:GID : 1021:1001
             netrc file : None
           offline mode : False

Training images not displaying

I am trying to start training my own MesoNet model to identify brain regions/landmarks but am running into issues with displaying the brain images in the MesoNet Trainer Interface. Although I am able to browse for the input folder and select the folder with my .png images, the images are not displaying on the screen and I am unable to paint in the cortex region. MesoNet was working and displaying these same images when initially testing on our data for the segmentations and landmark identifications. Do you have any guidance on how I can fix this issue and display these images? Thanks.

Issue with output_mask detection (command line)

Hello,

I was trying to analyze a subset of the provided test data on the command line, and I am running into an error. I was able to complete all of the steps up to mesonet.predict_dlc(config_file). When I try to run the DLC component, however, I get an apparent filepath error:

I've tried to change the file path from backslashes to forward-slashes in the mesonet_testing_config.yaml, but it doesn't seem to fix the issue.

Another issue (may or may not be related) is that when the progam is run, the file that the command is trying to reference (in this case, 1.png) is deleted. Running the mesonet.predict_regions command adds the file again.

Any help on this matter would be appreciated.

PNG write error macM1

Hi,

I am trying to get MesoNet running on a mac M1 laptop. I installed DeepCut using miniconda3 and MesoNet according to the instructions on the github main page. When I run 'test.py' it fails with the following output:

% python test.py
Loading DLC 2.3.5...
Successfully created the directory /Users/wtobin/MesoNet/mesonet/../tests/results/mesonet_output_atlas_brain
Successfully created the directory /Users/wtobin/MesoNet/mesonet/../tests/results/mesonet_output_brain_atlas
Successfully created the directory /Users/wtobin/MesoNet/mesonet/../tests/results/mesonet_output_sensory
Successfully created the directory /Users/wtobin/MesoNet/mesonet/../tests/results/mesonet_output_MBFM_U_Net
Successfully created the directory /Users/wtobin/MesoNet/mesonet/../tests/results/mesonet_output_voxelmorph
/Users/wtobin/MesoNet/mesonet/models/DongshengXiao_brain_bundary.hdf5

1. Atlas-to-brain warp with U-Net and DeepLabCut
   /Users/wtobin/MesoNet/mesonet
   /Users/wtobin/MesoNet/mesonet/models/DongshengXiao_brain_bundary.hdf5
   2023-07-05 19:30:30.923781: W tensorflow/tsl/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
   2023-07-05 19:30:30.946077: W tensorflow/c/c_api.cc:300] Operation '{name:'conv2d_24/bias/Assign' id:601 op device:{requested: '', assigned: ''} def:{{{node conv2d_24/bias/Assign}} = AssignVariableOp[_has_manual_control_dependencies=true, dtype=DT_FLOAT, validate_shape=false](conv2d_24/bias, conv2d_24/bias/Initializer/zeros)}}' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session.
   2023-07-05 19:30:32.398229: W tensorflow/c/c_api.cc:300] Operation '{name:'conv2d_21/bias/v/Assign' id:1423 op device:{requested: '', assigned: ''} def:{{{node conv2d_21/bias/v/Assign}} = AssignVariableOp[_has_manual_control_dependencies=true, dtype=DT_FLOAT, validate_shape=false](conv2d_21/bias/v, conv2d_21/bias/v/Initializer/zeros)}}' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session.
   /Users/wtobin/MesoNet/mesonet/../tests/sample_data/pipeline1_2
   /Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/mesonet-1.0.6.1-py3.9.egg/mesonet/predict_regions.py:93: UserWarning: Model.predict_generator is deprecated and will be removed in a future version. Please use Model.predict, which supports generators.
   /Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/keras/engine/training_v1.py:2359: UserWarning: Model.state_updates will be removed in a future version. This property should not be used in TensorFlow 2.0, as updates are applied automatically.
   updates=self.state_updates,
   2023-07-05 19:30:33.191665: W tensorflow/c/c_api.cc:300] Operation '{name:'conv2d_24/Sigmoid' id:607 op device:{requested: '', assigned: ''} def:{{{node conv2d_24/Sigmoid}} = SigmoidT=DT_FLOAT, _has_manual_control_dependencies=true}}' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session.
   Traceback (most recent call last):
   File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/PIL/PngImagePlugin.py", line 1277, in _save
   rawmode, mode = _OUTMODES[mode]
   KeyError: 'F'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/wtobin/MesoNet/tests/test.py", line 92, in
mesonet.predict_regions(config_file_atlas_brain)
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/mesonet-1.0.6.1-py3.9.egg/mesonet/predict_regions.py", line 153, in predict_regions
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/mesonet-1.0.6.1-py3.9.egg/mesonet/predict_regions.py", line 95, in predictRegion
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/mesonet-1.0.6.1-py3.9.egg/mesonet/mask_functions.py", line 116, in saveResult
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/skimage/io/_io.py", line 143, in imsave
return call_plugin('imsave', fname, arr, plugin=plugin, **plugin_args)
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/skimage/io/manage_plugins.py", line 205, in call_plugin
return func(*args, **kwargs)
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/imageio/v3.py", line 147, in imwrite
encoded = img_file.write(image, **kwargs)
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/imageio/core/v3_plugin_api.py", line 367, in exit
self.close()
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/imageio/plugins/pillow.py", line 123, in close
self._flush_writer()
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/imageio/plugins/pillow.py", line 457, in _flush_writer
primary_image.save(self._request.get_file(), **self.save_args)
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/PIL/Image.py", line 2413, in save
save_handler(self, fp, filename)
File "/Users/wtobin/miniconda/envs/DEEPLABCUT_M1/lib/python3.9/site-packages/PIL/PngImagePlugin.py", line 1280, in _save
raise OSError(msg) from e
OSError: cannot write mode F as PNG

Hoping you can help me sort this out,

-Willie