dattalab/moseq2-extract

Here it comes...

moseq2-extract's Issues

Test extraction parameters yield poor ROIs on extraction

Hello,

Thank you for creating this invaluable tool!

I am using the Interactive Arena Detection Tool to find the best parameters to fit an ROI (arena mask) around my arena and extract the mouse from our data. I found a good combination of parameters for each of my mice that gave good ROIs and a clean extraction of the mouse. For example:

However, upon extracting the data, 25% of the extracted MoSeq sessions give ROIs that do not match the test extraction's ROI. These ROIs did not fit the arena and did not give usable extractions. For example:

I double-checked that:

• the session_config file was being updated with the right parameters when I click "Save session parameters"
• the session_config parameter "manual_set_depth_range" is set to true
• I even tried changing the config parameter "manual_set_depth_range" to true manually, but this did not resolve the issue (I have since changed that parameter back to false in the config file)

Restarting the Jupyter Notebook also did not resolve the issue. My MoSeq2 app version is v1.3.1 and I am using a Windows subsystem for Linux on Windows 10. Why are the parameters yielding different results on the test extractions and the actual extractions?

We need more flip files

I'll mark as resolved once we had a few more flip files.

Installation errors related to ipython 6.0.0

Hi,

I got the following bunch of error related to IPython when installing moseq2-extract with Pip.

(/gpfs/data/afleisc2/moseq2) [apierre3@login003 data]$ pip install git+https://github.com/dattalab/moseq2-extract.git                                                                                       
Collecting git+https://github.com/dattalab/moseq2-extract.git                                                                                                                                               
  Cloning https://github.com/dattalab/moseq2-extract.git to /tmp/pip-req-build-ylww5p90                                                                                                                     
  Running command git clone -q https://github.com/dattalab/moseq2-extract.git /tmp/pip-req-build-ylww5p90                                                                                                   
Username for 'https://github.com': kir0ul                                                                                                                                                                   
Password for 'https://[email protected]':                                                                                                                                                                   
Requirement already satisfied: h5py in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from moseq2-extract==0.1.3) (2.10.0)                                                                         
Requirement already satisfied: matplotlib in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from moseq2-extract==0.1.3) (2.2.2)                                                                    
Collecting scipy<1.3.0,>=0.19                                                                                                                                                                               
  Downloading scipy-1.2.3-cp36-cp36m-manylinux1_x86_64.whl (24.8 MB)                                                                                                                                        
     |████████████████████████████████| 24.8 MB 11.9 MB/s                                                                                                                                                   
Requirement already satisfied: scikit-image>=0.13 in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from moseq2-extract==0.1.3) (0.14.5)                                                           
Requirement already satisfied: tqdm in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from moseq2-extract==0.1.3) (4.40.0)                                                                         
Requirement already satisfied: statsmodels>=0.8.0 in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from moseq2-extract==0.1.3)
Requirement already satisfied: numpy in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from moseq2-extract==0.1.3) (1.14.6)
Collecting joblib==0.13.1
  Downloading joblib-0.13.1-py2.py3-none-any.whl (278 kB)
     |████████████████████████████████| 278 kB 51.3 MB/s
Collecting scikit-learn
  Downloading scikit_learn-0.22.2.post1-cp36-cp36m-manylinux1_x86_64.whl (7.1 MB)
     |████████████████████████████████| 7.1 MB 55.5 MB/s
Collecting click
  Downloading click-7.1.1-py2.py3-none-any.whl (82 kB)
     |████████████████████████████████| 82 kB 154 kB/s
Collecting ruamel.yaml>=0.15.0
  Downloading ruamel.yaml-0.16.10-py2.py3-none-any.whl (111 kB)
     |████████████████████████████████| 111 kB 64.3 MB/s
Collecting six
  Downloading six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting pytz
  Downloading pytz-2019.3-py2.py3-none-any.whl (509 kB)
     |████████████████████████████████| 509 kB 57.0 MB/s
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.2.0-cp36-cp36m-manylinux1_x86_64.whl (88 kB)
     |████████████████████████████████| 88 kB 1.8 MB/s
Collecting cycler>=0.10
  Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting python-dateutil>=2.1
  Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1
  Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
     |████████████████████████████████| 67 kB 2.1 MB/s
Requirement already satisfied: networkx>=1.8 in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from scikit-image>=0.13->moseq2-extract==0.1.3) (2.4)
Requirement already satisfied: PyWavelets>=0.4.0 in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from scikit-image>=0.13->moseq2-extract==0.1.3) (1.1.1)
Requirement already satisfied: cloudpickle>=0.2.1 in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from scikit-image>=0.13->moseq2-extract==0.1.3) (1.2.2)
Requirement already satisfied: pillow>=4.3.0 in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from scikit-image>=0.13->moseq2-extract==0.1.3) (6.2.1)
Requirement already satisfied: patsy in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from statsmodels>=0.8.0->moseq2-extract==0.1.3) (0.5.1)
Requirement already satisfied: pandas in /gpfs_home/apierre3/.local/lib/python3.6/site-packages (from statsmodels>=0.8.0->moseq2-extract==0.1.3) (0.20.3)
Collecting ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.9"
  Using cached ruamel.yaml.clib-0.2.0-cp36-cp36m-manylinux1_x86_64.whl (548 kB)
Collecting decorator>=4.3.0
  Downloading decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Building wheels for collected packages: moseq2-extract
  Building wheel for moseq2-extract (setup.py) ... done
  Created wheel for moseq2-extract: filename=moseq2_extract-0.1.3-py3-none-any.whl size=28437 sha256=409505e1bc67332253f22478c3e6dd3d048419061449d3e7621effafac342221
  Stored in directory: /tmp/pip-ephem-wheel-cache-1vi2vs7c/wheels/b3/b0/2b/fc623850a7c3afb5140587dcf31f0aef3557bd930b1def0616
Successfully built moseq2-extract
ERROR: ipython 6.0.0 requires jedi>=0.10, which is not installed.
ERROR: ipython 6.0.0 requires pexpect; sys_platform != "win32", which is not installed.
ERROR: ipython 6.0.0 requires pickleshare, which is not installed.
ERROR: ipython 6.0.0 requires prompt-toolkit<2.0.0,>=1.0.4, which is not installed.
ERROR: ipython 6.0.0 requires pygments, which is not installed.
ERROR: ipython 6.0.0 requires simplegeneric>0.8, which is not installed.
ERROR: ipython 6.0.0 requires traitlets>=4.2, which is not installed.
Installing collected packages: scipy, joblib, scikit-learn, click, ruamel.yaml.clib, ruamel.yaml, moseq2-extract, six, pytz, kiwisolver, cycler, python-dateutil, pyparsing, decorator
Successfully installed click-7.1.1 cycler-0.10.0 decorator-4.4.2 joblib-0.13.1 kiwisolver-1.2.0 moseq2-extract-0.1.3 pyparsing-2.4.7 python-dateutil-2.8.1 pytz-2019.3 ruamel.yaml-0.16.10 ruamel.yaml.clib-0.2.0 scikit-learn-0.22.2.post1 scipy-1.2.3 six-1.14.0

So I'm not sure moseq2-extract has been correctly installed. Should I install these dependencies manually and try to reinstall moseq2-extract again?

Using moseq2-extract on Odyssey cluster

We run in to problems when running $moseq2-extract extract on Odyssey, but works fine running locally.

#########################################################################################
Error Running on Odyssey Cluster
#########################################################################################

rxie@rclogin03:~$ srun --pty --mem=50G -n 20 -N 1 -p test,shared -t 60 bash
rxie@holy7c09313:~$ source activate moseq2
(moseq2) rxie@holy7c09313:~$ cd /n/regal/uchida_lab/yuxie/MoSeq_Testing
(moseq2) rxie@holy7c09313:/n/regal/uchida_lab/yuxie/MoSeq_Testing$ ls
config.yaml  depth.dat	depth_ts.txt  metadata.json  rgb.mp4  rgb_ts.txt
(moseq2) rxie@holy7c09313:/n/regal/uchida_lab/yuxie/MoSeq_Testing$ moseq2-extract extract depth.dat
Getting background...
Getting roi...
Finding plane: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [00:00<00:00, 1494.88it/s]
Detected true depth: 678.0
Processing batches:   0%|                                                                                                         | 0/56 [00:00<?, ?it/sTraceback (most recent call last):██████████████████████████████████████████████████████████████████████████████████| 1000/1000 [00:00<00:00, 5893.07it/s]
  File "/n/home11/rxie/miniconda3/envs/moseq2/bin/moseq2-extract", line 11, in <module>
    load_entry_point('moseq2-extract', 'console_scripts', 'moseq2-extract')()
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/n/home11/rxie/python_repos/moseq2-extract/moseq2_extract/util.py", line 42, in invoke
    return super().invoke(ctx)
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/n/home11/rxie/python_repos/moseq2-extract/moseq2_extract/cli.py", line 293, in extract
    true_depth=true_depth)
  File "/n/home11/rxie/python_repos/moseq2-extract/moseq2_extract/extract/extract.py", line 122, in extract_chunk
    true_depth=true_depth)
  File "/n/home11/rxie/python_repos/moseq2-extract/moseq2_extract/extract/proc.py", line 423, in compute_scalars
    features['width_px'] = np.min(track_features['axis_length'], axis=1)
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 2442, in amin
    initial=initial)
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 83, in _wrapreduction
    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
FloatingPointError: invalid value encountered in reduce
Exception ignored in: <bound method tqdm.__del__ of Processing batches:   0%|                                                                                                         | 0/56 [00:11<?, ?it/s]>
Traceback (most recent call last):
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/tqdm/_tqdm.py", line 885, in __del__
    self.close()
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/tqdm/_tqdm.py", line 1090, in close
    self._decr_instances(self)
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/tqdm/_tqdm.py", line 454, in _decr_instances
    cls.monitor.exit()
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/site-packages/tqdm/_monitor.py", line 52, in exit
    self.join()
  File "/n/home11/rxie/miniconda3/envs/moseq2/lib/python3.6/threading.py", line 1053, in join
    raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread

#########################################################################################
Works fine when running locally
#########################################################################################

(moseq2) alex@UchidaGPU:/media/alex/noveltydisk/MoSeqData/MoSeq_Testing$ ls
config.yaml  depth.dat  depth_ts.txt  metadata.json  rgb.mp4  rgb_ts.txt
(moseq2) alex@UchidaGPU:/media/alex/noveltydisk/MoSeqData/MoSeq_Testing$ moseq2-extract extract depth.dat
Getting background...
Getting roi...
Finding plane: 100%|█████████████████████████████████████| 1000/1000 [00:00<00:00, 1037.60it/s]
Detected true depth: 678.0
Processing batches: 100%|██████████████████████████████████████| 56/56 [20:54<00:00, 22.40s/it]
Writing frames: 100%|███████████████████████████████████████| 685/685 [00:04<00:00, 139.17it/s]

User-defined Time to Start or End Extraction

I'm a MoSeq user at Uchida lab.
In our task, the first few seconds of the entire session is very important so we always start recording before putting our mice in the arena. I think the period when the mouse is not in the arena and when my arm is in the field of view might cause some problems to later analyses.
If there's a way to specify the starting time or and the ending time of an extraction, instead of extracting the entire file?

batch-extract ignores (some) parameters

I changed a couple of parameters in config.yaml such as num_frames or frame_trim.

This works with the extract CLI command.

However, when I change it to use batch-extract, these parameters are getting completely ignored and the whole video is processed.

I haven't tried it for other parameters, so it could be the case that other parameters are getting ignored as well which will lead to inconsistent behavior, e.g. when one tries to optimize parameters for his/her experiment and then assumes that he can process the other sessions with batch-extract.

Continued "Backing off..." messages followed by timeout

During moseq2-extract extract, I'm encountering continued "backing off..." messages. Eventually the extraction simply times out. It seems to be for a specific session as I've been extracting the same way every time. Oddly, I've also been recording the same way every time so I'm not sure why extracting this particular session is leading to this error. The roi, first frame, and bground tiffs all look very similar to other extractions as well. Any thoughts?

ModuleNotFoundError: No module named 'skimage.external'

Windows 10.0.19041.264
Debian WSL 1

When I cd to folder with depth.dat, depth_ts.txt, metadata.json and config.yaml,
and run moseq2-extract extract depth.data --config-file config.yaml
it gives an error message:
File "/home/username/miniconda3/envs/moseq2/bin/moseq2-extract", line 5, in
from moseq2_extract.cli import cli
File "/home/whr/miniconda3/envs/moseq2/lib/python3.6/site-packages/moseq2_extract/cli.py", line 3, in
from moseq2_extract.io.image import write_image, read_image
File "/home/whr/miniconda3/envs/moseq2/lib/python3.6/site-packages/moseq2_extract/io/image.py", line 1, in
from skimage.external import tifffile
ModuleNotFoundError: No module named 'skimage.external'

How can I load tifffile module manually ?

Feature request: pixel_format and movie_dtype shouldn't be overwritten by camera type params

Current behavior: passing in, eg, --pixel-format uint8 doesn't do anything because in extract > util.py > detect_and_set_camera_parameters(), the args are overwritten by the camera settings because camera_type = "auto." To pass in custom params to pixel and movie formats, you also have to specify camera_type = either azure or kinect.

Expected behavior: a few options. I don't know the full extent of what camera_type is used for throughout the extraction, so take all this with a grain of salt.

1. [not preferred but easiest] instead of failing silently, when you pass in custom args for pixel_format and movie_dtype without changing camera type from auto, it should raise an exception and specify that you need to specify camera type.
2. [preferred] in between setting the default param dictionary and the line where it gets applied to the config (config_data.update(**default_parameters[detected])), update default_parameters to include any user specified stuff instead.

does bg-roi-dilate do anything?

Comparing the masks from a small and large dilation reveal that they produce the same roi mask.

Minor bug: recompute-bg should probably be a flag

Current you have to pass --recompute-bg True. Seems excessive and can cause weird silent errors.

Failure to generate default config parameters

Generated by running:
moseq2-extract make-default-config

Traceback (most recent call last):
  File "/Users/wgillis/anaconda2/envs/py3/bin/moseq2-extract", line 11, in <module>
    load_entry_point('moseq2-extract', 'console_scripts', 'moseq2-extract')()
  File "/Users/wgillis/anaconda2/envs/py3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/Users/wgillis/anaconda2/envs/py3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2693, in load_entry_point
    return ep.load()
  File "/Users/wgillis/anaconda2/envs/py3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in load
    return self.resolve()
  File "/Users/wgillis/anaconda2/envs/py3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Users/wgillis/dev/moseq2/moseq2_extract/cli.py", line 121, in <module>
    @click.option("--config-file", type=click.Path())
  File "/Users/wgillis/anaconda2/envs/py3/lib/python3.6/site-packages/click/decorators.py", line 151, in decorator
    _param_memo(f, ArgumentClass(param_decls, **attrs))
  File "/Users/wgillis/anaconda2/envs/py3/lib/python3.6/site-packages/click/core.py", line 1699, in __init__
    Parameter.__init__(self, param_decls, required=required, **attrs)
TypeError: __init__() got an unexpected keyword argument 'help'

Question: Really struggling tuning parameters for find ROI, any advice?

Thank you all for your generous help.
We found it really tricky to find an proper ROI when extracting. Some of videos succeded in finding ROI , while some of them did not, shown as below (upper):

(unsuccesful finding ROI)

(succesful finding ROI)
I checked issue #54, and tried some parametre combinations, but sadly they did not work. Is there any tricks in tuning paremeters for finding ROI? Very grateful for your advice !

--spatial-filter-size cannot be even

Traceback (most recent call last):
  File "/Users/wgillis/anaconda/envs/py3/bin/moseq2-extract", line 11, in <module>
    load_entry_point('moseq2-extract', 'console_scripts', 'moseq2-extract')()
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/wgillis/dev/lab/moseq2/moseq2_extract/util.py", line 37, in invoke
    return super(custom_command_class, self).invoke(ctx)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/wgillis/dev/lab/moseq2/moseq2_extract/cli.py", line 247, in extract
    crop_size=crop_size)
  File "/Users/wgillis/dev/lab/moseq2/moseq2_extract/extract/extract.py", line 41, in extract_chunk
    progress_bar=progress_bar)
  File "/Users/wgillis/dev/lab/moseq2/moseq2_extract/extract/proc.py", line 258, in clean_frames
    filtered_frames[i, ...] = cv2.medianBlur(filtered_frames[i, ...], prefilter_space[j])
cv2.error: /Users/travis/build/skvark/opencv-python/opencv/modules/imgproc/src/smooth.cpp:3125: error: (-215) (ksize % 2 == 1) && (_src0.dims() <= 2 ) in function medianBlur

animal in background image

Trying to extract some new data and can't seem to get a background image without the animal in it.

Example:

I used from moseq2_extract.extract.proc import get_bground_im_file to try adjusting the stride length in a jupyter notebook, but that didn't seem to work. Any ideas? Can point you to the raw data if you'd like.

Error with find-roi

Traceback (most recent call last):
  File "/home/wgillis/miniconda3/bin/moseq2", line 11, in <module>
    load_entry_point('moseq2', 'console_scripts', 'moseq2')()
  File "/home/wgillis/miniconda3/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/wgillis/miniconda3/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/wgillis/miniconda3/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/wgillis/miniconda3/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/wgillis/miniconda3/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/wgillis/dev/moseq2/moseq2/cli.py", line 54, in find_roi
    rois, _, _, _, _ = get_roi(bground_im, strel_dilate=strel_dilate, weights=roi_weights)
ValueError: too many values to unpack (expected 5)

Feature request: CLI moseq2-extract extract should print which file it's working on

If using CLI to run moseq2-extract extract, I would find it useful to have filenames in the .out files from slurm so that if one fails, it's easy to tell which session it was. Often the error message due to failing includes the file name, but not always. A line like "Extracting from file ..." would suffice.

Too few parameters exposed for `convert_raw_to_avi`

This function needs more of the movie datatype parameters exposed for flexible movie conversion.

Allow user to specify `frames_is_timestamp` for .avi files

Right now, when reading avi files, the default behavior is to assume that timestamps increment linearly throughout the whole movie. This is in contrast to .mkv files, where the timestamps are loaded explicitly and then read_frames is called with frames_is_timestamp=True.

There should also be an option to explicitly use the timestamps in avi files, similar to how it works for mkv files.

batch extraction shooting blanks

1. Ran batch extraction
2. Aggregated extraction results
3. Noticed sessions missing from aggregate list
4. Checked to see that those sessions extracted
5. Missing results.mp4 files in those sessions 'proc' folder
6. Attempted to re-run extraction in the same directory as the first time

• Generate config successful, added directory for flip classifier
• When compiling list of jobs into batch_command.sh getting empty file
• No errors, no output at all

Symlinks to depth files convert to original paths

I have symlinks pointing to depth.mkv files. During batch extraction, the path to each depth file gets converted back to the depth file's original path. This results in the output files (everything under /proc/) getting saved under the original path instead of the symlink directory.

One potential fix for this is to change the line @click.argument('input-file', type=click.Path(exists=True, resolve_path=True)) in cli.py to @click.argument('input-file', type=click.Path(exists=True, resolve_path=False)). i.e. setting resolve_path to False fixes the issue.

cli parameters not overriding config-defined parameters: realsense branch

When I'm running an extraction using a config file, config params are not updated accordingly.
I.e.,
moseq2-extract extract depth-000.avi --config-file ../../../extraction-params/realsense-90hz-480p.yaml --floor-range 100 1000 --chunk-size 1000 --max-height 100 --min-height 10

The yaml file specifies a 1500 chunk size, yet is not being updated in the output.

ValueError with OpenCV

Hi,

I get the a ValueError: too many values to unpack (expected 2) when I run the moseq2-extract extract command on the good-session/depth.dat file. Here is the full stack trace:

$ moseq2-extract extract good-session/depth.dat 
Processing: /gpfs/scratch/apierre3/moseq_test_data/good-session/depth.dat
Loading background...
Loading ROI...
Detected true depth: 673.0
Cleaning frames: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 900/900 [00:00<00:00, 1734.03it/s]
Computing moments:   0%|                                                                                                                                                            | 0/900 [00:00<?, ?it/s]
Processing batches:   0%|                                                                                                                                                             | 0/1 [00:04<?, ?it/s]
Traceback (most recent call last):
  File "/gpfs_home/apierre3/data/moseq2/bin/moseq2-extract", line 8, in <module>
    sys.exit(cli())
  File "/gpfs/data/afleisc2/moseq2/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/gpfs/data/afleisc2/moseq2/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/gpfs/data/afleisc2/moseq2/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/gpfs/data/afleisc2/moseq2/lib/python3.6/site-packages/moseq2_extract/util.py", line 43, in invoke
    return super().invoke(ctx)
  File "/gpfs/data/afleisc2/moseq2/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/gpfs/data/afleisc2/moseq2/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/gpfs/data/afleisc2/moseq2/lib/python3.6/site-packages/moseq2_extract/cli.py", line 408, in extract
    tracking_model_init=tracking_model_init)
  File "/gpfs/data/afleisc2/moseq2/lib/python3.6/site-packages/moseq2_extract/extract/extract.py", line 77, in extract_chunk
    progress_bar=progress_bar)
  File "/gpfs/data/afleisc2/moseq2/lib/python3.6/site-packages/moseq2_extract/extract/proc.py", line 353, in get_frame_features
    cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
ValueError: too many values to unpack (expected 2)

Bug using bg-roi-depth-range in the CLI

Jonah found this bug when trying to set the depth range manually. Previously, this command expected a tuple of values, but now when running the extract command, the --bg-roi-depth-range flag expects a string and defaults to auto. Since this flag now expects a string, there is no clear way of allowing the user to specify their own depth range.

Options:

1. The user can feed in a tuple into the command line flag, like this: --bg-roi-depth-range "(600, 700)" and the flag must be preprocessed and turned into a tuple in the extract code somewhere.
2. This flag reverts to the old specification, and we add a new (boolean) flag that specifies if we want to automatically find the depth range.

Running moseq2-extract extract with all defaults

Traceback (most recent call last):
  File "/Users/wgillis/anaconda/envs/py3/bin/moseq2-extract", line 11, in <module>
    load_entry_point('moseq2-extract', 'console_scripts', 'moseq2-extract')()
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 1064, in invoke
    sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 621, in make_context
    self.parse_args(ctx, args)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 880, in parse_args
    value, args = param.handle_parse_result(ctx, opts, args)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 1396, in handle_parse_result
    value = self.full_process_value(ctx, value)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 1681, in full_process_value
    return Parameter.full_process_value(self, ctx, value)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 1368, in full_process_value
    value = self.get_default(ctx)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 1636, in get_default
    return Parameter.get_default(self, ctx)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 1312, in get_default
    return self.type_cast_value(ctx, rv)
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 1344, in type_cast_value
    return _convert(value, (self.nargs != 1) + bool(self.multiple))
  File "/Users/wgillis/anaconda/envs/py3/lib/python3.6/site-packages/click/core.py", line 1343, in _convert
    return tuple(_convert(x, level - 1) for x in value or ())
TypeError: 'int' object is not iterable

Additional smoothing of tracking parameters

We did this in a previous version of the pipeline for Inscopix data. Time to add back in for version v0.1.1.

unable to compress depth videos that are shorter than the specified chunk size

When I try to run moseq2-extract convert-raw-to-avi depth.dat --delete --chunk-size 4000 on a dataset that is 575 MB in size, the compression does not work, but I get no error messages.

Add in ability to save relative paths, rather than absolute paths in the moseq2-index.yaml file

This is a nice feature to have, especially if you're copying data between two locations and only one has moseq installed. Since one of the locations does not have moseq installed, you can't regenerate a new moseq2-index.yaml file with the correct paths

realsense branch: movie output does not support changes in fps

Regardless of the sampling rate set, movies are always written as 30fps

Is it normal that true_depth and depth_range are forced to fixed values ?

Hello,

Is that normal that many times in the extract code and imported functions there is variables forced : true_depth=673.1 or for example depth_range=(600, 800), which are quite different of my experiment ? I tried to change that but it is forced in several functions and I don't get how to solve that issue.
It might be explaining why I have a very strange ROI and a lack of detection of the mouse. Please could you please check and inform about how this can affect the different functions where it is defined (compute_scalars, roi,extract_chunk...)

Thank you very much for your help,

Julien

using deepcopy instead of np.copy

do you mind if I change instances of deepcopy to use the numpy array method .copy() instead? I think this is more expressable, and might be higher performance.

I.e., check out video.py

bug in movie_dtype for read_frames_raw

read_frames_raw sets movie_dtype='<i2', which converts data read in from depth.dat files as int16, when in reality they should be uint16 values. movie_dtype should actually be set to <u2.

Flipping at batch changes

We are seeing a lot of flips in some of our videos. The videos seem to flip arbitrarily at the thousandth frame (1000, 4000, 5000, 7000, etc). We tested changing the batch size to 500, 2000, 10000, and ~27000 (full length). 500, 1000, and 2000 all flipped a similar amount of frames but (with a couple exceptions) only flipped at multiples of 500, 1000, or 2000 respectively. I’ve attached a couple examples. The full length videos either flip all or none of the frames. My experience with this project is that (normal) flips typically occur when the mouse is rearing or scrunched into a ball, but that is not the case here.

We appreciate any advice you can offer. Thank you

" find_roi() got an unexpected keyword argument 'crop_size' " when running moseq2-extract from moseq2-batch generated command

I am trying to extract a recording with two ROIs (each with its own mouse).

I ran the line "moseq2-batch extract-batch --config-file config.yaml --bg-roi-index 0 --bg-roi-index 1" to get the appropriate lines for running moseq2-extract. The output is below

sbatch -n 4 --mem=5GB -p short -t 3:00:00 --wrap "source activate moseq2; moseq2-extract find-roi --config-file /home/csw14/moseq2/job_config_2019-01-12_18-56-57.yaml /n/groups/datta/caleb/socialbox_mini/session_20181030122317/depth.dat; moseq2-extract extract --config-file /home/csw14/moseq2/job_config_2019-01-12_18-56-57_roi0.yaml --bg-roi-index 0 /n/groups/datta/caleb/socialbox_mini/session_20181030122317/depth.dat; moseq2-extract extract --config-file /home/csw14/moseq2/job_config_2019-01-12_18-56-57_roi1.yaml --bg-roi-index 1 /n/groups/datta/caleb/socialbox_mini/session_20181030122317/depth.dat; "

Here is the same text but broken into separate lines and stripped of the SLURM shit and virtual env activation:

moseq2-extract find-roi --config-file /home/csw14/moseq2/job_config_2019-01-12_18-56-57.yaml /n/groups/datta/caleb/socialbox_mini/session_20181030122317/depth.dat
moseq2-extract extract --config-file /home/csw14/moseq2/job_config_2019-01-12_18-56-57_roi0.yaml --bg-roi-index 0 /n/groups/datta/caleb/socialbox_mini/session_20181030122317/depth.dat
moseq2-extract extract --config-file /home/csw14/moseq2/job_config_2019-01-12_18-56-57_roi1.yaml --bg-roi-index 1 /n/groups/datta/caleb/socialbox_mini/session_20181030122317/depth.dat

These commands generated an error when run directly or as a SLURM job. The traceback is below:

Traceback (most recent call last):
  File "/home/csw14/miniconda3/envs/moseq2/bin/moseq2-extract", line 11, in <module>
    sys.exit(cli())
  File "/home/csw14/miniconda3/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/csw14/miniconda3/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/csw14/miniconda3/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/csw14/miniconda3/envs/moseq2/lib/python3.6/site-packages/moseq2_extract/util.py", line 42, in invoke
    return super().invoke(ctx)
  File "/home/csw14/miniconda3/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/csw14/miniconda3/envs/moseq2/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
TypeError: find_roi() got an unexpected keyword argument 'crop_size'

RuntimeError: cannot join thread when attempting to process depth.avi files

Are depth.avi files supported yet for extraction? This is the error I get when attempting an extraction.

Exception ignored in: <bound method tqdm.__del__ of Processing batches:   0%|          | 0/54 [00:12<?, ?it/s]>
Traceback (most recent call last):
  File "/home/nn105/miniconda3/envs/moseq2/lib/python3.6/site-packages/tqdm/_tqdm.py", line 889, in __del__
    self.close()
  File "/home/nn105/miniconda3/envs/moseq2/lib/python3.6/site-packages/tqdm/_tqdm.py", line 1095, in close
    self._decr_instances(self)
  File "/home/nn105/miniconda3/envs/moseq2/lib/python3.6/site-packages/tqdm/_tqdm.py", line 454, in _decr_instances
    cls.monitor.exit()
  File "/home/nn105/miniconda3/envs/moseq2/lib/python3.6/site-packages/tqdm/_monitor.py", line 52, in exit
    self.join()
  File "/home/nn105/miniconda3/envs/moseq2/lib/python3.6/threading.py", line 1053, in join
    raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread

Bug using --bg-roi-depth-range and --autoset-depth-range

@wingillis just found that --bg-roi-depth-range should never equal to auto and the current setup may make the GUI tool unable to auto-detect parameters to generate config.yaml for ROI widget.

Since --bg-roi-depth-range is set to (650, 750) by default, and the type is expected to be (float, float), the flag should never equal a string. The current if statement shows that when autoset_depth_range is False, the depth will be auto-detected, but the auto-detection should happen when autoset_depth_range==True, which is more intuitive.

In the GUI, config.yaml is generated from default parameters from CLICK, therefore, the config.yaml needs to specify that the depth to be auto-detected.

Suggested changes:

1. moseq2_extract/cli.py line 61
   function = click.option('--autoset-depth-range', is_flag=True, -> function = click.option('--autoset-depth-range', is_flag=True, default=True
2. When using the GUI function, the flag is set up in such a way that the depth can be automatically detected. When using the CLI, when there is a user input for --bg-roi-depth-range, the --autoset-depth-range is set to False and the user input range is used.

Updated suggested changes:
Instead of click.option('--autoset-depth-range', is_flag=True, maybe it's a better idea to click.option('--manualset-depth-range', is_flag=True,, the flag will default to False to trigger the automatic depth detection for GUI. Then when users specify '--manualset-depth-range', the bg-roi-depth-range will be set to user input value, and this option should have no default value.

trimming frames throws error when saving to h5

Let's say I record 100k frames and want to use the parameter frame_trim to trim the first 10k and last 10k frames.

Then, nframes is 80k and frame_range goes from 10k to 90k.

When creating the h5 file, memory is allocated for scalars and frames for 80k frames.

helpers/data::create_extract_h5

# Creating scalar dataset
  for scalar in list(scalars_attrs.keys()):
      h5_file.create_dataset(f'scalars/{scalar}', (nframes,), 'float32', compression='gzip')
      h5_file[f'scalars/{scalar}'].attrs['description'] = scalars_attrs[scalar]

  # Cropped Frames
  h5_file.create_dataset('frames', (nframes, config_data['crop_size'][0], config_data['crop_size'][1]),
                   config_data['frame_dtype'], compression='gzip')

When extracting, the h5 file is filled by indexing via the framerange:

helpers/extract::write_extracted_chunk_to_h5

  # Writing computed scalars to h5 file
  for scalar in scalars:
      h5_file[f'scalars/{scalar}'][frame_range] = results['scalars'][scalar][offset:]

  # Writing frames and mask to h5
  h5_file['frames'][frame_range] = results['depth_frames'][offset:]

frame_range is between 10k-90k, but only 80k frames are allocated, so it will throw and error as soon as it tries to set the frame number 80001:

Traceback (most recent call last):
  File "/root/miniconda3/envs/moseq2-app/bin/moseq2-extract", line 8, in <module>
    sys.exit(cli())
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_extract/util.py", line 111, in invoke
    return super().invoke(ctx)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_extract/cli.py", line 179, in extract
    extract_wrapper(input_file, output_dir, config_data, num_frames=num_frames, skip=skip_completed)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_extract/helpers/wrappers.py", line 381, in extract_wrapper
    output_mov_path=movie_filename)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_extract/helpers/extract.py", line 161, in process_extract_batches
    write_extracted_chunk_to_h5(h5_file, results, config_data, scalars, frame_range, offset)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_extract/helpers/extract.py", line 38, in write_extracted_chunk_to_h5
    h5_file[f'scalars/{scalar}'][frame_range] = results['scalars'][scalar][offset:]
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/h5py/_hl/dataset.py", line 685, in __setitem__
    selection = sel.select(self.shape, args, dsid=self.id)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/h5py/_hl/selections.py", line 90, in select
    sel[args]
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/h5py/_hl/selections.py", line 401, in __getitem__
    start, count, step, scalar = _handle_simple(self.shape, vector)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/h5py/_hl/selections.py", line 466, in _handle_simple
    x,y,z = _translate_int(int(arg), length)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/h5py/_hl/selections.py", line 486, in _translate_int
    raise ValueError("Index (%s) out of range (0-%s)" % (exp, length-1))
ValueError: Index (59196) out of range (0-59195)

Solution would be to subtract first_frame_idx from frame_range before writing the data.

Make pixel format for extraction preview videos "yuv420p" so that they are compatible with viewers like quicktime

Instead of "rgb24", change the default pixel format to "yuv420p". This is a default pixel format that many video players know how to render.