Giter Site home page Giter Site logo

sgoldenlab / simba Goto Github PK

View Code? Open in Web Editor NEW
280.0 280.0 138.0 1.23 GB

SimBA (Simple Behavioral Analysis), a pipeline and GUI for developing supervised behavioral classifiers

Home Page: https://simba-uw-tf-dev.readthedocs.io/

License: GNU General Public License v3.0

Python 98.66% CSS 0.75% Jupyter Notebook 0.60%

simba's Issues

Add Simba to Open Neuroscience

Hello!

We are reaching out because we would love to have your project listed on Open Neuroscience, and also share information about this project:

Open Neuroscience is a community run project, where we are curating and highlighting open source projects related to neurosciences!

Briefly, we have a website where short descritptions about projects are listed, with links to the projects themselves, their authors, together with images and other links.

Once a new entry is made, we make a quick check for spam, and publish it.

Once published, we make people aware of the new entry by Twitter and a Facebook group.

To add information about their project, developers only need to fill out this form

In the form, people can add subfields and tags to their entries, so that projects are filterable and searchable on the website!

The reason why we have the form system is that it makes it open for everyone to contribute to the website and allows developers themselves to describe their projects!

Also, there are so many amazing projects coming out in Neurosciences that it would be impossible for us to keep track and log them all!

Please get in touch if you have any questions or would like to collaborate!

Cannot extract frames

Hi, First of all thank you for creating SIMBA! This is my 1st approach to this kind of tracking programs and I'm just learning to use the program.
I'm running SIMBA w/TF in Windows 10 with:

  • Intel i7 (8th gen)
  • NVIDIA GeForce MX250 GPU
  • 16GB RAM

I created a DLC project with one video, and when I want to extract the frames in this video (w/automatic form and uniform algorithm) I get the following in my anaconda prompt:

Exception in Tkinter callback
Traceback (most recent call last):
  File "c:\users\agost\anaconda3\envs\dlc-gpu\lib\tkinter\__init__.py", line 1705, in __call__
    return self.func(*args)
  File "c:\users\agost\anaconda3\envs\dlc-gpu\lib\site-packages\simba\SimBA.py", line 2148, in dlc_extractframes_command
    select_numfram2pick(config_path,self.label_numframes2pick.entry_get)
NameError: name 'select_numfram2pick' is not defined

Installation help

Describe the bug
After entering the command

git clone -b SimBA_no_DLC https://github.com/sgoldenlab/simba.git

files are updated correctly, then after the command

pip install -r simba/SimBA/requirements.txt

I receive the error

ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

Under the correct directory, then running the command

python SimBA.py

produces the following

Traceback (most recent call last):
File "SimBA.py", line 8, in
from tkinter_functions import *
ModuleNotFoundError: No module named 'tkinter_functions'

Any advice would be appreciated.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS] Windows 10
  • Browser [e.g. chrome, safari] Chrome
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Impossible to set video parameters.

Hello,

I'm using multi-animal pose-estimation data in SimBA. I already extracted the frames, and should now set the video parameters. But here the error message my prompt returns:

Untitled

Consequently, no video row appear, and I cannot define the scales on my videos.

Would someone please have a suggestion to fix this problem?

FYI: I'm using the 1.2.5 simba version and get the same error on windows 7 and 10. My videos are .avi.

Thank you,
Best,
Dorian

Error in Extract Features Rolling Filter

Describe the bug
I'm doing a test run using an hour long mp4 file and DLC labels for 1 animal, 8 body parts. Extract Features appears to be working until it hits "Calculating rolling windows: medians, medians, and sums..." at which I get the following error:


Simba error

Desktop (please complete the following information):

  • OS: Windows 10
  • Python Version: 3.6.2
  • Are you using anaconda? Yes

Feature: Skip outlier correction

Is your feature request related to a problem? Please describe.
This feature request is in direct connection to this issue

Describe the solution you'd like
As this is mainly a copy/paste of data, plus some minor adjustments to the additional logs you create:
My suggestion would be a button in the outlier correction tab that creates a dummy outlier correction (without the calculations) file and handles the data movement/creation just as the normal function would.
Another alternative would be to allow the import of filtered/corrected files that will be transformed into your style of logging/handling these kinds of files.
It probably depends if there is any use for a truly raw tracking file in SimBa's workflow if the advanced user has already the abililty to filter/correct them disconnected from SimBa.

Describe alternatives you've considered
An easy alternative is to set the parameters given to the correction higher than the max resolution of the video, so that no bodypart can ever reach that distance.

Additional context
I understand that this is a feature that helps refining raw tracking data further and is therefore intentionally not skippable. This is a minor request and the community can probably live without it for sure. But, as many groups work with tracking data like the one from DLC, some adapted their own tricks to handle outliers. Therefore it might be a usefull feature, for advanced users.

Thank you for your hard work so far.

Error in generateFeatureImportanceBarGraph

Describe the bug
After starting the training with the following settings, the error occured.

grafik

[Parallel(n_jobs=-1)]: Using backend ThreadingBackend with 12 concurrent workers.
[Parallel(n_jobs=-1)]: Done  26 tasks      | elapsed:    8.9s
[Parallel(n_jobs=-1)]: Done 176 tasks      | elapsed:   51.3s
[Parallel(n_jobs=-1)]: Done 426 tasks      | elapsed:  2.1min
[Parallel(n_jobs=-1)]: Done 776 tasks      | elapsed:  3.7min
[Parallel(n_jobs=-1)]: Done 1226 tasks      | elapsed:  5.8min
[Parallel(n_jobs=-1)]: Done 1776 tasks      | elapsed:  8.4min
[Parallel(n_jobs=-1)]: Done 2000 out of 2000 | elapsed:  9.4min finished
[Parallel(n_jobs=12)]: Using backend ThreadingBackend with 12 concurrent workers.
[Parallel(n_jobs=12)]: Done  26 tasks      | elapsed:    0.0s
[Parallel(n_jobs=12)]: Done 176 tasks      | elapsed:    0.2s
[Parallel(n_jobs=12)]: Done 426 tasks      | elapsed:    0.5s
[Parallel(n_jobs=12)]: Done 776 tasks      | elapsed:    1.0s
[Parallel(n_jobs=12)]: Done 1226 tasks      | elapsed:    1.6s
[Parallel(n_jobs=12)]: Done 1776 tasks      | elapsed:    2.4s
[Parallel(n_jobs=12)]: Done 2000 out of 2000 | elapsed:    2.7s finished
[Parallel(n_jobs=12)]: Using backend ThreadingBackend with 12 concurrent workers.
[Parallel(n_jobs=12)]: Done  26 tasks      | elapsed:    0.0s
[Parallel(n_jobs=12)]: Done 176 tasks      | elapsed:    0.2s
[Parallel(n_jobs=12)]: Done 426 tasks      | elapsed:    0.5s
[Parallel(n_jobs=12)]: Done 776 tasks      | elapsed:    1.0s
[Parallel(n_jobs=12)]: Done 1226 tasks      | elapsed:    1.6s
[Parallel(n_jobs=12)]: Done 1776 tasks      | elapsed:    2.4s
[Parallel(n_jobs=12)]: Done 2000 out of 2000 | elapsed:    2.7s finished
[Parallel(n_jobs=12)]: Using backend ThreadingBackend with 12 concurrent workers.
[Parallel(n_jobs=12)]: Done  26 tasks      | elapsed:    0.0s
[Parallel(n_jobs=12)]: Done 176 tasks      | elapsed:    0.2s
[Parallel(n_jobs=12)]: Done 426 tasks      | elapsed:    0.5s
[Parallel(n_jobs=12)]: Done 776 tasks      | elapsed:    1.0s
[Parallel(n_jobs=12)]: Done 1226 tasks      | elapsed:    1.6s
[Parallel(n_jobs=12)]: Done 1776 tasks      | elapsed:    2.3s
[Parallel(n_jobs=12)]: Done 2000 out of 2000 | elapsed:    2.6s finished
Exception in thread Thread-7:
Traceback (most recent call last):
  File "D:\Anaconda\envs\simba-gpu\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "D:\Anaconda\envs\simba-gpu\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "simba.py", line 3737, in trainsinglemodel
    trainmodel2(self.projectconfigini)
  File "C:\Users\schwa\PycharmProjects\simba\simba\train_model_2.py", line 302, in trainmodel2
    generateFeatureImportanceBarGraph(log_df, N_feature_importance_bars)
UnboundLocalError: local variable 'log_df' referenced before assignment

Additional context
I guess this is an error based on the missing selection/toggle of "Generate Features Importance Log". Do you need to select both?

3d-coordinates

Is it possible to use 3d coordinates with this software? theoretically it should make it more accurate, especially for videos shot at non-90* angles.

Frames input folder files empty

Hello! :)
I am having issues with labelling the behaviours. Whenever I try to do it, it seems like the GUI does not load the extracted frames correctly, leaving me with a blank space where the video frames should be presented. I have tried extracting the frames again, but it tells me the frames have already been extracted, despite the fact that the folders inside input folder are empty. Any ideas on how to solve this? I am using the no DLC version, and I obtained the csvs from colab.

Thank you!
Filipa

Incompatabilities when installing SimBA with TF in fresh conda env.

When installing SimBA with TF in a fresh conda env. on windows 10 I get the following error message:

ERROR: imgaug 0.4.0 has requirement numpy>=1.15, but you'll have numpy 1.14.5 which is incompatible.
ERROR: imbalanced-learn 0.6.2 has requirement scikit-learn>=0.22, but you'll have scikit-learn 0.19.2 which is incompatible.
ERROR: tensorflow-gpu 1.14.0 has requirement protobuf>=3.6.1, but you'll have protobuf 3.6.0 which is incompatible.
ERROR: yellowbrick 0.9.1 has requirement scikit-learn>=0.20, but you'll have scikit-learn 0.19.2 which is incompatible.

I input the following commands:

conda create --name SimBA python=3.6
conda activate SimBA
git clone -b master https://github.com/sgoldenlab/simba.git
pip install -r simba/simba/requirements.txt

It installs several of the packages before I encounter the error.

error importing SLEAP .slp file

When trying to import a .slp file into simba I get the following error in "project configuration / import tracking data":

simba_error

I have imported the corresponding video in "project configuration / import videos into project folder".

What is the cause of this issue? Any suggestions on how to solve it?

install issue/question

First of all, thank you for developing SimBA!

I'm having an install issue that I cant seem to get around with dlc-windowsGPU (windows environment)

(dlc-windowsGPU) C:\Users\xxxxx\simba\simba>simba.py
Traceback (most recent call last):
File "C:\Users\maplight\simba\simba\SimBA.py", line 5, in
import deeplabcut
ModuleNotFoundError: No module named 'deeplabcut'

I have all the proper dependencies installed, I'm assuming it's a path I'm missing somewhere. using Anaconda. I have DeepLab installed:

(dlc-windowsGPU) C:\Users\xxxxx\DeepLabCut\

Thank you in advance for any guidance.

ROI feature?

Hello everybody!

First of all, thank you for developing SimBA :) I was wondering if in the future it would be possible to add a feature that would allow users to define a ROI and use that feature as part of the classifier training.

Thanks!
Filipa

Error during Outlier correction

Setting an outlier correct C similar to described in the documentation (C = 1.5), or any other value causes the SimBA main window to be stuck at:
Pose-estimation body part setting for outlier correction: 7
Processing 1 files for movement outliers...
While cmd prompts the following:

Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Users\Ejdrup\AppData\Local\Programs\Python\Python36\lib\tkinter\__init__.py", line 1699, in __call__
    return self.func(*args)
  File "SimBA.py", line 3981, in correct_outlier
    dev_move_user_defined(configini)
  File "C:\ProgramData\simba\simba\outlier_scripts\movement\correct_devs_mov_user_defined.py", line 115, in dev_move_user_defined
    df_p_cols = pd.DataFrame([csv_df.pop(x) for x in p_cols]).T
  File "C:\Users\Ejdrup\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas\core\frame.py", line 450, in __init__
    arrays, columns = to_arrays(data, columns, dtype=dtype)
  File "C:\Users\Ejdrup\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas\core\internals\construction.py", line 471, in to_arrays
    data, columns, coerce_float=coerce_float, dtype=dtype
  File "C:\Users\Ejdrup\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas\core\internals\construction.py", line 522, in _list_of_series_to_arrays
    indexer = indexer_cache[id(index)] = index.get_indexer(columns)
  File "C:\Users\Ejdrup\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas\core\indexes\base.py", line 2985, in get_indexer
    "Reindexing only valid with uniquely" " valued Index objects"
pandas.core.indexes.base.InvalidIndexError: Reindexing only valid with uniquely valued Index objects

Movie filename mismatch in csv files: a bug and a fix

Hi again,

I am using the new version 1.1 of the SimBA environment, and came over a small glitch, I think related rather to my particular way of using it.

So, the bug:

after the "Outlier Correction" step I proceeded directly to "Extract Features" step, but got an error message in the main SimBA GUI window: "Error: make sure all the videos that are going to be analyzed are represented in the project_folder/logs/video_info.csv file". The step thus has failed.

Possible reason and working empiric fix:

The postures were analysed separately using DeepLabCut (not via SimBA pipeline), and then the .csv files with coordinates from DeepLabCut were loaded into the the resulting .csv files. Thus, the names of those files (something like "Dec0419_Trial1DLC_resnet50_resident_intruder_optogeneticsFeb14shuffle1_600000.csv") were not the same as the names of the movies loaded into the project ("Dec0419_Trial1.mp4"). The former names were listed in the log files project_folder\log\Outliers_location_nnnnnnn.csv and project_folder\log\Outliers_movement_nnnnnnnnnn.csv, but the latter (simpler) movie names were listed in the file project_folder\log\video_info.csv
The issue was resolved by pasting the values of the movie names from Outliers_*.csv log files into the video_info.csv log file.

Sincerely,
Olexiy.

Installation of the version 1.1

Hello,

I wanted to upgrade to SimBA 1.1 (installation of the previous 1.0 version went smoothly).
Using Anaconda3 command prompt.

Followed the instructions on Github page:

git clone -b master https://github.com/sgoldenlab/simba.git
pip install -r simba/simba/requirements.txt

FFmpeg was already installed.

Then, in Anaconda3 run

python SimBA.py

SimBA GUI did not start, and instead resulted in the error message listed below.

Tried to go around by cloning the Tensorflow-free installation, repeating all the steps above and the following command:

git clone -b SimBA_no_TF https://github.com/sgoldenlab/simba.git

As a result, got the same error message.

Tried uninstalling Anaconda, then installed Anaconda & DeepLabCut newly, then repeated SimBA installation steps.... unfortunately, the same outcome.

Could you please help?

Thank you very much in advance.

Sincerely,
Olexiy.

Error messages in Anaconda prompt

(base) c:\Alex\simba\simba>python SimBA.py
Traceback (most recent call last):
File "SimBA.py", line 51, in
from ROI_freehand_draw_3 import roiFreehand
File "c:\Alex\simba\simba\ROI_freehand_draw_3.py", line 6, in
from shapely.geometry import Polygon
File "C:\anaconda3\lib\site-packages\shapely\geometry_init_.py", line 4, in

from .base import CAP_STYLE, JOIN_STYLE
File "C:\anaconda3\lib\site-packages\shapely\geometry\base.py", line 18, in
from shapely.coords import CoordinateSequence
File "C:\anaconda3\lib\site-packages\shapely\coords.py", line 8, in
from shapely.geos import lgeos
File "C:\anaconda3\lib\site-packages\shapely\geos.py", line 145, in
lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))
File "C:\anaconda3\lib\ctypes_init
.py", line 364, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found

(base) c:\Alex\simba\simba>

Desktop (please complete the following information):

  • OS: Windows
  • Browser Firefox

Additional context
Add any other context about the problem here.

Install Issue Shapely

After installing from the requirements.txt, getting the following Traceback:

Traceback (most recent call last):
File "SimBA.py", line 51, in
from ROI_freehand_draw_3 import roiFreehand
File "C:\Users\dlhag\simba\simba\ROI_freehand_draw_3.py", line 6, in
from shapely.geometry import Polygon
File "C:\Users\dlhag\Anaconda3\envs\simba\lib\site-packages\shapely\geometry_init_.py", line 4, in
from .base import CAP_STYLE, JOIN_STYLE
File "C:\Users\dlhag\Anaconda3\envs\simba\lib\site-packages\shapely\geometry\base.py", line 18, in
from shapely.coords import CoordinateSequence
File "C:\Users\dlhag\Anaconda3\envs\simba\lib\site-packages\shapely\coords.py", line 8, in
from shapely.geos import lgeos
File "C:\Users\dlhag\Anaconda3\envs\simba\lib\site-packages\shapely\geos.py", line 145, in
lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))
File "C:\Users\dlhag\Anaconda3\envs\simba\lib\ctypes_init
.py", line 348, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found

Tried a few shapely versions, couldn't get anything other than this.

Any thoughts?

RuntimeError: main thread is not in main loop

Describe the bug
When training a model using the app function "train single model from global environment" the following Error popt up. It seems to be connected to yellowbrick (so reporting?) but messes with the functionality. The model will not be saved (even though it was successfully trained).

This is the simba report window

grafik

This is the console until the error

grafik

This is the error message

Exception in thread Thread-182:
Traceback (most recent call last):
  File "D:\Anaconda\envs\simba-gpu\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "D:\Anaconda\envs\simba-gpu\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "simba.py", line 3726, in trainsinglemodel
    trainmodel2(self.projectconfigini)
  File "C:\Users\schwa\PycharmProjects\simba\simba\train_model_2.py", line 290, in trainmodel2
    generateClassificationReport(clf, class_names)
  File "C:\Users\schwa\PycharmProjects\simba\simba\train_model_2.py", line 68, in generateClassificationReport
    g = visualizer.poof(outpath=visualizerPath, clear_figure=True)
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\yellowbrick\base.py", line 215, in poof
    plt.savefig(outpath, **kwargs)
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\matplotlib\pyplot.py", line 690, in savefig
    fig.canvas.draw_idle()   # need this if 'transparent=True' to reset colors
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\matplotlib\backends\_backend_tk.py", line 350, in draw_idle
    self._idle_callback = self._tkcanvas.after_idle(idle_draw)
  File "D:\Anaconda\envs\simba-gpu\lib\tkinter\__init__.py", line 764, in after_idle
    return self.after('idle', func, *args)
  File "D:\Anaconda\envs\simba-gpu\lib\tkinter\__init__.py", line 756, in after
    name = self._register(callit)
  File "D:\Anaconda\envs\simba-gpu\lib\tkinter\__init__.py", line 1372, in _register
    self.tk.createcommand(name, f)
RuntimeError: main thread is not in main loop

Desktop (please complete the following information):
Simba GPU installation (Anaconda) on Windows 10

Additional context
A restart of the whole procedure went completely smooth without any errors. No restart necessary or anything. Just clicked the button again.

Issue during features extraction.

@sgoldenlab , Hello, I'm unfortunately dealing with a new error message.
I have been able to analyze the ROI and to obtain: number of entries, cumulative time, distance and velocity data.
Now I would like to draw heat maps. But I should first extract features right?

When I click Extract Features, it seems that the process stops, and here what happens:

Schermata 2020-07-24 alle 15 35 17

I then click on append ROI data to features, and here what appears:

Schermata 2020-07-24 alle 15 39 29

Have you any idea about the reason why it's not working?

As reminder, I'm working on multi animal project, on windows 7, avi videos, SimBA = 1.2.7. Concerning the outlier correction, I skipped it.
Thank you,
Best,
Dorian

Report: Upper Limit in Memory with pandas [MemoryError]

Disclaimer
This is not a SimBa code bug. But probably a pandas or system specific behavior. This is why I called this "Report". Please feel free to close the issue if you see fit.

Describe the bug
As with all machine learning problems, I thought the best way to solve accuracy problems is to throw data at it:

Trying out 10 csv files from my experiments (about 600k frames in total) resulted in this error popping up.
I included my system info to give you a better overview. As mentioned below this might happen from overcommitment protection within the OS.

[Parallel(n_jobs=-1)]: Done  26 tasks      | elapsed:   46.8s
Exception in thread Thread-20:
Traceback (most recent call last):
  File "D:\Anaconda\envs\simba-gpu\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "D:\Anaconda\envs\simba-gpu\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "simba.py", line 3764, in trainmultimodel
    train_multimodel(self.projectconfigini)
  File "C:\Users\schwa\PycharmProjects\simba\simba\train_multiple_models_from_meta.py", line 164, in train_multimodel
    features = baseFeatureFrame.copy()
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\pandas\core\generic.py", line 5996, in copy
    data = self._data.copy(deep=deep)
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\pandas\core\internals\managers.py", line 788, in copy
    return self.apply("copy", axes=new_axes, deep=deep, do_integrity_check=False)
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\pandas\core\internals\managers.py", line 438, in apply
    applied = getattr(b, f)(**kwargs)
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\pandas\core\internals\blocks.py", line 771, in copy
    values = values.copy()
MemoryError: Unable to allocate 3.22 GiB for an array with shape (787, 549864) and data type float64

Simba console output

grafik

What google says:
A quick google hints at a OS issue/safety mechanism for memory allocation (although the thread is originally for Linux). Stackoverflow. There are a few provided solutions (including for windows) but I have not tried them yet.

Desktop (please complete the following information):

Simba-GPU installation with Anaconda
Operating System: Windows 10 Home 64-bit (10.0, Build 18362) (18362.19h1_release.190318-1202)
System Manufacturer: LENOVO
System Model: 81UH
Processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (12 CPUs), ~2.6GHz
Memory: 16384MB RAM

Additional context
As I was writing this issue, I restarted the process and I was able to reproduce the same result. But to my confusion the training continued despite of the memory issue.

grafik

'Append ROI data to features' is not functioning

Describe the bug
'Append ROI data to features' is not functioning as expected

To Reproduce
Steps to reproduce the behavior:
After successful execution of 'Analyze ROI' and 'Visualize ROI' steps:

  1. Load the project
  2. Go to 'Extract features' tab, click on 'Extract features', wait until 'All feature extraction complete.' message
  3. Click 'Append ROI data to features'
  4. See error

Screenshots
Bug
Bug2

Desktop (please complete the following information):

  • OS: Windows 10
  • Browser: Chrome

Error in Outlier correction

Describe the bug
After starting the outlier correction with both parameters set to 0 (the plan was to skip the correction with this), i got the following error message:

Exception in Tkinter callback Traceback (most recent call last): File "D:\Anaconda\envs\simba-gpu\lib\tkinter\__init__.py", line 1705, in __call__ return self.func(*args) File "simba.py", line 3818, in correct_outlier dev_loc_user_defined(configini) File "C:\Users\schwa\PycharmProjects\simba\simba\outlier_scripts\location\correct_devs_loc_user_defined.py", line 100, in dev_loc_user_defined currentArray[currentPosition][0] = reliableCoordinates[currentPosition][0] IndexError: index 7 is out of bounds for axis 0 with size 7

The csv file was transfered/created in the correct folder i assume "project_folder\csv\outlier_corrected_movement" but there is no file in "outlier_corrected_movement_location".
I checked your code and i guess it did not go through as the error occurs in one of the essential scripts.

To Reproduce
My user defined settings are: 9 points , 1 Animal (directly from DLC analysis)
The csv file is ca. 55k rows long

I am using a Windows 10 computer and your newest simba version (updated at 10 am local time)

Additional context
I cannot continue with extraction of features however.
It results in a message within your app:
Pose-estimation body part setting for feature extraction: user_defined Extracting features from 0 files... All feature extraction complete.
which is probably due a non working import in your code:

def extract_features_wotarget(inifile): .... csv_dir_in = os.path.join(csv_dir, 'outlier_corrected_movement_location') ....

########### FIND CSV FILES ########### for i in os.listdir(csv_dir_in): if i.__contains__(".csv"): fname = os.path.join(csv_dir_in, i) filesFound.append(fname) print('Extracting features from ' + str(len(filesFound)) + ' files...')

I can however continue if i copy+paste the same csv file (from outlier_corrected_movement) into the other folder. I haven't read through all of your code, but i assume that you keep the general structure of the dataframe that comes from dlc so this can work as a workaround for now (especially since i did not want to do a outlier correction anyway).

Issue for ROI analyze.

Hello,

I'm using SimBA with data obtained from multi animal pose estimation from DLC.
I have been able to extract my frames, to scale the video and to draw the ROIs. But when I want to run the ROI analysis (2 animals, one body part), there is an error. Here a screen shot of the prompt:

Schermata 2020-07-23 alle 18 06 04

I'm analyzing 4 videos, and used the function "apply to all" after drawing the ROI of the first video. The first video seems ok, but the issue occurs at the second video:

Schermata 2020-07-23 alle 18 19 27

Someone would have a solution to help me fixing this issue?

FYI: I'm using SimBA 1.2.5 on windows 7. My videos are ".avi".
I already successfully run ROI analysis with the same program configuration (but never using "apply to all" function) with other videos. So I don't know why this time it is not working.

Thank you,
Best,
Dorian

portability

I try to run SimBA.py on OpenSUSE 15.1 x64 and ran into various portability errors. Mostly it's case-sensitivity (ie TheGoldenLab.PNG -> TheGoldenLab.png) and various instances of '\' in the code.
As both linux, mac and windows works fine with '/' as directory delimiter, I think it would be nice to change the \'s to /

Remove restriction on osf.io repository

Might be the wrong place, but it was the place I thought easiest to contact you and its semi a part of the software.

I suggest removing the current restriction on your osf.io repositories. They are current closed and you have to request access. It puts a unnecessary(?) delay on working with the software.

Conflicting Dependencies + Failure to Run

Hello! This is my first time writing an issue, so feel free to request lots of clarification.

I'm running the most recent version of Anaconda and attempting to install Simba. I've followed the instructions here for installing the standalone package w/o tf. I've been trying to resolve this for the past few hours to no avail.

Describe the bug
Simba will not run and has trouble installing.

To Reproduce

  1. Create environment with python 3.6: conda create --name new_env python==3.6
  2. Activate environment: conda activate new_env
  3. Within new_env, run pip install simba-uw-no-tf. This is using pip version 20.1.1. Everything appears to build correctly except with the following warning:
    ** ERROR: imbalanced-learn 0.7.0 has requirement scikit-learn>=0.23, but you'll have scikit-learn 0.22.2 which is incompatible.
  4. Ignoring that, run simba in new_env. This fails and produces a long traceback.
    **Traceback (most recent call last):
    File "c:\programdata\anaconda3\envs\new_env\lib\runpy.py", line 193, in _run_module_as_main "__main__", mod_spec)
    File "c:\programdata\anaconda3\envs\new_env\lib\runpy.py", line 85, in _run_code exec(code, run_globals)
    File "C:\ProgramData\Anaconda3\envs\new_env\Scripts\simba.exe\__main__.py", line 4, in <module>
    File "c:\programdata\anaconda3\envs\new_env\lib\site-packages\simba\SimBA.py", line 47, in <module> from simba.ROI_freehand_draw_3 import roiFreehand
    File "c:\programdata\anaconda3\envs\new_env\lib\site-packages\simba\ROI_freehand_draw_3.py", line 6, in <module> from shapely.geometry import Polygon
    File "c:\programdata\anaconda3\envs\new_env\lib\site-packages\shapely\geometry\__init__.py", line 4, in <module> from .base import CAP_STYLE, JOIN_STYLE
    File "c:\programdata\anaconda3\envs\new_env\lib\site-packages\shapely\geometry\base.py", line 18, in <module> from shapely.coords import CoordinateSequence
    File "c:\programdata\anaconda3\envs\new_env\lib\site-packages\shapely\coords.py", line 8, in <module> from shapely.geos import lgeos
    File "c:\programdata\anaconda3\envs\new_env\lib\site-packages\shapely\geos.py", line 145, in <module> _lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))
    File "c:\programdata\anaconda3\envs\new_env\lib\ctypes\__init__.py", line 348, in __init__ self._handle = _dlopen(self._name, mode)
    OSError: [WinError 126] The specified module could not be found

Expected behavior
Expect no conflicting dependencies and simba to start when simba is run in new_env

Desktop (please complete the following information):

  • OS: Windows 10
  • Python Version: 3.6.0 in new_env, 3.8 in base environ
  • Are you using anaconda? Yes

Things I have tried

  • making sure shapely is installed in new_env
  • same thing with Python 3.7 and 3.8 (has even more issues)
  • Interestingly, imbalanced-learn 0.7 has listed on their pypi site that it is compatible with scikit-learn versions >= 0.22...I tried installing scikit-learn version 0.23 anyways, which conflicts with simba but satisfies imbalanced-learn. Produces same error when trying to run simba.
  • same thing with imbalanced-learn 0.62 in the hopes it doesn't need scikit-learn 0.23... same error when trying to run simba

Any help/thoughts much appreciated!

Pose Configuration: Window sizing bug

Describe the bug
When importing a small image to create a pose configuration (96x161) the autosizing of the window is buggy (see image below).

Screenshots
grafik

Expected behavior
A clear and concise description of what you expected to happen.

Desktop (please complete the following information):

  • OS: Windows 10
  • Anaconda
  • simba_no_tf

Additional context
I guess you will run into this issue with apps if you autosize and go to the limits of tkinter and such. So this is more of a note, than a real big issue. But as the original video resolution was 848x480 (quite common for tracking) the cut out might be something that happens more than once :)

Simba installation issue on Win10

You might know about this bug, and I know I'm not installing simba as per the instructions (using a conda environment etc.), but it seems like it should be an acceptable way to install it.

Describe the bug
Installing simba on Windows 10 in a conda environment fails due to geos DLL import error.

To Reproduce
Steps to reproduce the behavior:

conda create --name simba_bug_test python=3.6
conda activate simba_bug_test
git clone -b master https://github.com/sgoldenlab/simba.git
pip install -r simba\simba\requirements.txt
conda install cudatoolkit=10.0 cudnn
cd simba\simba
python SimBA.py

gives:

  File "C:\Users\adam.\.conda\envs\simba_bug_test\lib\site-packages\shapely\coords.py", line 8, in <module>
    from shapely.geos import lgeos
  File "C:\Users\adam\.conda\envs\simba_bug_test\lib\site-packages\shapely\geos.py", line 145, in <module>
    _lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))
  File "C:\Users\adam\.conda\envs\simba_bug_test\lib\ctypes\__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found

If I try:

pip uninstall shapely
pip install shapely

Then the error persists. However, if I install shapely from conda forge conda install -c conda-forge shapely, then simba loads.

N.B. it's also a bit annoying that simba has to be run from simba\simba

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Python Version 3.6.10
  • Are you using anaconda? yep (well miniconda)

Labeling session reloading at wrong frame

After saving a labeling section and reopening at a later timepoint, SimBA keeps reopening at an older save. The file last_framelog remains unchanged as the following when saving at different timepoints:

[Last saved frames]' 'tracking_video2020-02-26t10_19_20 = 3736

Additionally, I'm not seeing any labels in the csv under targets_inserted, however, that might be because they are stored somewhere else?

Path not working on MacOS

Describe the bug
A clear and concise description of what the bug is.
Some of your scripts use backslashes for generating a path, which does not work on MacOS.
For example, in the create_project_init.py,
project_folder = str(directory +'\\' + project_name + '\\project_folder')
It would be better to use os.path.join()

The backslashes resulted in folders like this:
image

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Python Version [e.g. 3.6.0]
  • Are you using anaconda?

Additional context
Add any other context about the problem here.

Simba 1.2 Error annotating/labeling behaviors in larval zebrafish

SIMBA v1.2 error while trying to label behaviors in larval zebrafish (TL= 100px = 5mm) in a 30x30mm arena (1024x1024px videos). Seven body part labeled (two eyes, swim bladder, four along tail)

Successful steps:
Project ini file created and loaded,
20 Videos loaded top video folder,
DLC csv files imported to csv/input_csv/original filename folder
Video parameters adjusted
Outlier correction settings of 0.7 (movement) and 1.5 (location) attempted but main screen window/log says:
Processing 0 files for location outliers...
Log for corrected "location outliers" saved in project_folder/logs
Outlier correction complete.
ROIs drawn
Features (6000 png frames) extracted to 20 folders (one per video)

Error occurs when Labeling behaviors:
video frames loaded and displayed in GUI window,
rheotaxis behaviors check box selected,
frame navigation works,
frame range selected (e.g. 3280-5005) and click save and advance to next frame:
Annotated behavior: Rheotaxis. Start frame: 3280. End frame: 5005
Click “generate / save csv” button gives the following file not found error:

(SIMBA) C:\Users\Fish_Behavior>simba
warning: Error opening file (/build/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp:901)
warning: C:/Users/Fish_Behavior/Desktop/SIMBA/1LZF_model/1LZF_model_07_28/project_folder\videos\0 (/build/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp:902)
Exception in Tkinter callback
Traceback (most recent call last):
File "c:\users\fish_behavior.conda\envs\simba\lib\tkinter_init_.py", line 1705, in call
return self.func(*args)
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\simba\SimBA.py", line 3389, in
button_setscale = Button(label_setscale,text='Set video parameters',command=lambda:video_info_table(self.projectconfigini))
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\simba\SimBA.py", line 1050, in init
self.getdata()
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\simba\SimBA.py", line 1100, in getdata
self.button = Button_getcoord(self.xscrollbar, self.data_lists[1], self.data_lists[5],self.pixel_list)
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\simba\SimBA.py", line 877, in init
self.ppmvar[i].set(ppmlist[i])
IndexError: list index out of range
Exception in Tkinter callback
Traceback (most recent call last):
File "c:\users\fish_behavior.conda\envs\simba\lib\tkinter_init_.py", line 1705, in call
return self.func(*args)
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\simba\labelling_aggression.py", line 135, in
self.generate = Button(self.window, text="Generate / Save csv", command=lambda: save_video(self.window))
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\simba\labelling_aggression.py", line 474, in save_video
data = pd.read_csv(input_file)
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\pandas\io\parsers.py", line 685, in parser_f
return _read(filepath_or_buffer, kwds)
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\pandas\io\parsers.py", line 457, in _read
parser = TextFileReader(fp_or_buf, **kwds)
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\pandas\io\parsers.py", line 895, in init
self._make_engine(self.engine)
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\pandas\io\parsers.py", line 1135, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "c:\users\fish_behavior.conda\envs\simba\lib\site-packages\pandas\io\parsers.py", line 1917, in init
self._reader = parsers.TextReader(src, **kwds)
File "pandas/_libs/parsers.pyx", line 382, in pandas._libs.parsers.TextReader.cinit
File "pandas/_libs/parsers.pyx", line 689, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: [Errno 2] File b'C:\Users\Fish_Behavior\Desktop\SIMBA\1LZF_model\1LZF_model_07_28\project_folder\csv\features_extracted\\20200318_AB_7dpf_ctl.csv' does not exist: b'C:\Users\Fish_Behavior\Desktop\SIMBA\1LZF_model\1LZF_model_07_28\project_folder\csv\features_extracted\\20200318_AB_7dpf_ctl.csv'

A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Python Version [e.g. 3.6.0]
  • Are you using anaconda?

Additional context
Add any other context about the problem here.

Issue with graphviz not being recognized

Hi again, @sgoldenlab, I had hoped it would be longer before I pestered you again!

The "Train Machine Model" step ran well for a few minutes before running into the below error:
Screen Shot 2020-07-22 at 5 13 08 PM

Thoughts on how to circumvent this? I did a bit of looking around other forums (not related to SimBA) and found a suggestion that I add Graphviz to the Windows path. Thanks in advance for your help!

Feature: Validate Settings Input before training start

Is your feature request related to a problem? Please describe.
The Settings for training a machine model in SimBa have multiple options that present a predefined Value (e.g. NaN) when they are toggled. If you accept invalid settings (such as NaN) there is no feedback for the user but training the model will result in an error (often at the end of training).
See below for an example:

[Parallel(n_jobs=-1)]: Using backend ThreadingBackend with 12 concurrent workers.
[Parallel(n_jobs=-1)]: Done  26 tasks      | elapsed:    4.9s
[Parallel(n_jobs=-1)]: Done 176 tasks      | elapsed:   26.0s
[Parallel(n_jobs=-1)]: Done 426 tasks      | elapsed:  1.0min
[Parallel(n_jobs=-1)]: Done 776 tasks      | elapsed:  1.9min
[Parallel(n_jobs=-1)]: Done 1226 tasks      | elapsed:  3.0min
[Parallel(n_jobs=-1)]: Done 1776 tasks      | elapsed:  4.4min
[Parallel(n_jobs=-1)]: Done 2000 out of 2000 | elapsed:  5.0min finished
Exception in thread Thread-1:
Traceback (most recent call last):
  File "D:\Anaconda\envs\simba-gpu\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "D:\Anaconda\envs\simba-gpu\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "simba.py", line 3737, in trainsinglemodel
    trainmodel2(self.projectconfigini)
  File "C:\Users\schwa\PycharmProjects\simba\simba\train_model_2.py", line 259, in trainmodel2
    shuffle_splits = config.getint('create ensemble settings', 'LearningCurve_shuffle_k_splits')
  File "D:\Anaconda\envs\simba-gpu\lib\configparser.py", line 818, in getint
    fallback=fallback, **kwargs)
  File "D:\Anaconda\envs\simba-gpu\lib\configparser.py", line 808, in _get_conv
    **kwargs)
  File "D:\Anaconda\envs\simba-gpu\lib\configparser.py", line 802, in _get
    return conv(self.get(section, option, **kwargs))
ValueError: invalid literal for int() with base 10: 'NaN'

I assumed that there was a default value that would be taken if i leave the textbox 'NaN'.

Describe the solution you'd like
This is a user created issue (and i won't do it again), but it might be worthwhile to either validate the input when saving the settings (e.g. comparing if it is 'int', 'float', etc.). Then if it is invalid, setting the value to default (while telling the user) or making the user choose again. This would save some users (like me) from their own ignorance and others from wasting time training a machine model that will ultimately not be saved, because the validation process causes an error and everything is gone.

Describe alternatives you've considered
An alternative would be to validate the settings before starting the training. Simply throwing out an error (e.g. "ValueError: Some of your parameters are invalid. Consider reading the readme files again.")

Another alternative would be to check user input when pressing the save button. Similiar to a contact formular on a website. I guess this would take much more time (as it's related to gui programming) though.

Additional context
I would not document this kind of request if your overall goal wasn't to create a tool that can be used without any (or without a lot of) programming experience. SimBa is already simplifying a complicated process so much and it is doing great work!

Error in Train model

Describe the bug
After getting the error in issue #22 , I toggled both boxes (see below) to circumvent the error #22 .
The following error occured after initializing training (or rather before it). I checked the code and it's the exception for an error in training (fitting), so could be anything? I checked both csv files if they contained the array and if it was valid (see further below).

simba/train_model_2.py:237

        try:
           clf.fit(data_train, target_train)
        except ValueError:
           print('ERROR: The model contains a faulty array. This may happen when trying to train a model with 0 examples of the behavior of interest')

I tried both train methods using global environment for 1 model and specific for 3 seperate models (as visible in the SimBa log). The console output is the same, so i didn't post the second.
Removing both boxes (Generate Features Importance Log and Graph) does not change the error (tested in the global settings method).

Settings:

grafik

SimBa - Log:

grafik

Console output:

(simba-gpu) C:\Users\schwa\PycharmProjects\simba\simba>python simba.py
Exception in thread Thread-1:
Traceback (most recent call last):
  File "D:\Anaconda\envs\simba-gpu\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "D:\Anaconda\envs\simba-gpu\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "simba.py", line 3734, in trainmultimodel
    train_multimodel(self.projectconfigini)
  File "C:\Users\schwa\PycharmProjects\simba\simba\train_multiple_models_from_meta.py", line 263, in train_multimodel
    generateClassificationReport(clf, class_names, classifierName, saveFileNo)
  File "C:\Users\schwa\PycharmProjects\simba\simba\train_multiple_models_from_meta.py", line 51, in generateClassificationReport
    visualizer.score(data_test, target_test)
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\yellowbrick\classifier\classification_report.py", line 130, in score
    y_pred = self.predict(X)
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\sklearn\ensemble\_forest.py", line 612, in predict
    proba = self.predict_proba(X)
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\sklearn\ensemble\_forest.py", line 654, in predict_proba
    check_is_fitted(self)
  File "D:\Anaconda\envs\simba-gpu\lib\site-packages\sklearn\utils\validation.py", line 967, in check_is_fitted
    raise NotFittedError(msg % {'name': type(estimator).__name__})
sklearn.exceptions.NotFittedError: This RandomForestClassifier instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.

pd.Dataframe.describe() and np.sum() for both target_inserted csv files over all labelled frames (model names: 'active', 'resting' and 'inner_circle')

Dataframe 1

Dataframe.describe()
            Active       Resting  Inner_circle
count  54892.000000  54892.000000  54892.000000
mean       0.750000      0.250000      0.394065
std        0.433017      0.433017      0.488653
min        0.000000      0.000000      0.000000
25%        0.750000      0.000000      0.000000
50%        1.000000      0.000000      0.000000
75%        1.000000      0.250000      1.000000
max        1.000000      1.000000      1.000000

np.sum()
Active          41169.0
Resting         13723.0
Inner_circle    21631.0

Dataframe 2

Dataframe.describe()
             Active       Resting  Inner_circle
count  54892.000000  54892.000000  54892.000000
mean       0.750000      0.250000      0.394065
std        0.433017      0.433017      0.488653
min        0.000000      0.000000      0.000000
25%        0.750000      0.000000      0.000000
50%        1.000000      0.000000      0.000000
75%        1.000000      0.250000      1.000000
max        1.000000      1.000000      1.000000

np.sum()
Active          41169.0
Resting         13723.0
Inner_circle    21631.0
dtype: float64

As you can see both Dataframes (csv files) have labelled frames (with both True/1 and False/0 values). As I did the exact same thing, but with 1 csv file less, just yesterday. I am confused.

Additional context
The additional csv file was added using the provided functions. It's noteworthy that I did not label the csv file manually but added columns to the outlier corrected version via some automatic labelling method. This method worked before.

Desktop (please complete the following information):

  • OS: Windows 10
  • SimBa-GPU install (newest version 07.04.2020) with Anaconda

Issue when uploading bx.h5 files.

Hello,

I am working on a maDLC project, and I have an error message when I try to import my h5 files on Simba.

Here the screenshot:
Untitled

Would someone have a suggestion to fix this problem?

FYI: I labelled the body parts through DLC 2.b.5 on windows 7, then run the training and analysis on colab, modifying the first cell code to use DLC 2.b.7, and now I tried to switch on Simba on two different windows computer, one with windows7 the other with windows 10. Nothing works, always the same error.

Thank you for the help,
Best,
Dorian

Issue when importing DLC .H5 file

Hello,

I'm attempting to create a new project using multi-animal pose estimation data from DLC. However, I am unable to import the tracking data in .H5 file format from DeepLabCut.

First I generated the Project Config, defined the tracking as multi-animal and imported the raw videos used to train the network on DLC. Yet, when i try to import the tracking data in .H5 file format from DeepLabCut from the DLC output folder "\training-datasets\iteration-0\UnaugmentedDataSet", with the .H5 file, I receive the following message on SimBA:
"Importing 0 multi-animal DLC h5 files to teh current project. All multi-animal DLC .h5 tracking files ordered and imported into SimBA project in CSV file format".

How can i solve these problems?

Thank you.

Feature: Additional models after Project Creation

Disclaimer: This is a feature that mainly is directed at easy use and therefore is only meant as a suggestion.

I am currently trying to merge different automatic labelling approaches into one project and noticed that there is no option to add additional models after the initial project creation. This is probably connected to the consistency of the feature extraction tool, but could be easily implemented in a workaround solution (like the one i do).
Describe the solution you'd like
So far i saw the following necessary changes to add a model into an existing project:
in the project_config file add a new line in the following sections and increase the number of targets:

[SML settings]
model_dir = D:/SimBa\MovementDetector\models
model_path_1 = D:/SimBa\ProjectName\models\Existing_model1.sav
model_path_2 = D:/SimBa\ProjectName\models\Existing_model2.sav
model_path_3 = D:/SimBa\ProjectName\models\New_model.sav
no_targets = 3
target_name_1 = Existing_model1
target_name_2 = Existing_model2
target_name_3 = New_model

[threshold_settings]
threshold_1 =
threshold_2 =
threshold_3 =

[Minimum_bout_lengths]
min_bout_1 =
min_bout_2 =
min_bout_3 =

This addition to project_config file could be easily done via the app (i think) and would allow non advanced users to change the number of existing models without the need to create a new project.

With these changes and an external addition of extracted features to the necessary csv file i was able to create a new model and train it within the simba app.

Additional context
I have not tested if the feature extraction is working with all 3 models (Old_model1, old_model2, New_Model) if there are already feature extractions for the old models. You can probably elaborate if you are overwriting the feature extracted csv file or add columns to it.

Edit:
spelling

Issue when extracting features

Hello everybody!

I am attempting to extract the features, but I run into this error message
image

I created an environment containing only python 3.6.0, then (inside the environment) I pip install simba-uw-no-tf, then conda install shapely, and conda install h5py. SimBA launches, and all the steps went fine until the extract features bit, where I get that error message. Any idea how to solve this? I am using the 1 animal 8bp config

Unable to open simba gui

I have install Simba using pip install simba-uw-no-tf, but when I tried launching Simba, I got the traceback below. I know this it is successfully installed because I opened the directory in my terminal (also shown below). I am using a Linux virtual machine is that changes anything.
image
image

Issue when loading .h5 files. Fuzzy instructions.

Hello,

I'm using SimBA for the first time, and I have an issue just after clicking "import h5". The instructions are fuzzy and I cannot read what is asked. Here a picture:

Untitled

I tried to recreate a new project several times, but I obtain systematically the same issue. would you have some suggestion to fix that bug?

FYI: I am using windows 10, and my training has been done on maDeeplabcut 2.2.b6 with 8bp. I worked on mov format for dlc analysis, and then I converted them in mp4 to switch on SimBA.

Thank you for the help,
Best,
Dorian

Error when extecuting SimBA.py

When executing SimBA.py I get the following error:

My input:
C:\ProgramData\simba\simba>python SimBA.py
Error output:

C:\ProgramData\simba\simba>python SimBA.py
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
ImportError: numpy.core.multiarray failed to import

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 960, in _find_and_load
SystemError: <class '_frozen_importlib._ModuleLockManager'> returned a result with an error set
ImportError: numpy.core._multiarray_umath failed to import
ImportError: numpy.core.umath failed to import
2020-04-27 14:43:13.667999: F tensorflow/python/lib/core/bfloat16.cc:675] Check failed: PyBfloat16_Type.tp_base != nullptr

Passing:

python
import numpy as np
np.__version__

returns 1.14.5, which is what is specified in the requirements.txt.

Impossible to run simba on Mac os

Hello,

Thank you for the new release of Simba.
I tried to install it on my mac using "pip install simba-uw-no-tf" command.

During the installation I noticed a bug: whereas this command should specify for intallation without TensorFlow, an error message appeared:

"Could not find a version that satisfies the requirement tensorflow_gpu==1.14.0..."

I added '#' before the tensorflow line in requirments, and that fixed the issue.

Nevertheless, I can not launch the program from my terminal.
Once cd in my simba folder, I tried both: "simba.py" and "pythonw simba.py" commands.

Here the error message returned:

b95a3806-9308-4a56-b941-532f8d33f5c8

FYI: I'm using python 3.7.6, with anaconda. I'm also using DeepLabCut 2.2.b5. Finally one friend tried also to install Simba on her Macos, and she had the same error.

Have you some suggestion to fix the bug?

Thank you very much!
Best,
Dorian

ROI module not working

Describe the bug
ROI module is not working. After selecting the ROI's, the output CSV file appears to be empty.

To Reproduce
Steps to reproduce the behavior:
After loading the project

  1. Go to 'ROI' tab
  2. Define ROI's and click 'Show Shape Definitions table'
  3. Enter shape names, leave dimensions 0, click on Set Shape Definitions
  4. There's only one video, click on draw and draw shapes according to instructions
  5. Click on 'Analyse ROI Data'

Expected behavior
A CSV file with analyzed ROI's

Screenshots
bug1
bug2
bug3

Desktop (please complete the following information):

  • OS: Windows 10
  • Browser: Chrome

Additional context

  1. Outlier correction step was done successfully and non-empty .csv files are available at project_folder\csv\outlier_corrected_movement_location and project_folder\csv\outlier_corrected_movement
  2. Video name in the file project_folder\logs\video_info.csv was changed due to incompatibility of Simba with Deeplabcut while importing videos into Simba ( #8 )
  3. All .csv files highlighted in the screenshot are empty.

Problem with Analyse ROI Data

Hello everybody!

First of all, thank you so much for implementing the new features on simba! I can also see that you have added a lot of new (very clear!) documentation, and I really appreciate your effort, as I am sure it took a lot of hours. Thank you so much!

That being said, I am having some issues running the analysis of the ROI Data. All of the steps ran smoothly until then, but when I click the Analyse ROI Data button, the cmd terminal displays this:
Screen Shot 2020-04-25 at 22 12 05
The steps before this ran correctly, I believe, as an ROI_definitions h5 file was created
Screen Shot 2020-04-25 at 22 09 20

Any idea how to solve this? I am using Python 3.6.4

Automatically track frames annotated for each labeling session

Rather than having the user supply the exact frame range labeled in the session to avoid erasing all previous labeling when saving (ouch I lost a lot of hours), I suggest tracking the frames actively labeled for each session, and then overwriting only those.

name 'deeplabcut' is not defined

Describe the bug When I run simba and error about dlc appears.
(deeplabcut-py36) D:\Users\Administrator\Documents>simba
Exception in Tkinter callback
Traceback (most recent call last):
File "c:\programdata\anaconda3\envs\deeplabcut-py36\lib\tkinter_init_.py", line 1705, in call
return self.func(*args)
File "c:\programdata\anaconda3\envs\deeplabcut-py36\lib\site-packages\simba\SimBA.py", line 1904, in
button_generatetempyaml_single = Button(label_tempyamlsingle,text='Add single video',command=lambda:generatetempyaml(self.label_set_configpath.file_path,self.label_genyamlsinglevideo.file_path))
File "c:\programdata\anaconda3\envs\deeplabcut-py36\lib\site-packages\simba\dlc_change_yamlfile.py", line 15, in generatetempyaml
deeplabcut.add_new_videos(tempyaml,[videolist],copy_videos=True)
NameError: name 'deeplabcut' is not defined

Desktop (please complete the following information):

  • windows 10
  • Python Version [ 3.6.0]
  • Are you using anaconda? yes
    intalled with SimBAxTF.

Do you have any ideas? Thanks!

Simba Create Project

Hi

I installed the Simba (pip install simba-uw-no-tf) in my local. then i launched "simba". I am trying to create a project, after filling all fields the following error came out:

Traceback (most recent call last):
File "C:\users...\appdata\local\programspython\python37\lib\thinter_init_.py", line 1705, in call
return self.func(*args)
File "C:\users...\appdata\local\programs\python\python37\lib\site-packages\simba\simba.py", line 3056, in make_projectini
with open(no animalsPath, "r", encoding='utf8') as f:
FileNotFoundError: [errno 2] No such file or directory: 'C:\users\...\pose_configurations\no_animals\no_animals.csv

In fact the pose configurations folder is empty.
What should I do?

Running Simba as singularity container on HPC cluster

Hello, we are trying to run Simba as a singularity container on our cluster, the pip install fails (wxpython 4.0.1 fails to build because of older version of libgtk), we don't have root privileges on the machine so we decided to use singularity container instead.

The official 'pip3 install simba-uw-tf' ran fine and the binary was built but when we try to launch Simba get get the error(s):

` ImportError: Cannot load backend 'TkAgg' which requires the 'tk' interactive framework, as 'headless' is currently running

Singularity simba.sif:~/scratch> simba
Traceback (most recent call last):
File "/usr/local/bin/simba", line 7, in
from simba.SimBA import main
File "/usr/local/lib/python3.6/dist-packages/simba/SimBA.py", line 94, in
from simba.dpk_script.train_model import trainDPKmodel
File "/usr/local/lib/python3.6/dist-packages/simba/dpk_script/train_model.py", line 6, in
np.random.bit_generator = np.random._bit_generator
AttributeError: module 'numpy.random' has no attribute '_bit_generator'

Here is the singularity recipe file we used:

Bootstrap: docker
From: ubuntu:18.04

%post
apt-get -y update && DEBIAN_FRONTEND=noninteractive
apt-get install python3-pip -y
apt-get install git -y
DEBIAN_FRONTEND="noninteractive" apt-get install tzdata python3-tk -y
apt-get install make gcc libgtk-3-dev libwebkitgtk-dev libwebkitgtk-3.0-dev libgstreamer-gl1.0-0 freeglut3 freeglut3-dev python-gst-1.0 python3-gst-1.0 libglib2.0-dev libgstreamer-plugins-base1.0-dev -y
pip3 install simba-uw-tf
pip3 install tk

Would it be possible to build a docker container for Simba, or any tips?

Pointer to folder with dlc files

Hi, I'm using SimBA on some DLC output files, which I had to move to a different drive post-labelling. Now I'm not able to train the network.

Is there a config file with the project pointing to the DLC files I can access somewhere to edit to the new directory?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.