de-bivort-lab/margo

High-throughput real-time tracker for delivery of stimuli and measurement of animal behavior.

MARGO

Description

The Massively Automated Real-time GUI for Object-tracking (MARGO) is a MATLAB based tracking platform designed with an emphasis on high-throughput tracking of large groups of animals and tracking applications requiring closed-loop hardware control. See below for examples of MARGO applications.

Sample video clip from an experiment featuring continuous high-throughput tracking of 960 fruit flies tracked at 8Hz for 6 days

Installation

Prerequisites

MATLAB

For best results, use MARGO with MATLAB 2016b or newer. MARGO has generally been designed to be backwards compatible with older versions of MATLAB. In addition to the base installation of MATLAB, MARGO requires the following toolboxes:

• image acquisition toolbox
• image processing toolbox
• computer vision system toolbox
• instrument control toolbox
• statistics and machine learning toolbox

Psychtoolbox (optional)

MARGO requires on Psychtoolbox 3 for support of external displays.

MARGO installation

1. The MARGO repository can be cloned via the github UI by downloading and extracting a zip file of the repository (Clone or Download > Download ZIP)   OR  clone via the git command line API with the following command:

git clone https://github.com/de-Bivort-Lab/margo.git

2. After cloning the repository, add the MARGO directory to MATLAB's path by navigating to the margo directory and running:

addpath(genpath(pwd));

OR

Alternatively, permanently add MARGO and all sub folders to the MATLAB path by running:

    pathtool

3. Once the margo directory is added to the MATLAB path, launch the GUI from the command line:

margo

Getting Started

We recommend that new users read the overview of MARGO's functionality and use the sample video included in this repository to follow the tracking tutorial included in the documentation for more complete instructions on getting started in MARGO.

Documentation

Complete documentation of MARGO including tutorial examples, descriptions of parameters, data outputs, and hardware configurations can be found on the MARGO wiki.

Sample Applications

Closed-loop control of stimuli

   

Closed-loop applications: (left) triggering LEDs based on position of flies in a Y-shaped mazes, (right) targeting optomotor stimuli to individual flies in circular arenas

Multi-species tracking

The Massively Automated Real-time GUI for Object-tracking (MARGO) is a MATLAB based tracking platform designed with an emphasis on high-throughput tracking of large groups of animals and tracking applications requiring closed-loop stimulus control.

   

Sample tracking of bumblebees (left) in a nestbox and larval zebrafish (right) in a multi-well culture plate

   

Tracking of fruit fly larvae (left) in a chemotactic gradient and nematodes (right) in response to an optogenetic stimulus in the wormotel high-throughput platform

Reporting issues

Although the core functionality has been thoroughly tested, MARGO is still a work in progress. Please report any errors to this repository. To help solve issues quickly, please provide a detailed description of the error, the MATLAB error message, and the MARGO error log file (if possible).

Contributors

• Zach Werkhoven primarily developed and maintains MARGO.
• Chuan Qin contributed to the development of MARGO's multitracker algorithm.
• Christian Rohrsen contributed to the development of MARGO's camera/display co-registration system.

Acknowledgements

Support for external display detection and visual stimulus crafting and display is dependent on the Psychtoolbox, developed by Mario Kleiner et al. MARGO's random dot display registration uses Andriy Myronenko's Coherent Point Drift algorithm and mex implementation. Multispecies tracking example videos provided by: James Crall (bumblebees), Jess Kanwal (fly larvae), and Matt Churgin (C. Elegans).

License

MIT License. See LICENSE for details.

margo's Issues

larval_sleep_deprivation subgui load error

When pressing experiment parameters for larval sleep deprivation

No appropriate method, property, or field 'Position' for class 'matlab.graphics.GraphicsPlaceholder'.

Error in sleepdeprivation_parameter_sub_gui>slowphototaxis_parameter_gui_OpeningFcn (line 102)
handles.figure1.Position(1) = gui_fig.Position(1) + ...

Optional output data fields disabled when post-processing is disabled

Various Issues for Proj. Rig 1

Opposed to the recent batch of errors, Projector Rig 1 throws multiple that are different. Here is a log of two separate instances of running Margo today, 2/9/24
error_log_Proj_Rig1_02_09_24.txt

Errors trying to open MARGO

Hi there,
I'm using Matlab 2016b with the bug updates to try and open MARGO, but keep getting some errors. Would hugely appreciate any help on this topic- I got it going once before but it won't seem to work anymore :( I've attached the error code here:

ls

. .gitignore README.md data classes experiments hardware tracking
.. LICENSE.md analysis and plots examples gui profiles utilities

addpath(genpath(pwd));
margo
Error using serial/fwrite (line 199)
Unsuccessful write: A timeout or Ctrl-C occurred during the write operation.

Error in writeInfraredWhitePanel (line 25)
fwrite(COM_obj,writeData,'uchar');

Error in refreshCOM (line 54)
writeInfraredWhitePanel(expmt.hardware.COM.light,1,...

Error in margo>margo_OpeningFcn (line 118)
[expmt, handles] = refreshCOM(expmt, handles);

Error in gui_mainfcn (line 220)
feval(gui_State.gui_OpeningFcn, gui_hFigure, [], guidata(gui_hFigure), varargin{:});

Error in margo (line 48)
gui_mainfcn(gui_State, varargin{:});

Referencing periodic cycling to inaccurate reference.

This is occurring specifically in the noisy behavior room. Likely cause is refRawCrossPatch.m

display projector preview - set default to unchecked

COM port status during tracking initialization

COM object has no field "status" after device is disconnected. Re-check that devices exist before initialization.

Camera disconnected

Margo stopped experiment with 3:03:01 left in a 24h long experiment (computer 3). The camera was connected...I restarted margo and played with the camera settings, we'll see if it errors out again.

Referencing bugs

Bugs/errors occurring during referencing

plotTraces scale

plotTraces function scales the traces to the min/max of the traces rather than the ROI itself. This can create poorly scaled plots for animals with little activity. Recommend scaling to ROI and adding background image of ROI to place traces in their context within the image.

trackDat ROI# not edited after manual ROI editing

expmt raw data field not attaching on load

Raw data fields are not attaching on loading into the MATLAB workspace and do not prompt any warning or error.

Recurrent error for initialize tracking

When pressing "initialize tracking" dialogue box opens up often when running Margo across different computers. Dialogue may not always occur, and then the experiment can proceed. Matlab "red" code related to this issue is attached
margo_error.txt

logical precision data writing as uint8 precision

Use Saved ROI coordinates to define new ROI

I was wondering if there is functionality to use the ROI coordinates saved or copied from a user-defined grid ROI to position a new ROI on a new video. Essentially can an exact copy of a previosu ROI be imported for use on a new video? Thanks

COM selection error

When selecting "No COM detected":

Brace indexing is not supported for variables of this type.

Error in margo>microcontroller_popupmenu_Callback (line 539)
expmt.hardware.COM.light = serial(hObject.String{hObject.Value});

ExperimentData file path repair string incompatibility

Automatic file path repair in the ExperimentData object is not compatible with MATLAB strings (introduced in 2016b)

The tracking target is lost when it moves rapidly

Our laboratory used Margo to track zebrafish larvae, but zebrafish moved too fast, resulting in the target being often lost.So what is the solution? Thank you!

UI Occlusion - manually edit ROIs

Recommend launching an additional window for editing ROIs.

Raw data map returns error when indexing without being attached

The appropriate behavior is to prompt an error that says the map is not attached, and offer the user instructions on how to attach the map.

Changing camera settings

This error occurred when trying to change the brightness in camera settings while previewing. Branch: master_beta

Problem with Margo CPU usage when live-tracking

Previously, we had been using Margo without any issues.
Recently however, when we try to do live tracking in Margo, our total processor utilisation across all cores hits 100% and the software fails to work properly. We don't believe it is a hardware issue.
We have noticed that the issue does not happen when we are tracking from a pre-recorded video, only when live tracking.

Test implementation for luminance stabilization

suggest ID label range once ROIs are specified

Error after loading settings from LED ymaze profile and switching to Basic Tracking

isnan issue when labelling subsets of ROIs

de Bivort lab projector rig 1, MATLAB R2023a, latest version of margo

We set up a ymaze experiment and input the following labels:

Strain	Sex	Treatment	ROI start	ROI End	ID Start	ID End	Day #	Box #	Tray #	Comments
s1	f	t1	11	40	1	30	1	1	10
s1	m	t1	47	51	31	35	1	1	10

This resulted in the following "Error encountered - tracking stopped" popup:

Undefined function 'isnan' for input arguments of type 'cell'.
Error in autoInitialize (line 119)
    if ~isnan(expmt.meta.labels_table{1,i})

Error in run_ymaze (line 36)
[trackDat,expmt] = autoInitialize(trackDat,expmt,gui_handles);

Error in margo>run_pushbutton_Callback (line 770)
    expmt = feval(handlesexperiments(exp_idx).run, expmt, handles);

Error in gui_mainfcn (line 95)
    feval(varargin{:});

Error in margo (line 48)
    gui_mainfcn(gui_State, varargin{:});

Error in matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)margo('run_pushbutton_Callback',hObject,eventdata,guidata(hObject))

The error in the MATLAB command window:

Incorrect number or types of inputs or outputs for function 'isnan'.

Error in autoInitialize (line 119)
            if ~isnan(expmt.meta.labels_table{1,i})

Error in run_ymaze (line 36)
[trackDat,expmt] = autoInitialize(trackDat,expmt,gui_handles);

Error in margo>run_pushbutton_Callback (line 770)
    expmt = feval(handles.experiments(exp_idx).run, expmt, handles);

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in margo (line 48)
    gui_mainfcn(gui_State, varargin{:});

Error in matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)margo('run_pushbutton_Callback',hObject,eventdata,guidata(hObject))
 
Error while evaluating UIControl Callback.

When putting in the following ROI sets in the labels, there was no issue starting up tracking:

Strain	Sex	Treatment	ROI start	ROI End	ID Start	ID End	Day #	Box #	Tray #	Comments
s1	f	t1	11	40	1	30	1	1	10
s1	m	t1	1	10	31	40	1	1	10

Test Bug Report

DESCRIPTION

I clicked on the preview camera button and then clicked on auto-detect-ROIs, then it spit out this error!

ERROR MSG

Invalid or deleted object.

Error in autoROIs (line 149)
imh.CData = binaryimage;

Error in autotracker>auto_detect_ROIs_pushbutton_Callback (line 1559)
expmt = autoROIs(handles, expmt);

Error in gui_mainfcn (line 95)
feval(varargin{:});

Error in autotracker (line 42)
gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)autotracker('auto_detect_ROIs_pushbutton_Callback',hObject,eventdata,guidata(hObject))
Error while evaluating UIControl Callback

Output index for video sub-sampling

One failed file path repair results in no file paths being updated on expmt load

Error opening older-version files

I have been encountering a weird error with the newest MARGO version when trying to read in .raw files. Specifically, the error occurs when trying to assign an already attached .raw(). There is an orange notice referring to a path mismatch/repair when loading in the expmt.mat. When manually attaching the object, the raw mat shape does change to reflect the underlying structure (i.e., frames on one axis) but it still errors out when trying to assign/access the .raw().

Circadian ramp count out of bounds

Display none path

Failed to clear the Display none gui axes notification after ending an experiment early with the delete function.

Video Recording sub-sampling

GUI bug

Project targeting preview

Auto-save expmt meta data during error

Update the meta data during an error and output an error report.

addROI increment error

addROI does not add roi.pixIdx to the new ROI which can create errors when adding and deleting multiple ROIs.

Adjust default minimum trace duration to scale relative to the target rate

Failure to save backup of ExperimentData after error before initialization

Reason is filepath is unset.

Box 4 Camera Settings do not retain themselves

Box 4 of our PC's does not keep camera settings when you manually update them. Error log below appeared once, but not consistently (even though error is consistent over multiple attempts). Camera settings revert to a over-exposed default where flies cannot be tracked well
error_log_box4_02_09_24.txt

False warning for failed file path repair on loading expmt from external HDD

Warning was printed even though the file paths were repaired correctly.

Warning: File path change detected, attempting to repair path meta data for raw data files Unsuccessful repair will result in non-functional raw data maps

Reference mode controls

Add a separate control panel for different modes of reference calculation and other referencing parameters.

Circadian Lights ON parameter

The circadian Lights ON parameter parameter sometimes does not receive a minute assignment. Possibly related to time strings in the format H:MM instead of HH:MM

Can a MatLab executable be made from Margo?

I am not a MATLAB expert, and am trying to advise colleagues as to whether this is possible and seek your insights and corrections to my understanding of the issues in making a standalone MATLAB executable from Margo.

To answer this, we will need to refer in part to The following table shows the MATLAB toolboxes that you can use with MATLAB Compiler and describes which functionality can and cannot be compiled where we learn that, for the most part, only command line functionality of the toolboxes can be compiled into standalones (notably NOT UIs and Apps but also other aspects of selected toolboxes). Does Margo transgress any of these limitations (e.g. does it use UIs/Apps from the required toolboxes)? I further expect that your preprint's observation that "Software for control of all custom electronic hardware was written in C using Arduino libraries" will bear on what is possible in this regard. Do you agree? Finally, we only have an old compiler license - do you agree this will limit us (especially given the answer to this question: Is it possible the use of a MCR version different from the utilized version of Matlab Compiler? ).

Thanks for your consideration.

blob size/ target rate controls disabled after loading profile

Blob size/ target rate controls did not properly enable following ROI detection after loading parameters from profile.

Manually edit ROIs - subtract and add lacking expmt inputs

The addROI and subtractROI make a reference to expmt but the input is not provided.

missing vignette image after tracking session

After running initializeRef after first completing a tracking session.

Problem initializing reference

Had an issue, but it turned out to be the tracking settings in the options that changed.

LED Y-maze Serial write timeout

Occurred approximately 1hr into the experiment. Probably not a software issue. Most likely the hardware.

Camera selection error

When selected "No Camera Detected" from dropdown:

Index exceeds the number of array elements (0).

Error in margo>cam_select_popupmenu_Callback (line 169)
if ~isempty(handles.cam_list(get(hObject,'value')).adaptor)