benlyyan/mgt

Important:Since many problems in Matlab2016,I suggest if anybody is still using Matlab2015a, please test it in Matlab2015a. Then I will make a new version for Matlab2016a.

MGT is a music gestures toolbox which is useful to analyze video based music-related body motion. It has been developed in Matlab. Current version requires Matlab 2015, and helpful toolboxes like matlabPyrtools, mocaptoolbox, mirtoolbox. This repository includes data-set, example, source-code, MGT.zip, mirtoolbox.zip, mocaptoolbox.zip. User can choose to download .zip.

1. In the data-set folder, the pianist data set contains the pianist.tsv, pianist.wav, pianist.mp4, which are related to mocap data, audio, video, respectively. This data set is used for illustration of how to use the MGT to analyze music-related motion. The corresponing demo can be found in the example folder.

2. In the source-code, the functionalities of the MGT are listed, which are the same as the MGT.zip.

3. Mirtoolbox, Mocaptoolbox, matlabPyrtools are the helpful toolboxes. In order to use MGT, they need to be installed as well.

To install the MGT:

1. at first, download the MGT.zip and helpful toolboxes, put them in a directory and unzip them.

2. Launch your Matlab, enter the directory which contains the previous unzipped files. Run install.m in matlab command console. Note that if the helpful toolboxes have been installed in the matlab, you can simply install MGT like this: addpath('./MGT'), or you can by set path in the matlab "Home" section, find "set path" and click it. Then it opens a file selection dialog. Next, click the "Add Folder" and choose the unzipped folder and finally click "save".

mgt's Issues

mgread type string case sensitive.

Minor thing, but it would be slightly more user friendly to have case-insensitive type strings in video read. I.e. use strcmpi instead of strcmp on line 18, 22, 27 and 36.

MIDI

It would be very useful to be able to also import MIDI files, possibly using the MIDI toolbox

readme file

In the readme file it says in step 2 in the installation instructions that run.m should be run. But there is no run.m anywhere?

Weird counting from mgmotion

It is great to have the counting working in mgmotion, but could it be on its own line? Now it truncates the same line as the command, see screenshot for how it looks (on 2015b on OSX).

Example 1:

Example 2:

function helpful info

I added function helpful info. Just try it in Matlab2015b. I did not try 2015a. If anybody uses 2015a, just try it as well, and give me feedbacks. I will collect them and add them in matlab2016.

bounding box

Is it possible to calculate the bounding box in mgmotion, and also add that as an extra subplot next to the centroid?

Invert and colours in mgmotion

In mgmotion is it possible to have two new attributes:

• choose between colours and greyscale
• choose to invert the output video (black on white instead of white on black)

mgread for files outside search path

The mgread function requires audio and video files to be within the working directory or the search path.
I think this can be fixed in lines 26 and 32 of mgread by including the path in the file specification. Something like this:
mg = mgvideoreader([path, file]);

mgmotion: opticalflow fast than diff

How come that it goes faster to process with optical flow than diff in mgmotion?

errors in Matlab 2016

I develop the toolbox in Matlab2015. It seems the toolbox failed to run in Matlab 2016 because of failing to initiate videoreader object. I will check that in Matlab 2015.

Threshold not working in mgmotion

Have tried different settings for 'Regular' in mgmotion: 0.01, 0.1, 0.3, but the exported video looks the same. Does the threshold actually work?

Also, when using "Binary" I only get black videos regardless the threshold size.

missing progmeter

I get this when running the mgdemo1. Seems that the progmeter function is missing.

Undefined function or variable 'progmeter'.

Error in mgmotion (line 313)
progmeter(ind,numf);

Error in mgdemo1 (line 28)
mgsegmo = mgmotion(mgseg,'Diff');

MGT testing

Thank you for your testing. I have see many comments on the toolbox and new requests. I will collect them and imporve them later. After my final presentation tomorrow, I will put effort on that again. If you have any new requests, just tell me.

Problem with mgvideoreader

@benlyyan #bug

>> mg=mgvideoreader('pianist.mp4')
Error using VideoReader/init (line 619)
Could not read file due to an unexpected error. Reason: Unable to initialize the video obtain properties

Error in VideoReader (line 172)
            obj.init(fileName);

Error in mgvideoreader (line 29)
    obj = VideoReader(fn);

Problem reading folder

Trying mgdemo1 but getting:

>> mg=mgread('Folder')
Error using VideoReader/init (line 619)
Could not read file due to an unexpected error. Reason: Unable to initialize the video obtain properties

Error in VideoReader (line 172)
            obj.init(fileName);

Error in mgvideoreader (line 29)
    obj = VideoReader(fn);


Error in mgread (line 52)
            tmp = mgvideoreader(filename{i});

mgsam error - Expected input type different (Matlab2015b Windows)

mgsam = mgvideosample(mg,'Sampling',[2,2]);
Error using mgvideosample (line 35)
Expected input to be one of these types:

char

Instead its type was double.

mgvideoadjust - column/row for contrast adjustment

I really like that mgvideoadjust provides the possibility to adjust separately for RGB colors. However, when used only with a single value for lo_in hi_in and lo_out hi_out, it would be nice if the user could choose whether to input as row- or column vectors.
E.g. this:
mgvideoadjust(d,[.4; .6],[0; 1])
and this:
mgvideoadjust(d,[.4, .6],[0, 1])
would be the same.

Export opticalflow

mgmotion exports a video file when choosing diff, but can we also get export when using opticalflow?

Problem with mgread

@benlyyan #bug

Trying to load a video file:

>> mg=mgread('dance.mp4')
Output argument "mg" (and maybe others) not assigned during call to "mgread".

Also tried to run

>> mg=mginitstruct

but same problem.

Audio loads as video

Looks like importing only audio loads it in the video field?

>> mg=mgvideoreader('pianist.wav')

mg = 

         video: [1x1 struct]
          type: 'mg data'
    createtime: '09-Jun-2016'

>> mgwaveplot(mg)
Reference to non-existent field 'audio'.

Error in mgwaveplot (line 15)
audio = varargin{1}.audio.mir

Shouldn't it either give an error message or try to load it in the audio field?

mgvideocrop - three issues

1. I think it would be good if there was instructions to the user to select a cropping frame and double click to end the selection procedure. Maybe a printout in the command window.

2. I had some trouble getting the pos argument to work. The help file should probably specify with an example:
   croppedvideo = mgvideocrop(d,[20 40 30 50])

3. When specifying file name like this
   croppedvideo = mgvideocrop(d,[20 40 30 50],'newfilename')
   I get the following output:

_cropping video_
process in progress:100%
Error using VideoReader/init (line 607)
The filename specified was not found in the MATLAB path.

Error in VideoReader (line 172)
obj.init(fileName);

Error in mgvideoreader (line 27)
obj = VideoReader(fn);

Error in mgvideocrop (line 110)
s = mgvideoreader(filename);

updated function mgmotionhistory

I have updated mgmotionhistory function to create motion history image. It is within source code folder.

Average video and average motion video

request

It would be cool to have functions for creating average videos (mean of all frames) for

1. both regular and motion images
2. in both colour and greyscale
3. in both regular and inverted mode