recording error

[introlab]

recording error

[jamesshao8]

Now I can record the wav, and I can hear myself in the files, but I still can't do like in your youtube video to seperate several voices, is that because my mic array is not as good as yours or because of configuration problem?

This is my cfg file for respeaker mic array v1.0.

Configuration file for ReSpeaker circular sound card

version = "2.1";

Raw

raw:
{

fS = 16000;
hopSize = 128;
nBits = 16;
nChannels = 8; 

# Input with raw signal from microphones
interface: {
    type = "soundcard";
    card = 1;
    device = 0;
}

}

Mapping

mapping:
{

map: (1, 2, 3, 4, 5, 6, 7);

}

General

general:
{

epsilon = 1E-20;

size: 
{
    hopSize = 128;
    frameSize = 256;
};

samplerate:
{
    mu = 16000;
    sigma2 = 0.01;
};

speedofsound:
{
    mu = 343.0;
    sigma2 = 25.0;
};

mics = (
    
    # Microphone 1
    { 
        mu = ( +0.0000, +0.0000, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 2
    { 
        mu = ( -0.0160, +0.0277, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 3
    { 
        mu = ( -0.0320, +0.0000, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 4
    { 
        mu = ( -0.0160, -0.0277, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    },

    # Microphone 5
    { 
        mu = ( +0.0160, -0.0277, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );        
    },

    # Microphone 6
    { 
        mu = ( +0.0320, +0.0000, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );        
    },

    # Microphone 7
    { 
        mu = ( +0.0160, +0.0277, +0.0000 ); 
        sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
        direction = ( +0.000, +0.000, +1.000 );
        angle = ( 80.0, 90.0 );
    }
    
);

# Spatial filter to include only a range of direction if required
# (may be useful to remove false detections from the floor)
spatialfilter: {

    direction = ( +0.000, +0.000, +1.000 );
    angle = (80.0, 90.0);

};    

nThetas = 181;
gainMin = 0.25;

};

Stationnary noise estimation

sne:
{

b = 3;
alphaS = 0.1;
L = 150;
delta = 3.0;
alphaD = 0.1;

}

Sound Source Localization

ssl:
{

nPots = 4;
nMatches = 10;
probMin = 0.5;
nRefinedLevels = 1;
interpRate = 4;

# Number of scans: level is the resolution of the sphere
# and delta is the size of the maximum sliding window
# (delta = -1 means the size is automatically computed)
scans = (
    { level = 2; delta = -1; },
    { level = 4; delta = -1; }
);

# Output to export potential sources
potential: {

    format = "json";

    interface: {
        type = "socket";
        ip = "127.0.0.1";
    port = 9001;

    };

};

};

Sound Source Tracking

sst:
{

# Mode is either "kalman" or "particle"

mode = "kalman";

# Add is either "static" or "dynamic"

add = "dynamic";

# Parameters used by both the Kalman and particle filter

active = (
    { weight = 1.0; mu = 0.3; sigma2 = 0.0025 }
);

inactive = (
    { weight = 1.0; mu = 0.15; sigma2 = 0.0025 }
);

sigmaR2_prob = 0.0025;
sigmaR2_active = 0.0225;
sigmaR2_target = 0.0025;
Pfalse = 0.1;
Pnew = 0.1;
Ptrack = 0.8;

theta_new = 0.9;
N_prob = 5;
theta_prob = 0.8;
N_inactive = ( 150, 200, 250, 250 );
theta_inactive = 0.9;

# Parameters used by the Kalman filter only

kalman: {

    sigmaQ = 0.001;
    
};

# Parameters used by the particle filter only

particle: {

    nParticles = 1000;
    st_alpha = 2.0;
    st_beta = 0.04;
    st_ratio = 0.5;
    ve_alpha = 0.05;
    ve_beta = 0.2;
    ve_ratio = 0.3;
    ac_alpha = 0.5;
    ac_beta = 0.2;
    ac_ratio = 0.2;
    Nmin = 0.7;

};

target: ();

# Output to export tracked sources
tracked: {

    format = "json";

    interface: {
        type = "socket";
        ip = "127.0.0.1";
    port = 9000;
    };

};

}

sss:
{

# Mode is either "dds", "dgss" or "dmvdr"

mode_sep = "dds";
mode_pf = "ms";

gain_sep = 1.0;
gain_pf = 10.0;

dds: {

};

dgss: {

    mu = 0.01;
    lambda = 0.5;

};

dmvdr: {

};

ms: {

    alphaPmin = 0.07;
    eta = 0.5;
    alphaZ = 0.8;        
    thetaWin = 0.3;
    alphaWin = 0.3;
    maxAbsenceProb = 0.9;
    Gmin = 0.01;
    winSizeLocal = 3;
    winSizeGlobal = 23;
    winSizeFrame = 256;

};

ss: {

    Gmin = 0.01;
    Gmid = 0.9;
    Gslope = 10.0;

}

separated: {

    fS = 16000;
    hopSize = 512;
    nBits = 16;        

    interface: {
        type = "socket";
        ip = "127.0.0.1";
        port = 10000;
    }        

};

postfiltered: {

    fS = 16000;
    hopSize = 512;
    nBits = 16;        

    interface: {
        type = "socket";
        ip = "127.0.0.1";
        port = 10010
    }        

};

}

classify:
{

frameSize = 1024;
winSize = 3;
tauMin = 32;
tauMax = 200;
deltaTauMax = 7;
alpha = 0.3;
gamma = 0.05;
phiMin = 0.15;
r0 = 0.2;    

category: {

    format = "undefined";

    interface: {
        type = "blackhole";
    }

}

}

[GodCed]

Try updating to the latest commit in master branch. It’s supposed to fix the no such file or directory error when recording.

[jamesshao8]

Thanks for your reply. I tried to compile the newest code, but I didn't succeed. I am new to node.js. I saw in another issue that I should compile it with standard user? I will try that tomorrow.

[GodCed]

What you saw is correct. Normally, the whole process is automated once node js is installed. So what you want to do:

1. Install node js on your machine following these instructions You want node 8.

2. Clone the odas web repo

3. In the odas web root folder, as a standard user (no sudo), run npm install. This should take a few minutes.

4. Run using npm start still from the odas web root folder.

[jamesshao8]

Thanks. I tried your setup procedure and now I am able to compile and run the latest version. but I still can't record the audio files normally. They are all 44Bytes in size and their durations are either 0s or -1s.

[jamesshao8]

I think this was a configuration problem. I tried with respeaker mic array v1.0 using similar cfg file from another thread. I can now record to wav files. But with respeaker mic array v2.0 I still can't.

the cfg file I used for v2.0 is at https://github.com/respeaker/usb_4_mic_array/blob/master/odas.cfg

[GodCed]

With an array as small as the respeaker, some tuning may be required in order to obtain a good separation. I'll add @FrancoisGrondin to the conversation as he's more aware of the configuration details.

What would you recommend experimenting with @FrancoisGrondin ?

[jamesshao8]

Thanks. I have some other questions: What does post filtered and separated mean and when should I hit the button "determine if separated audio is recorded"?

[FrancoisGrondin]

@jamesshao8

Hi,

I still can't do like in your youtube video to seperate several voices

First, are you able to track multiple voices with the system? If so, how far apart in degrees are the sources? Can you share some separated/postfiltered samples? In the video we used the DGSS separation algorithm, which usually performs better than DDS, but may be unstable if not setup properly.

[jamesshao8]

Yes, I am able to track multiple voices. I can see different sources on Active sources locations. I think 2 sources will converge when they are less than around 40 degrees apart. But if the angle separation is larger than that, the tracking is quite precise. I will record the separated/postfiltered samples. But I still don't know what does that mean? So if I use 2 cell phone to play music at the same time. How many wav files should the odas studio produce if it's configured properly?

[jamesshao8]

When I do the recording should I hit the "determine if separated audio is recorded" button?

[jamesshao8]

I just tried to record from 2 cell phone playing music placed the on the desk, with an angle of 180 degree apart, and the "determine if separated audio is recorded" checked. I can now hear 2 separated music in those files that end with "pf".

[GodCed]

Yes, you must hit the « determine if... » button as it enables the recording function (no recording will be produced if the box is unchecked).

Regarding separated and post filtered:
Separated is the output of the separation module, which core is a beam forming algorithm. The files ends with sp.

Post filtered is the output of the post filtering module. It applies filters to the separated audio to enance the separation, but introduce audible artefacts in the stream. The files end with pf.

[GodCed]

Also, the system should produce a file set for each source. With two phones, you should get two set (4 files, 2 sp, 2 pf).

Note that if the music stops on one of them, and then restarts, this will stop the first recording set and start a new one.

[jamesshao8]

Yes, now I get the separated files, 2 sp and 2 pf, exactly as you said. the pf files do have better separation than sp files, and have some artefacts.

2phone_180_degree.tar.gz

I am uploading my record here, in case some one else would like some references.

And this is how I place my cell phones and the respeaker.

I think this issue is almost closed.

But I have another question, now the beamforming algorithm is automatically targeted at the DOA of sound sources. If it's possible that I can determine manually which angle I want to record in odas_web? Can I modify the odas_web to achieve that or do I need to modify the c++ code of odascore?

[GodCed]

No code modification is needed. You can specify a target source in the ODAS configuration file.
Please see this issue.

[jamesshao8]

Great! I will try that configuration file. Many thanks.

Later today, when I hit the delete all button, after a while I will get a javascript error saying it can't find some file... and sometimes I found files have been recorded even if I didn't check "determine if" box. I am thinking this could be a multi thread problem. will confirm that and get back to you later on.

[yesheysamang]

@jamesshao8 I have the same issue that you had regarding the recorded audio files being 44Bytes in size and their durations are either 0s or -1s. DId you end up solving this issue for the mic array v2.0? If so how?

[jamesshao8]

@yesheysamang Yes, mic array v2.0 can also record wav files successfully now. You need to modify the config file little bit.

just change line 293 294 307 308 in your config file according to my config file I used for mic array v1.0

Then run odas_web and odascore separately.

interface: {
    type = "socket";
    ip = "127.0.0.1";
    port = 10000;
}        

interface: {
    type = "socket";
    ip = "127.0.0.1";
    port = 10010
}        

[jamesshao8]

@GodCed
Hello GodCed,

I am trying to record sound in a fixed direction. So I set to static mode in sst block. But I found I am unable to record any sound. If, in stead, I firstly set sst block to dynamic and start recording in odas_web, then switch the sst block in static mode and restart the odas_live, then I am able to record sound. I wonder if it is a bug of odas_web? But the recording still doesn't show a beamforming result.

[GodCed]

@jamesshao8

If, in stead, I firstly set sst block to dynamic and start recording in odas_web, then switch the sst block in static mode and restart the odas_live, then I am able to record sound.

Recording must me enabled in ODAS Studio before the source you want to record appears. For static sources, this means that the record window must be open with the record checkbox checked before you launch ODAS.

But the recording still doesn't show a beamforming result

I'm not sure I understand what you mean by that. Is it that the recorded audio doesn't seem to match the expected result of a beam former in the specified direction? If that's the case, I'll suggest you refer to the ODAS repo, as ODAS Studio doesn't perform any processing on the incoming audio stream.

[jamesshao8]

@GodCed Thanks GodCed. After following your instruction, we are now able to see the beamforming effect in odas_web. I think beam former result is more obvious in the pf file than in the sp file, but according to my knowledge, all beamforming should be done in sp... I will refer to ODAS repo for the processing.

[GodCed]

@jamesshao8
Glad to hear that.

For the beam forming, the beam former (separation) and the post filtering work together. As the beam former isn’t perfectly « square cut » as in « there a transition period between full gain and full attenuation » , the post filtering remove some more unwanted energy.

That’s why the effect is more obvious after the post filtering. The side effect is some distortion, which can be problematic for some use case. That’s why both audio streams are available.

[jamesshao8]

@GodCed Thanks for confirming. Actually after decreasing gain_pf from 10 to 2, we are now able to get a good result, without much distortion but still obvious beamforming effect.

I think we still have some way to go in tuning the parameters to increase the effective range. According our experiment, if we record directly to wav using pyaudio and this respeaker device, we can record sound from a longer distance than when we are using odas. We'll try to find out what caused this.