vislab/eeg-clean-tools

Contains tools for EEG standardized preprocessing

EEG-Clean-Tools

Contains tools for the PREP pipeline for standardized preprocessing of EEG. You can find user documention at: http://vislab.github.io/EEG-Clean-Tools/

Note: For convenience, EEGLABPlugin directory contains the latest released version of the PREP that can be unzipped into your EEGLAB plugins directory.

Citing the PREP pipeline

The PREP pipeline is freely available under the GNU General Public License. Please cite the following publication if using:

Bigdely-Shamlo N, Mullen T, Kothe C, Su K-M and Robbins KA (2015)
The PREP pipeline: standardized preprocessing for large-scale EEG analysis
Front. Neuroinform. 9:16. doi: 10.3389/fninf.2015.00016

People

The PREP pipeline incorporates many algorithms that were developed at USCS SCCN over many years by Nima Bigdely-Shamlo, Tim Mullen and Christian Kothe. Kyung Min Su performed most of the machine learning evaluation of PREP. Cassidy Matousek and Jeremy Cockfield worked on the interfaces for the EEGLAB plugin as well as associated visualization tools. Kay Robbins of UTSA is the lead developer and maintainer of PREP.

Support:

This research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-10-2-0022. The views and conclusions contained in this document/software are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.

Releases

Version 0.56.0 Released 8/01/2021

• Corrected parfor failure when channel number not consecutive
• Fixed missing badChannelsFromDropout in updateBadChannels issue#28

Version 0.55.4 Released 7/26/2020

• Correctly restored EEGLAB options after execution
• Added functions to output errors from etc.noiseDetection
• Corrected findpeaks naming conflict in Chronux
• Post process does not execute if Prep had errors

Version 0.55.3 Released 10/19/2017

• Fixed issue with interpolated channels when interpolation order is pre-process
• Fixed issue with correct removal of interpolated channels during post-processing
• Reordered preprocessing and report buttons on master GUI

Version 0.55.2 Released 08/18/2017

• Fixed undefined reference to referenceOut in prepPipeline post process

Version 0.55.1 Released 06/03/2017

• Wrote printListCompressed to display channels more compactly
• Put in a MATLAB version check because legend titles not supported in 2014b
• Fixed spacing on output of interpolated channel numbers

Version 0.55.0 Released 05/29/2017

• Changed the EEG.etc.noiseDetection structure to contain removed channels and interpolated channels for easier access
• Fixed reporting to work when bad channels have been removed
• Added original channel labels to EEG.etc.noiseDetection for ease in reporting
• Added Blasst as an unsupported line noise removal option
• Moved legend of spectrum to right, put in checks for removed channels
• Corrected bug in smoothing in cleanline
• Corrected several reporting issues
• Default behavior now outputs errors to command line in addition to logging
• Renamed several functions to make naming scheme consistent
• Started supporting changelog in versions
• Fixed bug in struct2str and improved com return on pop_prepPipeline

Version 0.52 Released

• Modified code to handle EEG structures with empty EEG.error.
• Performed additional minor cleanup.

Version 0.51 Not released

• Developing bad window visualization plugin for EEGLAB

Version 0.50 Released

• Made several cleanup modifications to ready for release.

Version 0.48 (Not released -- version 0.47 with EEGLAB integration)

• Integrated EEGLAB plugin

• Changed the default structure value field name from defaults.default to default.value and propagated the change

• Changed default names of line noise and global trend to linenoise and globaltrend

• Modified the resampling step to allow an option low pass filter to remove downsampling artifacts just below Nyquist frequency.

  Version 0.47 (Not released -- version 0.46 with additional changes)

• Minor refactoring of performReference to avoid 1 extra filtering operation --- should not reflect results.

• Also added average and specific referencing methods -- not tested as yet.

Version 0.46 (Not released - version 0.45 with additional changes)

• Fixed remapping of bad evaluation channels into original channel numbers (relevant when there are none EEG channels interspersed in the channel locations.
• Passed detrend information in reference structure to allow detrending with other than the defaults
• Corrected several channel mapping issues in the reporting.

Version 0.45 (Not released - version 0.44 with additional changes)

• Refactored report to allow statistics to be gathered from noisy structures

Version 0.44 (Not released - version 0.43 with additional changes)

• Corrected a minor issue with reporting -- difference between robust and ordinary reference had axes reversed.
• Updated to run with plotting compatible with MATLAB 2014b
• Added box on to cummulative plots.

Version 0.43 (Not released - version 0.42 with additional changes)

• Corrected a minor issue with reporting -- mean scalp correlation map for beforeInterpolation was plotting the Original data rather than the beforeInterpolation data.

Version 0.42 (Not released - version 0.41 with additional changes)

• Added default line frequencies as multiples of 60 up to half nyquist.

Version 0.41 (Not released - version 0.40 with additional changes)

• Replaced default method with channel forgetting and median initialization
• Converted EEG to double at the beginning of the pipeline
• Added a noisyStatisticsForInterpolation field to the reference reporting structure.

Version 0.40 (Not yet released - major change in strategy)

• Changed the name from StandardLevel2 to PrepPipeline
• Implemented the HP filter-free strategy
• Added a keepFiltered version -- if false (the default) the data in the repository is not high pass filtered
• Added an option for removing global trend
• Incorporated the different reference schemes into a single performReference

Version 0.28 (Not yet released)

• Changed the name of the noisyParameter structure in EEG.etc to noiseDetection. This is a major change with corresponding change in ESS.
• Added a specificReferenceChannels field to reference structure
• Changed the averageReference field name to referenceSignal in reference structure
• Included a referenceType field in the reference structure (this can be 'robust', 'average', or 'specific')
• Eliminated the don't interpolateHFChannels flag.
• Added routines to do specificReference (mastoid or average)
• Modified showSpectrum to return the spectra of all of the channels.
• Detrending at 0.2 Hz has replaced FIR filtering as default trend removal.

Version 0.27 Released 1/7/2015

• Correct version of bug fix in cleanLineNoise -- watch that single precision conversion!

Version 0.26 Released 1/7/2015

• Release to fix bug in cleanLineNoise --- channels that are not lineNoiseChannels were set to zero rather than being carried forward.

  Version 0.25 Released 1/5/2015 (major)

• Removed saving of temporary file after line noise removal

• Fixed report of relative reference

• Modified findNoisyChannels to exclude NaN and constant channels from noisyChannel thresholding, but to designate them as bad channels

• Moved resampling step before high pass filter

• Assigned return values in a separate step

• Put error check in ShowSpectrum when invalid data is invalid

• Correct minor issues with PlotScalpMap

• Added extractReferenceStatistics -- which extracts summary statistics for an entire archive.

• Added iterations on the remove robust reference

• Added a summary reporting scheme for spotting problematic datasets.

Version 0.24 Released 12/7/2104 (major)

• Fixed channel selection bug in showSpectrum
• Added error handling for failures in standardLevel2Pipeline
• Added error reporting for failures
• Corrected time scale on visualization of difference between robust and mean reference
• Added channel labels as well as numbers to spectrum visualization
• Fixed major bug in robustReference so that original signal is rereferenced
• Revised and expanded the reporting

Version 0.23 Released 11/13/2014

• Removed the channel locations and channel information from noisyOut because it is already in the reference structure at top level.
• Added reporting of average fraction of channels bad in windows.
• Added first version of hdf5support -- rewrites the noisyParameters to an HDF5 file.

Version 0.22 Released 11/9/2014

• Revised the method of computing the windowed channel deviations
• Added summary reporting functions
• Added a check to only perform ransac when sufficiently good channels are available
• Added check to only perform ransac when channel locations are available
• Fixed the input parameter structure on findNoisyChannels
• Added the infrastructure for the summary of all datasets

Version 0.21 Released 10/30/2104

• Removed any reference to chanlocs in highPassFilter
• Full integration with ESS Study Level 2 code
• Preliminary version of Standard Level 2 Report finalized (gives pdf)

Version 0.20 Released 10/18/2014

• Converted standardLevel2Pipeline to a function
• Moved the computationTimes structure to standardLevel2Pipeline so that it is returned.

Version 0.19 Released 10/16/2014

• Refactored name is also included in the params structure.
• Renamed rereferencedChannels as channelsToBeReferenced to agree with ESS.

Version 0.18 Released 10/15/2014

• Refactored so that all input to the pipeline is in a single params structure.
• Fixed the HF noise reporting windows and several minor bugs
• Added visualizations to show number of bad channels in each window

eeg-clean-tools's Issues

Bad-by-dropout channels ignored during re-referencing?

Was reading through the PREP re-referencing code while working on PyPREP, and noticed something that looks to be a bug in updateBadChannels.m:

EEG-Clean-Tools/PrepPipeline/utilities/updateBadChannels.m

Lines 1 to 23 in 3ed337e

	function ref = updateBadChannels(ref, noisy)
	% Update the bad channel lists from ref based on bad channels in noisy
	ref.badChannelsFromNaNs = union(ref.badChannelsFromNaNs, ...
	noisy.badChannelsFromNaNs);
	ref.badChannelsFromNoData = union(ref.badChannelsFromNoData, ...
	noisy.badChannelsFromNoData);
	ref.badChannelsFromHFNoise = union(ref.badChannelsFromHFNoise, ...
	noisy.badChannelsFromHFNoise);
	ref.badChannelsFromCorrelation = union(ref.badChannelsFromCorrelation, ...
	noisy.badChannelsFromCorrelation);
	ref.badChannelsFromDeviation = union(ref.badChannelsFromDeviation, ...
	noisy.badChannelsFromDeviation);
	ref.badChannelsFromRansac = union(ref.badChannelsFromRansac, ...
	noisy.badChannelsFromRansac);
	ref.badChannelsFromDropOuts = union(ref.badChannelsFromDropOuts, ...
	noisy.badChannelsFromDropOuts);
	ref.all = union(...
	union(ref.badChannelsFromNaNs, ref.badChannelsFromNoData), ...
	union( ...
	union(ref.badChannelsFromHFNoise, ref.badChannelsFromCorrelation), ...
	union( ...
	union(ref.badChannelsFromDeviation,ref.badChannelsFromRansac), ...
	ref.badChannelsFromRansac)));

Specifically, dropout channels aren't included in the ref.all union at the end, meaning that they're never excluded/interpolated during re-referencing. Additionally, ref.badChannelsFromRansac is included in the union twice, which makes it look like bad-by-dropout channels were meant to be included but got copy/pasted incorrectly.

Thanks in advance for taking a look!

Add reference information in EEG struct after PREP

It seems that EEGLAB uses this on the main menu. Something to do for next release

should I do a low pass filter and another rereference after PREP, before ICA?

Thank you so much for your reply! I process my data with PREP again following your advices, step by step.
Veog channel's number is the 20. I still not clear with should I do the channel location firstly.

After the channel location process through eeglab edit. I ran Prep.



In China, the line frequency is 50hz

Excluded 20th channel under reference channnels and evaluation channels condition.

After the whole process, I went back to eeglab interface, which shows the data is still unrereference.

According to the introduction of PREP, I thought this process have completed the procedures for me:
1 high pass
2 line noise
3 rereference
So I just need to do a low pass process before ICA, without another rereference and other preprocesses?

I am still a amateur in processing EEG data, thx for your patience!

PREP removes non-interpolated channels

I run PREP on a 32 channel EEG data where many sets have 1-3 noisy channels. Since I want to continue with a matrix of full rank, I set 'removeInterpolatedChannels' to true. However if I do so, PREP seems to remove also channels that it does not note as interpolated in the report (and accordingly in EEG.etc.noiseDetection.reference.interpolatedChannels.all). I've seen this in multiple subjects and the only pattern I could recognize is that it seems to remove 2x the amount of channels that it tracks as interpolated (e.g. leaving me with 26 channels when 3 channels had been interpolated according to the report). Which channels are removed on top of the interpolated ones, seems sort of arbitrary to me (but most likely it isn't). Do you have any idea what could cause this issue? Is it a bug or a user-mistake?
If I set 'removeInterpolatedChannels' to false, it seems to work fine. The rank of the new EEG.data is then 32-nic-1 (nic = number of interpolated channels) which makes sense to me since it's now referenced to average (therefore the -1).
As a workaround, I'll keep it this way and remove the interpolated channels manually. I'm just unsure whether there's something else wrong with my usage of PREP. (Which is a great tool btw!)
These are the paramters that I used:

EEG = pop_prepPipeline(EEG,struct('ignoreBoundaryEvents', true, ...
    'referenceChannels', [1:4, 6:26, 28:32], ...
    'evaluationChannels',[1:4, 6:26, 28:32], ...
    'rereferencedChannels', [1:32], ...
    'ransacOff', false, ...
    'ransacSampleSize', 50, ...
    'ransacChannelFraction', 0.25, ...
    'ransacCorrelationThreshold', 0.75, ...
    'ransacUnbrokenTime', 0.4, ...
    'ransacWindowSeconds', 5, ...
    'srate', 250, ...
    'robustDeviationThreshold', 5, ...
    'correlationWindowSeconds', 1, ...
    'highFrequencyNoiseThreshold', 5, ...
    'correlationThreshold', 0.3, ...
    'badTimeThreshold', 0.01, ...
    'maxReferenceIterations', 4, ...
    'referenceType', 'Robust', ...
    'reportingLevel', 'Verbose', ...
    'interpolationOrder', 'Post-reference', ....
    'meanEstimateType', 'Median', ...
    'samples', EEG.pnts, ....
    'lineNoiseChannels', [1:32], ...
    'lineFrequencies', [50  100], ...
    'Fs', 250, ...
    'p', 0.01, ....
    'fScanBandWidth', 2, ...
    'taperBandWidth', 2, ...
    'taperWindowSize', 4, ...
    'pad', 0, ...
    'taperWindowStep', 1, ...
    'fPassBand', [0  125], ...
    'tau', 100, ...
    'maximumIterations', 10, ...
    'cleanupReference', false, ...
    'keepFiltered', true, ...
    'removeInterpolatedChannels', true, ...
    'reportMode', 'normal', ...
    'publishOn', true, ...
    'sessionFilePath', strcat(path_reports, filename,'_Report.pdf'), ...
    'summaryFilePath', strcat(path_reports, filename,'_Summary.html'), ...
    'consoleFID', 1));

Request to exclude certain intervals from bad channel detection

Experimenter performs a 20Hz impedance test for a long time, which causes Prep to flag all channels as bad. Would like to be able to exclude this period from the counting.

Change name of findpeaks

Conflicts with the name in the signal processing toolbox.

Out of memory error

Hi,

I tried preprocess a file (80 MB) but got an out of memory error. I was using parallel pool.

I am on Mac M1 MAX, 64 GB RAM, Matlab 2020b, PrepPipeline version 0.56.0 (8.01.2021).

Running Prep...
Using option file in directory ~
pop_editoptions: Options were not modified
Checking for boundary events
Preliminary detrend to compute reference
pop_eegfiltnew() - performing 1651 point highpass filtering.
pop_eegfiltnew() - transition band width: 1 Hz
pop_eegfiltnew() - passband edge(s): 1 Hz
pop_eegfiltnew() - cutoff frequency(ies) (-6 dB): 0.5 Hz
pop_eegfiltnew() - filtering the data (zero-phase, non-causal)
firfilt(): |====================| 100%, ETE 00:00
Line noise removal
Find reference
Robust referencing
pop_eegfiltnew() - performing 1651 point highpass filtering.
pop_eegfiltnew() - transition band width: 1 Hz
pop_eegfiltnew() - passband edge(s): 1 Hz
pop_eegfiltnew() - cutoff frequency(ies) (-6 dB): 0.5 Hz
pop_eegfiltnew() - filtering the data (zero-phase, non-causal)
firfilt(): |====================| 100%, ETE 00:00
Warning: [MATLAB:lang:StackOverflow]
Error using path (line 71)
Out of memory. The likely cause is an infinite recursion within the program. 
> In prepPipeline (line 174)

Warning: The following error was caught while executing 'onCleanup' class destructor:
Error using restoreEEGOptions (line 43)
Cannot restore EEG options. Unknown error occurred.
 Backup options file
 /private/var/folders/xz/_cqvg2j14rvb3bs26x3kwpr80000gn/T/tpedfbb5fc_11c8_43e8_9e26_59c31c22a50a.m
 Current options file
 ~/eeg_options.m

Error in prepPipeline>cleanup (line 187)
   restoreEEGOptions(backupFile, currentFile);

Error in prepPipeline>@()cleanup(backupOptionsFile,currentOptionsFile,warningsState) (line 46)
    finishup = onCleanup(@() cleanup(backupOptionsFile, currentOptionsFile, ...

Error in onCleanup/delete (line 80)
            obj.task();

Error in prepPipeline (line 182)
end

Best,
Dawid

Should I do the channel location first?

Dear PREP pipeline group
I have scaned your paper and read the introduction to prep, I appreciate what you have done, but I still felt confuse when I use this plugin on EEGlab. I check all the parameter setting choice, and find none directing to montage.

All in all, I want to want should I do with my data, before I use the tool. should I complete channel location first? should I discard the Veog channel. I cannot find any glue about that on your website.

After the process, I cannot see any change through this EEGlab interface. no channel location, no reference.

Errors in report publising functions

I spotted 2 typos in the report publishing functions:

• in getErrors.m, line 16:
  ~isfield('noiseDetection', 'errors')
  should be:
  isfield(noiseDetection, 'errors')
• in publishPrepReport.m, line 33:
  if (nargin < 5)
  should be:
  if (nargin < 4)

restoreEEGOptions not quite working correctly

Warning: The following error was caught while executing 'onCleanup' class destructor:
Error using restoreEEGOptions (line 43)
Cannot restore EEG options. No matching files were found.
Backup options file
C:\Users\KAYROB~1\AppData\Local\Temp\eeg_options_backup.m
Current options file
C:\Users\Kay Robbins\eeg_options.m

Error in prepPipeline>cleanup (line 225)
restoreEEGOptions(backupFile, currentFile);

Error in prepPipeline>@()cleanup(backupOptionsFile,currentOptionsFile,warningsState) (line 46)
finishup = onCleanup(@() cleanup(backupOptionsFile, currentOptionsFile, ...

Error in onCleanup/delete (line 60)
h.task();

Error in prepPipeline (line 41)
computationTimes= struct( 'detrend', 0, 'lineNoise', 0, 'reference', 0);

Error in level2Study/createLevel2Study (line 385)
[EEG, computationTimes] = prepPipeline(EEG, params);

Error in runESSLARGPrepPipelineNoInterpolation (line 30)
obj2.createLevel2Study(ess2Dir, 'forceRedo', false, ...

In prepPipeline (line 41)
In level2Study/createLevel2Study (line 385)
In runESSLARGPrepPipelineNoInterpolation (line 30)

Keep receiving errors when running the PREPpipeline

I have set the path in Matlab to EEGlab and both the VisLab-EEG-Clean-Tools and PrepPipeline before using the function. According to the report, the PREP pipeline has failed at the step of line noise removal with the following error:

prepPipeline failed removeLineNoise: Error using testSignificantFrequencies
Undefined function 'finv' for input arguments of type 'double'.
Line noise method: clean

May I ask how to solve the problem? Thank you so much.

Print out errors on command line when verbose option is set

Make it more obvious what the errors are when it fails.

Removing interpolated channels conflict with report creation

When running the pipeline with the parameter removeInterpolatedChannels set to true, and then trying to publish the report, prepReport can still try to access the removed channels (e.g. line 87 to plot the spectrum) leading to an error.

struct2str getNumerical

num2str must have a numerical input in some versions of matlab (Line 38) causing error

Why is EEGlab PREP not flagging up EEG files as "problematic" if the number of interpolated channels is > 25%?

Dear PREP group,

I am working with clinical EEG data and I am using the PREP pipeline as an early preprocessing step.
In some of my EEG files, a high number of channels get flagged as bad channels and get interpolated, e.g. 7 out of 21 in the example attached (see screenshot).
However, in the PREP summary report, the error status for these files is still labelled as "good" (see screenshot).

From the PREP reference paper (Bigdely-Shalmo et al., 2015) I understand that one of the criteria used to flag datasets with issues is if >25% of the channels get interpolated:
"The criteria we currently use to flag datasets with serious issues include: [...]
Robust referencing interpolated more than 25% of the reference channels."

I am wondering why this is not flagged up in the summary report (in the case of the file attached, 33% of the channels were interpolated)?

Thank you for any help you might be able to provide!

Best wishes,
Irene

Issues with Infinite Recursion

Hello,

I'm attempting the run the prep pipeline for a set of 72Hr long continuous EEG recordings I have collected.

After running Prep on my EEG file, I get the following error:

"prepPipeline failed performReference: Error using catdirs (line 28) Out of memory. The likely cause is an infinite recursion within the program."

Any idea why this might be the case? Has anyone else had this error?

PyPrep Noisy Channel Parameters

Hi everyone, I have been working on a project that requires the preprocessing of EEG data. I would like to use one of the PyPrep noisy channel functions to mark bad channels, and I am testing different parameters within these functions.

How do you recommend I choose the best parameters for my EEG dataset? I would appreciate your insight on this question.

Error using cleanLineNoise (line 57)

Hello,

I am getting an error when I run PrepPipeline0.55.4
Here is a snip of the aftermath:

The same errors were present when I had the cleanline plugin installed, except the source code path was directed at cleanline directly, rather than in the PrePipeline utilities.

Cancel button doesn't work on EEGPlugin

Cancel button doesn't cancel....

Minimum of three iterations in robust re-reference?

Hi there,

I think I've discovered a small bug in the re-referencing logic of the PREP pipeline, but I wanted to make sure it isn't intentional behaviour before modifying anything on my end.

EEG-Clean-Tools/PrepPipeline/utilities/robustReference.m

Lines 58 to 87 in 3ed337e

	%% Remove reference from signal iteratively interpolating bad channels
	iterations = 0;
	noisyChannelsOld = [];
	while true % Do at least 1 iteration
	noisyStatistics = findNoisyChannels(signalTmp, referenceOut);
	referenceOut.badChannels = ...
	updateBadChannels(referenceOut.badChannels, noisyStatistics.noisyChannels);
	noisyChannels = referenceOut.badChannels.all(:)';
	if (iterations > 1 && (isempty(noisyChannels) ||...
	(isempty(setdiff(noisyChannels, noisyChannelsOld)) ...
	&& isempty(setdiff(noisyChannelsOld, noisyChannels))))) || ...
	iterations > referenceOut.maxReferenceIterations
	break;
	end
	noisyChannelsOld = noisyChannels;
	sourceChannels = setdiff(referenceOut.referenceChannels, noisyChannels);
	if length(sourceChannels) < 2
	error('robustReference:TooManyBad', ...
	'Could not perform a robust reference -- not enough good channels');
	end
	if ~isempty(noisyChannels)
	signalTmp = interpolateChannels(signal, noisyChannels, sourceChannels);
	else
	signalTmp = signal;
	end
	referenceSignal = nanmean(signalTmp.data(referenceChannels, :), 1);
	signalTmp = removeReference(signal, referenceSignal, referenceChannels);
	iterations = iterations + 1;
	fprintf('Iteration: %d\n', iterations);
	end

In the above loop, iterations starts at zero and only get incremented at the end, after the "check if we should break the loop" logic. As a consequence, it means that the code actually does a maximum of 5 iterations by default, even though the default maximum is documented to be 4. Additionally, it means that PREP needs to do at least three iterations of this loop, even if the detected bad channels are unchanged between the first and second iterations. Looking at the original PREP publication, the pseudocode for this loop doesn't say anything about a minimum of iterations, so I'm wondering whether this is a bug or an intentional behaviour I don't understand.

Thanks in advance, and thank you for creating and maintaining this software!

adding back online reference

Hi,
as far as I could see it, there is currently no way to add back the online reference during the pipeline. In order to base the average reference on N + onlineRef channels I simply added onlineRef to params.referenceChannels and entering N in params.evaluationChannels. onlineRef is just a vector of zeros. The pipeline stopped with a matrix dimension mismatch error. This was because PREP checks also referenceChannels to be usable (in findUnusableChannels(signal, referenceOut.referenceChannels)), which is a good idea, but will at the end produce the error when reindexing channels that are not bad and one or more reference channels that are found to be bad but do not occur in evaluation channel list.

signal.data(targetChannels, :) = signaltemp.data(reindex, :); line 53 of interpolateChannels

I think simply replacing referenceChannels by evaluationChannels in findUnusableChannels is not a good idea as the reference signal should be clean.

Is there another to do it?
Best, Norman

Change to EEGLAB options

PREP changes the option file to 'option_savetwofiles' disabled but it's not changed back when it is done.

post processing need to check whether an error occurrs in previous steps

If reference or earlier step fails, post-processing should not occur. This must be fixed.

Also structure has both postprocess and postProcess fields

Computation of windowSize based on detrendCutoff in localDetrend

Hello! Using a version of the removeTrend function ported to Python, I noticed that frequencies below the detrendCutoff in removeTrend are not completely attenuated when using the local method: sappelhoff/pyprep#125

I was wondering if this is the desired output, since it isn't clear to me why the window size is computed by dividing 1.5 by the cutoff frequency:

EEG-Clean-Tools/PrepPipeline/utilities/removeTrend.m

Line 79 in 9ac9ea0

windowSize = 1.5/detrendOut.detrendCutoff;

Also, I tried to test the original MATLAB code, but I wasn't able to debug it myself. Any tips there would be appreciated.

Script:

%% simulate signal
duration = 120;
frequency = [0.2, 0.4, 0.8, 1.6, 3.2];
amplitude = [1, 2, 3, 2, 1];
srate = 200;

nSamples = round(duration * srate);
period = 1 / srate;
seconds = (1:nSamples).*period;
data = zeros(1, nSamples);
for i=1:length(frequency)
    data = data + amplitude(i) * sin(2 * pi * frequency(i) * seconds);
end
%% format input
EEG.data = data;
EEG.srate = srate;
detrendIn = struct('detrendChannels', 1, 'detrendType', 'linear', ...
                    'detrendCutoff', 0.5, 'detrendStepSize', 0.02, ...
                    'detrendCommand', []);
[EEG, detrendOut] = removeTrend(EEG, detrendIn);

Error:

testRemoveTrend

Unrecognized function or variable 'change_row_to_column'.

Error in localDetrend (line 35)
data = change_row_to_column(data);

Error in removeTrend (line 83)
        localDetrend(EEG.data(detrendOut.detrendChannels, :)', ...

Error in testRemoveTrend (line 30)
[EEG, detrendOut] = removeTrend(EEG, detrendIn);

Crash on movefile in publishPrepReport

Prepline 0.55 crashes during publishPrepReport, whether I specify additional parameters or not (e.g., use all defaults).

One example of parameters that leads to a crash:
'summaryFilePath', strcat('C:\Users\data', filename, '_PREP.html') ,...
'sessionFilePath', strcat('C:\Users\data', filename, '_PREP.pdf') ,...
'consoleID', 1,...
'publishOn', true);

In any test runs, Matlab credits line 73 of publishPrepReport: "movefile(tempReportLocation, sessionReportLocation);" for the error.

The code will run to completion if I comment out that line or if I set relativeReportLocation (line 48) equal to sessionFilePath. However, only the .html file will be properly named. The .PDF will only be named "prepReport" and is overwritten if more than 1 subject is processed in a loop.

PREP - Reference Error (Failed postProcess)

Dear VisLab Team,

I'm a master student at the Berlin School of Mind and Brain and I'm currently using the PREP Pipeline to preprocess the EEG dataset of my thesis.
I'm using it as an EEGlab plugin and I'm extremely impressed by your work.
However, I'm constantly encountering the same MATLAB error:

Warning: [MATLAB:nonExistentField]
Reference to non-existent field 'referenceOut'.
Error in prepPipeline (line 184)
EEG = removeTrend(EEG, EEG.referenceOut);
Error in pop_prepPipeline (line 75)
EEG = prepPipeline(EEG, params);

At the end of the pipeline the pdf Report coherently shows:

Reference errors: [prepPipeline failed postProcess: Reference to non-
existent field 'referenceOut'.
]

Do you have any idea about the possible cause of this error?
I've tried to "dig" into the different subfunctions but I haven't managed to find a solution yet.

The thing that puzzle me the most is that, as far as I can judge, the pipeline seems to correctly perform robust averaging and it is even able to remove bad interpolated channels (if specified).
But because of the aforementioned error messages, I don't know whether I can trust the completion of the process. What do you think?

Many thanks for your time and consideration.
Best regards,

Alberto Mariola

Interpolation prior to ICA

Dear PREP pipeline team

I have a question regarding the interpolation that happens based on robust average referencing.

Many researchers hold the view that interpolation of channels should not happen prior to ICA. What is your opinion about not interpolating prior to running ICA analysis? Does it differ from the rather conventional view as you implement interpolation prior to ICA, and if so, why?

Hence, it seems like I should remove interpolation in the post-process menu of PREP, would you agree?

Thank you in advance!

Channel locations missing after the pipeline

Hello, I was using the pipeline then did ICA, epoching and wanted to do further artifact rejection, which requires chanlocs.labels, but channel information seems removed after the pipeline. The original channel number was 132, it became 95 after the interpolation, but is there a way to know which channels got removed so I still have channel location information for artifact rejection and further analyses? Thank you!

performReference.m: 'interpoledChannels' field does not exist.

Hi,
I just tried out performReference.m using the 'pre-reference' mode.
Matlab throws an error at line 102:
Reference to non-existent field 'interpoledChannels'.

I guess this is just a typo and should be changed to interpolatedChannels.

Best,
Wanja

GUI issue with MATLAB 2016a

GUI portion seems to give error, probably to MATLAB's change in figure representation. We need to address.

Beginners help

I am attempting to run PREP mainly for the service of bad channel identification, removal, and interpolation. When I attempt to do this via GUI or command line on my 40 channel raw .set files, it does not seem to be working properly.

Within the matlab command window, I can see the plugin is doing things, but in the end, bad channels are not removed or interpolated (despite my checking that box in postprocessing or setting params.removeInterpolatedChannel = true). What am I missing?

Is there any example code I could see, or should the runVEPPrepPipeline in examples be sufficient?

Might there be some data prerequisite I am missing?

Anything you can do to help me would be greatly appreciated.

Many thanks!
Spencer Fix

bug in smooth average for overlapping windows in removeLinesMovingWindow ?

Hello,
I was looking at the code for the line noise removal and if I understand correctly you are doing the smooth average of two consecutive windows in this line where datafitwin should be the new window and datafitwin0 should be the old window. But in line 70 of the same function you already assign datafitwin to datafitwin0 so that the smooth average is computed with the first Noverlap samples and the last Noverlap samples both of the new window which would be wrong.
Here is the whole code snippet:

for n = 1:nw
        indx = winstart(n):(winstart(n) + Nwin - 1);
        datawin = data(indx);
         [datafitwin, f0Sig] = ...
            fitSignificantFrequencies(datawin, f0, lineNoise);
%         [datafitwin, f0Sig, FvalSig, aSig, fSig, sig] = ...
%             fitSignificantFrequencies(datawin, f0, lineNoise);
        datafitwin0 = datafitwin;
%         datafitWinSave(:, n) = datafitwin0;  % Debugging
%         f0SigSave{n} = f0Sig; % Debugging
%         FvalSigSave{n} = FvalSig;  % Debugging
%         aSigSave{n} = aSig;  % Debugging
%         fSigSave{n} = fSig;  % Debugging
%         sigSave{n} = sig;   % Debugging
        f0Mask = f0Mask | f0Sig;
        if n > 1
            datafitwin(1:Noverlap)= smooth.*datafitwin(1:Noverlap) + ...
                (1 - smooth).*datafitwin0((Nwin - Noverlap + 1):Nwin);
        end;
        datafit(indx, :) = datafitwin;
end

So I think the assignment to datafitwin0 should be after doing the smooth average.
I would appreciate if you could clarify this.