simexp/load_confounds

Load fMRIprep confounds in python

load_confounds

Load a sensible subset of the fMRI confounds generated with fMRIprep in python (Esteban et al., 2018).

⚠️ load_confounds is now a new feature in NiLearn 0.9.0. Development of this project will fully migrate to NiLearn. Please see the following links for the implementation. ⚠️

New API:

• nilearn.interfaces.fmriprep.load_confounds
• nilearn.interfaces.fmriprep.load_confounds_strategy

The usage examples:

• Temporal filtering with masker
• Extracting signals on a parcellation
• Example Gallery: create a connectome

Installation

Install with pip (Python >=3.5):

pip install load_confounds

TL;DR

Load confounds for a minimal denosing strategy commonly used in resting state functional connectivity. (Full motion parameters, WM/CSF signals, and high pass filter)

from load_confounds import Minimal
from nilearn.input_data import NiftiMasker

# load_confounds auto-detects the companion .tsv file (which needs to be in the same directory)
file = "path/to/file/sub-01_ses-001_bold.nii.gz"
confounds = Minimal().load(file)

# Use the confounds to load preprocessed time series with nilearn
masker = NiftiMasker(smoothing_fwhm=5, standardize=True)
img = masker.fit_transform(file, confounds=confounds)

It is also possible to fine-tune a subset of noise components and their parameters:

from load_confounds import Confounds
confounds = Confounds(strategy=['high_pass', 'motion', 'global'], motion="full").load(file)

You can check our tutorial on MyBinder for more info

Noise components

The following noise components are supported. Check the docstring of Confounds for more info on the parameters for each type of noise.

• motion the motion parameters including 6 translation/rotation (basic), and optionally derivatives, squares, and squared derivatives (full).
• high_pass basis of discrete cosines covering slow time drift frequency band.
• wm_csf the average signal of white matter and cerebrospinal fluid masks (basic), and optionally derivatives, squares, and squared derivatives (full).
• global the global signal (basic), and optionally derivatives, squares, and squared derivatives (full).
• compcor the results of a PCA applied on a mask based on either anatomy (anat), temporal variance (temp), or both (combined).
• ica_aroma the results of an idependent component analysis (ICA) followed by identification of noise components. This can be implementing by incorporating ICA regressors (basic) or directly loading a denoised file generated by fMRIprep (full).
• scrub regressors coding for time frames with excessive motion, using threshold on frame displacement and standardized DVARS (basic) and suppressing short time windows using the (Power et al., 2014) appreach (full).

Predefined strategies

Minimal

Minimal is suitable for data with minimal motion. Only includes motion parameters, wm and csf, with the option to add global.

Scrubbing

Like Minimal, but with scrubbing. Pros: Actual impact on data is pretty limited, but still good and offers the most control on what's being discarded. Cons: high loss of degrees of freedom, and messes up with the time axis in a way that may be difficult to handle for downstream analyses.

CompCor

CompCor includes anatomical or temporal compcor. The default is anatomical compcor with fully expanded motion parameters. Pros: large impact of denoising, efficient denoising, controlled loss of degrees of freedom. Cons: low control on what is being discarded (who knows what signal actually show up in the PCA for a given subject).

ICAAROMA

ICA-AROMA are only applicable to fMRIprep output generated with --use-aroma. Pros: pretty similar to CompCor, with better control of discarded components (those can be visually reviewed even though this is time consuming. Cons: may require retraining the noise detector and also requires to believe that ICA does efficiently separate noise from signal, which is not that clear, and the quality of separation may also vary substantially across subjects.

A note on nifti files and file collections

Note that if a .nii.gz file is specified, load_confounds will automatically look for the companion tsvconfound file generated by fMRIprep. It is also possible to specify a list of confound (or imaging) files, in which case load_confounds will return a list of numpy ndarray.

A note on low pass filtering

Low pass filtering is a common operation in resting-state fMRI analysis, and is featured in all preprocessing strategies of the Ciric et al. (2017) paper. fMRIprep does not output the discrete cosines for low pass filtering. Instead, this operation can be implemented directly with the nilearn masker, using the argument low_pass. Be sure to also specify the argument tr in the nilearn masker if you use low_pass.

A note on high pass filtering and detrending

Nilearn masker features two arguments to remove slow time drifts: high_pass and detrend. Both of these operations are redundant with the high_pass regressors generated by fMRIprep, and included in all load_confounds strategies. Do not use nilearn's high_pass or detrend options with these strategies. It is however possible to use a flexible Confounds loader to exclude the high_pass noise components, and then rely on nilearn's high pass filterning or detrending options. This is not advised with compcor or ica_aroma analysis, which have been generated with the high_pass components of fMRIprep.

A note on demeaning confounds

Unless you use the detrend or high_pass options of nilearn maskers, it may be important to demean the confounds. This is done by default by load_confounds, and is required to properly regress out confounds using nilearn with the standardize=False, standardize=True or standardize="zscore" options. If you want to use standardize="psc", you will need to turn off the demeaning in load_confounds, which can be achieved using, e.g.:

from load_confounds import Params6
conf = Params6(demean=False)

A note on the choice of strategies

We decided to focus our strategy catalogue on a reasonable but limited set of choices, and followed (mostly) the Ciric et al. (2017) reference. However, there are other strategies proposed in benchmarks such as (Parkes et al. 2018, Mascali et al. 2020). Advanced users can still explore these other choices using the flexible Confounds API, which can be used to reproduce most denoising strategies in a single short and readable command.

A note on denoising benchmarks

There has been a number of benchmarks you may want to refer to in order to select a denoising strategy (e.g. Ciric et al., 2017; Parkes et al. 2018; Mascali et al., 2020; Raval et al., 2020). However, a number of caveats do apply and the conclusions of these studies may not directly apply to load_confounds strategies. First, the noise regressors generated by fMRIprep do not necessarily follow the same implementations as these papers did. For example, the way load_confounds implements scrubbing is by adding regressors, while Ciric et al. (2017) excluded outlier time points prior to regressing other confounds. There are also other aspects of the fMRI preprocessing pipelines which are not controlled by load_confounds. For example, Ciric et al. (2017) did apply image distortion correction in all preprocessing strategies. This step is controlled by fMRIprep, and cannot be changed through load_confounds.

A note about ICA-AROMA denoising

ICA-AROMA related strategies are only applicable to fMRIprep output generated with --use-aroma. The approach predefined in load_confounds is the non-aggressive apporach, and the recommanded way of applying ICA-AROMA. fMRIprep produces files with suffix desc-smoothAROMAnonaggr_bold. Other noise regressors needed are retrieved by the predefined strategy in load_confounds. For details of the implementation, please refer to the documentation of load_confounds.ICAAROMA.

The aggressive approach was described in Pruim et al. (2015) and achieve denoising in one step by load_confound. Noise independent components along with other source of noise are included in confound regressors. The aggressive approach must be applied to the regular minimally processed fMRIprep output suffixed desc-prepro_bold. The name "aggressive" reflects that this approach doesn't consider the potential good signals regressed out by the noise independent compoenents. Please refer to table Recreating strategies from Ciric et al. 2017 for the relevant options.

Recreating strategies from Ciric et al. 2017

load_confounds can recreate the following strategies. The following table highlights the relevant options:

Strategy	high_pass	motion	wm_csf	global	compcor	ica_aroma	scrub
Params2	x		basic
Params6	x	basic
Params9	x	basic	basic	basic
Params9Scrub	x	basic	basic				full
Params24	x	full
Params36	x	full	full	full
Params36Scrub	x	full	full				full
AnatCompCor	x	full			anat
TempCompCor	x				temp
ICAAROMA	x		basic			full
AROMAGSR	x		basic	basic		full
AggrICAAROMA	x		basic	basic		basic

Funding

Development of this library was supported in part by the Canadian Consortium on Neurodegeneration in Aging (CCNA) and in part by the Courtois Foundation.

References

Behzadi Y, Restom K, Liau J, Liu TT. A component based noise correction method (CompCor) for BOLD and perfusion based fMRI. Neuroimage. 2007. doi:10.1016/j.neuroimage.2007.04.042

Ciric R, Wolf DH, Power JD, Roalf DR, Baum GL, Ruparel K, Shinohara RT, Elliott MA, Eickhoff SB, Davatzikos C., Gur RC, Gur RE, Bassett DS, Satterthwaite TD. Benchmarking of participant-level confound regression strategies for the control of motion artifact in studies of functional connectivity. Neuroimage. 2017. doi:10.1016/j.neuroimage.2017.03.020

Esteban O, Markiewicz CJ, Blair RW, Moodie CA, Isik AI, Erramuzpe A, Kent JD, Goncalves M, DuPre E, Snyder M, Oya H, Ghosh SS, Wright J, Durnez J, Poldrack RA, Gorgolewski KJ. fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat Meth. 2018. doi: 10.1038/s41592-018-0235-4

Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proceedings of the National Academy of Sciences. 2005; doi: 10.1073/pnas.0504136102.

Mascali, D, Moraschi, M, DiNuzzo, M, et al. Evaluation of denoising strategies for task‐based functional connectivity: Equalizing residual motion artifacts between rest and cognitively demanding tasks. Hum Brain Mapp. 2020; 1– 24. doi: 10.1002/hbm.25332

Parkes, L., Fulcher, B., Yucel, M., & Fornito, A. (2018). An evaluation of the efficacy, reliability, and sensitivity of motion correction strategies for resting-state functional MRI. NeuroImage, 171, 415-436. doi: 10.1016/j.neuroimage.2017.12.073

Power JD, Mitra A, Laumann TO, Snyder AZ, Schlaggar BL, Petersen SE. Methods to detect, characterize, and remove motion artifact in resting state fMRI. Neuroimage 2014 84:320-41. doi: 10.1016/j.neuroimage.2013.08.048

Pruim, R. H., Mennes, M., van Rooij, D., Llera, A., Buitelaar, J. K., & Beckmann, C. F. (2015). ICA-AROMA: A robust ICA-based strategy for removing motion artifacts from fMRI data. Neuroimage, 112, 267-277. doi: 10.1016/j.neuroimage.2015.02.064

V. Raval, K. P. Nguyen, C. Mellema and A. Montillo, "Improved motion correction for functional MRI using an omnibus regression model," 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), 2020, pp. 1044-1047, doi: 10.1109/ISBI45749.2020.9098688.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

François Paugam 🚇 💻 👀 ⚠️ 🔣	HanadS 💻 ⚠️ 🔣 🚇 📖 🤔	Elizabeth DuPre 🤔	Hao-Ting Wang 🤔 💻 🔣 📖 ⚠️ 🐛	Pierre Bellec 💻 🐛 🤔 🚇 ⚠️ 🔣 📋 🚧 📆	Steven Meisler 🐛 ⚠️ 🔣 💻 📖 🤔	Chris Markiewicz 🤔
Shima Rastegarnia 🐛	Thibault PIRONT 💻	m-w-w 📖

This project follows the all-contributors specification. Contributions of any kind welcome!

load_confounds's Issues

__main__ is not doing the right thing

I don't think it makes sense to have a __main__ method as load_confounds is designed to load data in a python environment, and not be executed through the command line.

add a pypi badge

confound_loader vs load_confounds

I think we should use the same name for the library and the main method. I like load_confounds much better.

only expose well known strategies

Of the strategies currently listed, only "minimal" is a true preprocessing strategy.
The others are "private" strategies, used internally to assemble more complex strategies.
They should not be listed in the documentation of the main public method.

add support for scrubbing.

bump version number to 0.5.dev

the tip of master is not version 0.5 anymore, this should be reflected in the version number.

implement standard strategies

We should try to replicate the strategies found in this paper:
https://www.sciencedirect.com/science/article/pii/S1053811917310972

make 0.2 release with multiple confounds

add initial volume suppression to reach steady state

this would be implemented just like scrubbing.

See this issue:
courtois-neuromod/cneuromod_docs#23

refactor `_confound_strat`

First the name is not really self-explanatory. Maybe _get_confounds_columns as this is literally what it does.

Second I suggest to have a single line of documentation, as this is a private function.
The parameters are redundant with other functions and do not need to be documented again here.

"""Get a list of all confound columns that fit the strategy."""

Third, the code uses an in-line double for loop with an if statement.
https://github.com/SIMEXP/load_confounds/blob/master/load_confounds.py#L51-L57
That is hard to read, and not compact.
I suggest using a regular double for loop with an if statement.
Also add a comment to explain what is being done here: the function grabs all columns in the raw confounds that contains a string from the confounds dictionary.

Add instructions on how to install and load the package in the Readme

add support for ICA-AROMA

make a 0.1 release.

check the motion reduction

it seems to be selecting very few components

readability of noise components

This section of the code is used to filter different regressors from the raw confounds:
https://github.com/SIMEXP/load_confounds/blob/master/load_confounds.py#L9-L31

It is however not straightforward to understand how this dictionary is structured, and unless you parse the rest of the code it is very hard to follow what is going on.

To increase readability it may be better to have dedicated functions like

_load_motion
_load_high_pass
_load_masks

and then a separate function to establish which components to add in the model based on the strategy and a few flags.

fill in project description on pipy

https://pypi.org/project/load-confounds/

write a function that creates a string of parameters for testing

add pip configuration files

return an error if confounds are missing

because the confounds list in fmriprep changes across versions, it could be that a strategy requires confounds that are absent in the file.

This should raise an error.

create a contributing.md file

we want people to feel welcome proposing changes. The tedana project has a great contributing file
https://github.com/ME-ICA/tedana/blob/master/CONTRIBUTING.md

Consider adapting it for fmriprep_load_confounds. Simplify a bit maybe though, because this project is pretty small, and nowhere as ambitious as tedana.

support multiple confounds

some procedures (eg nilearn's group dictionary learning) require to pass a list of datasets. It would be useful to support such a list, and return a list of confound arrays.

set up circleCI

refactor load_confounds.py to follow object oriented principles

adapt code to work for other types of confound tsv files, particularly different high_pass_filter and compcor parameters

add an intercept to the regression model

Heads up for potential name changes

We're going to be going through and conforming fMRIPrep derivatives to the derivatives spec soon, so it's possible that derivatives may change. It might be useful to have some functionality to switch strategies based on fMRIPrep version, if that doesn't alrleady exist.

format doc strings according to nilearn's standards

handle csf and white matter derivatives and powers

test strategies Params36 AnatCompCor and TempCompCor with nilearn

tests currently check that the regressors are properly filtered, but not that they can be fed to nilearn.

test denoising through nilearn

we should simulate a 4D dataset with known confounds, and check that nilearn indeed removes the confounds.

add test using confound files featuring all possible confounds

Many confounds are only calculated optionnally by fmriprep. It would be useful to have a file with all available options for testing, and adding new supported denoising strategies such as aroma.

switch name back to load_confounds

as the name on pip is load_confounds it would be easier to use it for the repo name too.

reorganize folders following nilearn standards

update setup.py too

automatically detect single vs multiple confounds

Instead of introducing a new method load_confounds_multi, it would be more elegant to test if the tsv file provided by the user is a string (in which case a single confound array is loaded) or an array (in which case the method would iterate on all inputs).

remove pandas dependencies

compor strategy to be implemented with anatomical, temporal and basic options

make a 0.6 release

weird verbose

The loader says 6params ??

high_pass vs low_pass

it's not clear how to deal with high_pass vs low_pass. At the moment we are only working from an example with high_pass regressors, but those should be controlled separately.

repo takes forever to clone

I suspect there is still data saved in there.
Would be useful to "doctor" the git history to get rid of this.

add testing for `pca_motion` and change that parameter

to be consistent with n_compcor, pca_motion should be renamed n_motion, and should have the following behaviour
n_motion=0 (default) no PCA is applied
0 < n_motion < 1 select the number of components by the percentage of explained variance.
1 < n_motion directly select the number of compontents (but select at most the 6, 12 or 24 available in the model, with a warning if the number specified by the user is higher than that).

add README with minimal documentation, including usage examples

hard-coded preprocessing strategy

this line will only work for some options in the fmriprep pipeline:
https://github.com/SIMEXP/load_confounds/blob/master/load_confounds.py#L212

It should be replaced by a general pattern, e.g. scrapping everything after _space-

organize pytest and make more readable

add testing for errors

re-write tests in a more compact format

The tests currently are a long list of variations on the same snippet, combining certain parameters. It would be possible to replace all the test by one nested for loop, with a list of parameters to be tested, and a list of expected confound labels associated with each parameter.

name, default and doc for `n_components`

the documentation of that parameter currently lives in a private function. It should be moved and integrated in the documentation of the main load_confounds function.

Also, the name of that parameter is not explicit enoough.
It is the number of motion components, so it should be called n_motion rather than n_component

The default is currently 0.95 but PCA reduction of motion parameters is not a standard preprocessing strategy. It should be set at 0 by default, i.e. use all the motion regressors.

name of private function `_load_confounds_helper` does not match what it does

the aim of this private function is to find the confounds associated with a particular imaging dataset. The name should reflect that. Maybe _get_confounds_files ?

support nifti file names

when working with nilearn, it will often be the case that users have built a list of nifti file names they want to load. It would be useful to let them pass such list directly to the confound loader. As fmriprep uses a systematic naming schemes, we could automatically search for the corresponding confound files. We may also support niftiimage object, which carry the file name in their attributes. https://nilearn.github.io/manipulating_images/input_output.html

This way users would simply need to pass their list of imgs to get back the confounds arrays, ready to be fed in a NiftiMasker or any nilearn function with a confound argument.

simplify dependencies

from what I see, the library uses only pandas and sklearn.
But requirements.txt list lots of different dependencies.