Comments (9)
Thanks for clarifying!
The process works fine now, the only thing I would need still is the implementation of scrubbing into the pipeline.
I guess I will wait until this feature is integrated in the future and thanks for your work!
Best wishes,
Klemens
from c-pac.
Hi! Thank you for reaching out. There are a couple of things I want to check.
- Could you attach the nuisance regression section of your pipeline file? It looks like
generate_regressors
andingress_regressors
might both be switched on, which could cause a crash like this - Could you post one subject of your data config file, so I can verify the layout?
I don't really understand what this workflow is trying to achieve. It connects nuisance regression with an anatomical T1w image, which seems not intuitive to me. Is this intended or am I missing something?
The screenshot you posted indicates that nuisance_regressors_generation_T1
and ingress_regressors
are both being connected to the workflow, when we just want ingress_regressors
. The T1
just refers to the template used in functional registration (T1 or EPI).
from c-pac.
Oops I just saw that you did attach your pipeline config - I think I know what the issue is. There is a sub-category under ingress_regressors
where you can specify the name of the regressor strategy. For some reason, that isn't showing up by default in the pipeline files on github. I'll open an issue to fix that! The name in this section has to match one of the strategies above under Regressors
(only one strategy can run at a time). It's not intuitive so I am working on a new workflow for ingressed regressors as well as documentation for the current implementation!
ingress_regressors:
run: On
Regressors:
Name: HMP24_acompcorr_with_gsr
Columns: [global_signal, motion, a_comp_cor, white_matter, csf]
Let me know if that fixes your problem, or if you have any other questions!
from c-pac.
Hi,
thanks for the quick response and thanks for adding some more documentation, that is very much appreciated!
I tried now to add the name of the regressor and commented the second regressor out as suggested by you, but unfortunately that did not change the process and still leads to the same error.
If I look at my data_config_participants.yml
file, I am also not sure if that worked properly and might be responsible for the error?
I will attach the data_setting.yml
file also, could you scan through those files and let me know if I might have done something incorrect already there that would explain the issue?
For example, I tried to also add the path to the fmriprep freesurfer folders, but somehow this does not show up in the data_config
file. I don't know if that would be necessary for the fmriprep_ingress
pipeline though.
And just to be sure, I'll also attach a screenshot of my BIDS and fmriprep directory to doublecheck.
data_config_participants.txt
data_settings.txt
from c-pac.
Hi, thank you for providing more information! It looks like the issue is from the data config file. Unfortunately, the data config builder tool is not yet compatible with the derivatives_dir
option. That feature is currently in progress. Here is the documentation on how to format it with fmriprep ingress, and specific documentation for running nuisance regression on fMRIPrep outputs will be released in the next few weeks!
For example, based on the information you've given me, the data config should look like this for one subject:
- site: site-1
subject_id: 01
unique_id: 02
derivatives_dir: /bids_dir/derivatives/fmriprep/sub-01/ses-01
Let me know if you have any questions!
from c-pac.
Hi again, thanks for the further suggestions and the link. I completely missed that section.
I tried to use this minimal config file now but it still gives me the same error.
-
site: site-1
subject_id: 01
unique_id: 01
derivatives_dir: /bids_dir/derivatives/fmriprep/sub-01/ses-01
-
site: site-1
subject_id: 01
unique_id: 02
derivatives_dir: /bids_dir/derivatives/fmriprep/sub-01/ses-02
Is it correct that the pipeline file accesses the data config file and assumes it is located in the /configs folder from the docker command I am running?
Do you have some further ideas on where the error might be rooted?
And another question regarding this data_config file: How would the pipeline now know to only take the resting-state scans from the derivatives_dir? My full dataset has 3 scans per session and only one is resting state and should be further processed here with cpac.
Thanks for you help!
from c-pac.
Is it correct that the pipeline file accesses the data config file and assumes it is located in the /configs folder from the docker command I am running?
Oh I completely missed that!! You need to add a flag to indicate that you are using a data config file, otherwise C-PAC will just auto-generate one without the fMRIPrep directory. Your run command would look like this:
docker run -i --rm -v /nfs/DMT_HAR_MED_mri_data/BIDS/TEST_cpac/BIDS:/bids_dir -v /nfs/DMT_HAR_MED_mri_data/BIDS/TEST_cpac/cpac_output:/outputs -v /nfs/DMT_HAR_MED_mri_data/BIDS/TEST_cpac/tmp:/tmp -v /nfs/DMT_HAR_MED_mri_data/BIDS/TEST_cpac/cpac_output/config:/configs -v /nfs/DMT_HAR_MED_mri_data/BIDS/TEST_cpac/resources:/resources fcpindi/c-pac:latest /bids_dir /outputs participant --pipeline-file /configs/pipeline_config_fmriprep-ingress.yml --data_config_file /configs/<data_config_name>
My full dataset has 3 scans per session and only one is resting state and should be further processed here with cpac.
Unfortunately C-PAC doesn't filter out non-resting state data when pulling in an fMRIPrep directory. I'm sorry for the inconvenience and I will look into adding this functionality, since other users probably have the same concern!
from c-pac.
Hi, I got the command working now thanks to your help!
I still have a few questions you probably have the answers for before I start the pipeline for the whole dataset.
-
Since you said that C-PAC does not filter out non-resting state data, I was wondering if the flag
--bold-label
could be used to tell the pipeline to only look fortask-rest
for example. I tried the command with this flag, which wouldn't start the pipeline, but maybe I did not use it correctly? -
The timeseries extraction that I ran without defining any Atlas to use did run for every atlas that is built-in CPAC, at least it seemed so. Can I specify ROIs to not create as much output that I probably don't need?
-
For nuisance regression, the output files contain three columns, but I am not quite sure what is actually presented in those. Is there anywhere information on what is stored in these files. The accompanying json files were not exclusively relating to these
desc_confounds_timeseries.1D
files created. Here are the first few lines for reference:
# C-PAC 1.8.6
# Ingressed nuisance regressors:
1053.563041422457217777 1026.012326040201969590 1791.467862074797494643
1050.515425125942329032 1023.296357298717225603 1789.194473524351678861
1053.647751243849597813 1025.503930312144575510 1780.198806144095669879
-0.252886093174172877 -0.650269953506046772 -7.174069838742070715
Also, I was not sure if this part of the pipeline would actually ingress the regressors as intended, i.e. does Columns
refer to the column names in the fmriprep desc_confounds_timeseries.tsv
file and ingresses every column that has one of the listed strings in its name? Or if that is not the behavior, how does it work?
I also tried to rund the pipeline with Censoring turned on, but that somehow did not work. When I ran fmriprep, I used the flag --FD_treshold 0.5
and I would like to use associated columns in the desc_confounds_timeseries.tsv
for scrubbing. Do you know how I would specify this in the pipeline config file?
ingress_regressors:
run: On
Regressors:
Name: HMP24_acompcorr_with_gsr
Columns: [global_signal, motion, a_comp_cor, white_matter, csf]
-
Can I also do for example fsl's dual_regression directly with fMRIprep ingressed data. This analysis was not presented in the preconfigured pipeline template and I was wondering if that means that this wouldn't work or if it was just not shown in the template.
-
A last thing I am not quite sure is, how and when the pipeline would smooth the data. fMRIprep does no spatial smoothing as a last step, but it is often recommended to do that before doing further analyses.
So I if run this (last sections in the pipeline config file), ill smoothing only be done as the very last step or already before something else happens? And what would be the implications?
post_processing:
spatial_smoothing:
# Smooth the derivative outputs.
# Set as ['nonsmoothed'] to disable smoothing. Set as ['smoothed', 'nonsmoothed'] to get both.
#
# Options:
# ['smoothed', 'nonsmoothed']
output: ['smoothed', 'nonsmoothed']
Sorry for the very long post, but answers to these questions would be very useful to understand my outputs in the end.
Thanks a lot for your work!
from c-pac.
Awesome, glad to hear it's working! To answer your questions:
- The C-PAC workflow is very different when pulling in raw data vs. pulling in preprocessed data. The --bold-label flag is specifically for raw data.
- These are the atlases used by default. You can paste this list under timeseries_extraction > tse_roi_paths in your pipeline config and modify the list however you want.
- That file is just the pipeline-specific regressors pulled from the regressors tsv file produced in fMRIPrep. C-PAC parses the confounds tsv from fMRIPrep to just include the regressor columns you selected in your pipeline config. The names have to match exactly. Those regressors are then applied when nuisance regression runs. The columns should be labeled, though, which I will look into. This file is basically the same as the C-PAC generated regressors file described here. Also, scrubbing is not yet compatible with fMRIPrep output directory ingress - I apologize for any inconvenience.
- Yes, you can run dual regression on fMRIPrep outputs! Here is more information on how to configure it in the pipeline config.
- Thank you for pointing this out! There was a bug preventing spatial smoothing and z-scoring from running with fMRIPrep outputs. I have fixed this and it will be in the next software update, which will be released very soon. Smoothing and z-scoring run after all of the derivatives have been generated. Here is more information on spatial smoothing in C-PAC.
Let me knows how it goes!
from c-pac.
Related Issues (20)
- ✨ [User-requested Feature] Susceptibility Distortion Correction via SynBold-Disc HOT 2
- fmriprep-ingress regressor naming issue
- 🐛 `Image Exception : #3 :: Attempted to multiply images/ROIs of different sizes` HOT 1
- ✨ Generalize N4 bias correction option for all coregistration inputs
- ✨ [User-requested Feature] Freesurfer-Ingress without data config
- 🐛 [User-reported Bug] FileNotFoundError for output 'out_file' of a TShift interface HOT 1
- 🐛 not running multiple subjects simultaneously HOT 2
- 🚀 Release v1.8.7 HOT 1
- 🐛 [User-reported Bug] Not Reading In Slice Timing json in Bids Dataset
- 🐛 Freesurfer-based pipeline options cause a hang in a specific use case
- ♻️ Replace `logger.error; sys.exit` pattern with `raise SpecificException` pattern
- ♻️ DRY `isc.py` and `isfc.py`
- ♻️ DRY "Gathering the output file paths from" in `cpac_group_runner`
- 🎨 This `ts` isn't used anywhere. Do we care to assign it just to annotate, or should we drop this line?
- ♻️ Always or by default have Function nodes "as module"s?
- 🐛 PyPI's `bids_validator` does not include CLI
- 🐛 User-provided brain masks are not being read in to the resource list HOT 11
- 🐛 Functional preprocessing only pipeline not working HOT 2
- ♻️bids2table library for BIDS utils in CPAC, replacing and abstracting chunky utility functions
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from c-pac.