QA

[bids-apps]

QA

[chrisgorgo]

It requires using a mini X server and I could not get it to work reliably under Docker and Singularity. It's an old and clunky solution - are there any other QA oriented FreeSurfer tools out there?

[fliem]

Mhh. I have a vague memory of @satra heading a past brainhack project about FS QA?

[satra]

@fliem - this depends on whether you are willing to create something from scratch or use some of the outputs in place.

for example, the brain is generally quite symmetric - so bad reconstructions often have abs(1 - L_struct/R_struct) > some number for each aparc/a2009 structure

but the things that need QA and are hard are:

• meninges part of cortical ribbon
• intensity bias affecting placement of surfaces in orbitofrontal cortex
• ripples caused by interaction between motion and fat shifts.
• outlines of subcortical boundaries (e.g., amygdala)

many of these should be detectable with enough training data.

[fliem]

Since surface plots now arrived in nilearn, and creating qc plots for freesurfer is such a drag, I suggest we revive this issue. As a starting point I have created a couple of plots in a notebook.
What do you think?

[chrisgorgo]

This looks great - looping in @effigies who did some surface QC for fmriprep. This is how it looks:

[alexlicohen]

I also came across this: https://github.com/akeshavan/mindcontrol Which might be work a look or forking some ideas from...

…

-Alex

On Mar 4, 2017, at 2:21 PM, Chris Filo Gorgolewski ***@***.***> wrote: This looks great - looping in @effigies who did some surface QC for fmriprep. This is how it looks: — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[fliem]

Those look very pretty. @effigies could you please point me towards that fmriprep code. Did you do other things for qc as well (like calculating metrics...)? Thanks.

[fliem]

I have refined the surface and subcortical plots, borrowing heavily from nilearn/niworkflows.
Next, I'll look into how to assemble them in htmls

[effigies]

Hi @fliem sorry that I totally missed this thread.

It looks like you found the visualization bit in niworkflows. We haven't really done any metrics at this point, though; we've reconstructed surfaces using ANTs skull-stripping instead of mri_watershed, and are now using bbregister to perform functional alignment. At present, my focus (nipreps/fmriprep#391) is on sampling functional data to the surface and ensuring that these surfaces align properly in native T1 space, so that they can be used smoothly in non-FreeSurfer tools.

[fliem]

I have created first examples for single subjects and group html reports. Any suggestions?

[chrisgorgo]

They look great, although the subcortical segmentation seem a little misaligned. I wonder if this would make more sense as a standalone python module and command line tool that could be reused in other projects.

…

On Mon, May 8, 2017 at 11:18 AM, Franz Liem ***@***.***> wrote: I have created first examples for single subjects and group html reports <https://github.com/fliem/freesurfer/tree/qc_nb/example_reports/00_qc>. Any suggestions? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOkpxGZiyhqEt7Bs_mgsBIDothPG40Lks5r31xzgaJpZM4LUmfj> .

[effigies]

I can't see the parcellation reports at all, either through Github or pulling locally and viewing...

But I do like @chrisfilo's idea of a separate module that would ease re-use in other projects. Since the base machinery for this is niworkflows-based, I think it'd be reasonable to pull this into niworkflows, itself. But I'd also entirely support an independent module that we then incorporate into ReconAllRPT.

[fliem]

• re misalignment: this was partially due to a bug in the code. However after fixing this, the contours still don't align perfectly. I have reverted to ROI overlays for now.
  

• new example reports are here

• Due to a nilearn issue with .add_contours, the white/pial-surface-plots show smaller brains than the subcortical overlays.

• Making it an own module or part of niworkflows sounds good to me. For the time being, I have created a command line tool as part of my freesurfer repo, and a docker container. If anyone wants to try it:

docker run --rm -ti \
-v /data/sourcedata:/data/in \
-v /data/freesurfer:/data/out \
--entrypoint=/code/run_qc.py \
fliem/freesurfer:qc_nb \
/data/in /data/out participant

[alexlicohen]

To view these without cloning, use rawgit.com https://cdn.rawgit.com/fliem/freesurfer/qc_nb/example_reports/00_qc/freesurfer_qc_group_report.html This looks pretty nice actually for scanning for large-scale FS failures. (For more detailed screening I think we'll still need the more detailed manual review and/or quantitative assessment like mindcontrol or mriqc

…

-Alex

---------------------------------------------------------------- Alexander Li Cohen, M.D., Ph.D. E-mail: [email protected] (Medical/Science Email) E-mail: [email protected] (Lifetime Email) ----------------------------------------------------------------

On Wed, May 10, 2017 at 9:18 AM, Franz Liem ***@***.***> wrote: - re misalignment: this was partially due to a bug in the code. However after fixing this, the contours still don't align perfectly. I have reverted to ROI overlays for now. [image: subcort_contours1] <https://cloud.githubusercontent.com/assets/5772811/25900037/57f2de4e-3592-11e7-832f-6d667897e7de.png> - new example reports are here <https://github.com/fliem/freesurfer/blob/qc_nb/example_reports/00_qc> - Due to a nilearn issue <nilearn/nilearn#1450> with .add_contours, the white/pial-surface-plots show smaller brains than the subcortical overlays. - Making it an own module or part of niworkflows sounds good to me. For the time being, I have created a command line tool as part of my freesurfer repo, and a docker container. If anyone wants to try it: docker run --rm -ti \ -v /data/sourcedata:/data/in \ -v /data/freesurfer:/data/out \ --entrypoint=/code/run_qc.py \ fliem/freesurfer:qc_nb \ /data/in /data/out participant — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIPa_4DZV1eRfZxxLEWJNSxnt-8l3o5vks5r4bkUgaJpZM4LUmfj> .

[satra]

@akeshavan - perhaps this can somehow be integrated with mindcontrol.

[effigies]

To be clear, the parcellation images are missing for everyone else, right? I'm seeing surfaces and segmentation only.

[alexlicohen]

Yes, I'm not seeing the cortical parcellations either.

…

-Alex

---------------------------------------------------------------- Alexander Li Cohen, M.D., Ph.D. E-mail: [email protected] (Medical/Science Email) E-mail: [email protected] (Lifetime Email) ----------------------------------------------------------------

On Wed, May 10, 2017 at 11:30 AM, Chris Markiewicz ***@***.*** > wrote: To be clear, the parcellation images are missing for everyone else, right? I'm seeing surfaces and segmentation only. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIPa_59EdmaoBYVrJuaJs7K7B4FzKpyJks5r4dgmgaJpZM4LUmfj> .

[fliem]

Sorry, I can't figure out why the parcellations won't show. Here's one of them:

[PeerHerholz]

Ahoi hoi gang,

should we revive this given the recent developments over at niworkflows ?

[satra]

this is an entire segment of our current nobrainer collaboration with the freesurfer folks to develop autoqc methods in general and with the short term goal for freesurfer specifically. initial goal discard samples that will not work "well" with freesurfer, medium term goal - modify the samples to run through freesurfer - long term goal make freesurfer faster and more robust to inputs.

this doesn't mean anybody should stop trying to improve manual quality verification. this will also be used in our distributed training mode, where freesurfer runs locally and the lab can help provide feedback as to the quality .

also kwyk can be used in the short term to generate a quality assessment metric for T1 scans in 30s on a GPU.

btw, if anyone is interested in joining the autoqc effort just drop me a line and i will keep you informed about when we meet next (likely in jan, i'm mostly offline over the next few weeks).

[PeerHerholz]

Thx for the update and pointers @satra!
I'm all for automatization and happy to not do QC manually (other than to train and inform a model), hehe.

Re the things in this thread: as far as I understood it, the idea would be to generate a BIDS App report (comparable to fmriprep and mriqc) focusing on FreeSurfer output QC, right? If so, the information generated through the autoqc methods could be included in this report besides some informative graphics, eh?

[satra]

he idea would be to generate a BIDS App report (comparable to fmriprep and mriqc) focusing on FreeSurfer output QC, right?

yes, and any useful thing created for the report itself could be used to improve autoqc.

If so, the information generated through the autoqc methods could be included in this report besides some informative graphics, eh?

yes. the autoqc stuff can benefit from the report and in turn benefit the report in the future.

[Shotgunosine]

This is an old issue, but I think outputting QA reports would still be a big improvement.

[Remi-Gau]

note that for QA based on freesurfer output there is this: https://github.com/Qoala-T/QC