Comments (27)
Thank you, Satra!
First let's tackle the thickinthehead:
I just ran thickinthehead from within iPython and it worked just fine, outputting to the command line and to a table reasonable values. i also ran the following command:
mindboggle Twins-2-2 --ants /Users/arno/antsCorticalThickness/Twins-2-2/antsBrainSegmentation.nii.gz --thickness --no_surfaces -p 3
This command also completed without error, and the resulting ants thickinthehead table looks fine. When looking at the github repo, I see that nothing's changed in io_tables.py or ants.py (thickinthehead) or LABELS.py or LUT.py since November, and you're running a version of mindboggle post jan 5th, so I don't think it is due to any changes in the main mindboggle program either. I assume you ran the --all option, in which case I will run that next...
from mindboggle.
Just to be sure, have you tried running the most recent version of mindboggle?
from mindboggle.
i'm using this version:
7cecab8
Date: Sun Feb 8 21:29:43 2015 -0800
also it looks like some of the volumes do not contain the full assortment of labels.
img = nb.load('/om/scratch/Mon/satra/SAD/MBwork/SAD_020/Mindboggle/Volume_labels/FreeSurfer_cerebellum_ANTs_cerebrum/wmparc.mgz.nii.gz_to_ANTs_labels.nii.gz_to_ANTs_labels.nii.gz_through_combined_segmentations.nii.gz')
In [5]: np.unique(img.get_data())
Out[5]:
array([ 0., 7., 8., 46., 47., 2012., 2019., 2020.,
2027., 2028.], dtype=float32)
from mindboggle.
I believe that was an ephemeral bug that you happened to catch in your net!
You need a more recent version to squash it!!!
from mindboggle.
current master fails to run:
TypeError: relabel_surface() got an unexpected keyword argument 'hemi'
Interface Function failed to run.
from mindboggle.
I am sorry, Satra -- it should work now.
from mindboggle.
Exception: Some connections were not found
Module Surface_labels has no input called Reindex_labels.hemi
from mindboggle.
Yeesh -- I removed this second reference to hemi as well.
Please try again!
from mindboggle.
more errors!
>>> relabel_surface(vtk_file, old_labels, new_labels, erase_remaining, erase_labels, erase_value, output_file)
>>> # View
>>> plot_surfaces('relabeled_rh.labels.DKT31.manual.vtk')
"""
import os
import numpy as np
from mindboggle.utils.io_vtk import read_vtk, write_vtk
# Load labeled vtk surfaces:
faces, lines, indices, points, npoints, scalars, \
name, input_vtk = read_vtk(vtk_file, return_first=True, return_array=True)
new_scalars = scalars[:]
# Raise an error if inputs set incorrectly:
if (new_labels and not old_labels) or \
(len(old_labels) != len(new_labels)) or \
(erase_remaining and not old_labels):
raise IOError("Please check inputs for relabel_surface().")
the error above is raised with these inputs below
> args
vtk_file = /om/scratch/Mon/satra/mdd/MBwork/50250/Mindboggle/Surface_labels/_hemi_lh/FreeSurfer_annot_to_vtk/lh.aparc.vtk
old_labels = [1002, 1003, 1005, 1006, 1007, 1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031, 1034, 1035, 2002, 2003, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023, 2024, 2025, 2026, 2027, 2028, 2029, 2030, 2031, 2034, 2035]
new_labels = []
erase_remaining = True
erase_labels = [0]
erase_value = -1
output_file =
from mindboggle.
Great catch, Satra -- thank you! I have been using this function with an equal number of old and new labels, so I didn't catch the silly error in the conditional statement that checks if they're the same size -- this of course doesn't make sense if you don't specify new labels. Anyway, I am sorry to ask you to try again, but please try again!
from mindboggle.
Issue 1
this output is not getting any input: 'features.@fundus_per_sulcus': None,
Issue 2
and this file is still empty:
'tables.@thicknesses_of_ANTs_filled_cortex_labels': '/om/scratch/Mon/satra/mdd/MBwork/50207/Mindboggle/Volume_feature_shapes/ANTs_filled_cortex_label_thicknesses/thickinthehead_ANTs_labels.csv'
inputs to the routines that are generating this file is only seeing:
np.unique(nb.load('/om/scratch/Mon/satra/mdd/MBwork/50207/Mindboggle/Volume_labels/FreeSurfer_cerebellum_ANTs_cerebrum/wmparc.mgz.nii.gz_to_ANTs_labels.nii.gz_to_ANTs_labels.nii.gz_through_combined_segmentations.nii.gz').get_data())
Out[3]: array([ 0., 7., 8., 46., 47.], dtype=float32)
np.unique(nb.load('/om/scratch/Mon/satra/mdd/MBwork/50207/Mindboggle/Volume_labels/Extract_FreeSurfer_cerebella/wmparc.mgz.nii.gz').get_data())
Out[6]: array([ 0, 7, 8, 46, 47], dtype=int32)
and this is the empty chain
np.unique(nb.load('/om/scratch/Mon/satra/mdd/MBwork/50207/Mindboggle/Volume_labels/Combine_ANTs_cortex_noncortex_labels/ANTs_labels.nii.gz_to_ANTs_labels.nii.gz_through_combined_segmentations.nii.gz').get_data())
Out[15]: array([ 0.], dtype=float32)
np.unique(nb.load('/om/scratch/Mon/satra/mdd/MBwork/50207/Mindboggle/Volume_labels/Fill_cortex_with_ANTs_labels/ANTs_labels.nii.gz_through_combined_segmentations.nii.gz').get_data())
Out[18]: array([ 0.], dtype=float32)
np.unique(nb.load('/om/scratch/Mon/satra/mdd/MBwork/50207/Mindboggle/Volume_labels/Extract_ANTs_cortex_labels/ANTs_labels.nii.gz').get_data())
Out[21]: array([ 0.])
np.unique(nb.load('/om/scratch/Mon/satra/mdd/MBwork/50207/Mindboggle/Volume_labels/antsApplyTransforms/ANTs_labels.nii.gz').get_data())
Out[23]: array([ 0.])
np.unique(nb.load('/om/user/satra/projects/mdd/set1/MBcache/ef2b6f7116e05b46746c044bfa995546/OASIS-TRT-20_jointfusion_DKT31_CMA_labels_in_MNI152_v2.nii.gz').get_data())
Out[25]:
array([ 0., 4., 5., 6., 7., 10., 11., 12.,
13., 14., 15., 16., 17., 18., 24., 26.,
28., 30., 43., 44., 45., 46., 49., 50.,
51., 52., 53., 54., 58., 60., 91., 92.,
630., 631., 632., 1002., 1003., 1005., 1006., 1007.,
1008., 1009., 1010., 1011., 1012., 1013., 1014., 1015.,
1016., 1017., 1018., 1019., 1020., 1021., 1022., 1023.,
1024., 1025., 1026., 1027., 1028., 1029., 1030., 1031.,
1034., 1035., 2002., 2003., 2005., 2006., 2007., 2008.,
2009., 2010., 2011., 2012., 2013., 2014., 2015., 2016.,
2017., 2018., 2019., 2020., 2021., 2022., 2023., 2024.,
2025., 2026., 2027., 2028., 2029., 2030., 2031., 2034.,
2035.], dtype=float32)
so the antsApplyTransforms step is taking the right set of inputs and then generating empty labels.
here are the transforms i see, and that doesn't make sense:
stdout 2015-03-07T01:29:21.599079: 1. inverse of /om/user/satra/projects/mdd/set1/MBcache/f36e3d5d99f7c4a9bb70e2494ed7340b/OASIS-30_Atropos_template_to_MNI152_affine.txt (type = MatrixOffsetTransformBase)
stdout 2015-03-07T01:29:21.599079: 2. /om/scratch/Mon/satra/mdd/metaflow/50207/antsct/antsCT_TemplateToSubject0Warp.nii.gz (type = DisplacementFieldTransform)
stdout 2015-03-07T01:29:21.599079: 3. /om/scratch/Mon/satra/mdd/metaflow/50207/antsct/antsCT_TemplateToSubject1GenericAffine.mat (type = AffineTransform)
the template being used in antsct is nick's adni template. shouldn't there be a transform linking nick's adni template to the oasis-30-atropos template?
here is the complete antsCorticalThickness call and parameter log:
Runtime info
------------
* command : antsCorticalThickness.sh -a /om/scratch/Mon/satra/mdd/metaflow/50207/copy_header/50207_anatomical_hea
derFix.nii.gz -m /om/scratch/Mon/satra/mdd/metaflow/50207/antsct/T_template0_BrainCerebellumProbabilityMask.nii.g
z -e /om/user/satra/projects/mdd/scripts/templates/T_template0.nii.gz -d 3 -f /om/user/satra/projects/mdd/scripts
/templates/T_template0_BrainCerebellumExractionMask.nii.gz -s nii.gz -o antsCT_ -p nipype_priors/BrainSegmentatio
nPrior%02d.nii.gz -t /om/user/satra/projects/mdd/scripts/templates/T_template0_BrainCerebellum.nii.gz
* duration : 38415.91558
* hostname : node005.cm.cluster
Terminal output
stdout 2015-02-22T00:56:54.864568:WM[4]
stdout 2015-02-22T00:56:54.869143:WM 4
stdout 2015-02-22T00:56:54.888290:The output directory "antsCT_" does not exist. Making it.
stderr 2015-02-22T00:56:54.896795:
stderr 2015-02-22T00:56:54.896795:
stderr 2015-02-22T00:56:54.896795:
stderr 2015-02-22T00:56:54.896795:
stderr 2015-02-22T00:56:54.896795:
stderr 2015-02-22T00:56:54.896795: anatomical image = /om/scratch/Mon/satra/mdd/metaflow/50207/copy_
header/50207_anatomical_headerFix.nii.gz
stderr 2015-02-22T00:56:54.896795: brain template = /om/user/satra/projects/mdd/scripts/templates/
T_template0.nii.gz
stderr 2015-02-22T00:56:54.896795: convergence = [45,0.0,10]
stderr 2015-02-22T00:56:54.896795: debug mode = 0
stderr 2015-02-22T00:56:54.896795: extraction prior = /om/scratch/Mon/satra/mdd/metaflow/50207/antsc
t/T_template0_BrainCerebellumProbabilityMask.nii.gz
stderr 2015-02-22T00:56:54.896795: extraction reg. mask = /om/user/satra/projects/mdd/scripts/templates/
T_template0_BrainCerebellumExractionMask.nii.gz
stderr 2015-02-22T00:56:54.896795: float precision = 0
stderr 2015-02-22T00:56:54.896795: gradient step size = 0.025
stderr 2015-02-22T00:56:54.896795: image dimension = 3
stderr 2015-02-22T00:56:54.896795: max iterations = 100x100x70x20
stderr 2015-02-22T00:56:54.896795: metric = CC[fixedImage,movingImage,1,4]
stderr 2015-02-22T00:56:54.896795: output image suffix = nii.gz
stderr 2015-02-22T00:56:54.896795: output prefix = antsCT_
stderr 2015-02-22T00:56:54.896795: prior combinations = WM[4]
stderr 2015-02-22T00:56:54.896795: registration template = /om/user/satra/projects/mdd/scripts/templates/
T_template0_BrainCerebellum.nii.gz
stderr 2015-02-22T00:56:54.896795: regularization =
stderr 2015-02-22T00:56:54.896795: run quick = 0
stderr 2015-02-22T00:56:54.896795: segmentation prior = nipype_priors/BrainSegmentationPrior%02d.nii.g
z
stderr 2015-02-22T00:56:54.896795: smoothing sigma = 1.5
stderr 2015-02-22T00:56:54.896795: thickness prior = 10
stderr 2015-02-22T00:56:54.896795: transformation = SyN[0.1,3,0]
stderr 2015-02-22T00:56:54.896795: use random seeding = 1
from mindboggle.
Satra --
Issue 1: The --all option doesn't result in fundus extraction, so unless you've explicitly called --fundi, they won't be generated. Did you call --fundi?
Issue 2: Thank you so much for this detailed account! I will try to think clearly after an hour's sleep (app launch imminent). As you rightly point out when referring to the chain of commands, antsApplyTransforms is the step to focus on. Since the correct labels are in the joint fusion atlas, that leaves the transforms, which you point out as suspicious: "shouldn't there be a transform linking nick's adni template to the oasis-30-atropos template?" I am sure you are right. This is a command Nick Tustison used on ADNI data using his ADNI template that you are using:
antsApplyTransforms -d 3 -i 20080416_1407261284011361922076945140808101991208323033126ADNI912INNOMEDSTUDYs002a1001_BrainSegmentation.nii.gz -o test.nii.gz -r ~/Data/Public/ADNI/Templates/Normal/T_template0.nii.gz -t 20080416_1407261284011361922076945140808101991208323033126ADNI912INNOMEDSTUDYs002a1001_SubjectToTemplate1Warp.nii.gz -t 20080416_1407261284011361922076945140808101991208323033126ADNI912INNOMEDSTUDYs002a1001_SubjectToTemplate0GenericAffine.mat -n NearestNeighbor
If I am thinking clearly, then the right way to do this would be to have the antsCorticalThickness.sh arguments -e and -t point to the same (ADNI) template in MNI space, or to follow up the command with the application of such a transform. This is because mindboggle expects one to use a template in MNI space where the joint fusion atlas resides.
Looking back, I am embarrassed to have sent you an incomplete command in a 5/6/14 email:
Satra -- Here is the antsCorticalThickness.sh command you should run with Nick's ADNI template in $Template_path:
antsCorticalThickness.sh -d 3 -n 3 -w 0.25
-a IMAGE
-o PREFIX
-e ${Template_path}/T_template0.nii.gz
-t ${Template_path}/T_template0_BrainCerebellum.nii.gz
-m ${Template_path}/T_template0_BrainCerebellumProbabilityMask.nii.gz
-f ${Template_path}/OASIS-30_Atropos_template/T_template0_BrainCerebellumExtractionMask.nii.gz
-p ${Template_path}/Priors2/priors%d.nii.gz
This was a blunder on my part, because a couple of months earlier, I wrote to Nick regarding this very question:
"If you've already run [the longitudinal 3T ADNI] all through FS and antsCT with the [ADNI] template, then all that would be left would be to run them through Mindboggle, yes? Or, since you probably didn't run it with -e and -t pointing to the same template in MNI space, an extra transform would need to be computed, yes?"
to which Nick agreed. Did we correct this in your Mindboggle runs of the ADNI data? Otherwise, we would have gotten empty volume outputs, yes?
To come back to the present, why are you using the ADNI template as opposed to the OASIS-30 Atropos template for the MDD data? Because Nick's template is constructed from a more representative (older) age group of healthy people?
If the above is correct, then I see five ways to proceed:
- Just use the unaffected surface measures.
- Re-run mindboggle to do #1
from mindboggle -h:
--no_transforms no transforms (antsCorticalThickness.sh run
without -t, for example), so no volume outputs,
and no data registered to MNI152 space - Re-run antsCT with -e and -t options and re-run mindboggle.
- Transform the antsCT output data and re-run mindboggle.
- Revise mindboggle to accept such transforms.
What would you recommend?
from mindboggle.
@binarybottle - in the ants command above why are there two paths for templates? it looks like the -f option uses OASIS_atropos template? should i just use the OASIS atropos template for everything?
how do i download the OASIS atropos template? i see one one synapse, but i don't want to download a file at a time. is there a zip file somewhere?
also is it different from this oasis template?
http://figshare.com/articles/ANTs_ANTsR_Brain_Templates/915436
once you let me know where i can download the OASIS template, i'm just going to rerun with the OASIS template for the mdd and sad data. too much of a time pressure right now to get this down. my talk is in a week and a half and i have a bunch of other analyses to do.
from mindboggle.
As you confirmed, the OASIS Atropos templates on mindboggle's site and on figshare.com are the same. I just added a clear section at the top of the Mindboggle-101 webpage:
http://www.mindboggle.info/data.html#mindboggle-software-data
from mindboggle.
Satra -- regarding your original message:
"- the csv files are all strings (i.e. requires me to look and figure out what's a number, a label, and a list). if you are going to store numbers and lists, this should be json
- some of the csv files are not formatted consistently or properly, they have spaces with numbers"
I understand why json is a good fit for specifying the data types, but I don't know how many people in medical image analysis or statistics deal with json files. Everybody understands and can deal with csv tables. Three options going forward:
- generate both tables and json files
- separate list entries (spectra, moments, positions) from tables of numbers
- fix the csv tables so that they are very clear
from mindboggle.
if do_fundi:
NameError: name 'do_fundi' is not defined
Return code: 1
from mindboggle.
do_fundi
should be False rather than commented out.
from mindboggle.
now these files have all zeros: tables/left_surface/[sulcus|label]_shapes.csv
from mindboggle.
Yeesh -- clearly I should never quickly debug after an hour's sleep!
I just set do_fundi = False by default.
from mindboggle.
@binarybottle -- any thoughts on why these are all zero?
now these files have all zeros: tables/left_surface/[sulcus|label]_shapes.csv
from mindboggle.
I assumed that the bug I introduced (commenting out do_fundi) threw a monkey wrench into the works, so just to be sure I am starting clean, I am reinstalling everything from scratch right now and will run a test once ANTs is finished installing...
from mindboggle.
thanks!
from mindboggle.
As I wrote to you outside of GIthub, I realized that when I recently moved the mindboggle.info website to Github pages, I also moved the files that are accessed and cached by mindboggle. Fixing now...
from mindboggle.
You're going to get a laugh out of this, Satra. Remember those references to 'hemi' above? We needed them after all! I put them back in and now we have volume and surface labels restored.
from mindboggle.
still 0s in the surface tables.
i deleted my work directory, cache directory and output directory and reran mindboggle based on current master. i don't think you pushed your changes.
from mindboggle.
After all that, I didn't push? I need to be more pushy!
(pushed and bushed)
from mindboggle.
I am also seeing all 0s for the frontmarginal sulcus row in sulcus_shapes.csv for all subjects.
I am using version 1.3.8 in a singularity image. Freesurfer v6.0.0 was previously ran outside the container, while ants and mindboggle were ran from within the container. This happened in independent datasets.
from mindboggle.
Related Issues (20)
- visualization of eigenfunctions in laplace-beltrami operator HOT 1
- mindboggle looking for /mri/orig/001.mgz when /mri/T1.mgz is more appropriate for longitudinal FreeSurfer data?
- MIndboggle (latest) crashing on processing freesurfer 6 data HOT 7
- Mindboggle runtime error HOT 1
- Mindboggle as node of nipype pipeline HOT 4
- Shape table moments - how are they calculated? HOT 2
- New UnicodeEncodeError in pipeline (--thickinthehead) HOT 1
- how to install mindboggle package? HOT 4
- Data error HOT 2
- where to download the source data ? HOT 5
- runing error HOT 13
- OASIS-TRT-20-1 appears can't allocate memory in mindboggle dataset
- datasets update HOT 2
- Importing mindboggle HOT 1
- 📌 Pin versions of dependencies
- in singularity cannot execute binary file: Exec format error HOT 3
- Which file should be selected as the result of freesurfer automatic segmentation?
- running error HOT 1
- Some Mindboggle-101 dataset v3 labels.DKT31.manual+aseg.MNI152.nii.gz files contain labels 1000, 2000, 1001, 2001, 1032, 2032, 1033, and 2033 HOT 2
- Difference between DKT40 and DKT31 OASIS? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mindboggle.