wanderine / broccoli Goto Github PK
View Code? Open in Web Editor NEWBROCCOLI: Software for Fast fMRI Analysis on Many-Core CPUs and GPUs
License: GNU General Public License v3.0
BROCCOLI: Software for Fast fMRI Analysis on Many-Core CPUs and GPUs
License: GNU General Public License v3.0
on ds000009 R2.0.1 - sub-29
docker run -ti -v D:\:/d bids/broccoli:v1.0.1 /d/data/ds000009_R2.0.1/ /d/scratch/broccoli participant --participant_label 29
Running the BIDS validator for the dataset
1: Not all subjects/sessions/runs have the same scanning parameters. (code: 39 - INCONSISTENT_PARAMETERS)
/sub-01/func/sub-01_task-balloonanalogrisktask_bold.nii.gz
/sub-02/func/sub-02_task-balloonanalogrisktask_bold.nii.gz
/sub-03/func/sub-03_task-balloonanalogrisktask_bold.nii.gz
/sub-04/func/sub-04_task-balloonanalogrisktask_bold.nii.gz
/sub-05/dwi/sub-05_dwi.nii.gz
/sub-06/func/sub-06_task-balloonanalogrisktask_bold.nii.gz
/sub-07/func/sub-07_task-balloonanalogrisktask_bold.nii.gz
/sub-09/func/sub-09_task-balloonanalogrisktask_bold.nii.gz
/sub-10/func/sub-10_task-balloonanalogrisktask_bold.nii.gz
/sub-11/func/sub-11_task-balloonanalogrisktask_bold.nii.gz
... and 14 more files having this issue (Use --verbose to see them all).
Summary: Available Tasks: Available Modalities:
855 Files, 6.07GB discounting T1w
24 - Subjects balloon analog risk task inplaneT2
1 - Session emotional regulation dwi
stop signal bold
Doing first level analysis
bids_dir is /d/data/ds000009_R2.0.1/, output_dir is /d/scratch/broccoli, analysis type is participant, participants are 29
Task names are
balloonanalogrisktask
discounting
emotionalregulation
stopsignal
Analyzing subject sub-29
Analyzing subject sub-29 task balloonanalogrisktask
Number of runs is 1
Creating '/d/scratch/broccoli/sub-29/balloonanalogrisktask/cond_run1_BALOON.txt'...
BROCOLLI exist with a non-zero exit code without producing the expected analysis outputs.
Hi Anders,
Thanks for all the hard work put into BROCCOLI! I have an issue that's similar to one that I contacted you about a while ago. I have a time series dataset of the bowel (NIFTI and .mat). I'm registering all volumes to the last volume (vol. 35) using BROCCOLI, but the registration doesn't seem to work. The output appears to be unchanged from the input, and there are no error messages. I did the registration using FSL as well for comparison, and that worked perfectly. The issue last time was the small signal intensities I was using (10^-14 to 10^-7). This dataset ranges from 10^-9 to 10^-6, and I've tried multiplying all signals by 10^7 as well as stretching out the min and max values to adjust the contrast, but nothing seems to help. Do you mind taking a look to see what might be wrong?
Thank you!
Satyam
Dear Anders,
The nipype nodes provided with BROCCOLI work fine when executed on their own. Ideally, however, one would want to include them along-side with other nodes (e.g. FSL nodes) in nipype's workflow objects. Unfortunately, running such workflows fails because any downstream nodes do not receive any input from earlier BROCCOLI nodes. For example smoothing a functional image and then performing maths on the results as described here would fail:
`
from nipype.interfaces import broccoli, fsl
from nipype import Node, Workflow
import os
nii_path = '/path/to/some/functional_data.nii'
sm_node = Node(broccoli.Smoothing(fwhm=8, platform=0, device=2), name='sm_node')
fslmaths_node = Node(fsl.ImageMaths(op_string='-sub .2'), name='fslmaths_node')
wf = Workflow(name='broccoli_smoothing_test')
wf.connect([(sm_node, fslmaths_node, [('smoothed_file', 'in_file')])])
wf.inputs.sm_node.in_file = nii_path
wf.base_dir = os.path.expanduser('~/broccoli_smoothing_test')
wf.run()
`
This will produce this error:
"ValueError: ImageMaths requires a value for input 'in_file'. For a list of required inputs, see ImageMaths.help()"
because the smoothed image, which the Smoothing node was supposed to create, does not exist although BROCCOLI does not report any errors.
This error exists both on macOS and Ubuntu.
What can be done about this?
Thanks and best,
Michael
I was wondering if it would be possible to allow saving the uncorrected results after running RandomiseGroupLevel, as in FSL. One could use this as input for a different multiple comparison correction. Many thanks!
Currently the task name is hardcoded as rhymejudgement
. It would be better to iterate over all tasks for each subject:
https://github.com/wanderine/BROCCOLI/blob/master/code/bids/broccolipipeline.sh#L137
I tried to run GetOpenCLInfo
but it is not able to find the clblas library. I installed AMD APP SDK 3.0, Boost, ACML
I want to know how to install from this ppa
https://launchpad.net/ubuntu/+source/clblas
I tried the .deb file but it said dependency not satisfied in synaptics package manager. I am not very familiar with ubuntu so I would rather not go through the cmake route. Something like apt-get would be great for me.
Thank you
Authored by K.A. Eklund
fMRI data size: 64 x 64 x 13586 x 1
fMRI voxel size: 3.500000 x 3.500000 x 3.500000 mm
fMRI TR: 1.980000 s
fMRI slice order: Seqential increasing
T1 data size: 155 x 208 x 190
T1 voxel size: 1.000000 x 1.000000 x 1.000000 mm
MNI data size: 182 x 218 x 182
MNI voxel size: 1.000000 x 1.000000 x 1.000000 mm
Number of original GLM regressors: 1
Number of total GLM regressors: 5
Number of contrasts: 1
Performing registration between T1 and MNI
Performing registration between fMRI and T1
Segmentation fault (core dumped)
Thank you for the fantastic work on BROCCOLI. I am still new at this toolbox and migrating from fsl's Feat.
I am able to run FirstLevelAnalysis
but I cannot get the permutations to work. Every time I use the -permute
switch I get the error that permutation testing is currently turned off
. If I do not use the -pemute switch the command runs and dhe resulting copes and betas look fine. Perhaps I am not using the correct syntax or completely misunderstanding the use of -permute
in the scope of First level analyses.
Thanks in advance for your help.
FYI, my command is:
FirstLevelAnalysis /misc/mansfield/lconcha/TMP/circe/forBroccoli/frs404.nii.gz \
/misc/mansfield/lconcha/TMP/circe/forBroccoli/brs404.nii.gz \
/home/inb/lconcha/fmrilab_software/fsl_5.0.6/data/standard/MNI152_T1_2mm_brain.nii.gz \
outbase_regressors.txt \
outbase_contrasts.txt \
-smoothing 5 \
-saveall \
-output outbase_brocproc \
-verbose \
-savedesignmatrix \
-saveoriginaldesignmatrix \
-regressmotion \
-permute \
-permutations 1000 \
-inferencemode 1 \
-cdt 2.3
Permutation testing is currently turned off!
Cheers,
Luis
Device info
---------------------------------------------
Platform number: 0
---------------------------------------------
Platform vendor: NVIDIA Corporation
Platform name: NVIDIA CUDA
Platform extentions: cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll
Platform profile: FULL_PROFILE
---------------------------------------------
---------------------------------------------
Device number: 0
---------------------------------------------
Device vendor: NVIDIA Corporation
Device name: Quadro K2000
Hardware version: OpenCL 1.1 CUDA
Software version: 304.135
OpenCL C version: OpenCL C 1.1
Device extensions: cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64
Global memory size in MB: 2047
Size of largest memory object in MB: 511
Global memory cache size in KB: 32
Local memory size in KB: 48
Constant memory size in KB: 64
Parallel compute units: 2
Clock frequency in MHz: 954
Max number of threads per block: 1024
Max number of threads in each dimension: 1024 1024 64
Just discovered this project through the frontiers article and am very impressed with the speed you've managed to achieve with OpenCL. However, given the lack of recent activity in the repo, I'm concerned that the funding or interest for maintaining/refining the project has died out, making it potentially a bad time investment to learn. If there is still a will and intent to keep working on BROCCOLI (which I really hope there is, being able to do first-level analyses in minutes instead of days would be amazing), is there a development roadmap that could be made public to reassure us that the project has a future?
If funding is an issue, I'd imagine a little publicity on some neuro blogs and a proper website with some simple tutorials would help get more people interested in the project, including people who would happily contribute code/donations.
Cheers,
Hello,
I would like to ask whether you plan on adding TFCE support in RandomiseGroupLevel? It is specified in the documentation and command-line arguments, but says it is not yet implemented.
(sponsor is sense of uploads to Debian ;) )
just wanted to reach out. unfortunately don't have time bandwidth to dive into it myself ATM
Cheers,
Check and update functions for permutation testing at the single subject level
Hi, I was wondering if it is possible to control for permutations within groups/subjects. This is possible in FSL randomise by adding -e design.grp file, which basically restrict randomisation within a group (so for repeated measures you need to set a different group identifier (e.g. number) for each subject).
many thanks
Ido
Hello.
I'm trying to run first level GLM. Everything works great until I try to add dummy regressors for subjects that did not complete a task. I followed the documentation and created files with NumEvents 0 and added these to the design_file.
When I add these dummy regressors, and only when I add these, all beta's, copes, and tscores files are empty images.
I checked both original and total design matrix files and they seem fine in both cases, with or without empty regressors. Columns for empty regressors are filed with 0, as expected.
The issue seem to affect both GLM and FIRSTLEVEL analysis function.
I use the linux version of the script on Ubuntu 12.04.
Hi,
1: In NITRC BROCCOLI is specified as ' Operating System: Windows, POSIX/UNIX-like'.
How I can get it run in Windows?
2: Running RandomiseGroupLevel I got the message 'TFCE is currently disabled'. Will it be enabled again? If so, when?
Best,
Bertram
Notice this is the same issue I opened for BIDS-Apps/BROCCOLI (issue #10); I don't think the problem is specific to the Docker version, but please update Docker as well after this issue is resolved:
Trying to run BROCCOLI docker on a BIDS dataset with multiple runs creates the condition *.txt files and then I assume it tries to do something (no logs during that time) and then crashed with:
/Downloads/BROCCOLI/code/bids/broccolipipeline.sh: line 4: 323 Segmentation fault FirstLevelAnalysis -runs ${num_runs} ${bold_files} ${output_dir}/${subject}/${subject}_T1w_brain.nii.gz /usr/local/fsl/data/standard/MNI152_T1_2mm_brain.nii.gz ${regressor_files} ${output_dir}/${subject}/${task_name}/contrasts.txt -output ${output_dir}/${subject}/${task_name}/${subject} -device 0 -savemnimask -saveallaligned -savedesignmatrix -saveoriginaldesignmatrix -regressmotion
I used BIDS dataset ds001 (I downloaded it from https://drive.google.com/drive/folders/0B2JWN60ZLkgkMGlUY3B4MXZIZW8)
This issue happens with both bids/broccoli:v1.0.0 and bids/broccoli:latest (which I believe should be v1.0.1)
Hello! I have tried to install Broccoli under Ubuntu 16.04 with an AMDgpu driver, but it fails to compile the following error:
Authored by K.A. Eklund
Data size: 91 x 109 x 91 x 23
Number of permutations: 5000
Number of regressors: 1
Number of contrasts: 1
Performing 1 t-tests
Mean design detected for t-contrast 1
Max number of sign flips is 8.38861e+06
Initialization error is "Unable to create a command queue for the selected device and platform."
OpenCL error is "CL_OUT_OF_HOST_MEMORY"
OpenCL initialization failed, aborting!
See buildInfo* for output of OpenCL compilation!
My OpenCL seems to be installed on platform 0 and device 0:
Device info
Device vendor: Advanced Micro Devices, Inc.
Device name: Oland
Hardware version: OpenCL 1.2 AMD-APP (2527.3)
Software version: 2527.3
OpenCL C version: OpenCL C 1.2
Device extensions: cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_image2d_from_buffer cl_khr_spir cl_khr_gl_event
Global memory size in MB: 1619
Size of largest memory object in MB: 1169
Global memory cache size in KB: 16
Local memory size in KB: 32
Constant memory size in KB: 64
Parallel compute units: 6
Clock frequency in MHz: 1050
Max number of threads per block: 256
Max number of threads in each dimension: 1024 1024 1024
Then I tried Docker and it failed to download the dropbox files.
I will be happy if you can help me running the Broccoli on either Ubuntu 16 or under the dropbox container.
Hi Anders,
Thanks for developing this cool package. I am trying to compile the Matlab wrapper on Mac and I notice that you guys never got to do it. I suspect there is a reason that I am missing right now? For one thing, I don't seem to find the OPENCL_INCLUDE_DIRECTORY1 and 2.
Any help would be much appreciated.
Thanks!
v
Add support for slice timings file, with arbitrary slice timings
Warning: You are using gcc version "4.6.3-1ubuntu5)". The version
currently supported with MEX is "4.4.6".
For a list of currently supported compilers see:
http://www.mathworks.com/support/compilers/current_release/
Do you use the gcc-4.4 and g++-4.4?
I tried gcc-4.4 and g++-4.4 and got the following:
In file included from ../code/Matlab_Wrapper/FirstLevelAnalysis.cpp:20:
../code/Matlab_Wrapper/help_functions.cpp:1: fatal error: broccoli_lib.h: No such file or directory
compilation terminated.
mex: compile of ' "../code/Matlab_Wrapper/FirstLevelAnalysis.cpp"' failed.
Error using mex (line 206)
Unable to complete successfully.
Error in FirstLevelAnalysis_MixedGamblesTask (line 44)
mex ../code/Matlab_Wrapper/FirstLevelAnalysis.cpp -lOpenCL -lBROCCOLI_LIB
-I/usr/local/cuda-5.0/include/ -I/usr/local/cuda-5.0/include/CL -L/usr/lib
-I/home/andek/Research_projects/BROCCOLI/BROC
And how do you update the addPath and basePath?
Hi Anders,
I would like to use RegisterTwoVolumes to coregister a whole 4D time series of functionals with an anatomical scan. However, the resulting images "...aligned_linear.nii.gz" and "...aligned_nonlinear.nii.gz" are just 3D.
Is there an easy way to do that in BROCCOLI? Or is it possible to output the linear/non-linear transformations from one call to RegisterTwoVolumes and then apply them to another 4D series?
Cheers,
Michael
I am trying to use BROCCOLI to speed up permutations testing for TBSS on skeletonized WM data. I was able to run permutations testing successfully on my platform and device (AMD GPU) with your sample data:
BROCCOLI/code/testing_scripts/randomise/betas49.nii.gz
However, when I now try with my own 4D NIFTI file consisting of FA data registered to a group-derived template, I get an error stating that there are 0 possible permutations, even when just attempting -groupmean (which is meaningless for FA data). It does read in the correct dimensions of my file, and does find the correct # of subjects (n = 84) in the 4th dimension.
Command:
RandomiseGroupLevel all_FA_skeletonised.nii.gz -groupmean -platform 0 -device 0 -permutations 10000 -output tbss_parallel -verbose
Output:
Warning: No mask being used, doing permutations for all voxels.
It took 64.409843 seconds to read the nifti file(s)
Warning: Number of possible sign flips for group mean is 0, but 10000 permutations were requested. Lowering number of permutations to number of possible sign flips.
Authored by K.A. Eklund
Data size: 176 x 512 x 512 x 84
Number of permutations: 0
Number of regressors: 1
Number of contrasts: 1
Here also are info output on my dataset:
fslinfo all_FA_skeletonised.nii.gz:
data_type FLOAT32
dim1 176
dim2 512
dim3 512
dim4 84
datatype 16
pixdim1 1.000000
pixdim2 0.500000
pixdim3 0.500000
pixdim4 1.000000
cal_max 1.5417
cal_min 0.0000
file_type NIFTI-1+
3dinfo all_FA_skeletonised.nii.gz:
Dataset File: all_FA_skeletonised.nii.gz
Identifier Code: NII__AL_eWytYUadLGoiKqBqBQ Creation Date: Wed Apr 22 09:02:21 2015
Template Space: ORIG
Dataset Type: Echo Planar (-epan)
Byte Order: LSB_FIRST {assumed} [this CPU native = LSB_FIRST]
Storage Mode: NIFTI
Storage Space: 15,502,147,584 (16 billion [giga]) bytes
Geometry String: "MATRIX(1,0,0,-91.02654,0,-0.5,0,94.37949,0,0,0.5,-146.8557):176,512,512"
Data Axes Tilt: Plumb
Data Axes Orientation:
first (x) = Right-to-Left
second (y) = Posterior-to-Anterior
third (z) = Inferior-to-Superior [-orient RPI]
R-to-L extent: -91.027 [R] -to- 83.973 [L] -step- 1.000 mm [176 voxels]
A-to-P extent: -161.121 [A] -to- 94.379 [P] -step- 0.500 mm [512 voxels]
I-to-S extent: -146.856 [I] -to- 108.644 [S] -step- 0.500 mm [512 voxels]
Number of time steps = 84 Time step = 1.00000s Origin = 0.00000s
-- At sub-brick #0 '?' datum type is float
-- At sub-brick #1 '?' datum type is float
-- At sub-brick #2 '?' datum type is float
** For info on all 84 sub-bricks, use '3dinfo -verb' **
Any ideas why this might be happening? Many thanks!
Is anyone allowed to copy and use this code? If so, under the terms of which license?
--participant_label
can be specified in the group analysis level to analyse only a subset of subjects in the input dataset. Currently it is ignored.
Hi, guys!
I try to compile python wrapper, but unfortunately there are error appears:
gcc: error: broccoli_lib_wrap.cxx: No such file or directory
gcc: fatal error: no input files
compilation terminated.
g++: error: broccoli_lib_wrap.o: No such file or directory
and there are no any file with name broccoli_lib_wrap* in BROCCOLI folder. What am I doing wrong?
Hi,
I was trying to analyze high-res fMRI using 'GLM' but received the error message below:
Authored by K.A. Eklund
Data size for run 1: 256 x 256 x 256 x 147
Data size for run 2: 256 x 256 x 256 x 147
Data size for run 3: 256 x 256 x 256 x 147
Data size for run 4: 256 x 256 x 256 x 147
Data size for run 5: 256 x 256 x 256 x 147
Data size for run 6: 256 x 256 x 256 x 147
Data size for run 7: 256 x 256 x 256 x 147
Data size for run 8: 256 x 256 x 256 x 147
Total data size: 256 x 256 x 256 x 1176
Number of original regressors: 7
Number of total regressors: 15
Number of contrasts: 13
Performing 13 t-tests
GLM -runs 8 data_dir/fmri_run1.nii.gz data_dir/fmri_run2.nii.gz data_dir/fmri_run3.nii.gz data_dir/fmri_run4.nii.gz data_dir/fmri_run5.nii.gz data_dir/fmri_run6.nii.gz data_dir/fmri_run7.nii.gz data_dir/fmri_run8.nii.gz -designfiles regress_run1.txt regress_run2.txt regress_run3.txt regress_run4.txt regress_run5.txt regress_run6.txt regress_run7.txt regress_run8.txt -mask data_dir/brainmask_crop.nii -contrasts contrast.txt -firstlevel -detrendingregressors 1 -output brc_: Segmentation fault
The size of each input file is ~850MB. Someone recommended me to put all the input & output files in the directory. I don't know why though. Any help will be highly appreciated.
Hello, I'm new to using BROCCOLI and have just installed it on my Mac. It set up and installed correctly as far as I can tell, but when I try to use the TFCE option for RandomiseGroupLevel, I get a "TFCE is currently turned off!" error. Am I missing something or is TFCE not implemented?
Here is the command I ran: RandomiseGroupLevel /Users/catcamacho/Box/FFnHK-Oddball/dataprocessing/workflows/L2workflow/_cope_cope1/mergecopes/cope1_merged.nii -design /Users/catcamacho/Box/FFnHK-Oddball/dataprocessing/misc/design.mat -contrasts /Users/catcamacho/Box/FFnHK-Oddball/dataprocessing/misc/tcon.con -inferencemode 3 -verbose
Thanks for you help!
Thank you for the great toolbox!
I'm getting different results for the t-maps outputted by BROCCOLI's RandomiseGroupLevel and fsl randomise where I'm only testing for the groupmean (-groupmean flag in Broccoli and -1 flag in randomise).
The commands I ran were:
RandomiseGroupLevel volumes.nii -groupmean
randomise -i volumes.nii -o output -1
The maps look the same but the t-values are off by a factor of ~470 for BROCCOLI. So like 0.06 instead of 28. I've also done a 1 sample t-test in spm and the results are very similar to the randomise ones.
I've attached the information about my system from getOpenCLInfo.
I can put the data on dropbox for you if you wan't to try it out yourself. Any idea what could be the problem?
I am performing permutation tests on results and want to use RandomiseGroupLevel. To test it out, I placed placed spatial volumes (beta-maps) for 10 subjects into a single nifti file and then created files for design/contrast matrices for 10 subjects. I ran the following to test the program.
RandomiseGroupLevel a.nii -design design.mat -contrasts design.con
The output was:
Warning: No mask being used, doing permutations for all voxels.
Input data is a single volume, nothing to permute!
I am not using a mask in this test. But, there are definitely 10 volumes in the "a.nii" file. Does this commonly occur?
I created the "a.nii" file by using AFNI's 3dbucket to compile them and then 3dAFNItoNIFTI to convert to nifti format. Could this be a source of error?
Thank you.
Currently broccolipipeline.sh
is relying on existence of cond files which are not part of the BIDS standard. Instead you should query the trial_type
column from the _events.tsv files to obtain similar information.
Dear Anders,
I was wondering if it would be feasible to implement elastic net regularization in GLM. It would be cool to estimate weights for very large design matrices with regularization at the speed of light :) Is there hope for this to come into existence?
Best,
Michael
I'm running Broccoli for permutation tests on MRI data. I'm getting an error of the type:
Run kernel error for kernel 'CalculateBetaWeightsGLM' is 'CL_MEM_OBJECT_ALLOCATION_FAILURE'
Run kernel error for kernel 'CalculateStatisticalMapsGLMTTest' is 'CL_MEM_OBJECT_ALLOCATION_FAILURE'
Run kernel error for kernel 'CalculateStatisticalMapsGLMTTestSecondLevelPermutation' is 'CL_MEM_OBJECT_ALLOCATION_FAILURE'
Run kernel error for kernel 'TransformData' is 'CL_MEM_OBJECT_ALLOCATION_FAILURE'
It seems this is a memory issue. Are there anyways getting around this? The data I'm permuting are spatial maps of dimension 256x256x256 for several hundred samples. I'm using a mask generated from "Smoothing".
What is more is that, prior to this error, the output of RandomiseGroupLevel states
Permutation threshold for contrast 1 for a significance level of 0.050000 is 0.000000
Permutation threshold for contrast 2 for a significance level of 0.050000 is 0.000000
Permutation threshold for contrast 3 for a significance level of 0.050000 is 0.000000
Could this imply that the data is too highly correlated over voxels for "RandomiseGroupLevel" to work properly or is it most likely a memory issue? The volumes file is 2.5 GB in total with 356 subjects. My device information is:
Device info
Platform number: 0
Platform vendor: NVIDIA Corporation
Platform name: NVIDIA CUDA
Platform extentions: cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_nv_create_buffer
Platform profile: FULL_PROFILE
Device number: 0
Device vendor: NVIDIA Corporation
Device name: Tesla K80
Hardware version: OpenCL 1.2 CUDA
Software version: 375.66
OpenCL C version: OpenCL C 1.2
Device extensions: cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_nv_create_buffer
Global memory size in MB: 11439
Size of largest memory object in MB: 2859
Global memory cache size in KB: 208
Local memory size in KB: 48
Constant memory size in KB: 64
Parallel compute units: 13
Clock frequency in MHz: 823
Max number of threads per block: 1024
Max number of threads in each dimension: 1024 1024 64
It seems the hardware could theoretically handle loading 2.5 Gigs of data (but not sure if it is enough for Permutation tests).
thank you.
Best,
Chintan
I'm testing out first-level analysis using BROCCOLI with some data I'd previously analyzed with FSL. When I run FirstLevelAnalysis using my CPU as the OpenCL device, everything builds perfectly fine and the analysis seems to run as intended. When I try running the same analysis with my GPU, however, I get the following error:
Source build error for kernelBayesian.cpp is CL_BUILD_PROGRAM_FAILURE
One or several kernels were not created correctly, check buildInfo* !
Looking at the build log for that file for my GPU, it contains just this single line:
ptxas error : Program using constant pointers passed as entry function parameter cannot use cvta.const
The only reference I could turn up to this error on Google was another OpenCL developer having the same issue on a Mac, but he never posted a solution. Here's my GetOpenCLInfo output:
Device info
---------------------------------------------
Platform number: 0
---------------------------------------------
Platform vendor: Apple
Platform name: Apple
Platform extentions: cl_APPLE_SetMemObjectDestructor cl_APPLE_ContextLoggingFunctions cl_APPLE_clut cl_APPLE_query_kernel_names cl_APPLE_gl_sharing cl_khr_gl_event
Platform profile: FULL_PROFILE
---------------------------------------------
---------------------------------------------
Device number: 0
---------------------------------------------
Device vendor: Intel
Device name: Intel(R) Core(TM) i7-4771 CPU @ 3.50GHz
Hardware version: OpenCL 1.2
Software version: 1.1
OpenCL C version: OpenCL C 1.2
Device extensions: cl_APPLE_SetMemObjectDestructor cl_APPLE_ContextLoggingFunctions cl_APPLE_clut cl_APPLE_query_kernel_names cl_APPLE_gl_sharing cl_khr_gl_event cl_khr_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_image2d_from_buffer cl_APPLE_fp64_basic_ops cl_APPLE_fixed_alpha_channel_orders cl_APPLE_biased_fixed_point_image_formats cl_APPLE_command_queue_priority
Global memory size in MB: 8192
Size of largest memory object in MB: 2048
Global memory cache size in KB: 0
Local memory size in KB: 32
Constant memory size in KB: 64
Parallel compute units: 8
Clock frequency in MHz: 3500
Max number of threads per block: 1024
Max number of threads in each dimension: 1024 1 1
---------------------------------------------
Device number: 1
---------------------------------------------
Device vendor: NVIDIA
Device name: GeForce GTX 780M
Hardware version: OpenCL 1.2
Software version: 10.30.25 355.11.10.10.30.120
OpenCL C version: OpenCL C 1.2
Device extensions: cl_APPLE_SetMemObjectDestructor cl_APPLE_ContextLoggingFunctions cl_APPLE_clut cl_APPLE_query_kernel_names cl_APPLE_gl_sharing cl_khr_gl_event cl_khr_byte_addressable_store cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_APPLE_fp64_basic_ops cl_khr_fp64 cl_khr_3d_image_writes cl_khr_depth_images cl_khr_gl_depth_images cl_khr_gl_msaa_sharing cl_khr_image2d_from_buffer cl_APPLE_ycbcr_422 cl_APPLE_rgb_422
Global memory size in MB: 4096
Size of largest memory object in MB: 1024
Global memory cache size in KB: 0
Local memory size in KB: 48
Constant memory size in KB: 64
Parallel compute units: 8
Clock frequency in MHz: 648
Max number of threads per block: 1024
Max number of threads in each dimension: 1024 1024 64
I've tried installing the NVIDIA Web Drivers and CUDA drivers for my GPU to see if that would make a difference, but without any luck. I'm running macOS 10.13.4 (High Sierra). Any idea what's going on?
Thanks in advance!
I have not find anywhere in the documentation information about estimated inverse transformations for non-linear fields.
I would like to perform template-to-image transforms based on the fields estimated with RegisterTwoVolumes during image-to-template transform. Is there a currently working solution to that, or could it be implemented?
I assume the affine matrix can be inverted using fsl's convert_xfm function
Hi,
I have the issue that when I use RegisterTwoVolumes and apply the transformation matrix to the moving object in using TransformVolume, the MyImage_allign_linear and the MyImage_warpped don't overlap (when I apply linear transform).
This was the case on linux, up until I downloaded BROCCOLI's latest version (22Feb2017). when it seemed to have been fixed. However the issue remain on my Mac, despite having downloaded BROCCOLI's latest.
Code I use is as follow:
RegisterTwoVolumes MyImage standard -savematrix
TransformVolume MyImage standard -matrix MyImage_affinematrix.txt
fslview MyImage_aligned_linear MyImage_warpped
Hi, I am trying to run RandomiseGroupLevel for the first time using CPU and am getting the following error. Do you know how to resolve this? Thank you!
Authored by K.A. Eklund
Data size: 197 x 233 x 189 x 976
Number of permutations: 5000
Number of regressors: 4
Number of contrasts: 2
Performing 2 t-tests
Two sample t-test design detected for t-contrast 1, 590 subjects in group 1 and 386 subjects in group 2
Two sample t-test design detected for t-contrast 2, 590 subjects in group 1 and 386 subjects in group 2
Max number of permutations for contrast 1 is 7.84211e+282
Max number of permutations for contrast 2 is 7.84211e+282
Unable to write to binary file for kernel kernelConvolution.cpp, null file pointer!
Both operands to ICmp instruction are not of the same type!
%41 = icmp slt i32 %39, i64 0
Both operands to ICmp instruction are not of the same type!
%61 = icmp sgt i64 %53, i32 %40
Invalid operands for select instruction!
%62 = select i1 %61, i64 %53, i32 %40
Broken module found, compilation aborted!
Stack dump:
0. Running pass 'Function Pass Manager' on module 'main'.
Running pass 'Module Verifier' on function '@CalculatePhaseGradientsY'
Aborted (core dumped)
Hi all,
so I recently installed an NVIDIA driver using (NVIDIA-Linux-x86_64-375.66.run). Now if I run clinfo I get the following message attached to this message. However, when I cd into the appropriate folder and run ./GetOpenCLInfo then I get an error message:
./GetOpenCLInfo: error while loading shared libraries: libclBLAS.so.2: cannot open shared object file: No such file or directory
I'm not sure exactly why this is happening and I'm very very new to linux so I'm having some trouble figuring it out. I was hoping somebody on here might be able to help. Thank you in advance!
Best,
Sri
I have spent many hours trying to make Broccoli run on Microsoft Surface Pro 7+ i7, inside WSL2. I have installed all sorts of packages and GetOpenCLInfo shows that OpenCL is available. Yet, the Kernel code cannot be compiled:
$ /opt/brp/depend/Broccoli/compiled/Bash/Linux/Release/RegisterTwoVolumes ~/SampleData/mprage_reorg.nii ~/SampleData/atlas.nii -output ~/SampleData/OUTPUT.nii -saveinterpolated
Authored by K.A. Eklund
Volume 1 size: 176 x 224 x 256
Volume 1 voxel size: 1.000000 x 1.000000 x 1.000000 mm
Volume 2 size: 182 x 218 x 182
Volume 2 voxel size: 1.000000 x 1.000000 x 1.000000 mm
Source build error for kernelMisc.cpp is CL_BUILD_PROGRAM_FAILURE
Source build error for kernelBayesian.cpp is CL_BUILD_PROGRAM_FAILURE
One or several kernels were not created correctly, check buildInfo* !
ll
Create kernel error for kernel 'SliceTimingCorrection' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateMagnitudes' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateColumnSums' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateRowSums' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateColumnMaxs' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateRowMaxs' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateMaxAtomic' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'ThresholdVolume' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'Memset' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MemsetDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MemsetInt' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MemsetFloat2' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'IdentityMatrix' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'IdentityMatrixDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'GetSubMatrix' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'GetSubMatrixDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'PermuteMatrix' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'PermuteMatrixDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'LogitMatrix' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'LogitMatrixDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MultiplyVolume' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MultiplyVolumes' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MultiplyVolumesOverwrite' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'MultiplyVolumesOverwriteDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'AddVolume' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'AddVolumes' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'AddVolumesOverwrite' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'SubtractVolumes' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'SubtractVolumesOverwrite' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'SubtractVolumesOverwriteDouble' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'RemoveMean' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Create kernel error for kernel 'CalculateStatisticalMapsGLMBayesian' is 'CL_INVALID_PROGRAM_EXECUTABLE'
Run kernel error for kernel 'CalculateColumnMaxs' is 'CL_INVALID_KERNEL'
Run kernel error for kernel 'CalculateRowMaxs' is 'CL_INVALID_KERNEL'
Run kernel error for kernel 'Memset' is 'CL_INVALID_KERNEL'
Run kernel error for kernel 'MemsetFloat2' is 'CL_INVALID_KERNEL'
Run kernel error for kernel 'MultiplyVolumes' is 'CL_INVALID_KERNEL'
Run kernel error for kernel 'AddVolumesOverwrite' is 'CL_INVALID_KERNEL'
Affine registration parameters
0.977015 -0.016522 -0.000435 0.274039
0.011848 0.928244 -0.168395 -3.405550
0.011702 0.173381 0.876269 5.861003
0.000000 0.000000 0.000000 1.000000
The output of the above is two files, one for affine registration and one non-linear registration. Both are identical though, looks like Broccoli did not manage to do any nonlinear registration.
This is the output of GetOpenCLInfo:
$ GetOpenCLInfo
Device info
---------------------------------------------
Platform number: 0
---------------------------------------------
Platform vendor: Intel(R) Corporation
Platform name: Intel(R) OpenCL Graphics
Platform extentions: cl_khr_byte_addressable_store cl_khr_device_uuid cl_khr_fp16 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_intel_command_queue_families cl_intel_subgroups cl_intel_required_subgroup_size cl_intel_subgroups_short cl_khr_spir cl_intel_accelerator cl_intel_driver_diagnostics cl_khr_priority_hints cl_khr_throttle_hints cl_khr_create_command_queue cl_intel_subgroups_char cl_intel_subgroups_long cl_khr_il_program cl_intel_mem_force_host_memory cl_khr_subgroup_extended_types cl_khr_subgroup_non_uniform_vote cl_khr_subgroup_ballot cl_khr_subgroup_non_uniform_arithmetic cl_khr_subgroup_shuffle cl_khr_subgroup_shuffle_relative cl_khr_subgroup_clustered_reduce cl_intel_device_attribute_query cl_khr_suggested_local_work_size cl_intel_split_work_group_barrier cl_intel_spirv_media_block_io cl_intel_spirv_subgroups cl_khr_spirv_linkonce_odr cl_khr_spirv_no_integer_wrap_decoration cl_intel_unified_shared_memory cl_khr_mipmap_image cl_khr_mipmap_image_writes cl_ext_float_atomics cl_khr_external_memory cl_intel_planar_yuv cl_intel_packed_yuv cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_3d_image_writes cl_intel_media_block_io cl_intel_subgroup_local_block_io cl_khr_integer_dot_product cl_khr_gl_sharing cl_khr_gl_depth_images cl_khr_gl_event cl_khr_gl_msaa_sharing cl_intel_sharing_format_query cl_khr_pci_bus_info
Platform profile: FULL_PROFILE
---------------------------------------------
---------------------------------------------
Device number: 0
---------------------------------------------
Device vendor: Intel(R) Corporation
Device name: Intel(R) Graphics [0x9a49]
Hardware version: OpenCL 3.0 NEO
Software version: 23.33.0
OpenCL C version: OpenCL C 1.2
Device extensions: cl_khr_byte_addressable_store cl_khr_device_uuid cl_khr_fp16 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_intel_command_queue_families cl_intel_subgroups cl_intel_required_subgroup_size cl_intel_subgroups_short cl_khr_spir cl_intel_accelerator cl_intel_driver_diagnostics cl_khr_priority_hints cl_khr_throttle_hints cl_khr_create_command_queue cl_intel_subgroups_char cl_intel_subgroups_long cl_khr_il_program cl_intel_mem_force_host_memory cl_khr_subgroup_extended_types cl_khr_subgroup_non_uniform_vote cl_khr_subgroup_ballot cl_khr_subgroup_non_uniform_arithmetic cl_khr_subgroup_shuffle cl_khr_subgroup_shuffle_relative cl_khr_subgroup_clustered_reduce cl_intel_device_attribute_query cl_khr_suggested_local_work_size cl_intel_split_work_group_barrier cl_intel_spirv_media_block_io cl_intel_spirv_subgroups cl_khr_spirv_linkonce_odr cl_khr_spirv_no_integer_wrap_decoration cl_intel_unified_shared_memory cl_khr_mipmap_image cl_khr_mipmap_image_writes cl_ext_float_atomics cl_khr_external_memory cl_intel_planar_yuv cl_intel_packed_yuv cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_3d_image_writes cl_intel_media_block_io cl_intel_subgroup_local_block_io cl_khr_integer_dot_product cl_khr_gl_sharing cl_khr_gl_depth_images cl_khr_gl_event cl_khr_gl_msaa_sharing cl_intel_sharing_format_query cl_khr_pci_bus_info
Global memory size in MB: 6490
Size of largest memory object in MB: 1024
Global memory cache size in KB: 3840
Local memory size in KB: 64
Constant memory size in KB: 1048576
Parallel compute units: 96
Clock frequency in MHz: 1300
Max number of threads per block: 512
Max number of threads in each dimension: 512 512 512
Trying to compile libraries also fails to find opencl.h, although I installed the neo driver from source:
$ ./compile_broccoli_library.sh
broccoli_lib.cpp:48:10: fatal error: opencl.h: No such file or directory
48 | #include <opencl.h>
| ^~~~~~~~~~
compilation terminated.
ar: broccoli_lib.o: No such file or directory
mv: cannot stat 'libBROCCOLI_LIB.a': No such file or directory
There are many examples showing that OpenCL works on WSL2, and I can see it is there, but I can't find what is Broccoli's problem. Any help?
To improve user experience the broccolipipeline.sh
script should check if bids-validator
is available and call it (bids-validator $bids_dir
) to check input data compatibility before running any analysis.
I'll send a PR to BIDS-App/BROCCOLI updating the Dockerfile to include the validator.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.