samuelstjean/nlsam

The reference implementation for the Non Local Spatial and Angular Matching (NLSAM) denoising algorithm for diffusion MRI

nlsam's Issues

how to use nlsam denoising

I used autodmri to estimate N and sigma from my noisy data , as a result I have sigma.nii.gz and N.nii.gz images. How do I use them in order to denoise my dataset given the noise distribution previously estimated? thank you,
Rosella

Move parts of the script to the lib

For having a simpler format for scilpy, move some code from the script to the main core nlsam, so that way the script should only be a thin wrapper over a function instead of doing some setup/type checking like now.

Add verbose option to nlsam

If possible, can you "quiet" nlsam by default and/or add a verbose option.

Update versions between this one, scilpy and my dev version

Well, once everything for the stabilization is merged in scilpy proper, use this version in the repo since it will be the one with all the fancy features.

Look into replacing multiprocessing with joblib

Note to self : seems like it could be lighter to use and easier than multiprocessing, as it removes the need for unpacking functions and huge arglist variables like currently used.

got multiple values for argument 'n_cores'

Sorry, never mind

kill off ez_setup.py

Seems like it's not recommended anymore pypa/setuptools#581
Plus it was probably never really used since setuptools is a fairly basic to have package.

Noise maps

Hi
is it possible to obtain noise maps resulting from nlsam denoising?
best,
Rosella

check what happened to warm restart

Seems like it disappeared between 0.3 and 0.5 during the big refactor, but the rejection branch might help bring it back.

bundle dipy_slim

Since I only use a few functions, no need to install the whole thing, especially for people using their own fork. Might as well bundle the 2-3 functions in a separate file to prevent building the whole thing,

floating point exception

rosella@vivi:nlsam_denoising unproc_3T.nii.gz denoised.nii.gz auto bval bvec 5 -m mask_noise.nii.gz --load_sigma sigma.nii.gz --fix_implausible --verbose
2020-01-10 15:59 - nlsam - INFO - Verbosity is on
2020-01-10 15:59 - nlsam - WARNING - Option --implausible_signal_fix has been deprecated
2020-01-10 15:59 - nlsam - INFO - Loading data /media/rosella/TOSHIBA EXT/dkeTest/115825/Diffusion_3T/unproc_3T.nii.gz
2020-01-10 15:59 - nlsam - INFO - Loading mask /media/rosella/TOSHIBA EXT/dkeTest/115825/Diffusion_3T/mask_noise.nii.gz
2020-01-10 15:59 - nlsam - INFO - Loading sigma /media/rosella/TOSHIBA EXT/dkeTest/115825/Diffusion_3T/sigma.nii.gz
2020-01-10 15:59 - nlsam - INFO - Now performing stabilization
[Parallel(n_jobs=8)]: Using backend LokyBackend with 8 concurrent workers.
[Parallel(n_jobs=8)]: Done 34 tasks | elapsed: 2.5s
[Parallel(n_jobs=8)]: Done 111 out of 111 | elapsed: 7.5s finished
2020-01-10 15:59 - nlsam - INFO - Now denoising
2020-01-10 15:59 - nlsam - INFO - Found 6 b0s at position [ 0 16 32 48 64 80]
2020-01-10 15:59 - nlsam - INFO - Now denoising volumes (0, 1, 47, 38, 30, 85) / block 1 out of 24.
Floating point exception

### What is this error due to?
thanks

Add --symmetry option

Currently we enforce symmetry on the bvecs file, the user should be able to disable this behavior in case his data is already symmetrized (DSI grid for example).

omp_set_nested routine deprecated

When running nlsam on CentOS server,
nlsam_denoising data.nii.gz denoised.nii.gz auto bvals bvecs 3 -m mask.nii.gz --noise_est auto --cores 15 -v
during performing of stabilization the following warning starts iteratively appear in loop
OMP: Info #274: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
How can this be solved? Is it just a warning? Can I let the code go despite this message appearing in loop?
My OS features are:
lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.8.2003 (Core)
Release: 7.8.2003
Codename: Core
thank you
Rosella

Add the various speedfix

They are in my dev_stuff branch from scilpy, so once the scilpy version is up to date with this one, merge my personal fixes.

change the sh warning to output the number of dwis only

it currently output the total number of volumes including b0s, change it also in scilpy for good measure.

pre-compiled Windows version with new features

While we are able to use nlsam_denoise on linux (and I've managed to get it working on Compute Canada as well, which is a cloud cluster we can take advantage of), I'll be interested in making it working on Windows as well (some of our users use ExploreDTI on Matlab and this would allow them to run this pre-processing all on one platform).

The pre-compiled version available on GitHub seems to be from previous iteration of work, so it doesn't support the automated estimation. I have so far not been able to install (and make things work) under Python on Windows - it seems that despite no errors during installing packages, spams/spams_wrap is missing some DLLs (?)

Is there any trick you would be to recommend to get this installed, or, even better, any chance you might be able to generate updated version of pre-compiled exe file?

We have successfully integrated your NLSAM method into processing of our DTI data. Especially your recent addition of automating of estimating all parameters helps a great deal, and we are very grateful your work is available!

Update shebang in scripts

Apparently JC mentioned we should use something like nipy/nibabel#434 for proper packaging with setuptools.

Add long and short command switches

-m, --mask, -o, --output and such.

Will break all user scripts, but at least it will be more consistent with scilpy currently.

Check that 4D padding is consistent

Currently, the 4D array are always overlapping but the padding factor is still computed nevertheless if no padding is required, thus enlarging arrays without any need to.

Either re-put the non overlap 4D function or make the naming consistent that overlap is actually happening to less confuse people looking at the code.

Add option to still compute SD without stabilization

Currently one has to pass --load_sigma along with --no_stabilization. Would be simple enough to still allow this computation in the main script by just moving some stuff around.

Check if mkl threading is slower in multiprocessing

I currently disable openblas threading for linear algebra, but it seems like mkl is doing the same thing.

Compare if it is slower by setting export MKL_NUM_THREADS=1 or if it would not be needed (I read that mkl took care of that, maybe they just spawn unused threads), as the binaries are built with mkl.

Check the size of sigma is compatible with the size of data

Currently, the check is only with data and mask, so checking the size of sigma also would be accurate in case people come up with their own estimated volume and just want to run the denoising purely.

Check if convergence on signal is faster than on coefficients

Currently, for a system Ax = b with a solution x_star, convergence is reached when ||x_star - x||_inf < eps.

Check if it would be attained faster by checking ||b - b_star||_inf < eps, which would be identical for all practical purpose.

nlsam version

Hi I have just download and installed 6.0.1 nlsam version but the option "auto" for N does not work. I would like to know which version has this feature.
Thanks
Rosella

Add a --fast option

It would be nice to have a --fast option that turns on the greedy_subsampler True.

Should it be True by default?

installing nlsam on centos

when trying to install nlsam on Centos the following error appears:

pip install nlsam_0.6.1_centos6_x64.zip --user rosella
Processing ./nlsam_0.6.1_centos6_x64.zip
    ERROR: Command errored out with exit status 1:
     command: /home/rosella/anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-p5dlbqnh/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-p5dlbqnh/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-e5y0g4co
         cwd: /tmp/pip-req-build-p5dlbqnh/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/rosella/anaconda3/lib/python3.7/tokenize.py", line 447, in open
        buffer = _builtin_open(filename, 'rb')
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-req-build-p5dlbqnh/setup.py'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

what shoud it be due to? I have python 3.7.3 installed with conda package.
My OS features are:
lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.8.2003 (Core)
Release: 7.8.2003
Codename: Core

Thanks,
Rosella