Giter Site home page Giter Site logo

reals's Introduction

Robust and Efficient Image Alignment through Simultaneous Low Rank and Sparse Decomposition

This repository is the official implementation of Robust and Efficient Image Alignment through Simultaneous Low Rank and Sparse Decomposition. This paper is accepted in WACV 2023.

Requirements

To install requirements:

conda create -n reals python=3.8.10
conda activate reals
pip install -r requirements.txt

Data for REALS is required for execution of the code.

  • 2-D Zebrafish brain data: Static data with size [512(x), 512(y), 60(z)]. Additional perturbations or noise can be applied.
  • 3-D Zebrafish brain data: Data with size [256(x), 512(y), 48(z), 60(t)]. Random translation and rotation are applied synthetically.
  • Mouse brain data: Data with size [480(x), 752(y), 1000(t)]. Y.tif is the original video, and Y_DoG.tif is high pass filtered video.
  • 2-D Zebrafish brain data with drift: Data with size [256(x), 512(y), 600(t)]. Original video has small drift.

To run run_reals.py, you have to generate data with perturbation from static 2-D Zebrafish brain data.

  • generate_oscillation.py generates randomly perturbed data with corresponding transformation from static 2-D Zebrafish brain data.

If you want to add noise to above generated data,

  • generate_gaussian_noise.py generates data with gaussian noise from perturbed data.
  • generate_poisson_noise.py generates data with poisson noise from perturbed data.

Training

run_reals.py can reproduce most of the results in the paper. Run following command:

python -m scripts.run_reals --type normal

To perform REALS on 3-D Zebrafish brain data, run following command:

python -m scripts.run_reals_3d

We also provide the mini-batch and multi-resolution version of REALS in run_reals_minibatch.py, run_reals_multi.py respectively.

After running REALS on one sample data, there will be composite transformation tau, original images Y, aligned images L, low-rank component L_stat, and sparse component S in the resulting directory.

Evaluation

We provide 3 sample codes for evaluation. However, the codes can perfectly reproduce the results in original paper only when required results are fully available. So, it may require to modify some parts of the code to analyze the result with below codes.

  • eval_heatmap.py shows the alignment inconsistency heatmap.
  • eval_barplot.py shows the alignment inconsistency bar plot of REALS. It is to check performance with different batch size.
  • eval_timeplot.py shows the (time, alignment inconsistency) plot. You need to save time and transformation at each iteration to plot the result.

Results

We will make it available after the review.

Contributing

The base code is from BEAR.

reals's People

Contributors

junmokane avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar

Forkers

saglag bienseo

reals's Issues

pytorch version incompatible with 3090 GPU

torch==1.8.1

When running python -m scripts.run_reals --type normal, I encounter the following warning: The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

As well as:
RuntimeError: CUDA error: no kernal image is available for the execution on the device.

Adding the suggestion of copying the installation notes from SUPPORT:
Install Pytorch with the version compatible with your OS and platform from https://pytorch.org/get-started/locally/

conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

Running python -m scripts.run_reals --type normal correctly targets dedicated GPU for processing after performing the suggestion.

setuptools==52.0.0.post20210125 does not exist in PyPI

setuptools==52.0.0.post20210125

setuptools==52.0.0.post20210125 cannot be completed as this version doesn't exist in the PyPI repository. Likely a carryover from a previously yanked version 51.0.0.post20210125 that was misnamed.

Changing to 52.0.0 solves the error and doesn't throw and error when calling python -m script.run_reals --type normal

numpy version dependency conflict in requirements.txt

numpy==1.20.2

This version of numpy throws the following dependency conflict:
ERROR: Cannot install -r requirements.txt (line 20), -r requirements.txt (line 22), -r requirements.txt (line 28), -r requirements.txt (line 29), -r requirements.txt (line 30), -r requirements.txt (line 31) and numpy==1.20.2 because these package versions have conflicting dependencies.

The conflict is caused by:
The user requested numpy==1.20.2
h5py 3.3.0 depends on numpy>=1.17.5; python_version == "3.8"
imageio 2.9.0 depends on numpy
mat73 0.50 depends on numpy
matplotlib 3.4.2 depends on numpy>=1.16
mkl-fft 1.3.0 depends on numpy>=1.16
mkl-random 1.2.2 depends on numpy<1.23.0 and >=1.22.3

Changing to numpy==1.22.3 solves the issue without throwing an error calling python -m scripts.run_reals --type normal

I haven't examined outputs to confirm this and the setuptools change yet.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.