matejak/imreg_dft

Image registration using discrete Fourier transform.

imreg_dft

Project status

imreg_dft has reached a stable state. This means that although you won't see many commits, the project is not dead, there are just no outstanding issues and people are not complaining. Addition of some nice-to-have features is planned in Q3/2017. Until then, I will spend time on the Argbash project - you might want to check it out if you write shell scripts.

Overview

Image registration using discrete Fourier transform.

Given two images, imreg_dft can calculate difference between scale, rotation and position of imaged features. Given you have the requirements, you can start aligning images in about five minutes! Check the documentation on readthedocs.org (bleeding-edge) or pythonhosted.org (with images).

If you are a part of the education process that has something to do with phase correlation (doesn't matter whether you are a student or a teacher), the ird-show utility that is part of the imreg_dft project can help you understand (or explain) how the phase correlation process works. If you are a researcher and you are having problems with the method on your data, you can use ird-show to find out what causes your problems quite easily.

Features

• Image pre-processing options (frequency filtration, image extension).
• Under-the-hood options exposed (iterations, phase correlation filtration).
• Visualization of various stages of the image registration (so you can more easily find out how it works or what went wrong).
• Command-line interface for image registration (ird - text output and/or image output), for image transformation (ird-tform, cooperates with ird) and inspection (ird-show).
• Documented Python API with examples.
• Permissive open-source license (3-clause BSD).

Project facts

• The project is written in pure Python.
• Essentially requires only numpy and scipy (RHEL7-safe).
• Includes quickstart documentation and example data files.
• Per-commit tests and documentation (see badges under the heading).
• Originally developed by Christoph Gohlke (University of California, Irvine, USA)
• Currently developed by Matěj Týč (Brno University of Technology, CZ)

imreg_dft's Issues

Discrepancy in command line utility (ird) and python function (similarity)

I am trying to register two images that have been spatially translated (no rotations). When I use the command line utility ird, I get the exact (x, y) coordinates. But when I use the python utility, I get erroneous results. This is not the case for all the images, but it does happen in a few cases. I am unable to attach the actual images for copyright reasons.

Using python

template = cv2.imread(template_image_path, 0)
print template.shape # 512x512
subject = cv2.imread(subject_image_path, 0)
print subject.shape # 512x512

from imreg_dft.imreg import similarity
similarity(template, subject)

Output

'tvec': array([-14.03552074, -0.08051801]), 'Dt': 0.25

Using ird command-line

ird template_image_path subject_image_path

Output

shift (x, y): 58.9625, -0.0204523 +-0.25

I am not able to figure out what is wrong here.

Consider second pass imreg

If the common area of two images is determined after the first pass, perform the second pass on the truncated images containing that common area. The results could be somewhat better and it could be quite fast since the images would be smaller.
Resampling for sub-pixel resolution could also be an option here.

Enable multiprocessing for tiles

Evaluation of tiles can be done in parallel, potentially saving a lot of time.

Investigate image registration on complex arrays

ird doesn't produce impressive results when used on complex numbers arrays.

[Need Help] What's that FILE OBJECT?

I'm a green hand.
Successfully installed your function.
It's working when I input "image's path and name" in
im1 = sp.misc.imread(ImagePathName, True)

But in my project, I need read an image and manipulate it, then convert it into PIL-object.
Currently I had to write that PIL-object to hdd.

Is there any way to read that PIL-object directly.
I read the function's doc, it's working in file object. But I tried this, but failed.
fo=open('image.png','r'); im1 = sp.misc.imread(fo, True)

Please give me some guide, thanks a lot.

Multichannel Images

Hello,
I am trying to use your library for registration of a series of 13 images. They have four channels (RGB + IR). When i use the similarity func. on all four channels they all have slightly different translation and rotation results. Could it be possible to just calculate the transformation data on all 4 channels without actually transforming them (would save time) and then choose a transformation of a single channel (the one with the best success rate) and apply it to all 4 ? The 4 channels would then align when they are written to a single image.

Could you also please define the success rate ? How is it calculated ?

Nice Greetings, Jure

Add support for fixing transform parameters

If we know translation, scale or angle, we can take it into account and then come up with a different result than when guessing the transformation blindly.

np.bool deprecated in utils.get_borderval( )

The function similarity( ) calls utils.get_borderval( ) in imreg.py:

bgval = utils.get_borderval(im1, 5)

but utils.get_borderval( ) points to the type np.bool (which is now deprecated as an alias for bool as of NumPy 1.20):

mask = np.zeros_like(img, dtype=np.bool)

This generates an AttributeError. np.bool needs to be replaced with bool in utils.get_borderval( ). Thanks!

Make a web app

Code a web application that would allow any user to check imreg-dft out just using the browser if someone decides to host it.

Add outright support for sub-pixel precision

This can be achieved by doing smart things when evaluating the phase correlation.

Polish documentation

Add some nice real-world images to docs, investigate possibility of using ird-generated images to be embedded in docs.

Think of improving the "success" criterion

Evaluation of phase correlation success is flaky ATM

Using constraints from python

I'm unable to grok how to use the parameter contraints (eg. translation(..., constriaints)) when using imreg_dft from python. Do you have an example?

sample PNGs for LargeTemplate [tiling] example do not open with Pillow 7.2

Thanks for this fascinating and useful code.
I was perplexed when attempting to run the LargeTemplate [tiling] example, always getting an out-of-range error.
imread was dropped from scipy.misc after v1.2, so I tried: forcing a scipy version < 1.2, using imageio.imread (as suggested in the SciPy documentation), and cv2.imread.
I finally realized that all of these attempts resulted in an all-zero numpy-array after opening resources/examples/tiling/big.png and small.png
I am an imaging novice, so I converted those sample images to a white background and the Large Template example went perfectly as expected (I think the sample PNGs have a transparent background, alpha?)
Not entirely sure what module is actually loading these PNG images, but I am using Pillow 7.2
Would it be possible to update the imreg_dft repository to include sample PNGs with a white background under resources/examples/tiling?
Thank you in advance!

Make a debug tool

A CLI utility (and possibly an API, too) that would allow peeking into registration internals would be great for debugging, but also for education.

GPU Support

It seems like this is the right kind of application which a GPU can speed up. Have you considered this?

How to convert padded border of transformed image to black

Is there a simple way to make it so the padded border that is added on a registered image which is smaller than the reference image is black instead of some unpredictable grey/brown color. I trained a neural network on thousands of images with black borders (from rectification) and I am worried about what will happen when I start predicting on images with brown/gray borders.

images come out like this:

but I need them to be like this:

One issue is that the border images vary in pixel values from image to image and arent even homogenous within a single image. Which ruined my attempt at the following:
border = np.where((timg[:,:,0] ==114 ) & (timg[:,:,1]==101) & (timg[:,:,2] ==86) )
timg[border] = (0,0,0)

Make a GUI desktop app

Make a GUI app and make an easy way how to produce its Windows binaries.
Investigate how to distribute those binaries.

Problem installing using pip when requirements are not met.

The top-level __init__.py is imported because of the version number defined there and it uses the convenience import *, which imports virtually all dependencies.

Polish the setup.py script

Ensure that it conforms to standards and has all features needed for PyPi support

Error installing imreg_dft

I would appreciate some help on getting imreg_dft to install and work properly. The issue might be due to having different versions of python in my system (CentOs-6.1) but I am not sure. I recently installed python 2.7.12 but sudo still uses python 2.6.

So when I tried to install imreg_dft using command 'python setup.py install', I ended up with this error "ImportError: No module named numpy.distutils.core"

But when I tried to install imreg_dft using command '/usr/bin/python setup.py install', it appeared to work fine.

---

Installed /usr/lib/python2.6/site-packages/imreg_dft-2.0.0-py2.6.egg
Processing dependencies for imreg-dft==2.0.0
Searching for Pillow==3.3.0
Best match: Pillow 3.3.0
Adding Pillow 3.3.0 to easy-install.pth file

Using /usr/lib64/python2.6/site-packages
Searching for scipy==0.17.1
Best match: scipy 0.17.1
Adding scipy 0.17.1 to easy-install.pth file

Using /usr/lib64/python2.6/site-packages
Searching for numpy==1.11.1
Best match: numpy 1.11.1
Adding numpy 1.11.1 to easy-install.pth file

Using /usr/lib64/python2.6/site-packages
Searching for setuptools==24.0.0
Best match: setuptools 24.0.0
Adding setuptools 24.0.0 to easy-install.pth file
Installing easy_install-3.5 script to /usr/bin
Installing easy_install script to /usr/bin

Using /usr/lib/python2.6/site-packages

Finished processing dependencies for imreg-dft==2.0.0

But when I ran ird, I got these errors:

Traceback (most recent call last):
File "/usr/local/bin/ird", line 5, in
from pkg_resources import load_entry_point
File "build/bdist.linux-x86_64/egg/pkg_resources/init.py", line 2927, in
File "build/bdist.linux-x86_64/egg/pkg_resources/init.py", line 2913, in _call_aside
File "build/bdist.linux-x86_64/egg/pkg_resources/init.py", line 2940, in _initialize_master_working_set
File "build/bdist.linux-x86_64/egg/pkg_resources/init.py", line 635, in _build_master
File "build/bdist.linux-x86_64/egg/pkg_resources/init.py", line 943, in require
File "build/bdist.linux-x86_64/egg/pkg_resources/init.py", line 829, in resolve
pkg_resources.DistributionNotFound: The 'scipy>=0.12' distribution was not found and is required by imreg-dft

Any help would be appreciated.

Error defining mask in utils.py

In imreg_dft/src/imreg_dft/utils.py, line 710-713, you define a border mask of with radius:
mask[:, :radius] = True
mask[:, -radius:] = True
mask[radius, :] = True
mask[-radius:, :] = True
However, line 712 is incorrect as it masks a single row (radius) not a span (:radius).

Add possibility of customizing the log rad transform.

Customizing under-the-hood details of transforms could be appreciated.

how to uninstall imreg_dft

Hello, I have installed it according to the instructions, but the test failed. It may be a version dependency problem. I don't know. I have tried a lot, but I still haven't solved it. Now I want to uninstall it, but it's not clean and can't be reinstalled. How can I solve this problem?

Polish loaders

Ensure that .mat loader works, bring it on a par with PIL.
Expand the documentation and add (regression) tests.

[Help needed] Different shape images

Hello,
I have tested the translation tool on the sample images you provided and I noticed that they are all the same shape. So I wanted to ask you if it's possible to use it for different shaped images or is it necessary to apply padding on the smaller image. Can you suggest a possible solution?

Use a smart border processing (extending, apodization)

The current way is convoluted and even doesn't have such great results.

Pixel values are changed after registration.

I'm trying to register a segmentation mask where each segmented particle is given an integer value. These values are altered during registration resulting in loss of data. I assume that this is cause by an interpolation step. Is there a way around this?

For anyone experiencing a similar problem, rounding the transformation vector to nearest integer was a fix in my case: transformation["tvec"].round(0)

Make imreg_dft transformations accessible

Would be nice to have registration and transformation separated, so if one computes the registration, one can transform by himself later or using another means.
This should be possible both through CLI tools (imreg-tform) and Python API

Array dtype conversion

I'm working with 16-bit numpy arrays and when passed into the similarity function the transformed image has a dtype of float64. I haven't been able to find where this conversion is taking place.

img0.dtype
dtype('uint16')
result = ird.similarity(img0, img1, numiter = 1, order = 3)
result['timg'].dtype
dtype('float64')

Image is shifted out of frame

imreg_dft/src/imreg_dft/imreg.py

Lines 566 to 570 in 3eb7137

	if tvec[0] != 0 or tvec[1] != 0:
	dest0 = ndii.shift(dest0, tvec, order=order, mode=mode, cval=bgval)
	bg = np.zeros_like(img) + bgval
	dest = utils.embed_to(bg, dest0)

I ran into an issue where the shift applied dest0 shifted my image out of the frame resulting in a registered image that appears cropped. I moved lines 566-7 to after line 570 so that the shift operates on the dest variable after it has been embedded into the new background. Although this solution worked for my example, I am unsure if it will work in general.

I think this should say anticlockwise instead of clockwise

imreg_dft/src/imreg_dft/imreg.py

Line 521 in 3eb7137

angle (float): Degrees of rotation (clock-wise)

installation

Hi, how to install this package in python

change apodization type

When we are trying to guess rotation / scale, use circular (rather than edge-based) apodization. And maybe re-apodize to rectangular when doing translation?

add spectrum filtration tuning

It turns out that the spectrum filtration is likely THE most important factor in phase correlation. It should be more configurable (sample/template separately) and maybe based on multiplication of exponentials.

The translation.py demo is broken

/imreg_dft/resources/code (master)$ python translation.py
Traceback (most recent call last):
File "translation.py", line 15, in
t0, t1 = ird.translation(im0, im1)
TypeError: translation() takes at least 3 arguments (2 given)

Memory issue

I have an array of images of size 1010x1010. I would like to register all of the images in this array onto the first image. When I run the following code my 8GB of memory gets full:

import numpy as np
import imreg_dft as ird

def register(im1,im2):
  result = ird.similarity(im1, im2, numiter=1)
  assert "timg" in result
  return result['timg']

im1 = img_array[:,:,0] # first image in the array

for i in xrange(1, num_images):
 img_array[:, :, i] = register(im1, img_array[:,:,i])

Do you have any idea what could be the problem?
Thanks!

Edit: Images are float and the pixel values are not necessarily within 0-255 range.

Using tiles in image registration

Hi!

I'm trying to perform an image registration between thermal image and visible image.
These images' fields of view don’t match and are in subset-superset relationship.
I have read that "tiles" option/module should solve it.
The problem is that the code fails when using the command-line (some of the functions are not implemented - e.g. _load2reg)
Also, it is unclear how to use the tiles module directly with python code.

Can you please please please add instructions for how to use tiles (or how to perform image registration with slightly different fields?)

Thank you so much!
Lior Mathan

mode is ignored in transform_img

Currently, the mode parameter is ignored in transform_img, i.e. the following

from scipy.misc import ascent
import scipy.ndimage.interpolation as ndii

im = ascent()
out1 = imreg_dft.transform_img(im, tvec=[100,100], mode="reflect")
out2 = ndii.shift(im, [100,100], mode ="reflect")

is not behaving as expected from the docs.

Possible solutions
a) to update the docstring reflecting that
b) to rewrite the utils.embed_to function to include padding with the correct mode (which is what I do right now, see that gist)

cheers and thanks for the package!

Please update your release that the np.bool change reflects in the download

Hi, could you please release a new version with the changes made - specifically the np.bool?
I use this a lot (thanks).

Nothing runs.

I get a long list of errors when I try to run the ird script.

File "/home/nuc/.local/lib/python3.11/site-packages/imreg_dft_nw-2.0.0-py3.11.egg/imreg_dft/loader.py", line 72, in _str2nptype
File "/usr/lib/python3.11/site-packages/numpy/init.py", line 324, in getattr
raise AttributeError(former_attrs[attr])
AttributeError: module 'numpy' has no attribute 'float'.
np.float was a deprecated alias for the builtin float. To avoid this error in existing code, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'cfloat'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/nuc/.local/bin/ird", line 33, in
sys.exit(load_entry_point('imreg-dft-nw==2.0.0', 'console_scripts', 'ird')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nuc/.local/bin/ird", line 25, in importlib_load_entry_point
return next(matches).load()
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/metadata/init.py", line 202, in load
module = import_module(match.group('module'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1147, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/nuc/.local/lib/python3.11/site-packages/imreg_dft_nw-2.0.0-py3.11.egg/imreg_dft/cli.py", line 40, in
File "/home/nuc/.local/lib/python3.11/site-packages/imreg_dft_nw-2.0.0-py3.11.egg/imreg_dft/loader.py", line 500, in
File "/home/nuc/.local/lib/python3.11/site-packages/imreg_dft_nw-2.0.0-py3.11.egg/imreg_dft/loader.py", line 113, in init
File "/home/nuc/.local/lib/python3.11/site-packages/imreg_dft_nw-2.0.0-py3.11.egg/imreg_dft/loader.py", line 113, in
File "/home/nuc/.local/lib/python3.11/site-packages/imreg_dft_nw-2.0.0-py3.11.egg/imreg_dft/loader.py", line 322, in init
File "/home/nuc/.local/lib/python3.11/site-packages/imreg_dft_nw-2.0.0-py3.11.egg/imreg_dft/loader.py", line 234, in init
File "/home/nuc/.local/lib/python3.11/site-packages/imreg_dft_nw-2.0.0-py3.11.egg/imreg_dft/loader.py", line 252, in setOpts
File "/home/nuc/.local/lib/python3.11/site-packages/imreg_dft_nw-2.0.0-py3.11.egg/imreg_dft/loader.py", line 74, in _str2nptype
AttributeError: 'AttributeError' object has no attribute 'message'

transform_img clips image, doesn't have option to specify output size

I noticed that transform_img clips the output image to the same size as the input image, and it would be very useful if one could specify an option for an output-image size or a suitable flag so the entire input image was shown after applying the transformation. For example, I need to find the shift and rotations to geometrically register images, but I am then scaling them up (zooming in) by some factor and combining them together to produce an enhanced, larger image (this is the initial step for super-resolution).

skimage allows this with e.g. warp(). Could you consider adding such a feature?