ValueError: Big-endian buffer not supported on little-endian compiler

[yas-nakajima]

ValueError: Big-endian buffer not supported on little-endian compiler

[martinberoiz]

Hello,

Yes, the issue is similar to the one in scikit-image issue you mentioned. The endian-ness from the source of your data doesn't match the endian-ness of your local computer where you're running astroaling.

Fortunately Numpy lets you swap the endianness quite easy:

https://numpy.org/doc/stable/user/basics.byteswapping.html

target = target.newbyteorder()
source = source.newbyteorder()

I don't think astroalign should deal with different endian issues but the above workaround works well.
Let me know if that fixes the issue for you.

[Jayesh6999]

#Allgignment of Images
image_path='image path'
filename='image / file name'
files=sorted(glob.glob(os.path.join(image_path,filename)))
print(len(files))

ref_image=fits.open(files[0], )
reference_image=ref_image[0].data

for i in range(0,len(files)):
    image_data=fits.open(files[i])
    source_image=image_data[0].data + 0  #here the addition of zero (0) solves the the endian compiler issue.
    header=image_data[0].header
    image_aligned,footprint=aa.register(source_image,reference_image)
    aligned_file=files[i].replace('.fits','')
    fits.writeto(aligned_file+'_aligned'+'.fits',image_aligned,header,overwrite=True)
    print('No. %i alignment done'%i)

(after for loop the rest of the code is in indentation)

[Jayesh6999]

i hope it helps

[yas-nakajima]

Thank you for your reply.

It doesn't work.

import astroalign as aa
from astropy.io import fits

target = fits.getdata('target.fits')
source = fits.getdata('source.fits')

target = target.newbyteorder()
source = source.newbyteorder()

reg_im, footprint = aa.register(source, target)

ValueError: Troublesome data array: [ nan nan nan nan 755. nan nan nan nan nan nan nan
nan 151. nan nan nan nan nan nan nan nan nan nan
nan nan nan nan nan nan nan nan nan nan nan nan
nan nan nan nan nan nan 1296. nan nan nan nan nan
nan nan]

I am using an intel-Mac with OS 10.15.
I did not expect facing an endian issue.

Thanks

[martinberoiz]

Well, at least this time you got a different error, it seems like the endian issues you were having before are not a problem.

The problem now is that you have a bunch of nans in your arrays. Astroalign will not work if there is missing data. You should fill your nan's with some value, like the background median, or gaussian noise.

[yas-nakajima]

The program without .newbyteorder() lines works for the same FITS files with scikit-image==0.15.0.
The FITS files does not contain nans and , for example, DS9 displays the FITS file properly.

It seems .newbyteorder() destroyed data.

target = fits.getdata('target.fits')
source = fits.getdata('source.fits')
print('original')
print(np.max(target), np.min(target)) 
print(np.max(source), np.min(source))

target_new = target.newbyteorder()
source_new = source.newbyteorder()

print('newbyte')
print(np.max(target_new), np.min(target_new)) 
print(np.max(source_new), np.min(source_new))

original
38990.938 345.18137
39516.312 -2829.9492
newbyte
nan nan
nan nan

[martinberoiz]

That is strange. Would you mind sending me the problem images so I can take a look? Thanks.

[yas-nakajima]

Thanks, Martin. I sent you the FITS files.
By the way, the FITSVERIFY yields no errors for them.
I tried with an intel 64bit PC + CentOS 7 and obtained the same results.

[yas-nakajima]

target_fixed = np.array(target, dtype="<f4")
source_fixed = np.array(source, dtype="<f4")

is a workaround.

[yas-nakajima]

- target = target.newbyteorder()
- source = source.newbyteorder()
+ target = target.byteswap().newbyteorder()
+ source = source.byteswap().newbyteorder()

FYI. byteswap() is needed.

Because skimage.transform.warp does not accept Big-endian,
I would suggest adding .byteswap().newbyteorder() in def apply_transform or def _data of astroalign.py.

[martinberoiz]

Yes, this is a known issue that was introduced in a recent version of scikit-image.