Segmentation fault during fir_convolve_impl_mirror_mirror_antisymmetric_avxfmaN

[ssidorenko]

Segmentation fault during fir_convolve_impl_mirror_mirror_antisymmetric_avxfmaN

[svenpeter42]

Thanks for the detailed report. I've heard about this bug before but so far no one had been able to produce a small testcase!

I'm busy teaching this week but will try to fix it afterwards.

@stuarteberg @k-dominik I think this is the bug we've run into a few times already.

[svenpeter42]

Running hessianOfGaussianEigenvalues twice also reliably triggers this segfault for me.
I think the first call overwrites some memory which triggers the fault in the second call.

[ssidorenko]

Interestingly, using two consecutive hessianOfGaussianEigenvalues but with scale and windowSize set to 1.2 (or lower) does not trigger the segmentation fault.

Compiling with FMA and AVX disabled still gives a segmentation fault.

[svenpeter42]

I think in this case the problem is that you try to filter the result of an eigenvalue filter again. Fastfilters will assume 3d convolution for the second one and then fault because one axis only has a length of two.

Did you want to do this or is the testcase just coincidentally faulting and the real error is somewhere else?

[ssidorenko]

Indeed, the testcase is coincidentally faulting. I'll try to make another testcase closer to what I wanted to do.

[ssidorenko]

After more investigation, it turns out that the downsampled data I was using for development was too small (3D array with depth of 12 while computing LoG with window_size=4). I think that #14 should then be the only change needed.