Integer overflow issue with large image (copy and FL_RGB_SCALING_NEAREST)

[fire-eggs]

Integer overflow issue with large image (copy and FL_RGB_SCALING_NEAREST)

[fire-eggs]

Further analysis suggests that (unsigned int) would work as well, at least for this image.

[Albrecht-S]

Thanks for the report and the patch, this appears to be a bug. However, the real solution might be more involved, at least in FLTK 1.4.

Note 1: Regarding FLTK 1.3 and 1.4 (patch given above): I'd suggest to keep the signedness (sy and line_d are both int's) and use (long) for the cast rather than (ulong).

Note 2: I'm afraid that there are similar overflows in bilinear scaling mode below (lines 506 etc. in current 1.4 (master) code):

  memcpy(left, array + lefty * line_d + leftx * d(), d());
  ...

If this is true then this should be fixed as well. It's just an observation, I don't have enough time right now to dig deeper.

[fire-eggs]

Thank you for the follow-up. When I had a chance to try bilinear scaling mode, I would report any bug there as well.