examples build KO

[rimaille]

examples build KO

[undisbeliever]

superfamiconv also crashes on my machine (Arch Linux x64, g++).

gdb wasn't helpful, so I switched to valgrind, which gave:

==15484== Invalid write of size 4
==15484==    at 0x43DA91: void std::__unguarded_linear_insert<__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, __gnu_cxx::__ops::_Val_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}> >(__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, __gnu_cxx::__ops::_Val_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}>) (stl_algo.h:1827)
==15484==    by 0x439B5F: void std::__unguarded_insertion_sort<__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, __gnu_cxx::__ops::_Iter_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}> >(__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, __gnu_cxx::__ops::_Iter_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}>, __gnu_cxx::__ops::_Iter_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}>) (stl_algo.h:1864)
==15484==    by 0x433AAF: void std::__final_insertion_sort<__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, __gnu_cxx::__ops::_Iter_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}> >(__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, __gnu_cxx::__ops::_Iter_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}>, __gnu_cxx::__ops::_Iter_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}>) (stl_algo.h:1883)
==15484==    by 0x42FAC0: void std::__sort<__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, __gnu_cxx::__ops::_Iter_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}> >(__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, __gnu_cxx::__ops::_Iter_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}>, __gnu_cxx::__ops::_Iter_comp_iter<sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}>) (stl_algo.h:1968)
==15484==    by 0x42D571: void std::sort<__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}>(__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}, sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&)::{lambda(unsigned int const&, unsigned int const&)#1}) (stl_algo.h:4739)
==15484==    by 0x42C60A: sfc::sort_colors(std::vector<unsigned int, std::allocator<unsigned int> >&) (Common.h:329)
==15484==    by 0x42CF9B: sfc::Subpalette::sort() (Palette.h:63)
==15484==    by 0x42A6B6: sfc::Palette::sort() (Palette.cpp:51)
==15484==    by 0x41B3A5: sfc_palette(int, char**) (sfc_palette.cpp:159)
==15484==    by 0x4077B9: main (superfamiconv.cpp:225)

Testing has confirmed that the bug is in the comparison lambda in sfc::sort_colors. Apparently gnu libstdc++ std::sort can write outside the vector and crash if the comparison function does not honour strict weak ordering.

I couldn't see any reason why sfc::rgba_color::operator> did not honour strict weak ordering, but on a whim I did this:

diff --git a/src/Common.h b/src/Common.h
index 1254762..783a8cc 100644
--- a/src/Common.h
+++ b/src/Common.h
@@ -320,7 +320,7 @@ inline bool rgba_color::operator>(const rgba_color& o) const {
   int lo = (int)(segments * sqrt(0.241f * o.r + 0.691f * o.g + 0.068f * o.b));
   int vo = (int)(segments * hsva_o.v);
 
-  return (h > ho) ? true : (l > lo) ? true : (v > vo) ? true : false;
+  return std::tie(h, l, v) > std::tie(ho, lo, vo);
 }
 
 inline void sort_colors(std::vector<rgba_t>& colors) {

And the crash went away on my system.

[Optiroc]

Hi! Hmm, yeah, SuperFamiconv seems to struggle with large input images when built with gcc (like the Mode 7 backgrounds in those examples). I'll try to get a decent debugging environment up and running to diagnose those...

The SuperFX example isn't supposed to do anything but set the background color – it's just a test for the assembly output basically.

[rimaille]

Ok for SuperFX example, so sfc works. But i did provided the output for this error :
ld65: Error: Missing memory area assignment for segment `ZNMI'
make: *** [../../libSFX.make:182: SuperFX.sfc] Error 1
exit code 2

I tried compiling SuperFamiconv with clang++, but i got problems with the linker. I switched ld to gold, -fuse-ld=gold as many forums said, but no luck so far (invalid character/syntax error invalid end in *.o).

If you need help to diagnose the famiconv problem, feel free to ask, i'll do my best.

[Optiroc]

Ah, good catch. I need to update the memory map for that one.

[Optiroc]

The GSU/SuperFX memory map is now fixed.

Regarding the trouble with SuperFamiconv: do you have any experience with debugging using gdb (or any graphical frontend for it)? It would certainly be of great help to get more information about the crashes!

[rimaille]

Thanks, i can confirm that SuperFX example is now building without error.

Unfortunately, i have no experience with debugging using gdb. However, i uploaded the core file, with the binary compiled with gcc (debug enabled) :

core.zip
superfamiconv.zip

I don't know if it's usefull, please tell me what i can do :
http://sprunge.us/bajV
(please note the truncated arguments Core was generated by `../../tools/superfamiconv/bin/superfamiconv -9 ette -v --colors 128 --in-image'.)

[Optiroc]

Wow, that's really helpful, thanks! It's amazing how many things works just fine using clang/libc++ that wreaks complete havoc with gcc/libstdc++, actually.

[rimaille]

@undisbeliever your patch worked for me, thank you

[Optiroc]

@undisbeliever Ohh, you're a star!

[rimaille]

@Optiroc

On a side, when i launched make for Mode7-Transform, i had to modify the shebang of Sin.py
Yours was pointing to /usr/local/bin when mine and potentially all linux distribution point to /usr/bin
Is there any way to make it "universal" (i mean that could work out of the box for all of us) ?

[Optiroc]

@rimaille Ah, that's really just an old habit of mine, but in this case there's really no reason not to use the python distributed with the OS (even macOS doesn't lag behind too horribly on python 2.x anymore).