Building for Go

[animetosho]

Building for Go

[mnightingale]

I've pushed my currently working one to https://github.com/mnightingale/go-yencode

The main things I needed were set(CMAKE_MACOSX_RPATH OFF) because otherwise on Mac the install_path was being prefixed with @rpath/ and Go and then couldn't find it.

Then as with the plain Makefile I'm making a shared library and just giving it all the sources.

At some point I want to see how I can make it work for multiple platforms, I'm not sure how at this stage.
Go will find all the .syso files and I believe use their name to determine which to load, so I can have avx2_amd64.syso, avx2_386.syso, avx2_arm.syso, etc.

I haven't tried it yet but I thought maybe I could split it and produce yencoder_avx2_arm.syso etc. with just the encoder/decoder, if that won't work I could just make a single shared library for each platform with all the sources, yencode_amd64.syso, yencode_arm.syso, etc. - I think this is probably simpler.

---

I haven't had chance to see how fast it is yet but I'm expecting great things!

[animetosho]

For your build, maybe the dependent objects aren't being included in your library. I don't believe a static library actually does any linking - it just packages up the object files into an archive, so the target_link_libraries directive might not be doing much.
You can use ar -t [library file] command to list the contents of your .syso file - it should contain all the compiled objects (a .o file for every .cc file compiled), otherwise you'll be missing symbols during link.

The source files are separated with different options to cater for running a single binary on a variety of systems. If you're just testing and don't need to distribute the compiled binary, it'll be easier to just throw all the source files together with -march=native compile flag, and forget the other -m* flags.

If you do want to build a distributable binary, perhaps include all the source files in a single library, but change the compile options for the files specifically (try set_source_files_properties([file] PROPERTIES COMPILE_FLAGS [flags])).

I'm not sure how Go does it's linking with a static library, but an alternative to having it all in one library, might be to have multiple (like you have now), but explicitly link the Go code to all of them.

---

do_decode was meant to be called from C/C++, as it's a simple inlined wrapper around a function pointer. For a library, you'd need to make a small exported function which calls do_decode.

Enums are effectively labelled numbers, so you can typecast YencDecoderState to/from an int.

I had thought about making a C library out of this at some point, but never got around to it. Maybe it's time...

[mnightingale]

Thanks so much, your comments pointed me in the right direction.

It seems my biggest issue apart from as you suggested that I should be making a shared library was even the times I had CMake setup nearly correctly it was failing to link or compile.

I think it was using gcc instead of g++ or equivalent, I was getting errors like:

Undefined symbols for architecture x86_64:
  "vtable for __cxxabiv1::__class_type_info", referenced from:
      typeinfo for crcutil_interface::CRC in interface.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "vtable for __cxxabiv1::__si_class_type_info", referenced from:
      typeinfo for crcutil_interface::Implementation<crcutil::GenericCrc<crcutil::uint128_sse2, crcutil::uint128_sse2, unsigned long, 4>, crcutil::RollingCrc<crcutil::GenericCrc<crcutil::uint128_sse2, crcutil::uint128_sse2, unsigned long, 4> > > in interface.o
...

For now as suggested I've gone back to basics, made a Makefile in the yencode directory and called g++ with -march=native and a few of the cflags, added a small wrapper to simplify init and I think so that do_decode is exported and it works!

Result length: 128
Decoded: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]

Have you any idea what I'd need to make CMake work?

I would like to be able to create distributable package eventually, so instead of a single package ideally I'd create libyencode_sse2_amd64.syso, etc. and it should load them as needed.

[mnightingale]

I've figured out getting cmake to generate a working -march=native library which is good enough for now.

[animetosho]

Glad you got it working!

Have you any idea what I'd need to make CMake work?

I'll look at writing one up.

[animetosho]

I knocked together a quick C library: https://github.com/animetosho/rapidyenc
Hopefully the CMake configuration there might be of some help.

[mnightingale]

That's great thanks!