How to deal with "static inline functions" from a static library?

[modulovalue]

How to deal with "static inline functions" from a static library?

[modulovalue]

It appears to me that (by looking through the amalgamation header) this issue exists for the following functions:

• bitset_size_in_bytes
• bitset_size_in_bits
• bitset_size_in_words
• bitset_set
• bitset_set_to_value
• bitset_get
• bitset_next_set_bit
• bitset_next_set_bits
• bitset_for_each
• bitset_print
• roaring_bitmap_create (can be simulated with roaring_bitmap_create_with_capacity(0))
• roaring_bitmap_init_cleared (can be simulated with roaring_bitmap_init_with_capacity(r, 0))
• roaring_bitmap_get_copy_on_write
• roaring_bitmap_set_copy_on_write
• roaring_bitmap_add_range
• roaring_bitmap_remove_range

If there's no workaround, would you accept a PR that would remove the "static inline" modifiers from the implementations and headers? Oops, they seem to only exist in the header.

[lemire]

Try this, create a header file with the following content (call it "myheader.h"):

static inline void myfunction() {
}

Then create a source file (say "source.c"):

#include "myheader.h"
extern inline void myfunction();

Then compile it:

cc -std=c99 -c source.o

Look at the symbols...

$ objdump -T source.o 

You will find that myfunction is one of the symbols.

[lemire]

The maintainers of ffigen appear to claim that this is the case because static inline functions are not available in static libraries.

We had a few missing 'extern'. It does not explain your issue because many of the functions that you provided had extern declaration. Nevertheless, release 1.2.0 has a more complete list of extern declarations.

[lemire]

I ran the following command nm -gU libroaring.dylib to look for exported symbols, and bitset_set (which is static inline) does not appear to be exported.

As I wrote above, we had a few missing externs. As of 1.2.0, they should be all present. You can verify...

$ cc -std=c99 -c roaring.c && objdump -T roaring.o | grep roaring_bitmap_add_range
In file included from roaring.c:57:
0000000000020150 l       0e SECT   01 0000 [.text] _roaring_bitmap_add_range
000000000001f2bc g       0f SECT   01 0000 [.text] _roaring_bitmap_add_range_closed

$ cc -std=c99 -c roaring.c && objdump -T roaring.o |grep bitset_size_in_bytes
0000000000002d90 l       0e SECT   01 0000 [.text] _bitset_size_in_bytes

So I am going to close this issue.

Pull requests are invited, but we do want several of our functions to be inline.

We have multiple wrappers for CRoaring in other programming languages. I personally wrote two (Go and Swift) and I have contributed to others (e.g., Python, JavaScript). It definitively does work.

[modulovalue]

@lemire Thank you for the quick and thorough response. I agree that inline should be kept (it was intended as a rather naive attempt in proposing a solution as I'm not intimately familiar with the inner workings of the C/C++ toolchains.)

I think the issue still persists. It looks to me like the difference in results can be explained by me trying to make use of CRoaring as a statically linked library (i.e. a dylib on macos) and you are compiling it to an object file, could that be it?

Furthermore, if I search for bitset_set in RoaringBitmap.swift I can't find it. Could it be that most bindings do not make use of bitsets?

If I change the "static" to "extern" in the amalgamated header file, then bitset_set becomes visible:

// get version 1.2.0
./amalgamation.sh
// change some "static" in roading.h to "extern"
cc  -O3 -std=c11 -shared -o libroaring.dylib -fPIC roaring.c
nm -gU libroaring.dylib | grep bitset_set                   
0000000000001f84 T _bitset_set

Could it be that the amalgamated header file is also missing those extern annotations?

[modulovalue]

I.e., when I change the static here to extern, then that appears to solve the issue in the issue description:

CRoaring/include/roaring/bitset/bitset.h

Line 91 in e29535b

static inline void bitset_set(bitset_t *bitset, size_t i) {

[modulovalue]

Proposed solution: #489

[lemire]

The extern qualifier is fine but we can’t export the same function more than once in the library. This means that, typically, you must use the extern declarations in the source files, not in the header files.

[modulovalue]

Got it, thank you. See: #489 (comment)

[lemire]

statically linked library

Note that dylib refers to a shared library, not a static library.

[lemire]

Furthermore, if I search for bitset_set in RoaringBitmap.swift I can't find it. Could it be that most bindings do not make use of bitsets?

The bindings rarely cover the entire API. However, the Swift wrapper includes roaring_bitmap_add_range.

https://github.com/RoaringBitmap/SwiftRoaring/blob/7c89e041370103d7956fc977821d72391eb4d768/Sources/SwiftRoaring/RoaringBitmap.swift#L514

The Python wrapper also includes it...

https://github.com/Ezibenroc/PyRoaringBitMap/blob/89c361ab0961cd3b51249dde4fce95937ec10bdd/pyroaring/bitmap.pxi#L243

The D wrapper includes it...

https://github.com/yuce/droaring/blob/e1e0e53ae6b37c22da1ecc101888d11e4b48c2dc/source/roaring/roaring.d#L224

The Zig wrapper includes it...

https://github.com/jwhear/roaring-zig/blob/779eccf26cdc6c807f0d1f0830c2d8ed4bd2e3bc/src/roaring.zig#L233

I could keep going...

[modulovalue]

@lemire thank you for pointing this out. I apologize for any confusion that my misuse of terminology must have caused.