Investigate static linking and/or support other binary distributions.

[comby-tools]

Investigate static linking and/or support other binary distributions.

[abitrolly]

The same problem with Fedora 31.

$ comby
comby: error while loading shared libraries: libpcre.so.3: cannot open shared object file: No such file or directory
$ sudo dnf provides "/usr/lib64/libpcre.so*"
Last metadata expiration check: 0:02:46 ago on Tue 21 Jan 2020 07:32:56 AM +03.
pcre-8.43-2.fc31.1.x86_64 : Perl-compatible regular expression library
Repo        : @System
Matched from:
Filename    : /usr/lib64/libpcre.so.1
Filename    : /usr/lib64/libpcre.so.1.2.11

pcre-8.43-2.fc31.1.x86_64 : Perl-compatible regular expression library
Repo        : fedora
Matched from:
Filename    : /usr/lib64/libpcre.so.1
Filename    : /usr/lib64/libpcre.so.1.2.11

pcre-devel-8.43-2.fc31.1.x86_64 : Development files for pcre
Repo        : fedora
Matched from:
Filename    : /usr/lib64/libpcre.so

[rvantonder]

Appreciate the note. My plan is to release a cross platform implementation in Javascript through npm in the next couple of weeks, which will help with this issue.

[steinuil]

Just as a heads up, I installed Comby on Fedora 35 Workstation with the curl | bash method and it also complained about not finding libcpre3.

Apparently Fedora doesn't link libpcre3 with the version number in /lib64 so I fixed it by replacing the name of the library from libpcre.so.3 to libcpre.so with patchelf.

sudo patchelf --replace-needed libpcre.so.3 libpcre.so $(which comby)

I'm not very familiar with how this works on the build side but maybe libpcre.so could also be added as a possible name of the dynamic library?

[rvantonder]

#129 comby is now distributed under homebrew for Mac OS X

[abitrolly]

Docker image works for initial example.

✗ cat ~/bin/comby 
#!/bin/bash

podman run -a stdin -a stdout -i --rm comby/comby "$@"

[coolaj86]

I just ran into this.

I am glad that Hombrew works, however, having to install 400+mb of Homebrew for a small library is less than ideal. I'm hoping for a static build in the future. :)

[PapyElGringo]

Appreciate the note. My plan is to release a cross platform implementation in Javascript through npm in the next couple of weeks, which will help with this issue.

Is that done?

[cal101]

Long time unix sysop here. Besides doing a static link which may be problematic to some build/link environments it often is possible to set dynamic library pathes. So it may be possible to bundle a SHARED pcre library and just call comby via a shell script wrapper that sets those path(es). I remember some support for alternative pathes inside the binary but don't remember if that includes relative pathes.
If there is any interest I think I should be able to provide an example for shell wrapping.

[rvantonder]

@PapyElGringo I haven't packaged a JS lib on npm yet. But if you want to experiment with this, have a look at the .js files here: https://github.com/comby-tools/comby.js/tree/master/js. See this codepen for using comby.js. One note though, comby.js doesn't support regular expressions inside holes.

@cal101

via a shell script wrapper that sets those path(es).

Interesting, are there examples of other projects that use this? If it can be done robustly, that might be a nice idea.

[cal101]

I don't know if some project uses this.
We used it locally to install multiple versions or some package but avoiding absolute pathes.

The idea is as follow:

cd /tmp
mkdir comby
cd comby/
cp /home/cal/cal01/bin/comby .
ldd comby 
	linux-vdso.so.1 (0x00007ffcd1970000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe96c61d000)
	libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007fe96c314000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fe96c0f7000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fe96be85000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fe96bc7d000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe96b8df000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe96b6db000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe96b2ea000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fe96e4e6000)
mkdir lib
cp /lib/x86_64-linux-gnu/libpcre.so.3 lib
LD_LIBRARY_PATH=$PWD/lib:${LD_LIBRARY_PATH} ldd comby
	linux-vdso.so.1 (0x00007fff0f9c7000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f4de43d9000)
	libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f4de40d0000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f4de3eb3000)
	libpcre.so.3 => /tmp/comby/lib/libpcre.so.3 (0x00007f4de3c41000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f4de3a39000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4de369b000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4de3497000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4de30a6000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f4de62a2000)
LD_LIBRARY_PAH=$PWD/lib:${LD_LIBRARY_PATH} comby -h
Run a rewrite pass. Comby runs in current directory by default. The '-stdin' option rewrites input on stdin.

  comby [MATCH_TEMPLATE REWRITE_TEMPLATE [FULL_FILE_PATHS_OR_FILE_SUFFIXES ...]]
...

[aryx]

Note that we recently had a similar linking problem with PCRE in semgrep, and a solution a colleague found was to
delete the pcre.dylib file before compiling semgrep, to force static linking.
see https://github.com/returntocorp/semgrep/pull/2629/files

[Strum355]

I've managed to implement (almost fully) statically linked comby via Nix. I've included muslc and glibc targets. Going to tidy it up a bit yet, and it may not ever get merged in that repo, but someone may find some inspiration from it. I saw some mention of ppx using dlopen which may be problematic under statically linked binaries, but I've not seen that be hit in some light testing.
sourcegraph/sourcegraph#49526

glibc:

# ldd ./result/bin/comby
        linux-vdso.so.1 (0x00007ffd19f9d000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f561447c000)
        librt.so.1 => /usr/lib/librt.so.1 (0x00007f5614477000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007f561438f000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f56141a8000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f5614483000)

musl:

# ldd comby 
        /lib/ld-musl-x86_64.so.1 (0x7fe5466d2000)
        libc.so => /lib/ld-musl-x86_64.so.1 (0x7fe5466d2000)