CREATE EXTENSION multicorn, undefined symbol: sem_wait

[segfault-inc]

CREATE EXTENSION multicorn, undefined symbol: sem_wait

[rdunklau]

How did you install multicorn ?
What version of python are you using ?
What is the output of running pg_config ?

Thanks !

[johnmudd]

install multicorn:
git clone git://github.com/Kozea/Multicorn.git
cd Multicorn
make && make install

Python 2.7.5

$ pg_config
BINDIR = /home/mudd/pkgs/postgresql-9.3.0.install/bin
DOCDIR = /home/mudd/pkgs/postgresql-9.3.0.install/share/doc
HTMLDIR = /home/mudd/pkgs/postgresql-9.3.0.install/share/doc
INCLUDEDIR = /home/mudd/pkgs/postgresql-9.3.0.install/include
PKGINCLUDEDIR = /home/mudd/pkgs/postgresql-9.3.0.install/include
INCLUDEDIR-SERVER = /home/mudd/pkgs/postgresql-9.3.0.install/include/server
LIBDIR = /home/mudd/pkgs/postgresql-9.3.0.install/lib
PKGLIBDIR = /home/mudd/pkgs/postgresql-9.3.0.install/lib
LOCALEDIR = /home/mudd/pkgs/postgresql-9.3.0.install/share/locale
MANDIR = /home/mudd/pkgs/postgresql-9.3.0.install/share/man
SHAREDIR = /home/mudd/pkgs/postgresql-9.3.0.install/share
SYSCONFDIR = /home/mudd/pkgs/postgresql-9.3.0.install/etc
PGXS = /home/mudd/pkgs/postgresql-9.3.0.install/lib/pgxs/src/makefiles/pgxs.mk
CONFIGURE = '--with-python' '--prefix=/home/mudd/pkgs/postgresql-9.3.0.install'
CC = gcc
CPPFLAGS = -D_GNU_SOURCE
CFLAGS = -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
CFLAGS_SL = -fpic
LDFLAGS = -L../../../src/common -Wl,--as-needed -Wl,-rpath,'/home/mudd/pkgs/postgresql-9.3.0.install/lib',--enable-new-dtags
LDFLAGS_EX = 
LDFLAGS_SL = 
LIBS = -lpgport -lpgcommon -lz -lreadline -lcrypt -ldl -lm 
VERSION = PostgreSQL 9.3.0
$

[johnmudd]

-lpthread is not listed in LIBS above in pg_config output. I think that's the problem.

[johnmudd]

Progress!

All I needed to do was add -lpthread when building postgres.

$ cd postgresql-9.3.0
$ export LIBS=-lpthread
$ ./configure --with-python --prefix=$(pwd).install
$ make; make install

Now -lpthread appears in the pg_config output:

$ pg_config | grep LIBS
CONFIGURE = '--with-python' '--prefix=/home/mudd/pkgs/postgresql-9.3.0.install' 'LIBS=-lpthread'
LIBS = -lpgport -lpgcommon -lz -lreadline -lcrypt -ldl -lm -lpthread
$

Then rebuild, install Multicorn.

Then create extension:

$ echo 'CREATE EXTENSION multicorn;' | psql postgres
ERROR:  could not load library "/home/mudd/pkgs/postgresql-9.3.0.install/lib/multicorn.so": /home/mudd/pkgs/postgresql-9.3.0.install/lib/multicorn.so: undefined symbol: openpty
$

It still fails but it's looking for a different function now. I'll report back once I have the complete list of libs I need to add when building postgres.

[johnmudd]

openpty requires -lutil

So I set LIBS='-lpthread -lutil', reconfigured and built postgres.

-lutil appears in the pg_config output:
LIBS = -lpgport -lpgcommon -lz -lreadline -lcrypt -ldl -lm -lpthread -lutil

But it doesn't appear in ldd output for postgres executable:

$ ldd ~/pkgs/postgresql-9.3.0.install/bin/postgres | grep util
$

The pthread lib is present though.

$ ldd ~/pkgs/postgresql-9.3.0.install/bin/postgres | grep thread
        libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xb7700000)
$

Currently the 'CREATE EXTENSION multicorn;' still fails with "undefined symbol: openpty"

[johnmudd]

Looks like I need to build postgres with clang instead of gcc if I want libutil included.

$ gcc test.c -o test
$ ldd test
        linux-gate.so.1 =>  (0xb7713000)
        libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb7543000)
        /lib/ld-linux.so.2 (0xb7714000)
$ gcc test.c -lutil -o test
$ ldd test
        linux-gate.so.1 =>  (0xb7765000)
        libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb7595000)
        /lib/ld-linux.so.2 (0xb7766000)
$ clang test.c -o test
$ ldd test
        linux-gate.so.1 =>  (0xb77cd000)
        libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb75fd000)
        /lib/ld-linux.so.2 (0xb77ce000)
$ clang test.c -lutil -o test
$ ldd test
        linux-gate.so.1 =>  (0xb7740000)
        libutil.so.1 => /lib/i386-linux-gnu/libutil.so.1 (0xb771f000)
        libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb756b000)
        /lib/ld-linux.so.2 (0xb7741000)
$ 

[rdunklau]

What OS are you running ?
Multicorn builds fine with gcc. Is there any OS-provided package of postgresql 9.3 ? If so, you should try to build against it.

[johnmudd]

Ubuntu 13.04

$ uname -a
Linux mudd-Precision-WorkStation-T3400 3.8.0-31-generic #46-Ubuntu SMP Tue Sep 10 19:56:49 UTC 2013 i686 i686 i686 GNU/Linux
$

I didn't mean to imply that I would build Multicorn with clang, only postgres. Alas, I just tried clang for postgres and the postgres executable still does not include libutil. I'll go back to gcc and see if there is an option to help me.

I could install a prebuilt postgres but that won't help me in the long run. I have to build postgres on a very old Linux so that it will run on any of our customer machines. For now I'm just testing on modern Unbntu.

[johnmudd]

Problem solved. I think.

I had to force the inclusion of libutil in the postgres executable by adding a dummy call to openpty() at the top of PostgresMain() in postgres.c as shown below. For some reason this step was not necessary to get libpthread included. I went ahead and added a call to pthread_exit() anyway just to be sure.

The "if" statement is necessary so that the call is never executed. I used the condition "argc < 0" because it was the first thing that I could think of that will always be false but there's no way for the compiler to know this and optimize away the calls.

Diff of postgresql-9.3.0/src/backend/tcop/postgres.c:

diff -bw -r1.1 postgres.c

 * NOTES
 *    this is the "main" module of the postgres backend and
 *    hence the main module of the "traffic cop".
 *
 *-------------------------------------------------------------------------
 */

#include "postgres.h"

21a22,24
> #include <pthread.h>
> #include <pty.h>
> 

void
PostgresMain(int argc, char *argv[],
             const char *dbname,
             const char *username)
{
    int         firstchar;
    StringInfoData input_message;
    sigjmp_buf  local_sigjmp_buf;
    volatile bool send_ready_for_query = true;
3521a3525,3529
>     if (argc < 0) {  // Always false but but the compiler doesn’t know this.
>         pthread_exit(0);
>         openpty(0,0,0,0,0);
>     }
> 

I still need to set LIBS before configuring postgres.

export LIBS='-lpthread -lutil'

Here's the final result:

$ which postgres
/home/mudd/pkgs/postgresql-9.3.0.install/bin//postgres
$ ldd /home/mudd/pkgs/postgresql-9.3.0.install/bin//postgres
        linux-gate.so.1 =>  (0xb77b4000)
        libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xb7792000)
        libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xb774e000)
        libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xb7733000)
        libutil.so.1 => /lib/i386-linux-gnu/libutil.so.1 (0xb772f000)
        libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb757c000)
        /lib/ld-linux.so.2 (0xb77b5000)
$
$ echo 'CREATE EXTENSION multicorn;' | psql postgres
CREATE EXTENSION
$