Docker Image does NOT build

[badoo]

Docker Image does NOT build

[anton-povarov]

Hey. This is a known issue with gcc8 (maybe 7 as well). Will fix.
Should definitely work with gcc6.

Try using fedora25 based image, linked in readme.

[alexanderilyin]

@anton-povarov here is a section from README you're talking about:

• https://github.com/anton-povarov/pinba2#docker — Fedora 25 (kinda works)

The link is pointing to this directory:

• https://github.com/anton-povarov/pinba2/tree/master/docker/fedora-25

As you can see from the full log attached in 1st message when I'm trying to build it I'm getting the following error:

AF6428:fedora-25 ailyin$ docker build .
...
make: *** [all] Error 2
Makefile:383: recipe for target 'all' failed
The command '/bin/sh -c /_src/pinba2/docker/build-from-source.sh' returned a non-zero code: 2

P. S.

@anton-povarov also in the log you can see that gcc 6.4.1 was installed. Why are we talking about gcc7/8?

 gcc                      x86_64   6.4.1-1.fc25                 updates    20 M
 gcc-c++                  x86_64   6.4.1-1.fc25                 updates    11 M

[anton-povarov]

Because we've found the same issue with gcc8 internally at badoo.
And i've been using gcc 6.3 to build on fedora.

This is a real issue, gimme a bit of time (should be a day or less) to fix.
The workaround in the meantime might be to downgrade to g++ (GCC) 6.3.1 20161221, that i've had success building with.

Or, go to configure.ac and remove "-Werror" from build options there, there will be warnings but it should build and run fine.

[alexanderilyin]

I've tried to remove -Werror option:

diff --git a/configure.ac b/configure.ac
index aea5c22..264d797 100644
--- a/configure.ac
+++ b/configure.ac
@@ -31,7 +31,7 @@ AC_CHECK_FUNCS([sysconf recvmmsg])

 # compiler flags
 common_flags=" -pthread"
-common_flags="$common_flags -Wall -Wextra -Wformat-security -Werror"
+common_flags="$common_flags -Wall -Wextra -Wformat-security"
 common_flags="$common_flags -Wno-unused -Wno-unused-parameter -Wno-unused-variable -Wno-missing-field-initializers"
 common_flags="$common_flags -fno-omit-frame-pointer"
 common_flags="$common_flags -fPIC -DPIC"

It did not help to build i succesfully and failed with other errors

libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -std=gnu++11 -fno-rtti -pthread -Wall -Wextra -Wformat-security -Wno-unused -Wno-unused-parameter -Wno-unused-variable -Wno-missing-field-initializers -fno-omit-frame-pointer -fPIC -DPIC -D_GNU_SOURCE -D__STDC_FORMAT_MACROS -D_POSIX_SOURCE -maes -msse4 -msse4.2 -O3 -ffast-math -ggdb3 -mtune=native -DMYSQL_DYNAMIC_PLUGIN -DMYSQL_SERVER=1 -I/usr/include -I/usr/include/mysql -I/usr/include/mysql/private -DPINBA_ENGINE_DEBUG_OFF -Wno-error -Wno-macro-redefined -pthread -I/_install/nanomsg/include -I/_src/meow -I/usr/include -I/usr -I.. -I../include -MT plugin.lo -MD -MP -MF .deps/plugin.Tpo -c plugin.cpp  -fPIC -DPIC -o .libs/plugin.o
plugin.cpp: In lambda function:
plugin.cpp:268:43: error: 'STDERR_FILENO' was not declared in this scope
   return std::make_shared<mysql_logger_t>(STDERR_FILENO, log_level);
                                           ^~~~~~~~~~~~~
plugin.cpp: In function 'int pinba_engine_init(void*)':
plugin.cpp:269:4: error: 'void logger' has incomplete type
  }();
    ^
At global scope:
cc1plus: warning: unrecognized command line option '-Wno-macro-redefined'
In file included from ../include/pinba/histogram.h:8:0,
                 from handler.cpp:7:
../include/pinba/hdr_histogram.h: In instantiation of 'bool hdr_histogram___impl_t<CounterT>::increment(const config_t&, int64_t, hdr_histogram___impl_t<CounterT>::counter_t) [with CounterT = unsigned int; hdr_histogram___impl_t<CounterT>::config_t = hdr_histogram_conf_t; int64_t = long int; hdr_histogram___impl_t<CounterT>::counter_t = unsigned int]':
../include/pinba/histogram.h:150:56:   required from here
../include/pinba/hdr_histogram.h:259:27: warning: throw will always call terminate() [-Wterminate]
      throw std::bad_alloc();
                           ^
Makefile:474: recipe for target 'plugin.lo' failed
make[2]: *** [plugin.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
cc1plus: warning: unrecognized command line option '-Wno-macro-redefined'
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -std=gnu++11 -fno-rtti -pthread -Wall -Wextra -Wformat-security -Wno-unused -Wno-unused-parameter -Wno-unused-variable -Wno-missing-field-initializers -fno-omit-frame-pointer -fPIC -DPIC -D_GNU_SOURCE -D__STDC_FORMAT_MACROS -D_POSIX_SOURCE -maes -msse4 -msse4.2 -O3 -ffast-math -ggdb3 -mtune=native -DMYSQL_DYNAMIC_PLUGIN -DMYSQL_SERVER=1 -I/usr/include -I/usr/include/mysql -I/usr/include/mysql/private -DPINBA_ENGINE_DEBUG_OFF -Wno-error -Wno-macro-redefined -pthread -I/_install/nanomsg/include -I/_src/meow -I/usr/include -I/usr -I.. -I../include -MT view_conf.lo -MD -MP -MF .deps/view_conf.Tpo -c view_conf.cpp -o view_conf.o >/dev/null 2>&1
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -std=gnu++11 -fno-rtti -pthread -Wall -Wextra -Wformat-security -Wno-unused -Wno-unused-parameter -Wno-unused-variable -Wno-missing-field-initializers -fno-omit-frame-pointer -fPIC -DPIC -D_GNU_SOURCE -D__STDC_FORMAT_MACROS -D_POSIX_SOURCE -maes -msse4 -msse4.2 -O3 -ffast-math -ggdb3 -mtune=native -DMYSQL_DYNAMIC_PLUGIN -DMYSQL_SERVER=1 -I/usr/include -I/usr/include/mysql -I/usr/include/mysql/private -DPINBA_ENGINE_DEBUG_OFF -Wno-error -Wno-macro-redefined -pthread -I/_install/nanomsg/include -I/_src/meow -I/usr/include -I/usr -I.. -I../include -MT handler.lo -MD -MP -MF .deps/handler.Tpo -c handler.cpp -o handler.o >/dev/null 2>&1
make[2]: Leaving directory '/_src/pinba2/mysql_engine'
Makefile:493: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/_src/pinba2'
Makefile:383: recipe for target 'all' failed
make: *** [all] Error 2
ERROR: Service 'fedora-25' failed to build: The command '/bin/sh -c /_src/pinba2/docker/build-from-source.sh' returned a non-zero code: 2

You can see full log here:

• Log at 2018-10-10 00-08-54.txt

[anton-povarov]

@alexanderilyin these fixes should work for now. please check latest master.

Thank you very much for your pull requests, will definitely try to check them out and accept improvements.
At the moment (vacaction) - i have no time to check them out (not very experienced with docker stuff at the moment anyway, will need some time to learn).

feel free to ping me on direct email as well (come on, we've known each other for years!) :)

[alexanderilyin]

I was able to build image Log at 2018-10-10 16-26-27.txt and start container:

AF6428:pinba2 ailyin$ docker-compose ps
         Name                      Command             State     Ports
------------------------------------------------------------------------
pinba2_fedora-25_run_1   docker-entrypoint.sh mysqld   Up      30003/tcp

But container died Log at 2018-10-10 16-31-54.txt with the following error:

ERROR 1126 (HY000) at line 1: Can't open shared library '/usr/lib64/mysql/plugin/libpinba_engine2.so' (errno: 13, undefined symbol: my_print_error_service)

/usr/lib64/mysql/plugin/libpinba_engine2.so is present and has no zero size:

$ docker exec -ti pinba2_fedora-25_run_1 ls -alFh /usr/lib64/mysql/plugin/libpinba_engine2.so
-rwxr-xr-x 1 root root 31M Oct 10 23:20 /usr/lib64/mysql/plugin/libpinba_engine2.so*

[alexanderilyin]

@tony2001 maybe you can help me with the problem above?

[tony2001]

Wrong MySQL version? Wrong MySQL options?
Did you build MySQL yourself or did you use a prepackaged version?
Does it start without Docker with the same MySQL and libpinba_engine2.so?

[mkevac]

I can reproduce this problem. Didn't have time yet to fix it, but I will try to fix it.

[alexanderilyin]

@tony2001 please see below:

Did you build MySQL yourself or did you use a prepackaged version?
I used prepackaged version, unfortunately, I don't have logs for that package installation Log at 2018-10-10 16-26-27.txt since it was used from the cached layer.

Does it start without Docker with the same MySQL and libpinba_engine2.so?
Unfortunately, I not interested in running it on the host system, so I was not looking into this option.

@mkevac thanks for taking your time, please let me know if you need any help.

[mkevac]

@alexanderilyin I've fixed the build. Docker container is not ideal and we would want to improve it, but build should at least work now.
Can you try it, please?

[alexanderilyin]

@mkevac you're the man! Here is container log after starting:

• Log at 2018-11-01 11-43-34 PM.txt

There are few minor issues with ports in Dockerfile, for example:

pinba2-fedora-25 | 2018-11-02 06:42:11 1 [Note] PINBA: engine initialized on 0.0.0.0:3002

Doesn't match an exposed port:

AF6469:pinba2 ailyin$ docker-compose ps
      Name                    Command               State     Ports
---------------------------------------------------------------------
pinba2-fedora-25   /usr/local/bin/docker-entr ...   Up      30003/tcp

MySQL port is not listed in Dockerfile as well:

pinba2-fedora-25 | Version: '10.1.26-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server

I will take care of this and a few other things in #4. Here some other thing just for the record:

AF6469:pinba2 ailyin$ docker stats --no-stream pinba2-fedora-25
CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
4242c2ae17b2        pinba2-fedora-25    1.04%               324.3MiB / 1.952GiB   16.23%              2kB / 0B            356kB / 545MB       32

AF6469:pinba2 ailyin$ docker top pinba2-fedora-25
PID                 USER                TIME                COMMAND
20282               27                  0:07                mysqld -umysql

P. S.

I didn't get why you need a dedicated user builder but since I'm going to use multistage build it doesn't matter at all.

[alexanderilyin]

Here are couple more smoke checks:

[root@4242c2ae17b2 /]# nc -uvz localhost 3002
Ncat: Version 7.40 ( https://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:3002.
Ncat: UDP packet sent successfully
Ncat: 1 bytes sent, 0 bytes received in 2.01 seconds.

[root@4242c2ae17b2 /]# nc -vz localhost 3306
Ncat: Version 7.40 ( https://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:3306.
Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds.

[root@4242c2ae17b2 /]# mysql -e "SHOW CREATE DATABASE pinba"
+----------+------------------------------------------------------------------+
| Database | Create Database                                                  |
+----------+------------------------------------------------------------------+
| pinba    | CREATE DATABASE `pinba` /*!40100 DEFAULT CHARACTER SET latin1 */ |
+----------+------------------------------------------------------------------+

[alexanderilyin]

Some detils on image/layers size.

AF6469:pinba2 ailyin$ docker images badoo/pinba2:fedora-25
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
badoo/pinba2        fedora-25           af5fdbbf61f9        24 minutes ago      686MB

AF6469:pinba2 ailyin$ docker history badoo/pinba2:fedora-25
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
af5fdbbf61f9        23 minutes ago      /bin/sh -c #(nop)  CMD ["mysqld"]               0B
feb3dd8c92d7        23 minutes ago      /bin/sh -c #(nop)  EXPOSE 30003                 0B
bcd18cc08c42        23 minutes ago      /bin/sh -c #(nop)  ENTRYPOINT ["/usr/local/b…   0B
6551d9fcefc7        23 minutes ago      /bin/sh -c #(nop) COPY file:4c53e3d12cf134a7…   1.89kB
04a82d47e402        23 minutes ago      /bin/sh -c #(nop) COPY file:3e852c5b42805802…   32MB
bac3685f7dbd        23 minutes ago      /bin/sh -c dnf install -y file hostname mari…   421MB
b2cd8a1877f5        About an hour ago   /bin/sh -c #(nop)  MAINTAINER Anton Povarov …   0B
9cffd21a45e3        12 months ago       /bin/sh -c #(nop) ADD file:7e506fea1c7b06484…   232MB
<missing>           12 months ago       /bin/sh -c #(nop)  ENV DISTTAG=f25container …   0B
<missing>           12 months ago       /bin/sh -c #(nop)  MAINTAINER [Adam Miller <…   0B

[alexanderilyin]

@mkevac looks like I got a general idea why you are using a dedicated user builder based on the following:

• https://fedoraproject.org/wiki/Using_Mock_to_test_package_builds
• https://blog.packagecloud.io/eng/2015/05/11/building-rpm-packages-with-mock/

In logs, I see following errors so looks like something is not going as expected and builder not really used?

warning: user mockbuild does not exist - using root
warning: group mockbuild does not exist - using root

[mkevac]

@alexanderilyin I've tried building with root and got an error that "you should not use root for building". Because of that, I've used a dedicated user. It works because the build is succeeding. You can try removing this part.