dh_proxy_1 not running

dh_proxy_1 not running

[alexbn71]

Identical issue here on a fresh CentOS 7

+ [ ! -f /etc/ssl/ssl_certificate -o ! -f /etc/ssl/ssl_certificate_key ]
+ openssl req -x509 -newkey rsa:2048 -keyout /etc/ssl/ssl_certificate_key -out /etc/ssl/ssl_certificate -days 365 -nodes -subj /CN=localhost -sha256
Can't open /usr/lib/ssl/openssl.cnf for reading, No such file or directory
140652013532416:error:02001002:system library:fopen:No such file or directory:../crypto/bio/bss_file.c:74:fopen('/usr/lib/ssl/openssl.cnf','r')
140652013532416:error:2006D080:BIO routines:BIO_new_file:no such file:../crypto/bio/bss_file.c:81:
Generating a 2048 bit RSA private key
...............................+++
..................................+++
writing new private key to '/etc/ssl/ssl_certificate_key'
-----
unable to find 'distinguished_name' in config
problems making Certificate Request
140652013532416:error:0E06D06A:configuration file routines:NCONF_get_string:no conf or environment variable:../crypto/conf/conf_lib.c:272:

Linux xxx.ovh.net 3.10.0-693.21.1.el7.x86_64 #1 SMP Wed Mar 7 19:03:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Containers: 9
 Running: 6
 Paused: 0
 Stopped: 3
Images: 10
Server Version: 1.13.1
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: docker-runc runc
Default Runtime: docker-runc
Init Binary: docker-init
containerd version:  (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1)
runc version: N/A (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f)
init version: N/A (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
 seccomp
  WARNING: You're not using the default seccomp profile
  Profile: /etc/docker/seccomp.json
Kernel Version: 3.10.0-693.21.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 3
CPUs: 1
Total Memory: 1.749 GiB
Name: xxx.ovh.net
ID: xxx
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Registries: docker.io (secure)

[sitnik]

Due to issues that it created and can possibly create I removed generation of self-signed certificates in devicehive-proxy. You can still use your own TLS certificate, mounted as Docker volumes.
Issue resolved in release 3.5.0.

[sitnik]

I've tested fresh devicehive-docker-3.4.5.1 deployment on several distributions and can't reproduce this issue. For example it starts fine on CentOS 7 with updates (m4.large AWS instance, 30GB GP2 root volume). Following procedure were used to configure server and launch DeviceHive on this machine:

sudo yum install -y epel-release
sudo yum update -y
sudo yum install -y docker python2-pip
sudo pip install -U docker-compose
sudo systemctl enable docker
sudo reboot
<reconnect to instance>
curl -L -o - https://github.com/devicehive/devicehive-docker/archive/3.4.5.1.tar.gz | tar xzf -
cd devicehive-docker-3.4.5.1/rdbms-image/
sudo docker-compose up -d
sudo docker logs -f rdbms-image_dh_proxy_1

dh_proxy container output, without issues and restarts:

+ [ ! -f /etc/ssl/dhparam.pem ]
+ openssl dhparam -out /etc/ssl/dhparam.pem 2048
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
..........................................................................................................+............................................................+...........................................................................................+..........................+........................................................................................................................+.+........+...................................................................................................................................................................................................................................+....................................................................+...............+.........................................................................................................................................................................................................................................................+......................................................................................................+........................................................+...+........+............................................................................+................................................+.............................................+........................................................................+.......................+..................................................................................................................................................................................................................................................+.............................................................................................................................................................................................+....+.................................................................................................................+...........................................................+.....................+......+......+.......................................................................................+...........................................................................+.................................................+.............................................................+..................................................................................................................................................................................................+...................................................................................................................................................................................................................................................................................................................................................+.............................+........................................................+......................+..............................................++*++*
+ [ ! -f /etc/ssl/ssl_certificate -o ! -f /etc/ssl/ssl_certificate_key ]
+ openssl req -x509 -newkey rsa:2048 -keyout /etc/ssl/ssl_certificate_key -out /etc/ssl/ssl_certificate -days 365 -nodes -subj /CN=localhost -sha256
Generating a 2048 bit RSA private key
...................................................+++
....................................................................+++
writing new private key to '/etc/ssl/ssl_certificate_key'
-----
+ awk BEGIN{ORS=" "} $1=="nameserver" {print $2} /etc/resolv.conf
+ echo resolver 127.0.0.11 ;
+ getent hosts dh_plugin
+ getent hosts wsproxyext
+ nginx
2018/04/13 16:43:20 [warn] 10#10: "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/ssl/ssl_certificate"
nginx: [warn] "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/ssl/ssl_certificate"

Docker engine version:

Client:
 Version:         1.13.1
 API version:     1.26
 Package version: <unknown>
 Go version:      go1.8.3
 Git commit:      774336d/1.13.1
 Built:           Wed Mar  7 17:06:16 2018
 OS/Arch:         linux/amd64

Server:
 Version:         1.13.1
 API version:     1.26 (minimum version 1.12)
 Package version: <unknown>
 Go version:      go1.8.3
 Git commit:      774336d/1.13.1
 Built:           Wed Mar  7 17:06:16 2018
 OS/Arch:         linux/amd64
 Experimental:    false

Docker Compose version:

docker-compose version 1.21.0, build 5920eb0
docker-py version: 3.2.1
CPython version: 2.7.5
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013

In next few days I'm planning to release new version of devicehive-proxy container with updated base nginx image, and issue another fixup release for devicehive-docker.

[orino76]

I also ran into the same issue as well. It seems that OpenSSL no longer included in nginx docker image according to nginxinc/docker-nginx#182.

The log of dh_proxy docker image is as following:

+ openssl dhparam -out /etc/ssl/dhparam.pem 2048 ./proxy-start.sh: 9: ./proxy-start.sh: openssl: not found

Please help to have a look. thank you.

[steiniks]

It looks like dh_proxy_1 is missing openssl support and breaks on startup check / creating selfsigned certificate.
To get it running you can do:

openssl req -x509 -newkey rsa:2048 -keyout /var/lib/docker/volumes/rdbms-image_dh-proxy-ssl/_data/ssl_certificate_key -out /var/lib/docker/volumes/rdbms-image_dh-proxy-ssl/_data/ssl_certificate -days 365 -nodes -subj /CN=localhost -sha256;openssl dhparam -out /var/lib/docker/volumes/rdbms-image_dh-proxy-ssl/_data/dhparam.pem 2048

To use ssl see: https://github.com/devicehive/devicehive-docker/tree/master/rdbms-image#using-custom-certificate

[sitnik]

Thanks for report, I'll check what is wrong with dh_proxy.

[tomyc]

The same problem. I've checked with Docker under Windows and Centos 7(Google Cloud instance). With command docker-compose ps, I have the same situation on both OS:
rdbms-image_dh_proxy_1 /bin/sh ./proxy-start.sh Restarting

[sitnik]

@bpl3 , @tomyc, I suspect that issue can be in Docker itself, because file /usr/lib/ssl/openssl.cnf exists in devicehive-proxy:3.4.5.1 image. Could you please provide versions of Docker Engine in your environment and versions of Linux distrubitions. Thanks.

[tomyc]

@sitnik, I can confirm, that according to the procedure you've submitted, I was able to run the instance of devicehive on Google Compute Engine.
So, that is clear for me that the problem was in wrong installation procedure that I've prepared myself upon the Readme file.
Then you very much for your help and support.

The successful specification:

VM instance:

Machine type: custom (6 vCPUs, 22 GB memory)
CPU platform: Intel Skylake

Centos:
centos-release-7-4.1708.el7.centos.x86_64

Docker

Client:
 Version:         1.13.1
 API version:     1.26
 Package version: <unknown>
 Version:         1.13.1
 Go version:      go1.8.3
 Git commit:      774336d/1.13.1
 Built:           Wed Mar  7 17:06:16 2018
 OS/Arch:         linux/amd64
Server:
 Version:         1.13.1
 API version:     1.26 (minimum version 1.12)
 Package version: <unknown>
 Go version:      go1.8.3
 Git commit:      774336d/1.13.1
 Built:           Wed Mar  7 17:06:16 2018
 OS/Arch:         linux/amd64
 Experimental:    false

Docker-compose
docker-compose version 1.21.0, build 5920eb0

[sitnik]

@orino76, this issue was resolved in devicehive-proxy-3.4.5.1, just with added openssl installation in image.
devicehive-proxy-3.4.5.1 should be used now, it references the right devicehive-proxy version.

And thank you for posting link to docker-nginx ticket, we probably should rethink self-singed certificate generation as suggested in it.

[bahuma20]

I updated to devicehive-proxy-3.4.5.1 but i still cannot access the admin interface.
In the logs i geht this error messages: https://pastebin.com/sx3SiJYJ
Something about the nginxconf file missing...

[sitnik]

@bahuma20, similar problem already reported in this issue. It's probably related to outdated version of Docker daemon, because mentioned file /usr/lib/ssl/openssl.cnf do exists in devicehive-proxy image. I can't reproduce it my environments where every distribution updated to latest available package versions (CentOS, Fedora, Arch).

Which distribution you run and which version of Linux kernel and Docker daemon you have installed?
uname -a
sudo docker info

[bahuma20]

@sitnik Here is the output of the commands:

Linux zw-testing 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Containers: 9
 Running: 7
 Paused: 0
 Stopped: 2
Images: 9
Server Version: 18.03.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.15.0-20-generic
Operating System: Ubuntu 18.04 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.9GiB
Name: zw-testing
ID: 3KKI:KSWM:DJMA:QVS3:AW6F:2YN3:6YVO:QHCM:2BWB:IOBY:S4OQ:SMJR
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

[aldrinc]

@sitnik Having a similar issue as @bahuma20 on Mac using 3.4.5.1 with running the proxy.

Docker daemon is updated to latest version.

docker-compose version 1.21.1, build 5a3f1a3

Containers: 17
 Running: 0
 Paused: 0
 Stopped: 17
Images: 21
Server Version: 18.03.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.87-linuxkit-aufs
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.786GiB
Name: linuxkit-025000000001
ID: IUMX:KB3E:FYGC:GNDG:N5PG:CZPJ:AVSF:4JDG:S46L:PH7J:JHKC:IJM4
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 21
 Goroutines: 40
 System Time: 2018-05-10T22:47:10.512037286Z
 EventsListeners: 2
HTTP Proxy: docker.for.mac.http.internal:3128
HTTPS Proxy: docker.for.mac.http.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Error I'm facing

+ [ ! -f /etc/ssl/dhparam.pem ]
+ [ ! -f /etc/ssl/ssl_certificate -o ! -f /etc/ssl/ssl_certificate_key ]
+ openssl req -x509 -newkey rsa:2048 -keyout /etc/ssl/ssl_certificate_key -out /etc/ssl/ssl_certificate -days 365 -nodes -subj /CN=localhost -sha256
Can't open /usr/lib/ssl/openssl.cnf for reading, No such file or directory
140312761206016:error:02001002:system library:fopen:No such file or directory:../crypto/bio/bss_file.c:74:fopen('/usr/lib/ssl/openssl.cnf','r')
140312761206016:error:2006D080:BIO routines:BIO_new_file:no such file:../crypto/bio/bss_file.c:81:
Generating a 2048 bit RSA private key
...............................................................+++
..................................+++
writing new private key to '/etc/ssl/ssl_certificate_key'
req: Can't open "/etc/ssl/ssl_certificate_key" for writing, Is a directory
140312761206016:error:0E06D06A:configuration file routines:NCONF_get_string:no conf or environment variable:../crypto/conf/conf_lib.c:272:

[aldrinc]

@sitnik @bahuma20 Any update on this front? I've been playing with the playground for my POC work but would actually like to go ahead and deploy this in my own environment. I know you guys are busy so if you could just let us know if you might not get to it for 1/2 weeks I would appreciate it. At least if we know I can probably play around with the docker files locally and hack together a solution but if you guys are working on it theres no point for me to do that.