[Help]: Configurations in Docker and kubernetes

[amardeep2006]

[Help]: Configurations in Docker and kubernetes

[morrisonlevi]

Environment variables for PHP are more difficult, because every SAPI tends to handle them differently. INI settings are generally much more consistent across SAPIs, so I recommend setting them that way. The datadog-setup.php script has a config mode that can be used to set them as part of your script, e.g.:

FROM myrepo/ubuntu_drupal_apache2:latest
RUN curl -LO https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php \
	&& php datadog-setup.php --php-bin=all \
	&& php datadog-setup.php config set \
		-d datadog.service=app-name \
		-d datadog.env=prod \
		-d datadog.version=1.3.2 \
		-d datadog.trace.agent_url=http://localhost:8216

[amardeep2006]

Thanks @morrisonlevi for quick response .
If I pass the configs with -d at build time then which file/location can I see these after configuration ?
I still see 2 issues :

1. Since my docker image is reusable it will be confusing if all apps report traces with same service , env and version.
2. The agent in my case is running on kubernetes host node as daemonset. I can inject the IP of kubernetes node via helm while deploying but I need to set agent_host that can point to OS environment variable having IP address of kubernetes node.

[amardeep2006]

I tried following configs :

Config 1 :

FROM myrepo/ubuntu_drupal_apache2:latest
RUN apt-get update -yq \
	&& apt-get upgrade -yq
RUN curl -LO https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php \
	&& php datadog-setup.php --php-bin=all
RUN echo "env[DD_ENV] = \$DD_ENV" >>/etc/php/8.1/fpm/pool.d/www.conf \
	&& echo "env[DD_SERVICE] = \$DD_SERVICE" >>/etc/php/8.1/fpm/pool.d/www.conf \
	&& echo "env[DD_VERSION] = \$DD_VERSION" >>/etc/php/8.1/fpm/pool.d/www.conf \
	&& echo "env[DD_AGENT_HOST] = \$KUBERNETES_HOST_IP" >>/etc/php/8.1/fpm/pool.d/www.conf 

Config 2 with clear_env = no:

FROM myrepo/ubuntu_drupal_apache2:latest
RUN apt-get update -yq \
	&& apt-get upgrade -yq
RUN curl -LO https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php \
	&& php datadog-setup.php --php-bin=all
RUN echo "env[DD_ENV] = \$DD_ENV" >>/etc/php/8.1/fpm/pool.d/www.conf \
	&& echo "env[DD_SERVICE] = \$DD_SERVICE" >>/etc/php/8.1/fpm/pool.d/www.conf \
	&& echo "env[DD_VERSION] = \$DD_VERSION" >>/etc/php/8.1/fpm/pool.d/www.conf \
	&& echo "env[DD_AGENT_HOST] = \$KUBERNETES_HOST_IP" >>/etc/php/8.1/fpm/pool.d/www.conf \	
	&& echo "clear_env = no" >>/etc/php/8.1/fpm/pool.d/www.conf 

I am passing KUBERNETES_HOST_IP via helm chart.

  - name: KUBERNETES_HOST_IP
    valueFrom:
      fieldRef:
        fieldPath: status.hostIP

ISSUE I faced: It does not set datadog.agent_host and it still points to localhost. Interestingly datadog.env , datadog.service and datadog.version are set.

I then Injected DD_AGENT_HOST via helm chart like this and now datadog.agent_host has the correct IP even without making entry in www.conf file.

  - name: DD_AGENT_HOST 
    valueFrom:
      fieldRef:
        fieldPath: status.hostIP

Seems like configurations I have set in /etc/php/8.1/fpm/pool.d/www.conf are not effective and It's picking up OS environment variables directly if I run below commands.

php -i or php -c /etc/php/8.1/apache2/conf.d/98-ddtrace.ini -i commands.

However when I created a phpinfo() via echo "<?php phpinfo();" >/var/www/html/info.php but it does not show the actual values I passed from env variables.

I will try few more configs and keep updated.

[amardeep2006]

I tried one more config but still phpinfo() does not use the values I passed. Kind of hitting end of road now.

FROM myrepo/ubuntu_drupal_apache2:latest
RUN apt-get update -yq \
	&& apt-get upgrade -yq
RUN curl -LO https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php \
	&& php datadog-setup.php --php-bin=all
RUN echo "env[DD_ENV] = \$DD_ENV" >>/etc/php/8.1/fpm/pool.d/www.conf \
	&& echo "env[DD_SERVICE] = \$DD_SERVICE" >>/etc/php/8.1/fpm/pool.d/www.conf \
	&& echo "env[DD_VERSION] = \$DD_VERSION" >>/etc/php/8.1/fpm/pool.d/www.conf \
	&& echo "env[DD_AGENT_HOST] = \$DD_AGENT_HOST" >>/etc/php/8.1/fpm/pool.d/www.conf
RUN echo "PassEnv \$DD_ENV" >>/etc/apache2/apache2.conf \
	&& echo "PassEnv \$DD_SERVICE" >>/etc/apache2/apache2.conf \
	&& echo "PassEnv \$DD_VERSION" >>/etc/apache2/apache2.conf \
	&& echo "PassEnv \$DD_AGENT_HOST" >>/etc/apache2/apache2.conf

[amardeep2006]

Update : I tries below config

FROM myrepo/ubuntu_drupal_apache2:latest
ARG PHP_VERSION=8.1
ARG DDTRACE_LIB_VERSION=0.99.1
RUN apt-get update -yq \
	&& apt-get upgrade -yq
# Install dd-trace-php Library and configure it for all php installations
RUN curl -LO https://github.com/DataDog/dd-trace-php/releases/download/${DDTRACE_LIB_VERSION}/datadog-setup.php \
	&& php datadog-setup.php --php-bin=all \
	&& php datadog-setup.php config set --php-bin=php --php-bin=php-fpm${PHP_VERSION}  \
		-d datadog.service=\${DD_SERVICE} \
		-d datadog.env=\${DD_ENV} \
		-d datadog.version=\${DD_VERSION} \
		-d datadog.agent_host=\${KUBERNETES_HOST_IP}
RUN echo "<?php phpinfo();" >/var/www/html/info.php

The start command in base docker image is
CMD ["/bin/bash", "-c", "service php8.1-fpm start && apachectl '-DFOREGROUND'"]

Here are installation logs

#6 265.1 Installed required source files to '/opt/datadog/dd-library/0.99.1'
#6 265.1 Installing to binary: php (/usr/bin/php8.1)
#6 265.2 Copied '/tmp/dd-install/dd-library-php/trace/ext/20210902/ddtrace.so' to '/usr/lib/php/20210902/ddtrace.so'
#6 265.2 Copied '/tmp/dd-install/dd-library-php/profiling/ext/20210902/datadog-profiling.so' to '/usr/lib/php/20210902/datadog-profiling.so'
#6 265.2 Copied '/tmp/dd-install/dd-library-php/appsec/ext/20210902/ddappsec.so' to '/usr/lib/php/20210902/ddappsec.so'
#6 265.2 Created INI file '/etc/php/8.1/cli/conf.d/98-ddtrace.ini'
#6 265.2 Installation to 'php (/usr/bin/php8.1)' was successful
#6 265.2 Created INI file '/etc/php/8.1/apache2/conf.d/98-ddtrace.ini'
#6 265.2 Installation to 'php (/usr/bin/php8.1)' was successful
#6 265.2 Installing to binary: php8.1 (/usr/bin/php8.1)
#6 265.2 Copied '/tmp/dd-install/dd-library-php/trace/ext/20210902/ddtrace.so' to '/usr/lib/php/20210902/ddtrace.so'
#6 265.2 Copied '/tmp/dd-install/dd-library-php/profiling/ext/20210902/datadog-profiling.so' to '/usr/lib/php/20210902/datadog-profiling.so'
#6 265.2 Copied '/tmp/dd-install/dd-library-php/appsec/ext/20210902/ddappsec.so' to '/usr/lib/php/20210902/ddappsec.so'
#6 265.2 Updating existing INI file '/etc/php/8.1/cli/conf.d/98-ddtrace.ini'
#6 265.2 Installation to 'php8.1 (/usr/bin/php8.1)' was successful
#6 265.2 Updating existing INI file '/etc/php/8.1/apache2/conf.d/98-ddtrace.ini'
#6 265.2 Installation to 'php8.1 (/usr/bin/php8.1)' was successful
#6 265.2 Installing to binary: php-fpm8.1 (/usr/sbin/php-fpm8.1)
#6 265.3 Copied '/tmp/dd-install/dd-library-php/trace/ext/20210902/ddtrace.so' to '/usr/lib/php/20210902/ddtrace.so'
#6 265.3 Copied '/tmp/dd-install/dd-library-php/profiling/ext/20210902/datadog-profiling.so' to '/usr/lib/php/20210902/datadog-profiling.so'
#6 265.3 Copied '/tmp/dd-install/dd-library-php/appsec/ext/20210902/ddappsec.so' to '/usr/lib/php/20210902/ddappsec.so'
#6 265.3 Created INI file '/etc/php/8.1/fpm/conf.d/98-ddtrace.ini'
#6 265.3 Installation to 'php-fpm8.1 (/usr/sbin/php-fpm8.1)' was successful
#6 265.3 --------------------------------------------------
#6 265.3 SUCCESS
#6 265.3
#6 265.5 Setting configuration for binary: php (/usr/bin/php8.1)
#6 265.5 Set 'datadog.service' to '${DD_SERVICE}' in INI file: /etc/php/8.1/cli/conf.d/98-ddtrace.ini.
#6 265.5 Set 'datadog.service' to '${DD_SERVICE}' in INI file: /etc/php/8.1/apache2/conf.d/98-ddtrace.ini.
#6 265.5 Set 'datadog.env' to '${DD_ENV}' in INI file: /etc/php/8.1/cli/conf.d/98-ddtrace.ini.
#6 265.5 Set 'datadog.env' to '${DD_ENV}' in INI file: /etc/php/8.1/apache2/conf.d/98-ddtrace.ini.
#6 265.5 Set 'datadog.version' to '${DD_VERSION}' in INI file: /etc/php/8.1/cli/conf.d/98-ddtrace.ini.
#6 265.5 Set 'datadog.version' to '${DD_VERSION}' in INI file: /etc/php/8.1/apache2/conf.d/98-ddtrace.ini.
#6 265.5 Set 'datadog.agent_host' to '${KUBERNETES_HOST_IP}' in INI file: /etc/php/8.1/cli/conf.d/98-ddtrace.ini.
#6 265.5 Set 'datadog.agent_host' to '${KUBERNETES_HOST_IP}' in INI file: /etc/php/8.1/apache2/conf.d/98-ddtrace.ini.
#6 265.5 Setting configuration for binary: php-fpm8.1 (/usr/sbin/php-fpm8.1)
#6 265.6 Set 'datadog.service' to '${DD_SERVICE}' in INI file: /etc/php/8.1/fpm/conf.d/98-ddtrace.ini.
#6 265.6 Set 'datadog.env' to '${DD_ENV}' in INI file: /etc/php/8.1/fpm/conf.d/98-ddtrace.ini.
#6 265.6 Set 'datadog.version' to '${DD_VERSION}' in INI file: /etc/php/8.1/fpm/conf.d/98-ddtrace.ini.
#6 265.6 Set 'datadog.agent_host' to '${KUBERNETES_HOST_IP}' in INI file: /etc/php/8.1/fpm/conf.d/98-ddtrace.ini.
#6 DONE 265.6s

When I start container in kubernetes and run commands like php -i it picks up the datadog configurations passed as OS variables but if I open then phpinfo page it still does not show the correct configs.

I know I am so close but no idea why it's now working .

[amardeep2006]

@morrisonlevi
Finally I was able to solve it. Here is what worked for me for future reference if anyone faces the same

FROM myrepo/ubuntu_drupal_apache2:latest
ARG PHP_VERSION=8.1
ARG DDTRACE_LIB_VERSION=0.99.1
RUN apt-get update -yq \
	&& apt-get upgrade -yq
RUN curl -LO https://github.com/DataDog/dd-trace-php/releases/download/${DDTRACE_LIB_VERSION}/datadog-setup.php \
	&& php datadog-setup.php --php-bin=all \
	&& php datadog-setup.php config set --php-bin=php --php-bin=php-fpm${PHP_VERSION} \
		-d datadog.service=\${DD_SERVICE} \
		-d datadog.env=\${DD_ENV} \
		-d datadog.version=\${DD_VERSION} \
		-d datadog.agent_host=\${DD_AGENT_HOST} 	
RUN echo "PassEnv DD_ENV" >>/etc/apache2/apache2.conf \
	&& echo "PassEnv DD_SERVICE" >>/etc/apache2/apache2.conf \
	&& echo "PassEnv DD_VERSION" >>/etc/apache2/apache2.conf \
	&& echo "PassEnv DD_AGENT_HOST" >>/etc/apache2/apache2.conf

Here are how I am passing env variable via helm chart. The environment names like DD_AGENT_HOST , DD_ENV etc must be exactly same.

  - name: DD_AGENT_HOST 
    valueFrom:
      fieldRef:
        fieldPath: status.hostIP
  - name: DD_ENV
     value: dev
  - name: DD_SERVICE
     value: drupal-app-name
  - name: DD_VERSION
    value: 1.0.16

I also found one issue with DD official documentation . For Apache + mod_php in VirtualHost or server configuration file the example shows Dollar sign like this which is wrong.

https://app.datadoghq.com/apm/service-setup?architecture=container-based&collection=Helm%20Chart%20%28Recommended%29&environment=kubernetes&language=php

PassEnv $DD_ENV
PassEnv $DD_SERVICE
PassEnv $DD_VERSION

The correct configs for apache2.conf

PassEnv DD_ENV
PassEnv DD_SERVICE
PassEnv DD_VERSION
PassEnv DD_AGENT_HOST 

This documentation should be corrected. I cannot send PR as this seems part of datadog cloud private docs.
There was one more issue in public documentation which I got corrected by sending PR to datadog DataDog/documentation#23186

Today I will try one more configuration . I will share results and simplified configs in case it works with pure Apache PassEnv + Standard DD_ OS variables combo.

[amardeep2006]

Here is the most simple config that also worked.

FROM myrepo/ubuntu_drupal_apache2:latest
RUN apt-get update -yq \
	&& apt-get upgrade -yq
RUN curl -LO https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php \
	&& php datadog-setup.php --php-bin=all	
RUN echo "PassEnv DD_ENV" >>/etc/apache2/apache2.conf \
	&& echo "PassEnv DD_SERVICE" >>/etc/apache2/apache2.conf \
	&& echo "PassEnv DD_VERSION" >>/etc/apache2/apache2.conf \
	&& echo "PassEnv DD_AGENT_HOST" >>/etc/apache2/apache2.conf

[morrisonlevi]

If you want to use env vars, that's fine. I'm glad you figured out how to pipe them through! I'll pass along the documentation issue you mentioned.