Trouble spinning up docker instances to test

[friendica]

Trouble spinning up docker instances to test

[focalfury]

Sorry for my long absence. Big life event just happened a few weeks earlier than expected, which is the point of getting a private social media setup :).

So I took a few steps back and focused on some basics and now have a better grasp of Docker, certs, etc.

Currently I have setup a letsencrypt docker image that is acting as SSL termination and reverse proxy for a few items. If I'm using this Letsencrypt/nginx container separate from Friendica, it sounds like I want to be looking at building my docker-compose based on the insecure fpm image. https://github.com/friendica/docker/blob/master/.examples/docker-compose/insecure/mariadb-cron-redis/fpm-alpine/docker-compose.yml

If I understand this correctly letsencrypt/nginx is going to act as the secure method, webserver/proxy to friendica DB & App in a single container, providing I keep the app part on the frontend network that connects to LE/nginx.

Does this sound about the right path for doing this? Are there any gotchyas or good examples with nginx running separate from friendica that you could point me to? I found one but it seemed a bit old and looked like there might be issues with URL rewrites which I admit I don't know much about quite yet.

Thanks again for your insight and help.

[focalfury]

OK so I spent some time tonight and feel confident I have a good docker-compose file for getting the FPM version going with services: db,redis,app, and cron. They are running with what looks like no errors :)

What I really am in over my head with is the proper nginx settings in another container that serves as my reverse proxy/webserver & letsencrypt wildcard.

I've learned a bit more about PHP-FPM and understand that we need to get php requests from nginx over but I just don't quite grasp where I'm supposed to make these settings changes inside nginx. Proxy confs, main nginx.conf, something else?

I realize this is quite out of scope for Friendica but if there is any type of resources you can help point me to or any experience it would be greatly appreciated. I really am trying to learn all about this as I go and not want to just be given answers without understanding.

Thanks for your help.

[nupplaphil]

Which docker-compose example did you use?
At least this one should work: https://github.com/friendica/docker/tree/master/.examples/docker-compose/insecure/mariadb-cron-redis/apache
(don't forget to adapt the environment variables)

For the two containers:

• Did you create the friendicaDB with this sql-file: https://github.com/friendica/friendica/blob/master/database.sql ? (or similar if you use the develop/rc branch)
• Is there a working local.config.php in your /var/www/html/config/ directory inside your friendica container (otherwise the server doesn't know where to connect to)?
• If not, you can create a plain one, based on this template: https://github.com/friendica/friendica/blob/master/config/local-sample.config.php

Can you paste the output of docker logs $(docker ps | grep 'friendica/server' | awk '{ print $1 }' | head -1) ? It should output the log of the friendica/server container you started.

[nupplaphil]

A and please add a custom network for the two container-examples instead of the (legacy) --link feature (https://docs.docker.com/network/links/).

Like:

$> docker network create friendica_network
$> docker create --name db --network friendica_network -e MYSQL_ROOT_PASSWORD=test -e MYSQL_PASSWORD=test -e MYSQL_USERNAME=test -e MYSQL_DATABASE=test mysql:5.7
$> docker create --name friendica  --network friendica_network --publish 8080:80 -e MYSQL_ROOT_PASSWORD=test -e MYSQL_PASSWORD=test -e MYSQL_USERNAME=test -e MYSQL_DATABASE=test -e MYSQL_HOST=db -e "[email protected]" -e "FRIENDICA_URL=https://test.it" friendica/server

This should auotmatically install friendica

[focalfury]

Thanks for the information and help. Disclaimer I think a lot of this is my relative newness to docker.
Also for reference in testing I'm running Docker on Windows for the base work before I decide which Linux hosts I'll have the real apps running from, just in case it makes a difference.

My first time I tried docker-compose I used the Base version - apache one here: https://hub.docker.com/r/friendica/server
My understanding is that all I needed to do is create the docker-compose.yaml with that in the body, save and run docker-compose up -d and I should be able to go to localhost:8080 to view. When I do this it tries to move me to friendica/install in browser and it doesn't connect.
Logs for these two containers show the following
APP1
Initializing Friendica 2019.09 ... Initializing finished New Friendica instance Running web-based installer on first connect! [Sat Dec 28 23:28:57.754008 2019] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.25 (Debian) PHP/7.3.12 configured -- resuming normal operations [Sat Dec 28 23:28:57.754055 2019] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND' 172.21.0.1 - - [28/Dec/2019:23:29:02 +0000] "GET / HTTP/1.1" 302 413 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"

DB1 (last few lines)
2019-12-28 23:29:39 0 [Note] InnoDB: Completed initialization of buffer pool 2019-12-28 23:29:39 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2019-12-28 23:29:39 0 [Note] InnoDB: 128 out of 128 rollback segments are active. 2019-12-28 23:29:39 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2019-12-28 23:29:39 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2019-12-28 23:29:39 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2019-12-28 23:29:39 0 [Note] InnoDB: 10.4.11 started; log sequence number 19884017; transaction id 14628 2019-12-28 23:29:39 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2019-12-28 23:29:39 0 [Note] Plugin 'FEEDBACK' is disabled. 2019-12-28 23:29:39 0 [Note] Server socket created on IP: '::'. 2019-12-28 23:29:39 0 [Warning] 'proxies_priv' entry '@% root@252fde0bcedb' ignored in --skip-name-resolve mode. 2019-12-28 23:29:39 0 [Note] InnoDB: Buffer pool(s) load completed at 191228 23:29:39 2019-12-28 23:29:39 0 [Note] Reading of all Master_info entries succeeded 2019-12-28 23:29:39 0 [Note] Added new Master_info '' to hash table 2019-12-28 23:29:39 0 [Note] mysqld: ready for connections. Version: '10.4.11-MariaDB-1:10.4.11+maria~bionic' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution

=================
With your method you sent me in your response for docker-compose I created a new folder and made sure the file structure was identical as it is shown here: https://github.com/friendica/docker/tree/master/.examples/docker-compose/insecure/mariadb-cron-redis/apache
and then ran docker-compose up -d but it didn't navigate at all to the site

===============

When I tried again the 2 container method and this time ran the sql file to get the DB ready, verified the local config was present and set as expected, I was able to get to the first page but then when I clicked register it is trying to always redirect to https and so without a certificate I couldn't proceed further.

This highlights another area I know I'm not very proficient right now with. My goal was to get things into a very basic state and then start learning how to get this into a stable, secure and more permanent method through trial/error. However I thought docker-compose was going to be the easiest way to going and I'm still having issues with it.

Any advice or other information I can provide you with that could help me get going?

Thanks for taking the time to help, I really appreciate it.

[nupplaphil]

Thanks for the information and help. Disclaimer I think a lot of this is my relative newness to docker.
Also for reference in testing I'm running Docker on Windows for the base work before I decide which Linux hosts I'll have the real apps running from, just in case it makes a difference.

This is fine, don't worry :-) .

My first time I tried docker-compose I used the Base version - apache one here: https://hub.docker.com/r/friendica/server
My understanding is that all I needed to do is create the docker-compose.yaml with that in the body, save and run docker-compose up -d and I should be able to go to localhost:8080 to view. When I do this it tries to move me to friendica/install in browser and it doesn't connect.

The problem is that Friendica needs a running mysql-server and some basic configurations to be useful (yet). That's why even the .examples needs some basic configuration.
Based on https://github.com/friendica/docker/tree/master/.examples/docker-compose/insecure/mariadb-cron-redis/apache , you need at least setting the following configs:

• db.env (the database environment settings):
  

  docker/.examples/docker-compose/insecure/mariadb-cron-redis/apache/db.env

  Line 1 in 1e61667

  MYSQL_PASSWORD=

• docker-compose.yml (general settings for Friendica):
  

  docker/.examples/docker-compose/insecure/mariadb-cron-redis/apache/docker-compose.yml

  Lines 24 to 29 in 1e61667

  	- FRIENDICA_ADMIN_MAIL=
  	- FRIENDICA_TZ=
  	- FRIENDICA_LANG=
  	- FRIENDICA_URL=
  	- FRIENDICA_SITENAME=
  	- SMTP=

Otherwise the automatic installer isn't capable of setting up Friendica the right way

Logs for these two containers show the following
APP1
Initializing Friendica 2019.09 ... Initializing finished New Friendica instance Running web-based installer on first connect! [Sat Dec 28 23:28:57.754008 2019] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.25 (Debian) PHP/7.3.12 configured -- resuming normal operations [Sat Dec 28 23:28:57.754055 2019] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND' 172.21.0.1 - - [28/Dec/2019:23:29:02 +0000] "GET / HTTP/1.1" 302 413 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"

DB1 (last few lines)
2019-12-28 23:29:39 0 [Note] InnoDB: Completed initialization of buffer pool 2019-12-28 23:29:39 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2019-12-28 23:29:39 0 [Note] InnoDB: 128 out of 128 rollback segments are active. 2019-12-28 23:29:39 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2019-12-28 23:29:39 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2019-12-28 23:29:39 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2019-12-28 23:29:39 0 [Note] InnoDB: 10.4.11 started; log sequence number 19884017; transaction id 14628 2019-12-28 23:29:39 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2019-12-28 23:29:39 0 [Note] Plugin 'FEEDBACK' is disabled. 2019-12-28 23:29:39 0 [Note] Server socket created on IP: '::'. 2019-12-28 23:29:39 0 [Warning] 'proxies_priv' entry '@% root@252fde0bcedb' ignored in --skip-name-resolve mode. 2019-12-28 23:29:39 0 [Note] InnoDB: Buffer pool(s) load completed at 191228 23:29:39 2019-12-28 23:29:39 0 [Note] Reading of all Master_info entries succeeded 2019-12-28 23:29:39 0 [Note] Added new Master_info '' to hash table 2019-12-28 23:29:39 0 [Note] mysqld: ready for connections. Version: '10.4.11-MariaDB-1:10.4.11+maria~bionic' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution

The logs are fine. It means that both instances (db and Friendica) are on their basic state, but no installer was running.

=================
With your method you sent me in your response for docker-compose I created a new folder and made sure the file structure was identical as it is shown here: https://github.com/friendica/docker/tree/master/.examples/docker-compose/insecure/mariadb-cron-redis/apache
and then ran docker-compose up -d but it didn't navigate at all to the site

As said above, you need some basic setup to make it work. I take it as a todo for a "test environment" setting, where I fill out the basic setups for test-only environments :-).

===============

When I tried again the 2 container method and this time ran the sql file to get the DB ready, verified the local config was present and set as expected, I was able to get to the first page but then when I clicked register it is trying to always redirect to https and so without a certificate I couldn't proceed further.

This could happen in the following cases:

• FRIENDICA_URL is set to https, like https://friendica.local
• In your config/local.config.php, in the table config is set
  system.ssl_policy to 2 (which means force SSL)
• or during the installing wizzard, you set the "SSL link policy" to "Force all links to use SSL"

Please check the config/local.config.php file or the config table if you find such settings.

This highlights another area I know I'm not very proficient right now with. My goal was to get things into a very basic state and then start learning how to get this into a stable, secure and more permanent method through trial/error. However I thought docker-compose was going to be the easiest way to going and I'm still having issues with it.

Yes, you're right. I need to simplify it a little bit more, so your feedback is very welcome :-)

[nupplaphil]

If you don't want to play around with PHP-FPM, you could replace it with the apache version of the image. This would make the usage of an extra nginx obsolete.

Because beware that you need two different types of proxies for routing FPM images to the world wide web :)

The Asset Proxy

This proxy is responsible for serving the static assets of the friendica image, because PHP-FPM is not able to serve it
Definition of PHP-FPM:

FPM (FastCGI Process Manager) is an alternative PHP FastCGI implementation with some additional features (mostly) useful for heavy-loaded sites.

FastCGI is not possible to serve any assets, it's "just" responsible to generating the right PHP response for a request as fast as possible. So you have to mount the same friendica-html-root-directory as you use for the php-fpm container inside an extra nginx-image and the nginx proxy is now responsible for:

• Calling the PHP-FPM image with the parameters he gets (fast PHP excecution) and respond it to the caller
• Responding assets (images, css, js, ...) directly without calling the PHP-FPM image

A sample config for this type of nginx proxy is available here:
https://github.com/friendica/docker/blob/master/.examples/docker-compose/insecure/mariadb-cron-redis/fpm-alpine/web/nginx.conf

This is the place where the config defines which PHP-FPM container the nginx container has to call (in this example, it's the host app in its local docker network with the port 9000):

docker/.examples/docker-compose/insecure/mariadb-cron-redis/fpm-alpine/web/nginx.conf

Lines 33 to 35 in 41ac697

	upstream php-handler {
	server app:9000;
	}

The Reverse Proxy

This is a extra proxy for routing a web address to the right destination - including SSL/TLS (Let's encrypt).
You can use nginx for this type of proxy too (there are configs for docker environments to automatically route web addresses to another docker image, I must confess, I've to look for it too).

I personally use a more "docker-way" for loadbalancing/routing requests:
https://docs.traefik.io/
As the front-image already suggests, it's possible to route every kind of request from outside of the docker-environment to a specific, running docker image (including scaling, loadbalancing, ...). It has native Let's encrypt support.
And it's fully flexible based on docker-image labels (so no config-changes of nginx necessary because of changed routes, new addresses, ...)

So for a first start, you could skip the first type of proxy and use the apache-image instead. This would make it possible to setup your environment step by step

1. Setup Apache friendica image
2. Setup Reverse Proxy (nginx / traefik / ...)
3. Switch from Apache to PHP-FPM image and setup the Asset proxy

btw. Maybe it's be possible to mix both types of proxies in one nginx instance, but I don't have a lot of experiences here to help you. So I'd say you should ask these kind of questions better at stack overflow with a lot more experts :-)

[nupplaphil]

@focalfury I close this issue for now. If you need any guidance, feel free to reopen it or open another issue with questions