connecting to db in docker container fails on windows

[barbalex]

connecting to db in docker container fails on windows

[kevin-dp]

Hi @barbalex, npm run backend:start should start containers for postgres and electric that are named <app name>-postgres-1 and <app name>-electric-1. When running npm run db:migrate the script checks that those containers are running but it appears they are not. You said the containers seem to be named postgres-1 and electric-1 which is odd as they are missing the app name prefix.

A few things to check:

• Can you check the container names with docker container ls to make sure the names are postgres-1 and electric-1 (i.e. are missing the app name prefix)
• If the container names are missing the app name prefix, can you check the contents of the backend/compose/.envrc file. It is in this file that an environment variable is defined that contains the app name, e.g.:

export POSTGRESQL_IMAGE=postgres:14-alpine
export APP_NAME=bug_repro
export ELECTRIC_PORT=5134
export ELECTRIC_PROXY_PORT=65431
export ELECTRIC_IMAGE=electricsql/electric@sha256:4f21be46b836c6f3a59649cf6c8d03479c5df45ef28012c51d61706cfd8d2892

Notice the APP_NAME environment variable, this should be picked up by our compose file (cf. https://github.com/electric-sql/electric/blob/main/examples/starter/template/backend/compose/docker-compose.yaml#L2):

name: "${APP_NAME:-electric}"

The way how these environment variables are picked up is because when running npm run backend:start we invoke our dockerCompose helper function (https://github.com/electric-sql/electric/blob/main/examples/starter/template/util/util.js#L34) which passes the envrc file using the --env-file option.

Since you mentioned that you are on Windows, perhaps there is a problem with the paths that we define to the compose file and to the envrc file:

const envrcFile = path.join(__dirname, '../backend/compose/.envrc')
const composeFile = path.join(__dirname, '../backend/compose/docker-compose.yaml')

You may want to console.log them in your app and manually try to run the compose file as how our dockerCompose function would do, to see if there's a problem there.

[barbalex]

@kevin-dp

Regarding container names: The problem was my container noobness:

PS C:\Users\alexa\ps> docker container ls
CONTAINER ID   IMAGE                         COMMAND                  CREATED        STATUS                    PORTS                                              NAMES
8f6966d545f5   electricsql/electric:latest   "/app/bin/entrypoint…"   19 hours ago   Up 30 seconds             0.0.0.0:5133->5133/tcp, 0.0.0.0:65432->65432/tcp   ps-electric-1
dd15cedf6d46   postgis/postgis:16-3.4        "docker-entrypoint.s…"   19 hours ago   Up 30 seconds (healthy)   0.0.0.0:53411->5432/tcp                            ps-postgres-1

I didn't understand how they are named and saw this:

So they are named correctly. Sorry for the noise.
More in next comment.

[barbalex]

To find out more, I added a console.log in db/util.js/fetchHostPort:

const envrcFile = path.join(__dirname, '../backend/compose/.envrc')
  const composeFile = path.join(
    __dirname,
    '../backend/compose/docker-compose.yaml',
  )
  console.log('util.js, fetchHostPort', {
    container,
    containerPort,
    service,
    envrcFile,
    composeFile,
    output: `docker inspect --format='{{(index (index .NetworkSettings.Ports "${containerPort}/tcp") 0).HostPort}}' ${container}`,
  })

the output part was copied from the shell.exec command of fetchHostPort itself.

This returns:

util.js, fetchHostPort {
  container: 'ps-electric-1',
  containerPort: 65432,
  service: 'Electric',
  envrcFile: 'C:\\Users\\alexa\\ps\\backend\\compose\\.envrc',
  composeFile: 'C:\\Users\\alexa\\ps\\backend\\compose\\docker-compose.yaml',
  output: `docker inspect --format='{{(index (index .NetworkSettings.Ports "65432/tcp") 0).HostPort}}' ps-electric-1`
}

When I manually run a text.js file with this content:

const shell = require('shelljs')
shell.exec(
  `docker inspect --format='{{(index (index .NetworkSettings.Ports "65432/tcp") 0).HostPort}}' ps-electric-1`,
)

I get this error:

PS C:\Users\alexa\ps> node test.js      
template parsing error: template: :1: unclosed action

[barbalex]

Oh, seems to be a windows quoting thing: https://stackoverflow.com/a/38886047/712005

[barbalex]

I have tried these versions, all failed, with different errors:

shell.exec(
  `docker inspect --format="{{(index (index .NetworkSettings.Ports \"65432/tcp\") 0).HostPort}}" ps-electric-1`,
) // template parsing error: template: :1: unexpected "/" in operand
shell.exec(
  `docker inspect --format="{{(index (index .NetworkSettings.Ports ""65432/tcp"") 0).HostPort}}" ps-electric-1`,
) // template parsing error: template: :1: unterminated quoted string
shell.exec(
  `docker inspect --format="{{(index (index .NetworkSettings.Ports "65432/tcp") 0).HostPort}}" ps-electric-1`,
) // template parsing error: template: :1: unexpected "/" in operand
shell.exec(
  `docker inspect --format="{{(index (index .NetworkSettings.Ports /"65432/tcp/") 0).HostPort}}" ps-electric-1`,
) // template parsing error: template: :1: unexpected "/" in operand
shell.exec(
  `docker inspect --format="{{(index (index .NetworkSettings.Ports '65432/tcp') 0).HostPort}}" ps-electric-1`,
)
// template parsing error: template: :1: unexpected "/" in operand
// template parsing error: template: :1: malformed character constant: '65432/tcp'

Seems fun to execute commands in the shell (would that be powershell?) on windows...

[barbalex]

I found the documentation on the docker inspect --format command: https://docs.docker.com/config/formatting/

So pasting this command directly into powershell works:

docker inspect --format='{{(index (index .NetworkSettings.Ports \"65432/tcp\") 0).HostPort}}' ps-electric-1

(returns: 65432)

But running this file does not:

const shell = require('shelljs')
shell.exec(
  `docker inspect --format='{{(index (index .NetworkSettings.Ports \"65432/tcp\") 0).HostPort}}' ps-electric-1`,
)

error:

PS C:\Users\alexa\ps> node test.js
template parsing error: template: :1: unclosed action

[barbalex]

I wonder if the docker command could be written without using --format. That might be a way to prevent the error? Unfortunately that seems not to be documented: https://docs.docker.com/engine/reference/commandline/inspect/#find-a-specific-port-mapping.

I know nothing about go templates unfortunately.

[kevin-dp]

This is the command we execute (cf. https://github.com/electric-sql/electric/blob/main/examples/starter/template/db/util.js#L51):

`docker inspect --format="{{(index (index .NetworkSettings.Ports \\"${containerPort}/tcp\\") 0).HostPort}}" ${container}`

Note the double backslashes before the double quotes \\"${containerPort}/tcp\\"
It's a double backslash because we need to escape the " so we want to write \" but \ is a special escape character in JS string so we need to escape the backslash \\"

Does that work?

[barbalex]

Calling a test.js file containing this works:

const shell = require('shelljs')
shell.exec(
  `docker inspect --format="{{(index (index .NetworkSettings.Ports \\"65432/tcp\\") 0).HostPort}}" ps-electric-1`,
)

(Pasting it directly into powershell gives: template parsing error: template: :1: unexpected "\\" in operand)

[barbalex]

So it seems that line has recently changed. In the new version of the app I built using create-electric-app today, that line is correct. Now it works on windows for me.

Well, actually, that means: I am now getting this error on windows too: #750

[kevin-dp]

So it seems that line has recently changed. In the new version of the app I built using create-electric-app today, that line is correct. Now it works on windows for me.

Well, actually, that means: I am now getting this error on windows too: #750

Great! Yes, we knew we had some issues with Windows but @samwillis fixed them recently. Glad that those fixes address this issue.