Need to change default port on container execution

[microsoft]

Need to change default port on container execution

[twright-msft]

We plan on addressing this in the CTP 2.0 release in mid-April. The approach will be to allow specifying the port that SQL Server listens on as an environment variable at docker run time. Does that in combination with -p/ports: satisfy the requirement for the presto scenario?

[schmunk42]

Can't you just do a port mapping?

[maciejgrzybek]

@schmunk42 if you mean docker's port mapping, the answer is no. E.g. if you use Docker Toolbox on Mac (https://docs.docker.com/toolbox/toolbox_install_mac/) then you may have some other service running on the SQL Server's default port inside the VM. Simple mapping won't work then.
Also on Docker for Mac we've seen similar issues with conflicting ports, please have a look (on the example of socks proxy listening on 1080): prestodb/presto#6182 (comment)
After this issue discovered we decided to proceed with non-standard ports for all services we use in product tests, to avoid clashes with dev environment.

[schmunk42]

You mean something like this?

Do you need that for local development only or for communication between containers?

Sidenote: I am working on a PR for a framework, where I changed hostnames in testing to use inter-container communication without any port mappings.

We don't use port mappings (or network mode host) in the CI at all, since it completely breaks the ability of running multiple stacks in parallel (on the same host). For local-development we include port mappings in another file, but the developer is responsible to take care about the mappings.
Bonus: Makefile open targets

TL;dr

The container (or image) can only set EXPOSE in a Dockerfile, while ports: can only be set in docker-compose.yml, so this isn't actually a problem of this repo. Because ... changing the exposed port won't really help - or?!

[maciejgrzybek]

@schmunk42 Okay, maybe the mentioned by me case was a misconfiguration/problem with early version of Docker for Mac but there's one use case which is viable for us and not achievable by port mapping in docker itself.
Presto is a SQL query engine so as a part of its product tests we spin multiple containers for different databases and we query them.
Product tests are fully dockerized so both end-points (databases) and Presto itself run in docker, therefore internal container's port is used for communication between them. However, sometimes we use product tests infrastructure but with Presto running in IDE (outside Docker) instead of in docker. In such a case a mapped port had to be used (we don't want to use default ports for outside docker communication because some of the databases may be running on developers environment for other than developing purposes as well, so we want to avoid clashes).
It's important to keep product tests environment and debugging environment as close to each other as possible, that's why we reuse the same config files (configuration of end points, e.g.: https://github.com/prestodb/presto/blob/master/presto-product-tests/conf/presto/etc/catalog/mysql.properties) for product tests and debugging sessions in IDE.
With docker port mapping and keeping default port of database not changed, we'd need to maintain two configurations: for product tests auto-run (databases and Presto all inside docker) - with default ports (3306, 5432, 1433 etc.) and separate for outside docker, debugging sessions - with ports e.g. 13306, 15432, 11433.

That's why we change default port of Postgres and MySQL and map it 1-to-1, so it's consistent for both - automatic run and debugging. Please have a look on the example here: https://github.com/prestodb/presto/blob/master/presto-product-tests/conf/docker/common/jdbc_db.yml

And regardless of our use case, it's just reasonable to have such a trivial thing as changing port to listen on configurable.

[schmunk42]

I just noticed this from the docs

The Docker network feature supports creating networks without the need to expose ports within the network, for detailed information see the overview of this feature).

So, I didn't know that :) Looks to me like we (almost) do not longer need EXPOSE.

---

Do you also "fake" the hostname mysql somewhere during debug/dev, when using this config.

Because having

    ports:
      - '13306:3306'

would work on the host for debugging with 13306 (but not via mysql?) and inside the stack with the default ports.

[maciejgrzybek]

@schmunk42: yes, it's expected that mysql, postgres etc. will be mapped properly in /etc/hosts/, please have a look: https://github.com/prestodb/presto/tree/master/presto-product-tests#debugging-java-based-tests

[maciejgrzybek]

@twright-msft: yes, this will work. Thank you!

[agates4]

@twright-msft was this added? I need to specify the port as well