How/where do I add docker build args?

[accetto]

How/where do I add docker build args?

[Andres9713]

I've continued working on this and read through the very thorough wiki entries on building the images with 'builder.sh' (this Discussion topic got me thinking that this might be how to pass these args). However, I was able to get 'builder.sh' 'running' in that it starts and produces some output, but it then returns the following error (I'm only including the last few lines of the output):

bash command: ./builder.sh latest-firefox all-no-push
.
.
.

[1/14] STEP 4/4: RUN     --mount=type=bind,target=/tmp/cache2     mkdir -p /tmp/g3-cache     && if [ -d /tmp/cache2/.g3-cache/ ] ; then cp -r /tmp/cache2/.g3-cache/* /tmp/g3-cache/ ; fi
[2/14] STEP 1/2: FROM ubuntu:22.04 AS stage_essentials
Error: error building at STEP "RUN --mount=type=bind,target=/tmp/cache2 mkdir -p /tmp/g3-cache     && if [ -d /tmp/cache2/.g3-cache/ ] ; then cp -r /tmp/cache2/.g3-cache/* /tmp/g3-cache/ ; fi": error resolving mountpoints for container "bb0cd30fffc8e016e433181f98117f1814e7ddf7609d536a6a123314991a84ac": invalid mount type "bind"
+ set +x

==> FINISHED  @2023-04-16_02-13-04: ./docker/hooks/build

==> Built 'latest-firefox'

==> FINISHED  @2023-04-16_02-13-04: ./builder.sh

Running as sudo results in the same error. This does result in an image being generated but this image is not usable in two ways: 1) this image does not have 'Repository' or 'Tag' info (both of these are listed as 'none') it wasn't clear to me where I can define/pass these tags either in 'examples-secrets.rc' or in builder.sh or in what I got from the wiki. 2) More importantly, when I run the resulting (untagged) image using its ID in detached (-d) mode it immediately dies (does not show up in 'podman ps') - running it in interactive mode (-it) gets me bash, but I still can not connect to the vnc/noVNC servers...

Not sure how to proceed. Suggestions are welcome. Thank you!

[accetto]

Hello @Andres9713,

thank you for your feedback. I understand your confusion, especially if you're a beginner, as you say. The projects have already grown pretty complex, so I'll try to sort it out a little bit for you.

The accetto projects on the GitHub are aimed to image developers. If you only want to use or to extend my Docker images, you don't need to use the GitHub projects at all. It's enough to use the images I'm publishing to the accetto repositories on the Docker Hub the usual way.

The feature you're looking for - changing the user ID and the group - is actually already included. Check the Overriding container user parameters section of the full README file.

Section 1

If you would like to build your own customized images using the provided script builder.sh, then you should provide the additional build arguments like this:

./builder.sh latest-firefox build \
--build-arg ARG_HEADLESS_USER_NAME=hairless \
--build-arg ARG_HEADLESS_USER_GROUP_NAME=hairygroup \
--build-arg ARG_HEADLESS_USER_ID=2002 \
--build-arg ARG_HEADLESS_USER_GROUP_ID=3003 \
--build-arg ARG_SUDO_INITIAL_PW=docker

Be sure to execute the command from the project's root folder (where the builder script is).

Also be sure to set at least the folowing environment variables before you start building:

• REPO_OWNER_NAME
• BUILDER_REPO

Note also that I've used the build command instead of all-no-push in the example above. That way you can spare building the helper image first, if you don't need the up-to-date version sticker.

Check the files Local building example, Utility builder.sh and Wiki for more info.

Section 2

The building errors you've encountered are related to maintaining the builder cache. It sometimes happens, especially if you try many subsequent builds of different images. It all depends on the concrete images, building sequence and also on the environment.

One or more of following things usually help.

1. Use the additinal --no-cache by building.

./builder.sh latest-firefox build --no-cache

2. Purge the builder cache before or between builds

docker builder prune

3. Re-open the terminal. This is sometimes required if you're building in Visual Studio Code on Windows/WSL. Sometimes I even need to re-open the VSCode itself.

4. If you're extending the images, you sometimes need to remove the lines containing rm -rf /var/lib/apt/lists/* from your Dockerfile - if you have blocks similar to this:

RUN \
    apt-get update \
    && apt-get install -y --no-install-recommends \
        vim \
    && apt-get -y autoremove \
    && rm -rf /var/lib/apt/lists/*

Section 3

PS: I don't think it matters too much, but I am using podman instead of docker.

I can imagine, that this also can impact your building experience.

Unfortunatlly I didn't test it with podman yet. I would recommend to test it with docker first.

Regards,
accetto

[Andres9713]

First of all, thank you @accetto for your very fast response.

About using the accetto repositories: these images come 'pre-built' so they preclude the possibility of changing the username/password using the 'docker build' instructions? Or is there a way to use 'docker build' with the repo images? (Again, sorry these questions are so basic.)

If the answer to the above is 'yes' (i.e. one cannot rebuild the images from the repo), then is the only other alternative using '.builder.sh'? If yes, then I would suggest that the current documentation ('Overriding user parameters in build time' in the full readme) is somewhat out of date/misleading. Otherwise, how is it possible to actually use the 'docker build' command as shown there?

I'm thinking that perhaps the easiest solution to my original problem would be to change the username and password by 'extending' your pulled image? I tried this briefly and it didn't work (the server did not startup correctly and I couldn't connect) but I'm thinking this may be fastest way to go after all. Do you know any reason that this shouldn't work or anything that I need to pay special attention to? I see you mentioned removing 'rm -rf /var/lib/apt/lists/*' commands from the Dockerfile. The Dockerfile you are referencing here is my 'costume-made' Dockerfile that uses your image as a base, correct? In other words, when I am copying and pasting 'extensions' to your base image I should make sure to not include these lines, correct?

Regarding using './builder.sh' I don't believe 'docker builder purge' is a command, am I wrong? At any rate, what I was able to find online suggests that 'docker prune -a' should also remove all the cached images. But this did not fix my problem (it gave the same error as above). So far, I've been trying this on WSL2 using podman. However, I also just tried this on a machine natively running Ubuntu22.04 and docker (not podman). I cloned your repo, made the required changes to the permissions, and generated and sourced a 'secrets.rc' file, then ran the following which resulted in a different error:

console command: ./builder.sh latest-firefox build --no-cache

.
<skipping over several pages of output>
.
[stage_firefox 1/2] RUN     --mount=type=cache,from=stage_cache,sharing=locked,source=/var/cache/apt,target=/var/cache/apt     --mount=type=cache,from=stage_cache,sharing=locked,source=/var/lib/apt,target=/var/lib/apt     KEY_SERVER_URL="keyserver.ubuntu.com"     FIREFOX_PPA_FINGERPRINT="0x0AB215679C571D1C8325275B9BDB3D89CE49EC21"     FIREFOX_PPA_GPG_KEY="mozillateam-ubuntu-ppa.gpg"     && gpg --status-fd 1 --keyserver "${KEY_SERVER_URL}" --recv-keys "${FIREFOX_PPA_FINGERPRINT}"     && gpg --export "${FIREFOX_PPA_FINGERPRINT}" > "${FIREFOX_PPA_GPG_KEY}"     && sudo install -o root -g root -m 644 "${FIREFOX_PPA_GPG_KEY}" /etc/apt/trusted.gpg.d/     && gpg --batch --yes --delete-keys "${FIREFOX_PPA_FINGERPRINT}"     && rm "${FIREFOX_PPA_GPG_KEY}"     && echo "deb https://ppa.launchpadcontent.net/mozillateam/ppa/ubuntu jammy main\n# deb-src https://ppa.launchpadcontent.net/mozillateam/ppa/ubuntu jammy main\n"     >> /etc/apt/sources.list     && echo "Package: firefox*\nPin: release o=LP-PPA-mozillateam\nPin-Priority: 501"     > /etc/apt/preferences.d/mozilla-firefox     && apt-get update     && DEBIAN_FRONTEND=noninteractive apt-get install -y "--no-install-recommends"         firefox:
#19 0.135 gpg: directory '/home/headless/.gnupg' created
#19 0.135 gpg: keybox '/home/headless/.gnupg/pubring.kbx' created
#19 60.52 gpg: keyserver receive failed: Connection timed out
#19 60.52 [GNUPG:] FAILURE recv-keys 167805060
------
ERROR: failed to solve: executor failed running [/bin/sh -c KEY_SERVER_URL="keyserver.ubuntu.com"     FIREFOX_PPA_FINGERPRINT="0x0AB215679C571D1C8325275B9BDB3D89CE49EC21"     FIREFOX_PPA_GPG_KEY="mozillateam-ubuntu-ppa.gpg"     && gpg --status-fd 1 --keyserver "${KEY_SERVER_URL}" --recv-keys "${FIREFOX_PPA_FINGERPRINT}"     && gpg --export "${FIREFOX_PPA_FINGERPRINT}" > "${FIREFOX_PPA_GPG_KEY}"     && sudo install -o root -g root -m 644 "${FIREFOX_PPA_GPG_KEY}" /etc/apt/trusted.gpg.d/     && gpg --batch --yes --delete-keys "${FIREFOX_PPA_FINGERPRINT}"     && rm "${FIREFOX_PPA_GPG_KEY}"     && echo "deb https://ppa.launchpadcontent.net/mozillateam/ppa/ubuntu jammy main\n# deb-src https://ppa.launchpadcontent.net/mozillateam/ppa/ubuntu jammy main\n"     >> /etc/apt/sources.list     && echo "Package: firefox*\nPin: release o=LP-PPA-mozillateam\nPin-Priority: 501"     > /etc/apt/preferences.d/mozilla-firefox     && apt-get update     && DEBIAN_FRONTEND=noninteractive apt-get install -y "${ARG_APT_NO_RECOMMENDS:+--no-install-recommends}"         firefox]: exit code: 2
+ set +x

==> FINISHED  @2023-04-16_18-24-24: ./docker/hooks/build --no-cache


==> FINISHED  @2023-04-16_18-24-24: ./builder.sh --no-cache

I'm including this here for your development work (so you can better understand points of failure, not necessarily for your help troubleshooting this particular problem) - I'm just interested in getting a version of this image running with a different username and password.... well, that's all I'm trying to do for this step - the next step is having this image also running nvidia drivers so it can have access to the gpu (which I can pass as a device through podman or docker). So ideally, I would like to start from the CUDA Ubuntu22 image rather than the base one, but I know that's moving the goal post.

Thank you sincerely for your feedback!

[accetto]

About using the accetto repositories: these images come 'pre-built' so they preclude the possibility of changing the username/password using the 'docker build' instructions? Or is there a way to use 'docker build' with the repo images?

This is not correct for two reasons.

Firstly, my images already provide the possibility to change the user ID and also the group ID by the container start.

For example, if you start the container like this

docker run -d -p "36901:6901" --user "1002:2002" accetto/ubuntu-vnc-xfce-g3:latest

and then you connect to it by http://localhost:36901, then you can check in the terminal that the user ID and the group ID are indeed changed:

headless@8ebc57230e90:~$ id
uid=1002(headless) gid=2002(headless) groups=2002(headless)

Note that the default VNC password is headless.

Secondly, you can always use my images as a base and extend them any way you wish. Just start you Dockerfile like this:

FROM ccetto/ubuntu-vnc-xfce-g3:latest

You can use the files in the examples folder as the starting point.

[accetto]

If you want to build the images with different and permanent user ID and group IDs, you can indeed provide the required build to the arguments to the builder script:

./builder.sh latest build --build-arg ARG_HEADLESS_USER_ID=2002  --build-arg ARG_HEADLESS_USER_GROUP_ID=3003

You can also modify the Dockerfile and build the images the usual way or some other of several possible ways.

If you only want to use the images, you can use them directly from the Docker Hub. After pulling them you can extend them if you need (see the answer above).

Note that the related GitHub projects do not container Docker images, but the generators of images, more exactly generators of groups of images. You don't need to use them at all, if you don't plan to customize them. The generators are pretty complex and they use a lot of variables, as you can see from the Docker files, hook scripts and also, for example, Docker stage diagram. You need some knowledge in shell scripting and how Docker works to customize the generators.

[accetto]

I'm sorry, but I cannot help with technical problems, if you don't describe in details your exact use case and your environment.

Generally speaking, I use my images and the generators on different computers in different environments for several years and they work just fine.

I've got a feeling that you are trying to experiment with the graphics related stuff. Maybe the image accetto/ubuntu-vnc-xfce-opengl-g3 from the sibling project accetto/headless-drawing-g3 will be better suited for it.

[accetto]

Regarding using './builder.sh' I don't believe 'docker builder purge' is a command, am I wrong?

You're right, thanks for the hint. It should be prune, not purge. I've already corrected it.

[accetto]

Closing because of no other communication from @Andres9713.

[Andres9713]

Sorry - for the lack of reply @accetto - what happened is that I actually got all this working, (not in the way you described, in a worse way) - that is I got the container fully working with a new user name and with full access to the NVIDIA card, and then through testing this container I learned that containerization itself was problematic in terms of performance for my given application. Therefore I switched gears.

I can report a few things: 1) podman definitely does cause conflicts when trying to build the images, however, I could build the images in Docker, export them and then open them in podman easily, 2) the last error I reported was specifically with your Firefox build and wasn't due to any obvious mistake I could discern on my part (by that I mean that the identical set of build commands went through no problem on the 'base', non-firefox, version of container), and 3) I still think that at least for beginners the instructions on re-building your images (as for example, to create images with particular usernames) are not so very clear, some more less abstract examples would help - but that is not to say that I know it would be worth your effort to make that documentation beginner friendly, I don't know that, and I do know that you've already put a lot of time and effort into it, and its in very good shape as is.