sh: write error: Invalid argument - Centos 7

[multiarch]

sh: write error: Invalid argument - Centos 7

[junaruga]

Here is the alternative workflow for your environment possibly.

$ docker run --rm --privileged multiarch/qemu-user-static:register --reset

or

$ docker run --rm --privileged multiarch/qemu-user-static --reset

Above commands are just to add binfmt_misc files to your host environment.

Then for example if you want to run arm64v8/ubuntu Ubuntu ARM 64-bit (= aarch64) container, it is like this with your container "my-aarch64-ubuntu".

$ docker build --rm -t my-aarch64-ubuntu -<<EOF
FROM multiarch/qemu-user-static:x86_64-aarch64 as qemu
FROM arm64v8/ubuntu
COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
EOF

$ docker run --rm -t my-aarch64-ubuntu uname -m
aarch64

[junaruga]

I'd prefer not to have to include the binary in the docker images and it looks like my only option is to update to Centos 8 which is currently kernel 4.18.0 and does in fact work as expected.

There is an alternative way not to include the binary in the docker images for your case.

That is to copy the binary qemu-*-static from the container to the host and use docker run -v host_dir:container_dir.
I referred this document to copy the binary from the container to the host.

$ docker run --rm --privileged multiarch/qemu-user-static:register --reset

$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64
enabled
interpreter /usr/bin/qemu-aarch64-static
flags: 
offset 0
magic 7f454c460201010000000000000000000200b700
mask ffffffffffffff00fffffffffffffffffeffffff

$ docker create -it --name dummy multiarch/qemu-user-static:x86_64-aarch64 bash

$ docker container ls -a
CONTAINER ID        IMAGE                                       COMMAND             CREATED             STATUS              PORTS               NAMES
6ab622a76dfa        multiarch/qemu-user-static:x86_64-aarch64   "bash"              3 minutes ago       Created                                 dummy

$ docker cp dummy:/usr/bin/qemu-aarch64-static qemu-aarch64-static

$ ls qemu-aarch64-static
qemu-aarch64-static*

$ docker rm -f dummy

$ docker container ls -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

$ docker run --rm -t -v $(pwd)/qemu-aarch64-static:/usr/bin/qemu-aarch64-static arm64v8/ubuntu uname -m
aarch64

[junaruga]

One possibility is your kernel 3.10.0-1062.9.1.el7.x86_64 (= uname -r) is too old to use the command or the option -p yes.

Does following command without -p yes work on your environment?

docker run --rm --privileged multiarch/qemu-user-static --reset

Because flags: F used for -p yes is relatively newer feature in binfmt_misc itself. I saw the option worked on Travis CI Ubuntu xenial, but did not work on Travis CI Ubuntu trusty.

Here is the document on kernel 4.10.
https://www.kernel.org/doc/html/v4.10/admin-guide/binfmt-misc.html
flags - F - fix binary is the used function for the -p yes.

[jc21]

Update for anyone who really wants this to work on Centos 7:

1. Install the latest unofficial mainline kernel
2. Select the newly installed kernel to be default in grub
3. Profit

[junaruga]

Here is the binfmt_misc document for kernel 3.10. There is no F option of flags there.

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Documentation/binfmt_misc.txt?h=v3.10#n37

[jc21]

Thanks those steps worked. But the situation sucks.

I'd prefer not to have to include the binary in the docker images and it looks like my only option is to update to Centos 8 which is currently kernel 4.18.0 and does in fact work as expected.

[wanganhong]

@junaruga

Hi,junaruga! I repeat your steps, but got an error at the last step.

Error detail:

$ docker run --rm -t -v $(pwd)/qemu-aarch64-static:/usr/bin/qemu-aarch64-static arm64v8/ubuntu uname -m
: error while loading shared libraries: cannot apply additional memory protection after relocation: Permission denied

My device is a virtual machine，operating system info:

$ cat /etc/centos-release
CentOS Linux release 7.4.1708 (Core)

$ uname -a
Linux localhost.localdomain 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

[zyystudio]

Update for anyone who really wants this to work on Centos 7:

1. Install the latest unofficial mainline kernel
2. Select the newly installed kernel to be default in grub
3. Profit
   +1
   This is the right one.

[junaruga]

@wanganhong Sorry when I was checking past notification emails which I was mentioned in, I found this ticket's notification email.

: error while loading shared libraries: cannot apply additional memory protection after relocation: Permission denied

Sorry I have no idea about the message.

Can you check the binary file type of the qemu-aarch64-static binary file you copied to the host by the command $ file ./qemu-aarch64-static? You can see if you copied the correct aarch64 binary.

Sorry my mistake. The qemu-aarch64-static is not aarch64 binary but x86_64 binary. I checked the steps with podman on my Fedora 33 , where the kernel version is 5.11.7-200.fc33.x86_64 now.

$ uname -r
5.11.7-200.fc33.x86_64

$ podman --version
podman version 3.0.1

$ sudo podman run --rm --privileged multiarch/qemu-user-static:register --reset

$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64
enabled
interpreter /usr/bin/qemu-aarch64-static
flags:
offset 0
magic 7f454c460201010000000000000000000200b700
mask ffffffffffffff00fffffffffffffffffeffffff

$ podman create -it --name dummy multiarch/qemu-user-static:x86_64-aarch64 bash

$ podman container ls -a
CONTAINER ID  IMAGE                                                COMMAND  CREATED         STATUS   PORTS   NAMES
03c6c59f1678  docker.io/multiarch/qemu-user-static:x86_64-aarch64  bash     21 seconds ago  Created          dummy

$ podman cp dummy:/usr/bin/qemu-aarch64-static qemu-aarch64-static

$ ls qemu-aarch64-static
qemu-aarch64-static*

$ file ./qemu-aarch64-static
./qemu-aarch64-static: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=30efd3930fb9519b21470b113679376f2ffbb41a, for GNU/Linux 3.2.0, stripped

$ podman rm -f dummy

$ podman container ls -a
CONTAINER ID  IMAGE   COMMAND  CREATED  STATUS  PORTS   NAMES

$ podman run --rm -t -v $(pwd)/qemu-aarch64-static:/usr/bin/qemu-aarch64-static arm64v8/ubuntu uname -m
aarch64

[junaruga]

@wanganhong Or you might be able to try the way @zyystudio shared. I have not checked the links. But thanks for the info!

[constanine]

@junaruga Excuse me,I want to ask how to use command docker build with the same issue.
I had try use your command and at last use command like this

$ docker run --rm --privileged multiarch/qemu-user-static:register --reset

$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64
enabled
interpreter /usr/bin/qemu-aarch64-static
flags: 
offset 0
magic 7f454c460201010000000000000000000200b700
mask ffffffffffffff00fffffffffffffffffeffffff

$ docker create -it --name dummy multiarch/qemu-user-static:x86_64-aarch64 bash

$ docker container ls -a
CONTAINER ID        IMAGE                                       COMMAND             CREATED             STATUS              PORTS               NAMES
6ab622a76dfa        multiarch/qemu-user-static:x86_64-aarch64   "bash"              3 minutes ago       Created                                 dummy

$ docker cp dummy:/usr/bin/qemu-aarch64-static qemu-aarch64-static

$ ls qemu-aarch64-static
qemu-aarch64-static*

$ docker rm -f dummy

$ docker container ls -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

$ docker build --force-rm -v $(pwd)/qemu-aarch64-static:/usr/bin/qemu-aarch64-static -t basic-centos-arm-env:1.0 .

and Dockerfile is like

FROM arm64v8/centos:7
MAINTAINER bokesoft.com

RUN rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7; \
    yum update -y;
RUN \
    yum install -y wget; \
    yum install -y telnet; \
    yum install -y nano; \
    yum install -y vim;
	
ENV  TIME_ZONE Asia/Shanghai
RUN ln -sf /usr/share/zoneinfo/${TIME_ZONE} /etc/localtime;

RUN yum install -y kde-l10n-Chinese; \
    yum install -y glibc-common; \
    yum clean all; \
    localedef -c -f UTF-8 -i zh_CN zh_CN.utf8; \
    echo 'LANG="zh_CN.UTF-8"' > /etc/locale.conf && echo 'LC_ALL="zh_CN.UTF-8"' >> /etc/locale.conf && source /etc/locale.conf; \
    localedef -c -f UTF-8 -i zh_CN zh_CN.utf8;
ENV LANG zh_CN.UTF-8
	
RUN yum install -y vixie-cron; \
    yum -y install crontabs;

...