Build Redroid 8 armv8a (64-bit) Exec format error

[brunoaduarte]

Build Redroid 8 armv8a (64-bit) Exec format error

[zhouziyang]

Try redriod 10:

• disable omx, and enable codec2
• build your own 64bit only redroid image

It's possible to implement a native bridge which translate 32bit to 64bit, but it needs lot of efforts.

[zhouziyang]

Seems you packed the wrong binaries. Try run file <path-to-system-bin-audioserver >, and make sure to use the correct ABI.

[brunoaduarte]

Seems you packed the wrong binaries. Try run file <path-to-system-bin-audioserver >, and make sure to use the correct ABI.

@zhouziyang the binaries seems correct. ARM64 is compatible with ARM32 binaries, right?

redroid_arm64:/ # file /system/bin/audioserver
/system/bin/audioserver: ELF shared object, 32-bit LSB arm, dynamic (/system/bin/linker), for Android 27, BuildID=bb2923923c96b26856ee7a09a4909a13, stripped

redroid_arm64:/ # file /system/bin/cameraserver
/system/bin/cameraserver: ELF shared object, 32-bit LSB arm, dynamic (/system/bin/linker), for Android 27, BuildID=750ba28bf28fd24ade0bb63ac84a4139, stripped

redroid_arm64:/ # file /system/bin/app_process32
/system/bin/app_process32: ELF shared object, 32-bit LSB arm, dynamic (/system/bin/linker), for Android 27, BuildID=49a7e2847a198ffe94663c555d039833, stripped

redroid_arm64:/ # file /system/bin/drmserver
/system/bin/drmserver: ELF shared object, 32-bit LSB arm, dynamic (/system/bin/linker), for Android 27, BuildID=6593029d96ba32effecce87cfb3b2330, stripped

redroid_arm64:/ # file /system/bin/mediadrmserver
/system/bin/mediadrmserver: ELF shared object, 32-bit LSB arm, dynamic (/system/bin/linker), for Android 27, BuildID=5e9d3e0f26c7e108a9506df1b75bd193, stripped

redroid_arm64:/ # file /vendor/bin/hw/[email protected]
/vendor/bin/hw/[email protected]: ELF shared object, 32-bit LSB arm, dynamic (/system/bin/linker), for Android 27, BuildID=f19ffc8977bef2360287a430d5d59137, stripped

redroid_arm64:/ # file /vendor/bin/hw/[email protected]
/vendor/bin/hw/[email protected]: ELF shared object, 32-bit LSB arm, dynamic (/system/bin/linker), for Android 27, BuildID=b227b7cc85d822ff782afd3ac794a304, stripped

dpi@redroid-builder:/src$ file out/target/product/redroid_arm64/system/bin/audioserver
out/target/product/redroid_arm64/system/bin/audioserver: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[md5/uuid]=bb2923923c96b26856ee7a09a4909a13, stripped

[brunoaduarte]

Same error with Redroid 10 arm64 build

In this case audioserver for example is an aarch64 binary

dpi@ubuntu:~$ file ~/redroid/out/target/product/redroid_arm64/system/bin/audioserver
/home/dpi/redroid/out/target/product/redroid_arm64/system/bin/audioserver: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /system/bin/linker64, BuildID[md5/uuid]=012e21baa599f2cc479f400e7f215346, stripped

But mediaserver for example is not aarch64 (it's 32bit) thus triggering the error

dpi@ubuntu:~$ file ~/redroid/out/target/product/redroid_arm64/system/bin/mediaserver
/home/dpi/redroid/out/target/product/redroid_arm64/system/bin/mediaserver: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, BuildID[md5/uuid]=aa44b40bbb0508d612183b677489d57c, stripped

[  105.140217] init: mkdir("/dev/pts", 0755) failed File exists
[  105.140979] init: mount("proc", "/proc", "proc", 0, "hidepid=2,gid=" MAKE_STR(AID_READPROC)) failed Device or resource busy
[  105.142613] init: mount("sysfs", "/sys", "sysfs", 0, NULL) failed Device or resource busy
[  105.143220] init: mount("selinuxfs", "/sys/fs/selinux", "selinuxfs", 0, NULL) failed No such file or directory
[  105.143802] init: mknod("/dev/kmsg", S_IFCHR | 0600, makedev(1, 11)) failed File exists
[  105.144234] init: mknod("/dev/random", S_IFCHR | 0666, makedev(1, 8)) failed File exists
[  105.144676] init: mknod("/dev/urandom", S_IFCHR | 0666, makedev(1, 9)) failed File exists
[  105.145185] init: mknod("/dev/ptmx", S_IFCHR | 0666, makedev(5, 2)) failed File exists
[  105.145659] init: mknod("/dev/null", S_IFCHR | 0666, makedev(1, 3)) failed File exists
[  105.153157] init: Could not set 'ro.hardware' to 'redroid' while loading .prop filesRead-only property was already set
[  105.155084] init: Could not set 'ro.secure' to '1' while loading .prop filesRead-only property was already set
[  105.159834] cgroup1: Unknown subsys name 'schedtune'
[  105.160005] libprocessgroup: Failed to mount schedtune cgroup: Invalid argument
[  105.195968] init: couldn't write 13 to /dev/cpuset/system-background/tasks: No space left on device
[  105.196228] init: couldn't write 14 to /dev/cpuset/system-background/tasks: No space left on device
[  105.196570] init: couldn't write 15 to /dev/cpuset/system-background/tasks: No space left on device
[  105.197761] init: couldn't write 16 to /dev/cpuset/system-background/tasks: No space left on device
[  105.201806] init: couldn't write 22 to /dev/cpuset/foreground/tasks: No space left on device
[  105.240847] apexd: Failed to walk /product/apex : Can't open /product/apex for reading : No such file or directory
[  105.243782] apexd: Failed to activate packages from /product/apex : Failed to scan /product/apex : Can't open /product/apex for reading : No such file or directory
[  105.272420] init: Could not load single persistent property file, trying legacy directory: Unable to read persistent property file: open() failed: No such file or directory
[  105.299045] init: couldn't write 48 to /dev/cpuset/system-background/tasks: No space left on device
[  105.314660] init: property 'dev.mnt.blk.data' doesn't exist while expanding '/sys/fs/f2fs/${dev.mnt.blk.data}/cp_interval'
[  105.316462] init: couldn't write 59 to /dev/cpuset/foreground/tasks: No space left on device
[  105.317198] init: couldn't write 61 to /dev/cpuset/foreground/tasks: No space left on device
[  105.323487] init: couldn't write 73 to /dev/cpuset/foreground/tasks: No space left on device
[  105.323626] init: couldn't write 66 to /dev/cpuset/system-background/tasks: No space left on device
[  105.324529] init: cannot execve('/vendor/bin/hw/[email protected]'): Exec format error
[  105.327125] init: couldn't write 75 to /dev/cpuset/system-background/tasks: No space left on device
[  105.333797] init: Failed to bind socket 'pdx/system/vr/display/client': No such file or directory
[  105.335085] init: Failed to bind socket 'pdx/system/vr/display/manager': No such file or directory
[  105.336061] init: Failed to bind socket 'pdx/system/vr/display/vsync': No such file or directory
[  105.339607] init: Could not start service 'flash_recovery' as part of class 'main': Cannot find '/system/bin/install-recovery.sh': No such file or directory
[  105.342418] init: couldn't write 91 to /dev/cpuset/foreground/tasks: No space left on device
[  105.342529] init: setpgid failed for console: Operation not permitted
[  105.343874] init: couldn't write 94 to /dev/cpuset/camera-daemon/tasks: No such file or directory
[  105.346499] init: couldn't write 92 to /dev/cpuset/foreground/tasks: No space left on device
[  105.347471] init: couldn't write 95 to /dev/cpuset/foreground/tasks: No space left on device
[  105.347518] init: cannot execve('/system/bin/cameraserver'): Exec format error
[  105.348029] [email protected]: wakealarm_init: timerfd_create failed
[  105.350941] init: cannot execve('/system/bin/app_process32'): Exec format error
[  105.352288] init: cannot execve('/system/bin/drmserver'): Exec format error
[  105.359292] init: couldn't write 100 to /dev/cpuset/foreground/tasks: No space left on device
[  105.360607] init: couldn't write 101 to /dev/cpuset/foreground/tasks: No space left on device
[  105.361168] init: couldn't write 111 to /dev/cpuset/foreground/tasks: No space left on device
[  105.361492] init: couldn't write 103 to /dev/cpuset/foreground/tasks: No space left on device
[  105.361772] init: couldn't write 104 to /dev/cpuset/foreground/tasks: No space left on device
[  105.362397] init: couldn't write 102 to /dev/cpuset/foreground/tasks: No space left on device
[  105.363091] init: couldn't write 107 to /dev/cpuset/system-background/tasks: No space left on device
[  105.363967] init: Failed to open file '/d/mmc0/mmc0:0001/ext_csd': No such file or directory
[  105.365927] init: couldn't write 109 to /dev/cpuset/system-background/tasks: No space left on device
[  105.367783] init: cannot execve('/system/bin/mediadrmserver'): Exec format error
[  105.368981] init: cannot execve('/vendor/bin/hw/[email protected]'): Exec format error
[  105.374429] init: cannot execve('/system/bin/mediaserver'): Exec format error
[  105.379135] init: couldn't write 121 to /dev/cpuset/foreground/tasks: No space left on device
[  105.409990] init: couldn't write 139 to /dev/cpuset/system-background/tasks: No space left on device
[  105.410682] init: couldn't write 140 to /dev/cpuset/system-background/tasks: No space left on device

[zhouziyang]

Seems only 64bit mode supported by your platform (such as Apple Silicon). Try a 64 bit only redroid image (provided since redroid 12, redroid/redroid-12.0.0_64only-latest).

[brunoaduarte]

Seems only 64bit mode supported by your platform (such as Apple Silicon). Try a 64 bit only redroid image (provided since redroid 12, redroid/redroid-12.0.0_64only-latest).

You're right, what a POS this Apple M1 chip, they removed native 32bit support. Now I must get an Raspberry Pi 5 or a use Azure ARM64 VM.

redroid-12.0.0_64only works fine... I need Redroid 8 and 10 images for arm64. Any ways of making it work? Can I force 64 bit binaries into these images?

Or maybe it's possible to embed a qemu-arm-static as binfmt interpreter for arm on the Redroid container @zhouziyang ?
https://github.com/dbhi/qus

Check this demonstration

ubuntu@ubuntu:~$ file /usr/bin/hello
/usr/bin/hello: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=659134ba2144781617657cbc03c4aa20e0defa64, for GNU/Linux 3.2.0, stripped

ubuntu@ubuntu:~$ /usr/bin/hello
-bash: /usr/bin/hello: cannot execute binary file: Exec format error

ubuntu@ubuntu:~$ docker run --rm --privileged aptman/qus -s -- -p arm
cat ./qemu-binfmt-conf.sh | sh -s -- --path=/qus/bin -p arm --suffix -static
Setting /qus/bin/qemu-arm-static as binfmt interpreter for arm

ubuntu@ubuntu:~$ /usr/bin/hello
Hello, world!

[brunoaduarte]

Try redriod 10:

• disable omx, and enable codec2
• build your own 64bit only redroid image

It's possible to implement a native bridge which translate 32bit to 64bit, but it needs lot of efforts.

Followed the redroid:12.0.0_arm64_only template to create redroid:10.0.0_arm64_only

When trying to lunch redroid_arm64_only-userdebug; m -j$(nproc) got this error

FAILED: ninja: 'frameworks/av/services/mediacodec/seccomp_policy/mediacodec-arm64.policy', needed by 'out/target/product/a01core/obj/ETC/mediacodec.policy_intermediates/mediacodec.policy', missing and no known rule to make it

So, copied the file from Redroid 12 AOSP source

~/redroid/frameworks/av/services/mediacodec/seccomp_policy/mediacodec-arm64.policy

# Organized by frequency of systemcall - in descending order for
# best performance.
futex: 1
ioctl: 1
write: 1
prctl: 1
clock_gettime: 1
getpriority: 1
read: 1
close: 1
writev: 1
dup: 1
ppoll: 1
mmap2: 1
getrandom: 1
memfd_create: 1
ftruncate: 1
ftruncate64: 1
# mremap: Ensure |flags| are (MREMAP_MAYMOVE | MREMAP_FIXED) TODO: Once minijail
# parser support for '<' is in this needs to be modified to also prevent
# |old_address| and |new_address| from touching the exception vector page, which
# on ARM is statically loaded at 0xffff 0000. See
# http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0211h/Babfeega.html
# for more details.
mremap: arg3 == 3 || arg3 == MREMAP_MAYMOVE
munmap: 1
mprotect: 1
madvise: 1
openat: 1
sigaltstack: 1
clone: 1
setpriority: 1
getuid32: 1
fstat64: 1
fstatfs64: 1
pread64: 1
faccessat: 1
readlinkat: 1
exit: 1
rt_sigprocmask: 1
set_tid_address: 1
restart_syscall: 1
exit_group: 1
rt_sigreturn: 1
pipe2: 1
gettimeofday: 1
sched_yield: 1
nanosleep: 1
lseek: 1
_llseek: 1
sched_get_priority_max: 1
sched_get_priority_min: 1
statfs64: 1
sched_setscheduler: 1
fstatat64: 1
ugetrlimit: 1
getdents64: 1
getrandom: 1
@include /system/etc/seccomp_policy/crash_dump.arm.policy
@include /system/etc/seccomp_policy/code_coverage.arm.policy

then tried to lunch redroid_arm64_only-userdebug; m -j$(nproc) again and got another error

[ 90% 21872/24200] Verifying vendor VINTF manifest.
FAILED: out/target/product/redroid_arm64_only/verified_assembled_vendor_manifest.xml
/bin/bash -c "PRODUCT_ENFORCE_VINTF_MANIFEST=true               out/host/linux-x86/bin/assemble_vintf               -c out/target/product/redroid_arm64_only/obj/ETC/verified_assembled_system_matrix.xml_intermediates/verified_assembled_system_matrix.xml             -i out/target/product/redroid_arm64_only/obj/ETC/device_manifest.xml_intermediates/manifest.xml         \$([ -d out/target/product/redroid_arm64_only/vendor/etc/vintf/manifest ] &&                find out/target/product/redroid_arm64_only/vendor/etc/vintf/manifest -type f -name \"*.xml\" |               sed \"s/^/-i /\" | tr '\\n' ' ') -o out/target/product/redroid_arm64_only/verified_assembled_vendor_manifest.xml"
Not compatible: HALs incompatible. Matrix level = 3. Manifest level = 3. The following requirements are not met:
android.hardware.media.omx:
    required: @1.0::IOmxStore/default
    provided:
android.hardware.media.omx:
    required: @1.0::IOmx/default
    provided:

19:41:53 ninja failed with: exit status 1

#### failed to build some targets (28:24 (mm:ss)) ####

so added the requirements to the file

~/redroid/device/redroid/manifest.xml

    <hal format="hidl">
        <name>android.hardware.media.omx</name>
        <transport>hwbinder</transport>
        <version>1.0</version>
        <interface>
            <name>IOmx</name>
            <instance>default</instance>
        </interface>
        <interface>
            <name>IOmxStore</name>
            <instance>default</instance>
        </interface>
        <fqname>@1.0::IOmx/default</fqname>
        <fqname>@1.0::IOmxStore/default</fqname>
    </hal>

And it successfully builded. But when I try to run the container it connects to adbd but when I try to scrcpy got this other error

07-25 20:00:30.545  1676  1676 D AndroidRuntime: >>>>>> START com.android.internal.os.RuntimeInit uid 2000 <<<<<<
07-25 20:00:30.547  1676  1676 I AndroidRuntime: Using default boot image
07-25 20:00:30.547  1676  1676 I AndroidRuntime: Leaving lock profiling enabled
07-25 20:00:30.605  1676  1676 D ICU     : Time zone APEX file found: /apex/com.android.tzdata/etc/icu/icu_tzdata.dat
07-25 20:00:30.616  1676  1676 W app_process: JNI RegisterNativeMethods: attempt to register 0 native methods for android.media.AudioAttributes
07-25 20:00:30.618  1676  1676 D AndroidRuntime: Calling main entry com.genymobile.scrcpy.Server
07-25 20:00:30.620  1676  1676 I scrcpy  : Device: [redroid] redroid redroid_arm64_only (Android 10)
07-25 20:00:30.633  1676  1694 W scrcpy  : Audio disabled: it is not supported before Android 11
07-25 20:00:30.637  1676  1695 I app_process: Thread[18,tid=1695,Native,Thread*=0xe989e4bce800,peer=0x12c3e880,"video"] recursive attempt to load library "libmedia_jni.so"
07-25 20:00:30.637  1676  1695 D MtpDeviceJNI: register_android_mtp_MtpDevice
07-25 20:00:30.637  1676  1695 I app_process: Thread[18,tid=1695,Native,Thread*=0xe989e4bce800,peer=0x12c3e880,"video"] recursive attempt to load library "libmedia_jni.so"
07-25 20:00:30.637  1676  1695 I app_process: Thread[18,tid=1695,Native,Thread*=0xe989e4bce800,peer=0x12c3e880,"video"] recursive attempt to load library "libmedia_jni.so"
07-25 20:00:30.637  1676  1695 I ServiceManager: Waiting for service 'media.player' on '/dev/binder'...
07-25 20:00:30.660  1689  1689 D AndroidRuntime: >>>>>> START com.android.internal.os.RuntimeInit uid 2000 <<<<<<
07-25 20:00:30.661  1689  1689 I AndroidRuntime: Using default boot image
07-25 20:00:30.661  1689  1689 I AndroidRuntime: Leaving lock profiling enabled
07-25 20:00:30.685  1689  1689 D ICU     : Time zone APEX file found: /apex/com.android.tzdata/etc/icu/icu_tzdata.dat
07-25 20:00:30.694  1689  1689 W app_process: JNI RegisterNativeMethods: attempt to register 0 native methods for android.media.AudioAttributes
07-25 20:00:30.695  1689  1689 D AndroidRuntime: Calling main entry com.genymobile.scrcpy.CleanUp
07-25 20:00:31.011   207   347 W ServiceManagement: Waited one second for [email protected]::IOmxStore/default
07-25 20:00:31.012   207   347 I ServiceManagement: getService: Trying again for [email protected]::IOmxStore/default...
07-25 20:00:31.214   370   462 W ServiceManagement: Waited one second for [email protected]::IOmxStore/default
07-25 20:00:31.215   370   462 I ServiceManagement: getService: Trying again for [email protected]::IOmxStore/default...
07-25 20:00:31.311  1554  1554 I ServiceManager: Waiting for service 'media.camera' on '/dev/binder'...
07-25 20:00:31.642  1676  1695 I ServiceManager: Waiting for service 'media.player' on '/dev/binder'...

[zhouziyang]

Check device/redroid/redroid.mk, and try get codec2 working

ifneq ($(REDROID_DISABLE_OMX),true)
    $(call inherit-product, hardware/redroid/omx/omx.mk)
else
    $(call inherit-product, hardware/redroid/c2/c2.mk)
endif