Comments (46)
Sharing gist note written by @toricls (thank you!) for running Lima on ARM Mac
https://gist.github.com/toricls/d3dd0bec7d4c6ddbcf2d25f211e8cd7b
Screenshot: https://twitter.com/toricls/status/1432925285377142789
Just updated my installation guide to use Homebrew to install QEMU and Lima. It's dead simple now 😍
https://gist.github.com/toricls/d3dd0bec7d4c6ddbcf2d25f211e8cd7b
from lima.
Just reporting I've been working with this for the past two days with great success.
After creating a vm, I run lima sh -c 'sudo apt update && sudo apt install -y qemu-user-static binfmt-support'
to also support amd64 containers.
from lima.
Upstream brew patched qemu, so it is no longer necessary to use the tap.
Homebrew/homebrew-core@5e8eb54#diff-5fe005c786b1e905434aaa353962cc6a58b0a6431e7f9a785076f0ad5f277a3d
─❯ qemu-system-aarch64 -accel help
Accelerators supported in QEMU binary:
hvf
tcg
from lima.
Just tested on an M1 Mac Mini using what's available in Homebrew:
% limactl start default INFO[0000] Using the existing instance "default" INFO[0000] Downloading "https://github.com/containerd/nerdctl/releases/download/v0.9.0/nerdctl-full-0.9.0-linux-arm64.tar.gz" INFO[0000] Using cache "/Users/adam/Library/Caches/lima/download/by-url-sha256/08927199d45dbbfa40ed207c68567a6913b7b347c62c033a5c501220bd8958b2/data" INFO[0000] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/adam/.lima/default/serial.log") INFO[0000] SSH Local Port: 60022 INFO[0000] [hostagent] Waiting for the essential requirement 1 of 4: "ssh" INFO[0000] [hostagent] QEMU has exited FATA[0000] exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/adam/.lima/default/ha.stderr.log")
Here's the latter file:
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/adam/.lima/default/serial.log\")","time":"2021-06-22T21:38:31+01:00"} {"level":"debug","msg":"qCmd.Args: [/opt/homebrew/bin/qemu-system-aarch64 -cpu cortex-a72 -machine virt,accel=hvf,highmem=off -smp 4,sockets=1,cores=4,threads=1 -m 4096 -drive if=pflash,format=raw,readonly,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd -boot order=c,splash-time=0,menu=on -drive file=/Users/adam/.lima/default/diffdisk,if=virtio -cdrom /Users/adam/.lima/default/cidata.iso -net nic,model=virtio -net user,net=192.168.5.0/24,hostfwd=tcp:127.0.0.1:60022-:22 -device virtio-rng-pci -display none -vga none -device ramfb -device usb-ehci -device usb-kbd -device usb-mouse -parallel none -chardev socket,id=char-serial,path=/Users/adam/.lima/default/serial.sock,server,nowait,logfile=/Users/adam/.lima/default/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/adam/.lima/default/qmp.sock,server,nowait -qmp chardev:char-qmp -name lima-default -pidfile /Users/adam/.lima/default/qemu.pid]","time":"2021-06-22T21:38:31+01:00"} {"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-06-22T21:38:31+01:00"} {"level":"debug","msg":"executing script \"ssh\"","time":"2021-06-22T21:38:31+01:00"} {"level":"debug","msg":"stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 60022: Connection refused\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 60022: Connection refused\\r\\n\": exit status 255","time":"2021-06-22T21:38:31+01:00"} {"error":"exit status 1","level":"info","msg":"QEMU has exited","time":"2021-06-22T21:38:31+01:00"} exit status 1
I got the same issue with my Mac on ARM.
Lima version:0.6.1
I have installed lima use brew,but after installed i follow the readme.md ,uninstall the qemu 0.6.1
brew uninstall --ignore-dependencies qemu
reinstall qemu with patch
brew install simnalamburt/x/qemu-hvf
verify qemu version shows
$ qemu-system-aarch64 -accel help
Accelerators supported in QEMU binary:
tcg
hvf
after all these , lima works fine
from lima.
Thanks, testing is highly appreciated.
I'm planning to purchase an ARM Mac probably after they release the 2021 model of MacBookPro.
from lima.
Sharing gist note written by @toricls (thank you!) for running Lima on ARM Mac
https://gist.github.com/toricls/d3dd0bec7d4c6ddbcf2d25f211e8cd7b
Screenshot: https://twitter.com/toricls/status/1432925285377142789
from lima.
I was able to resolve this by setting the
http_proxy
andhttps_proxy
environment variables to my local proxy.
There were various issues with environment variables in general, and proxy settings in particular in earlier lima releases, but I'm not aware of any remaining issues in v0.7.3 or later.
If the latest lima release does not automatically inherit proxy setting, then please file a separate Github issue with the details, so we can investigate.
@AkihiroSuda I'm tempted to close this bug, as it seems to just collect all manner of feedback related to M1 support instead of being specific to a single issue. I.e. I cannot tell what actions would be required to resolve it.
from lima.
@AkihiroSuda nixpkgs user so can't help here, although FYI there's a bit of work at nixpkgs to improve entitlements and signatures for qemu on aarch64-darwin, so it should be as easy as e.g nix-env -iA nixpkgs.qemu
on that platform soon enough (it already works as expected on x86_64-darwin)
from lima.
This is due to containerd/nerdctl#458 .
nerdctl run --rm
(or nerdctl rm CONTAINERID
) should work as a workaround.
from lima.
@AkihiroSuda Is it because of lack of hardware? I may be able to lend a hand and do some testing on mine.
from lima.
A quick test with a custom-built† QEMU 5.2.0 installed in /opt/qemu-5.2.0
shows the following:
$ curl -L -O https://github.com/AkihiroSuda/lima/releases/download/v0.2.0/lima-0.2.0-Darwin-arm64.tar.gz
$ mkdir lima
$ cd lima
$ tar xvzf ../lima-0.2.0-Darwin-arm64.tar.gz
$ find . -type f | xargs xattr -d com.apple.quarantine # necessary when downloaded via Safari
$ export PATH="/opt/qemu-5.2.0/bin:$PWD/bin:$PATH"
$ limactl start # vim popped up: reviewed and accepted the defaults
INFO[0034] Downloading "https://github.com/containerd/nerdctl/releases/download/v0.8.3/nerdctl-full-0.8.3-linux-arm64.tar.gz"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 645 100 645 0 0 2041 0 --:--:-- --:--:-- --:--:-- 2041
100 152M 100 152M 0 0 9987k 0 0:00:15 0:00:15 --:--:-- 10.7M
INFO[0050] Downloaded "nerdctl-full-0.8.3-linux-arm64.tar.gz"
INFO[0050] Attempting to download the image from "~/Downloads/hirsute-server-cloudimg-arm64.img"
INFO[0050] Attempting to download the image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-arm64.img"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 520M 100 520M 0 0 10.4M 0 0:00:49 0:00:49 --:--:-- 10.1M
INFO[0101] Downloaded image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-arm64.img"
INFO[0101] Starting QEMU (hint: to watch the boot progress, see "/Users/lloeki/.lima/default/serial.log")
INFO[0101] SSH: 127.0.0.1:60022
INFO[0101] Waiting for the essential requirement 1 of 3: "ssh"
INFO[0120] The essential requirement 1 of 3 is satisfied
INFO[0120] Waiting for the essential requirement 2 of 3: "sshfs binary to be installed"
INFO[0129] The essential requirement 2 of 3 is satisfied
INFO[0129] Waiting for the essential requirement 3 of 3: "the guest agent to be running"
INFO[0129] The essential requirement 3 of 3 is satisfied
INFO[0129] Mounting "/Users/lloeki"
INFO[0130] Mounting "/tmp/lima"
INFO[0131] Waiting for the optional requirement 1 of 1: "containerd binaries to be installed"
INFO[0131] Forwarding "/run/user/501/lima-guestagent.sock" (guest) to "/Users/lloeki/.lima/default/ga.sock" (host)
INFO[0134] The optional requirement 1 of 1 is satisfied
INFO[0134] READY. Run `lima bash` to open the shell.
As was suggested while waiting for ssh, I looked into the boot output in a second term:
$ tail -f /Users/lloeki/.lima/default/serial.log # looked good
cloud-init-local.service
Starting Network Service...
[ OK ] Started Network Service.
Starting Wait for Network to be Configured...
----8<----
-----END SSH HOST KEY KEYS-----
[ 38.032196] cloud-init[1297]: Cloud-init v. 21.1-19-gbad84ad4-0ubuntu3 finished at Thu, 10 Jun 2021 12:04:29 +0000. Datasource DataSourceNoCloud [seed=/dev/vdb][dsmode=net]. Up 38.02 seconds
Alright, everything seems up, let's try in a third term:
$ export PATH="/opt/qemu-5.2.0/bin:$PWD/bin:$PATH"
$ lima bash
lloeki@lima-default:/Users/lloeki/Source/sandbox/lima$ ls
bin share
lloeki@lima-default:/Users/lloeki/Source/sandbox/lima$ echo foo > bar
bash: bar: Read-only file system
lloeki@lima-default:/Users/lloeki/Source/sandbox/lima$ mount | grep sshfs
:/Users/lloeki on /Users/lloeki type fuse.sshfs (ro,nosuid,nodev,relatime,user_id=501,group_id=1000)
:/tmp/lima on /tmp/lima type fuse.sshfs (rw,nosuid,nodev,relatime,user_id=501,group_id=1000)
lloeki@lima-default:/Users/lloeki/Source/sandbox/lima$ echo foo > /tmp/lima/bar
lloeki@lima-default:/Users/lloeki/Source/sandbox/lima$ exit
$ cat /tmp/lima/bar
foo
$ lima uname -a
Linux lima-default 5.11.0-18-generic #19-Ubuntu SMP Fri May 7 14:21:20 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
I'd say it works fine.
† followed this, only I added this patch as well
from lima.
qemu-system-x86_64
(5.2.0) is untested because it doesn't seem to operate properly (unrelated to lima, it goes down with Could not allocate dynamic translator buffer
with FreeDOS 1.2) for some reason, although 6.0.0 works, but I did not test the latter with lima (I don't seem to be able to enable HVF).
from lima.
Thanks!
from lima.
Just tested on an M1 Mac Mini using what's available in Homebrew:
% limactl start default
INFO[0000] Using the existing instance "default"
INFO[0000] Downloading "https://github.com/containerd/nerdctl/releases/download/v0.9.0/nerdctl-full-0.9.0-linux-arm64.tar.gz"
INFO[0000] Using cache "/Users/adam/Library/Caches/lima/download/by-url-sha256/08927199d45dbbfa40ed207c68567a6913b7b347c62c033a5c501220bd8958b2/data"
INFO[0000] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/adam/.lima/default/serial.log")
INFO[0000] SSH Local Port: 60022
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0000] [hostagent] QEMU has exited
FATA[0000] exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/adam/.lima/default/ha.stderr.log")
Here's the latter file:
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/adam/.lima/default/serial.log\")","time":"2021-06-22T21:38:31+01:00"}
{"level":"debug","msg":"qCmd.Args: [/opt/homebrew/bin/qemu-system-aarch64 -cpu cortex-a72 -machine virt,accel=hvf,highmem=off -smp 4,sockets=1,cores=4,threads=1 -m 4096 -drive if=pflash,format=raw,readonly,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd -boot order=c,splash-time=0,menu=on -drive file=/Users/adam/.lima/default/diffdisk,if=virtio -cdrom /Users/adam/.lima/default/cidata.iso -net nic,model=virtio -net user,net=192.168.5.0/24,hostfwd=tcp:127.0.0.1:60022-:22 -device virtio-rng-pci -display none -vga none -device ramfb -device usb-ehci -device usb-kbd -device usb-mouse -parallel none -chardev socket,id=char-serial,path=/Users/adam/.lima/default/serial.sock,server,nowait,logfile=/Users/adam/.lima/default/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/adam/.lima/default/qmp.sock,server,nowait -qmp chardev:char-qmp -name lima-default -pidfile /Users/adam/.lima/default/qemu.pid]","time":"2021-06-22T21:38:31+01:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-06-22T21:38:31+01:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2021-06-22T21:38:31+01:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 60022: Connection refused\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 60022: Connection refused\\r\\n\": exit status 255","time":"2021-06-22T21:38:31+01:00"}
{"error":"exit status 1","level":"info","msg":"QEMU has exited","time":"2021-06-22T21:38:31+01:00"}
exit status 1
from lima.
Thanks for testing, but for ARM Mac you have to apply a patch to QEMU https://gist.github.com/nrjdalal/e70249bb5d2e9d844cc203fd11f74c55
If it still does not work, please show serial.log .
from lima.
Followed the steps on m1 mac
installed the qemu patch from https://gist.github.com/nrjdalal/e70249bb5d2e9d844cc203fd11f74c55
when I try to start qemu
~ ❯ qemu-system-aarch64 \ 10:47:49 AM
-machine virt,accel=hvf,highmem=off
-cpu cortex-a72 -smp 4 -m 4G
-device virtio-gpu-pci
-device virtio-keyboard-pci
-drive "format=raw,file=edk2-aarch64-code.fd,if=pflash,readonly=on"
-drive "format=raw,file=ovmf_vars.fd,if=pflash"
-drive "format=qcow2,file=virtual-disk.qcow2"
-cdrom ubuntu-lts.iso
qemu-system-aarch64: invalid accelerator hvf
**~ ❯ limactl --debug start** 7s 10:49:31 AM
DEBU[0000] interpreting argument "default" as an instance name "default"
INFO[0000] Using the existing instance "default"
INFO[0000] Downloading "https://github.com/containerd/nerdctl/releases/download/v0.11.0/nerdctl-full-0.11.0-linux-arm64.tar.gz" (sha256:865d8648e2378e10dc2311f5a373fd0b85cbb6c03d2adafb1ca752d0e1267649)
DEBU[0000] file "/var/folders/h2/1n08pbk124d2488vvfssb13c0000gr/T/lima-download-nerdctl146249405/nerdctl-full-0.11.0-linux-arm64.tar.gz" is cached as "/Users/kiran_chavala/Library/Caches/lima/download/by-url-sha256/d6cadf5f656696cdf565db5101a16b6c6bc837cab7886b4fd0e48ac3a58dd6f4/data"
DEBU[0000] Comparing digest "sha256:865d8648e2378e10dc2311f5a373fd0b85cbb6c03d2adafb1ca752d0e1267649" with the cached digest file "/Users/kiran_chavala/Library/Caches/lima/download/by-url-sha256/d6cadf5f656696cdf565db5101a16b6c6bc837cab7886b4fd0e48ac3a58dd6f4/sha256.digest", not computing the actual digest of "/Users/kiran_chavala/Library/Caches/lima/download/by-url-sha256/d6cadf5f656696cdf565db5101a16b6c6bc837cab7886b4fd0e48ac3a58dd6f4/data"
DEBU[0000] res.ValidatedDigest=true
INFO[0000] Using cache "/Users/kiran_chavala/Library/Caches/lima/download/by-url-sha256/d6cadf5f656696cdf565db5101a16b6c6bc837cab7886b4fd0e48ac3a58dd6f4/data"
INFO[0001] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/kiran_chavala/.lima/default/serial.log")
DEBU[0001] [hostagent] qCmd.Args: [/opt/homebrew/bin/qemu-system-aarch64 -cpu cortex-a72 -machine virt,accel=hvf,highmem=off -smp 4,sockets=1,cores=4,threads=1 -m 4096 -drive if=pflash,format=raw,readonly,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd -boot order=c,splash-time=0,menu=on -drive file=/Users/kiran_chavala/.lima/default/diffdisk,if=virtio -cdrom /Users/kiran_chavala/.lima/default/cidata.iso -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:60022-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:a7:10:91 -device virtio-rng-pci -display none -vga none -device ramfb -device usb-ehci -device usb-kbd -device usb-mouse -parallel none -chardev socket,id=char-serial,path=/Users/kiran_chavala/.lima/default/serial.sock,server,nowait,logfile=/Users/kiran_chavala/.lima/default/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/kiran_chavala/.lima/default/qmp.sock,server,nowait -qmp chardev:char-qmp -name lima-default -pidfile /Users/kiran_chavala/.lima/default/qemu.pid]
DEBU[0001] received an event event="{2021-08-23 10:49:38.526942 +0530 IST {false false false [] 60022}}"
INFO[0001] SSH Local Port: 60022
INFO[0001] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
DEBU[0001] [hostagent] executing script "ssh"
DEBU[0001] [hostagent] qemu[stderr]: qemu-system-aarch64: -drive if=pflash,format=raw,readonly,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd: warning: short-form boolean option 'readonly' deprecated
DEBU[0001] [hostagent] qemu[stderr]: Please use readonly=on instead
DEBU[0001] [hostagent] stdout="", stderr="ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n", err=failed to execute script "ssh": stdout="", stderr="ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n": exit status 255
DEBU[0001] [hostagent] qemu[stderr]: qemu-system-aarch64: -chardev socket,id=char-serial,path=/Users/kiran_chavala/.lima/default/serial.sock,server,nowait,logfile=/Users/kiran_chavala/.lima/default/serial.log: warning: short-form boolean option 'server' deprecated
DEBU[0001] [hostagent] qemu[stderr]: Please use server=on instead
INFO[0001] [hostagent] QEMU has exited
DEBU[0001] [hostagent] qemu[stderr]: qemu-system-aarch64: -chardev socket,id=char-serial,path=/Users/kiran_chavala/.lima/default/serial.sock,server,nowait,logfile=/Users/kiran_chavala/.lima/default/serial.log: warning: short-form boolean option 'nowait' deprecated
DEBU[0001] received an event event="{2021-08-23 10:49:38.71089 +0530 IST {false false true [] 0}}"
FATA[0001] exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/kiran_chavala/.lima/default/ha.stderr.log")
**~ ❯ cat /Users/kiran_chavala/.lima/default/ha.stderr.log**
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see "/Users/kiran_chavala/.lima/default/serial.log")","time":"2021-08-23T10:49:38+05:30"}
{"level":"debug","msg":"qCmd.Args: [/opt/homebrew/bin/qemu-system-aarch64 -cpu cortex-a72 -machine virt,accel=hvf,highmem=off -smp 4,sockets=1,cores=4,threads=1 -m 4096 -drive if=pflash,format=raw,readonly,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd -boot order=c,splash-time=0,menu=on -drive file=/Users/kiran_chavala/.lima/default/diffdisk,if=virtio -cdrom /Users/kiran_chavala/.lima/default/cidata.iso -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:60022-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:a7:10:91 -device virtio-rng-pci -display none -vga none -device ramfb -device usb-ehci -device usb-kbd -device usb-mouse -parallel none -chardev socket,id=char-serial,path=/Users/kiran_chavala/.lima/default/serial.sock,server,nowait,logfile=/Users/kiran_chavala/.lima/default/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/kiran_chavala/.lima/default/qmp.sock,server,nowait -qmp chardev:char-qmp -name lima-default -pidfile /Users/kiran_chavala/.lima/default/qemu.pid]","time":"2021-08-23T10:49:38+05:30"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 4: "ssh"","time":"2021-08-23T10:49:38+05:30"}
{"level":"debug","msg":"executing script "ssh"","time":"2021-08-23T10:49:38+05:30"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-aarch64: -drive if=pflash,format=raw,readonly,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd: warning: short-form boolean option 'readonly' deprecated","time":"2021-08-23T10:49:38+05:30"}
{"level":"debug","msg":"qemu[stderr]: Please use readonly=on instead","time":"2021-08-23T10:49:38+05:30"}
{"level":"debug","msg":"stdout="", stderr="ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n", err=failed to execute script "ssh": stdout="", stderr="ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n": exit status 255","time":"2021-08-23T10:49:38+05:30"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-aarch64: -chardev socket,id=char-serial,path=/Users/kiran_chavala/.lima/default/serial.sock,server,nowait,logfile=/Users/kiran_chavala/.lima/default/serial.log: warning: short-form boolean option 'server' deprecated","time":"2021-08-23T10:49:38+05:30"}
{"level":"debug","msg":"qemu[stderr]: Please use server=on instead","time":"2021-08-23T10:49:38+05:30"}
{"error":"exit status 1","level":"info","msg":"QEMU has exited","time":"2021-08-23T10:49:38+05:30"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-aarch64: -chardev socket,id=char-serial,path=/Users/kiran_chavala/.lima/default/serial.sock,server,nowait,logfile=/Users/kiran_chavala/.lima/default/serial.log: warning: short-form boolean option 'nowait' deprecated","time":"2021-08-23T10:49:38+05:30"}
{"level":"debug","msg":"qemu[stderr]: Please use wait=off instead","time":"2021-08-23T10:49:38+05:30"}
from lima.
Getting the same error even after
~ ❯ sudo codesign -s - --entitlements entitlements.xml --force /usr/local/bin/qemu-system-aarch64
Password:
/usr/local/bin/qemu-system-aarch64: replacing existing signature
~ ❯ cd /usr/local/bin
/usr/local/bin ❯ qemu-system-aarch64 -accel help
Accelerators supported in QEMU binary:
tcg
from lima.
@kiranchavala I just followed the same steps from the gist manually, and it worked for me:
% qemu-system-aarch64 -accel help
Accelerators supported in QEMU binary:
hvf
tcg
This was on an (almost) freshly install M1 mini:
% brew ls
==> Formulae
autoconf gdbm glib m4 ninja pcre pkg-config readline xz
automake gettext libffi mpdecimal [email protected] pixman [email protected] sqlite
There was one comment in the gist discussion from a person having the same issue: they had python 3.8 installed. Once they moved to Python 3.9 the problem went away.
from lima.
Another easy route is to get UTM, which embeds up to date scratch that, it seems to not include the CLI binaries anymore.qemu-system-aarch64
and qemu-system-x86_64
that work right away.
from lima.
Just tested on an M1 Mac Mini using what's available in Homebrew:
% limactl start default INFO[0000] Using the existing instance "default" INFO[0000] Downloading "https://github.com/containerd/nerdctl/releases/download/v0.9.0/nerdctl-full-0.9.0-linux-arm64.tar.gz" INFO[0000] Using cache "/Users/adam/Library/Caches/lima/download/by-url-sha256/08927199d45dbbfa40ed207c68567a6913b7b347c62c033a5c501220bd8958b2/data" INFO[0000] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/adam/.lima/default/serial.log") INFO[0000] SSH Local Port: 60022 INFO[0000] [hostagent] Waiting for the essential requirement 1 of 4: "ssh" INFO[0000] [hostagent] QEMU has exited FATA[0000] exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/adam/.lima/default/ha.stderr.log")
Here's the latter file:
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/adam/.lima/default/serial.log\")","time":"2021-06-22T21:38:31+01:00"} {"level":"debug","msg":"qCmd.Args: [/opt/homebrew/bin/qemu-system-aarch64 -cpu cortex-a72 -machine virt,accel=hvf,highmem=off -smp 4,sockets=1,cores=4,threads=1 -m 4096 -drive if=pflash,format=raw,readonly,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd -boot order=c,splash-time=0,menu=on -drive file=/Users/adam/.lima/default/diffdisk,if=virtio -cdrom /Users/adam/.lima/default/cidata.iso -net nic,model=virtio -net user,net=192.168.5.0/24,hostfwd=tcp:127.0.0.1:60022-:22 -device virtio-rng-pci -display none -vga none -device ramfb -device usb-ehci -device usb-kbd -device usb-mouse -parallel none -chardev socket,id=char-serial,path=/Users/adam/.lima/default/serial.sock,server,nowait,logfile=/Users/adam/.lima/default/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/adam/.lima/default/qmp.sock,server,nowait -qmp chardev:char-qmp -name lima-default -pidfile /Users/adam/.lima/default/qemu.pid]","time":"2021-06-22T21:38:31+01:00"} {"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-06-22T21:38:31+01:00"} {"level":"debug","msg":"executing script \"ssh\"","time":"2021-06-22T21:38:31+01:00"} {"level":"debug","msg":"stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 60022: Connection refused\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 60022: Connection refused\\r\\n\": exit status 255","time":"2021-06-22T21:38:31+01:00"} {"error":"exit status 1","level":"info","msg":"QEMU has exited","time":"2021-06-22T21:38:31+01:00"} exit status 1
I got the same issue with my Mac on ARM.
Lima version: 0.6.1
from lima.
Yay, thank you for sharing my note here @AkihiroSuda!
One issue I've faced during my Lima party was something related to bind mount (which is not on the gist note now).
I tried to run lima nerdctl run -d -v $(pwd):/usr/share/nginx/html -p 127.0.0.1:8080:80 nginx:alpine
but it just got stuck after successfully pulled the container image, without any errors.
Either way I'll create an issue in the nerdctl
repository with the repro.
from lima.
it just got stuck
I had the same thing happen to me (on Intel), and I thought it was because the host directories are mounted read-only into the VM, and then the app wanted to write to it from inside the container. Got distracted with something else; will test it again tomorrow.
from lima.
I thought it was because the host directories are mounted read-only
This doesn't seem to be causing the issue (although it would/should have caused the command to fail with an error).
I've now mounted the home directory as writable, and the container still hangs. It is supposed to copy a single file out of the container image to the mounted volume:
$ docker run -v /Users/jan/tmp/screen/bin:/mnt screen-static-coreos:latest cp -v /home/core/screen-4.8.0/screen /mnt
from lima.
Hi all,
Thanks for the great work on lima!
May I know if anyone tried intel-on-arm on M1 Mac?
I followed the gist to install the patched(HVF) qemu 6.0 and the default lima instance worked like a charm.
While for x86_64
, it's stuck. The log is saying on SSH but I guess it's just not booting(serial log is empty)?
Should I use x86_64 brew-ed qemu instead, or?
Thanks!
$ limactl start x
x
? Creating an instance "x" Open an editor to override the configuration
INFO[0204] Downloading "https://github.com/containerd/nerdctl/releases/download/v0.11.1/nerdctl-full-0.11.1-linux-amd64.tar.gz" (sha256:ce7a6e119b03c3fb8ded3d46d929962fd17417bea1d5bbc07e0fce49494d8a09)
INFO[0204] Using cache "/Users/weyl/Library/Caches/lima/download/by-url-sha256/3304d173f1e1824e5be6cf84bf2f78825cf0db416c4c975dbb2458776942629e/data"
INFO[0205] Attempting to download the image from "~/Downloads/hirsute-server-cloudimg-amd64.img"
INFO[0205] Attempting to download the image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-amd64.img"
INFO[0205] Using cache "/Users/weyl/Library/Caches/lima/download/by-url-sha256/e1fed960ebd29619676c7ab7535bc83f7fb2ad71739edb6fde4e17bce0b61a47/data"
INFO[0205] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/weyl/.lima/x/serial.log")
INFO[0205] SSH Local Port: 62022
INFO[0205] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0293] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0380] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0467] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0553] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0640] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0727] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
FATA[0805] did not receive an event with the "running" status
$ tail -f *.log
==> ha.stderr.log <==
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EDX.spec-ctrl [bit 26]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.fma [bit 12]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.x2apic [bit 21]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.tsc-deadline [bit 24]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.avx [bit 28]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.f16c [bit 29]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.avx2 [bit 5]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.invpcid [bit 10]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EDX.spec-ctrl [bit 26]","time":"2021-09-06T16:27:19+08:00"}
==> ha.stdout.log <==
{"time":"2021-09-06T16:27:19.070538+08:00","status":{"sshLocalPort":62022}}
==> serial.log <==
==> ha.stderr.log <==
{"level":"debug","msg":"stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\": exit status 255","time":"2021-09-06T16:28:36+08:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-09-06T16:28:46+08:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2021-09-06T16:28:46+08:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\": exit status 255","time":"2021-09-06T16:30:03+08:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-09-06T16:30:13+08:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2021-09-06T16:30:13+08:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\": exit status 255","time":"2021-09-06T16:31:30+08:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-09-06T16:31:40+08:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2021-09-06T16:31:40+08:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\": exit status 255","time":"2021-09-06T16:32:57+08:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-09-06T16:33:07+08:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2021-09-06T16:33:07+08:00"}
$ qemu-system-x86_64 -accel help
Accelerators supported in QEMU binary:
tcg
from lima.
While for
x86_64
, it's stuck. The log is saying on SSH but I guess it's just not booting(serial log is empty)?
I don't know; how long did you wait?
I once tried running the ARM version on an Intel Mac, and thought it didn't work, but I just didn't wait long enough; it took 30min to boot the VM.
Maybe I've done something wrong, but I think emulating the whole OS with a software-level emulator is just too slow. We can run just containers from a different architecture using qemu-user mode together with binfmt_misc
. See also containerd/nerdctl#280
from lima.
Thank you @jandubois!
I come to know software emulation of the VM level should be unusably slow and costly. I am waiting for nerdctl
to support --platform
:-D and before that, will give a try on bintfmt
with ctr run --platform
!
And it was 13 min waiting by the time I collected the log, maybe to wait longer will have the OS booted up.
Thank you so much! You rock!!!
from lima.
Updating installation steps for ARM (brew install simnalamburt/x/qemu-hvf
)
Can anyone confirm whether this works with Lima?
from lima.
Updating installation steps for ARM (
brew install simnalamburt/x/qemu-hvf
)Can anyone confirm whether this works with Lima?
Verified it's working, replied in #215
from lima.
Just reporting I've been working with this for the past two days with great success.
After creating a vm, I run
lima sh -c 'sudo apt update && sudo apt install -y qemu-user-static binfmt-support'
to also support amd64 containers.
Thanks for this tip! I've added this as a provision script in lima config:
provision:
- mode: system
script: |
#!/bin/bash
set -eux -o pipefail
export DEBIAN_FRONTEND=noninteractive
apt update
apt-get install -y qemu-user-static binfmt-support
from lima.
I'm planning to purchase an ARM Mac probably after they release the 2021 model of MacBookPro.
@AkihiroSuda Can highly recommend Scaleway for testing on Apple Silicon (Mac Mini) in the cloud...
from lima.
Happy to contribute to testing on M1, I'm on Monterey beta 10/MBA M1 16GB. FWIW default image works (ubuntu) but fedora does not nor centos when I created a centos.yaml (pretty much the same as the fedora.yaml but using centos cloud image URLs and sha). It looks like the initial difficulty is Waiting for the essential requirement 1 of 4: "ssh"
and also sshfs. I will open an issue with details later when I have more time to test.
from lima.
Thanks for testing, ARM edition of Fedora is now supported in the master #329
Will be included in v0.7.2 (probably next week)
from lima.
The fedora/centos issues were related to legacyBIOS: true
, the ubuntu/debian should use UEFI by default and work better...
Since CentOS (CentOS Linux) is going end-of-life soon (2021-12-31), it probably needs replacing with "Stream" (CentOS Stream).
from lima.
i have tried on my mac M1. it seems the data in shared media is always lost when stopped and restarted on the vm container and I have problems with the network. vm can't connect to the internet like doing a clone repository action on github.com
from lima.
Smooth sailing so far on M1 Pro, though that's not surprising because I wouldn't expect silicon-level breaking changes.
Cool project!
from lima.
It's been working pretty well for me however I'm receiving an error when I stop a container and try to run it again on the same port:
$ lima nerdctl run -p 8080:80 nginxdemos/hello:plain-text
.. running and http://localhost:8080 returns results as expected ..
ctrl+c to quit
.. i see the container quit but if i go to http://localhost:8080 the connection hangs ..
$ lima nerdctl run -p 8080:80 nginxdemos/hello:plain-text
FATA[0000] failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: time="2021-11-02T17:26:06Z" level=fatal msg="conflict with ID 1"
Failed to write to log, write /home/myname.linux/.local/share/nerdctl/1935db59/containers/default/b3af05e40ede34fd3e6eada2537704900fc221c17384ba8af5c7b9e842f02ff2/oci-hook.createRuntime.log: file already closed: unknown
If I then try a third time the container starts but the port still hangs. If I stop and start with limactl it works again.
UPDATE: I see same behavior on intel macbook, so looks like it is not an ARM specific issue
from lima.
Just a question: Why do we have to rely on qemu? It's a wonderful concoction, but not something that anybody would ever want to rely on. It often fails, and performance is awful. And both binaries and docker images can be built for both platforms.
from lima.
performance is awful
When using the accelerators, such as kvm/hvf/whpx, the cpu performance should be rather acceptable.
Running in emulation is slow, but it also opens up possibilities - like running "the other" architecture, etc etc
from lima.
Docker Desktop on Mac M1 provides qemu. It makes things happen that could never happen otherwise. But the mysterious failures and performance issues make it something to take a pass on wherever possible. For "other platforms" it's also possible to build native images and native binaries. DDEV and many other tools have native binaries and docker images.
from lima.
Please move any further discussion of the merits of qemu
to a discussion item; it is off-topic in respect to "Test on ARM macOS".
from lima.
Sorry, moved the conversation about architecture-native binaries and images to #406
from lima.
I could offer some testing (currently Air M1, soon probably another M1 Pro). But I'm unsure what tests I can contribute.
I did some very simple cpubench
tests here: https://levelup.gitconnected.com/running-amd64-linux-on-apple-m1-with-qemu-utm-64d67cccd6f8?source=friends_link&sk=35ca87f9537f73bf1e3ed74eb04e0abb
Today I tried running a docker-compose
setup fully with lima. It did work quite nicely, but I also got some compose errors, especially when using -f
multiple times and files basing on each other - likely nothing specific to arm.
from lima.
Hi, trying to get working on an M1 Mac - Big Sur 11.4.
I've been stuck at this for a while now:
? Creating an instance "default" Proceed with the default configuration
INFO[0007] Attempting to download the image from "~/Downloads/hirsute-server-cloudimg-arm64.img" digest=
INFO[0007] Attempting to download the image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-arm64.img" digest=
535.75 MiB / 535.75 MiB [-----------------------------------] 100.00% 1.08 MiB/s
INFO[0505] Downloaded image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-arm64.img"
INFO[0506] Attempting to download the nerdctl archive from "https://github.com/containerd/nerdctl/releases/download/v0.13.0/nerdctl-full-0.13.0-linux-arm64.tar.gz" digest="sha256:8a744638b01cd53c437a0d2a54c66f0ec4a918b75e93056393e7076bbf06a173"
INFO[0506] Using cache "/Users/viveksubramanian/Library/Caches/lima/download/by-url-sha256/b8e4d6c61f4294e7e6808bbb6c9d63e6515065b53f377046e0a42a4b4731e5cc/data"
INFO[0507] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/viveksubramanian/.lima/default/serial.log")
INFO[0507] SSH Local Port: 60022
INFO[0507] [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
INFO[0517] [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
INFO[0525] [hostagent] The essential requirement 1 of 5 is satisfied
INFO[0525] [hostagent] Waiting for the essential requirement 2 of 5: "user session is ready for ssh"
INFO[0525] [hostagent] The essential requirement 2 of 5 is satisfied
INFO[0525] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0565] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0605] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0645] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0685] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0725] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0766] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0806] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0846] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0886] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0926] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0966] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[1007] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[1047] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[1087] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
FATA[1106] did not receive an event with the "running" status
Is there something that I am doing wrong? I installed lima via brew install lima
from lima.
Is there something that I am doing wrong?
There might be more clues in that "serial.log" file
from lima.
Is there something that I am doing wrong? I installed lima via
brew install lima
I'm not sure if this applies to your situation, but in my case this error was due to my local HTTP proxy configuration in macOS. After enabling video output to view the logs on init, I noticed that when installing sshfs
the guest was trying to access the proxy on 127.0.0.1
rather than the address of my host. I was able to resolve this by setting the http_proxy
and https_proxy
environment variables to my local proxy. By default, Lima will detect this and replace localhost
and 127.0.0.1
with the host address.
After this the default ubuntu image works flawlessly on the M1 for me.
from lima.
Sure, closing this issue.
If somebody is facing a problem, please open a new issue.
from lima.
Related Issues (20)
- Provide standard values "default" and "none" for most options HOT 1
- Cross-compiler for building "darwin" target HOT 5
- Failed to start VM when I want to modify PATH env var from env section in yaml template HOT 1
- Error: msg="did not receive an event with the \"running\" status" HOT 2
- Lima VM will randomly lose network connectivity HOT 5
- Call `limadriver.Validate` as part of `limayaml.Validate` HOT 2
- v0.23.0 planning HOT 4
- Inconsistent {{.Home}} and {{.User}} values between guest and host templates HOT 2
- Bound ports are not released immediately when container is stopped/deleted HOT 11
- Figure out why archlinux CI is broken HOT 2
- nerdctl start fails 2 of 3 times with bridge errors HOT 2
- Dependabot wasn't able to update downloading
- cross-vm network connectivity issue when using socket_vmnet
- Lima fails to install on fedora 40 using homebrew HOT 1
- docker build using colima with lima v0.23.1 hangs HOT 10
- guest agent events closed unexpectedly error="rpc error: code = Unavailable desc = error reading from server: EOF" HOT 7
- v0.23.2 planning HOT 1
- Replace al.essio.dev/pkg/shellescape HOT 1
- The ls-lint is missing from "make lint", but is only run in github CI HOT 1
- Host agent process running in the foreground even when not using --foreground
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lima.