s-u/macosvm

Tool for running macOS guest virtual machines in macOS 12 host or higher on M1 arm64 Macs

macosvm's Issues

Question: route traffic through guest vm

Is it possible to route traffic to specific subnet from host to guest vm? Let's say, we have a VPN connection in guess vm and would like to access certain vpn-subnet from host.

Question: The VZ limit number of VM vs. VM shutdown

Hi, first let me thank for this brilliant tool!

Sometimes, I'm experiencing troubles that after several ephemeral VM startups and shutdowns the next startup ends with the The maximum supported number of active virtual machines has been reached. error.

At that point none of the macosvm or com.apple.Virtualization.VirtualMachine processes are running.

I've tried both terminating the macosvm process and sudo shutdown -h now from within the VM.

I cannot confidently reproduce this problem, it happens quite randomly and the only solution seems to reboot the host machine.

Is there a way of better VM management, maybe a list of running VMs, or somehow refresh the VZ framework so it doesn't indicate the limit?

Or more reliable option to shutdown the VM?

Any idea would be much appreciated :)

Issues building from master branch

Hit this issue when trying to build from master branch (revision f044ad8).

Thanks!

Using xcodebuild:

pmoore@Peters-MacBook-Pro:~/git/macosvm master $ xcodebuild 
2022-08-05 05:35:30.350 xcodebuild[90375:7223177] Requested but did not find extension point with identifier Xcode.IDEKit.ExtensionSentinelHostApplications for extension Xcode.DebuggerFoundation.AppExtensionHosts.watchOS of plug-in com.apple.dt.IDEWatchSupportCore
2022-08-05 05:35:30.350 xcodebuild[90375:7223177] Requested but did not find extension point with identifier Xcode.IDEKit.ExtensionPointIdentifierToBundleIdentifier for extension Xcode.DebuggerFoundation.AppExtensionToBundleIdentifierMap.watchOS of plug-in com.apple.dt.IDEWatchSupportCore
Command line invocation:
    /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild

User defaults from command line:
    IDEPackageSupportUseBuiltinSCM = YES

note: Using new build system
note: Planning
Analyze workspace

Create build description
Build description signature: a37167f2ce8cd2707ce805f47b4e7d39
Build description path: /Users/pmoore/git/macosvm/build/XCBuildData/a37167f2ce8cd2707ce805f47b4e7d39-desc.xcbuild

note: Build preparation complete
note: Building targets in dependency order
CreateBuildDirectory /Users/pmoore/git/macosvm/build
    cd /Users/pmoore/git/macosvm/macosvm.xcodeproj
    builtin-create-build-directory /Users/pmoore/git/macosvm/build

CreateBuildDirectory /Users/pmoore/git/macosvm/build/Release
    cd /Users/pmoore/git/macosvm/macosvm.xcodeproj
    builtin-create-build-directory /Users/pmoore/git/macosvm/build/Release

WriteAuxiliaryFile /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/DerivedSources/Entitlements.plist (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    write-file /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/DerivedSources/Entitlements.plist

ProcessProductPackaging /Users/pmoore/git/macosvm/macosvm/macosvm.entitlements /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm.xcent (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    
    Entitlements:
    
    {
    "com.apple.security.app-sandbox" = 0;
    "com.apple.security.device.usb" = 1;
    "com.apple.security.get-task-allow" = 1;
    "com.apple.security.network.client" = 1;
    "com.apple.security.network.server" = 1;
    "com.apple.security.virtualization" = 1;
}
    
    builtin-productPackagingUtility /Users/pmoore/git/macosvm/macosvm/macosvm.entitlements -entitlements -format xml -o /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm.xcent

WriteAuxiliaryFile /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm.hmap (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    write-file /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm.hmap

WriteAuxiliaryFile /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-project-headers.hmap (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    write-file /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-project-headers.hmap

WriteAuxiliaryFile /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-all-target-headers.hmap (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    write-file /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-all-target-headers.hmap

WriteAuxiliaryFile /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-own-target-headers.hmap (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    write-file /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-own-target-headers.hmap

WriteAuxiliaryFile /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/all-product-headers.yaml (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    write-file /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/all-product-headers.yaml

WriteAuxiliaryFile /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-generated-files.hmap (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    write-file /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-generated-files.hmap

WriteAuxiliaryFile /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-all-non-framework-target-headers.hmap (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    write-file /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-all-non-framework-target-headers.hmap

WriteAuxiliaryFile /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/macosvm.LinkFileList (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    write-file /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/macosvm.LinkFileList

CompileC /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/main.o /Users/pmoore/git/macosvm/macosvm/main.m normal arm64 objective-c com.apple.compilers.llvm.clang.1_0.compiler (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    export LANG\=en_US.US-ASCII
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x objective-c -target arm64-apple-macos12.0 -fmessage-length\=245 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit\=0 -fcolor-diagnostics -std\=gnu11 -fobjc-arc -fobjc-weak -fmodules -gmodules -fmodules-prune-interval\=86400 -fmodules-prune-after\=345600 -fbuild-session-file\=/var/folders/kc/p5r32hgd42xcwxhc9lyfxtwm0000gn/C/org.llvm.clang/ModuleCache.noindex/Session.modulevalidation -fmodules-validate-once-per-build-session -Wnon-modular-include-in-framework-module -Werror\=non-modular-include-in-framework-module -Wno-trigraphs -fpascal-strings -Os -fno-common -Wno-missing-field-initializers -Wno-missing-prototypes -Werror\=return-type -Wdocumentation -Wunreachable-code -Wquoted-include-in-framework-header -Wno-implicit-atomic-properties -Werror\=deprecated-objc-isa-usage -Wno-objc-interface-ivars -Werror\=objc-root-class -Wno-arc-repeated-use-of-weak -Wimplicit-retain-self -Wduplicate-method-match -Wno-missing-braces -Wparentheses -Wswitch -Wunused-function -Wno-unused-label -Wno-unused-parameter -Wunused-variable -Wunused-value -Wempty-body -Wuninitialized -Wconditional-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wconstant-conversion -Wint-conversion -Wbool-conversion -Wenum-conversion -Wno-float-conversion -Wnon-literal-null-conversion -Wobjc-literal-conversion -Wshorten-64-to-32 -Wpointer-sign -Wno-newline-eof -Wno-selector -Wno-strict-selector-match -Wundeclared-selector -Wdeprecated-implementations -DNS_BLOCK_ASSERTIONS\=1 -DOBJC_OLD_DISPATCH_PROTOTYPES\=0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -fstrict-aliasing -Wprotocol -Wdeprecated-declarations -g -fvisibility\=hidden -Wno-sign-conversion -Winfinite-recursion -Wcomma -Wblock-capture-autoreleasing -Wstrict-prototypes -Wno-semicolon-before-method-body -Wunguarded-availability -iquote /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-generated-files.hmap -I/Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-own-target-headers.hmap -I/Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-all-target-headers.hmap -iquote /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-project-headers.hmap -I/Users/pmoore/git/macosvm/build/Release/include -I/Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/DerivedSources-normal/arm64 -I/Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/DerivedSources/arm64 -I/Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/DerivedSources -F/Users/pmoore/git/macosvm/build/Release -MMD -MT dependencies -MF /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/main.d --serialize-diagnostics /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/main.dia -c /Users/pmoore/git/macosvm/macosvm/main.m -o /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/main.o
In file included from /Users/pmoore/git/macosvm/macosvm/main.m:3:
/Users/pmoore/git/macosvm/macosvm/VMInstance.h:31:40: warning: pointer is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified) [-Wnullability-completeness]
@property (strong) VZMacOSRestoreImage *restoreImage;
                                       ^
/Users/pmoore/git/macosvm/macosvm/VMInstance.h:31:40: note: insert '_Nullable' if the pointer may be null
@property (strong) VZMacOSRestoreImage *restoreImage;
                                       ^
                                         _Nullable 
/Users/pmoore/git/macosvm/macosvm/VMInstance.h:31:40: note: insert '_Nonnull' if the pointer should never be null
@property (strong) VZMacOSRestoreImage *restoreImage;
                                       ^
                                         _Nonnull 
/Users/pmoore/git/macosvm/macosvm/main.m:94:7: warning: extra tokens at end of #endif directive [-Wextra-tokens]
#endif;
      ^
      //
/Users/pmoore/git/macosvm/macosvm/main.m:93:19: error: expected ';' after expression
    fflush(stdout)
                  ^
                  ;
2 warnings and 1 error generated.

CompileC /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/VMInstance.o /Users/pmoore/git/macosvm/macosvm/VMInstance.m normal arm64 objective-c com.apple.compilers.llvm.clang.1_0.compiler (in target 'macosvm' from project 'macosvm')
    cd /Users/pmoore/git/macosvm
    export LANG\=en_US.US-ASCII
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x objective-c -target arm64-apple-macos12.0 -fmessage-length\=245 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit\=0 -fcolor-diagnostics -std\=gnu11 -fobjc-arc -fobjc-weak -fmodules -gmodules -fmodules-prune-interval\=86400 -fmodules-prune-after\=345600 -fbuild-session-file\=/var/folders/kc/p5r32hgd42xcwxhc9lyfxtwm0000gn/C/org.llvm.clang/ModuleCache.noindex/Session.modulevalidation -fmodules-validate-once-per-build-session -Wnon-modular-include-in-framework-module -Werror\=non-modular-include-in-framework-module -Wno-trigraphs -fpascal-strings -Os -fno-common -Wno-missing-field-initializers -Wno-missing-prototypes -Werror\=return-type -Wdocumentation -Wunreachable-code -Wquoted-include-in-framework-header -Wno-implicit-atomic-properties -Werror\=deprecated-objc-isa-usage -Wno-objc-interface-ivars -Werror\=objc-root-class -Wno-arc-repeated-use-of-weak -Wimplicit-retain-self -Wduplicate-method-match -Wno-missing-braces -Wparentheses -Wswitch -Wunused-function -Wno-unused-label -Wno-unused-parameter -Wunused-variable -Wunused-value -Wempty-body -Wuninitialized -Wconditional-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wconstant-conversion -Wint-conversion -Wbool-conversion -Wenum-conversion -Wno-float-conversion -Wnon-literal-null-conversion -Wobjc-literal-conversion -Wshorten-64-to-32 -Wpointer-sign -Wno-newline-eof -Wno-selector -Wno-strict-selector-match -Wundeclared-selector -Wdeprecated-implementations -DNS_BLOCK_ASSERTIONS\=1 -DOBJC_OLD_DISPATCH_PROTOTYPES\=0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -fstrict-aliasing -Wprotocol -Wdeprecated-declarations -g -fvisibility\=hidden -Wno-sign-conversion -Winfinite-recursion -Wcomma -Wblock-capture-autoreleasing -Wstrict-prototypes -Wno-semicolon-before-method-body -Wunguarded-availability -iquote /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-generated-files.hmap -I/Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-own-target-headers.hmap -I/Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-all-target-headers.hmap -iquote /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/macosvm-project-headers.hmap -I/Users/pmoore/git/macosvm/build/Release/include -I/Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/DerivedSources-normal/arm64 -I/Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/DerivedSources/arm64 -I/Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/DerivedSources -F/Users/pmoore/git/macosvm/build/Release -MMD -MT dependencies -MF /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/VMInstance.d --serialize-diagnostics /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/VMInstance.dia -c /Users/pmoore/git/macosvm/macosvm/VMInstance.m -o /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/VMInstance.o
In file included from /Users/pmoore/git/macosvm/macosvm/VMInstance.m:1:
/Users/pmoore/git/macosvm/macosvm/VMInstance.h:31:40: warning: pointer is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified) [-Wnullability-completeness]
@property (strong) VZMacOSRestoreImage *restoreImage;
                                       ^
/Users/pmoore/git/macosvm/macosvm/VMInstance.h:31:40: note: insert '_Nullable' if the pointer may be null
@property (strong) VZMacOSRestoreImage *restoreImage;
                                       ^
                                         _Nullable 
/Users/pmoore/git/macosvm/macosvm/VMInstance.h:31:40: note: insert '_Nonnull' if the pointer should never be null
@property (strong) VZMacOSRestoreImage *restoreImage;
                                       ^
                                         _Nonnull 
/Users/pmoore/git/macosvm/macosvm/VMInstance.m:326:20: warning: unused variable 'slavefd' [-Wunused-variable]
            int masterfd, slavefd;
                          ^
2 warnings generated.

** BUILD FAILED **


The following build commands failed:
	CompileC /Users/pmoore/git/macosvm/build/macosvm.build/Release/macosvm.build/Objects-normal/arm64/main.o /Users/pmoore/git/macosvm/macosvm/main.m normal arm64 objective-c com.apple.compilers.llvm.clang.1_0.compiler (in target 'macosvm' from project 'macosvm')
(1 failure)

Using make:

pmoore@Peters-MacBook-Pro:~/git/macosvm master $ make
make -C macosvm macosvm
clang -I/opt/homebrew/opt/ruby/include -Wall -c main.m
In file included from main.m:3:
./VMInstance.h:31:40: warning: pointer is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified) [-Wnullability-completeness]
@property (strong) VZMacOSRestoreImage *restoreImage;
                                       ^
./VMInstance.h:31:40: note: insert '_Nullable' if the pointer may be null
@property (strong) VZMacOSRestoreImage *restoreImage;
                                       ^
                                         _Nullable 
./VMInstance.h:31:40: note: insert '_Nonnull' if the pointer should never be null
@property (strong) VZMacOSRestoreImage *restoreImage;
                                       ^
                                         _Nonnull 
main.m:94:7: warning: extra tokens at end of #endif directive [-Wextra-tokens]
#endif;
      ^
      //
main.m:93:19: error: expected ';' after expression
    fflush(stdout)
                  ^
                  ;
2 warnings and 1 error generated.
make[1]: *** [main.o] Error 1
make: *** [all] Error 2
pmoore@Peters-MacBook-Pro:~/git/macosvm master $ 

Feature request: container registry interface

Do you take feature requests?

If I knew Swift, I might contribute these myself, but I don't, yet.

github.com/cirruslabs/tart/ is very much like this but has a much more polished command line interface. it's not interactive or anything, but it's more mature. That would be a "nice to have."

One thing I would really, really like, though, is Tart's ability to upload and download VM images from a container registry. If that is something that's easy to do, I'd like to request that. If it's not, then never mind, and you can close this issue without response.

I ask because tart is soon moving to a horrendous license that I can't use.

I don't mean to make demands of any open source maintainer, so if these feel like things you must do, don't do them, please! I do not want to be a cause of or contribute to anyone's increasing stress level. This is a request. A "no" from you is equally as valid as a "yes." Like I said, I'm new to Macs and Swift and I intend on taking some of Apple's starter code from WWDC 2022 and expanding it as I learn the language.

Thank you, and have a wonderful [insert time period]!

new Sonoma features

Features added in macOS 14 (Sonoma) to the VZ framework that may be of interest and could be made use of:

• new storage devices:
  • raw (hardware) disk support
  • NBD - network block device protocol
• save/restore VM state

Request for clipboard support

First of all a very big "THANK YOU" for creating this wrapper to virtualization framework. In fact, this wrapper has good number of features (including changing of screen resolution, storage size, CPU, memory etc...) that I was not able to find in other commercial applications.

When you get a chance, please look at the possibility of sharing the clipboard from the host mac os to guest mac os.

Thank you so much in advance,
Guru R.

Serial Port Access from Host

Thank you very much for the great application. I have seen that it has support for serial adapters in the code for Linux Guests. I have now released this code for macOS as well. In macOS I see /dev/cu.virtio device which is great. How can I access the serial adapter of the VM from the host OS (/dev/tty?)? Thanks a lot!

The bridge network does not work even when sip is off

Thanks a lot for the great application.
I have a problem when I try to use the bridge network instead of nat.

I must define the interface name manually because when I use a blank parameter then I get an error with listing available interfaces, but this is not a problem to define the interface manually.

If I used bridge with a manually set interface name then I got an error with missing entitlement com.apple.vm.networking even when the sip is off

Do you have any suggestions on how to solve this problem?

This problem is occurred on these version of systems:

• 12.3
• 12.4

This is output from console

./macosvm --disk /Volumes/MacVMs4/VMS/OSX12-1/osx12-1-bridge-disk.img,size=64g --aux /Volumes/MacVMs4/VMS/OSX12-1/osx12-1-bridge-aux.img --restore /Volumes/MacVMs4/UniversalMac_12.4_21F79_Restore.ipsw --net bridge:"en0" /Volumes/MacVMs4/VMS/OSX12-1/osx12-1-bridge.json
INFO: creating new disk image /Volumes/MacVMs4/VMS/OSX12-1/osx12-1-bridge-disk.img with size 68719476736 bytes
INFO: add storage '/Volumes/MacVMs4/VMS/OSX12-1/osx12-1-bridge-disk.img' type 'disk' read-write 
INFO: add storage '/Volumes/MacVMs4/VMS/OSX12-1/osx12-1-bridge-aux.img' type 'aux' read-write 
INFO: restore from /Volumes/MacVMs4/UniversalMac_12.4_21F79_Restore.ipsw
INFO: add bridged network on interface en0
2022-06-21 10:45:30.595 macosvm[911:17777] Restoring from /Volumes/MacVMs4/UniversalMac_12.4_21F79_Restore.ipsw
2022-06-21 10:45:30.689 macosvm[911:17777]   image load: OK
2022-06-21 10:45:30.689 macosvm[911:17777] Creating instance ...
2022-06-21 10:45:30.689 macosvm[911:17777] <VMSpec: 0x600002394000> - configure for restore, OS: macos
2022-06-21 10:45:30.689 macosvm[911:17777] configure with restore, minimum requirements: 2 CPUs, 4294967296 RAM
2022-06-21 10:45:30.693 macosvm[911:17777]  + Bridged network to Ethernet (en0)
2022-06-21 10:45:30.693 macosvm[911:17777]  + display: 2560 x 1600 @ 200
2022-06-21 10:45:30.693 macosvm[911:17777]  + disk image file:///Volumes/MacVMs4/VMS/OSX12-1/osx12-1-bridge-disk.img (read-write)
2022-06-21 10:45:30.693 macosvm[911:17777]  + new aux storage /Volumes/MacVMs4/VMS/OSX12-1/osx12-1-bridge-aux.img
2022-06-21 10:45:30.693 macosvm[911:17777]  + 2 CPUs
2022-06-21 10:45:30.693 macosvm[911:17777]  + 4294967296 RAM
2022-06-21 10:45:30.693 macosvm[911:17777] validateWithError = Error Domain=VZErrorDomain Code=2 "Using VZBridgedNetworkDeviceAttachment in a process that lacks the “com.apple.vm.networking” entitlement." UserInfo={NSLocalizedFailure=Invalid virtual machine configuration., NSLocalizedFailureReason=Using VZBridgedNetworkDeviceAttachment in a process that lacks the “com.apple.vm.networking” entitlement.}
2022-06-21 10:45:30.693 macosvm[911:17777] Exception in VM init: Error Domain=VZErrorDomain Code=2 "Using VZBridgedNetworkDeviceAttachment in a process that lacks the “com.apple.vm.networking” entitlement." UserInfo={NSLocalizedFailure=Invalid virtual machine configuration., NSLocalizedFailureReason=Using VZBridgedNetworkDeviceAttachment in a process that lacks the “com.apple.vm.networking” entitlement.}

File paths in vm.json are saved exactly as given during creation, rather than relative to vm.json

One thing I noticed while trying to make lots of VMs and copying VMs around is that the vm.json files end up having paths saved into it that are saved exactly as given when I created the VM.

This is fine if I never change anything about how and where I execute my VM, but this makes it complicated to change folder names or move execute around or make extra copies of a VM. Depending on the style of path specified, I end up having to modify the vm.json to account for other changes in the environment.

Ideally, I should be able to specify a single vm.json and have it work regardless of the path of where I moved the VM around to or where I'm executing from.

The issue

Relative paths

Relative paths function relative to the working directory used when invoking macosvm

For example, if I make a VM with relative paths like this:

cd ~/Desktop
macosvm -g --disk ./macos_13.6/disk.img,size=100g --aux ./macos_13.6/aux.img -c 4 -r 8g --restore ~/Desktop/Installers/UniversalMac_13.6_22G120_Restore.ipsw ./macos_13.6/vm.json

All of the files for the resulting VM will be saved under ~/Desktop/macos_13.6.

And inside the ~/Desktop/macos_13.6/vm.json it has saved the same ./macos_13.6/disk.img and ./macos_13.6/aux.img that were specified on the commandline.

Changing working directory:

Because the relative working directory structure is saved into the json, nothing will work if I change where I'm executing from:

cd ~
~/Desktop/macosvm --ephemeral -g ~/Desktop/macos_13.6/vm.json
2023-11-14 15:34:30.221 macosvm[21423:1426906]  . cloning ./macos_13.6/disk.img to ephemeral ./macos_13.6/disk.img-clone-21423
2023-11-14 15:34:30.222 macosvm[21423:1426906] *** Terminating app due to uncaught exception 'FSClone', reason: 'Failed to clone './macos_13.6/disk.img' to './macos_13.6/disk.img-clone-21423': [errno=2] No such file or directory'
*** First throw call stack:
(
	0   CoreFoundation                      0x000000019b4b70e4 __exceptionPreprocess + 176
	1   libobjc.A.dylib                     0x000000019afd5fd0 objc_exception_throw + 60
	2   macosvm                             0x0000000104a0215c -[VMSpec cloneAllStorage] + 1316
	3   macosvm                             0x00000001049fe83c main + 9080
	4   dyld                                0x000000019b007f28 start + 2236
)
libc++abi: terminating due to uncaught exception of type NSException
Abort trap: 6

Changing folder names:

Because the relative working directory structure is saved into the json, nothing will work if I change the name of the VM's folder:

cd ~/Desktop
mv ~/Desktop/macos_13.6 ~/Desktop/macos_13.6_backup
~/Desktop/macosvm --ephemeral -g ~/Desktop/macos_13.6_backup/vm.json
2023-11-14 15:46:51.380 macosvm[21449:1429392]  . cloning ./macos_13.6/disk.img to ephemeral ./macos_13.6/disk.img-clone-21449
2023-11-14 15:46:51.380 macosvm[21449:1429392] *** Terminating app due to uncaught exception 'FSClone', reason: 'Failed to clone './macos_13.6/disk.img' to './macos_13.6/disk.img-clone-21449': [errno=2] No such file or directory'
*** First throw call stack:
(
	0   CoreFoundation                      0x000000019b4b70e4 __exceptionPreprocess + 176
	1   libobjc.A.dylib                     0x000000019afd5fd0 objc_exception_throw + 60
	2   macosvm                             0x0000000102dfa15c -[VMSpec cloneAllStorage] + 1316
	3   macosvm                             0x0000000102df683c main + 9080
	4   dyld                                0x000000019b007f28 start + 2236
)
libc++abi: terminating due to uncaught exception of type NSException
Abort trap: 6

Absolute paths

I can avoid the problems of execution directories if I use absolute paths, as then there's no relative path structure to worry about. But I still have to fix up the paths in the vm.json if any names change anywhere along the hierarchy.

Ideal:

In my ideal world, the vm.json could store paths to the disk.img and aux.img relative to itself. By doing so, I could freely copy a VM folder around, rename it, change the relative execution path of my machine, etc., without needing to make the vm.json match.

Feature Request: Support multiple automount volumes

The virtualization framework supports having multiple automounted volumes with different readonly vs readwrite statuses, as can be seen described in Apple's documentation for VZVirtioFileSystemDeviceConfiguration

But the current support for --vol only allows for one automounted volume. If two are specified, an error is thrown due to reusing the same VZVirtioFileSystemDeviceConfiguration.macOSGuestAutomountTag tag.

I see that there is a method already in macosvm called addAutomountDirectoryShares:readOnly: which does know how to share describe multiple automounted directories, but it forces them all to have the same value for readOnly, and the code is currently unreachable.

Motivating example:

I want to start my VM with two automounted directories, one that is readonly so that the guest VM can't ruin it, and one that has readwrite capabilities so that the guest VM can write some data back to the host.

~/Desktop/macosvm -g \
--vol ~/Desktop/sharedWithVM/fromHost,ro,automount \
--vol ~/Desktop/sharedWithVM/fromGuest,automount \
macos_14.1/vm.json

Potential implementation:

I've got a working implementation of this in my fork:
aaronclarke:feature/multiple_automount

There I've hacked this together by making macosvm loop over the shares twice:

1. Loop one: Loop over all shares
   • if share is an automount share and automounting is available, add it to a list of automount shares and continue
   • otherwise do normal share specifications
2. Loop two:
   • Loop over all discovered automount shares. Batch each directory together into a single VZMultipleDirectoryShare and set that as the share for VZVirtioFileSystemDeviceConfiguration with the VZVirtioFileSystemDeviceConfiguration.macOSGuestAutomountTag tag.

If that seems acceptable I could send a pull request.

Possibility to run more than 2 VMs on same host

Is it possible to somehow overcome the limitations of the 2 VM per host machine limit and run more VMs on the same host machine?

When I try to run three times ./macosvm vm.json --ephemeral then I get an error

2022-08-12 09:04:06.328 macosvm[13625:684496] start completed err=Error Domain=VZErrorDomain Code=6 "The maximum supported number of active virtual machines has been reached." UserInfo={NSLocalizedFailure=No value exists in nvram for the provided key., NSLocalizedFailureReason=The maximum supported number of active virtual machines has been reached.}
2022-08-12 09:04:06.328 macosvm[13625:684496] *** Terminating app due to uncaught exception 'VMStartError', reason: 'Error Domain=VZErrorDomain Code=6 "The maximum supported number of active virtual machines has been reached." UserInfo={NSLocalizedFailure=No value exists in nvram for the provided key., NSLocalizedFailureReason=The maximum supported number of active virtual machines has been reached.}'

Help screen wrong

When running the macosvm -h it gives the following output:
Usage: ./macosvm [-g|--[no-]gui] [--[no-]audio]
[--restore <path>] [--ephemeral] [--recovery]
[--{disk|usb} <path>[,ro][,size=<spec>][,keep]] [--aux <path>]
[--vol <path>[,ro][,{name=<\name>|automount}]]
[--net <spec>] [--mac <addr>] [-c <cpu>] [-m <ram>]
[--no-serial] [--pty] <config.json>

which indicates that the command line switch to specify the amount of RAM is '-m'. This is wrong... The command line switch to specify the amount of RAM is actually '-r' ...

"--mac" parameter is not in the list of "multi_options", address gets interpreted as a json

I've been trying to use the --mac <addr> parameter and I think it's misconfigured:

~/Desktop/macosvm --version
macosvm 0.2-1

Copyright (C) 2022 Simon Urbanek
There is NO warranty.
Licenses: GPLv2 or GPLv3
Capabilites: macOS guest, vol, vol:automount, net:unix:mtu, usb

~/Desktop/macosvm -g --mac 8e:e0:62:8a:1d:e2 --vol ~/Desktop/sharedWithVM,automount ~/Desktop/macos_13.6/vm.json 
2023-10-20 15:39:35.903 macosvm[1374:8455] Cannot open '8e:e0:62:8a:1d:e2': Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory" UserInfo={_kCFStreamErrorCodeKey=2, _kCFStreamErrorDomainKey=1}
2023-10-20 15:39:35.904 macosvm[1374:8455] ERROR: Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory" UserInfo={_kCFStreamErrorCodeKey=2, _kCFStreamErrorDomainKey=1}

From digging around in the code, I believe the issue here is that --mac isn't included in the list of multi_options.

macosvm/macosvm/main.m

Lines 364 to 366 in f133574

	const char *multi_options[] = {
	"--restore", "--vol", "--disk", "--usb", "--aux", "--initrd", "--net", "--save", 0
	};

As a result the parameter to --mac gets read as an file path and assumed to be the path to configuration json, which can be seen if I move the arguments around:

~/Desktop/macosvm -g --vol ~/Desktop/sharedWithVM,automount ~/Desktop/macos_13.6/vm.json --mac 8e:e0:62:8a:1d:e2
ERROR: configuration path can only be specified once

Network connect issue

Can not connect to network by nat (ethernet), it shows following errors:

2022-08-10 14:17:57.073 macosvm[13593:402196] validateWithError = Error Domain=VZErrorDomain Code=2 "Using VZBridgedNetworkDeviceAttachment in a process that lacks the “com.apple.vm.networking” entitlement." UserInfo={NSLocalizedFailure=Invalid virtual machine configuration., NSLocalizedFailureReason=Using VZBridgedNetworkDeviceAttachment in a process that lacks the “com.apple.vm.networking” entitlement.}
2022-08-10 14:17:57.073 macosvm[13593:402196] Exception in VM init: Error Domain=VZErrorDomain Code=2 "Using VZBridgedNetworkDeviceAttachment in a process that lacks the “com.apple.vm.networking” entitlement." UserInfo={NSLocalizedFailure=Invalid virtual machine configuration., NSLocalizedFailureReason=Using VZBridgedNetworkDeviceAttachment in a process that lacks the “com.apple.vm.networking” entitlement.}

Possible to set or determine IP or MAC address for ephemeral instances?

I am using the --ephemeral flag when running for macosvm in a GitLab Runner custom executor. If 2 jobs start at the same time, is there a trick to mapping the process ID with the IP or MAC address that gets assigned to the instance?

Is it possible to specify a MAC address via config file or command line argument? (Then I could identify the IP using arp -a.)

Thanks for your work on this project!

Number of pointing devices is greater than the maximum number supported

% ../macosvm/macosvm -g linux.json
2022-12-24 20:41:54.450 macosvm[56871:2611793] Creating instance ...
2022-12-24 20:41:54.450 macosvm[56871:2611793] <VMSpec: 0x120f041a0> - configure for run, OS: linux
2022-12-24 20:41:54.450 macosvm[56871:2611793] + Linux kernel vmlinuz-5.10.70 -- file:///Projects/macosvm/linux/
2022-12-24 20:41:54.450 macosvm[56871:2611793] + kernel boot parameters: console=hvc0 root=/dev/vda1
2022-12-24 20:41:54.450 macosvm[56871:2611793] + inital RAM disk: initrd -- file:///Projects/macosvm/linux/
2022-12-24 20:41:54.450 macosvm[56871:2611793] + NAT network
2022-12-24 20:41:54.450 macosvm[56871:2611793] + network: ether 2a:04:4a:5d:0d:bf
2022-12-24 20:41:54.450 macosvm[56871:2611793] + display: 2560 x 1600 @ 200
2022-12-24 20:41:54.450 macosvm[56871:2611793] + disk image debian-rootfs-aarch64.img -- file:///Projects/macosvm/linux/ (read-write)
2022-12-24 20:41:54.451 macosvm[56871:2611793] + 2 CPUs
2022-12-24 20:41:54.451 macosvm[56871:2611793] + 4294967296 RAM
2022-12-24 20:41:54.453 macosvm[56871:2611793] validateWithError = Error Domain=VZErrorDomain Code=2 "Number of pointing devices is greater than the maximum number supported." UserInfo={NSLocalizedFailure=Invalid virtual machine configuration., NSLocalizedFailureReason=Number of pointing devices is greater than the maximum number supported.}
2022-12-24 20:41:54.453 macosvm[56871:2611793] Exception in VM init: Error Domain=VZErrorDomain Code=2 "Number of pointing devices is greater than the maximum number supported." UserInfo={NSLocalizedFailure=Invalid virtual machine configuration., NSLocalizedFailure
Reason=Number of pointing devices is greater than the maximum number supported.}

linux.json.zip