danielfullmer/nixos-nvidia-vgpu

NixOS NVIDIA vGPU Module

nixos-nvidia-vgpu's Issues

Licensing server?

I was wondering if you had a derivation for the licensing server too necessary to use vGPU functionality.

Support recent kernels (>= 5.10)

Thanks for packaging this up! I have been interested in trying this since I first heard about the hack.

Unfortunately, I came across some compilation errors, possibly due to kernel version mismatch? I ​wasn't able to immediately figure it out, so I thought I would share here

I'm building via a NixOS flake build of nixos-unstable:

build/kernel/nvidia/nv-vgpu-vmbus.c: In function 'nv_vmbus_open':
/build/kernel/nvidia/nv-vgpu-vmbus.c:911:55: error: 'struct vmbus_channel' has no member named 'target_vp'; did you mean 'target_cpu'?
  911 |     open_msg.target_vp = nv_hyperv_ctx.guest_channel->target_vp;
      |                                                       ^~~~~~~~~
      |                                                       target_cpu
/build/kernel/nvidia/nv-vgpu-vmbus.c: In function 'nv_vmbus_setup':
/build/kernel/nvidia/nv-vgpu-vmbus.c:1272:68: error: 'PAGE_KERNEL_RX' undeclared (first use in this function); did you mean 'PAGE_KERNEL_ROX'?
 1272 |     nv_hyperv_ctx.nv_hypercall_pg = vmap(&page, num_pages, VM_MAP, PAGE_KERNEL_RX);
      |                                                                    ^~~~~~~~~~~~~~
      |                                                                    PAGE_KERNEL_ROX
/build/kernel/nvidia/nv-vgpu-vmbus.c:1272:68: note: each undeclared identifier is reported only once for each function it appears in
/build/kernel/nvidia/nv-caps.c: In function 'nv_cap_close_fd':
/build/kernel/nvidia/nv-caps.c:592:5: error: implicit declaration of function 'sys_close' [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.>
  592 |     sys_close(fd);
      |     ^~~~~~~~~

The failures are all undefined symbols. The sys_close failure especially made me think of a kernel version mismatch. I later found this discussion of what looks like the same problem.

This may also be influenced by other Nvidia-related NixOS configuration options. I'm happy to share my settings for those here as well, if useful.

Full build log: 1f09h5bw0sq4ndwwnsj51c69cr2sik3l-nvidia-x11-460.73.01-5.11.14.drv.log

Stopping the nvidia-vgpu-mgr service fails

The ExecStopPost phase fails because the -r argument to rm is missing at https://github.com/danielfullmer/nixos-nvidia-vgpu/blob/master/default.nix#L123 which causes this error:

Apr 23 12:58:25 luna rm[2131770]: /nix/store/485sqh570qixsv9wk0d59vrvs35pqja0-coreutils-8.32/bin/rm: cannot remove '/var/run/nvidia-vgpu-mgr': Is a directory

Buid fails in nixOS 22.11 with Kernel 5.4.231

Was trying to make this work on my nixOS machine, changed my kernel to 5.4 to make sure the environment was as close as possible to the tested one:

  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
      (builtins.fetchTarball "https://github.com/danielfullmer/nixos-nvidia-vgpu/archive/master.tar.gz")
    ];

  boot.kernelPackages = pkgs.linuxPackages_5_4; # needed for this
  hardware.nvidia.vgpu.enable = true; # Enable NVIDIA KVM vGPU + GRID driver
  hardware.nvidia.vgpu.unlock.enable = true; # Unlock vGPU functionality on consumer cards using DualCoder/vgpu_unlock project.

Added the files NVIDIA-Linux-x86_64-460.32.04-vgpu-kvm.run and NVIDIA-Linux-x86_64-460.32.03-grid.run as the first error logs suggested with the command nix-store --add-fixed sha256 <filename>. But it seems to not be working in nixOS 22.11.
I'm using flakes, but also tried in a VM without flakes and the error is the same

these are the error logs:

...
For full logs, run 'nix log /nix/store/qal4yqqph4hipk086jp744pmp72v4my7-nvidia-x11-460.32.04-460.32.03-5.4.231.drv'
...

> nix log /nix/store/qal4yqqph4hipk086jp744pmp72v4my7-nvidia-x11-460.32.04-460.32.03-5.4.231.drv
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
Creating directory NVIDIA-Linux-x86_64-460.32.03-grid
Verifying archive integrity... OK
Uncompressing NVIDIA Accelerated Graphics Driver for Linux-x86_64 460.32.03............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
source root is NVIDIA-Linux-x86_64-460.32.03-grid
cp: cannot create regular file 'kernel/common/inc//nv-vgpu-vfio-interface.h': No such file or directory

Could this be a problem related to the nix version?

(Also the link in "Additional Notes" in the README goes to a "PAGE NOT FOUND" error)

system info:

> nix-shell -p neofetch --run neofetch
          ▗▄▄▄       ▗▄▄▄▄    ▄▄▄▖            yeshey@nixos 
          ▜███▙       ▜███▙  ▟███▛            ------------ 
           ▜███▙       ▜███▙▟███▛             OS: NixOS 22.11.20230219.0cf4274 (Raccoon) x86_64 
            ▜███▙       ▜██████▛              Host: LENOVO LNVNB161216 
     ▟█████████████████▙ ▜████▛     ▟▙        Kernel: 5.4.231 
    ▟███████████████████▙ ▜███▙    ▟██▙       Uptime: 1 hour, 24 mins 
           ▄▄▄▄▖           ▜███▙  ▟███▛       Packages: 1885 (nix-system), 1917 (nix-user), 25 (flatpak) 
          ▟███▛             ▜██▛ ▟███▛        Shell: bash 5.2.15 
         ▟███▛               ▜▛ ▟███▛         Resolution: 1920x1080, 1920x1080 
▟███████████▛                  ▟██████████▙   DE: Plasma 
▜██████████▛                  ▟███████████▛   WM: KWin 
      ▟███▛ ▟▙               ▟███▛            Icons: breeze-dark [GTK2/3] 
     ▟███▛ ▟██▙             ▟███▛             Terminal: .konsole-wrappe 
    ▟███▛  ▜███▙           ▝▀▀▀▀              CPU: Intel i7-9750H (12) @ 4.500GHz 
    ▜██▛    ▜███▙ ▜██████████████████▛        GPU: NVIDIA GeForce RTX 2060 Mobile 
     ▜▛     ▟████▙ ▜████████████████▛         Memory: 14204MiB / 15924MiB 
           ▟██████▙       ▜███▙
          ▟███▛▜███▙       ▜███▙                                      
         ▟███▛  ▜███▙       ▜███▙                                     
         ▝▀▀▀    ▀▀▀▀▘       ▀▀▀▘

> nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.231, NixOS, 22.11 (Raccoon), 22.11.20230219.0cf4274`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.1`
 - channels(root): `"nixos"`
 - channels(yeshey): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Referals

@danielfullmer

mdevctl doesn't depend on jq

I installed this on my system and didn't have jq installed. Upon executing mdevctl I got:

/run/current-system/sw/bin/mdevctl: line 97: jq: command not found