clayfreeman/gpu-passthrough

A GPU passthrough tutorial using libvirt and KVM on GNU/Linux

gpu-passthrough's Issues

Why I encounter the rom file I/O error when exporting the gpu rom file?

When I export the rom file as:

root@myComputer:/home/clock/Work/temp# cat /sys/bus/pci/devices/0000\:"${PCIE_BDF_IDENT}"/rom > \
>     /usr/share/qemu/gpu-"$(date +%Y%m%dT%H%m%S)".rom
cat: '/sys/bus/pci/devices/0000:01:00.0/rom': Input/output error

all the previous step work correctly, except this one. Thanks in advance.

network cards are not visible/accessible after enabling passthrough (intel_iommu=on)

Hey,

I'm trying to execute the tutorial of the gpu-passthrough, sadly i get stuck already at step 1)
After I enable the gpu-passthrough (kernel option intel-iommu=on) and reboot the network interfaces that where configured before are not accessible anymore.

Is this normal behavior?
From what I did read here or on the RHV doc it doesn't sound to happen in this way.
(RHV: https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/html/setting_up_an_nvidia_gpu_for_a_virtual_machine_in_red_hat_virtualization/proc_nvidia_gpu_passthrough_nvidia_gpu_passthrough#Detaching_the_GPU_device_from_the_host_nvidia_gpu_passthrough )

I did also try to add the option "iommu=pt", no difference here.
Do I need to enable the unsafe interupt support or is this totally not related?

Currently I'm using the next harwarde:

Supermicro 4029GP-TRT 5U Server
2 x Intel xeon Gold 5217
12 x Certified 16GB DDR4 2666MHZ ECC REG
1 x Samsung 480GB PM883
1 x Nvidia RTX 2080Ti OEM card

Any tips?

Already many thanks!

Br, Nico

Where can i find the script to load my vGPU mdev configuration on boot ?

Rebooting the host removes hostdev configuration, How do i automate this using modules ?

error: failed to read '/sys/bus/mdev/devices/301b7a7e-6704-4393-aa72-0255ccf3fe80/mdev_type/device_api': No such file or directory

Clarify or eliminate the need for ROM, console, CPU model, hv_*

Thank you for documenting. Confirming this works without libvirt and with AMD Ryzen as well, I would like extend this. Before I write it; a few things surprised me and may improve my additions if explained:

0. Whats the rationale on choosing q35 vs i440fx? I just used q35 because docs say its "modern".
1. Did you need an actual console for the hypervisor? I did not. Network or serial access sufficed when starting qemu with -monitor unix:/run/qemu/1.sock
2. Using just one GPU, I had to manually unbind vtconsole and efi-framebuffer via /sys. Did you avoid this by configuring your mainboard UEFI to initialize your non-passthrough GPU first? Or does libvirt implement this?
3. Did you need the ROM backup? I did not. I have tested the same ZOTAC 1060 Mini rev a1.
4. Did you get EFI boot menu to work? I did not. I have tested OVMF 20180205.
5. You say it is important, but did you notice any practical difference for the CPU model choice?
6. Did any of the Hyper-V enlightenments do anything useful? I failed to measure any effect.

when comand line "nvidia-smi“ in linux terminal，no devices were found in gpu-passthrough for GPU T4

hello,I make gpu-passthroug with GPU Tesla T4 on kvm。host device is redhat,but virtual machine is centos.
when comand line "nvidia-smi“ in linux terminal，no devices were found,as follow;

then vim /var/log/message，find as follows:
![image](https://user-images.githubusercontent.com/42164855/92296743-36039e00-ef6a-11ea-91e1-23d03c4c9d44.png,
can you give some suggestion thank you

WSL2 support?

Hello, great tutorial!
Did you by any chance validate if your system supports wsl2 ?
I've been trying to get it running on my virtual win10 with gpu passtrough, but I don't seem to be able to enable hyper-v.
I would love to be able to use the new docker client in windows with wsl2, but just can't seem to get my head around it.
The kvm settings are correct. I can run virtualbox, even tried ubuntu livecd with kvm enabled and run ubuntu in a vm in the live environment. I just don't seem to find a way to make hyper-v and my GPU work.
As soon as hyper-v get enabled during boottime, my RX580 gets the Error 43 code again and on top of that, hyper-v isn't even working, so no wsl2 either

TroubleShooting with the Tesla M40 24GB

just a note in the future when using a GPU with a larger than normal VRAM size such as my experience with the NVIDIA TESLA M40 24GB which has a 24GB Vram size, QEMU will give you an error code 12 in the windows operating system, this can be fixed by following the libvirt documentation for adding launch arguments for the VM QEMU command-line passthrough, also make sure to enable "Above 4G decoding / Resizable Bar" in your systems bare metal bios

ive also had issues with the script for dumping the GPU Rom for it as it doesn't have /rom accessible to the file system I had better luck using a rom already dumped by someone on the techpowerup you can also dump it yourself using NVFlash also provided by techpowerup, the GPU has no video output as its a datacenter card but the rom is still required or else you will get a code 10 error.

Command that worked for me by Laszlo Ersek

<domain
 type='kvm'
 xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <qemu:commandline>
    <qemu:arg value='-fw_cfg'/>
    <qemu:arg value='opt/ovmf/X-PciMmio64Mb,string=65536'/>
  </qemu:commandline>
</domain>

Some feedback on this tutorial

First off, I want to say that this is a wonderful and really helpful tutorial that does a great job of making sure the reader doesn't break anything. Second off, because you encourage feedback, I'll say that I feel like the tutorial is missing a major section. Right at the beginning of the "Preparing the Virtual Machine" section, it says:

Now that the hypervisor is prepared for GPU passthrough, we need to configure the Windows® 10 virtual machine

But, if the reader has been following the tutorial word-for-word, they don't have a Windows® 10 virtual machine to configure! At no point does the tutorial instruct the reader to install/set up a virtual machine - it skips right from setting up the hypervisor to configuring the virtual machine.
I am intentionally being obtuse here - I know that the tutorial intends for the reader to set up a VM using libvirt and KVM, but I think there are a lot of people are reading this tutorial who might not know how to use libvirt/KVM, so it's important to explicitly instruct the reader on how to do this in a way that will ensure there's no conflicts with the rest of the tutorial.

BTW, I think you can do this hypervisor/VM setup with only one graphics device. I've never tried it, but I've read that you can run the hypervisor in "headless" mode, with no graphics, and just let the VM take the single graphics device for itself. I'm not sure how advisable that is, but you do say "The hypervisor will need a way to display the console" which as far as I can tell is not strictly true.

Also, a minor nitpick: under the "Preparing the Hypervisor" section, it says:

Assuming that you use GRUB as your boot loader

I think you should add a line saying what to do if you're not using GRUB.

Update tutorial to accommodate the inclusion of VFIO as a statically-linked driver

Since Ubuntu 20.04 (or possibly earlier), vfio-pci is now statically compiled with the kernel, allowing it to preempt other modules by default.

As a result, this tutorial should recommend placing hardware identifiers in GRUB_CMDLINE_LINUX_DEFAULT instead of a configuration file in /etc/modprobe.d/* and make the soft module dependencies conditional on this premise.