rcore-os/virtio-drivers

VirtIO guest drivers in Rust.

VirtIO-drivers-rs

VirtIO guest drivers in Rust. For no_std environment.

Support status

Device types

Device	Supported
Block	✅
Net	✅
GPU	✅
Input	✅
Console	✅
Socket	✅
Sound	✅
...	❌

Transports

Transport	Supported
Legacy MMIO	✅	version 1
MMIO	✅	version 2
PCI	✅	Memory-mapped CAM only, e.g. aarch64 or PCIe ECAM

Device-independent features

Feature flag	Supported
VIRTIO_F_INDIRECT_DESC	✅	Indirect descriptors
VIRTIO_F_EVENT_IDX	✅	avail_event and used_event fields
VIRTIO_F_VERSION_1	TODO	VirtIO version 1 compliance
VIRTIO_F_ACCESS_PLATFORM	❌	Limited device access to memory
VIRTIO_F_RING_PACKED	❌	Packed virtqueue layout
VIRTIO_F_IN_ORDER	❌	Optimisations for in-order buffer usage
VIRTIO_F_ORDER_PLATFORM	❌	Platform ordering for memory access
VIRTIO_F_SR_IOV	❌	Single root I/O virtualization
VIRTIO_F_NOTIFICATION_DATA	❌	Extra data in device notifications

Examples & Tests

x86_64

cd examples/x86_64
make qemu

aarch64

cd examples/aarch64
make qemu

RISCV

cd examples/riscv
make qemu

You will see device info & GUI Window in qemu.

virtio-drivers's Issues

The examples/riscv has a stuck bug

Running examples/riscv will get stuck.

Like below:

Publish to crates.io

I would like to use this crate in a project I'm working on, and so would like to be able to get a stable version from crates.io. I'm happy to publish it myself, but let me know if you'd prefer to.

`VirtioVsockHdr` and `rx_queue_buffers` should be allocated from DMA range.

All data which will be access by device directly should be allocated from DMA range. But in vsock.rs, VirtioVsockHdr is allocated in stack and rx_queue_buffers is allocated in heap.

  pub fn connect(&mut self, connection_info: &ConnectionInfo) -> Result {
      let header = VirtioVsockHdr {
          op: VirtioVsockOp::Request.into(),
          ..connection_info.new_header(self.guest_cid)
      };
      // Sends a header only packet to the TX queue to connect the device to the listening socket
      // at the given destination.
      self.send_packet_to_tx_queue(&header, &[])
  }

      // Allocate and add buffers for the RX queue.
      let mut rx_queue_buffers = [null_mut(); QUEUE_SIZE];
      for (i, rx_queue_buffer) in rx_queue_buffers.iter_mut().enumerate() {
          let mut buffer: Box<[u8; RX_BUFFER_SIZE]> = FromZeroes::new_box_zeroed();
          // Safe because the buffer lives as long as the queue, as specified in the function
          // safety requirement, and we don't access it until it is popped.
          let token = unsafe { rx.add(&[], &mut [buffer.as_mut_slice()]) }?;
          assert_eq!(i, token.into());
          *rx_queue_buffer = Box::into_raw(buffer);
      }
      let rx_queue_buffers = rx_queue_buffers.map(|ptr| NonNull::new(ptr).unwrap());

It's unsafe and not reasonable to share the stack range or heap range to device.

Audit and fix endianness issues

I suspect that this crate currently assumes little-endian byte order in a bunch of places, and will break on big-endian architectures. We should go through and make the endianness explicit so that it will work on either. (I plan on doing so, this is just to keep track of it.)

Virtio GPU: Redrawing with timer interrupt on RISCV

Hi, first of all, thanks for the great bindings.

I am having problems redrawing the frame buffer periodically using a timer. The first succeeds, but all further draw calls fail with an IO Error. Maybe I do not understand the procedure correctly...

pub struct VirtGPU {
    gpu: VirtIOGpu<HalImpl, MmioTransport>,
}

pub static VGPU: OnceLock<Spin<VirtGPU>> = OnceLock::new();

impl VirtGPU {
    pub fn new(transport: MmioTransport) -> Self {
        Self {
            gpu: VirtIOGpu::new(transport).expect("initialize gpu driver"),
        }
    }

    pub fn draw(&mut self) -> virtio_drivers::Result<()> {
        let (width, height) = self.gpu.resolution()?;
        let width = width as usize;
        let height = height as usize;
        serial!("GPU resolution is {width}x{height}");
        let fb = self.gpu.setup_framebuffer()?; // <- IO Error happens here
        let fb = unsafe {
            core::slice::from_raw_parts_mut(fb.as_ptr() as *mut u32, fb.len() / size_of::<u32>())
        };
        serial!("Framebuffer size is {} ({})", fb.len(), width * height);

        /// Update the framebuffer...

        self.gpu.flush()?;
        Ok(())
    }
}

The MmioTransport is initialized as in the RISCV example.
A timer interrupt calls draw periodically. The first draw call succeeds; all subsequent ones fail in setup_framebuffer.

QEMU command line:

qemu-system-riscv32 -m 2G -machine virt -device virtio-gpu-device -serial mon:stdio -gdb tcp::1234 -no-shutdown -no-reboot target/debug/kernel

is there any means to unload virtio driver?

In the current version (as of 0.5.0), I noticed that this library does not provide a way to unload the virtio driver. I need to unload the driver in order to reload or use another driver in a given situation.

While the library provides the ability to load the driver, it lacks a way to unload it. I think adding support for unloading drivers would make the library more complete and flexible.

I would like to be able to dynamically load and unload virtio drivers within an application to allow flexibility to adjust as needed. Currently, the only way to uninstall a driver seems to be to restart the application or the OS（this way seem to be too rude）.

Please consider adding a function or method to uninstall the virtio driver to allow developers to uninstall the driver at runtime. This would add functionality and flexibility to the library and match the functionality of the loaded driver.

Thanks for your interest and contribution!

Additional information:

Library version: 0.5.0
Operating system: rcore-os，(yeah I'am following this os tutorial)

能通过 example/riscv 下的测试，但没有显示 GPU 那张红色的图

我尝试了用第 0 章提供的 docker 镜像和本机跑了一下，结果都是一样的

Expose writability of memory regions to Hal interface

It is necessary for some implementations of the Hal interface to know whether the driver or the device should be allowed to write to a memory region that is being allocated, so that it can map it appropriately in the page tables and IOMMUs or equivalent. For example the driver area and descriptor queue should be mapped read-write for the driver but read-only to the device, and the device area vice-versa.

This could be done by adding a parameter to the Hal::dma_alloc method. However, VirtQueue currently allocates all three regions together, as the legacy interface requires them to be laid out contiguously. This is awkward. Options I can see are:

1. Remove support for the legacy interface. Then the three areas could be allocated separately, with different permissions. This would also reduce the size of each individual allocation which might make things easier for the allocator.
2. Have dma_alloc take two parameters, something like driver_pages and device_pages, and be expected to allocate a contiguous region of the given total number of pages, but with different permissions for the two subranges. This is an awkward API, but would at least allow us to maintain all current functionality.

Thoughts? Is anyone particularly attached to the legacy MMIO interface? Could we just remove it entirely?

run example and got error

I change target from riscv64imac-unknown-none-elf to riscv64gc-unknown-none-elf （seems newer）, and use rustc 1.51-nightly, qemu-5.0
Compiling OK.
But when make qemu, I got runtime error:

"Guest has not initialized the display (yet)."

How to fix it?

The shared buffer address is not in the dma allocation address range.

I implemented Hal trait for VirtioHal and use it to init VirtIONet, However, the address of the input parameter buf of the share interface is not within the range of the previous dma allocated address. I am not sure whether this is the correct behavior.

[dma_alloc] paddr: 0x84256000, vaddr: 0xd6000
[dma_alloc] paddr: 0x85250000, vaddr: 0x10d0000
[share] vaddr: 0xd480

virtio-drivers = { git = "https://github.com/rcore-os/virtio-drivers", rev = "a35c6e6" }

compilation error due to the use of spin_loop

When I run the code in ch7 without any modification, I ran into a compilation error. I think '#![feature(renamed_spin_loop)]' should be added to lib.rs.
My rust version is rustc 1.46.0-nightly (50fc24d8a 2020-06-25)

VirtIOInput stops accepting input after certain number of events

After 30 keyboard events were sent, the VirtIO Input (keyboard in my test case) stops working.
I do ack the interupt and I do pop the pending event.

Problems with asynchronous interfaces

考虑我们有这样的结构，它主要实现了这样的功能：

1. 发送读请求，将 waker 保存，让出 CPU。
2. 中断到来，根据 token 取出 waker，唤醒 Future。
3. 完成读取。

这套方案存在两个问题：

1. 在非理想情况下，Future 可能被其他事件取消（信号/进程终止等），此时后续的 complete_read_blocks 不会被执行，因此需要驱动支持直接取消一个 token，而不需要完成数据复制工作。（不能创建一个新的 buffer 放到全局区，把 complete 操作移到 handle_irq 处，因为这会导致额外的复制开销。）
2. peek_used 方法只会取出一个待处理的 token（而不会消费），而在这个 token 被处理前无法得知其他 token 是否完成，从而将对应任务唤醒。这会导致其他的 Future 因为后续无法收到中断信号而永远被挂起（假设中断发生时所有任务都已经就绪）。

这样的问题在目前的异步接口下很难解决。

struct ReqGuard<'a>(&'a VirtIOBlkDevice, u16);

impl Drop for ReqGuard<'_> {
    fn drop(&mut self) {
        let mut inner = self.0.inner.lock();
        inner.tokens.remove(&self.1);
        inner.block.// How: cancel_request(self.1);
    }
}

async fn read_block(&self, block_id: usize, buf: &mut [u8]) -> SyscallResult {
    let mut req = BlkReq::default();
    let mut resp = BlkResp::default();
    let token = loop {
        let token = unsafe {
            self.inner.lock().block.read_blocks_nb(block_id, &mut req, buf, &mut resp)
        };
        // Error handling...
    };
    let guard = ReqGuard(self, token);
    poll_fn(|cx| {
        self.inner.lock().tokens.insert(token, cx.waker().clone());
        Poll::<()>::Pending
    }).await;
    core::mem::forget(guard);
    let res = unsafe {
        self.inner.lock().block.complete_read_blocks(token, &mut req, buf, &mut resp)
    };
    // Error handling...
}

fn handle_irq(&self) {
    let mut inner = self.inner.lock();
    if let Some(token) = inner.block.peek_used() {
        if let Some(waker) = inner.tokens.remove(&token) {
            waker.wake();
        }
    }
}

`VirtIOBlk::read_blocks` returns `IoError` if `buf` is >= 2GiB

I'm using the PciTransport on a Rust unikernel. 2047MiB copies work without issue, but any greater and I get an IoError.

how to configure interrupt for virtio-blk / virtio-console on riscv-qemu?

Hello, I have a question. I'm trying to configure the virtio console driver on riscv qemu. I found that the fdt has an interrupt number, but I'm not sure how to enable the interrupt of this device in the virtio drivers library. Could you please ask me how to do it?

Some question about virtio-net

Hello, I am using virtio-net to implement an interrupt-controlled network driver. However, I found that interrupts cannot be triggered unless the recv method of VirtIONet is called. Do you think VirtIONet should also provide a non-blocking method like VirtIOBlk?

impl<H: Hal> VirtIOBlk<'_, H> {
  pub unsafe fn read_block_nb(
          &mut self,
          block_id: usize,
          buf: &mut [u8],
          resp: &mut BlkResp,
      ) -> Result<u16> {
        let token = self.queue.add(&[req.as_buf()], &[buf, resp.as_buf_mut()])?;
        self.header.notify(0);
    }
}

I guess the header.notify in the above code is what makes the interrupt mechanism work properly, right?

Support VirtIO 1.2 `VIRTIO_F_RING_RESET`

If VIRTIO_F_RING_RESET is negotiated then PCI transport queues can be reset with queue_reset.

getting a IoError but no clue what it means or how to fix

https://github.com/walksanatora/oc2kernel/blob/master/src/main.rs#L119-L136
is the virtio code
but yet it always returns with a IO error
you can see my qemu args in run.sh

Unable to build with feature `tcp` enabled

Hi guys, I'm trying to run the x86_64 example on a Proxmox QEMU KVM.

If I set the tcp feature to on, the compilation fails as reported below.

Any idea of the possible cause?

Thanks in advance for your availability.

编码建议

清华大学这么牛逼组织，能不能持续迭代一些基础系统的包？ 操作系统以后的趋势，就是由无数个包组织。 长期坚持必然能立于世界之巅。
不要很狭隘的放到rcore下面去，目标不是一个操作系统，而是一个可以生产操作系统的工具集合。 我们要的是天下， 不是一城一池。
工程里面不要有一些第几章这种字样，要极为严谨。

Support memory buffers from noncontagious physical pages

In VirtQueue::add(Here we take outputs as an example):

for output in outputs.iter() {
    let desc = &mut self.desc[self.free_head as usize];
    desc.set_buf(output);
    desc.flags.write(DescFlags::NEXT | DescFlags::WRITE);
    last = self.free_head;
    self.free_head = desc.next.read();
}

For every output memory buffer, we only add one Descriptor to the Descriptor Table now. This is because we assume that the given memory buffer resides on contagious physical pages, which means that it is required that the current VA->PA mapping is somehow "smooth" near the position of the memory buffer in the virtual memory space. If this condition is not satisfied, the second part of the output will be written to a wrong physical page.

One possible solution is that we add a Descriptor for each contagious physical memory region where the given memory buffer resides in.

By the way, the BlkReq we allocate in Blk::VirtIOBlk::read/write_block(_nb) can also be across 2 physical pages depending on that current page table, which is more difficult to think about.

Network driver issues on multicore systems

Hello, I'd like to ask if this package supports network drivers on multicore systems. When I create multiple TCP connections using the send interface of virtio-net on a multicore machine, the captured packets in tcpdump are incomplete and disordered.

let responese = "response from server";
// write a response
syscall::write!(client_fd, responese.as_bytes());

Additionally, it works fine if there is only a single TCP connection.

a bug in blk::read/write_block_nb

In these methods, we allocate a BlkReq instance on the stackframe. The memory block which the BlkReq instance resides in will be reused after the methods return, and there is a possibility that the virtio device is also accessing it at the same time, leading to a data race. Mention that the blocking version of there methods do not have this problem since when the methods returns and the BlkReq is deallocated, it can be guaranteed that the virtio device has completed handling the request before.

I have no idea how to fix it yet.

Async support?

Would it be possible to implement an async version of the driver?

`write_volatile` on aarch64 may not properly set ESR

With the new version of LLVM in rustc 1.78.0, pointer::write_volatile can sometimes get optimised into something like str w9, [x0], #4. This writes to the MMIO memory as expected, but suppresses the syndrome register (e.g. ESR_EL1) update, which at least in the case of KVM with crosvm prevents the VMM from servicing the fault as it relies on the ESR for this.

This could arguably be considered a bug in rustc as write_volatile is explicitly intended for MMIO, but we may want to consider working around this in the meantime by using inline assembly for volatile memory access on aarch64.

2022-08-27最新版本可能存在问题

根据 rCore-Tutorial-Book 进行实验环境的配置

http://rcore-os.cn/rCore-Tutorial-Book-v3/chapter0/5setup-devel-env.html

不论是在 Linux 系统上还是在 wsl 中或是 docker中，在最后一步 make run 时都会出现这样的 error

error[E0107]: this struct takes 1 generic argument but 0 generic arguments were supplied
  --> src/drivers/block/virtio_blk.rs:18:32
   |
18 |     virtio_blk: UPIntrFreeCell<VirtIOBlk<'static>>,
   |                                ^^^^^^^^^ expected 1 generic argument
   |

error[E0107]: this struct takes 1 generic argument but 0 generic arguments were supplied
  --> src/drivers/gpu/mod.rs:19:25
   |
19 |     gpu: UPIntrFreeCell<VirtIOGpu<'static>>,
   |                         ^^^^^^^^^ expected 1 generic argument
   |

error[E0107]: this struct takes 1 generic argument but 0 generic arguments were supplied
  --> src/drivers/input/mod.rs:21:35
   |
21 | struct VirtIOINPUT(UPIntrFreeCell<VirtIOInput<'static>>);
   |                                   ^^^^^^^^^^^ expected 1 generic argument
   |

但是如果我用一位 GitHub 用户在第29次 commit 时 fork 的版本，可以成功通过编译
rCore-Tutorial-v3/os/Cargo.toml中修改

...
virtio-drivers = { git = "a forked version" }
...

我觉得可能是最新的几次 commit 的代码中出现了一些问题导致的