wenzel/libmicrovmi

A cross-platform unified Virtual Machine Introspection API library

libmicrovmi

A cross-platform unified Virtual Machine Introspection API library

Table of Contents

• Overview
• Project Status
• Getting Started
• Documentation
• Maintainers
• License

Overview

libmicrovmi aims to provide a cross-platform unified Virtual Machine Introspection API. (See What's VMI ?)

The term micro (μ) refers to the library's simplicity as well as the letter U standing for Unified interface.

Virtual Machine Introspection has been around since 2003, yet the ecosystem is still heavily fragmented and lacks standards as well as interoperability. (See VMI Ecosystem Fragmentation)

The main objective is to provide the simplest virtual machine introspection abstraction, offering a standard API to interact with any VMI provider, with a high degree of compatibility and composability to be integrated with any high-level VMI application.

Project Status

Below you can find a table describing the apps and drivers that can be used with libmicrovmi.

App	Driver
✅ MemProcFS / LeechCore ✅ volatility3 🟧 LibVMI 🔲 HyperDBG	🟧 Xen 🟧 KVM 🟧 VirtualBox ✅ memflow

API	Platform
✅ Rust ✅ C ✅ Python	✅ Linux ✅ Windows 🔲 MacOS

Legend:

• ✅: full support
• 🟧: partial support
• 🔲: TODO

Getting Started

The documentation is here to guide you, whether you are a user or developer.

User

• I would like to install libmicrovmi on my system
• I would like to know how to setup my VMI app with libmicrovmi
• I would like to know which drivers are available and how to initialize them

Developer

• I am developing a memory forensic / VM introspection app, and I want an API that supports multiple hypervisors at glance
• I want to add a new driver for libmicrovmi

Documentation

Our documentation is hosted online at

You can find it at doc/ as an mdbook 📖

To build the docs locally:

$ cargo install mdbook
$ mdbook build doc
$ xdg-open doc/book/index.html

Maintainers

• @Wenzel
• @rageagainsthepc

License

GNU General Public License v3.0

libmicrovmi's Issues

Physical write function should not require a mutable buffer

When doing a physical write the input byte buffer that is being written to the target is usually not mutated directly. It will just cause the memory of the target to be mutated. Currently, the libmicrovmi API exposes a physical write function that enforces mutability of the input byte buffer though.

    fn write_physical(&self, _paddr: u64, _buf: &mut [u8]) -> Result<(), Box<dyn Error>> {
        unimplemented!();
    }

Follow Rust API guidelines for libraries

Rust has an official guide to write llibraries and APIs:
https://rust-lang.github.io/api-guidelines/checklist.html

we should strive to follow it.

HyperDBG integration

HyperDbg debugger is an open-source, hypervisor-assisted user-mode, and kernel-mode Windows debugger with a focus on using modern hardware technologies. It is a debugger designed for analyzing, fuzzing and reversing.

How does it listen for VM events such as EPT pagefaults ?
What hypervisors are supported ?

Could HyperDBG make use of libmicrovmi ?

cc @SinaKarvandi here :)

API: replying should be implemented on an Event

It's more natural to call reply on an Event than on the driver:

// old
drv.reply_event(&ev, EventReplyType::Continue)
// new
ev.reply(EventReplyType::Continue)

• find a way to hide the API reply_event in the Introspectable trait
• expose a new API in impl Event, so users should call this API instead

related: https://github.com/Wenzel/libmicrovmi/pull/49/files

thoughts @kylerky, @rageagainsthepc ?

Driver initialization should be fallible

Each driver initialization through new should return a Result instead of panicking.

PR #152 is working toward this goal by refactoring error handling.

KVM: support to init via VSock

Support was added in the kvmi crate to init using a VSock socket:
Wenzel/kvmi#39

changes required in libmicrovmi:

• the DriverInitParam 's KVMiSocket field should use a SocketType`: https://github.com/Wenzel/kvmi/blob/master/src/lib.rs#L129
• the examples should be updated to take a new parameter to pass the vsock socket: https://github.com/Wenzel/libmicrovmi/blob/master/examples/mem-dump.rs#L19
• kvm driver should be updated

Logging

use log crate instead of println! macro

RUSTSEC-2020-0036: failure is officially deprecated/unmaintained

failure is officially deprecated/unmaintained

Details
Status	unmaintained
Package	failure
Version	0.1.8
URL	rust-lang-deprecated/failure#347
Date	2020-05-02

The failure crate is officially end-of-life: it has been marked as deprecated
by the former maintainer, who has announced that there will be no updates or
maintenance work on it going forward.

The following are some suggested actively developed alternatives to switch to:

• anyhow
• eyre
• fehler
• snafu
• thiserror

See advisory page for additional details.

CI: Release on Github

Xen driver not compatible with Ubuntu 16.04

When trying to setup multi distro testing under Github Actions, the Xen driver failed to compiled,
as it relied on xenforeignmemory, which is an API which appeared recently in Xen.

Add more compatibility for the Xen driver.
#45

API: write_physical shouldn't take a mutable buffer

libmicrovmi/src/api.rs

Line 204 in 5c15b94

fn write_physical(&self, _paddr: u64, _buf: &mut [u8]) -> Result<(), Box<dyn Error>> {

Volatility3 integration

Writing a new address space for volatility3, to run vol plugins directly on the VM physical memory.

From Ikelos:

Sure, so if you define an appropriate handler (using the python handlers system) similar to the JarHandler we have at https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/layers/resources.py#L202, then people will be able to use python vol.py --single-location vmi://whatever to run against the VMI image directly.

In volatility 3, we don't pass parameters, but instead every "Configurable" (some that needs information) provides requirements through the get_requirements method, which contains simple types like int, str, etc, to act as parameters.
You can see this at work for every plugin, but you can also see it in the intel layer here: https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/layers/intel.py#L231 which can then be accessed via self.config[<requirement name>]. Hopefully that makes a bit more sense? I'd probably go for the handler method, if you've already done something similar for rekall...

TODO:

• create libmicrovmi python bindings

Build system: opportunistic drivers

PR #151 attempts to have default drivers enabled.

The issue raised by this system is that it requires to have

• Xen headers
• libkvmi headers
• VirtualBox FDP headers

all installed and detected to compile, unless you explicitely disable a driver.

An improvment would be to do like libvmi's cmake build system, and have opportunistic features, where we try to build the crates on which the driver depend upon, and on failure we simply disable the feature and continue.

I don't know how to implement this with cargo, build.rs.

Next step would be to ask on Rust language forums.

Xen API changes in xenctrl

Xen breaks the API between 4.9.2 and 4.11.2

4.9.2: xc_monitor_enable(domid_t domid)
4.11.2: xc_monitor_enable(uint32_t domid)

How to deal with this in Bindgen ?

KVM: Handle read size > 4K

From #161 , Volatility3 is performing huge read operations, far beyong the usual 4K.
However, in KVM invalid_page_access returns true if the size requested is superior to the PAGE_SIZE, which is 4K.

This means that the libmicrovmi should split the read into 4K chunks.

Add cargo doc in Github Actions CI

Translate a virtual address

Another goal is to translate a virtual address to a physical address.

This implies parsing the page tables.
Now either the hypervisor supports this already and can offer us an API, or we have to implement this ourselves.

Check from existing crates like x86_64 from rust-osdev if they can share page tables structures definition, and even the translation functions.

Clippy integration

Integrate Clippy in Travis CI to catch errors early

KVM: disable interception of breakpoint events by default

While reviewing the code:
https://github.com/Wenzel/libmicrovmi/blob/master/src/driver/kvm.rs#L122

We shouldn't be intercepting INT3 by default, unless we are explicitely asked to do so.
if a debugger is running, the INT3 will be catched and put the in the queue, (and if the user never listens, never processed).

Libvmi implementation:
https://github.com/libvmi/libvmi/blob/master/libvmi/driver/kvm/kvm_events.c#L615

Github Actions: avoid dual build problem

At the moment we are triggering a dual build for internal pull requests (a PR originating from this repo, to master branch for example).

After a quick search, it's possible to avoid this. examples:

• decidim/decidim#5883
• microsoft/vscode#83158

I recommend to limit the push to master branch and let pull_request trigger on any branches.

First publication on crates.io

This issue tracks what needs to be done for a first publication on crates.io

• update Cargo.toml
• convert all git dependencies to crates.io equivalent

upload git dependencies to crates.io

• xenctrl
  • xenctrl-sys
• xenstore
  • xenstore-sys
• xenforeignmemory
  • xenforeignmemory-sys
• kvmi
  • kvmi-sys
• fdp
  • fdp-sys

Xen: Handle permission denied error while reading xenstore entries

According to #125 (comment)

When reading xenstore from a dedicated virtual machine, the access is more limited, and reading some entries may fail with a permission denied error.

In Libvmi, NULL entries are ignored.

We should implement the same behavior, handling explicitely the permission denied error, and continue the loop.

hypervisor detection

Libmicrovmi's api needs to explicitely specify the hypervisor that we want to use.

We could improve that behavior by detecting if a given hypervisor is present, and test all possible hypervisors (the ones compiled in the library).

As we are only compiling a specific hypervisor, there will be only 2 hypervisors available:

• Dummy
• the selected feature (xen or kvm at this point)

How can we modify the init function, so that it takes an optional driver type ?

pub fn init(driver_type: Option<DriverType>, domain_name: &String) -> Box<Introspectable> {

So we could pass None and let the function decide.

What do you think @tathanhdinh, how to implement this feature ?

Thanks.

Design arbitrary argument passing for driver initialization

The KVM driver needs more arguments than the domain name.
It also needs a socket, which is hardcoded in the KVM driver ATM:

https://github.com/Wenzel/libmicrovmi/blob/master/src/driver/kvm.rs#L32

Redesign the API to allow specifying a set of parameters, or passing a dictionary for example.

Release Debian package with cargo-deb

https://github.com/mmstick/cargo-deb

initial setup with example code results in a segmentation fault / xen on linux

Context

As discussed on twitter (@deiderajr) i receive a segmentation fault after using the mem-dump example code (same for other examples).

Process

This is our testing environment:

xlinfo.txt

Steps to reproduce the error:

1. Downloaded the repository from github (master branch)
2. cd to libmicrovmi/target/debug/examples
3. RUST_LOG=debug cargo run --features xen --example mem-dump

Retrieving data with libvmi works (see terminal_libvmi.png).

Expected result

Getting a correct .dump file

Current result

Segmentation fault (see terminal_libmicrovmi.png)

End

If i missed important information please let me know. Thanks!

API design: events interfaces and traits

Following #49 I'm opening this issue to start the discussion on how the event API should be implemented.

I'm reposting an idea from @kylerky, where each driver event (KVMEvent, XenEvent...) should implement a set of traits, and Event should contain generic type:

pub struct Event {
    common: EventCommon,
    extra: EventExtra,
}

pub enum EventExtra<CR>
    where CR: CREvent
{
    CR(CR),
    Pause,
}

pub trait CREvent {
    fn get_old() -> u64;
    fn get_new() -> u64;
}

pub struct KVMCrEvent {
    pub fn construct_reply(...) -> KVMCrReply {

    }
}

impl CREvent for KVMCrEvent {...}

struct KVMCrReply {...}
impl Reply for KVMCrReply {...}

fn main() {
    let kvm_cr_event = ...;
    let reply = kvm_cr_event.construct_reply(Continue, new);
    dom.send(reply);
}

fn send<T: Reply>(reply: T) { ... }

mem-dump.rs writes invalid memory dumps

Volatility was unable to read a requested page: Page error 0xffff81e13858 in layer primary (Page Fault at entry 0x0 in table page directory)

Strange, are you using Volatility3 ?
Maybe open an issue on their repo, they will get back to you.

The problem is that mem-dump.rs does not create a valid memory dump, in the sense that it simply skips non-accessible pages, and thus physical address and offset in dump file will not match. Volatility depends on both being the same.

Something like this results in a working dump file (as sparse file, so not wasting disk space). Tested on Linux only, I am not familiar with Windows, if it supports sparse file that way as well.

--- a/examples/mem-dump.rs
+++ b/examples/mem-dump.rs
@@ -1,6 +1,9 @@
 use std::fs::File;
 use std::io::Write;
 use std::path::Path;
+use std::io::SeekFrom;
+use std::io::Seek;
+use std::convert::TryInto;
 
 use clap::{App, Arg, ArgMatches};
 use env_logger;
@@ -56,7 +59,9 @@ fn main() {
                     .write_all(&buffer)
                     .expect("failed to write to file");
             }
-            Err(_error) => (),
+            Err(_error) => {
+               dump_file.seek(SeekFrom::Current(PAGE_SIZE.try_into().unwrap())).expect("failed to seek in file");
+           },
         }
     }

RUSTSEC-2020-0095: difference is unmaintained

difference is unmaintained

Details
Status	unmaintained
Package	difference
Version	2.0.0
URL	johannhof/difference.rs#45
Date	2020-12-20

The author of the difference crate is unresponsive.

Maintained alternatives:

• dissimilar

• treediff

• diffus

See advisory page for additional details.

Driver dynamic loading

Today libmicrovmi has 2 limitations:

• only one driver is compiled-in
• the resulting binary links with the driver libraries

We should produce a library that detects at runtime which libraries are available and initializes the compatible drivers.

However, we rely on bindgen, and this feature is not ready yet:
rust-lang/rust-bindgen#1541

libmicrovmi python bindings

Create Python bindings to call libmicrovmi from Python

TODO (updated):

• decide how to integrate this new library in our build system
• expose DriverType and DriverInitParam
• expose read_physical
• package and upload to PyPI
• update setup.py to accept --features parameter, and set features in RustExtension
• create new Dockerfile based on quay.io/pypa/manylinux2014_x86_64 and install all the drivers required development headers
• update build-wheels.sh to accept --features and pass them to setup.py
• read all relevant fields of python/Cargo.toml to fill setup.py metadata

We should use the rust-cpython project

rust-cpython will help us to generate a Python extension module, wrapping the libmicrovmi Rust API.

how to integrate this new library

1. create a separate repository
2. use cargo workspace to manage both Rust library and Python extension in the same git repository, in tandem.
3. maybe generate both with the same cargo.toml (I doubt this is possible)

@rageagainsthepc if you have an opinion

Hyper-V: filter by VM name

The Hyper-V driver currenty loops over all vmwp.exe and returns when he found a partition handle.

This means that the vm_name parameter is not checked.

We need to list runnings Hyper-V VMs, and match the vm_name parameter.

Move mem-dump as example

Cargo already knows how to deal with examples binaries with the examples folder.
Move it there.

Use log instead of println!

API: remove get_page_access

get_page_access is actually not implemented by hypervisor. the library is supposed to maintain the state of page whose permissions have been modified, and return this state, via this API.
So it's a higher-level than we expected.

Example in Libvmi:
https://github.com/libvmi/libvmi/blob/master/libvmi/events.c#L667

Enabling logging output from C

libmicrovmi is using the log crate to define its log messages.
These message are enabled by the env_logger, initialized at beginning of each example.

However, how can we initialize the env_logger with the C API ? 🤔

Even if I export RUST_LOG=debug now, I have no debug output when running a LibVMI example :/

CAPI: return boolean instead of custom Success/Failure values

Libvmi has been built with (too) strong compatibility in mind, and used a VMI_SUCCESS/FAILURE as a return type when it could have included stdbool.h and returned a boolean.

stdbool.h only exists since C99:
https://stackoverflow.com/a/1608350/3017219

I think we should not try to go lower than this standard.

KVM driver build fail with latest kvmi crate

https://github.com/Wenzel/libmicrovmi/pull/115/checks?check_run_id=1000813980#step:10:87

The tests are failing if we use kvmi master branch (or the one I uploaded on crates.io)
This should be a simple fix.

Publish on crates.io using Github Actions CI

A similar issue is opened on actions-rs repo:
actions-rs/meta#6

Let's see where it leads

TODO:

• use Github Actions to publish the crate on crates.io
• add a new crates.io badge in the README

Libvmi integration

This issue tracks the integration status of libmicrovmi with LibVMI

➡️ A fork of libvmi is available with a basic integration of libmicrovmi:
https://github.com/Wenzel/libvmi on the libmicrovmi branch

A pull request has been opened (only on the forked repo) to track the changes:
Wenzel/libvmi#1

A github actions has been configured to test the build:
https://github.com/Wenzel/libvmi/runs/1527128223?check_suite_focus=true

Registers: add support for IDT register

While trying to integrate libmicrovmi into LibVMI, I noticed that one of the KPCR search algorithm required the IDTR_BASE LibVMI register.

Even though there are fallback algorithms, they failed for unclear reasons.
So I decided to hardcode my IDTR_BASE value:

And vmi-process-list example works, on top of libmicrovmi 🎉

cc @rageagainsthepc
the current implementation (without IDTR_BASE hardcoded) is here:
https://github.com/Wenzel/libvmi/tree/libmicrovmi

Automatic reply from events on Drop

On feature that I would like on the API is when an Event is being Dropped from the scope, it would automatically send a default response, which would be EventReplyType::Continue.

tasks:

• add reference to Introspectable in the Event, so the event can call the driver
• impl Drop for Event
• add boolean in Event so that you can't reply twice to the same event

related: https://github.com/Wenzel/libmicrovmi/pull/49/files

thoughts @kylerky, @rageagainsthepc ?

Default implementation for Introspectable trait

Provide default implementation for Introspectable so that the driver doesn't have to define and implement everything to compile

Init function should be fallible

The init function should fail instead of panicking

KVM: read maximum physical address via v6 API

libkvmi v6 offers a method to get max physical address:
https://github.com/KVM-VMI/kvm/blob/kvmi/tools/kvm/kvmi/include/kvmi/libkvmi.h#L125

memflow integration

memflow is a library that allows live memory introspection of running systems and their snapshots. Due to its modular approach it trivial to support almost any scenario where Direct Memory Access is available.

Github

A libmicrovmi connector is already available ! 🎉
https://github.com/memflow/memflow-microvmi

Thanks to @ko1N !

Xen events ABI compatiblity

The Xen events ABI has changed a lot along the Xen releases.

Right now, we are following the ABI from Xen 4.11 and onwards.
If we want to be compatible with all release, we need to follow what LibVMI implemented with xen_events_abi.h

And deduce the vm_event version from the Xen version:
https://github.com/libvmi/libvmi/blob/master/libvmi/driver/xen/xen_events.c#L3213

Icebox integration

Icebox is a Virtual Machine Introspection solution that enable you to stealthily trace and debug any process (kernel or user).

The project is based on the Fast Debugging Protocol library, which ties it to VirtualBox for now.

A libmicrovmi integration could be handy for the Icebox project to support KVM or Xen

Redesign the init function so it returns result objects

When using libmicrovmi in an external project it is rather unfortunate to have the init function just panic if something goes wrong. Imho it would be better to have it return result objects instead so the caller can decide how to handle the situation.
One example is libvmi which tests hypervisor connections by initializing them. If one fails, libvmi just moves on to the next one.

Expose a C-compatible API

Libmicrovmi aims to be compatible with LibVMI and other already existing C projects.

We need to build and expose a C API.