tinkoff/libvirt-exporter

Prometheus metrics exporter for libvirt.

Prometheus libvirt exporter

Docker image is available at dockerhub.

• Dockerfile - creates a docker container with dynamically linked libvirt-exporter. Make an image and run with docker container run -p9177:9177 -v /var/run/libvirt:/var/run/libvirt yourcontainername. Based on the latest golang:alpine.
• build-with - builds dynamically linked libvirt-exporter in the container based on Dockerfile specified as an argument. Ex.: build-with ./build_container/Dockerfile.ubuntu2004 will build libvirt-exporter for Ubuntu 20.04.

Metrics

The following metrics/labels are being exported:

libvirt_domain_block_meta{bus="scsi",cache="none",discard="unmap",disk_type="network",domain="instance-00000337",driver_type="raw",serial="5f1a922c-e4b5-4020-9308-d70fd8219ac8",source_file="somepool/volume-5f1a922c-e4b5-4020-9308-d70fd8219ac8",target_device="sda"} 1
libvirt_domain_block_stats_allocation{domain="instance-00000337",target_device="sda"} 2.1474816e+10
libvirt_domain_block_stats_capacity_bytes{domain="instance-00000337",target_device="sda"} 2.147483648e+10
libvirt_domain_block_stats_flush_requests_total{domain="instance-00000337",target_device="sda"} 5.153142e+06
libvirt_domain_block_stats_flush_time_seconds_total{domain="instance-00000337",target_device="sda"} 473.56850521
libvirt_domain_block_stats_limit_burst_length_read_requests_seconds{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_length_total_requests_seconds{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_length_write_requests_seconds{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_read_bytes{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_read_bytes_length_seconds{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_read_requests{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_total_bytes{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_total_bytes_length_seconds{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_total_requests{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_write_bytes{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_write_bytes_length_seconds{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_burst_write_requests{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_read_bytes{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_read_requests{domain="instance-00000337",target_device="sda"} 640
libvirt_domain_block_stats_limit_total_bytes{domain="instance-00000337",target_device="sda"} 1.572864e+08
libvirt_domain_block_stats_limit_total_requests{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_write_bytes{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_limit_write_requests{domain="instance-00000337",target_device="sda"} 320
libvirt_domain_block_stats_physicalsize_bytes{domain="instance-00000337",target_device="sda"} 2.147483648e+10
libvirt_domain_block_stats_read_bytes_total{domain="instance-00000337",target_device="sda"} 1.7704034304e+11
libvirt_domain_block_stats_read_requests_total{domain="instance-00000337",target_device="sda"} 1.9613982e+07
libvirt_domain_block_stats_read_time_seconds_total{domain="instance-00000337",target_device="sda"} 161803.085086353
libvirt_domain_block_stats_size_iops_bytes{domain="instance-00000337",target_device="sda"} 0
libvirt_domain_block_stats_write_bytes_total{domain="instance-00000337",target_device="sda"} 9.2141217792e+11
libvirt_domain_block_stats_write_requests_total{domain="instance-00000337",target_device="sda"} 2.8434899e+07
libvirt_domain_block_stats_write_time_seconds_total{domain="instance-00000337",target_device="sda"} 530522.437009019

libvirt_pool_info_allocation_bytes{pool="default"} 5.4276182016e+10
libvirt_pool_info_available_bytes{pool="default"} 5.1278647296e+10
libvirt_pool_info_capacity_bytes{pool="default"} 1.05554829312e+11

libvirt_domain_info_cpu_time_seconds_total{domain="instance-00000337"} 949422.12
libvirt_domain_info_maximum_memory_bytes{domain="instance-00000337"} 8.589934592e+09
libvirt_domain_info_memory_usage_bytes{domain="instance-00000337"} 8.589934592e+09
libvirt_domain_info_meta{domain="instance-00000337",flavor="someflavor-8192",instance_name="name.of.instance.com",project_name="instance.com",project_uuid="3051f6f46d394ab98f55a0670ae5c70b",root_type="image",root_uuid="155e5ab9-d28c-48f2-bd8d-f193d0a6128a",user_name="master_admin",user_uuid="240270fa2a3e4fd3baa6d6e776669b19",uuid="1bac351f-242e-4d53-8cf3-fd91b061069c"} 1
libvirt_domain_info_virtual_cpus{domain="instance-00000337"} 2
libvirt_domain_info_vstate{domain="instance-00000337"} 1

libvirt_domain_interface_meta{domain="instance-00000337",source_bridge="br-int",target_device="tapa7e2fe95-a7",virtual_interface="a7e2fe95-a7cf-4bec-8180-d835cf342d72"} 1
libvirt_domain_interface_stats_receive_bytes_total{domain="instance-00000337",target_device="tapa7e2fe95-a7"} 7.9182281e+09
libvirt_domain_interface_stats_receive_drops_total{domain="instance-00000337",target_device="tapa7e2fe95-a7"} 0
libvirt_domain_interface_stats_receive_errors_total{domain="instance-00000337",target_device="tapa7e2fe95-a7"} 0
libvirt_domain_interface_stats_receive_packets_total{domain="instance-00000337",target_device="tapa7e2fe95-a7"} 4.378193e+06
libvirt_domain_interface_stats_transmit_bytes_total{domain="instance-00000337",target_device="tapa7e2fe95-a7"} 1.819996331e+09
libvirt_domain_interface_stats_transmit_drops_total{domain="instance-00000337",target_device="tapa7e2fe95-a7"} 0
libvirt_domain_interface_stats_transmit_errors_total{domain="instance-00000337",target_device="tapa7e2fe95-a7"} 0
libvirt_domain_interface_stats_transmit_packets_total{domain="instance-00000337",target_device="tapa7e2fe95-a7"} 2.275386e+06

libvirt_domain_memory_stats_actual_balloon_bytes{domain="instance-00000337"} 8.589934592e+09
libvirt_domain_memory_stats_available_bytes{domain="instance-00000337"} 8.363945984e+09
libvirt_domain_memory_stats_disk_cache_bytes{domain="instance-00000337"} 0
libvirt_domain_memory_stats_major_fault_total{domain="instance-00000337"} 3.34448e+06
libvirt_domain_memory_stats_minor_fault_total{domain="instance-00000337"} 5.6630255354e+10
libvirt_domain_memory_stats_rss_bytes{domain="instance-00000337"} 8.7020544e+09
libvirt_domain_memory_stats_unused_bytes{domain="instance-00000337"} 7.72722688e+08
libvirt_domain_memory_stats_usable_bytes{domain="instance-00000337"} 2.27098624e+09
libvirt_domain_memory_stats_used_percent{domain="instance-00000337"} 72.84790881786736

libvirt_domain_vcpu_cpu{domain="instance-00000337",vcpu="0"} 7
libvirt_domain_vcpu_delay_seconds_total{domain="instance-00000337",vcpu="0"} 880.985415109
libvirt_domain_vcpu_state{domain="instance-00000337",vcpu="0"} 1
libvirt_domain_vcpu_time_seconds_total{domain="instance-00000337",vcpu="0"} 315190.41
libvirt_domain_vcpu_wait_seconds_total{domain="instance-00000337",vcpu="0"} 0

libvirt_up 1

Libvirt/qemu version notice

Some of the above might be exposed only with:

libvirt >= v7.2.0: libvirt_domain_vcpu_delay_seconds_total

Historical

Project forked from https://github.com/kumina/libvirt_exporter and substantially rewritten. Implemented support for several additional metrics, ceph rbd (and network block devices), ovs. Implemented statistics collection using GetAllDomainStats

And then forked again from https://github.com/rumanzo/libvirt_exporter_improved and rewritten. Implemented meta metrics and more info about disks, interfaces and domain.

This repository provides code for a Prometheus metrics exporter for libvirt. This exporter connects to any libvirt daemon and exports per-domain metrics related to CPU, memory, disk and network usage. By default, this exporter listens on TCP port 9177.

This exporter makes use of libvirt-go, the official Go bindings for libvirt. This exporter make use of the GetAllDomainStats()

libvirt-exporter's Issues

Cant Compile

build_static.sh fails with:

...
make[2]: Leaving directory '/tmp/libvirt-3.8.0/examples'
make[1]: Leaving directory '/tmp/libvirt-3.8.0/examples'
+ sed -i 's/^Libs:.*/& -lnl -ltirpc -lxml2/' /usr/local/lib/pkgconfig/libvirt.pc
+ cd /libvirt-exporter
+ export 'GOPATH=/gopath'
+ go get -d ./...
+ go build --ldflags '-extldflags "-static"' -o libvirt-exporter
# github.com/prometheus/client_golang/prometheus
/gopath/src/github.com/prometheus/client_golang/prometheus/go_collector.go:388:15: undefined: debug.ReadBuildInfo```

[FEATURE] stealtime metrics :)

Is your feature request related to a problem? Please describe.
A very powerfull metrics would be the stealttime of a vm, for balancing the while VMs between Servers.

Describe the solution you'd like
High Stealtime leads to a low performance of the vm (20% ST means only 80% of CPU is available, and to maximize the hardwareusage this is a powerfull metric)

Describe alternatives you've considered
Load of the libvirt Compute Node is not powerfull enough.

Build error

Want to make libvirt-exporter docker image, command: "docker build -t prometheus-libvirt-exporter -f Dockerbuildimage ."

I use Ubuntu18.04 on HP server where prometheus, prometheus alert manager and other exporters (everything in docker containers) and multiple VMs are running.

Is it mandatory static linking of libvirt library? Or Im doing something in wrong dirrection?

The error is:
rm util/virkeyname-linux.pod util/virkeycode-rfb.pod util/virkeyname-osx.pod util/virkeycode-linux.pod util/virkeycode-osx.pod util/virkeycode-usb.pod util/virkeycode-xt.pod util/virkeycode-win32.pod util/virkeycode-atset3.pod util/virkeycode-atset1.pod util/virkeycode-atset2.pod util/virkeycode-xtkbd.pod util/virkeyname-win32.pod
make[3]: Leaving directory '/tmp/libvirt-3.8.0/src'
make[2]: *** [Makefile:5808: all] Error 2
make[2]: Leaving directory '/tmp/libvirt-3.8.0/src'
make[1]: *** [Makefile:2116: all-recursive] Error 1
make[1]: Leaving directory '/tmp/libvirt-3.8.0'
make: *** [Makefile:2009: all] Error 2
The command '/bin/sh -c ./configure --disable-shared --enable-static --localstatedir=/var --without-storage-mpath && make -j$(nproc) && make install && sed -i 's/^Libs:.*/& -lnl -ltirpc -lxml2/' /usr/local/lib/pkgconfig/libvirt.pc' returned a non-zero code: 2

Could attach whole log if needed.

Use specific version of docker image instead of latest version

Dear guys

In the Dockerfile, It doesn't have a specific version; I think it's not good practice for Dockerfile.
Also, in the Dockerfile, it uses the go image, and again it adds (upgrades) the go; It's better to change the docker image instead of it.

I created this PR based on my points.

[FEATURE] Disk usage statistics

Is it possible to support disk usage on virtual instance? Like it is visible size of disk:
libvirt_domain_block_stats_capacity_bytes{domain="instance-00000222",target_device="sda"}
and it will be useful have also used_bytes.

Thanks a lot!

[FEATURE] add --version argument

At the moment it is not possible to determine the version of the binary. It would be helpful to have a -version argument to be able to read the version.

[BUG] error message an no metrics with blkiotune

Describe the bug
2021/04/08 05:53:38 Failed to scrape metrics: virError(Code=1, Domain=10, Message='internal error: block_io_throttle inserted entry was not in expected format')

To Reproduce
Steps to reproduce the behavior:

1. Go to your node, start exporter
2. See error
   2021/04/08 05:53:38 Failed to scrape metrics: virError(Code=1, Domain=10, Message='internal error: block_io_throttle inserted entry was not in expected format')

Expected behavior
`# HELP libvirt_domain_block_stats_size_iops_bytes The size of IO operations per second permitted through a block device

TYPE libvirt_domain_block_stats_size_iops_bytes gauge

libvirt_domain_block_stats_size_iops_bytes{domain="dc533c58-8902-4db6-933b-e626721a5221",target_device="hda"} 0`
Additional context
libvirt prozess startet with:

/usr/bin/kvm -name guest=UUID,process=qemu:UUIS,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-49-dc533c58-8902-4db6-9/master-key.aes -machine pc-i440fx-2.4,accel=kvm,usb=off,dump-guest-core=off,memory-backend=pc.ram -cpu Haswell-noTSX,ss=on,pdpe1gb=on,abm=on -m 8192 -object memory-backend-ram,id=pc.ram,size=8589934592 -overcommit mem-lock=off -smp 8,maxcpus=64,sockets=64,cores=1,threads=1 -uuid UUID -smbios type=0,vendor=unspecified,version=1.0,date=10/09/2015,release=166.0 -smbios type=1,manufacturer=unspecified,product=Legacy Virtual Machine,version=1.0,serial=0123456789,uuid=UUID -smbios type=2,manufacturer=unspecified,product=Legacy Virtual Machine,version=1.0,serial=0123456789 -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=98,server=on,wait=off -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -blockdev {"driver":"rbd","pool":"rbd","image":"CEPH-UUID","server":[{"host":"mon1","port":"6789"},{"host":"mon2","port":"6789"},{"host":"mon3","port":"6789"}],"node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":false,"discard":"unmap","cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-2-storage"} -device ide-hd,bus=ide.0,unit=0,drive=libvirt-2-format,id=ide0-0-0,bootindex=7,write-cache=on -blockdev {"driver":"rbd","pool":"isos","image":"ISO-UUID","server":[{"host":"mon1","port":"6789"},{"host":"mon2","port":"6789"},{"host":"mon3","port":"6789"}],"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"} -device ide-cd,bus=ide.1,unit=1,drive=libvirt-1-format,id=ide0-1-1,bootindex=5 -netdev tap,fd=104,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=02:39:3e:56:fa:01,bus=pci.0,addr=0x10,romfile= -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc [::0]:37 -device VGA,id=video0,vgamem_mb=16,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on

[FEATURE] - Add end-to-end tests

There're no tests right now, except spelling and linter.

Exporter must have a test CI workflow to test most of functionality with real libvirt instances. Not sure about implementation details, but as a result tests must proof that all available metrics are collected and they are collected properly.

It could be useful to check libvirt-exporter data with virsh data when dealing with some metrics.

Might be some issues running libvirt instance during CI.

Extract IP address of instances

Dear Aleksei,

I hope you are doing well.
If possible, please add ip address of instance as a label metric.

Best regards,
Alireza

[FEATURE] - Get binary release for major platforms

Is your feature request related to a problem? Please describe.
Would it be possible to build binary release for Linux x86_64/aarch64?

Describe the solution you'd like
As for this exporter: https://github.com/openstack-exporter/openstack-exporter/releases
I'd like to be able to download an archive of the statically build exporter for linux.

Describe alternatives you've considered
I'll have to get a two step build phase as our platform is an airgapped platform that can't reach internet and so can't build correctly the exporter without cloning the whole github platform.

Additional context
Any offline platform won't be able to build this exporter as dependencies are refering to github or internet hosted repos.

libvirt_up = 0 when one or more VM get qemuProcessReportLogError upon creation

Hi all,

We have recently experienced the following error:

Jul 20 06:13:31 server1 libvirtd[47045]: 2022-07-20 06:13:31.181+0000: 47045: error : qemuProcessReportLogError:1914 : internal error: qemu unexpectedly closed the monitor: 2022-07-20T06:13:31.090440Z qemu-system-x86_64: -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1: Failed to get "write" lock
Jul 20 06:13:31 server1 libvirtd[47045]: 2022-07-20 06:13:31.181+0000: 47049: error : qemuProcessReportLogError:1914 : internal error: process exited while connecting to monitor: 2022-07-20T06:13:31.090440Z qemu-system-x86_64: -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1: Failed to get "write" lock

Jul 20 07:01:27 server1 libvirt_exporter[94868]: 2022/07/20 07:01:27 Failed to scrape metrics: virError(Code=38, Domain=10, Message='Unable to read from monitor: Connection reset by peer')

• While the above happened, the libvirt_exporter flagged libvirt_up to be == 0
• What we understand from it is that if even 1 VM fails to be created due to this error, then libvirt_up will be 0
• During this time, except for this specific failure, libvirt/virsh was working totally fine.

Please let us know if any more information is needed.

Add block device IOPS and byte/s limit metric

Need to expose disk-related limits.

Something like this:

libvirt_domain_block_limit_read_bytes
libvirt_domain_block_limit_write_bytes
libvirt_domain_block_limit_read_requests
libvirt_domain_block_limit_write_requests
libvirt_domain_block_limit_total_bytes
libvirt_domain_block_limit_total_requests

and burst stats:

libvirt_domain_block_limit_burst_read_bytes
libvirt_domain_block_limit_burst_read_bytes_seconds
libvirt_domain_block_limit_burst_write_bytes
libvirt_domain_block_limit_burst_write_bytes_seconds
libvirt_domain_block_limit_burst_read_requests
libvirt_domain_block_limit_burst_read_requests_seconds
libvirt_domain_block_limit_burst_write_requests
libvirt_domain_block_limit_burst_write_requests_seconds

grafana仪表盘太老

https://grafana.com/grafana/dashboards/13633
仪表盘有些指标和新版本的exporter对不上

[BUG] - Network statistics cannot be obtained when instances have multiple SRIOV ports

Describe the bug
This appears to be a similar issue to the one resolved in #4. When a libvirt instance has multiple SRIOV attached interfaces, the /metrics resource returns a 500 error with output similar to the following:

An error has occurred while serving metrics:

8 error(s) occurred:
* collected metric "libvirtd_domain_net_rx_bytes" { label:<name:"interface" value:"" > label:<name:"uuid" value:"40e776cf-2187-4b73-9ab5-9168a1c567da" > counter:<value:0 > } was collected before with the same name and label values
* collected metric "libvirtd_domain_net_rx_packets" { label:<name:"interface" value:"" > label:<name:"uuid" value:"40e776cf-2187-4b73-9ab5-9168a1c567da" > counter:<value:0 > } was collected before with the same name and label values
* collected metric "libvirtd_domain_net_rx_errors" { label:<name:"interface" value:"" > label:<name:"uuid" value:"40e776cf-2187-4b73-9ab5-9168a1c567da" > counter:<value:0 > } was collected before with the same name and label values
* collected metric "libvirtd_domain_net_rx_drop" { label:<name:"interface" value:"" > label:<name:"uuid" value:"40e776cf-2187-4b73-9ab5-9168a1c567da" > counter:<value:0 > } was collected before with the same name and label values
* collected metric "libvirtd_domain_net_tx_bytes" { label:<name:"interface" value:"" > label:<name:"uuid" value:"40e776cf-2187-4b73-9ab5-9168a1c567da" > counter:<value:0 > } was collected before with the same name and label values
* collected metric "libvirtd_domain_net_tx_packets" { label:<name:"interface" value:"" > label:<name:"uuid" value:"40e776cf-2187-4b73-9ab5-9168a1c567da" > counter:<value:0 > } was collected before with the same name and label values
* collected metric "libvirtd_domain_net_tx_errors" { label:<name:"interface" value:"" > label:<name:"uuid" value:"40e776cf-2187-4b73-9ab5-9168a1c567da" > counter:<value:0 > } was collected before with the same name and label values
* collected metric "libvirtd_domain_net_tx_drop" { label:<name:"interface" value:"" > label:<name:"uuid" value:"40e776cf-2187-4b73-9ab5-9168a1c567da" > gauge:<value:0 > } was collected before with the same name and label values

This is seen in OpenStack deployments, and it could be that they are failing to insert an interface name where they should be, but a more graceful failure mode in the exporter would be useful if so.

Version of libvirt-exporter

Is libvirt-exporter used in docker?
No

Libvirt version
6.0.0

To Reproduce

• Start an instance
• Attach one SRIOV interface (no issue observed)
• Attach another SRIOV interface (errors are produced)

Expected behavior
Metrics resource continues to produce output without error

Additional context
I've included a snapshot of the libvirt XML and virsh output below:

   <interface type='bridge'>
      <mac address='fa:16:3e:2a:68:25'/>
      <source bridge='brq862b83ec-b7'/>
      <target dev='tapdd8b8ce2-c2'/>
      <model type='virtio'/>
      <mtu size='1500'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='fa:16:3e:f1:a1:ac'/>
      <source bridge='brq681aa992-37'/>
      <target dev='tapa737598a-b4'/>
      <model type='virtio'/>
      <mtu size='1500'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </interface>
    <interface type='hostdev' managed='yes'>
      <mac address='fa:16:3e:5d:cd:ce'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x65' slot='0x02' function='0x2'/>
      </source>
      <vlan>
        <tag id='2021'/>
      </vlan>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
    </interface>
    <interface type='hostdev' managed='yes'>
      <mac address='fa:16:3e:ee:35:a1'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x65' slot='0x04' function='0x6'/>
      </source>
      <vlan>
        <tag id='2022'/>
      </vlan>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
    </interface>

:~# virsh domiflist 4
 Interface        Type      Source           Model    MAC
-------------------------------------------------------------------------
 tapdd8b8ce2-c2   bridge    brq862b83ec-b7   virtio   fa:16:3e:2a:68:25
 tapa737598a-b4   bridge    brq681aa992-37   virtio   fa:16:3e:f1:a1:ac
 -                hostdev   -                -        fa:16:3e:5d:cd:ce
 -                hostdev   -                -        fa:16:3e:ee:35:a1

[FEATURE] project_id label in libvirt_domain_info_meta metric

This exporter is the standard libvirtd exporter for Kolla/Kolla-Ansible.
https://github.com/openstack/kolla/blob/master/docker/prometheus/prometheus-libvirt-exporter/Dockerfile.j2

I deploy kolla-ansible with cloudkitty as rating module and with prometheus as metric source. So cloudkitty has to be configured to use the promQL to retrieve the data from Prometheus. Unfortunately there is no label 'project_id' in the libvirt_domain_info_meta metric. project_id is the standard label name for cloudkitty to filter the domains by. This is configurable, but then other exporters like openstack exporter can't retrieve data.

So for better compatibility i hope you can insert a label with name project_id in the libvirt_domain_info_meta metric.
I'm not a programmer, but think it is not much work to do.

Can`t compile

Compiled via: build_static.sh
Branch: Master (fresh pull)
OS: Alpine 3.8 (within docker)
HOST: MacOS BigSur

Testtrun (without strip) on ArchLinux & Ubuntu 20.04 (segfault)

make[1]: Entering directory '/tmp/libvirt-3.8.0/po'
if test "libvirt" = "gettext-tools"; then \
  ../build-aux/install-sh -c -d /usr/local/share/gettext/po; \
  for file in Makefile.in.in remove-potcdate.sin quot.sed boldquot.sed [email protected] [email protected] insert-header.sin Rules-quot   Makevars.template; do \
    /usr/bin/install -c -m 644 ./$file \
		    /usr/local/share/gettext/po/$file; \
  done; \
  for file in Makevars; do \
    rm -f /usr/local/share/gettext/po/$file; \
  done; \
else \
  : ; \
fi
make[1]: Leaving directory '/tmp/libvirt-3.8.0/po'
Making install in examples
make[1]: Entering directory '/tmp/libvirt-3.8.0/examples'
make[2]: Entering directory '/tmp/libvirt-3.8.0/examples'
make[2]: Nothing to be done for 'install-exec-am'.
../build-aux/install-sh -c -d ""/usr/local/etc/libvirt/nwfilter""
for f in ./xml/nwfilter/no-other-rarp-traffic.xml ./xml/nwfilter/qemu-announce-self-rarp.xml ./xml/nwfilter/allow-dhcp-server.xml ./xml/nwfilter/no-mac-broadcast.xml ./xml/nwfilter/allow-incoming-ipv4.xml ./xml/nwfilter/no-other-l2-traffic.xml ./xml/nwfilter/allow-ipv4.xml ./xml/nwfilter/no-arp-mac-spoofing.xml ./xml/nwfilter/allow-arp.xml ./xml/nwfilter/no-arp-ip-spoofing.xml ./xml/nwfilter/clean-traffic.xml ./xml/nwfilter/no-arp-spoofing.xml ./xml/nwfilter/allow-dhcp.xml ./xml/nwfilter/no-mac-spoofing.xml ./xml/nwfilter/no-ip-spoofing.xml ./xml/nwfilter/no-ip-multicast.xml ./xml/nwfilter/qemu-announce-self.xml; do \
	/usr/bin/install -c -m 644 $f ""/usr/local/etc/libvirt/nwfilter""; \
done
make[2]: Leaving directory '/tmp/libvirt-3.8.0/examples'
make[1]: Leaving directory '/tmp/libvirt-3.8.0/examples'
+ sed -i 's/^Libs:.*/& -lnl -ltirpc -lxml2/' /usr/local/lib/pkgconfig/libvirt.pc
+ cd /libvirt-exporter
+ export 'GOPATH=/gopath'
+ mkdir -p /gopath/
+ ln -s /libvirt-exporter/vendor /gopath/src
+ mkdir -p /gopath/src/github.com/AlexZzz
+ ln -s /libvirt-exporter /gopath/src/github.com/AlexZzz/libvirt-exporter
+ go build --ldflags '-extldflags "-static"' -o libvirt-exporter
go build _/libvirt-exporter: build output "libvirt-exporter" already exists and is not an object file

Is it possible to monitoring hypervisor stats with this exporter?

Hi, I'm new in prometheus and exporter and looking for a openstack hypervisor monitoring.
As I look through source, I can find the exporter offers vm metrics. Then, is it possible to get metrics of hypervisor?

I think the node stats of libvirt is hypervisor stats. (I'm not sure for this because I'm a poor junior dev 😢 )
If i'm right, then the exporter can offer a hypervisor metrics with NodeCPUStats in libvirt-exporter/vendor/libvirt.org/go/libvirt/connect.go for example?

Don't scrape vcpu metrics when instance is in a wrong state

Feb 25 11:07:07 somehost.example.com libvirt-exporter[780603]: 2021/02/25 11:07:07 Failed to scrape metrics: virError(Code=55, Domain=10, Message='Requested operation is not valid: cannot retrieve vcpu information for inactive domain')

Some metric doesn't collected

some metrics like libvirt_domain_stat_memory_available_bytes doesn't collected through libvirt-export.
but I cant see it by virsh command in console.

Is there any suggestion to solve this? Or do you have any clues as to why it wasn't collected?

[BUG] Metrics for memory not working correctly

At least for:
libvirt_domain_memory_stats_unused_bytes
libvirt_domain_memory_stats_used_percent

I got wrong values. For 1st it is 3,66GB and 2nd is 7,29%, but should be total-used=1,6Gi and 2,2/3,8*100=58%.

System output:

free -h

          total        used        free      shared  buff/cache   available

Mem: 3.8Gi 2.2Gi 197Mi 3.0Mi 1.4Gi 1.3Gi
Swap: 0B 0B 0B

cat /etc/lsb-release

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.4 LTS"