nebn/unraid-simple-monitoring-api

Simple REST API to monitor basic metrics. Created for Unraid for use with Homepage.

Unraid Simple Monitoring API

Simple rest API to monitor basic metrics, currently supports:

• Disk utilization and temperature
• Network traffic
• CPU load and temperature
• Memory utilization

Originally created for Unraid for use with Homepage.

Table of Contents

1. Utilization with Unraid
   1. Installation
   2. Configuration
   3. ZFS
   4. Calling the API
2. Integration with Homepage
   1. Configuration
3. How reliable are the measurements?
4. Installing a QA build

Utilization with Unraid

Installation

Install from the Unraid community apps

Configuration

By default the application expects a configuration file in

/mnt/user/appdata/unraid-simple-monitoring-api/conf.yml

You can find an example file here. It should look like this

networks:
  - eth0
  - anotherNetwork
disks:
  cache:
    - /mnt/cache
    - /another/cache/mount
  array:
    - /mnt/disk1
    - /mnt/disk2

CPU Temperature file

You can specify which file to read to obtain the correct CPU temperature.

cpuTemp: /path/to/temp/file

To see where this information might be, you can try running the following command:

for dir in /sys/class/hwmon/hwmon*; do
  echo "Directory: $dir"
  for file in $dir/temp*_input; do
    echo "Reading from: $file"
    cat $file
  done
done

If no file is specified in the configuration, the software will attempt to figure it out by running a very quick stress test (a few seconds) while monitoring plausible files. You can find the result of this search in the application's logs. This method is of questionable reliability, specifying which file should be read is the preferred option.

Logging level

loggingLevel: DEBUG

Accepted values are DEBUG INFO WARN and ERROR, it defaults to INFO.

ZFS

If any of the mount points listed in the configuration are using ZFS, the application needs to be run as privileged in order to obtain the correct utilization of ZFS datasets. The command zfs list is being used to obtain the correct information, as conventional disk reading methods do not seem to work.

If you are comfortable with running the container as privileged, follow these steps:

• Unraid Docker Tab
• unraid-simple-monitoring-api > Edit
• Change Privileged: to ON
• Apply

You can always decide to turn Privileged: back to OFF.

Tip

If you are not using ZFS, there is no reason to run the container as privileged.

Calling the API

Make a request to

http://your-unraid-ip:24940

The response will be formatted this way.

{
   "array":[
      {
         "mount":"/mnt/disk1",
         "total":3724,
         "used":1864,
         "free":1860,
         "used_percent":50.05,
         "free_percent":49.95,
         "temp":32
      },
      {
         "mount":"/mnt/disk2",
         "total":3724,
         "used":1366,
         "free":2358,
         "used_percent":36.68,
         "free_percent":63.32,
         "temp":34
      },
      {
         "mount":"/mnt/disk3",
         "total":931,
         "used":7,
         "free":924,
         "used_percent":0.75,
         "free_percent":99.25,
         "temp":0
      }
   ],
   "cache":[
      {
         "mount":"/mnt/cache",
         "total":465,
         "used":210,
         "free":255,
         "used_percent":45.16,
         "free_percent":54.84,
         "temp":37
      }
   ],
   "parity":[
      {
         "name":"parity",
         "temp":31
      },
      {
         "name":"parity2",
         "temp":0
      }
   ],
   "network":[
      {
         "interface":"docker0",
         "rx_MiBs":0,
         "tx_MiBs":0,
         "rx_Mbps":0,
         "tx_Mbps":0
      },
      {
         "interface":"eth0",
         "rx_MiBs":0.02,
         "tx_MiBs":5.22,
         "rx_Mbps":0.13,
         "tx_Mbps":43.8
      }
   ],
   "array_total":{
      "mount":"total",
      "total":13034,
      "used":3342,
      "free":9692,
      "used_percent":25.64,
      "free_percent":74.36
   },
   "cache_total":{
      "mount":"total",
      "total":465,
      "used":210,
      "free":255,
      "used_percent":45.16,
      "free_percent":54.84
   },
   "network_total":{
      "interface":"total",
      "rx_MiBs":0.02,
      "tx_MiBs":5.22,
      "rx_Mbps":0.13,
      "tx_Mbps":43.8
   },
   "cpu":{
      "load_percent":10.6,
      "temp":41
   },
   "memory":{
      "total":15788,
      "used":1288,
      "free":14500,
      "used_percent":8.16,
      "free_percent":91.84
   },
   "error":null
}

Integration with Homepage

Configuration

Check out Hompage's official custom API widget documentation.
Your homepage services.yaml should look like this if you want it to look like the above example, showing cache and network data.

- Category:
   - Unraid:
        icon: unraid.png
        href: http://<your-unraid-ip>
        widget:
          type: customapi
          url: http://<your-unraid-ip>:24940
          method: GET # this doesn't matter
          mappings:
            - field:
                cache_total: free
              label: cache free
              format: number
              suffix: GiB
            - field:
                cache_total: free_percent
              label: percent
              format: percent
            - field:
                network_total: rx_MiBs
              label: rx
              format: float
              suffix: MiB/s
            - field:
                network_total: tx_MiBs
              label: tx
              format: float
              suffix: MiB/s

The following are examples for each currently available field.

• Array Total

- field:
    array_total: free # or used, total, used_percent, free_percent
  label: your label
  format: number # or percentage
  suffix: GiB # or nothing in case of percentages, or whatver you prefer

• Cache Total

- field:
    cache_total: free # or used, total, used_percent, free_percent
  label: your label
  format: number # or percentage
  suffix: GiB # or nothing in case of percentages, or whatver you prefer

• Specific Disk

- field:
    array: # or cache
      0: free 
      # '0' is the index of the disk, 0 = the first 
      # 'free' is the field you wish to read
      # specific disks (or cache disks using btrfs) also have the 'temp' field
  label: your label
  format: number
  suffix: GiB

• Parity

- field:
    parity: 
      0: temp 
      # '0' is the index of the parity disk, 0 = the first 
      # 'temp' is the field you wish to read
  label: your label
  format: number
  suffix: °

• Network Total

- field:
    network_total: rx_MiBs # or tx_MiBs, rx_Mbps, tx_Mbps
  label: your label
  format: float # or 'number' to round to the nearest integer
  suffix: MiB/s # or Mbps, or whatever you prefer

• Specific Network

- field:
    network:
      0: rx_MiBs 
      # '0' is the index of the network, 0 = the first 
      # 'rx_MiBs' is the field you wish to read
  label: your label
  format: float
  suffix: MiB/s 

• CPU

- field:
    cpu: load_percent # or temp
  label: your label
  format: percent # or number

• Memory

- field:
    memory: used_percent # or free_percent, total, used, free
  label: your label
  format: percent

Tip

If you wish to show more than the usual 4 allowed fields, you can set the widget property display: list to have the fields displayed in a vertical list that can be arbitrarily long.

widget:
  type: customapi
  display: list
  mappings:
     ...

How reliable are the measurements?

The goal of this API is to be simple, fast, and lightweight.
For these reasons, the measurements provided are not as accurate as they could be.

Disk

Disk utilization is rounded down to the nearest GiB.

Network and CPU

Both Network and CPU usage need to be measured for some time interval. Typically, to get an accurate measurement, you would monitor these for a few seconds before providing a response.
To avoid having to either:

• wait for the measurement to be completed before responding
• continuosly measure them to have a recent measurement ready to respond with

A different approach has been taken: a snapshot of Network and CPU usage is taken every time the API is called, and the response is the average Network and CPU usage between the current and last API call. This ensures that the response is quick and reasonably accurate, without having the process continuously read Network and CPU data even when not required.

Installing a QA build

Everyone's Unraid setup is different, therefore, when implementing a new feature or fixing a bug specific to a certain setup, it might be necessary that the end user (you) install a testing deployment to verify that everything works as expected.

To do so follow these steps:

• Unraid Docker Tab
• unraid-simple-monitoring-api > Stop
• Add container
• Template > unraid-simple-monitoring-api
• Change the name to something else, e.g.: unraid-simple-monitoring-api-QA
• Change Repository: to ghcr.io/nebn/unraid-simple-monitoring-api:qa (The actual tag might change, currently using qa)
• Apply

You should now have 2 installations on your Docker Tab, and can switch between them by stopping/starting them.

Note

Avoid having both active at the same time, as they share the same port and would therefore be unable to start the HTTP service.

Warning

It is a good idea to switch back to the official build as soon as whatever fix you were testing is deployed to it. QA builds are unstable and are likely to not work correctly if you update them further.

unraid-simple-monitoring-api's Issues

Debug logging

There is currently very little debug logging, making it difficult to figure out what's going wrong if there ever is a problem.
Logging level could be added as an option to the already existing conf.yml. Using INFO or WARN as default.

Temperatures

Would it be possible to also add temperatures for each hdd and cpu? And maybe other sensors?

Add option to exclude reading of unused data

Most users probably are not using all the data collected each time the API is called.
There should be an option to specify which information to include or exclude.

No CPU temp reading

Using this as part of a little desktop app I'm making but am finding that no CPU temps are returned.

CPU & RAM usage readouts work fine. Just no temp from the CPU.

Temp is reported correctly via UnRaid.

Unraid App cannot locate conf file

I installed the app today on Unraid and keep getting the following error:

`text error warn system array login

2024/04/08 21:59:59 ERROR Cannot read conf.yml error="read /app/conf.yml: is a directory"

** Press ANY KEY to close this window ** `

I tried putting the actual conf.yml file in several folder locations and tried to map a different location and they all failed.

I think there is confusion within the image looking for a file named "conf.yml" but also trying to look at a folder named "conf.yml" in the coding.

ZFS Volume Size Reporting

Any volume (or my 4 disk cache pool) that is formatted under zfs always reports 0 bytes used. This also throws off the percentages. Are there any plans to support these volumes better? It also affects other monitoring tools and the unraid dashboard seems to be the only place that reports the sizes correctly.

Thank you!

Parity Information

Would it be possible to pull the parity drive? Specifically the temperature if possible?

Also, is there any plans to get data from other drives in the cache? As I'm runnning my cache in RAID 1, it would be really nice to get that drive temp too.

Prevent divisions by 0 and guarantee a valid JSON as response

There are a few very rare situations where divisions by zero might occur. In addition to this a simple string displaying the error is returned, instead of a valid JSON.

Use smaller images

I believe you can use smaller containers if you use stages:

FROM golang:alpine as build

RUN apk update && apk upgrade

WORKDIR /app

COPY go.mod go.sum ./
RUN go mod download

COPY cmd ./cmd 
COPY internal ./internal 

RUN CGO_ENABLED=0 GOOS=linux go build -o /unraid-simple-monitoring-api ./cmd

FROM alpine as run

COPY --from=build /unraid-simple-monitoring-api .

ARG HOSTFS_PREFIX

ENV CONF_PATH=/app/conf.yml
ENV HOSTFS_PREFIX=${HOSTFS_PREFIX}

CMD ["/unraid-simple-monitoring-api"]

Testing this:

❯ docker build -t nebn/unraid-simple-monitoring-api:example --file deploy/Dockerfile .
...


❯ docker image ls nebn/*
REPOSITORY                          TAG       IMAGE ID       CREATED         SIZE
nebn/unraid-simple-monitoring-api   example   4ffda7763a11   6 minutes ago   15.6MB
nebn/unraid-simple-monitoring-api   latest    b167c6a06642   8 minutes ago   332MB

Due to my limited availability this week I can't create a pull request 😅

Better Homepage documentation

There should ideally be an example of using every output from the API in Homepage.

Guarantee order of disks and networks

Mapping specific disks / networks in Homepage is not possible if they are not predictably sorted in the response. They should be sorted according to the order in the configuration.

Memory stats

It would be nice to also be able to see stats about memory.

Almost same layout as a disk with total, used, free, free_percent and used_percent, leaving mount out.

Can API surface CPU usage?

Firstly, great unraid app! Great documentation, I am knowledgeable but not a pro and could figure it out. Request, can the API be modified to surface CPU usage so that I can add that to homepage? I like to be able to see CPU usage on my various servers when in homepage.