subuk/vmango

Your own personal IaaS cloud

vmango

Vmango is a virtual machines management web interface written using Go.

The main goal of project is not to provide a hypervisor configuration tool, because that problem already solved by many configuration management systems like Ansible or Puppet, but provide a convenient way to manage virtual machines on existing hypervisors.

Current features:

• SSH keys management and injection
• Volume management
• KVM machines via libvirt
• Web console
• Support for cloud OS images (with cloud-init installed)
• Custom userdata for cloud-init
• Bridged network

Installation

There are two RPM repositories:

• vmango for the latest tagged release, it may be considered stable
• vmango-devel contains packages built automatically from the latest commit in master branch

CentOS 7 and 8

1. Enable copr reposotory, use subuk/vmango-devel for the latest version, subuk/vmango for stable
2. Install package
3. Edit configuration file /etc/vmango.conf
4. Start and enable systemd service vmango

yum install -y yum-plugin-copr && yum copr -y enable subuk/vmango
yum install -y vmango
systemctl enable --now vmango

Ubuntu 18.04

1. Follow instructions on vmango or vmango-devel ppa page https://launchpad.net/~subuk/+archive/ubuntu/vmango
2. Install package
3. Edit configuration file /etc/vmango.conf

sudo add-apt-repository ppa:subuk/vmango
sudo apt-get install vmango

Hypervisor configuration

Install libvirt and qemu-kvm.

Ubuntu:

sudo apt-get install libvirt-bin qemu-kvm qemu-system

Centos:

yum install -y libvirt qemu-kvm
systemctl enable --now libvirtd

Allow your user to access libvirt socket:

sudo usermod -aG libvirtd [username]
newgrp libvirtd

Download vm images to default libvirt pool location:

cd /var/lib/libvirt/images/
wget https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1901.qcow2
wget https://cloud-images.ubuntu.com/minimal/releases/bionic/release/ubuntu-18.04-minimal-cloudimg-amd64.img

Define default volume pool (if not exists) and start it:

virsh pool-define-as default dir --target /var/lib/libvirt/images/
virsh pool-start default
virsh pool-autostart default

Local run

Copy vmango.dist.conf to vmango.conf and change configuration if needed.

Run app

make && ./bin/vmango web

View it on http://localhost:8080 (login with admin / admin by default)

Dependencies for Ubuntu 14.04+

Install libvirt and kvm

sudo apt-get install libvirt-dev libvirt-bin qemu-kvm qemu-system genisoimage

Install Go compiler. Configure libvirt as described above. Now you can use your own computer as hypervisor.

Dependencies for MacOS

Install Go compiler, libvirt C library and mkisofs util (for configdrive creation)

brew install go
brew install libvirt
brew install dvdrtools

You need a linux hypervisor somewhere in the world, because libvirt doesn't support MacOS. Make sure to add ?socket option to remote libvirt urls.

Build RPM

With docker for Centos 7:

./dockerbuild.sh centos-7 make rpm

Locally:

make rpm

Build DEB

With docker for Ubuntu 18.04:

./dockerbuild.sh ubuntu-1804 make deb

Locally:

make deb

vmango's Issues

Network Filter

Have you experiencing errors regarding nwfilter clean-traffic with CTRL_IP_LEARNING=any filter or any other related to arp and ip spoofing prevention with default network on ubuntu 16.04? While I am using clean-traffic vm cannot assign ip address from dhcp. I also trying to use examples from libvirt manual, but nothing works.

For example:

<interface type='bridge'>
  <source bridge='virbr0'/>
  <model type='virtio'/>
  <filterref filter='clean-traffic'/>
</interface>

IPv6 support!

please add IPv6 support! or guide me a little bit so, i can do it and make a pull request

Problems and suggestions - V2 :)

Hi again,
after reviewing the latest changes in release 0.10.0 I found the following:

1. 
   Host memory is still shown in bytes.
2. Please, add the possibility to hot add / remove memory to the VM + hot add CPU (all 3 things are required as live VM parameters updating) - all those are supported by qemu/kvm
   It will be nice to have those live updates alongside CPU and NUMA pinning :)
3. Maybe if possible to make the values of the memory translated from megabytes to gigabytes when creating/updating machines. I mean I know that qemu requires those values in MB, but you can expose the values in GB in the web and make some translation on the background + have some predefined values alongside with the option to write the values yourself. Also possibility to choose if want it in MB for fine tuning or in GB will be great. Just if you can do this - it will save me some time from calculating the values every time for bigger machines.
4. Still lacking support of connecting between hosts and showing them all in the web interface (I know this was postponed, just marking it here to avoid forgetting it)

The other stuff all have open issues for them, so won't mention all.
If it is possible you can add 1,2 and 3 in the next release. I know 4 is a complex thing.

Thanks for the recent changes also! They work great at the moment!

Console tool for password hash generation

Something like:

$ echo secret | ./bin/vmango genpw
$2a$10$m8hh/nNMb.2krzysVyoRVOS3LoSZha7rwV4lfBnCvasTBKAGC.X4i

#35

Feature request: Add vnc web console

One suggestion is to use noVNC (https://novnc.com)

feature request: autostart on boot

There should be an option to autostart a vm on boot

Add support for bridge networking

Add support for bridge networking using
https://www.isc.org/kea/
as DHCP server listening on bridge uplink port

Feature Request: Mark Volume as "Protected"

Maybe all the objects, VMs, networks, volumes, etc.
Protected from accidental deletion. Maybe this could go along with with #33 option to delete volumes when deleting a VM.

How to isolate KVM VMs from each other on same host with dhcp?

Without creating new subnet for each VM

feature request: images management from the interface

this could be done within the web interface rather than having to edit the config file.

hard limit the CPU allowance to X percent

How do i limit the CPU allowance to 25% of a single core (for example: 1 ghz of physical 4gz core)
is there a way to hard limit this?

Network management

In Networks section, instead of just viewing the networks, to be able to edit the network settings of the host - like DHCP range, reserved addresses, rules, etc. supported by libvirt network.

IP addresses not displayed when using bridged network

Compile Problem?

GOPATH=/root/vmango/vendor:/root/vmango go build -o vendor/bin/go-bindata github.com/jteeuwen/go-bindata/go-bindata
/bin/sh: 1: go: not found
Makefile:28: recipe for target 'vendor/bin/go-bindata' failed
make: *** [vendor/bin/go-bindata] Error 127

Debian 8

Feature Request: Create VM from iso

a'la virt-install

[RFE] Provide an RPM package

It would be nice if you can create a copr repo for easy consumption of the vmango RPM.

Some problems and suggestions

Hi again,
Problems:

1. Console
   I have several VMs running and I cannot open console to them. Here is what I am getting:
   
2. IP of running machines not showing in the Machines section under the IP column.
   Example:
   

Suggestions:

1. Memory everywhere to be displayed in GB, not in MB as for bigger hosts, those are a lot of numbers. Examples:
   
2. Volumes size to be displayed in GB everywhere.
   Example:
   
3. In Networks section, instead of just viewing the networks, to be able to edit the network settings of the host - like DHCP range, reserved addresses, rules, etc. supported by libvirt network
4. Possibility to connect to another host from the same interface, to be able to see multiple hosts and machines. LIbvirtd and qemu are supporting such connection over http, ssh and other. You can expose some of those options to make this possible.
5. Users management - possibility to add more users and define some roles for them like staff user, admin, superuser, normal user, guest, etc.
6. Create VM from ISO file - like virt-install.
7. Possibility to mark a VM as template and use it only for cloning and deploying new VMs. Should not be able to be started until marked as template - only cloning option available.
8. Swap the positions of Size and Free columns for the Volumes in the Host page. To me it seems more logical to be ordered Free / Total Size than Total Size / Free.
   Example:
   
9. Autostart on boot - mark the VM to be atustarted on host boot.
10. Ability to mark the disk of the VM to be deleted when the VM is deleted.
11. Images management from the web interface - upload and download images like ISO and IMG from and to the WEB interface and choose on which volume they should go.
12. Ability to extend (change) VM size - CPUs, RAM, Disk size, etc.

Please, share your thoughts!
Thank you!

IP of the VM is not listed

Latest version of vamango is installed in CentOS 7.6 using the following commands.

yum install -y yum-plugin-copr && yum copr -y enable subuk/vmango
yum install -y vmango
systemctl enable --now vmango

A new VM was created from CentOS-7-x86_64-GenericCloud-1901.qcow2. The IP of the VM was not listed in the vamango web page. virsh net-dhcp-leases vmango command showed the IP address correctly. After rebooting the VM, the IP was shown properly in the vmango web page too.

failed to start machine: Unsupported machine type

failed to start machine: virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: disk0,bootindex=1 -drive file=/opt/libvirt/images/test1_config.iso,format=raw,if=none,id=drive-ide0-1-0,readonly=on -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:56:6c:53,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 127.0.0.1:0 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on (process:44260): GLib-WARNING **: 16:25:31.918: gmem.c:489: custom memory allocation vtable not supported kvm-spice: -machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,dump-guest-core=off: Unsupported machine type Use -machine help to list supported machines!')

AltCloud

Hello, how about to participate in another cloud solution development? I really need some knowledge about kvm, qemu and libvirt-go capabilities.

feature request: post-create-hook script

I'd like to be able to write a script that would get called after the vm is created with the following parameters:

$1 name
$2 volume pool
$3 volume name

Bridge Network

Routed network with libvirt managed dhcp server. Bridged networks not supported due to impossibility to determine machine ip address.
But it's not true. You can use qemu-guest-agent to determine ip address

No valid certificate for https ?

Vmango.org has no valid certificate ?
Safari refuses entry

With kind regards,
Remke

Shows no provider or images

Hi, i just installed vmango. configured it and started the server i can login but cannot see any provider (LOCAL1) isn't showing up. i checked the storage pool. checked the network. restarted vmango but cannot see anything.

listen = "127.0.0.1:8000"
session_secret = "shdihvidi"
static_cache = "24h"
# trusted_proxies = ["127.0.0.1"]
# ssl_key = "test_ssl_key.pem"
# ssl_cert = "test_ssl_cert.pem"

hypervisor "LOCALjk" {
    url = "qemu:///system"
    image_storage_pool = "vmango-images"
    root_storage_pool = "default"
    network = "vmango"
    vm_template = "vm.xml.in"
    volume_template = "volume.xml.in"
}

user "admin" {
     password = "$2a$10$UJzjXoWvraZ3o8as7sULtO77KYbGBYjI/DkojHiuDqxmV8aLZiSXq"
}

plan "small" {
    memory = 512
    cpus = 1
    disk_size = 5
}

plan "medium" {
    memory = 512
    cpus = 1
    disk_size = 10
}

screenshot

Add license to project

Hello there,
I am the maintainer of the Arch Linux User Repository (AUR) packages for vmango (vmango and vmango-git). While packaging vmango for Arch Linux I couldn't find a license for this project. Of course you have the choice on which license to use. I personally prefer the GPL version 3 (my 2 cents).

Anyway. Awesome project! I am happy to maintain the AUR packages for this.

feature request: lxd/lxc

Support the management of lxc containers

PCI passthrough

Ability to passthrough specified pci host device to vm.

Add hostdev with vfio driver to devices xml section of vm.
Host configuration process should be documented in readme.
New configuration option to limit device list.
New pretty page with available host devices.
Display attached devices somewhere on the vm details page (parse hostdev element).

make problems

root@foobar:~/vmango# make && ./bin/vmango
go install github.com/go-bindata/go-bindata/go-bindata
can't load package: package github.com/go-bindata/go-bindata/go-bindata: cannot find package "github.com/go-bindata/go-bindata/go-bindata" in any of:
        /usr/lib/go-1.10/src/github.com/go-bindata/go-bindata/go-bindata (from $GOROOT)
        /root/vmango/src/github.com/go-bindata/go-bindata/go-bindata (from $GOPATH)
Makefile:41: recipe for target 'bin/go-bindata' failed
make: *** [bin/go-bindata] Error 1

Documentation regarding "Custom userdata for cloud-init"

Hello. Is there any documentation regarding "Custom userdata for cloud-init". I though that "Specify a userdata" field will do the job, but it seems to pass options as qemu custom fields, so passing it to qemu or something else which will query qemu, but not passing things to cloud-init / OS.

The vmango.org site is not accessible.

The https://vmango.org/docs/ redirects to github source repository page.

Improve host info page

• Swap the positions of Size and Free columns for the Volumes

Feature Request: Add support for VMware and oVirt

if possible can these be added?

feature request: oidc, ldap support and u2f

Hello,
Thanks for a great project.

It would be supper usefull to implement:

• oidc ( major clouds, generic oidc, keycloak -> https://github.com/dexidp/dex can be used)
• ldap
• u2f ( yubikey, google authenticator/otp, etc.. )

Or is this something to consider to potentional enterprise version?

GUI spinners should not go below 1

On the create page
CPUs, Ram, volume size, vlan

On the volumes page
Size

Vcpu pinning

Vcpu pinning is a tedious thing and should be automated.

New config option enable_vcpu_pinning which implies that all vcpus of all vms should be pinned to exactly one host cpu logical core.
If option is enabled and vm has no pins configured, warning should be displayed.
CPU pinning doesn't work without NUMA pinning.

Add ability to change cpu count and memory size

Currently it requires to delete vm and create a new one with same network interfaces and volumes.

feature request: user management from web

at least being able to change the admin password.

and isn't blowfish insecure, or did they fix that?

I found this page to generate a new hash: http://www.passwordtool.hu/blowfish-password-hash-generator

Improve size formatting in web interface

FYI: setup step... virsh pool-define-as

I had to use virsh pool-create-as

(but this may have had to do with this being on my test box where I've installed/uninstalled so much stuff. I'll be trying this on a clean install later.)

NUMA pinning

Only one NUMA can be selected.

Accessing Windows VM

I have created a Windows Server 2012 R2 Virtual Machine. Its cloud-base image was downloaded from https://cloudbase.it/windows-cloud-images/#download. The VM instance is up and running. vmango shows its IP in the machine details web page.

My question is, how can I access this VM through terminal or using DRP? For openstack users, the following command can be used to retrieve password for Admin user. Reference: https://cloudbase.it/windows-cloud-images/#download

nova get-password <instance> <SSH_private_key>

Does vmango have such support?

failed to fetch plan list: bucket "plans" not found

Hi,
After I start the server,the dashboard show 'failed to fetch plan list: bucket "plans" not found'.

Go binaries on release page?

Can be vmango go binaries included in assets on release page (https://github.com/subuk/vmango/releases) ?
Thanks

Error 500: failed to find root disk

500
Internal server error
failed to query provider &{LOCAL1 default %!s(*libvirt.Connect=&{0x1e60660}) %!s(*dal.LibvirtMachinerep=&{0xc42000e8b0 0xc42027a040 0xc42027a280 vmango default LOCAL1 []}) %!s(*dal.LibvirtImagerep=&{vmango-images 0xc42000e8b0})}: failed to find root disk

not sure what that means but i cannot open machines or providers tab

any ideas?

Proper developer/contributor documentation!

i don't see a documentation listing all the function and types, there should such documentation.

feature request: delete image when VM is deleted

maybe a checkbox in the confirmation dialog if you want to delete the image file and iso too?

Volume content management

Ability to download and upload volume content

feature request: API

It would be super usefull to implement API for further automation and integration with tools like Ansible, Terraform, etc ..

For example: major competitors ( +- same feature set for KVM/QEMU ) like https://github.com/mistio/mist-ce and https://github.com/cockpit-project/cockpit does not offer (afaik) full API for machine creation ( cloud-init is must have in this century, PXE and cloning is bit weird in 21.century .. )

I guess it depends also on potentional RBAC and #29

Hugepages support

Always add <hugepages/> xml element.

Readme section how to configure hugepages on the host for ubuntu and centos.

Problem connecting libvirtd socket

HI @subuk
I am having problem
Error: hostinfo failed: cannot acquire libvirt connection: cannot open libvirt connection: virError(Code=38, Domain=7, Message='Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied')
I am running ubuntu 18.04 and installed via the Ubuntu 18.04 repo.
I am not getting which user must I give permissions to the libvirtd socket? If I am using the default admin/admin I should create such user and give him rights to the socket or?
Please, explain!
Thanks!