Giter Site home page Giter Site logo

ovirt / go-ovirt-client Goto Github PK

View Code? Open in Web Editor NEW
12.0 14.0 19.0 686 KB

An easy-to-use overlay for the Go SDK.

Home Page: https://pkg.go.dev/github.com/ovirt/go-ovirt-client

License: BSD 3-Clause "New" or "Revised" License

Go 99.36% Assembly 0.22% Smarty 0.42%
ovirt ovirt-engine api golang

go-ovirt-client's Introduction

go-ovirt-client: an easy-to-use overlay for the oVirt Go SDK

This library provides an easy-to-use overlay for the automatically generated Go SDK for oVirt. It does not replace the Go SDK. It implements the functions of the SDK only partially and is primarily used by the oVirt Terraform provider.

Using this library

To use this library you will have to include it as a Go module dependency:

go get github.com/ovirt/go-ovirt-client github.com/ovirt/go-ovirt-client-log/v3

You can then create a client instance like this:

package main

import (
	"crypto/x509"

	ovirtclient "github.com/ovirt/go-ovirt-client/v3"
	ovirtclientlog "github.com/ovirt/go-ovirt-client-log/v3"
)

func main() {
	// Create a logger that logs to the standard Go log here:
	logger := ovirtclientlog.NewGoLogger()

	// Create an ovirtclient.TLSProvider implementation. This allows for simple
	// TLS configuration.
	tls := ovirtclient.TLS()

	// Add certificates from an in-memory byte slice. Certificates must be in PEM format.
	tls.CACertsFromMemory(caCerts)

	// Add certificates from a single file. Certificates must be in PEM format.
	tls.CACertsFromFile("/path/to/file.pem")

	// Add certificates from a directory. Optionally, regular expressions can be passed that must match the file
	// names.
	tls.CACertsFromDir(
		"/path/to/certs",
		regexp.MustCompile(`\.pem`),
	)

	// Add system certificates. This doesn't work on Windows before Go 1.18.
	tls.CACertsFromSystem()

	// Add a custom cert pool as a source of certificates. This option is
	// incompatible with CACertsFromSystem.
	// tls.CACertsFromCertPool(x509.NewCertPool())

	// Disable certificate verification. This is a bad idea, please don't do this.
	tls.Insecure()

	// Create a new goVirt instance:
	client, err := ovirtclient.New(
		// URL to your oVirt engine API here:
		"https://your-ovirt-engine/ovirt-engine/api/",
		// Username here:
		"admin@internal",
		// Password here:
		"password-here",
		// Pass the TLS provider here:
		tls,
		// Pass the logger here:
		logger,
		// Pass in extra settings here. Must implement the ovirtclient.ExtraSettings interface.
		nil,
	)
	if err != nil {
		// Handle error, here in a really crude way:
		panic(err)
	}
	// Use client. Please use the code completion in your IDE to
	// discover the functions. Each is well documented.
	upload, err := client.StartUploadToNewDisk(
		//...
	)
	//....
}

Test helper

The test helper can work in two ways:

Either it sets up test fixtures in the mock client, or it sets up a live connection and identifies a usable storage domain, cluster, etc. for testing purposes.

The ovirtclient.NewMockTestHelper() function can be used to create a test helper with a mock client in the backend:

helper := ovirtclient.NewMockTestHelper(ovirtclientlog.NewNOOPLogger())

The easiest way to set up the test helper for a live connection is by using environment variables. To do that, you can use the ovirtclient.NewLiveTestHelperFromEnv() function:

helper := ovirtclient.NewLiveTestHelperFromEnv(ovirtclientlog.NewNOOPLogger())

This function will inspect environment variables to determine if a connection to a live oVirt engine can be estabilshed. The following environment variables are supported:

  • OVIRT_URL: URL of the oVirt engine API.
  • OVIRT_USERNAME: The username for the oVirt engine.
  • OVIRT_PASSWORD: The password for the oVirt engine
  • OVIRT_CAFILE: A file containing the CA certificate in PEM format.
  • OVIRT_CA_BUNDLE: Provide the CA certificate in PEM format directly.
  • OVIRT_INSECURE: Disable certificate verification if set. Not recommended.
  • OVIRT_CLUSTER_ID: The cluster to use for testing. Will be automatically chosen if not provided.
  • OVIRT_BLANK_TEMPLATE_ID: ID of the blank template. Will be automatically chosen if not provided.
  • OVIRT_STORAGE_DOMAIN_ID: Storage domain to use for testing. Will be automatically chosen if not provided.
  • OVIRT_VNIC_PROFILE_ID: VNIC profile to use for testing. Will be automatically chosen if not provided.

You can also create the test helper manually:

import (
	"os"
	"testing"

	ovirtclient "github.com/ovirt/go-ovirt-client/v3"
	ovirtclientlog "github.com/ovirt/go-ovirt-client-log/v3"
)

func TestSomething(t *testing.T) {
	// Create a logger that logs to the standard Go log here
	logger := ovirtclientlog.NewTestLogger(t)

	// Set to true to use in-memory mock, otherwise this will use a live connection
	isMock := true

	// The following parameters define which infrastructure parts to use for testing
	params := ovirtclient.TestHelperParams().
		WithClusterID(ovirtclient.ClusterID(os.Getenv("OVIRT_CLUSTER_ID"))).
		WithBlankTemplateID(ovirtclient.TemplateID(os.Getenv("OVIRT_BLANK_TEMPLATE_ID"))).
		WithStorageDomainID(ovirtclient.StorageDomainID(os.Getenv("OVIRT_STORAGE_DOMAIN_ID"))).
		WithSecondaryStorageDomainID(ovirtclient.StorageDomainID(os.Getenv("OVIRT_SECONDARY_STORAGE_DOMAIN_ID"))).
		WithVNICProfileID(ovirtclient.VNICProfileID(os.Getenv("OVIRT_VNIC_PROFILE_ID")))

	// Create the test helper
	helper, err := ovirtclient.NewTestHelper(
		"https://localhost/ovirt-engine/api",
		"admin@internal",
		"super-secret",
		// Leave these empty for auto-detection / fixture setup
		params,
		ovirtclient.TLS().CACertsFromSystem(),
		isMock,
		logger,
	)
	if err != nil {
		t.Fatal(err)
	}
	// Fetch the cluster ID for testing
	clusterID := helper.GetClusterID()
	//...
}

Tip: You can use any logger that satisfies the Logger interface described in go-ovirt-client-log

Retries

This library attempts to retry API calls that can be retried if possible. Each function has a sensible retry policy. However, you may want to customize the retries by passing one or more retry flags. The following retry flags are supported:

  • ovirtclient.ContextStrategy(ctx): this strategy will stop retries when the context parameter is canceled.
  • ovirtclient.ExponentialBackoff(factor): this strategy adds a wait time after each time, which is increased by the given factor on each try. The default is a backoff with a factor of 2.
  • ovirtclient.AutoRetry(): this strategy will cancel retries if the error in question is a permanent error. This is enabled by default.
  • ovirtclient.MaxTries(tries): this strategy will abort retries if a maximum number of tries is reached. On complex calls the retries are counted per underlying API call.
  • ovirtclient.Timeout(duration): this strategy will abort retries if a certain time has been elapsed for the higher level call.
  • ovirtclient.CallTimeout(duration): this strategy will abort retries if a certain underlying API call takes longer than the specified duration.

Mock client

This library also provides a mock oVirt client that doesn't need working oVirt engine to function. It stores all information in-memory and simulates a working oVirt system. You can instantiate the mock client like so:

client := ovirtclient.NewMock()

We recommend using the ovirtclient.Client interface as a means to declare it as a dependency in your factory so you can pass both the mock and the real connection as a parameter:

func NewMyoVirtUsingUtility(
    client ovirtclient.Client,
) *myOVirtUsingUtility {
    return &myOVirtUsingUtility{
        client: client,
    }
}

FAQ

Why doesn't the library return the underlying oVirt SDK objects?

It's a painful decision we made. We want to encourage anyone who needs a certain function to submit a PR instead of simply relying on the SDK objects. This will lead to some overhead when a new function needs to be added, but leads to cleaner code in the end and makes this library more comprehensive. It also makes it possible to create the mock client, which would not be possibly if we had to simulate all parts of the oVirt engine.

If you need to access the oVirt SDK client you can do so from the ovirtclient.New() function:

client, err := ovirtclient.New(
    //...
)
if err != nil {
    //...
}
sdkClient := client.GetSDKClient()

You can also get a properly preconfigured HTTP client if you need it:

httpClient := client.GetHTTPClient()

๐Ÿšง Warning: If your code relies on the SDK or HTTP clients you will not be able to use the mock functionality described above for testing.

Contributing

You want to help out? Awesome! Please head over to our contribution guide, which explains how this library is built in detail.

go-ovirt-client's People

Contributors

amberdante avatar darth-mera avatar dependabot[bot] avatar emesika avatar engelmi avatar eslutsky avatar gal-zaidman avatar janosdebugs avatar mgold1234 avatar michalskrivanek avatar mnecas avatar mrnold avatar vatson112 avatar vjuranek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-ovirt-client's Issues

Constant overflows uint on 32 bit platforms

Describe the bug

When building on 32 bit platforms here is an error message appears:

newmock.go:43:3: constant 10737418240 overflows uint

Expected behavior

Since the storage is provided in bytes all storage-related numbers should be provided in uint64 and not uint.

Simulate VNIC name rules

There cannot be two VNICs on the same network with the same name. The mock implementation should reflect this.

Test failure on missing initialization

Describe the bug

When the oVirt Engine has a VM without initialization the listing fails with the following error:

bug: non-retryable error encountered while getting vm 7eb20a91-58bf-4c53-b12e-26b08deec768, giving up (bug: failed to convert vm 7eb20a91-58bf-4c53-b12e-26b08deec768 (field_missing: no initialization field found on VM object))

To Reproduce

If you can please provide the steps to reproduce the issue.

  1. Create a VM without initialization.
  2. Run the test suite.

Expected behavior

Tests should not fail.

Parallel image upload via ImageIO

Please describe what you would like to see

Image uploads should be parallelized if the source supports seeking to take advantage of faster uploads.

Please describe the solution you'd like

Fully implement ImageIO

Please describe your use case

Faster OCP installations on oVirt.

Add sparse support on VM creation

Please describe what you would like to see

On VM creation I would like to be able to specify that the disks from the template should be sparse or not.

Please describe the solution you'd like

The VM creation should have an optional parameter to toggle setting the template disks sparse or non-sparse.

Please describe your use case

The OpenShift integration needs to be able to create sparse disks.

add ReportedDevice and FindVirtualMachineIP methods

Please describe what you would like to see

for cluster-api we need to find each VM IP address (management)

  • add to NIC client reporteddevices slice (IP address string).
  • expose a method to FindVirtualMachineIP with logic to determine which is the management.

FindVirtualMachineIP(id string, excludeAddr map[string]int) (string, error)

  • id - vm id
  • excludeaddr - address to be excluded from the selection

example:
https://github.com/openshift/cluster-api-provider-ovirt/blob/70617f6447ca544a79850f8fbe5ec41479e7cb3e/pkg/clients/ovirt/impl.go#L394

VM start/stop/shutdown and wait

Please describe what you would like to see

Start, stop, shutdown a VM and wait for status.

Please describe the solution you'd like

It should be possible to start, stop, and shutdown a VM. It should also be possible to wait for the VM to reach a certain status.

create New VM on a Different storageDomain then set in a template

Please describe what you would like to see

during VM creation we would like to specify target storageDomainID

required engine API calls

  1. Search for template (to get the disk ID)
  2. get Disk information (to get the SDs)
  3. copy the disks between SDs
  4. create the VM

protype code

	sdkClient := client.GetSDKClient()
	newSD := "abc480ae-498f-42ad-ad84-e8f1380eb79b"
	vmName := "new-sd-vmaa-2"
	clusterName := "Default"
	templateName := "centos-7"

	// disk exists in the new SD
	diskExistsInSD := false


	//search for template name
	var templateID string

	templatesService := sdkClient.SystemService().TemplatesService()
	tpsResp, err := templatesService.List().Search(templateName).Send()
	if err != nil {
		logger.Errorf("Failed to search templates, reason: %v\n", err)
		return
	}
	tpSlice, _ := tpsResp.Templates()

	for _, tp := range tpSlice.Slice() {
		templateID = tp.MustId()
		break

	}

	//find the master diskID  from the template
	var diskId string
	if templateID != "" {
		tpService := templatesService.TemplateService(templateID)
		tpGetResp, _ := tpService.Get().Send()

		tp, _ := tpGetResp.Template()
		disks, _ := sdkClient.FollowLink(tp.MustDiskAttachments())
		if disks, ok := disks.(*ovirtsdk.DiskAttachmentSlice); ok {
			disk := disks.Slice()[0].MustDisk()
			logger.Debugf("ddd %v",disk.MustId())
			diskId =  disk.MustId()
		}
	}

	// check if disk contains our new SD
	dsk := sdkClient.SystemService().DisksService().DiskService(diskId)
	sds := dsk.Get().MustSend().MustDisk().MustStorageDomains()

	for _, sd := range sds.Slice() {
		if sd.MustId() == newSD {
			diskExistsInSD=true
		}
	}

	// if disk doesnt exists in the new storage domain copy it
	if diskExistsInSD == false {
		_, err = dsk.Copy().StorageDomain(ovirtsdk.NewStorageDomainBuilder().Id(newSD).MustBuild()).Send()
		if err != nil {
			logger.Errorf("Failed to copy disk to different SD, reason: %v\n", err)
			return
		}
	}

	//create VM  based on the new template Disk
	vmsService := sdkClient.SystemService().VmsService()

	_ , err = vmsService.Add().
		Vm(
		ovirtsdk.NewVmBuilder().Name(vmName).
			Cluster(ovirtsdk.NewClusterBuilder().Name(clusterName).MustBuild()).
			Template(ovirtsdk.NewTemplateBuilder().Id(templateID).MustBuild()).
			StorageDomain(ovirtsdk.NewStorageDomainBuilder().Id(newSD).MustBuild()).
			MustBuild()).Send()

	if err != nil {
		logger.Errorf("error adding new vm : %v ",err)
	}

NIC unique names

Network interfaces need to have unique names. This should be reflected in the mock.

when creating a new VM from template specify optional disk type

// Clone makes sure that the disks are cloned from the template and are not linked.
// Defaults to false.
// +optional
Clone *bool json:"clone,omitempty"

// Sparse indicates that sparse provisioning should not be used and disks should be preallocated.
// Defaults to false.
// +optional
Sparse *bool json:"sparse,omitempty"

// Format is the disk format that the disks are in. Can be "cow" or "raw". "raw" disables several features that
// may be needed, such as incremental backups. Defaults to "cow".
// +optional
Format string json:"format,omitempty"

add AffinityGroup support

Please describe what you would like to see

add AffinityGroupClient support

this PR can be used for reference #63

support for AutoPinning - NewVmPlacementPolicyBuilder for all the hosts in the cluster

for each VM :

  • we need to set a list of Hosts that Can run this VM (migratable) with similar CPU pinning option (taken from the hosts in the same cluster) [0]
  • have placement_policy string, currently, the policy is not stored anywhere [1]

[0] https://github.com/openshift/cluster-api-provider-ovirt/blob/3643d913dcfd76ad84987e6295672b46bbf8dcac/pkg/clients/ovirt/impl.go#L90-L109

[1]

vmPlacementPolicy.Affinity()

Prevent disk lockups on sparse/raw on NFS

Describe the bug

When creating disks with sparse/raw on NFS the real oVirt Engine will lock up and not release the disk. We should prevent that.

To Reproduce

If you can please provide the steps to reproduce the issue.

  1. Create a disk with sparse/raw on NFS
  2. Check the VDSM logs

Expected behavior

go-ovirt-client should prevent this.

Create Function for creating new templates + tests + mockup

Please describe what you would like to see

A clear and concise description of what you would like to see improved.

Please describe the solution you'd like

A clear and concise description of what you want to happen.

Please describe your use case

A clear and concise description of how you would use this feature.

add more optional disk parameters when creating new Disks

// Clone makes sure that the disks are cloned from the template and are not linked.
// Defaults to false.
// +optional
Clone *bool `json:"clone,omitempty"`

// Sparse indicates that sparse provisioning should not be used and disks should be preallocated.
// Defaults to false.
// +optional
Sparse *bool `json:"sparse,omitempty"`

// Format is the disk format that the disks are in. Can be "cow" or "raw". "raw" disables several features that
// may be needed, such as incremental backups. Defaults to "cow".
// +optional
Format string `json:"format,omitempty"`

disk update operation is not checking disk status

Describe the bug

Please enter a clear and concise description of what the bug is.

To Reproduce

If you can please provide the steps to reproduce the issue.

  1. ...
  2. ...
  3. ...

Expected behavior

Please describe what you would expect to happen.

add support for tags

Please describe what you would like to see

add the ability to manage tags in the engine.

tags are used during ocp-on-rhv deployment to mark cluster VMs and ephemeral bootstrap VM.

in ocp-on-rhv case:
tags are attached to all the OCP VMs and used when there is a need to remove them. (cluster destroy)
https://github.com/openshift/installer/blob/4a00fa716eb4f548e7a4648fe8528084771217e7/pkg/destroy/ovirt/destroyer.go#L57

example tags returned from engine with single cluster installed:

<tags>
<tag href="/ovirt-engine/api/tags/31c44901-acbd-4d9d-9684-b31872c1c06c" id="31c44901-acbd-4d9d-9684-b31872c1c06c">
<name>ovirt10-zbgjn</name>
<description/>
<parent href="/ovirt-engine/api/tags/00000000-0000-0000-0000-000000000000" id="00000000-0000-0000-0000-000000000000"/>
</tag>
<tag href="/ovirt-engine/api/tags/00000000-0000-0000-0000-000000000000" id="00000000-0000-0000-0000-000000000000">
<name>root</name>
<description>root</description>
</tag>
</tags>

we can use this PR for reference https://github.com/oVirt/go-ovirt-client/pull/63/files

Add format support to VM creation

Please describe what you would like to see

On VM creation I should be able to specify the disk format for the disks from the template.

Please describe the solution you'd like

I would like to see a format optional parameter for the template disks.

Please describe your use case

OpenShift needs to be able to create VMs with RAW and QCOW2 disks independently of the template disk format.

Not identifying hot-plug issues

The oVirt Engine can issue an error when it fails to hot-plug a disk. This error should be identified with its own error code.

E1021 09:17:11.828831       1 server.go:124] /csi.v1.Controller/ControllerPublishVolume returned with error: failed creating disk attachment, error: timeout: timeout while attaching disk 080fd0e5-baad-4c47-919f-9a72f3b05dca to vm d1e99420-c9eb-42ad-827c-393b22d22adb (generic_error: failed to attach disk 080fd0e5-baad-4c47-919f-9a72f3b05dca to VM d1e99420-c9eb-42ad-827c-393b22d22adb using virtio_scsi (Fault reason is "Operation Failed". Fault detail is "[Failed to hot-plug disk]". HTTP response code is "400". HTTP response message is "400 Bad Request".))

add a InstanceTypeId to a VM

string id - describes pre-set hardware configuration of the virtual machines,
cannot be combined with individual Memory/CPU VM config.

image

Image download commands

I would like to use the client as a replacement for ovirt sdk in CDI. Here you can find list of commands here.

Currently we use the sdk to import disks from RHV to CNV and we had to implement bunch of interfaces to be able to mock the calls during tests. This client would help us to delete good amount of code without changing functionality.

implement GetTemplateByName in template_client

we need this method for simple Template lookup by name:

ovirtclient.Template, err = GetTemplateByName(vmName)

should return a single Template (throw an error if more than 1 are found)
return nil if none are found.

Auto-recover from invalid_grant failure

Describe the bug

When an ovirt connection has been alive for too long it may run into this error:

failed to parse oVirt Engine fault response: <html><head><title>Error</title></head><body>invalid_grant: The provided authorization grant for the auth code has expired.</body></html> (Tag not matched: expect <fault> but got <html>

Expected behavior

go-ovirt-client should attempt to reauthenticate.

ListVMTags API call

Please describe what you would like to see

We need an API call to list the tags of a VM, or the VMs belonging to a tag.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.