kubernetes/metrics

Kubernetes metrics-related API types and clients

metrics

Kubernetes metrics API type definitions and clients.

Purpose

This repository contains type definitions and client code for the metrics APIs that Kubernetes makes use of. Depending on the API, the actual implementations live elsewhere.

Consumers of the metrics APIs can make use of this repository to access implementations of the APIs, while implementors should make use of this library when implementing their API servers.

APIs

This repository contains types and clients for several APIs.

Custom Metrics API

This API allows consumers to access arbitrary metrics which describe Kubernetes resources.

The API is intended to be implemented by monitoring pipeline vendors, on top of their metrics storage solutions.

If you want to implement this as an API server for this API, please see the kubernetes-sigs/custom-metrics-apiserver library, which contains the basic infrastructure required to set up such an API server.

Import Path: k8s.io/metrics/pkg/apis/custom_metrics.

Resource Metrics API

This API allows consumers to access resource metrics (CPU and memory) for pods and nodes.

The API is implemented by metrics-server and prometheus-adapter.

Import Path: k8s.io/metrics/pkg/apis/metrics.

Compatibility

The APIs in this repository follow the standard guarantees for Kubernetes APIs, and will follow Kubernetes releases.

Community, discussion, contribution, and support

Learn how to engage with the Kubernetes community on the community page.

You can reach the maintainers of this repository at:

• Slack: #sig-instrumentation channel, you can get an invite at slack.kubernetes.io.
• Mailing List

Code of Conduct

Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.

Contibution Guidelines

See CONTRIBUTING.md for more information.

metrics's Issues

The container.Usage.Cpu().AsInt64

Dear community:
I use this function to get one pod of a container's CPU,Memory

`
func PrintAllResourceOfPodContainers(podContainers []v1beta1.ContainerMetrics){

for _,container := range podContainers{

    fmt.Println("container: ",container)

    cpuQuantiy    := container.Usage.Cpu().AsApproximateFloat64()

    memQuantiy    := container.Usage.Memory().AsApproximateFloat64()

    memQuantiy     = memQuantiy / Mi

    stoQuantiy,ok := container.Usage.Storage().AsInt64()

    containerName := container.Name

    if !ok{
        return 
    }
    msg := fmt.Sprintf("Container Name: %s \n Cpu usage: %f \n memory usage: %f MB\n storage usage: %d\n",containerName,cpuQuantiy,memQuantiy,stoQuantiy)
    fmt.Println(msg)
}

}
`
and I get this output:

container: {free5gc-upf-container map[cpu:{{59076 -9} {<nil>} 59076n DecimalSI} memory:{{30302208 0} {<nil>} BinarySI}]}
According my best knowledge, I know 1CPU = 1000m = 1000 * 1000 n, so is the cpu:{{59076 -9}} is wrong ? I think it is -6 not -9,can anyone know what wrong with this output?

implement the Object interface of k8s.io/apimachinery/pkg/apis/meta/v1

Plan to use kubectl get to display the metrics of custom-metrics-apiserver, but the Object object of TableRow in the k8s.io/apimachinery/pkg/apis/meta/v1beta1 package needs to be implemented in k8s.io/apimachinery/pkg/apis/meta/v1.
So we need to implement it in custom_metrics.MetricValue.
/cc @dgrisonnet

Create a SECURITY_CONTACTS file.

As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.

The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".

Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)

Thanks so much, let me know if you have any questions.

(This issue was generated from a tool, apologies for any weirdness.)

[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md

Necessity of metrics server exposing metrics to kube apiserver.

As we know , kube apiserver periodically fetches status reports from kubelets, why should metrics server collect mertrics from kubelete and expose the same to kube apiserver ? Please explain.

Support mutli-cluster for HPA

Multi-cluster is a reasonable scenario in a production environment，but now，can not support it，can you support mutli-cluster for HPA ？

Need a document on how to use this for custom metrics api server

Hi

We are actually planning to implement this custom metrics API server on our cluster and use it for pod auto scaling. So, looking forward to an introductory document or a blog post on how to use this project.

Thanks.

some problems about heapster

Heapster has been archived. But I used heapster in my project ,and my k8s cluster upgrade 1.14 ,now l can't get any data from heapster. someone can tell me what can i do if i wnat to use heapster in 1.14

Add pod/node labels

I'm using metrics-server to keep track of certain CPU/memory usage, but I'd like to aggregate on pod labels.

I asked this over here kubernetes-sigs/metrics-server#347 and was told that I should ask it here.

Length restrictions on External Metrics

It appears that using the generic label in MetricIdentifier.Selector comes with the following validation

invalid label value: "prod-redacted-queue": must be no more than 63 characters

I think this might need to be a different struct as it's possible for the values you're looking for to be longer then 63 characters, and RFC-1123 doesn't come into play when looking up metrics by key value.

MarshalToSizedBuffer undefined (type *resource.Quantity has no field

Getting below error while running metrics program.

var kubeconfig, master string //empty, assuming inClusterConfig
config, err := clientcmd.BuildConfigFromFlags(master, kubeconfig)
if err != nil {
	panic(err)
}

mc, err := metrics.NewForConfig(config)
if err != nil {
	panic(err)
}

metricsList, err := mc.MetricsV1beta1().NodeMetricses().List(metav1.ListOptions{})

# k8s.io/metrics/pkg/apis/metrics/v1beta1
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:281:44: (*resource.Quantity)(&v).MarshalToSizedBuffer undefined (type *resource.Quantity has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:338:44: (*resource.Quantity)(&v).MarshalToSizedBuffer undefined (type *resource.Quantity has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:358:24: m.Window.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Duration has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:368:27: m.Timestamp.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Time has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:378:28: m.ObjectMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ObjectMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:425:26: m.ListMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ListMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:472:24: m.Window.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Duration has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:482:27: m.Timestamp.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Time has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:492:28: m.ObjectMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ObjectMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:539:26: m.ListMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ListMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:539:26: too many errors
# k8s.io/metrics/pkg/apis/metrics/v1alpha1
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:281:44: (*resource.Quantity)(&v).MarshalToSizedBuffer undefined (type *resource.Quantity has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:338:44: (*resource.Quantity)(&v).MarshalToSizedBuffer undefined (type *resource.Quantity has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:358:24: m.Window.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Duration has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:368:27: m.Timestamp.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Time has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:378:28: m.ObjectMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ObjectMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:425:26: m.ListMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ListMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:472:24: m.Window.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Duration has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:482:27: m.Timestamp.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Time has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:492:28: m.ObjectMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ObjectMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:539:26: m.ListMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ListMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:539:26: too many errors
make: *** [build-cma] Error 2

Deprecation of "github.com/gogo/protobuf"

"k8s.io/metrics" is using the deprecated "github.com/gogo/protobuf"
Is it possible to remove this deprecated module and replace it with a suitable alternative?

question: context is building custom controller for `NodeMetricsList` kind

wanted to ask how should I build a custom controller using kubebuilder
for watching this resource

metrics/pkg/apis/metrics/v1beta1/types.go

Lines 50 to 58 in 538a35d

	type NodeMetricsList struct {
	metav1.TypeMeta `json:",inline"`
	// Standard list metadata.
	// More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
	metav1.ListMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
	// List of node metrics.
	Items []NodeMetrics `json:"items" protobuf:"bytes,2,rep,name=items"`
	}

here are my kubbuilder commands which failed

kubebuilder init --domain kubesimplify.com --repo kubesimplify.com/ksctl
kubebuilder create api --group metrics.k8s.io --version v1beta1 --kind NodeMetricsList

but it is failing
code

func (r *NodeMetricsListReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	_ = log.FromContext(ctx)

	// TODO(user): your logic here

	fmt.Println("TRIGGERED CONTROLLER!!")

	return ctrl.Result{}, nil
}

// SetupWithManager sets up the controller with the Manager.
func (r *NodeMetricsListReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		// Uncomment the following line adding a pointer to an instance of the controlled resource as an argument
		For(&metricsServer.NodeMetrics{}).
		Complete(r)
}

Linter error

1. level=warning msg="[runner] Can't run linter goanalysis_metalinter: buildir: failed to load package cache: could not load export data: no export data for \"sigs.k8s.io/controller-runtime/pkg/cache\""
   level=error msg="Running error: 1 error occurred:\n\t* can't run linter goanalysis_metalinter: buildir: failed to load package cache: could not load export data: no export data for \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\n"

terminal

➜  ksctl-controller git:(main) ✗ make run
/home/dipankar/Onedrive/gitlab/go-playground/ksctl-controller/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/home/dipankar/Onedrive/gitlab/go-playground/ksctl-controller/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
# sigs.k8s.io/controller-runtime/pkg/cache
../../../../go/pkg/mod/sigs.k8s.io/[email protected]/pkg/cache/multi_namespace_cache.go:308:9: cannot use handles (variable of type map[string]"k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration) as "k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration value in return statement: map[string]"k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration does not implement "k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration (missing method HasSynced)
../../../../go/pkg/mod/sigs.k8s.io/[email protected]/pkg/cache/multi_namespace_cache.go:321:9: cannot use handles (variable of type map[string]"k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration) as "k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration value in return statement: map[string]"k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration does not implement "k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration (missing method HasSynced)
../../../../go/pkg/mod/sigs.k8s.io/[email protected]/pkg/cache/multi_namespace_cache.go:326:17: impossible type assertion: h.(map[string]toolscache.ResourceEventHandlerRegistration)
	map[string]"k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration does not implement "k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration (missing method HasSynced)
make: *** [Makefile:55: vet] Error 1

Fake metrics list only "pods" resources

While trying to make unitest for metrics and using "k8s.io/metrics/pkg/client/clientset/versioned/fake" the following was observed.

the object provided as input to NewSimpleClientset() is of type v1beta1.PodMetricsList. The addition of the object is of Kind: "PodMetrics" as this is extracted in ObjectKinds() and the schema.GroupVersionKind. At the end this results in resource

schema.GroupVersionResource{Group: "metrics.k8s.io", Version: "v1beta1", Resource: "podmetricses"}

The metrices are requested using :

metricsClient.MetricsV1beta1().PodMetricses(namespace).List(getContext(options), getK8sRequestOptions(options))

However when getting the pod metrics the podmetricsesResource is predefined as Resource: "pods" thus no data found when requesting the pod metricses

var podmetricsesResource = v1beta1.SchemeGroupVersion.WithResource("pods")

// Get takes name of the podMetrics, and returns the corresponding podMetrics object, and an error if there is any.
func (c *FakePodMetricses) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1beta1.PodMetrics, err error) {
	obj, err := c.Fake.
		Invokes(testing.NewGetAction(podmetricsesResource, c.ns, name), &v1beta1.PodMetrics{})

What type of object should be provided in NewSimpleClientset for v1beta1 version to store the kind as Resource: "pods" for podmetricsesResource in fakepodmetrics.go. Is this functionality correct?