cloud-bulldozer / go-commons Goto Github PK
View Code? Open in Web Editor NEWCode repository with all go common packages and libraries
License: Apache License 2.0
Code repository with all go common packages and libraries
License: Apache License 2.0
Currently the Grafana dashboards linked in the CPT dashboard are broken for the bare metal platform because they lack the masterNodeType field (https://github.com/cloud-bulldozer/go-commons/blob/main/ocp-metadata/ocp-metadata.go#L265).
It would be great to fill this field for bare metal deployments also, some standard key like i.e.: baremetal
of just any non empty string, i.e.: -
will do.
Looking at https://github.com/cloud-bulldozer/go-commons/blob/main/prometheus/prometheus.go#L89 which is used for the kube-burner ocp wrapper reporting mode, it looks like we are extracting all the datapoints over the timeseries and then feeding them into methods in stats package as per the aggregation required. It looks like prometheus supports aggregations natively (https://prometheus.io/docs/prometheus/latest/querying/functions/#aggregation_over_time) and we should switch over to using those for better consistency and confidence in our aggregations.
We should setup a CI for each tool using go-commons
to ensure changes to go-commons doesn't break functionality in our main tools.
Integrate Prometheus library in Go-Commons
Kube-burner's node-density uses the number of worker nodes to calculate the number of pods to deploy in the cluster.
With this change recently pushed, kube-burner's failes because the nodes in the CI cluster has the scheduler configured with mastersSchedulable: true
, hence the nodes are labeled as master and workers simultaneosly
go-commons/ocp-metadata/ocp-metadata.go
Lines 215 to 226 in 97b6154
We should fix this corner case.
Add more fields to metadata collection, the ones at https://github.com/cloud-bulldozer/e2e-benchmarking/blob/bf5ac71356e1f128f35cb231ad67e39729837345/utils/index.sh#L183C1-L190C53 can be potentially included
Im not sure about the actual benefits of indexing them along with the tests results though. wdyt @jtaleric @dry923 @afcollins , etc?
At the moment go-commons is an all-in-one module, this is coming with several caveats.
Cutting a new release affects all the libraries, this is not desired since a project could want to stick on an specific release of a library.
Creates too many dependencies, a project only using one of the libraries will inherit all go-commons dependencies.
Splitting the project in different modules comes also with some downsides, the versioning handling of the different modules will be the most problematic one.
Let's use this issue to discuss about the possibilities
The current readme does not have any data. We should add a good description of the repository, how to contribute, etc.
As a next step to standardize our (golang based) tooling we should enable this project to collect OCP cluster metadata. So different go based tools such as k8s-netperf, kube-burner, ocm-api-load... can use it and index use the same format of cluster metadata.
We can use kube-burner's cluster metadata collection as a starting point.
type clusterMetadata struct {
MetricName string `json:"metricName,omitempty"`
UUID string `json:"uuid"`
Platform string `json:"platform"`
OCPVersion string `json:"ocpVersion"`
K8SVersion string `json:"k8sVersion"`
MasterNodesType string `json:"masterNodesType"`
WorkerNodesType string `json:"workerNodesType"`
InfraNodesType string `json:"infraNodesType"`
WorkerNodesCount int `json:"workerNodesCount"`
InfraNodesCount int `json:"infraNodesCount"`
TotalNodes int `json:"totalNodes"`
SDNType string `json:"sdnType"`
Benchmark string `json:"benchmark"`
Timestamp time.Time `json:"timestamp"`
EndDate time.Time `json:"endDate"`
ClusterName string `json:"clusterName"`
Passed bool `json:"passed"`
Metadata map[string]interface{} `json:"metadata,omitempty"`
}
At the moment we're providing a elasticsearch comparison mechanism at
https://github.com/cloud-bulldozer/go-commons/blob/33ff99845ce3847d5b03177601cd7cb882029616/comparison/elastic.go#L47C4-L47C4
which uses the result from a ES query and compares it with the input value.
We need another method to perform thresholds comparisons with the passed value, including a tolerancy percent and comparison direction
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.