Comments (12)
those numbers are definitely looks high:
# HELP node_total_hourly_cost node_total_hourly_cost Total node cost per hour
# TYPE node_total_hourly_cost gauge
node_total_hourly_cost{arch="amd64",instance="kubectrl01a",instance_type="rke2",node="kubectrl01a",provider_id="rke2://kubectrl01a",region=""} 2.717651809769061e+08
node_total_hourly_cost{arch="amd64",instance="kubectrl01b",instance_type="rke2",node="kubectrl01b",provider_id="rke2://kubectrl01b",region=""} 2.71763839646976e+08
node_total_hourly_cost{arch="amd64",instance="kubectrl01c",instance_type="rke2",node="kubectrl01c",provider_id="rke2://kubectrl01c",region=""} 2.717649127106552e+08
node_total_hourly_cost{arch="amd64",instance="kubegateway01a",instance_type="rke2",node="kubegateway01a",provider_id="rke2://kubegateway01a",region=""} 1.6326061544289812e+07
node_total_hourly_cost{arch="amd64",instance="kubeworker01a",instance_type="rke2",node="kubeworker01a",provider_id="rke2://kubeworker01a",region=""} 2.717651809769061e+08
node_total_hourly_cost{arch="amd64",instance="kubeworker01b",instance_type="rke2",node="kubeworker01b",provider_id="rke2://kubeworker01b",region=""} 2.717630348506068e+08
node_total_hourly_cost{arch="amd64",instance="kubeworker01c",instance_type="rke2",node="kubeworker01c",provider_id="rke2://kubeworker01c",region=""} 2.717654492432893e+08
node_total_hourly_cost{arch="amd64",instance="kubeworker01d",instance_type="rke2",node="kubeworker01d",provider_id="rke2://kubeworker01d",region=""} 1.788777011721537e+10
node_total_hourly_cost{arch="amd64",instance="kubeworker01e",instance_type="rke2",node="kubeworker01e",provider_id="rke2://kubeworker01e",region=""} 1.7941222081961483e+10
node_total_hourly_cost{arch="amd64",instance="kubeworker01f",instance_type="rke2",node="kubeworker01f",provider_id="rke2://kubeworker01f",region=""} 1.788778317587711e+10
for the individual containers, for me the CPU looks more or less plausible. but the RAM costs are there way to high:
but if I click on "view right sizing" the numbers again looks fine:
Let me know if you need further informations.
from features-bugs.
Hi @aeggerd could you possible share the results of the metric
node_total_hourly_cost
?
As well as a secreenshot of the Assets page to see what is driving this cost?
from features-bugs.
Hi @aeggerd I am guessing something is wrong with the node total RAM reporting. Can you share the results of
kube_node_status_capacity_memory_bytes
from features-bugs.
those are the results from the metric:
in case it helps here are the metrics from k9s:
from features-bugs.
feels like the metric keeps increasing:
i still assume that it must be a bug in the version or within the setting we set during the deployment. Because we have this issue consistent in all of our deployed instances.
here are the prometheus metrics:
kube_node_status_capacity_memory_bytes
from features-bugs.
Hmm, those byte counts look reasonable. How about node_ram_hourly_cost ? Are you supplying any custom prices? If so, how?
from features-bugs.
those are the node_ram_hourly_cost
those are the settings from the configmap kubecost/kubecost-cost-analyzer
:
from features-bugs.
I also now deleted the deployment, and all of his configurations. But still the issue occurs.
The issue is occurring in all of our 7 clusters / installations
version we are using is: 2.1.1
Maybe this screenshot is can give so some glue:
from features-bugs.
@aeggerd can you share a little bit more about how kubecost is being installed as well as any logs in the cost-model container?
from features-bugs.
@aeggerd I have another report of this on rancher kubernetes. Can you try to set custom prices enabled to true here? https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values.yaml#L3159
from features-bugs.
@AjayTripathy i have now updated my values file according to your recommendation.
so far it looks good, but i would wait some more days, if stays correct or if it summarize up again
if you are interested, this is the helm values that we are using:
global:
prometheus:
enabled: false
fqdn: http://rancher-monitoring-prometheus.cattle-monitoring-system.svc:9090
grafana:
enabled: false
domainName: rancher-grafana.kubectrl01.XXXXXXX
scheme: https
prometheus:
kube-state-metrics:
disabled: true
nodeExporter:
enabled: false
serviceAccounts:
nodeExporter:
create: false
kubeStateMetrics:
enabled: false
prometheusRule:
enabled: true
serviceMonitor:
enabled: true
networkCosts:
enabled: false
networkCosts:
enabled: false
kubecostProductConfigs:
customPricesEnabled: true
defaultModelPricing:
enabled: true
CPU: "28.0"
spotCPU: "4.86"
RAM: "3.09"
spotRAM: "0.65"
GPU: "693.50"
spotGPU: "225.0"
storage: "0.04"
zoneNetworkEgress: "0.01"
regionNetworkEgress: "0.01"
internetNetworkEgress: "0.12"
ingress:
enabled: true
from features-bugs.
it is still working, with the customer pricing :)
from features-bugs.
Related Issues (20)
- [Feature] Support `LOG_FORMAT` and `LOG_LEVEL` in cluster controller and network costs
- [Bug] network monitoring is empty HOT 5
- [Bug] cloud integration - spot datafeed - no datas HOT 2
- [Bug] full diagnostics cannot access (No change from "Running Diagnostics...") HOT 9
- [Bug] Turndown schedule deleted a few seconds after being deployed HOT 2
- [Feature] Support Cilium - add parsing of conntrack information HOT 5
- [Feature] - Add ability to aggregate allocations by Provider ID HOT 1
- [Bug] Duckdb corruption after 2.2.5 upgrade HOT 7
- Request Right Sizing action created via Helm shows incorrect in UI HOT 13
- [Bug] cost-model image not available on aws ecr for release 2.3.2 HOT 1
- [Bug] Aggregator panics on QueryAssetCTE HOT 7
- [Bug] Cluster Controller stopped targeting namespace level annotations HOT 1
- [Bug] Alibaba Instance Price Error HOT 1
- [Bug] Allocation query fails due to a max concurrency semaphore error HOT 1
- You found an enterprise feature! HOT 1
- [Feature] Provide a configuration to restrict out of cluster cost dashboard HOT 1
- [Bug] Operation error Organizations: ListAccounts HOT 3
- [Bug] No data HOT 1
- [Feature] Support Kubernetes 1.30 in AWS Marketplace listing HOT 2
- [Bug] Crash Aggregator container HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from features-bugs.