sustainable-computing-io/kepler-doc

Kepler uses eBPF to probe energy related system stats and exports as Prometheus metrics

kepler-doc's Issues

GPU Nvidia H100 PCIe Not Supported

I've deployed Kepler on a Kubernetes to monitor a cluster with a GPU node with a NIVIDIA H100 PCIe.

In the kepler logs from this node, I've this error. In parallel I'm monitoring this GPU with a dgcm-exporter instance and it can collect gpu energy consumption metrics correctly.

I0125 07:04:48.972351 1 power.go:86] Failed to collect GPU metrics, trying to initizalize again: failed to get processes' utilization on device {0x7f639b40bdf8}: Not Supported
I0125 07:04:48.972407 1 gpu_nvml.go:62] found 1 gpu devices
I0125 07:04:48.972416 1 gpu_nvml.go:73] GPU 0 NVIDIA H100 PCIe

Do you have an idea ?

Outline for kepler technical deep dive

Kepler Deep Dive

• Components and what they do

• What metrics are gathered on various systems

  • BM
  • VM
  • Linux standalone

• Why these metric?

• How is the power consumption attribution done?

• Explain the models. How the models are different and is there a right use case/scenario for when to apply a particular model over another?

  • AbsComponentModelWeight
  • AbsComponentPower
  • AbsModelWeight
  • AbsPower
  • DynComponentModelWeight
  • DynComponentPower
  • XGBoost

• Each model has three sub models BPFOnly, CgroupOnly, CounterOnly but we use only one of these models. Why is that?

Let's move to Mkdocs for easier contribution and integration

I found that Mkdocs Is easier for contributing in documentation than Sphinx for the following reasons:

• It is based on markdown format which we already use in github page and wiki. So that, we can copy&paste the content that we already put in README or Wiki directly.
• It can serve and see live change in the same theme that will show in github page.
• CI workflow is already available. Check https://squidfunk.github.io/mkdocs-material/publishing-your-site/#with-github-actions

Integrate CNCF blog article as comprehensive kepler overview

Hey,

after reading this CNCF blog article, I think it would make for a great starting point when people check out kepler.

We could then from there link to more specific design subpages, what do you think?

Installation on Prometheus Operator on Openshift can break Cluster Monitoring

The Deploy section of the Kepler Doc recommends installing Prometheus Operator. This would install 2 instances of Prometheus Operators that if not properly configured can render the cluster's in-platform monitoring unusable as the new Prometheus Operator can reconcile the prometheus-k8s in openshift-monitoring namespace.

MicroShift installation instructions missing | not working

The documentation refers to MicroShift as manifest install.
The manifest installation instructions are referring to a make command which does not exist if you clone the kepler git repo as instructed

Document for BMC

Hi @rootfs ,
for prepare kubecon kiosk for kubecon China 2023.
I hope we can have BMC related document update in this repo.
for both English and Chinese.

Document How we now have an SBOM for releases

Now that we have an SBOM in the releases we should document it for people's awareness.

The SBOM has just been produced for the latest release here: https://github.com/sustainable-computing-io/kepler/releases/tag/v0.5

This information would properly be best suited to the install.md

I will create a PR on Monday for discussion/approval

The documents are not rendered in Chinese

Env: Google Chrome Version 114.0.5735.106 (Official Build) (64-bit)

zh support for training

for kubecon China kiosk, hope to have zh support for model server and training process.

Include CNCF logo and title

Website footers must include trademark guidelines by either linking to Trademark Usage (directly or via a "Terms of service" page), or by including the following text: "The Linux Foundation® (TLF) has registered trademarks and uses trademarks. For a list of TLF trademarks, see Trademark Usage".

Set GH Actions for Sphinx

check if local cluster document is up to date

          Is this doc updated? I don't think we do local cluster set up using yaml anymore.

@SamYuan1990

Originally posted by @husky-parul in #70 (comment)

OpenSSF questions related for basic informations

Home page,

• The project website MUST succinctly describe what the software does (what problem does it solve?).
  Kepler (Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe energy related system stats and exports as Prometheus metric.
• The project website MUST provide information on how to: obtain, provide feedback (as bug reports or enhancements), and contribute to the software.
• The information on how to contribute SHOULD include the requirements for acceptable contributions (e.g., a reference to any required coding standard). (URL required)

Document metrics and their usage

This issue is to track sustainable-computing-io/kepler#334

If there is a copyright notice at the bottom of the page, copyright should be to the project authors or to CNCF, not the origin company. For details, see Copyright notices.

Website: Analytics transferred to [email protected]

Follow as per cncf/toc#1054 (comment)

Add platform validation to legend / table of contents

Hey,

I saw that #91 was merged but realized the corresponding page is not included in the legend / table of contents:
https://sustainable-computing.io/platform-validation/

Where would it fit best? Or should we create a new section for it?

Document Kepler-RPM install configuration options.

The idea is to document the configuration options for the kepler rpm install under /etc/kepler/kepler.config

Show rendered page when PR

It should be nice if we can have a rendered page when PR is pushed.
So that, reviewer can confirm the rendered result before approving the PR in the case that the pusher forget to do so (like me).

I was experienced that it is possible from kwok project before. It has a rendered page per PR.
https://deploy-preview-316--k8s-kwok.netlify.app/docs/adopters/

docs: Missing Figures

Building the documents shows several figures missing:

mkdocs build
:
WARNING -  Doc file 'installation/kepler-operator.md' contains a relative link '../fig/ocp_installation/kind_grafana.png', but the target
           'fig/ocp_installation/kind_grafana.png' is not found among documentation files.
:
WARNING -  Doc file 'installation/community-operator.zh.md' contains a relative link '../fig/ocp_installation/operator_installation_ocp_1_0.6.z.png', but the target
           'fig/ocp_installation/operator_installation_ocp_1_0.6.z.png' is not found among documentation files.
WARNING -  Doc file 'installation/community-operator.zh.md' contains a relative link '../fig/ocp_installation/operator_installation_ocp_7_0.6.z.png', but the target
           'fig/ocp_installation/operator_installation_ocp_7_0.6.z.png' is not found among documentation files.
WARNING -  Doc file 'installation/community-operator.zh.md' contains a relative link '../fig/ocp_installation/operator_installation_ocp_2_0.6.z.png', but the target
           'fig/ocp_installation/operator_installation_ocp_2_0.6.z.png' is not found among documentation files.
WARNING -  Doc file 'installation/community-operator.zh.md' contains a relative link '../fig/ocp_installation/operator_installation_ocp_5a_0.6.z.png', but the target
           'fig/ocp_installation/operator_installation_ocp_5a_0.6.z.png' is not found among documentation files.
WARNING -  Doc file 'installation/community-operator.zh.md' contains a relative link '../fig/ocp_installation/operator_installation_ocp_5b_0.6.z.png', but the target
           'fig/ocp_installation/operator_installation_ocp_5b_0.6.z.png' is not found among documentation files.
WARNING -  Doc file 'installation/community-operator.zh.md' contains a relative link '../fig/ocp_installation/operator_installation_ocp_6_0.6.z.png', but the target
           'fig/ocp_installation/operator_installation_ocp_6_0.6.z.png' is not found among documentation files.
WARNING -  Doc file 'installation/community-operator.zh.md' contains a relative link '../fig/ocp_installation/operator_installation_ocp_3_0.6.z.png', but the target
           'fig/ocp_installation/operator_installation_ocp_3_0.6.z.png' is not found among documentation files.
WARNING -  Doc file 'installation/kepler-operator.zh.md' contains a relative link '../fig/ocp_installation/kind_grafana.png', but the target
           'fig/ocp_installation/kind_grafana.png' is not found among documentation files.
:

@vprashar2929 - fig/ocp_installation/kind_grafana.png was removed in #148, should the link be updated/removed or the image restored?
@vprashar2929 - I think the rest were removed in #106, should the link be updated/removed or the image restored?

[CNCF Sandbox Onboarding] Track website tasks

This issue addresses the website tasks from Kepler being onboarded as a Sandbox project.
Main tracking issue: sustainable-computing-io/kepler#698

• Website: ensure LF footer is there and website guidelines followed (if your project doesn't have a dedicated website, please adopt those guidelines to the README file of your project on GitHub).
  • CNCF projects are strongly encouraged to host the source of their websites in an open source repository (and under the same organization) so that requests to change can be done via pull requests and the discussions are archived in a transparent manner.
  • It is OK to say that, e.g., “Prometheus was originally created by Soundcloud” or “Kubernetes builds upon 15 years of experience of running production workloads at Google,” but the origin company should not otherwise be referred to on the project homepage.
  • There should be no links or forms for capturing enterprise support leads. Instead, it is fine to have an enterprise support, commercial partners or similar page. Companies must be listed on that page in alphabetical order, or the order can be changed randomly on each page load. It’s OK to have different categories of support offered. Simple vetting by the project is needed to ensure that all companies listed really can provide the support promised. Projects are welcome to outsource this vetting to CNCF staff if it becomes a burden.
  • Links to companies offering support are expected to go a page that at least mentions support of the project. This can either be the company homepage or a project-specific landing page.
  • #52
  • #50
  • #53
• #54

Add documentation for how to enable a new hardware platform support

@rootfs ,
pre discussed with @marceloamaral , @jiangphcn , @jichenjc , @sunya-ch
personally I want to add a chapter in kepler-doc as enable a new hardware platform for kepler.
as a checklist for people when on board kepler with a new hardware platform as s390.
Here is the check list as overall design for this chapter, and assigned to people as task? and I hope we can make it with in this week, then submit to kubecorn as a proposal from our community.

• Over all description. (#60 )
  Phase 1
• build kepler code on specific hardware platform from binary to container image.(@SamYuan1990 )
• find out all existing metric as ebfp counter and cgroup to identify resource usage.(@SamYuan1990 , @marceloamaral , @jichenjc )
• find power API to play role as rapi or acpi.(@jiangphcn )
  Phase 2
• model training logic(@sunya-ch , @jiangphcn )
• model training docker image and deployment(@sunya-ch, @jiangphcn , @SamYuan1990 )

Hence, we can use an existing content as proposal to kubecorn.
WDYT?