Expand server/agent config reference to include full config.yaml spec on consolidated page

[rancher]

Expand server/agent config reference to include full config.yaml spec on consolidated page

[brandond]

There is no "consolidated" view of the yaml files or their syntax. There is no schema for the config files. The YAML keys are just just translated directly to CLI flags internally, and appended to the CLI args, as described at https://docs.rke2.io/install/configuration#configuration-file

The configuration file parameters map directly to CLI arguments, with repeatable CLI arguments being represented as YAML lists. Boolean flags are represented as true or false in the YAML file.

For this reason, the CLI flags remain the authoritative list of keys that can be set in the config file.

[buzzsurfr]

The YAML keys are just just translated directly to CLI flags internally, and appended to the CLI args

Even so, I would like for that translation to be explicit, including names of the YAML properties, etc. That way, I could review my config.yaml file instead of the provided flags/arguments to determine how a cluster is setup.

[brandond]

I'm not sure I see the value in maintaining the same list in two formats. What currently prevents you from searching for the yaml keys in the server or agent config reference, and how would this be improved if they were identically listed out in a similar table for the config YAML?

[buzzsurfr]

If I had a config.yaml file and want to review/compare with another config.yaml file, I would then have to search for the different commands between the two (assuming the two config.yaml files are distinct) and either match the syntax from the first or lookup the correct syntax for the second. With a consolidated list, I can use a single page to determine the differences.

I also don't think it would be necessary to maintain two formats of the same list, but rather to generate one format from another format. The logic for un-marshalling the YAML is already in code (in order to translate to CLI syntax), so ideally use that to generate the spec.

[brandond]

either match the syntax from the first or lookup the correct syntax for the second. With a consolidated list, I can use a single page to determine the differences.

I'm not understanding what you have to look up across multiple pages. All the agent flags are also valid for servers (as the server embeds a local agent), so if in doubt, just look at the list of server flags. If something differs between config files, then find the key in the flag list and see what it does.

[buzzsurfr]

If something differs between config files, then find the key in the flag list and see what it does

That's where the optimization exists.

For background: I will be deploying tens to hundreds of RKE2 clusters across an enterprise. On some clusters, I'll need to disable the snapshot controller (side note: that is not listed on the server configuration reference). On other clusters I'll need to change the default CIDRs to avoid IP conflicts, and some clusters may have different node taints on them based on external factors (dev/test/prod, etc.)

I can lookup each one of those items separately and get the config.yaml for it, but because there's not a complete example for a config.yaml file, then I'm left to guess whether the syntax is correct. If I were to use CLI arguments, I could rely on the config references alone.

Thinking about this from a declarative context, this would be similar to having a values.yaml file as an example in helm charts. While it's not fully detailed, it can serve as a template for using base configurations in a helm release, and the same would apply to config.yaml files in RKE2.

A good example of the values.yaml for a helm chart is https://github.com/rancher/rke2-charts/blob/main-source/packages/rke2-canal/charts/values.yaml, where basic info is shared and those config items can be deleted or commented but gives a complete view of all configuration items in a given cluster.

[brandond]

The logic for un-marshalling the YAML is already in code (in order to translate to CLI syntax), so ideally use that to generate the spec.

I don't think you're quite getting what I mean when I say that the YAML is schemaless. It is literally just unmarshalled to a map[string]interface{}, converted pairwise from key: value to --key=value, and appended to os.Args[]. If you put an invalid key in the YAML, you will not get an in error from the yaml parser, you will get an invalid flag error from the arg parser.

On some clusters, I'll need to disable the snapshot controller (side note: that is not listed on the server configuration reference)
--disable value (components) Do not deploy packaged components and delete any deployed components (valid items: rke2-coredns, rke2-ingress-nginx, rke2-metrics-server)

You are correct that it's not listed the example list of things that can be disabled, as we don't update the CLI flag help text every time we add a packaged component. However, the fact that any packaged component can be disabled is covered elsewhere: https://docs.rke2.io/advanced?#disabling-server-charts

there's not a complete example for a config.yaml file

There is, as I've said before it is just the flag list. There are no yaml keys that are not valid flags. There are no valid flags that are not yaml keys.

this would be similar to having a values.yaml file as an example in helm charts.

it sounds like what you'd really like to see is an example of the default server or agent configuration, with every option's default value explicitly specified, in YAML format? Again, this is non-trivial to maintain, as the defaulting is handled based on CLI args, not YAML keys, and occurs across several stages within the server and agent setup process, partially based on the state of the node at runtime.

[buzzsurfr]

Does that mean logical grouping is not used in config.yaml either?

Take the S3 Compatible API Support as an example. Reading the options, this means the config.yaml would look like this:

etcd-disable-snapshots: true
etcd-snapshot-schedule-cron: "0 */12 * * *"
etcd-snapshot-retention: 5
etcd-snapshot-dir: "${data-dir}/db/snapshots"
etcd-snapshot-compress: true

and not this:

etcd:
  disable-snapshots: true
  snapshot:  
    schedule-cron: "0 */12 * * *"
    retention: 5
    dir: "${data-dir}/db/snapshots"
    compress: true

While you are correct that by RTFM it should be the former--many Kubernetes/cloud native admins will look at the config file and expect the latter by default. Having the explicit config file with all of the values listed would ease that complication.

It is literally just unmarshalled to a map[string]interface{}, converted pairwise from key: value to --key=value, and appended to os.Args[]

I am not suggesting to maintain both formats--that would be wasteful and prone to more human error. However, if it's as simple as you stated, then can we generate one from the other?

[brandond]

Reading the options, this means the config.yaml would look like this:
and not this:

Now you're getting it. The keys are literally just the CLI flags, with an int, string, bool, or list as the value, depending on what value the CLI flag would take. There are no complex nested fields or prefix groupings; was there something you saw in the docs that suggested that was possible?

[buzzsurfr]

I'm not sure where you got that from.

Sadly, from spending too much time with Kubernetes. It's a default position for me, but shared by others that also work on Kubernetes. While I am speculating that others see it this way, I don't know for certain. I'm leaning on my experience with helm and other cloud-native configuration items as a basis for my hypothesis.

I'll agree it seems silly--but it would help. I'm all in favor of generating that snippet instead of having to manually maintain it.

On some clusters, I'll need to disable the snapshot controller (side note: that is not listed on the server configuration reference)
--disable value (components) Do not deploy packaged components and delete any deployed components (valid items: rke2-coredns, rke2-ingress-nginx, rke2-metrics-server)

You are correct that it's not listed the example list of things that can be disabled, as we don't update the CLI flag help text every time we add a packaged component. However, the fact that any packaged component can be disabled is covered elsewhere: https://docs.rke2.io/advanced?#disabling-server-charts

Back to the disable, should we then change the CLI message to say that valid items are found at {{website}}. Stating "Valid: 1, 2, 3" implies that only those three are valid.

[brandond]

If we listed a full spec of the config keys it would literally just be a 1-1 copy of the current list dumped from the CLI help:

CLI	YAML File	Description
--debug	debug: true	(logging) Turn on debug logs [$RKE2_DEBUG]
--bind-address value	bind-address: value	(listener) rke2 bind address (default: 0.0.0.0)
--advertise-address value	advertise-address: value	(listener) IPv4 address that apiserver uses to advertise to members of the cluster (default: node-external-ip/node-ip)
--tls-san value	tls-san: [value]	(listener) Add additional hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the server TLS cert
--data-dir value	data-dir: value	(data) Folder to hold state (default: "/var/lib/rancher/rke2")
--cluster-cidr value	cluster-cidr: [value]	(networking) IPv4/IPv6 network CIDRs to use for pod IPs (default: 10.42.0.0/16)

and so on.

Providing full values.yaml style dump of the post-processed default configuration is non-trivial to maintain; currently (as you noticed) we struggle to keep even just the flag list up to date with what the binary actually supports.

[brandond]

Back to the disable, should we then change the CLI message to say that valid items are found at {{website}}. Stating "Valid: 1, 2, 3" implies that only those three are valid.

The RKE2 docs haven't been synced with some changes from K3s, but the text at https://docs.k3s.io/installation/packaged-components#disabling-manifests does a good job of describing what --disable actually does.

AddOns for any additional manifests placed in the manifests directory can be disabled with the --disable flag. Disabled AddOns are actively uninstalled from the cluster, and the source files deleted from the manifests directory.