I'm working on the ability for users to set configuration parameters locally (per project/repository) and I’m wondering about how we might handle the semantics of configuration, internally and (perhaps most importantly) in the CLI.
For context, we may eventually end up having several different types of configuration:
- Custom project parameters that are intended to be populated locally by each user of a project. This could for example apply to user-specific keys, flags that individual users can set while developing/debugging etc. and can in general be used to supply secrets (although we will also need to provide a specific secrets storage in the hosted platform).
- Custom project parameters that are stored remotely and shared across the team and all environments. For example authentication keys for 3rd party services, that are not specific to individual developers or environments.
- Custom project parameters that are stored remotely but are specific to a single development environment (which may or may not be shared across the team). For example authentication keys for 3rd party services that need to be different for individual environments (e.g. a specific key for an analytics service in the shared staging environment).
- Built-in local per-project configuration parameters, such as authentication info for the hosted platform, CLI preferences (e.g. defaulting to output JSON or YAML). The names/schemas of these parameters could be specified by plugins.
- Built-in project parameters that are stored remotely and shared across the team. Could apply to anything that should be kept secret but would otherwise belong in the
garden-project.yml
file, for example custom SSL certificates for the ingress endpoint.
- Built-in global parameters for the garden framework. This could again apply to authentication info for the hosted platform, CLI preferences (e.g. defaulting to output JSON or YAML) etc. I'm unsure if we want something like that, or if we should stick to just per-project/repo configuration.
Or to frame this another way, configuration parameters may be:
- Built-in at the framework level
- Defined by plugins and integrations/apps
- Custom (i.e. implicitly defined in the project config files via template strings)
And may have the following scopes:
- Account/Team (we may introduce a distinction between those at some point, for enterprise accounts with multiple teams)
- Project
- Environment
- User/local (should those have a distinction, i.e. remote user parameters and local user parameters?)
So that's all in all quite a few things to consider (and I may be missing something as well?). We won't implement all of the above in the short term, but what I'm grappling with now is how we would like to see this look in our CLI and config files, such that it is easy to understand and elegant, since it would be annoying to change substantially later.
To try and decompose this a bit, I'm going to raise some questions that I've bumped into, discuss each of those, and then try and converge on something that looks pretty good.
Do we want to have a semantic distinction between remotely hosted custom configuration variables and secrets?
Seeing as secrets are basically configuration variables except with a clear indication that they're, well, secret (meaning encrypted remotely etc.) I see no reason why you would want non-secret variables on the remote platform, I'm leaning towards having no distinction there, but rather considering just the two "dimensions" listed above.
When referencing these parameters, should we explicitly reference them with a scope (e.g. user.some.key
and team.some.key
) or implicitly collapse values based on scope (so just write some.key
and use the value defined in the most granular scope)? Should we allow both?
There's no cut and dry correct answer here. The former has the benefit of being simpler to implement, as well as being more explicit so it's less likely to cause confusion as to where something is configured. The latter in turn allows more flexible overrides at different levels of scope (e.g. having default values at the account level, but allowing users to override). Allowing both would raise some edge case issues, like having to disallow user
, team
etc. as top level names for parameters, and my feeling is I'd rather have it explicitly work one way or the other. That is unless we have another "scope" that called something like all
, combined
, collapsed
, coalesced
or something like that (ideas welcome), so you can explicitly choose to use the most granularly defined value.
How should we separate between built-in parameters, plugin parameters and custom/project parameters?
I think each of the above should have its own namespace at same same level, but a follow-on question could be whether other template resolvers (e.g. the one for getting environment variables in the user's environment) are also at the same level or whether these should fall under a top-level config.
prefix. I'm leaning towards the latter, for consistency and clarity, so parameter values could be accessed with ${config.<namespace>.<key>}
.
As for the namespace names, the built-in namespace can simply be called garden
. Plugins could have their own namespace, which will simply be the name of the plugin. I'm not 100% sure what to call the custom namespace. custom
doesn't sound quite right to me, but maybe it's fine + I can't think of a clear alternative. Another way could be to not have a namespace for custom parameters, and instead use some symbol prefix for built-in/plugin parameters, e.g. ${[email protected]}
. Looks a bit weird though, because it doesn't look like a normal identifier, no matter which prefix we use (except _
, which doesn't work because it makes it look like a private property).
Should there be a distinction between user config values and local config values?
This one is at least fairly easy. We can just start with local
and if we see a need for user configuration that is stored remotely per user, we can add it later under a user
scope.
Should there be a special indication of namespace and/or scope in the CLI when setting configuration values (e.g. Heroku-style garden config:local set bla=ble
as opposed to garden config set local.bla ble
)?
This may depend a bit on how we answer the above questions on namespaces/scope, but I'd lean towards "nah" and say keep it simple. If we make this distinction, we'd kinda need it to look similar in template strings for consistency, and I can't think of a solid way to do that or a good reason to.
... Anyway, by now you likely understand why I'm struggling a bit with this. 😖
I'll keep thinking on this and put in some suggestions on how this might look, but thoughts would be appreciated.