Comments (6)
This seems very opinionated and not necessarily generalisable.
I'm a bit confused by this. It should be generalizable. The file names in the directory form the keys to the conf map and the values are the file content. The idea is to support any file name.
That being said, I'm likely doing a very poor job explaining the desired behavior. I'm happy to build a 3rd party provider to support this.
Thanks for the help.
from koanf.
Hi @jalaziz. Do these secret files have a specific format? Could you point to a sample? I'm not familiar with them.
from koanf.
@knadh no specific format. They typically contain whatever the secret is (be it a string, JSON, a PEM-encoded key, etc).
For example, you might have a file named /run/secrets/db_password
that contains a password in plaintext. Or you might have /run/secrets/db_creds
that may contain a JSON blob of the username and password.
This is way of passing secrets is common in containerization and used by:
- Docker
- Kubernetes Secrets Store CSI Driver (which supports AWS Secrets Manager, Vault, and several other solutions)
- Google Cloud Run
from koanf.
I'm a little confused by this. If secrets are just strings that can be in any file, JSON, YAML, TOML, or env, or even just a file with no format, how do you propose that a new provider handle that? How will the provider work with arbitrary data?
You can just use the existing providers and parses to read a file (YAML, JSON etc.) and just access that key, right? koanf.String("myapp.mysecret")
.
from koanf.
Maybe I'm mistaken, but the issue with the current providers and parsers is two fold:
- All current parsers expect structured data.
- The current file provider only supports a single file.
What I believe is needed is:
- A new "directory" provider that reads files into map keyed by file name. This would
koanf.String("db-secret")
to map to the value of a file and allow mapping the value to a struct key when unmarshalling. - A new provider or parser that maps the value of a provider to a specific map key (rather than the config map itself).
- If possible, some combination of a provider and parser that would allow you to map a directory of files to different "types" of values.
It actually be easier to talk about specific examples of what I'm trying to accomplish.
In the simplest case, I would like the value of koanf.String("myapp.foo")
to be the contents of a file named /var/run/secrets/foo
. I don't believe this is possible today because I can't seem to find a way to parse the contents of a file into a single key.
For example, if /var/run/secrets/foo
contained the simple text content of secret-value
. I would like koanf.String("foo")
to return secret-value
. The important detail here is that key in the config maps to a specific file on disk.
In the more complex case, given a directory (e.g. /run/secrets/
) with the following files:
.
├── bar // Contains basic text API Key
└── foo // Contains JSON value
I would like to be able to unmarshal into the following structs:
type Nested struct {
a string
b int64
}
type Config struct {
foo Nested
bar string
}
I think this case would be a more challenging given the current structure of providers and parsers as the parser would have to be determined lazily based on the expected type. A simple version would be to load all the files in the directory into a config map that looks like:
{ foo: "{\"a\": \"one\", \"b\": 2}", bar: "secret-value"}
This would then allow koanf.String("foo")
and koanf.String("bar")
to return the contents of the corresponding files and the end user would have to parse from there.
from koanf.
This seems very opinionated and not necessarily generalisable.
For example, if /var/run/secrets/foo contained the simple text content of secret-value. I would like koanf.String("foo") to return secret-value.
Someone would want that to be secrets/foo
, secrets/bar
etc. Mapping file names to key is thus arbitrary.
I think you can write a provider that does this for usecase, but it wouldn't make sense to merge it into koanf's core. You can publish this on your own repo and we can add it to the 3rd party providers list in the koanf docs.
from koanf.
Related Issues (20)
- How to stop file watcher? HOT 3
- Is it possible to define a parameter reference another parameter in mock.toml file? HOT 1
- How to retrieve/print the final YAML string after YAML files are loaded and merged? HOT 5
- Alternativ to pflag? HOT 4
- basicflag Provider does not exclude default values HOT 3
- Why not create new file when file not exists. HOT 1
- Read enviroments secrets from github HOT 2
- Can not parse environment variable HOT 1
- Switch from mitchellh/mapstructure to go-viper/mapstructure HOT 14
- Can't unmarshall env variable with multiple words
- Support for merging slices HOT 2
- File watching error when saving with vi/vim or neovim HOT 1
- How to populate a map[string]string from environment variables HOT 1
- When loading with providers/env, koanf.Unmarshal doesn't work. HOT 5
- koanf.Unmarshal doesn't work when loaded with env provider HOT 2
- Default values from Tags for missing config values HOT 2
- posflag provider overrides values even if flags are not set for special delimiters HOT 1
- Convert non-string type to string in StringMap() HOT 1
- feature: search file HOT 1
- Difficulty Handling Multi-Word Environment Variables HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from koanf.