Issue with configuration file to render the values from environment variables

[akoserwal]

Issue with configuration file to render the values from environment variables

[dosubot]

🤖

Hey @akoserwal, great to see you back! 😊 Hope you've been doing awesome since our last chat.

The issue you're experiencing is due to the way the default resolver in Kratos handles environment variables. The default resolver only supports flat keys, meaning it can only resolve environment variables that correspond to top-level keys in the configuration. It doesn't support nested keys like "data.testDB.endpoint".

Here's a snippet of the default resolver code for reference:

o := options{
		decoder:  defaultDecoder,
		resolver: defaultResolver,
		merge: func(dst, src interface{}) error {
			return mergo.Map(dst, src, mergo.WithOverride)
		},
	}

If you want to override nested keys with environment variables, you would need to implement a custom resolver that supports this feature. This custom resolver should be able to parse the environment variable names into nested keys.

Unfortunately, without the implementation of the defaultResolver, I can't provide a specific code example on how to modify it to support nested keys. However, the general idea would be to split the environment variable name on a certain character (like underscore or dot) and use the resulting parts as nested keys.

You can find more information about the default resolver and its limitations in the Kratos repository.

I hope this helps clarify the issue. If you have any further questions, feel free to ask.

Sources

• config/config.go
• transport/grpc/resolver/direct/builder.go

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[kvii]

I think here is a bug. Consider to change "$TEST_ENDPOINT" to "${ENDPOINT}" to avoid this temporary.

I made a reproduce project at local. Some of key codes:

server:
  http:
    addr: "${ADDR}"

c := config.New(
	config.WithSource(
		env.NewSource("TEST_"),
		file.NewSource(flagconf),
	),
)
c.Load()

var bc conf.Bootstrap
c.Scan(&bc)

fmt.Println(bc.Server.Http.Addr)
fmt.Println(c.Value("ADDR").String())
fmt.Println(c.Value("server.http.addr").String())

After I execute export TEST_ADDR=0.0.0.0:8000, codes run properly(print "0.0.0.0:8000").

I guess there has some mistakes of the format of placeholder. The comment say that the correct format is "${key:default}".

kratos/config/options.go

Lines 85 to 97 in 9106991

	// defaultResolver resolve placeholder in map value,
	// placeholder format in ${key:default}.
	func defaultResolver(input map[string]interface{}) error {
	mapper := func(name string) string {
	args := strings.SplitN(strings.TrimSpace(name), ":", 2) //nolint:gomnd
	if v, has := readValue(input, args[0]); has {
	s, _ := v.String()
	return s
	} else if len(args) > 1 { // default value
	return args[1]
	}
	return ""
	}

[kvii]

It's a mistake of doc. kratos don't support "$key" format by default. I found this test:

kratos/config/options_test.go

Lines 132 to 136 in 9106991

	{
	name: "test $PORT",
	path: "foo.bar.value2",
	expect: "$PORT",
	},

[akoserwal]

Thanks, @kvii, for clarifying.

It works if the export TEST_ADDR=0.0.0.0:8000 and use fmt.Println(bc.Server.Http.Addr)

For boolean, I am getting

Error
panic: proto: (line 1:133): invalid value for bool type: "true"

data:
  test:
    endpoint: "${ENDPOINT}"
    useTLS: "${TLS:false}"

export TEST_TLS=true

[kvii]

Same issue at #1565. Unfortunately, non-string types are not supported in Scan because that will include more complexity. You can use c.Value("data.test.useTLS").Bool() to get the bool value explicitly. But I think this doesn't match your case.