Lack of Support for Additional Conformance Test Flags

[rjsadow]

Lack of Support for Additional Conformance Test Flags

[rjsadow]

I believe we'll need to pass any extra args we take in via flags to the pod as an array of args. The E2E keys only work for a small subset of variables (https://github.com/kubernetes/kubernetes/blob/02aaad0de94708b00e95ba6e809ac07d34eff61c/test/conformance/image/go-runner/const.go#L35-L42) and don't have a direct one for one mapping to all possible conformance flags listed here https://github.com/kubernetes/kubernetes/blob/9b5e2dc54d2315402a462e75c168bfc4ce28ecf3/test/e2e/framework/test_context.go#L349-L466

[rjsadow]

I don't think it's reasonable or maintainable to keep a consistent mapping for all conformance image options. Likely the better solution would be to impliment a catch all flag that passes extra flags to the conformance image args. I don't have a good example for how to do that off the bat.

[rjsadow]

In the short term and to at minimum satisfy @soltysh concerns for testing openshift we could add specifically --allow-not-ready-nodes to be passed as an argument.

[Hii-Arpit]

/assign

[reetasingh]

I am interested in this . @Hii-Arpit are you looking at catch all flag option or specifically --allow-not-ready-nodes option? I can work on the other one

[reetasingh]

@rjsadow I thought about a solution for catch all flag option

1. Implement a CLI flag which takes multiple key value pair as input rootCmd.Flags().VarP(&keyValue, "keyvalue", "k", "Multiple key/value pairs (e.g., key1=value1,key2=value2)"). Since we now use viper for config management we can also pass this in a config file to not make it look messy on command line
2. for each of the defined key value pair pass it as input in the env variable list for the container by adding Name = "E2E_<key>" and Value = <value> https://github.com/kubernetes-sigs/hydrophone/blob/main/pkg/service/init.go#L149
   The caveat is since its a catch all flag, we will not be able to to do validation when setting up the env variable for the container. the basic validation check we still need to do is not to allow duplicate keys when passing the key as env var

what do you think?

[reetasingh]

I will try with passing the arguments to the container. Also, looks like there is option to pass extra args through E2E keys using E2E_EXTRA_ARGS https://github.com/kubernetes/kubernetes/blob/02aaad0de94708b00e95ba6e809ac07d34eff61c/test/conformance/image/go-runner/const.go#L50

[rjsadow]

Good spot, if we can use E2E_EXTRA_ARGS then I agree, let's go down that path. We'll need to make sure we build some e2e tests for this.

[rjsadow]

Hey @Hii-Arpit, I'm going to reassign this issue to @reetasingh , her PR looks like it's going to cover all cases for adding extra arguments for users and we'd like to get this issue resolved to unblock some users from testing hydrophone. Feel free to reach out in slack or check out other issues that you'd like to help with!

/unassign @Hii-Arpit
/assign @reetasingh

[Hii-Arpit]

Hey @rjsadow , Thanks for informing me.