Refactor Conformance Testing Process

[rjsadow]

Refactor Conformance Testing Process

[rjsadow]

Sounds great @SD-13. It might make more sense to break some of these down into several smaller PRs. We can talk about different aspects as you work on them and figure out what makes sense. Reach out on slack or via this message if you have any issues.

[rjsadow]

This issue will likely involve a bit of a rethinking on how we're handling the command line and default behavior. Some initial thoughts:

• ./hydrophone should:
  • run run the conformance test suite like with --conformance.
  • pending #94, should run the conformance image on the same version as the test cluster
  • should get the logs and output them to the output-dir
  • should cleanup all of the resources
• ./hydrophone --focus should remain as is and override setting cfg.Focus. Some work required https://github.com/kubernetes-sigs/hydrophone/blob/main/pkg/common/args.go#L139C1-L141 and https://github.com/kubernetes-sigs/hydrophone/blob/main/pkg/common/args.go#L139C1-L141.
• ./hydrophone --cleanup should remain the same
• Adding in a get-logs or fetch-logs option may require some significant rework and design since they're streamed out of the sidecar busybox container. To decouple this we'll need some kind of more persistent storage solution that can be ad-hoc accessed after a run of the conformance image completes.

Other notes:

• This may be a good time to introduce cobra or some other CLI wrapper to better organize main.go

[SD-13]

/assign
I would like to try

[rakshitgondwal]

This may be a good time to introduce cobra or some other CLI wrapper to better organize main.go

+1 to this.

[rakshitgondwal]

Hey @SD-13! Wanted to ask you that will you be interested in collaborating on this issue. Like while you are looking into one of the subparts, maybe I can look into the other one?

[dims]

@SD-13 please collabroate with @rakshitgondwal and @rjsadow on slack and may be pick up smaller issues / pr(s) as this is a larger piece of work that needs to be broken down.

[reetasingh]

Hi @rjsadow @rakshitgondwal from what I see as scope of the issue still need to figure out "get-logs or fetch-logs" part . is anyone working on it? and whats the plan? I am interested to contribute

[rakshitgondwal]

Hi @rjsadow @rakshitgondwal from what I see as scope of the issue still need to figure out "get-logs or fetch-logs" part . is anyone working on it? and whats the plan? I am interested to contribute

Hey @reetasingh, afaik no one is working on that part right now but I am not so sure if we are planning to implement this right now. Maybe Ricky can tell more about this.

[Bharadwajshivam28]

Hey 👋
I can also take some part of this issue and work on it with all

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.