kubernetes/sig-testing

Home for SIG Testing discussion and documents.

SIG Testing

Home for SIG Testing discussion and documents.

Join us!

Learn how to engage with the Kubernetes community on the community page.

• Slack
• Mailing List
• Meeting agenda and notes

Contribution

• Open Issues / PRs
• sig-testing issues project (Kubernetes)
• sig-testing issues project (Kubernetes SIGs)

Code of conduct

Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.

sig-testing's Issues

Describe and write down the CI policy

Describe the CI policy for jobs, like:

• having contact information
• resources defined

We have a policy for blocking and informing jobs, https://github.com/kubernetes/sig-release/blob/master/release-blocking-jobs.md .
If we compare this policy with what is proposed in kubernetes/test-infra#18599, what would we add? what would we change?
We should evaluate what changes we need to make to help ensure we are acting on useful information and check tht CI jobs are maintained and well.

/area release-eng
/area ci
/kind documentation
/priority important-soon
/milestone v1.20

Fix or remove continuously failing jobs listed under sig-testing testgrid dashboard group

We have a number of jobs that are continuously failing. I'd like to see us get our dashboards back to green.

suffixes=(
  boskos
  canaries
  e2e-framework
  images
  kind
  kubetest2
  maintenance
  misc
  prow
)
for suffix in "${suffixes[@]}"; do
  curl -s https://testgrid.k8s.io/sig-testing-$suffix/summary \
  | jq -r '
    to_entries 
    | map(
      select(.value.overall_status == "FAILING") 
      | "- [ ] https://testgrid.k8s.io/\(.value.dashboard_name)#\(.key)"
    )[]';
done

• https://testgrid.k8s.io/sig-testing-canaries#bazelbuild-canary
• https://testgrid.k8s.io/sig-testing-canaries#benchmark-demo
• https://testgrid.k8s.io/sig-testing-canaries#ci-kubernetes-coverage-conformance
• https://testgrid.k8s.io/sig-testing-canaries#ci-kubernetes-coverage-e2e-gci-gce
• https://testgrid.k8s.io/sig-testing-canaries#ci-kubernetes-coverage-unit
• https://testgrid.k8s.io/sig-testing-canaries#ci-kubernetes-generate-make-test-cache
• https://testgrid.k8s.io/sig-testing-canaries#ci-kubernetes-generate-make-test-count10
• https://testgrid.k8s.io/sig-testing-canaries#gcb-docker-gcloud-canary
• https://testgrid.k8s.io/sig-testing-canaries#node
• https://testgrid.k8s.io/sig-testing-images#bigquery: kubernetes/test-infra#23849
• https://testgrid.k8s.io/sig-testing-images#gcloud-in-go: kubernetes/test-infra#23849
• https://testgrid.k8s.io/sig-testing-images#krte: kubernetes/test-infra#23849
• https://testgrid.k8s.io/sig-testing-images#kubemci
• https://testgrid.k8s.io/sig-testing-maintenance#autoupdate-patch
• https://testgrid.k8s.io/sig-testing-maintenance#ci-janitor: kubernetes/test-infra#23848
• https://testgrid.k8s.io/sig-testing-maintenance#pull-janitor: kubernetes/test-infra#23848
• https://testgrid.k8s.io/sig-testing-maintenance#triage-update: kubernetes/test-infra#23850

/sig testing
/kind failing-test
/kind bug
/priority important-longterm
/milestone v1.23

We should surface the SIG project boards better

https://kubernetes.slack.com/archives/C09QZ4DQB/p1624532212202100?thread_ts=1624504790.197700&cid=C09QZ4DQB cc @YuviGold

We have one for github.com/kubernetes, and one for github.com/kubernetes-sigs (due to not being able to stick issues from the latter in the former). We should put these in our README etc.

We should also look at getting more people involved in maintaining the contents, xref: #7

We should have a SIG Testing triage party instance

I know some other SIGs have experiment with this for issue triage. I think most triage right now is myself + @spiffxp, we should really be scaling that more.

cc @dims @ameukam

xref discussions about issue triage kubernetes/kubernetes#103151

we should also consider working on making it easier to stamp out instances for SIGs: https://twitter.com/BenTheElder/status/1407851591319703552 🧵

I think the biggest question is what token to run it on.

see also: https://twitter.com/thomrstrom/status/1408152928926502919