Test discrepancies identified with kubetest2 - cloud-provider-gcp vs k/k #18865
Assignees
Labels
area/kubetest
area/provider/gcp
Issues or PRs related to gcp provider
kind/bug
Categorizes issue or PR as related to a bug.
lifecycle/frozen
Indicates that an issue or PR should not be auto-closed due to staleness.
sig/cloud-provider
Categorizes an issue or PR as relevant to SIG Cloud Provider.
Comments
|
/cc @amwat |
|
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
k8s-ci-robot
added
the
lifecycle/stale
|
/cc @jpbetz |
|
/remove-lifecycle stale |
|
/lifecycle frozen |
k8s-ci-robot
added
lifecycle/frozen
BenTheElder
added
area/provider/gcp
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What happened:
There are new CI jobs that use kubetest2 that are intended to serve as benchmarks to ensure parity of functionality between kubetest and kubetest2 before migrating more jobs from kubetest to kubetest2.
ci-cloud-provider-gcp-conformance-latest is a periodic that runs the conformance E2E tests against cloud-provider-gcp using kubetest2.
pull-kubernetes-e2e-gce-kubetest2 is a copy of pull-kubernetes-e2e-gce using kubetest2. This is the standard merge-blocking presubmit that runs on all PRs against k/k.
ci-kubernetes-gce-conformance-latest-kubetest2 is a copy of ci-kubernetes-gce-conformance-latest using kubetest2. This is a periodic that runs conformance E2E tests against k/k.
Interesting behavior is being observed in TestGrid as 1 and 3 run.
ci-cloud-provider-gcp-conformance-latestandci-kubernetes-gce-conformance-latest-kubetest2are essentially the same job - they both run conformance tests against a version of kubernetes (k/k vs cloud-provider-gcp) with kubetest2. However, comparing the dashboard for ci-kubernetes-gce-conformance-latest-kubetest2 with that of ci-cloud-provider-gcp-conformance-latest shows a striking difference. The easiest thing to point out is the set ofAdmissionWebhooktests that succeed onci-cloud-provider-gcp-conformance-latestyet fail onci-kubernetes-gce-conformance-latest-kubetest2. Both tests use the latest version of kubetest2 and pull the latest available Ginkgo E2E test package. Kubetest2 calls (effectively) the same script in k/k and in cloud-provider-gcp to bring the cluster up, and it does so with the same set of environment variables.The easy conclusion is that k/k has some regression or other difference when compared to cloud-provider-gcp. Yet, the original k/k conformance job
ci-kubernetes-gce-conformance-latesthas no failing tests.What you expected to happen:
The same set of tests fail on
ci-kubernetes-gce-conformance-latestas do onci-cloud-provider-gcp-conformance-latest.How to reproduce it (as minimally and precisely as possible):
N/A, see TestGrid.
Please provide links to example occurrences, if any:
See problem description.
The text was updated successfully, but these errors were encountered: