Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove most of gingko.skip for tests focusing on Conformance #12887

Merged
merged 1 commit into from
Jun 5, 2019

Conversation

spiffxp
Copy link
Member

@spiffxp spiffxp commented Jun 4, 2019

Any tests that would have been skipped by these regexes have been
demoted from Conformance since v1.12, and are now prevented from
promotion to Conformancea (ref:
kubernetes/kubernetes#78710)

The one exception is [Serial], there are some Conformance tests that
we have been as yet unable to rewrite to run safely in parallel.

ref: #9697

/area conformance

Any tests that would have been skipped by these regexes have been
demoted from Conformance since v1.12, and are now prevented from
promotion to Conformance

The one exception is [Serial], there are some Conformance tests that
we have been as yet unable to rewrite to run safely in parallel.
@k8s-ci-robot k8s-ci-robot added area/conformance Issues or PRs related to kubernetes conformance tests cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jun 4, 2019
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. area/config Issues or PRs related to code in /config sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/release Categorizes an issue or PR as relevant to SIG Release. sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Jun 4, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 4, 2019
Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks.

@oomichi
Copy link
Member

oomichi commented Jun 4, 2019

kubernetes/cloud-provider-azure/cloud-provider-azure-config.yaml contains complex ginkgo.skip like --ginkgo.skip=\\[Serial\\]|\\[sig-node\\]\\sMount\\spropagation|\\[sig-network\\]\\sNetwork\\sshould\\sset\\sTCP\\sCLOSE_WAIT\\stimeout|\\[sig-storage\\]\\sPersistentVolumes-local\\sStress\\swith\\slocal\\svolume\\sprovisioner\\s\\[Serial\\]\\sshould\\suse\\sbe\\sable\\sto\\sprocess\\smany\\spods\\sand\\sreuse\\slocal\\svolumes|should\\sunmount\\sif\\spod\\sis\\sgracefully\\sdeleted\\swhile\\skubelet\\sis\\sdown\\s\\[Disruptive\\]\\[Slow\\]|should\\sunmount\\sif\\spod\\sis\\sforce\\sdeleted\\swhile\\skubelet\\sis\\sdown\\s\\[Disruptive\\]\\[Slow\\]|\\[sig-network\\]\\sServices\\sshould\\sbe\\sable\\sto\\screate\\sa\\sfunctioning\\sNodePort\\sservice|\\[sig-scheduling\\]\\sSchedulerPredicates\\s\\[Serial\\]\\svalidates\\sMaxPods\\slimit\\snumber\\sof\\spods\\sthat\\sare\\sallowed\\sto\\srun\\s\\[Slow\\]|\\[sig-storage\\]\\sDynamic\\sProvisioning\\sDynamicProvisioner\\sDefault\\sshould\\screate\\sand\\sdelete\\sdefault\\spersistent\\svolumes\\s\\[Slow\\]|\\[sig-storage\\]\\sDynamic\\sProvisioning\\sDynamicProvisioner\\sshould\\sprovision\\sstorage\\swith\\sdifferent\\sparameters|\\[sig-storage\\]\\sDynamic\\sProvisioning\\sDynamicProvisioner\\sshould\\stest\\sthat\\sdeleting\\sa\\sclaim\\sbefore\\sthe\\svolume\\sis\\sprovisioned\\sdeletes\\sthe\\svolume.|\\[sig-apps\\]\\sStatefulSet\\s\\[k8s.io\\]\\sBasic\\sStatefulSet\\sfunctionality\\s\\[StatefulSetBasic\\]\\sshould\\sadopt\\smatching\\sorphans\\sand\\srelease\\snon-matching\\spods|\\[sig-apps\\]\\sStatefulSet\\s\\[k8s.io\\]\\sBasic\\sStatefulSet\\sfunctionality\\s\\[StatefulSetBasic\\]\\sshould\\snot\\sdeadlock\\swhen\\sa\\spod.s\\spredecessor\\sfails|\\[sig-apps\\]\\sStatefulSet\\s\\[k8s.io\\]\\sBasic\\sStatefulSet\\sfunctionality\\s\\[StatefulSetBasic\\]\\sshould\\sperform\\srolling\\supdates\\sand\\sroll\\sbacks\\sof\\stemplate\\smodifications\\swith\\sPVCs|\\[sig-apps\\]\\sStatefulSet\\s\\[k8s.io\\]\\sBasic\\sStatefulSet\\sfunctionality\\s\\[StatefulSetBasic\\]\\sshould\\sprovide\\sbasic\\sidentity|\\[sig-storage\\]\\sPersistentVolumes\\sDefault\\sStorageClass\\spods\\sthat\\suse\\smultiple\\svolumes\\sshould\\sbe\\sreschedulable|\\[sig-storage\\]\\sPVC\\sProtection|\\[sig-storage\\]\\sDynamic\\sProvisioning\\s\\[k8s.io\\]\\sGlusterDynamicProvisioner|\\[sig-storage\\]\\sVolumes\\sAzure\\sDisk\\sshould\\sbe\\smountable\\s\\[Slow\\]|\\[sig-apps\\]\\sNetwork\\sPartition\\s\\[Disruptive\\]\\s\\[Slow\\]|\\[sig-network\\]\\sDNS\\sconfigMap|\\[k8s.io\\]\\s\\[sig-node\\]\\sKubelet\\s\\[Serial\\]\\s\\[Slow\\]\\s\\[k8s.io\\]\\s\\[sig-node\\]\\sregular\\sresource\\susage\\stracking\\sresource\\stracking\\sfor\\s0\\spods\\sper\\snode|\\[k8s.io\\]\\s\\[sig-node\\]\\sKubelet\\s\\[Serial\\]\\s\\[Slow\\]\\s\\[k8s.io\\]\\s\\[sig-node\\]\\sregular\\sresource\\susage\\stracking\\sresource\\stracking\\sfor\\s100\\spods\\sper\\snode|Horizontal\\spod\\sautoscaling\\s\\(scale\\sresource:\\sCPU\\)|\\[sig-storage\\]\\sDynamic\\sProvisioning\\sDynamicProvisioner\\sExternal\\sshould\\slet\\san\\sexternal\\sdynamic\\sprovisioner\\screate\\sand\\sdelete\\spersistent\\svolumes\\s\\[Slow\\]|ESIPP|\\[sig-network\\]\\sServices\\sshould\\spreserve\\ssource\\spod\\sIP\\sfor\\straffic\\sthru\\sservice\\scluster\\sIP|In-tree\\sVolumes|PersistentVolumes-local|CSI\\sVolumes|should\\swrite\\sentries\\sto\\s/etc/hosts|\\[Slow\\]|\\[Serial\\]|\\[Disruptive\\]|\\[Flaky\\]|\\[Feature:.+\\]|\\[NodeFeature:.+\\]"
Is this also an exception at this time?
The other part is good for me, thanks for doing this.

@neolit123
Copy link
Member

/lgtm
/hold

trying to notify the owning SIGs:
@kubernetes/sig-cluster-lifecycle-pr-reviews
@kubernetes/sig-gcp-pr-reviews
@kubernetes/sig-aws-misc

@k8s-ci-robot k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. area/provider/gcp Issues or PRs related to gcp provider area/provider/aws Issues or PRs related to aws provider lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Jun 4, 2019
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: e90995c9310dc0ad67d4b694601bd114d60938d7

@oomichi
Copy link
Member

oomichi commented Jun 4, 2019

/cc @oomichi

@k8s-ci-robot k8s-ci-robot requested a review from oomichi June 4, 2019 22:47
@spiffxp
Copy link
Member Author

spiffxp commented Jun 4, 2019

@oomichi yeah their thing is a bit crazy and I'm not going to touch it, one would hope they could eventually just exclude [LinuxOnly] or some tag that denoted functionality they knew wasn't going to work... my goal was solely to strip out the cargo-culted regex

@oomichi
Copy link
Member

oomichi commented Jun 4, 2019

@spiffxp Thanks for your response, I agree with you.
I confirmed all files which contained these skipping patterns are covered with this current PR and that is good at this time.

/lgtm

@spiffxp
Copy link
Member Author

spiffxp commented Jun 5, 2019

/hold cancel
I'll act as proxy for GCP jobs. Pinged others in slack. I'll keep an eye on jobs today

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 5, 2019
@k8s-ci-robot k8s-ci-robot merged commit d69838b into kubernetes:master Jun 5, 2019
@k8s-ci-robot
Copy link
Contributor

@spiffxp: Updated the job-config configmap in namespace default using the following files:

  • key generated-security-jobs.yaml using file config/jobs/kubernetes-security/generated-security-jobs.yaml
  • key eks-periodics.yaml using file config/jobs/kubernetes/sig-aws/eks/eks-periodics.yaml
  • key kubeadm-kind.yaml using file config/jobs/kubernetes/sig-cluster-lifecycle/kubeadm-kind.yaml
  • key gce-conformance.yaml using file config/jobs/kubernetes/sig-gcp/gce-conformance.yaml
  • key 1.12.yaml using file config/jobs/kubernetes/sig-release/release-branch-jobs/1.12.yaml
  • key 1.13.yaml using file config/jobs/kubernetes/sig-release/release-branch-jobs/1.13.yaml
  • key 1.14.yaml using file config/jobs/kubernetes/sig-release/release-branch-jobs/1.14.yaml
  • key 1.15.yaml using file config/jobs/kubernetes/sig-release/release-branch-jobs/1.15.yaml
  • key coverage.yaml using file config/jobs/kubernetes/sig-testing/coverage.yaml

In response to this:

Any tests that would have been skipped by these regexes have been
demoted from Conformance since v1.12, and are now prevented from
promotion to Conformancea (ref:
kubernetes/kubernetes#78710)

The one exception is [Serial], there are some Conformance tests that
we have been as yet unable to rewrite to run safely in parallel.

ref: #9697

/area conformance

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@spiffxp spiffxp deleted the conformance-exclude-tags branch June 28, 2019 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/config Issues or PRs related to code in /config area/conformance Issues or PRs related to kubernetes conformance tests area/provider/aws Issues or PRs related to aws provider area/provider/gcp Issues or PRs related to gcp provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/release Categorizes an issue or PR as relevant to SIG Release. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants