Cluster not upgradeable between minor versions

[gerijani]

This is a fresh install. We started with 4.6.0-0.okd-2021-02-14-205305 and right after the cluster stand up I've tried to update it but I've got the following:

[root@****** ~]# oc patch clusterversion/version --patch '{"spec":"upstream":"https://amd64.origin.releases.ci.openshift.org/graph"}}' --type=merge
clusterversion.config.openshift.io/version patched
[root@******~]# oc adm upgrade
Cluster version is 4.6.0-0.okd-2021-02-14-205305

Upgradeable=False

  Reason: ClusterOperatorsNotUpgradeable
  Message: Multiple cluster operators cannot be upgraded between minor versions:
* Cluster operator authentication cannot be upgraded between minor versions: UnsupportedConfigOverrides_UnsupportedConfigOverridesSet: UnsupportedConfigOverridesUpgradeable: setting: [useUnsupportedUnsafeNonHANonProductionUnstableOAuthServer]
* Cluster operator etcd cannot be upgraded between minor versions: UnsupportedConfigOverrides_UnsupportedConfigOverridesSet: UnsupportedConfigOverridesUpgradeable: setting: [useUnsupportedUnsafeNonHANonProductionUnstableEtcd]

Updates:

VERSION                       IMAGE
4.7.0-0.okd-2021-02-23-121606 registry.ci.openshift.org/origin/release@sha256:8b10103388c3f5c4ea8854bad12a7b39b9399404e2d68b5db56efacf14ed1cfc
4.7.0-0.okd-2021-02-25-144700 registry.ci.openshift.org/origin/release@sha256:d0141818eca0f2d49fdea7f3ae5789f3bc747a669f2459f8ef189d1702405b08
4.7.0-0.okd-2021-03-07-090821 registry.ci.openshift.org/origin/release@sha256:2597be07a16ae6ef8df7752c039190bf5b238f81f88aca129d0c2d542239b537
4.7.0-0.okd-2021-03-21-094146 registry.ci.openshift.org/origin/release@sha256:3396c6a8a98484969ad728ce7b006e854a8f37649f685913b9cea2f1c9dacd26
4.7.0-0.okd-2021-03-28-152009 registry.ci.openshift.org/origin/release@sha256:02c01aaaed9d292376e7808b6365ed38c34856e2a13c99718b54a12958f4e6da
4.7.0-0.okd-2021-04-11-124433 registry.ci.openshift.org/origin/release@sha256:4414db9e8945504d1d5423ec11d64df9fa8b4aa60d978f8b4bdb3d0720800741
4.7.0-0.okd-2021-04-24-103438 registry.ci.openshift.org/origin/release@sha256:1c71a740375b34df53c6e2bfe389018f3160cea46c4022ed6080f583c54ecbc0
4.7.0-0.okd-2021-05-22-050008 registry.ci.openshift.org/origin/release@sha256:301cd36b2ab370027ddb0ec43e540a03a24748a25b69578dec504d130475304b
4.7.0-0.okd-2021-06-04-191031 registry.ci.openshift.org/origin/release@sha256:78ebb2404a0f3a5622fa141a6bcc923284d773ed0ccf65239a47ede1a3ad07d2
4.7.0-0.okd-2021-06-13-090745 registry.ci.openshift.org/origin/release@sha256:ab372d72a365b970193bc3369f6eecfd3d63a5d71c24edd263743b263c053155
4.7.0-0.okd-2021-07-03-190901 registry.ci.openshift.org/origin/release@sha256:66857f330c4b6199ce319c9b5b5ff3bd7ff3d4c75113cd3d33302216312d345f
4.7.0-0.okd-2021-08-07-063045 registry.ci.openshift.org/origin/release@sha256:78e4237a39e93e6f7b9cef579f135dc5d5dc11fc46ae53587351ac1d26261339
4.7.0-0.okd-2021-08-22-163618 registry.ci.openshift.org/origin/release@sha256:6e8ef1f76a56819a96cc70635487032f5d2b64822f26e16da34304d3fe792a17

Then I've tried on console Administration > Cluster Settings UPDATE STATUS showed Updates Available to 4.7 without choosing Channel. After I changed to stable-4.6, update possibility dissapeared and I only got :

[root@perf2-helper ~]# oc adm upgrade
Cluster version is 4.6.0-0.okd-2021-02-14-205305

Upgradeable=False

  Reason: ClusterOperatorsNotUpgradeable
  Message: Multiple cluster operators cannot be upgraded between minor versions:
* Cluster operator authentication cannot be upgraded between minor versions: UnsupportedConfigOverrides_UnsupportedConfigOverridesSet: UnsupportedConfigOverridesUpgradeable: setting: [useUnsupportedUnsafeNonHANonProductionUnstableOAuthServer]
* Cluster operator etcd cannot be upgraded between minor versions: UnsupportedConfigOverrides_UnsupportedConfigOverridesSet: UnsupportedConfigOverridesUpgradeable: setting: [useUnsupportedUnsafeNonHANonProductionUnstableEtcd]

No updates available. You may force an upgrade to a specific release image, but doing so may not be supported and result in downtime or data loss.

All co are OK, no degraded; mcp OK. This is a 3 master - 6 worker node cluster.
Also have this warning, but I'm not sure it has anithing to do with it.
"SamplesMissingTBRCredential
The samples operator cannot find credentials for 'registry.redhat.io'. Many of the sample ImageStreams will fail to import unless the 'samplesRegistry' in the operator configuration is changed."

[vrutkovs]

Single node clusters can be upgraded starting with OKD 4.8

[vrutkovs]

Why masters have different start time - did you upscale from one master to three?

[gerijani]

Yes, we didn't get the nodes in time so I started with what I had. With the bootstrap + 1 master.

[vrutkovs]

In that case you need to removed the overrides installer has set when you started with a single master

[gerijani]

OK, thanks! Just for sure, you talking about to delete at etcd cluster:

useUnsupportedUnsafeNonHANonProductionUnstableEtcd: *
useUnsupportedUnsafeNonHANonProductionUnstableOAuthServer: *

and by authentications.operator.openshift.io cluster as well the:

useUnsupportedUnsafeNonHANonProductionUnstableOAuthServer: *

values?

[gerijani]

OK, I've tried and it works, but now we have something new. I'll open a new topic for that one