No Azure Zone: Crate Cannot Start

[yannic-hamann-abb]

Just wanted to spin up some test crate cluster in Azure.
The host system is in no zone:

curl -s 'http://169.254.169.254/metadata/instance/compute/zone?api-version=2020-06-01&format=text' -H 'Metadata: true'

Returns an empty string.

Cluster start fails with:

starts CrateDB

Option             Description
------             -----------
-C <KeyValuePair>  Configure a setting
-V, --version      Prints CrateDB version information and exits
-h, --help         show help
-s, --silent       show minimal output
-v, --verbose      show verbose output
ERROR: setting [node.attr.zone] must not be empty

Setting as a workaround:

spec.cluster.settings:
   node.attr.zone: "test"

[WalBeh]

Hi @yannic-hamann-abb, first of all apologies for the late response!

Could you please share environment variable settings you pass to crate-operator?

I assume envvar:: CLOUD_PROVIDER is set to azure, in which case the crate-operator assumes that the nodepools/nodes a spread over several availability zones and consults the infrastructure metaservice during startup of the POD for this information. Furthermore cluster routing allocation awareness is setup in the cratedbs, which makes limited sense if there are no AZs on the infrastructure side.

I hope that makes sense, otherwise let me know.

---

From documentation:

Under the hood, the operator will pass a zone attribute to all CrateDB
nodes. This attribute can also be defined explicitly or override the one set
by the operator. To do this on a cluster level, set .spec.cluster.settings:

  kind: CrateDB
  spec:
    cluster:
      settings:
        node.attr.zone: "some-value"

[yannic-hamann-abb]

Hi, the only config for the operator is:

crate-operator:
    env:
        CRATEDB_OPERATOR_CLOUD_PROVIDER: "azure"  # https://github.com/crate/crate-operator/blob/master/docs/source/configuration.rst
        CRATEDB_OPERATOR_LOG_LEVEL: "WARNING"

As mentioned initially, cluster start fails when the azure metadata service returns an empty string.
In one way it makes sense to error because Azure instances are created in a way not ready for production with multi-zone setup.

---

But, are you really sure the operator always configures multi zone setup automatically?
When looking at the docs the operator should set cluster.routing.allocation.awareness.attributes but it appears to me that this setting is not applied with a minimal CrateDB resource definition.

So AFAIK the default cluster setup via the crate-operator is without multi-zone support and therefore not setting a zone shouldn't fail.

[WalBeh]

You are right, routing awareness would need to be configured in the cratedbs.

If CRATEDB_OPERATOR_CLOUD_PROVIDER: "azure" is configured, the nodes need to be configured with availability zones, which is mentioned in the link you shared.

Agree it could be more explicitly mentioned, that setting a cloud provider with CRATEDB_OPERATOR_CLOUD_PROVIDER requires nodes with availability zones configured.

[WalBeh]

Agree it could be more explicitly mentioned, that setting a cloud provider with CRATEDB_OPERATOR_CLOUD_PROVIDER requires nodes with availability zones configured.

939edd7