Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(self-managed): update OpenShift routes instructions #3997

Merged
merged 23 commits into from
Jul 23, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -193,23 +193,36 @@ To use these routes for the Zeebe Gateway, configure this through Ingress as wel

As the Zeebe Gateway also uses `gRPC` (which relies on `HTTP/2`), [HTTP/2 Ingress Connectivity has to be enabled](https://docs.openshift.com/container-platform/latest/networking/ingress-operator.html#nw-http2-haproxy_configuring-ingress).

Also, Zeebe Gateway needs to be encrypted with tls because the connection from HAProxy to the Zeebe Gateway service can use HTTP/2 only for re-encrypt or passthrough routes and not for edge-terminated or insecure routes.
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved

#### Required Steps

1. Provide [TLS secrets](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) for the Zeebe Gateway. The [Cert Manager](https://docs.openshift.com/container-platform/latest/security/cert_manager_operator/index.html) might be helpful here:
1. Provide two [TLS secrets](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) for the Zeebe Gateway.

- The first TLS secret is issued to the Zeebe Gateway Service Name. This must use the [pkcs8 syntax](https://en.wikipedia.org/wiki/PKCS_8) or [pkcs1 syntax](https://en.wikipedia.org/wiki/PKCS_1) as Zeebe only supports these, referenced as **Service Certificate Secret** or `<SERVICE_CERTIFICATE_SECRET_NAME>`.
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved
In the example below, a TLS certificate is generated for the Zeebe Gateway service with an [annotation](https://docs.openshift.com/container-platform/4.15/security/certificates/service-serving-certificate.html). The generated cert will be in the form of a secret.
hamza-m-masood marked this conversation as resolved.
Show resolved Hide resolved

```yaml
zeebeGateway:
service:
annotations:
service.beta.openshift.io/serving-cert-secret-name: <SERVICE_CERTIFICATE_SECRET_NAME>
```

- One issued to the Zeebe Gateway Service Name. This must use the [pkcs8 syntax](https://en.wikipedia.org/wiki/PKCS_8) or [pkcs1 syntax](https://en.wikipedia.org/wiki/PKCS_1) as Zeebe only supports these, referenced as **Service Certificate Secret** or `<SERVICE_CERTIFICATE_SECRET_NAME>`. For more details, review the [OpenShift documentation](https://docs.openshift.com/container-platform/latest/networking/routes/secured-routes.html#nw-ingress-creating-a-reencrypt-route-with-a-custom-certificate_secured-routes).
Another option would be to use [Cert Manager](https://docs.openshift.com/container-platform/latest/security/cert_manager_operator/index.html). For more details, review the [OpenShift documentation](https://docs.openshift.com/container-platform/latest/networking/routes/secured-routes.html#nw-ingress-creating-a-reencrypt-route-with-a-custom-certificate_secured-routes).

<details>
<summary>pkcs8, pkcs1 syntax</summary>

> PKCS1 private key encoding. PKCS1 produces a PEM block that contains the private key algorithm in the header and the private key in the body. A key that uses this can be recognised by its BEGIN RSA PRIVATE KEY or BEGIN EC PRIVATE KEY header. NOTE: This encoding is not supported for Ed25519 keys. Attempting to use this encoding with an Ed25519 key will be ignored and default to PKCS8.
> PKCS1 private key encoding. PKCS1 produces a PEM block that contains the private key algorithm in the header and the private key in the body. A key that uses this can be recognised by its BEGIN RSA PRIVATE KEY or BEGIN EC PRIVATE KEY header. NOTE: This encoding is not supported for Ed25519 keys. Attempting to use this encoding with an Ed25519 key will be ignored and default to PKCS8.

> PKCS8 private key encoding. PKCS8 produces a PEM block with a static header and both the private key algorithm and the private key in the body. A key that uses this encoding can be recognised by its BEGIN PRIVATE KEY header.

> PKCS8 private key encoding. PKCS8 produces a PEM block with a static header and both the private key algorithm and the private key in the body. A key that uses this encoding can be recognised by its BEGIN PRIVATE KEY header.
[pkcs1, pkcs8 syntax definitionfrom cert-manager](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.PrivateKeyEncoding)

[pkcs1, pkcs8 syntax definitionfrom cert-manager](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.PrivateKeyEncoding)
</details>

- One that is used on the exposed route, referenced as **External URL Certificate Secret** or `<EXTERNAL_URL_CERTIFICATE_SECRET_NAME>`.
- The second TLS secret is used on the exposed route, referenced as **External URL Certificate Secret** or `<EXTERNAL_URL_CERTIFICATE_SECRET_NAME>`. For example, this would be the same TLS secret you would use for ingress.
hamza-m-masood marked this conversation as resolved.
Show resolved Hide resolved

2. Configure your Zeebe Gateway Ingress to create a [Re-encrypt Route](https://docs.openshift.com/container-platform/latest/networking/routes/route-configuration.html#nw-ingress-creating-a-route-via-an-ingress_route-configuration):

Expand Down Expand Up @@ -272,6 +285,8 @@ operate:
value: "true"
- name: CAMUNDA_OPERATE_ZEEBE_CERTIFICATEPATH
value: /usr/local/operate/config/tls.crt
- name: CAMUNDA_OPERATE_ZEEBE_BROKERCONTACTPOINT
value: camunda-zeebe-gateway.camunda.svc.cluster.local:26500
extraVolumeMounts:
- name: certificate
mountPath: /usr/local/operate/config/tls.crt
Expand All @@ -297,6 +312,8 @@ tasklist:
value: "true"
- name: CAMUNDA_TASKLIST_ZEEBE_CERTIFICATEPATH
value: /usr/local/tasklist/config/tls.crt
- name: CAMUNDA_TASKLIST_ZEEBE_BROKERCONTACTPOINT
value: camunda-zeebe-gateway.camunda.svc.cluster.local:26500
extraVolumeMounts:
- name: certificate
mountPath: /usr/local/tasklist/config/tls.crt
Expand All @@ -313,7 +330,54 @@ tasklist:

The actual configuration properties can be reviewed [in the Tasklist configuration documentation](/self-managed/tasklist-deployment/tasklist-configuration.md#zeebe-broker-connection).

5. Configure all other applications running inside the cluster and connecting to the Zeebe Gateway to also use TLS.
5. For Connectors:

The Connectors component will only accept a `jks`(java keystore) certificate.
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved
If you have followed our previous recommendation of generating a TLS certificate using the OpenShift annotation, then you will have a `PKCS#1` certificate which the Connectors component will not accept. For convenience, here are a number of commands that will convertt the `PKCS#1` certirficate, generated by OpenShift to `jks` format, that Connectors component accepts:
hamza-m-masood marked this conversation as resolved.
Show resolved Hide resolved

```bash
#grab OpenShift generated tls crt
kubectl get secret -n camunda camunda-zeebe-gateway -o jsonpath="{.data['tls\.crt']}" | base64 --decode > tls.crt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in https://github.com/camunda/distribution/issues/254#issuecomment-2196227345, could we integrate this directly at the startup of the container?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@conceptualshark We continued our discussion and noticed that instead we should put a reference to a github issue in the docs: camunda/connectors#2806

Is it a good idea to put a link to a github issue in the docs or should I just mention that we are working on a fix that will not require this manual workaround and not add a link?

For now I will add a link and you can remove it if you think it's not appropriate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added link to github issue ✅

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this reference is fine - we can make a note to remove it once the issue is closed.

#grab OpenShift generated tls key
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved
kubectl get secret -n hamza camunda-zeebe-gateway -o jsonpath="{.data['tls\.key']}" | base64 --decode > zeebe-key.key
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved
#convert zeebeGateway unencrypted key to encrypted key. You will be prompted to enter a password when running this command. Please not down the password:
openssl pkcs8 -topk8 -inform PEM -outform PEM -in ./zeebe-key.key -out ./zeebe-encrypted-key-gen.pem -v2 des3
#convert pkcs#1 cert to pkcs#12. Again, you will be prompted to enter password.
openssl pkcs12 -export -in tls.crt -inkey zeebe-encrypted-key-gen.pem -out zeebe-p12-certificate.p12 -name "certificate"
#convert pkcs#12 cert to jks cert
keytool -importkeystore -srckeystore zeebe-p12-certificate.p12 -srcstoretype pkcs12 -destkeystore keystore.jks
```

Finally create a generic TLS secret from the jks file:

```bash
kubectl create secret generic keystore -n camunda --from-file keystore.jks
```

Once the secret is created, this example values.yaml config can be followed:

```yaml
connectors:
inbound:
mode: oauth
env:
- name: ZEEBE_CLIENT_BROKER_GATEWAY-ADDRESS
value: "camunda-zeebe-gateway.camunda.svc.cluster.local:26500"
- name: ZEEBE_CLIENT_SECURITY_PLAINTEXT
value: "false"
- name: JAVA_TOOL_OPTIONS
value: "-Djavax.net.ssl.trustStore=/usr/local/certificates/keystore.jks -Djavax.net.ssl.trustStorePassword=changeit"
extraVolumeMounts:
- name: keystore
readOnly: true
mountPath: /usr/local/certificates
extraVolumes:
- name: keystore
secret:
secretName: keystore
```

6. Configure all other applications running inside the cluster and connecting to the Zeebe Gateway to also use TLS.

<!--Intended space left for not breaking the build!-->

Expand Down
Loading