Moved bridge configuration setup within the operator #11032

ppatierno · 2025-01-14T14:47:03Z

Type of change

Refactoring

Description

This PR fixes #10959 by moving the configuration of the bridge to be generated by the operator within a ConfigMap (actually using the same one that today is used for the metrics and logging configuratio, but with a different entry in the data section) then mounted on the bridge pod and loaded from there.

NOTE: the temporary change in the .azure/templates/steps/system_test_general.yaml file is for using a custom bridge image which doesn't use its script to generate the configuration anymore. Of course the commit to the file will be reverted before merging the PR. It's there for running STs.

Checklist

Write tests
Make sure all tests pass
Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
Reference relevant issue(s) and close them after merging
Update CHANGELOG.md

ppatierno · 2025-01-14T14:47:15Z

This PR comes together with strimzi/strimzi-kafka-bridge#963
I guess the bridge PR should be merged first, and the bridge 0.32.0 released.
After that we can update this PR by setting the new bridge release as well and removing the temporary change in the mentioned azure/templates/steps/system_test_general.yaml to use the right bridge.

see-quick · 2025-01-14T21:19:34Z

/azp run bridge

azure-pipelines · 2025-01-14T21:19:48Z

Azure Pipelines successfully started running 1 pipeline(s).

ppatierno · 2025-01-15T12:56:01Z

/azp run bridge

azure-pipelines · 2025-01-15T12:56:13Z

Azure Pipelines successfully started running 1 pipeline(s).

ppatierno · 2025-01-15T16:42:30Z

/azp run regression

azure-pipelines · 2025-01-15T16:42:41Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj

i made just a quick pass ... but left bunch of comments.

systemtest/src/test/java/io/strimzi/systemtest/bridge/HttpBridgeST.java

...src/main/java/io/strimzi/operator/cluster/operator/assembly/KafkaBridgeAssemblyOperator.java

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBridgeConfigurationBuilder.java

...tor/src/test/java/io/strimzi/operator/cluster/model/KafkaBridgeConfigurationBuilderTest.java

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBridgeConfigurationBuilder.java

tinaselenge

LGTM, thanks :)

fvaleri

There are a couple of test failures, but the overall approach LGTM.

I also left a minor comment. Thanks.

CHANGELOG.md

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBridgeConfigurationBuilder.java

...src/main/java/io/strimzi/operator/cluster/operator/assembly/KafkaBridgeAssemblyOperator.java

cluster-operator/src/test/java/io/strimzi/operator/cluster/model/KafkaBridgeClusterTest.java

systemtest/src/test/java/io/strimzi/systemtest/bridge/HttpBridgeST.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaBridgeCluster.java

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBridgeConfigurationBuilder.java

systemtest/src/test/java/io/strimzi/systemtest/bridge/HttpBridgeST.java

ppatierno · 2025-01-20T18:18:16Z

/azp run regression

azure-pipelines · 2025-01-20T18:18:26Z

Azure Pipelines successfully started running 1 pipeline(s).

...test/java/io/strimzi/operator/cluster/operator/assembly/KafkaBridgeAssemblyOperatorTest.java

.azure/templates/steps/system_test_general.yaml

ppatierno · 2025-01-21T17:29:34Z

/azp run regression

azure-pipelines · 2025-01-21T17:29:45Z

Azure Pipelines successfully started running 1 pipeline(s).

ppatierno · 2025-01-22T14:50:33Z

@scholzj I think I addressed all your comments.
Regarding the changes you see on the values.yaml and HelmResource.java, they are temporary for running the STs and allowing to bring the right operator and bridge for the Helm Charts one. I also opened an issue about that #11066
Of course these changes will be reverted before merging the PR.

ppatierno · 2025-01-22T15:02:50Z

actually it was worse. With the above changes I was able to run HelmCharts system tests locally but now the build is failing because of some checks we do on the naming:

FAIL  strimzi operator deployment	strimzi-kafka-operator/tests/operator_deployment_test.yaml
	- should have latest image with default image registry when no image is provided

		- asserts[1] `matchRegex` fail
			Template:	strimzi-kafka-operator/templates/060-Deployment-strimzi-cluster-operator.yaml
			DocumentIndex:	0
			Path:	spec.template.spec.containers[0].image
			Expected to match:
				quay.io/strimzi/operator:(latest|[0-9]+\.[0-9]+\.[0-9]+)
			Actual:
				quay.io/ppatierno/operator:bridge-config

Tbh, there are too many changes to make for having all regression STs running as they are now. What should we do? @scholzj @im-konge ideas?
Should I revert back these changes and live with a failing Helm Charts system test until the bridge release?

[UPDATE]
I did that to make the build passing. We'll have one regression test (related to Helm Charts) failing for sure.

ppatierno · 2025-01-22T16:45:39Z

/azp run regression

azure-pipelines · 2025-01-22T16:45:54Z

Azure Pipelines successfully started running 1 pipeline(s).

ppatierno · 2025-01-23T09:47:46Z

/azp run regression

azure-pipelines · 2025-01-23T09:47:59Z

Azure Pipelines successfully started running 1 pipeline(s).

ppatierno · 2025-01-23T14:37:00Z

So as mentioned on the related bridge PR strimzi/strimzi-kafka-bridge#963, I made a change on the bridge.version to set "latest" and run make bridge_version and then push the changes.
This way all the needed deployment files were updated for running the STs and they are all green. Of course, we have to revert this change as soon as we have a bridge 0.32.0 release. I will open an issue for that when this PR is merged.
@scholzj would you mind to have another pass and checking if everything looks fine to you now?

scholzj

One nit for the CHANGELOG. LGTM otherwise.

CHANGELOG.md

Signed-off-by: Paolo Patierno <[email protected]> Reverted back original operator Deployment Set different bridge image for STs Signed-off-by: Paolo Patierno <[email protected]> Fixed missing env vars to setup the truststore Signed-off-by: Paolo Patierno <[email protected]> Removed ST using env vars not in place anymore Signed-off-by: Paolo Patierno <[email protected]> Fixed checkstyle errors Signed-off-by: Paolo Patierno <[email protected]> CHANGELOG update Signed-off-by: Paolo Patierno <[email protected]> Fixed updated systemtests bridge MD Signed-off-by: Paolo Patierno <[email protected]> Fixed scholzj comments Signed-off-by: Paolo Patierno <[email protected]> Factored out a dedicated withConfigProviders method for the bridge configuration builder Refactored bridge configuration builder to use isEquivalent Signed-off-by: Paolo Patierno <[email protected]> Fixed tinaselenge comments Signed-off-by: Paolo Patierno <[email protected]> Fixed checkstyle issues Signed-off-by: Paolo Patierno <[email protected]> Fixed rolling bridge pod on configuration change Signed-off-by: Paolo Patierno <[email protected]> Fixed scholzj and tinaselenge feedback Signed-off-by: Paolo Patierno <[email protected]> Fixed KafkaBridgeAssemblyOperator tests about the missing new hash annotation Signed-off-by: Paolo Patierno <[email protected]> Reverted back some deleted tests Signed-off-by: Paolo Patierno <[email protected]> Adapted system tests to changes in configuration Signed-off-by: Paolo Patierno <[email protected]> Fixed scholzj and fvaleri feedback Signed-off-by: Paolo Patierno <[email protected]>

Signed-off-by: Paolo Patierno <[email protected]>

Temporary charts related changes to run STs Signed-off-by: Paolo Patierno <[email protected]>

Signed-off-by: Paolo Patierno <[email protected]>

ppatierno added this to the 0.46.0 milestone Jan 14, 2025

ppatierno mentioned this pull request Jan 14, 2025

Removed the script to generate the configuration strimzi/strimzi-kafka-bridge#963

Merged

strimzi deleted a comment from azure-pipelines bot Jan 14, 2025

strimzi deleted a comment from see-quick Jan 15, 2025

strimzi deleted a comment from azure-pipelines bot Jan 15, 2025

ppatierno force-pushed the bridge-config-via-configmap branch from 76c9c35 to d889705 Compare January 15, 2025 16:32

ppatierno marked this pull request as ready for review January 15, 2025 16:42

ppatierno requested a review from scholzj January 15, 2025 16:44

scholzj reviewed Jan 15, 2025

View reviewed changes

tinaselenge reviewed Jan 16, 2025

View reviewed changes

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBridgeConfigurationBuilder.java Outdated Show resolved Hide resolved

scholzj mentioned this pull request Jan 16, 2025

Add proposal for integrating Bridge with Metrics Reporter strimzi/proposals#143

Merged

tinaselenge approved these changes Jan 17, 2025

View reviewed changes

fvaleri reviewed Jan 18, 2025

View reviewed changes

CHANGELOG.md Show resolved Hide resolved

scholzj reviewed Jan 19, 2025

View reviewed changes

tinaselenge reviewed Jan 19, 2025

View reviewed changes

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaBridgeCluster.java Show resolved Hide resolved

tinaselenge reviewed Jan 19, 2025

View reviewed changes

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaBridgeCluster.java Outdated Show resolved Hide resolved

tinaselenge reviewed Jan 19, 2025

View reviewed changes

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBridgeConfigurationBuilder.java Outdated Show resolved Hide resolved

ppatierno commented Jan 20, 2025

View reviewed changes

systemtest/src/test/java/io/strimzi/systemtest/bridge/HttpBridgeST.java Show resolved Hide resolved

scholzj reviewed Jan 20, 2025

View reviewed changes

ppatierno force-pushed the bridge-config-via-configmap branch 2 times, most recently from 1c964e1 to de427e5 Compare January 21, 2025 13:57

ppatierno mentioned this pull request Jan 22, 2025

[ST] Improve the way to configure custom HTTP bridge image for Helm Charts system tests #11066

Open

scholzj approved these changes Jan 23, 2025

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

ppatierno added 6 commits January 24, 2025 08:26

Added test to check bridge configuration hash changes

519136c

Signed-off-by: Paolo Patierno <[email protected]>

Enabling user to specify config providers

448d8d3

Temporary charts related changes to run STs Signed-off-by: Paolo Patierno <[email protected]>

Fixed checkstyle error

6bb42d8

Signed-off-by: Paolo Patierno <[email protected]>

Reverted Helm Chart STs workaround to make the build pass

2ab7f37

Signed-off-by: Paolo Patierno <[email protected]>

Set bridge.version to "latest" to use new bridge image for STs

90427e7

Signed-off-by: Paolo Patierno <[email protected]>

ppatierno force-pushed the bridge-config-via-configmap branch from 99a0d55 to 90427e7 Compare January 24, 2025 07:27

ppatierno merged commit 984f294 into strimzi:main Jan 24, 2025
13 checks passed

ppatierno deleted the bridge-config-via-configmap branch January 24, 2025 08:44

This was referenced Jan 24, 2025

Update bridge.version to latest released bridge when 0.32.0 is out #11074

Open

Refactoring around bridge ConfigMap naming and volume mounts #11075

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Moved bridge configuration setup within the operator #11032

Moved bridge configuration setup within the operator #11032

ppatierno commented Jan 14, 2025 •

edited

Loading

ppatierno commented Jan 14, 2025 •

edited

Loading

see-quick commented Jan 14, 2025

azure-pipelines bot commented Jan 14, 2025

ppatierno commented Jan 15, 2025

azure-pipelines bot commented Jan 15, 2025

ppatierno commented Jan 15, 2025

azure-pipelines bot commented Jan 15, 2025

scholzj left a comment

tinaselenge left a comment

fvaleri left a comment

ppatierno commented Jan 20, 2025

azure-pipelines bot commented Jan 20, 2025

ppatierno commented Jan 21, 2025

azure-pipelines bot commented Jan 21, 2025

ppatierno commented Jan 22, 2025

ppatierno commented Jan 22, 2025 •

edited

Loading

ppatierno commented Jan 22, 2025

azure-pipelines bot commented Jan 22, 2025

ppatierno commented Jan 23, 2025

azure-pipelines bot commented Jan 23, 2025

ppatierno commented Jan 23, 2025

scholzj left a comment

Moved bridge configuration setup within the operator #11032

Moved bridge configuration setup within the operator #11032

Conversation

ppatierno commented Jan 14, 2025 • edited Loading

Type of change

Description

Checklist

ppatierno commented Jan 14, 2025 • edited Loading

see-quick commented Jan 14, 2025

azure-pipelines bot commented Jan 14, 2025

ppatierno commented Jan 15, 2025

azure-pipelines bot commented Jan 15, 2025

ppatierno commented Jan 15, 2025

azure-pipelines bot commented Jan 15, 2025

scholzj left a comment

Choose a reason for hiding this comment

tinaselenge left a comment

Choose a reason for hiding this comment

fvaleri left a comment

Choose a reason for hiding this comment

ppatierno commented Jan 20, 2025

azure-pipelines bot commented Jan 20, 2025

ppatierno commented Jan 21, 2025

azure-pipelines bot commented Jan 21, 2025

ppatierno commented Jan 22, 2025

ppatierno commented Jan 22, 2025 • edited Loading

ppatierno commented Jan 22, 2025

azure-pipelines bot commented Jan 22, 2025

ppatierno commented Jan 23, 2025

azure-pipelines bot commented Jan 23, 2025

ppatierno commented Jan 23, 2025

scholzj left a comment

Choose a reason for hiding this comment

ppatierno commented Jan 14, 2025 •

edited

Loading

ppatierno commented Jan 14, 2025 •

edited

Loading

ppatierno commented Jan 22, 2025 •

edited

Loading