Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Ensure one cma is processed by one addon worker at the same time #654

Merged

Conversation

zhujian7
Copy link
Member

@zhujian7 zhujian7 commented Oct 17, 2024

Summary

Currently, there are 2 workers started for the addonConfigurationController, and this controller watches the ManagedClusterAddOn resource, when mca changes, it enqueues the managedClusterAddon namespace and name in the format "{namespace}/{name}", but in the sync func, it only uses the name to get clusterManagementAddOn.

Think about the case, there is an addon named addon1, and we created 2 managedClusterAddon, one is in the cluster1 namespace, and the other one is in the cluster2 namespace, so there will be 2 keys enqueued, cluster1/addon1 and cluster2/addon1, because these 2 keys are not the same, the k8s cache queue will not merge them to one, so 2 workers will process these 2 keys at the same time. But finally, these 2 workers will get the same clusterManagementAddon addon1 simultaneously. This may cause some race condition problems.

This PR changes the enqueue rule for the managedClusterAddon resources only to enqueue the name. so cluster1/addon1 and cluster2/addon1 changing will result in only enqueue one key addon1, only one worker will be assigned to process it.

Related issue(s)

#651

Fixes #

@@ -90,7 +90,7 @@ func NewAddonConfigurationController(
queue.QueueKeyByMetaNamespaceName,
c.addonFilterFunc,
clusterManagementAddonInformers.Informer()).
WithInformersQueueKeysFunc(queue.QueueKeyByMetaNamespaceName, addonInformers.Informer()).
WithInformersQueueKeysFunc(queue.QueueKeyByMetaName, addonInformers.Informer()).
Copy link
Member Author

@zhujian7 zhujian7 Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the key point of this PR. mca changes, only enqueue the name.

Copy link

codecov bot commented Oct 17, 2024

Codecov Report

Attention: Patch coverage is 20.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 64.13%. Comparing base (33b409d) to head (86b605b).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...addon/controllers/addonconfiguration/controller.go 0.00% 3 Missing ⚠️
...kg/addon/controllers/addonmanagement/controller.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #654      +/-   ##
==========================================
+ Coverage   64.10%   64.13%   +0.03%     
==========================================
  Files         182      182              
  Lines       14277    14268       -9     
==========================================
- Hits         9152     9151       -1     
+ Misses       4213     4206       -7     
+ Partials      912      911       -1     
Flag Coverage Δ
unit 64.13% <20.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zhujian7
Copy link
Member Author

/assign @qiujian16 @haoqing0110

@qiujian16
Copy link
Member

/approve
/lgtm

@openshift-ci openshift-ci bot added the lgtm label Oct 18, 2024
Copy link
Contributor

openshift-ci bot commented Oct 18, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: qiujian16, zhujian7

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot bot merged commit bc01437 into open-cluster-management-io:main Oct 18, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants