Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(clustering/sync): improve readability and debugability #14170

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

StarlightIbuki
Copy link
Contributor

Summary

  1. abstract functions and reshape the code, including error handling
  2. extra call to notify CP the state of sync;
  3. commit DB purge before full sync;
  4. more debuging logs;
  5. sync once retry will not count timeout as retry, allowing DP to sync deltas right after a full sync

Checklist

  • The Pull Request has tests
  • A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
  • There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference

KAG-6177

@github-actions github-actions bot added core/clustering cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee labels Jan 15, 2025
@StarlightIbuki StarlightIbuki removed the cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee label Jan 15, 2025
@github-actions github-actions bot added the cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee label Jan 15, 2025
@chronolaw chronolaw changed the title refactor(sync.v2): improve readability and debugability refactor(clustering/sync): improve readability and debugability Jan 15, 2025
kong/clustering/services/sync/rpc.lua Outdated Show resolved Hide resolved
kong/clustering/services/sync/rpc.lua Outdated Show resolved Hide resolved
kong/clustering/services/sync/rpc.lua Show resolved Hide resolved
kong/clustering/services/sync/rpc.lua Outdated Show resolved Hide resolved
kong/clustering/services/sync/rpc.lua Outdated Show resolved Hide resolved
kong/clustering/services/sync/rpc.lua Show resolved Hide resolved
@StarlightIbuki StarlightIbuki force-pushed the refactor/rpc-sync-ce branch 3 times, most recently from 537d6f2 to b18175b Compare January 15, 2025 09:45
@StarlightIbuki StarlightIbuki force-pushed the refactor/rpc-sync-ce branch 4 times, most recently from ee07453 to 43975e3 Compare January 16, 2025 06:37
Copy link
Contributor

@chobits chobits left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!! all my comments are included in its associated EE pr, which have been resolved by xuming already

Comment on lines +264 to +249
if is_full_sync then
local entity_not_found = err and err:match("MDB_NOTFOUND")
-- full sync needs extra tolerance for errors
if not res and not entity_not_found then
return nil, err
end
else
if not res then
return nil, err
end
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a block, i know you're struggling with this PR.

Suggested change
if is_full_sync then
local entity_not_found = err and err:match("MDB_NOTFOUND")
-- full sync needs extra tolerance for errors
if not res and not entity_not_found then
return nil, err
end
else
if not res then
return nil, err
end
end
if is_full_sync then
local entity_not_found = err and err:match("MDB_NOTFOUND")
-- full sync needs extra tolerance for errors
if not res and not entity_not_found then
return nil, err
end
end
if not res then
return nil, err
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are different. The later is incorrect.

@StarlightIbuki StarlightIbuki force-pushed the refactor/rpc-sync-ce branch 3 times, most recently from 5092ad4 to 5e42c09 Compare January 16, 2025 09:12
1. abstract functions and reshape the code, including error handling
2. extra call to notify CP the state of sync;
3. more debuging logs;
4. sync once retry will not count timeout as retry, allowing DP to sync deltas right after a full sync

KAG-6177
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee core/clustering size/L skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants