Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pulsar component doesn't respect key_shared delivery to only a single consumer replica #3601

Open
alicejgibbons opened this issue Nov 13, 2024 · 0 comments
Labels
kind/enhancement New feature or request

Comments

@alicejgibbons
Copy link

alicejgibbons commented Nov 13, 2024

Describe the feature

The implementation of the Pulsar Pubsub component maintains the order of replicas in all subscribers however still round-robin delivers messages to all the replicas of the same app-id. In native Pulsar, all the messages with the same partition key are delivered to the same consumer when key_shared is set as the subscription type. The Dapr implementation today delivers the messages to all consumer replicas (same app-id) regardless of partition key, keeping the relative order but not limiting a partition key to a specific consumer replica.

This should be remedied to ensure consistency across the Dapr implementation of Pulsar pub/sub. The docs here also imply that it works the same.

Thanks @rochabr for the reproduction repo here, see screenshot below where the two subscribers with the same partition key (authoring-event-topic) and app id (subscriber) are on the bottom and the publisher is on the top. You can see the messages being delivered to two replicas instead of being pinned to one.

image

Release Note

RELEASE NOTE: FIX Pulsar key_shared functionality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant