Pulsar component doesn't respect key_shared
delivery to only a single consumer replica
#3601
Labels
kind/enhancement
New feature or request
Describe the feature
The implementation of the Pulsar Pubsub component maintains the order of replicas in all subscribers however still round-robin delivers messages to all the replicas of the same app-id. In native Pulsar, all the messages with the same partition key are delivered to the same consumer when key_shared is set as the subscription type. The Dapr implementation today delivers the messages to all consumer replicas (same app-id) regardless of partition key, keeping the relative order but not limiting a partition key to a specific consumer replica.
This should be remedied to ensure consistency across the Dapr implementation of Pulsar pub/sub. The docs here also imply that it works the same.
Thanks @rochabr for the reproduction repo here, see screenshot below where the two subscribers with the same partition key (authoring-event-topic) and app id (subscriber) are on the bottom and the publisher is on the top. You can see the messages being delivered to two replicas instead of being pinned to one.
Release Note
RELEASE NOTE: FIX Pulsar key_shared functionality.
The text was updated successfully, but these errors were encountered: