You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have recently started using asyncio and aiokafka.
My requirement is to run n different kafka consumers consuming from different topics.
Each consumer on consumption pushes the data to a database and commits the offset once its done.
I am using manual commits for consumers to achieve atleast once delivery.
But i get error that committing wrong offset group has already rebalanced.
Logs from consumer are below :
Heartbeat failed for group .....-redis-cache-dev-dc-03 because it is rebalancing
Revoking previously assigned partitions frozenset...
(Re-)joining group .......-redis-cache-dev-dc-03
Joined group '.......-redis-cache-dev-dc-03' (generation 2123) with member_id aiokafka-0.8.0-a77413e1-a0ea-4f6c-80ea-44f8f665fd54
Elected group leader -- performing partition assignments using roundrobin
OffsetCommit failed for group .......-redis-cache-dev-dc-03 due to group error ([Error 22] IllegalGenerationError: .......-redis-cache-dev-dc-03), will rejoin
OffsetCommit failed for group .....-redis-cache-dev-dc-03 due to group error ([Error 22] IllegalGenerationError: .......-redis-cache-dev-dc-03), will rejoin
Caught exception: CommitFailedError: ('Commit cannot be completed since the group has already\n rebalanced and assigned the partitions to another member.\n This means that the time between subsequent calls to poll()\n was longer than the configured max_poll_interval_ms, which\n typically implies that the poll loop is spending too much\n time message processing. You can address this either by\n increasing the rebalance timeout with max_poll_interval_ms,\n or by reducing the maximum size of batches returned in poll()\n with max_poll_records.\n ', 'Commit cannot be completed since the group has already rebalanced and may have assigned the partitions to another member')
Shutting down...
The text was updated successfully, but these errors were encountered:
I have recently started using asyncio and aiokafka.
My requirement is to run n different kafka consumers consuming from different topics.
Each consumer on consumption pushes the data to a database and commits the offset once its done.
I am using manual commits for consumers to achieve atleast once delivery.
But i get error that committing wrong offset group has already rebalanced.
Logs from consumer are below :
The text was updated successfully, but these errors were encountered: