Skip to content

Commit

Permalink
feat: rdkafka max_send_limit_bytes
Browse files Browse the repository at this point in the history
Signed-off-by: Peter Wilcsinszky <[email protected]>
  • Loading branch information
pepov committed Sep 27, 2024
1 parent 984e4c9 commit ff35e01
Show file tree
Hide file tree
Showing 6 changed files with 24 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -3215,6 +3215,8 @@ spec:
x-kubernetes-map-type: atomic
type: object
type: object
max_send_limit_bytes:
type: integer
max_send_retries:
type: integer
message_key_key:
Expand Down Expand Up @@ -10616,6 +10618,8 @@ spec:
x-kubernetes-map-type: atomic
type: object
type: object
max_send_limit_bytes:
type: integer
max_send_retries:
type: integer
message_key_key:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3211,6 +3211,8 @@ spec:
x-kubernetes-map-type: atomic
type: object
type: object
max_send_limit_bytes:
type: integer
max_send_retries:
type: integer
message_key_key:
Expand Down Expand Up @@ -9886,6 +9888,8 @@ spec:
x-kubernetes-map-type: atomic
type: object
type: object
max_send_limit_bytes:
type: integer
max_send_retries:
type: integer
message_key_key:
Expand Down
4 changes: 4 additions & 0 deletions config/crd/bases/logging.banzaicloud.io_clusteroutputs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3215,6 +3215,8 @@ spec:
x-kubernetes-map-type: atomic
type: object
type: object
max_send_limit_bytes:
type: integer
max_send_retries:
type: integer
message_key_key:
Expand Down Expand Up @@ -10616,6 +10618,8 @@ spec:
x-kubernetes-map-type: atomic
type: object
type: object
max_send_limit_bytes:
type: integer
max_send_retries:
type: integer
message_key_key:
Expand Down
4 changes: 4 additions & 0 deletions config/crd/bases/logging.banzaicloud.io_outputs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3211,6 +3211,8 @@ spec:
x-kubernetes-map-type: atomic
type: object
type: object
max_send_limit_bytes:
type: integer
max_send_retries:
type: integer
message_key_key:
Expand Down Expand Up @@ -9886,6 +9888,8 @@ spec:
x-kubernetes-map-type: atomic
type: object
type: object
max_send_limit_bytes:
type: integer
max_send_retries:
type: integer
message_key_key:
Expand Down
6 changes: 6 additions & 0 deletions docs/configuration/plugins/outputs/kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,12 @@ Default: nil
### keytab (*secret.Secret, optional) {#kafka-keytab}
### max_send_limit_bytes (int, optional) {#kafka-max_send_limit_bytes}
Max byte size to send message to avoid MessageSizeTooLarge. Messages over the limit will be dropped
Default: no limit
### max_send_retries (int, optional) {#kafka-max_send_retries}
Number of times to retry sending of messages to a leader
Expand Down
2 changes: 2 additions & 0 deletions pkg/sdk/logging/model/output/kafka.go
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,8 @@ type KafkaOutputConfig struct {
ScramMechanism string `json:"scram_mechanism,omitempty"`
// Number of times to retry sending of messages to a leader (default: 1)
MaxSendRetries int `json:"max_send_retries,omitempty"`
// Max byte size to send message to avoid MessageSizeTooLarge. Messages over the limit will be dropped (default: no limit)
MaxSendLimitBytes int `json:"max_send_limit_bytes,omitempty"`
// The number of acks required per request (default: -1).
RequiredAcks int `json:"required_acks,omitempty"`
// How long the producer waits for acks. The unit is seconds (default: nil => Uses default of ruby-kafka library)
Expand Down

0 comments on commit ff35e01

Please sign in to comment.