Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reconsider max_inflight heuristic and warning message #8622

Open
jsvd opened this issue Nov 9, 2017 · 0 comments · May be fixed by #16597
Open

reconsider max_inflight heuristic and warning message #8622

jsvd opened this issue Nov 9, 2017 · 0 comments · May be fixed by #16597
Assignees

Comments

@jsvd
Copy link
Member

jsvd commented Nov 9, 2017

Currently there's a MAX_INFLIGHT_WARN_THRESHOLD = 10_000 constant that is used to signal the user if it's likely that too many events will be in memory, and the formula is:

max_inflight = batch_size * pipeline_workers

Then we print:

@logger.warn("CAUTION: Recommended inflight events max exceeded! Logstash will run with up to #{max_inflight} events in memory in your current configuration. If your message sizes are large this may cause instability with the default heap size. Please consider setting a non-standard heap size, changing the batch size (currently #{batch_size}), or changing the number of pipeline workers (currently #{pipeline_workers})", default_logging_keys)

In master this formula may not be correct if you're using the memory queue, and is much less correct if using the persisted queue, where the high cap for events in memory is an unknown.

This warning must be reconsidered, either removed or made more accurate/robust.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants