Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make max inflight warning global to all pipelines #16597

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

jsvd
Copy link
Member

@jsvd jsvd commented Oct 24, 2024

- pipeline.id: test_01
  config.string: "input { heartbeat {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
- pipeline.id: test_02
  config.string: "input { heartbeat {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
- pipeline.id: test_03
  config.string: "input { heartbeat {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
~/elastic/logstash improve_max_inflight_warning ❯ bin/logstash -r -b 125 --debug | grep resource                 
[2024-10-24T11:44:44,941][DEBUG][logstash.pipelineresourceusagevalidator] For a baseline of 2KB events, the maximum heap memory consumed across 3 pipelines may reach up to 0.72% of the entire heap (more if the events are bigger).
~/elastic/logstash improve_max_inflight_warning ❯ bin/logstash -r -b 12500 --debug | grep resource
[2024-10-24T11:45:13,008][WARN ][logstash.pipelineresourceusagevalidator] For a baseline of 2KB events, the maximum heap memory consumed across 3 pipelines may reach up to 71.53% of the entire heap (more if the events are bigger). The recommended percentage is less than 10%. Consider reducing the number of pipelines, or the batch size and worker count per pipeline.

closes #8622

@jsvd jsvd marked this pull request as ready for review October 24, 2024 13:37
Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@elasticmachine
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

History

@andsel andsel self-requested a review October 25, 2024 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

reconsider max_inflight heuristic and warning message
2 participants