Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Determine if data_streams => true can be possible with ECS compatibility effectively disabled #1176

Open
yaauie opened this issue Mar 22, 2024 · 0 comments

Comments

@yaauie
Copy link
Contributor

yaauie commented Mar 22, 2024

Currently, we prohibit the use of data_stream => true, and resolve the default data_stream => auto to false when ECS is effectively disabled. The primary driver of this is to prevent users from getting themselves into a situation implicitly that would cause them trouble.

We need to determine in what conditions a customer should be able enable data streams features when they also want to run the pipeline without ECS.

When running on Logstash 8, ECS is on-by-default, but that can be:

  • overridden for an individual plugin by providing ecs_compatibility => disabled,
  • or the default itself can be changed by setting pipeline.ecs_compatibility: disabled for an individual pipeline or for the process as a whole.

When ECS is disabled, the behaviour in this plugin is changed in a couple of ways:

  • when manage_templates => true, template defaults to a pre-ecs template
  • data_stream => auto resolves to false

When data streams is effectively enabled, manage_templates defaults to false.

Therefore, when a user explicitly asks for data_stream => true, and does not also set manage_templates => true, the effective value of ecs_compatibility should have no effect. We need to verify this assertion, and if it holds true, should begin allowing users who explicitly ask for data streams to operate with or without ECS enabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant