I had a look at the kubernetes provider and the inputs generated to ingest logs for containers and there may be a simpler solution to ensure that agents generates filestream inputs that match rotated container log files for container that crash/terminate quickly.

Looking at elastic-agent standalone k8s manifest, agent creates an input per (pod, container) pair and then one datastream per container (it's the same in the kustomize files):

In elastic agent helm chart, a single input is created with one datastream per container id (modeled after what the k8s integration does for fleet-managed agents):

The files in /var/log/containers are symlinks to the actual container logs under /var/log/pods/<namespace>_<pod name>_<pod_uid>/<container name>/*.log.

For example for pod fritzy-printer-6d65dcc658-mpzpr in namespace default container logs for echoer container are:

root@fastcontainers-control-plane:/var/log# ll containers/fritzy-printer-*.log
lrwxrwxrwx 1 root root 104 Jan 23 12:23 containers/fritzy-printer-6d65dcc658-mpzpr_default_echoer-3b25af57297705d7c0986df8e4817e1225c212d5df6dad27104b17357de92c9e.log -> /var/log/pods/default_fritzy-printer-6d65dcc658-mpzpr_67f3da94-d6d0-47b7-a60b-ea715174927c/echoer/56.log

The directory /var/log/pods/default_fritzy-printer-6d65dcc658-mpzpr_67f3da94-d6d0-47b7-a60b-ea715174927c/echoer/ contains all the available logs for the same container (even the rotated ones from previous container runs until they are deleted by kubelet).

In order to capture all the available logs it should be enough to change the paths in the container logs input, creating a single input and a stream per (pod,container name) pair, something similar to the yaml below

      # Input ID allowing Elastic Agent to track the state of this input. Must be unique.
      - id: filestream-container-logs
        type: filestream
        use_output: default
        ...
        streams:
          # Stream ID for this data stream allowing Filebeat to track the state of the ingested files. Must be unique.
          # Each filestream data stream creates a separate instance of the Filebeat filestream input.
          - id: container-log-${kubernetes.pod.namespace}-${kubernetes.pod.name}-${kubernetes.container.id}
            data_stream:
              dataset: kubernetes.container_logs
              type: logs
            parsers:
              - container: ~
              # - ndjson:
              #     target: json
              # - multiline:
              #     type: pattern
              #     pattern: '^\['
              #     negate: true
              #     match: after
            paths:
              - /var/log/pods/${kubernetes.pod.namespace}_${kubernetes.pod.name}_${kubernetes.pod.uid}/${kubernetes.container.name}/*.log

Note the addition of ${kubernetes.pod.namespace} in the stream id to ensure that agent can distinguish between pods with the same name specifying a container with the same name in different namespaces (could happen in case of pods with predictable network name like statefulsets).

I am going to modify the elastic-agent helm chart to check that the new input spec works as expected.

Huge props to @pkoutsovasilis that assisted me in the analysis of kubernetes provider and suggested looking into adjusting the paths passed to filebeat

@henrikno Could please have a look at my comment above and check if the proposed solution could help with the rotated container logs ?
This solution has a much smaller impact than adding a different mechanism for discovering pods and should still ingest all container log files available on the node.