Error in parsing of acknowledged field for Darktrace integration

[arvchristos]

Hello,

While using the Darktrace integration for Elastic, we encountered several parsing errors while ingesting model breaches. These errors refer to the acknowledged field of the JSON stream, which is a JSON field with the following format:

    "acknowledged": {
        "time":1728987633000,
        "username": "[email protected]"
    }

However, as you can see in the lines below, this is treated as boolean:
https://github.com/elastic/integrations/blob/main/packages/darktrace/data_stream/model_breach_alert/elasticsearch/ingest_pipeline/default.yml#L493-L497

- convert:
      field: json.acknowledged
      target_field: darktrace.model_breach_alert.is_acknowledged
      type: boolean
      ignore_missing: true

This parser seems quite off. The error messages we get are the following:

[{time=1728987633000, [email protected]}] is not a boolean value, cannot convert to boolean

This value is useful, however it does not add too much value as it is only a snapshot of the model breach at the time of ingestion. As far as I know, elastic agent cannot update documents already ingested and a breach that is not acknowledged on ingestion may be acknowledged later on so by the time of query this field is already stale.

I suggest either removing it completely or fixing the parsing.

Thank you for the great integration,

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[efd6]

@arvchristos ISTM that we can just set darktrace.model_breach_alert.is_acknowledged to true if acknowledged.time exists and is in the past. Would that work for you?

I was unable to find any documentation for the logging format/fields used by Darktrace, that is not published by someone else. Are you aware of any? Are you able to donate some sanitised logs for testing?