Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

m365_defender: fix message ID handling #12546

Merged
merged 1 commit into from
Feb 2, 2025
Merged

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented Jan 31, 2025

Proposed commit message

m365_defender: fix message ID handling

The current data flow for the fields changed here is NetworkMessageId[1] →
m365_defender.event.network.message_id → email.message_id and
InternetMessageId[1] → m365_defender.event.internet_message_id → email.local_id,
but the definition of email.message_id is that it represents the RFC5322
Message-ID[2], corresponding to the Defender InternetMessageId value, and
email.local_id[3] is the non-persistent identifier, reasonably corresponding to
the Defender NetworkMessageId value.

Also add m365_defender.event.internet_message_id to final remove processor.

[1]https://learn.microsoft.com/en-us/defender-xdr/advanced-hunting-emailevents-table#:~:text=NetworkMessageId,sending%20email%20system
[2]https://www.elastic.co/guide/en/ecs/current/ecs-email.html#field-email-message-id
[3]https://www.elastic.co/guide/en/ecs/current/ecs-email.html#field-email-local-id

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@efd6 efd6 added Integration:m365_defender Microsoft M365 Defender bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations Team [elastic/security-service-integrations] labels Jan 31, 2025
@efd6 efd6 self-assigned this Jan 31, 2025
The current data flow for the fields changed here is NetworkMessageId[1] →
m365_defender.event.network.message_id → email.message_id and
InternetMessageId[1] → m365_defender.event.internet_message_id → email.local_id,
but the definition of email.message_id is that it represents the RFC5322
Message-ID[2], corresponding to the Defender InternetMessageId value, and
email.local_id[3] is the non-persistent identifier, reasonably corresponding to
the Defender NetworkMessageId value.

Also add m365_defender.event.internet_message_id to final remove processor.

[1]https://learn.microsoft.com/en-us/defender-xdr/advanced-hunting-emailevents-table#:~:text=NetworkMessageId,sending%20email%20system
[2]https://www.elastic.co/guide/en/ecs/current/ecs-email.html#field-email-message-id
[3]https://www.elastic.co/guide/en/ecs/current/ecs-email.html#field-email-local-id
@efd6 efd6 force-pushed the 12530-m365_defender branch from 23baf99 to 0e6b438 Compare January 31, 2025 01:32
@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link

💚 Build Succeeded

cc @efd6

@efd6 efd6 marked this pull request as ready for review January 31, 2025 01:54
@efd6 efd6 requested a review from a team as a code owner January 31, 2025 01:54
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@efd6 efd6 merged commit e7e9e5a into elastic:main Feb 2, 2025
5 checks passed
@elastic-vault-github-plugin-prod

Package m365_defender - 2.21.1 containing this change is available at https://epr.elastic.co/package/m365_defender/2.21.1/

harnish-elastic pushed a commit to harnish-elastic/integrations that referenced this pull request Feb 4, 2025
The current data flow for the fields changed here is NetworkMessageId[1] →
m365_defender.event.network.message_id → email.message_id and
InternetMessageId[1] → m365_defender.event.internet_message_id → email.local_id,
but the definition of email.message_id is that it represents the RFC5322
Message-ID[2], corresponding to the Defender InternetMessageId value, and
email.local_id[3] is the non-persistent identifier, reasonably corresponding to
the Defender NetworkMessageId value.

Also add m365_defender.event.internet_message_id to final remove processor.

[1]https://learn.microsoft.com/en-us/defender-xdr/advanced-hunting-emailevents-table#:~:text=NetworkMessageId,sending%20email%20system
[2]https://www.elastic.co/guide/en/ecs/current/ecs-email.html#field-email-message-id
[3]https://www.elastic.co/guide/en/ecs/current/ecs-email.html#field-email-local-id
qcorporation pushed a commit that referenced this pull request Feb 4, 2025
The current data flow for the fields changed here is NetworkMessageId[1] →
m365_defender.event.network.message_id → email.message_id and
InternetMessageId[1] → m365_defender.event.internet_message_id → email.local_id,
but the definition of email.message_id is that it represents the RFC5322
Message-ID[2], corresponding to the Defender InternetMessageId value, and
email.local_id[3] is the non-persistent identifier, reasonably corresponding to
the Defender NetworkMessageId value.

Also add m365_defender.event.internet_message_id to final remove processor.

[1]https://learn.microsoft.com/en-us/defender-xdr/advanced-hunting-emailevents-table#:~:text=NetworkMessageId,sending%20email%20system
[2]https://www.elastic.co/guide/en/ecs/current/ecs-email.html#field-email-message-id
[3]https://www.elastic.co/guide/en/ecs/current/ecs-email.html#field-email-local-id
harnish-elastic pushed a commit to harnish-elastic/integrations that referenced this pull request Feb 5, 2025
The current data flow for the fields changed here is NetworkMessageId[1] →
m365_defender.event.network.message_id → email.message_id and
InternetMessageId[1] → m365_defender.event.internet_message_id → email.local_id,
but the definition of email.message_id is that it represents the RFC5322
Message-ID[2], corresponding to the Defender InternetMessageId value, and
email.local_id[3] is the non-persistent identifier, reasonably corresponding to
the Defender NetworkMessageId value.

Also add m365_defender.event.internet_message_id to final remove processor.

[1]https://learn.microsoft.com/en-us/defender-xdr/advanced-hunting-emailevents-table#:~:text=NetworkMessageId,sending%20email%20system
[2]https://www.elastic.co/guide/en/ecs/current/ecs-email.html#field-email-message-id
[3]https://www.elastic.co/guide/en/ecs/current/ecs-email.html#field-email-local-id
@efd6 efd6 deleted the 12530-m365_defender branch February 5, 2025 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix Pull request that fixes a bug issue Integration:m365_defender Microsoft M365 Defender Team:Security-Service Integrations Security Service Integrations Team [elastic/security-service-integrations]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[m365_defender]: Incorrectly mapped email fields
3 participants