Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[postgres] Fix unicode decode error from Azure PostgreSQL Flexible Server #18938

Merged
merged 5 commits into from
Oct 30, 2024

Conversation

dalextorres
Copy link
Contributor

@dalextorres dalextorres commented Oct 29, 2024

What does this PR do?

This PR fixes the bug encountered in Azure PostgreSQL Flexible Server in the backend_type column of pg_stat_activity.

There is a bug on Microsoft's side where they send a non valid utf-8 string in that column, leading to decoding errors when trying to fetch this column.

The workaround is to query that same column with ::bytea suffix to convert the field into bytes then parsing backend_type manually. The bug still exists but now we just handle those error cases instead of failing on the whole query.

Motivation

https://datadoghq.atlassian.net/browse/SDBM-1169
https://datadoghq.atlassian.net/browse/SDBM-1197

#incident-31599

A couple of customers where seeing errors utilizing DBM with Azure PG Flexible Server when agent tries to query backend_type from pg_stat_activity. We previously issued a fix but had to revert due to many warnings being logged from the agent.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

Copy link

codecov bot commented Oct 30, 2024

Codecov Report

Attention: Patch coverage is 77.77778% with 2 lines in your changes missing coverage. Please review.

Project coverage is 87.41%. Comparing base (97f5af0) to head (b7fe580).
Report is 2 commits behind head on master.

Additional details and impacted files
Flag Coverage Δ
activemq ?
cassandra ?
hive ?
hivemq ?
hudi ?
ignite ?
jboss_wildfly ?
kafka ?
postgres 93.27% <77.77%> (+3.93%) ⬆️
presto ?
solr ?

Flags with carried forward coverage won't be shown. Click here to find out more.

@dalextorres dalextorres added this pull request to the merge queue Oct 30, 2024
Merged via the queue into master with commit 3d527ca Oct 30, 2024
38 checks passed
@dalextorres dalextorres deleted the zhengda.lu/unicode branch October 30, 2024 17:57
datadog-agent-integrations-bot bot pushed a commit that referenced this pull request Oct 30, 2024
…rver (#18938)

* manually decode backend_type

* map backend_type to backend_type::bytea

* add comments

* add changelog

* run lint

---------

Co-authored-by: Zhengda Lu <[email protected]>
(cherry picked from commit 3d527ca)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants