Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend unique index for rhnpackagechangelogdata table #7531

Merged
merged 7 commits into from
Oct 25, 2023

Conversation

witekest
Copy link
Contributor

@witekest witekest commented Sep 12, 2023

What does this PR change?

In spite of client code trying to avoid duplicate records for changelog
data such records are observed in the database.
The change enforces unique combination of name, time and text
fields by updating the definition of the unique index on the database
level.

Additionally:

  • Add logging package details during failed import.
  • Clean up the logic for removing duplicate changelog entries.

Links

Tracks suse/spacewalk#22470

  • DONE

Changelogs

Make sure the changelogs entries you are adding are compliant with https://github.com/uyuni-project/uyuni/wiki/Contributing#changelogs and https://github.com/uyuni-project/uyuni/wiki/Contributing#uyuni-projectuyuni-repository

If you don't need a changelog check, please mark this checkbox:

  • No changelog needed

If you uncheck the checkbox after the PR is created, you will need to re-run changelog_test (see below)

Re-run a test

If you need to re-run a test, please mark the related checkbox, it will be unchecked automatically once it has re-run:

  • Re-run test "changelog_test"
  • Re-run test "backend_unittests_pgsql"
  • Re-run test "java_pgsql_tests"
  • Re-run test "schema_migration_test_pgsql"
  • Re-run test "susemanager_unittests"
  • Re-run test "javascript_lint"
  • Re-run test "spacecmd_unittests"

@witekest witekest requested a review from a team as a code owner September 12, 2023 12:20
Copy link
Member

@agraul agraul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extra logging looks good to me. I just wonder why the other change is needed.

Comment on lines 1369 to 1371
if 'importer' in locals():
e_message += f'\nPackage: {repr(importer)}'
if src_importer:
if 'src_importer' in locals():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this change needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the variables importer or src_importer is not assigned at that point and the UnboundLocalError is thrown.

Anyway, repr(importer) does not provide useful information, so I'll drop that part.

Inspite of client code trying to avoid duplicate records for changelog
data such records are observed in the database.
The change enforces unique combination of `name`, `time` and `text`
fields by updating the definition of the unique index on the database
level.

Add logging package details during failed import.

Clean up the logic for removing duplicate changelog entries.
@witekest witekest changed the title Extend reposync logging about wrong changelog data Extend unique index for rhnpackagechangelogdata table Oct 10, 2023
Use sha512 hashing to minimize collision risk
@witekest witekest requested a review from cbosdo October 11, 2023 08:25
@witekest
Copy link
Contributor Author

@cbosdo Could you take a look at the sonarcloud scanner failure?

@cbosdo
Copy link
Contributor

cbosdo commented Oct 11, 2023

@cbosdo Could you take a look at the sonarcloud scanner failure?

It looks like PRs coming form forks don't have access to the sonar token...

@witekest witekest requested a review from mackdk October 19, 2023 12:35
@witekest
Copy link
Contributor Author

@mackdk Could you please review the suggested new unique index for changelog entries? Thanks in advance.

Copy link
Contributor

@mackdk mackdk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but also file schema/spacewalk/common/tables/rhnPackageChangeLogData.sql should be updated to reflect the new index.

@@ -28,8 +28,9 @@ CREATE TABLE rhnPackageChangeLogData

;

CREATE INDEX rhn_pkg_cld_nt_idx
ON rhnPackageChangeLogData (name, time)
CREATE UNIQUE INDEX CONCURRENTLY rhn_pkg_cld_ntt_idx
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think CONCURRENTLY is accepted in a tables file because, as far as I know, these scripts are used to create the database from scratch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, thanks.

@@ -13,6 +13,7 @@
-- in this software or its documentation.
--

CREATE EXTENSION IF NOT EXISTS pgcrypto;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel this should go in schema/spacewalk/postgres/start.sql. I see we have there already something for dblink there but I'm not sure to understand how that works. Maybe @mcalmer has more clues.

@witekest witekest merged commit 965f322 into uyuni-project:master Oct 25, 2023
6 of 7 checks passed
r-richardson pushed a commit to r-richardson/uyuni that referenced this pull request Oct 26, 2023
…7531)

* Add unique index for rhnpackagechangelogdata table

Inspite of client code trying to avoid duplicate records for changelog
data such records are observed in the database.
The change enforces unique combination of `name`, `time` and `text`
fields by updating the definition of the unique index on the database
level.

Add logging package details during failed import.

Clean up the logic for removing duplicate changelog entries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants