AAP-36604 (analytics) Thousands of zombie/orphaned Slow/Stuck DB queries in controller querying active host count

[arrestle]

SUMMARY

Related #[AAP-36604]
(analytics) Thousands of zombie/orphaned Slow/Stuck DB queries in controller querying active host count

ISSUE TYPE

• Bug, Docs Fix or other nominal change

COMPONENT NAME

• API

AWX VERSION

awx: 24.6.2.dev198+ge2be654df3

ADDITIONAL INFORMATION

Identified that HostMetricSummaryMonthlyTask was running once per node, but only needed to be run once, so wrapped the execution with an advisory_lock similar to how it's done in system.py

From comment by Elijah DeLee on AAP-36604

...this one (Jira ticket) for running the analytics/hostmetrics stuff on every single node.


Note we see this exception on postgres:

2024-11-26 12:50:18.892 UTC [2431843] ERROR:  duplicate key value violates unique constraint "main_hostmetricsummarymonthly_date_key"
2024-11-26 12:50:18.892 UTC [2431843] DETAIL:  Key (date)=(2021-12-01) already exists.
2024-11-26 12:50:18.892 UTC [2431843] STATEMENT:  INSERT INTO "main_hostmetricsummarymonthly" ("date", "license_consumed", "license_capacity", "hosts_added", "hosts_deleted", "indirectly_managed_hosts") VALUES ('2021-12-01'::date, 0, 0, 0, 0, 0), ('2022-01-01'::date, 0, 0, 0, 0, 0), ('2022-02-01'::date, 0, 0, 0, 0, 0), ('2022-03-01'::date, 0, 0, 0, 0, 0), ('2022-04-01'::date, 0, 0, 0, 0, 0), ('2022-05-01'::date, 0, 0, 0, 0, 0), ('2022-06-01'::date, 0, 0, 0, 0, 0), ('2022-07-...
 
further underlining issue that we are running same task on multiple controller nodes when we really should acquire some kind of advisory lock and only run it on one for each schedule.

[AlanCoding]

Conflict warning with #15676, but it's not an actual problem, that just changes the import pattern and the code that uses the method won't change.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud