Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(dav): Mark removal of dav object properties as expensive #50391

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ChristophWurst
Copy link
Member

@ChristophWurst ChristophWurst commented Jan 24, 2025

  • Resolves: Nextcloud upgrade taking hours on large installations. This is because the oc_properties table has indices for userid+propertypath and propertypath. The DELETE query filters on propertyname and propertyvalue. There are no other queries during normal operation that would benefit from an index, so it's not a valuable alternative.

Summary

Moves the repair step from a synchronour repair step to an expensive ones. These are executed via occ.

TODO

  • Change registration
  • Test repair without expensive steps ✔️ step doesn't run anymore
  • Test repair with expensive steps ✔️ step runs

Checklist

@ChristophWurst
Copy link
Member Author

@tcitworld correct me if my assumption is wrong but we have had this step since #30368 so at late upgrades the repair step will not remove a lot of rows.

@ChristophWurst
Copy link
Member Author

/backport to stable31

@ChristophWurst
Copy link
Member Author

/backport to stable30

@ChristophWurst
Copy link
Member Author

/backport to stable29

@@ -215,6 +216,7 @@ public static function getExpensiveRepairSteps() {
),
\OC::$server->get(ValidatePhoneNumber::class),
\OC::$server->get(DeleteSchedulingObjects::class),
\OC::$server->get(RemoveObjectProperties::class),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a huge fan of using dav app classes in server but there is no other way right now. #50390 tracks the possibility to declare expensiveness directly in info.xml.

@juliusknorr
Copy link
Member

Make sense, though the query still does not seem to use an index. Can we be sure that the delete does not cause locks on the table for too long or may cause other problems?

@ChristophWurst
Copy link
Member Author

Correct, the query does not use an index.

This is because the oc_properties table has indices for userid+propertypath and propertypath. The DELETE query filters on propertyname and propertyvalue. There are no other queries during normal operation that would benefit from an index, so it's not a valuable alternative.

Adding an index would probably also take a couple of hours, and be counterproductive for normal operations outside of the repair step. Let me double check for any queries that could benefit from the query nevertheless.

@ChristophWurst
Copy link
Member Author

The most fitting index would be one on propertyname, propertypath and userid:

\OCA\DAV\DAV\CustomPropertiesBackend::getUserProperties -> either userid+propertypath OR propertyname+propertypath+userid
\OCA\DAV\DAV\CustomPropertiesBackend::createDeleteQuery -> propertyname+propertypath+userid
\OCA\DAV\DAV\CustomPropertiesBackend::createUpdateQuery -> propertyname+propertypath+uderid
\OCA\DAV\BackgroundJob\UserStatusAutomation::getAvailabilityFromPropertiesTable -> propertyname+propertypath+userid
\OCA\DAV\Db\PropertyMapper::findPropertyByPathAndName -> propertyname+propertypath+userid
\OCA\DAV\Migration\RemoveObjectProperties::run -> propertyname

Other queries that only filter by propertypath and userid would still use properties_path_index

@tcitworld
Copy link
Member

correct me if my assumption is wrong but we have had this step since #30368 so at late upgrades the repair step will not remove a lot of rows.

That's right.

Btw, a repair step that we should skip on upgrades is #13430, which should be a one time job and not run on every upgrade.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🏗️ In progress
Development

Successfully merging this pull request may close these issues.

3 participants