Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@@contenttree-fetch slows down fetching indexes on brains for objects with large catalog indexes #32

Open
fredvd opened this issue Oct 27, 2017 · 0 comments

Comments

@fredvd
Copy link
Member

fredvd commented Oct 27, 2017

We ran into this issue on a site with a folder with 260 items that had a lot of words indexed in the SearchableText catalog. (Custom Dexterity contenttype with a file field containing a PDF that's indexed).

The related items behavior in plone.app.relationfield is using @@contenttree-fetch as well, default setup on Plone 4 with plone.app.contenttypes and trying to browse into the folder mentionned above takes between 30 and 40 seconds to render the listing.

Trying to profile and looking at a thread dump while the request is in progress there was always access to the method isBrainSelectable on PathSource at https://github.com/plone/plone.formwidget.contenttree/blob/master/plone/formwidget/contenttree/source.py#L161-L165 .

   def isBrainSelectable(self, brain):
        if brain is None:
            return False
        index_data = self.catalog.getIndexDataForRID(brain.getRID())
        return self.selectable_filter(brain, index_data)

These last two lines pull in a lot of index_data from the catalog, which takes a l a lot of time, especially when each content item has a lot of words.

Pull request #31 fixes the case when there are no criteria at all in CustomFilter to operate on, which makes fetching the index_data unnecessary in the first place. This fixes the related items behavior in plone.app.relationfield which doesn't use any filtering on the content type.

Plone 5 uses a different querying mechanism for plone.app.widgets related items, so it's not much use fixing this, just documented the behavior if someone else still runs into this.

@fredvd fredvd changed the title @@contenttree-fetch slows down fetching indexes on brains with many object and large indexes @@contenttree-fetch slows down fetching indexes on brains for objects with large catalog indexes Oct 27, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant