Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize indexing #14

Merged
merged 1 commit into from
Aug 22, 2024
Merged

Optimize indexing #14

merged 1 commit into from
Aug 22, 2024

Conversation

loevgaard
Copy link
Member

@loevgaard loevgaard commented Aug 21, 2024

This PR will split the indexing into batches dispatched on the command bus. This will speed up the indexing process proportionally with the number of workers.

Copy link

codecov bot commented Aug 21, 2024

Codecov Report

Attention: Patch coverage is 24.19355% with 47 lines in your changes missing coverage. Please review.

Project coverage is 26.46%. Comparing base (5991095) to head (c66bfea).

Files Patch % Lines
src/Message/Command/IndexEntities.php 0.00% 13 Missing ⚠️
src/DataProvider/DefaultIndexableDataProvider.php 0.00% 12 Missing ⚠️
src/Indexer/IndexBuffer.php 0.00% 11 Missing ⚠️
src/Indexer/DefaultIndexer.php 0.00% 6 Missing ⚠️
src/Config/Index.php 0.00% 4 Missing ⚠️
src/Event/QueryBuilderForDataProvisionCreated.php 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master      #14      +/-   ##
============================================
- Coverage     26.51%   26.46%   -0.05%     
- Complexity      361      373      +12     
============================================
  Files            78       80       +2     
  Lines          1192     1232      +40     
============================================
+ Hits            316      326      +10     
- Misses          876      906      +30     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@loevgaard loevgaard requested a review from jakubtobiasz August 21, 2024 12:12
src/DataProvider/DefaultDataProvider.php Outdated Show resolved Hide resolved
src/Indexer/IndexBuffer.php Outdated Show resolved Hide resolved
@loevgaard loevgaard force-pushed the data-provider branch 4 times, most recently from 4f3cf37 to 637346a Compare August 21, 2024 18:14
@loevgaard loevgaard requested a review from jakubtobiasz August 21, 2024 18:15
@loevgaard loevgaard force-pushed the data-provider branch 2 times, most recently from 922a7ef to c66bfea Compare August 21, 2024 18:22
@loevgaard loevgaard changed the title Add data provider for indexes Optimize indexing Aug 21, 2024
@loevgaard loevgaard merged commit a3565f2 into master Aug 22, 2024
33 of 36 checks passed
@loevgaard loevgaard deleted the data-provider branch August 22, 2024 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants