Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

increase insert xxx select xxx speed #460

Conversation

osdba
Copy link

@osdba osdba commented Nov 24, 2023

when increase polar_bulk_read_size to 64, but "insert xxx select" not getting faster, because ring_size is low,so increase ring_size in function GetAccessStrategy.

test:

postgres=# reset polar_bulk_read_size;
RESET
Time: 0.200 ms
postgres=# show polar_bulk_read_size;
 polar_bulk_read_size
----------------------
 128kB
(1 row)

Time: 0.216 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 96621.768 ms (01:36.622)

postgres=# set polar_bulk_read_size to 64;
SET
postgres=# set max_parallel_workers_per_gather to 0;
SET
postgres=# \timing
Timing is on.
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 73271.417 ms (01:13.271)

postgres=# set max_parallel_workers_per_gather to 8;
SET
Time: 0.173 ms
postgres=# explain select count(*) from test2;
                                         QUERY PLAN
---------------------------------------------------------------------------------------------
 Finalize Aggregate  (cost=9009314.43..9009314.44 rows=1 width=8)
   ->  Gather  (cost=9009313.60..9009314.41 rows=8 width=8)
         Workers Planned: 8
         ->  Partial Aggregate  (cost=9008313.60..9008313.61 rows=1 width=8)
               ->  Parallel Seq Scan on test2  (cost=0.00..8945812.48 rows=25000448 width=0)
(5 rows)

Time: 0.454 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 92410.190 ms (01:32.410)

postgres=# set polar_bulk_read_size to 64;
SET
Time: 0.168 ms
postgres=# show polar_bulk_read_size;
 polar_bulk_read_size
----------------------
 512kB
(1 row)

Time: 0.154 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 75722.028 ms (01:15.722)

@polardb-bot
Copy link

polardb-bot bot commented Nov 24, 2023

Hi @osdba ~ Thanks for your contribution in this PR. ❤️

Please make sure that your PR conforms the standard, and has passed all the checks.

We will review your PR as soon as possible.

@CLAassistant
Copy link

CLAassistant commented Nov 24, 2023

CLA assistant check
All committers have signed the CLA.

@polardb-bot
Copy link

polardb-bot bot commented Nov 24, 2023

Hey @osdba :

Congratulations~ 🎉 Your commit has passed all the checks. Please wait for further manual review.

@polardb-bot polardb-bot bot added the ci/success CI status is success label Nov 24, 2023
@osdba osdba force-pushed the feature_increase_insert_select_speed branch from 61f646c to 1d6ee47 Compare November 24, 2023 09:53
@osdba
Copy link
Author

osdba commented Nov 24, 2023

change my email for CLA,so close it and retry.

@osdba osdba closed this Nov 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/success CI status is success
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants