Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GUC for segmentwise recompression #7413

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

kpan2034
Copy link
Contributor

Add GUC option to enable or disable segmentwise recompression. If disabled, then a full recompression is done instead when recompression is attempted through compress_chunk. If recompress_chunk_segmentwise is used when GUC is disabled, then an error is thrown.

Closes #7381.

Copy link
Contributor

@mkindahl mkindahl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good, but I would suggest to make that message NOTICE instead of DEBUG1 to avoid flaky tests.

ERROR: segmentwise recompression functionality disabled, enable it by first setting timescaledb.enable_segmentwise_recompression to on
\set ON_ERROR_STOP 1
-- When GUC is OFF, entire chunk should be fully uncompressed and compressed instead
SET client_min_messages TO DEBUG;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be very flaky once in main, so it might be better to not use debug level in tests. You might be able to avoid it by making the message in question a notice rather than debug1.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the kind of flakiness I meant. Look at recompress_chunk_segmentwise. Debug outputs are rarely stable and will change depending on the run or the environment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you mean, makes sense. I'll change that to NOTICE instead.

Comment on lines 931 to 935
elog(DEBUG1,
"segmentwise recompression is disabled, performing full recompression on "
"chunk \"%s.%s\"",
NameStr(chunk->fd.schema_name),
NameStr(chunk->fd.table_name));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might make more sense as a NOTICE rather than DEBUG1. The message will only be shown when you have disabled the GUC, so it should not affect the default setup, but would be a good hint for users that disable segmentwise recompression.

Add GUC option to enable or disable segmentwise recompression. If
disabled, then a full recompression is done instead when recompression
is attempted through `compress_chunk`. If `recompress_chunk_segmentwise`
is used when GUC is disabled, then an error is thrown.
@kpan2034 kpan2034 force-pushed the segmentwise-recompression-guc branch from 796d690 to bcaea08 Compare November 1, 2024 13:53
Copy link
Contributor

@mkindahl mkindahl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

codecov bot commented Nov 1, 2024

Codecov Report

Attention: Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.

Project coverage is 82.62%. Comparing base (59f50f2) to head (bcaea08).
Report is 572 commits behind head on main.

Files with missing lines Patch % Lines
tsl/src/compression/api.c 83.33% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7413      +/-   ##
==========================================
+ Coverage   80.06%   82.62%   +2.55%     
==========================================
  Files         190      229      +39     
  Lines       37181    42740    +5559     
  Branches     9450    10739    +1289     
==========================================
+ Hits        29770    35314    +5544     
- Misses       2997     3143     +146     
+ Partials     4414     4283     -131     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement]: Add GUC for disabling/enabling partial recompression
2 participants