Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix the mem leak of VectorizedFunctionCallExpr #44275

Merged
merged 1 commit into from
Apr 18, 2024
Merged

[BugFix] Fix the mem leak of VectorizedFunctionCallExpr #44275

merged 1 commit into from
Apr 18, 2024

Conversation

trueeyu
Copy link
Contributor

@trueeyu trueeyu commented Apr 17, 2024 •
edited
Loading

Why I'm doing:

This bug was introduced by #43838

The default value of _fn_context_index is -1, if open success, the value should be >=0.

So this is no chance to call close if _fn_context_index=0, the mem leak will happend.

All functions using prepare/close will leak memory.

Direct leak of 506880 byte(s) in 4224 object(s) allocated from:
    #0 0xb52d247 in operator new(unsigned long) ../../.././libsanitizer/asan/asan_new_delete.cpp:99
    #1 0x173763a4 in starrocks::LikePredicate::like_prepare(starrocks::FunctionContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/like_predicate.cpp:108
    #2 0x13c789d3 in starrocks::VectorizedFunctionCallExpr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/function_call_expr.cpp:118
    #3 0x11f4004c in starrocks::ExprContext::clone(starrocks::RuntimeState*, starrocks::ObjectPool*, starrocks::ExprContext**) /root/starrocks/be/src/exprs/expr_context.cpp:135
    #4 0x16683aa8 in starrocks::ColumnExprPredicate::_add_expr_ctx(starrocks::ExprContext*) /root/starrocks/be/src/storage/column_expr_predicate.cpp:70
    #5 0x16682f4f in starrocks::ColumnExprPredicate::make_column_expr_predicate(std::shared_ptr<starrocks::TypeInfo>, unsigned int, starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::SlotDescriptor const*) /root/starrocks/be/src/storage/column_expr_predicate.cpp:36
    #6 0x149b2711 in starrocks::PredicateParser::parse_expr_ctx(starrocks::SlotDescriptor const&, starrocks::RuntimeState*, starrocks::ExprContext*) const /root/starrocks/be/src/storage/predicate_parser.cpp:97
    #7 0xd617121 in starrocks::OlapScanConjunctsManager::get_column_predicates(starrocks::PredicateParser*, std::vector<std::unique_ptr<starrocks::ColumnPredicate, std::default_delete<starrocks::ColumnPredicate> >, std::allocator<std::unique_ptr<starrocks::ColumnPredicate, std::default_delete<starrocks::ColumnPredicate> > > >*) /root/starrocks/be/src/exec/olap_scan_prepare.cpp:672
    #8 0xf645829 in starrocks::pipeline::OlapChunkSource::_init_reader_params(std::vector<std::unique_ptr<starrocks::OlapScanRange, std::default_delete<starrocks::OlapScanRange> >, std::allocator<std::unique_ptr<starrocks::OlapScanRange, std::default_delete<starrocks::OlapScanRange> > > > const&, std::vector<unsigned int, std::allocator<unsigned int> > const&, std::vector<unsigned int, std::allocator<unsigned int> >&) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:220
    #9 0xf64addf in starrocks::pipeline::OlapChunkSource::_init_olap_reader(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:427
    #10 0xf63c370 in starrocks::pipeline::OlapChunkSource::prepare(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:91
    #11 0xe538310 in starrocks::pipeline::ScanOperator::_pickup_morsel(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:548
    #12 0xe5331a8 in starrocks::pipeline::ScanOperator::_try_to_trigger_next_scan(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:340
    #13 0xe531b19 in starrocks::pipeline::ScanOperator::pull_chunk(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:254
    #14 0xe859778 in starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/pipeline_driver.cpp:306
    #15 0xe81a05b in starrocks::pipeline::GlobalDriverExecutor::_worker_thread() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:152
    #16 0xe818a7f in operator() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:57
    #17 0xe824a9b in __invoke_impl<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:60
    #18 0xe823da0 in __invoke_r<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:110
    #19 0xe822e3a in _M_invoke /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/std_function.h:291
    #20 0xb5b4833 in std::function<void ()>::operator()() const /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/std_function.h:622
    #21 0xc10c991 in starrocks::FunctionRunnable::run() (/home/disk1/sr/be/lib/starrocks_be+0xc10c991)
    #22 0xc108e96 in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:562
    #23 0xc126089 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:73
    #24 0xc125b74 in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:95
    #25 0xc124abd in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/functional:416
    #26 0xc123a2b in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/functional:499
    #27 0xc121585 in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) (/home/disk1/sr/be/lib/starrocks_be+0xc121585)
    #28 0xc11def3 in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:110
    #29 0xc11a1de in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/std_function.h:291

What I'm doing:

Fixes https://github.com/StarRocks/StarRocksTest/issues/7088

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.3
    • 3.2
    • 3.1
    • 3.0
    • 2.5

@trueeyu trueeyu marked this pull request as ready for review April 17, 2024 10:59
@trueeyu trueeyu requested a review from a team as a code owner April 17, 2024 10:59
@trueeyu
Copy link
Contributor Author

trueeyu commented Apr 17, 2024

@Mergifyio rebase

@mergify Mergify
Copy link
Contributor

mergify bot commented Apr 17, 2024

rebase

✅ Branch has been successfully rebased

@trueeyu
Copy link
Contributor Author

trueeyu commented Apr 18, 2024

https://github.com/Mergifyio rebase

@mergify Mergify
Copy link
Contributor

mergify bot commented Apr 18, 2024

rebase

✅ Branch has been successfully rebased

@github-actions GitHub Actions
Copy link

[FE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

@github-actions GitHub Actions
Copy link

[BE Incremental Coverage Report]

✅ pass : 1 / 1 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/exprs/function_call_expr.cpp 1 1 100.00% []

@trueeyu trueeyu merged commit e406099 into StarRocks:main Apr 18, 2024
60 checks passed
@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.3

@github-actions github-actions bot removed the 3.3 label Apr 18, 2024
@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.2

@github-actions github-actions bot removed the 3.2 label Apr 18, 2024
@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.1

@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.0

@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-2.5

@mergify Mergify
Copy link
Contributor

mergify bot commented Apr 18, 2024 •
edited
Loading

backport branch-3.3

✅ Backports have been created

@github-actions github-actions bot removed the 2.5 label Apr 18, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented Apr 18, 2024 •
edited
Loading

backport branch-3.2

✅ Backports have been created

@mergify Mergify
Copy link
Contributor

mergify bot commented Apr 18, 2024 •
edited
Loading

backport branch-3.1

✅ Backports have been created

@mergify Mergify
Copy link
Contributor

mergify bot commented Apr 18, 2024 •
edited
Loading

backport branch-3.0

✅ Backports have been created

@mergify Mergify
Copy link
Contributor

mergify bot commented Apr 18, 2024 •
edited
Loading

backport branch-2.5

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Apr 18, 2024
@trueeyu @mergify
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (#44275)
f973bcb 
Signed-off-by: trueeyu <[email protected]>
(cherry picked from commit e406099)
@mergify mergify bot mentioned this pull request Apr 18, 2024
Merged
42 tasks
mergify bot pushed a commit that referenced this pull request Apr 18, 2024
@trueeyu @mergify
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (#44275)
2efe562 
Signed-off-by: trueeyu <[email protected]>
(cherry picked from commit e406099)
@mergify mergify bot mentioned this pull request Apr 18, 2024
Merged
42 tasks
mergify bot pushed a commit that referenced this pull request Apr 18, 2024
@trueeyu @mergify
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (#44275)
7cec713 
Signed-off-by: trueeyu <[email protected]>
(cherry picked from commit e406099)
@mergify mergify bot mentioned this pull request Apr 18, 2024
Merged
42 tasks
mergify bot pushed a commit that referenced this pull request Apr 18, 2024
@trueeyu @mergify
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (#44275)
fcc35f4 
Signed-off-by: trueeyu <[email protected]>
(cherry picked from commit e406099)
@mergify mergify bot mentioned this pull request Apr 18, 2024
Merged
42 tasks
mergify bot pushed a commit that referenced this pull request Apr 18, 2024
@trueeyu @mergify
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (#44275)
938da70 
Signed-off-by: trueeyu <[email protected]>
(cherry picked from commit e406099)

# Conflicts:
#	be/src/exprs/function_call_expr.cpp
#	be/test/exprs/vectorized/function_call_expr_test.cpp
@mergify mergify bot mentioned this pull request Apr 18, 2024
Merged
42 tasks
wanpengfei-git pushed a commit that referenced this pull request Apr 18, 2024
@mergify @trueeyu
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (backport #44275
64a578a 
) (#44363)

Co-authored-by: trueeyu <[email protected]>
wanpengfei-git pushed a commit that referenced this pull request Apr 18, 2024
@mergify @trueeyu
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (backport #44275
a947b18 
) (#44364)

Co-authored-by: trueeyu <[email protected]>
wanpengfei-git pushed a commit that referenced this pull request Apr 18, 2024
@mergify @trueeyu
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (backport #44275
2a72321 
) (#44365)

Co-authored-by: trueeyu <[email protected]>
wanpengfei-git pushed a commit that referenced this pull request Apr 18, 2024
@mergify @trueeyu
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (backport #44275
fee4022 
) (#44367)

Signed-off-by: trueeyu <[email protected]>
Co-authored-by: trueeyu <[email protected]>
wanpengfei-git pushed a commit that referenced this pull request Apr 18, 2024
@mergify @trueeyu
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (backport #44275
c9836ed 
) (#44366)

Co-authored-by: trueeyu <[email protected]>
leiyang0324 pushed a commit to leiyang0324/starrocks that referenced this pull request Apr 19, 2024
@trueeyu @leiyang0324
[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (StarRocks#44275
9caa6ee 
)

Signed-off-by: trueeyu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Seaven Seaven Seaven approved these changes

@stdpain stdpain stdpain approved these changes

@andyziye andyziye andyziye approved these changes

Assignees

@trueeyu trueeyu

Labels
2.5-merged 3.0-merged 3.1-merged 3.2-merged 3.3-merged
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@trueeyu @Seaven @stdpain @andyziye