[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (backport #44275) #44367

mergify · 2024-04-18T12:16:21Z

Why I'm doing:

This bug was introduced by #43838

The default value of _fn_context_index is -1, if open success, the value should be >=0.

So this is no chance to call close if _fn_context_index=0, the mem leak will happend.

All functions using prepare/close will leak memory.

Direct leak of 506880 byte(s) in 4224 object(s) allocated from:
    #0 0xb52d247 in operator new(unsigned long) ../../.././libsanitizer/asan/asan_new_delete.cpp:99
    #1 0x173763a4 in starrocks::LikePredicate::like_prepare(starrocks::FunctionContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/like_predicate.cpp:108
    #2 0x13c789d3 in starrocks::VectorizedFunctionCallExpr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/function_call_expr.cpp:118
    #3 0x11f4004c in starrocks::ExprContext::clone(starrocks::RuntimeState*, starrocks::ObjectPool*, starrocks::ExprContext**) /root/starrocks/be/src/exprs/expr_context.cpp:135
    #4 0x16683aa8 in starrocks::ColumnExprPredicate::_add_expr_ctx(starrocks::ExprContext*) /root/starrocks/be/src/storage/column_expr_predicate.cpp:70
    #5 0x16682f4f in starrocks::ColumnExprPredicate::make_column_expr_predicate(std::shared_ptr<starrocks::TypeInfo>, unsigned int, starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::SlotDescriptor const*) /root/starrocks/be/src/storage/column_expr_predicate.cpp:36
    #6 0x149b2711 in starrocks::PredicateParser::parse_expr_ctx(starrocks::SlotDescriptor const&, starrocks::RuntimeState*, starrocks::ExprContext*) const /root/starrocks/be/src/storage/predicate_parser.cpp:97
    #7 0xd617121 in starrocks::OlapScanConjunctsManager::get_column_predicates(starrocks::PredicateParser*, std::vector<std::unique_ptr<starrocks::ColumnPredicate, std::default_delete<starrocks::ColumnPredicate> >, std::allocator<std::unique_ptr<starrocks::ColumnPredicate, std::default_delete<starrocks::ColumnPredicate> > > >*) /root/starrocks/be/src/exec/olap_scan_prepare.cpp:672
    #8 0xf645829 in starrocks::pipeline::OlapChunkSource::_init_reader_params(std::vector<std::unique_ptr<starrocks::OlapScanRange, std::default_delete<starrocks::OlapScanRange> >, std::allocator<std::unique_ptr<starrocks::OlapScanRange, std::default_delete<starrocks::OlapScanRange> > > > const&, std::vector<unsigned int, std::allocator<unsigned int> > const&, std::vector<unsigned int, std::allocator<unsigned int> >&) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:220
    #9 0xf64addf in starrocks::pipeline::OlapChunkSource::_init_olap_reader(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:427
    #10 0xf63c370 in starrocks::pipeline::OlapChunkSource::prepare(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:91
    #11 0xe538310 in starrocks::pipeline::ScanOperator::_pickup_morsel(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:548
    #12 0xe5331a8 in starrocks::pipeline::ScanOperator::_try_to_trigger_next_scan(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:340
    #13 0xe531b19 in starrocks::pipeline::ScanOperator::pull_chunk(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:254
    #14 0xe859778 in starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/pipeline_driver.cpp:306
    #15 0xe81a05b in starrocks::pipeline::GlobalDriverExecutor::_worker_thread() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:152
    #16 0xe818a7f in operator() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:57
    #17 0xe824a9b in __invoke_impl<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:60
    #18 0xe823da0 in __invoke_r<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:110
    #19 0xe822e3a in _M_invoke /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/std_function.h:291
    #20 0xb5b4833 in std::function<void ()>::operator()() const /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/std_function.h:622
    #21 0xc10c991 in starrocks::FunctionRunnable::run() (/home/disk1/sr/be/lib/starrocks_be+0xc10c991)
    #22 0xc108e96 in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:562
    #23 0xc126089 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:73
    #24 0xc125b74 in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:95
    #25 0xc124abd in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/functional:416
    #26 0xc123a2b in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/functional:499
    #27 0xc121585 in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) (/home/disk1/sr/be/lib/starrocks_be+0xc121585)
    #28 0xc11def3 in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:110
    #29 0xc11a1de in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/std_function.h:291

What I'm doing:

Fixes https://github.com/StarRocks/StarRocksTest/issues/7088

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

Bugfix cherry-pick branch check:

This is an automatic backport of pull request #44275 done by [Mergify](https://mergify.com). ## Why I'm doing:

This bug was introduced by #43838

The default value of _fn_context_index is -1, if open success, the value should be >=0.

So this is no chance to call close if _fn_context_index=0, the mem leak will happend.

All functions using prepare/close will leak memory.

Direct leak of 506880 byte(s) in 4224 object(s) allocated from:
    #0 0xb52d247 in operator new(unsigned long) ../../.././libsanitizer/asan/asan_new_delete.cpp:99
    #1 0x173763a4 in starrocks::LikePredicate::like_prepare(starrocks::FunctionContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/like_predicate.cpp:108
    #2 0x13c789d3 in starrocks::VectorizedFunctionCallExpr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope) /root/starrocks/be/src/exprs/function_call_expr.cpp:118
    #3 0x11f4004c in starrocks::ExprContext::clone(starrocks::RuntimeState*, starrocks::ObjectPool*, starrocks::ExprContext**) /root/starrocks/be/src/exprs/expr_context.cpp:135
    #4 0x16683aa8 in starrocks::ColumnExprPredicate::_add_expr_ctx(starrocks::ExprContext*) /root/starrocks/be/src/storage/column_expr_predicate.cpp:70
    #5 0x16682f4f in starrocks::ColumnExprPredicate::make_column_expr_predicate(std::shared_ptr<starrocks::TypeInfo>, unsigned int, starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::SlotDescriptor const*) /root/starrocks/be/src/storage/column_expr_predicate.cpp:36
    #6 0x149b2711 in starrocks::PredicateParser::parse_expr_ctx(starrocks::SlotDescriptor const&, starrocks::RuntimeState*, starrocks::ExprContext*) const /root/starrocks/be/src/storage/predicate_parser.cpp:97
    #7 0xd617121 in starrocks::OlapScanConjunctsManager::get_column_predicates(starrocks::PredicateParser*, std::vector<std::unique_ptr<starrocks::ColumnPredicate, std::default_delete<starrocks::ColumnPredicate> >, std::allocator<std::unique_ptr<starrocks::ColumnPredicate, std::default_delete<starrocks::ColumnPredicate> > > >*) /root/starrocks/be/src/exec/olap_scan_prepare.cpp:672
    #8 0xf645829 in starrocks::pipeline::OlapChunkSource::_init_reader_params(std::vector<std::unique_ptr<starrocks::OlapScanRange, std::default_delete<starrocks::OlapScanRange> >, std::allocator<std::unique_ptr<starrocks::OlapScanRange, std::default_delete<starrocks::OlapScanRange> > > > const&, std::vector<unsigned int, std::allocator<unsigned int> > const&, std::vector<unsigned int, std::allocator<unsigned int> >&) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:220
    #9 0xf64addf in starrocks::pipeline::OlapChunkSource::_init_olap_reader(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:427
    #10 0xf63c370 in starrocks::pipeline::OlapChunkSource::prepare(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:91
    #11 0xe538310 in starrocks::pipeline::ScanOperator::_pickup_morsel(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:548
    #12 0xe5331a8 in starrocks::pipeline::ScanOperator::_try_to_trigger_next_scan(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:340
    #13 0xe531b19 in starrocks::pipeline::ScanOperator::pull_chunk(starrocks::RuntimeState*) /root/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:254
    #14 0xe859778 in starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int) /root/starrocks/be/src/exec/pipeline/pipeline_driver.cpp:306
    #15 0xe81a05b in starrocks::pipeline::GlobalDriverExecutor::_worker_thread() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:152
    #16 0xe818a7f in operator() /root/starrocks/be/src/exec/pipeline/pipeline_driver_executor.cpp:57
    #17 0xe824a9b in __invoke_impl<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:60
    #18 0xe823da0 in __invoke_r<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:110
    #19 0xe822e3a in _M_invoke /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/std_function.h:291
    #20 0xb5b4833 in std::function<void ()>::operator()() const /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/std_function.h:622
    #21 0xc10c991 in starrocks::FunctionRunnable::run() (/home/disk1/sr/be/lib/starrocks_be+0xc10c991)
    #22 0xc108e96 in starrocks::ThreadPool::dispatch_thread() /root/starrocks/be/src/util/threadpool.cpp:562
    #23 0xc126089 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:73
    #24 0xc125b74 in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:95
    #25 0xc124abd in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/functional:416
    #26 0xc123a2b in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/functional:499
    #27 0xc121585 in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) (/home/disk1/sr/be/lib/starrocks_be+0xc121585)
    #28 0xc11def3 in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/invoke.h:110
    #29 0xc11a1de in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) /opt/rh/gcc-toolset-10/root/usr/include/c++/10.3.0/bits/std_function.h:291

What I'm doing:

Fixes https://github.com/StarRocks/StarRocksTest/issues/7088

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

Signed-off-by: trueeyu <[email protected]> (cherry picked from commit e406099) # Conflicts: # be/src/exprs/function_call_expr.cpp # be/test/exprs/vectorized/function_call_expr_test.cpp

mergify · 2024-04-18T12:16:23Z

Cherry-pick of e406099 has failed:

On branch mergify/bp/branch-2.5/pr-44275
Your branch is up to date with 'origin/branch-2.5'.

You are currently cherry-picking commit e406099f2b.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	deleted by us:   be/src/exprs/function_call_expr.cpp
	both modified:   be/test/exprs/vectorized/function_call_expr_test.cpp

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

mergify · 2024-04-18T12:16:57Z

@mergify[bot]: Backport conflict, please reslove the conflict and resubmit the pr

Signed-off-by: trueeyu <[email protected]>

This reverts commit 31f8a11.

Signed-off-by: trueeyu <[email protected]>

This reverts commit 81f329f.

Signed-off-by: trueeyu <[email protected]>

[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (#44275)

938da70

Signed-off-by: trueeyu <[email protected]> (cherry picked from commit e406099) # Conflicts: # be/src/exprs/function_call_expr.cpp # be/test/exprs/vectorized/function_call_expr_test.cpp

mergify bot added the conflicts label Apr 18, 2024

mergify bot mentioned this pull request Apr 18, 2024

[BugFix] Fix the mem leak of VectorizedFunctionCallExpr #44275

Merged

24 tasks

github-actions bot assigned trueeyu Apr 18, 2024

mergify bot closed this Apr 18, 2024

github-actions bot added the automerge label Apr 18, 2024

mergify bot deleted the mergify/bp/branch-2.5/pr-44275 branch April 18, 2024 12:17

trueeyu restored the mergify/bp/branch-2.5/pr-44275 branch April 18, 2024 12:18

trueeyu reopened this Apr 18, 2024

wanpengfei-git enabled auto-merge (squash) April 18, 2024 12:19

trueeyu added 6 commits April 18, 2024 20:21

fix

31f8a11

Signed-off-by: trueeyu <[email protected]>

Revert "fix"

3d97148

This reverts commit 31f8a11.

fix

60e14ff

Signed-off-by: trueeyu <[email protected]>

fix

81f329f

Signed-off-by: trueeyu <[email protected]>

Revert "fix"

8e2cca2

This reverts commit 81f329f.

fix

5f033d1

Signed-off-by: trueeyu <[email protected]>

imay approved these changes Apr 18, 2024

View reviewed changes

wanpengfei-git merged commit fee4022 into branch-2.5 Apr 18, 2024
26 of 27 checks passed

wanpengfei-git deleted the mergify/bp/branch-2.5/pr-44275 branch April 18, 2024 13:00

github-actions bot added the version:2.5.21 label Apr 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (backport #44275) #44367

[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (backport #44275) #44367

mergify bot commented Apr 18, 2024 •

edited by wanpengfei-git

Loading

mergify bot commented Apr 18, 2024

mergify bot commented Apr 18, 2024

[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (backport #44275) #44367

[BugFix] Fix the mem leak of VectorizedFunctionCallExpr (backport #44275) #44367

Conversation

mergify bot commented Apr 18, 2024 • edited by wanpengfei-git Loading

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

What I'm doing:

What type of PR is this:

Checklist:

mergify bot commented Apr 18, 2024

mergify bot commented Apr 18, 2024

mergify bot commented Apr 18, 2024 •

edited by wanpengfei-git

Loading