Skip to content

[fix](olap) Avoid polluting the schema cache in OlapScanner#61510

Open
mrhhsg wants to merge 1 commit intoapache:masterfrom
mrhhsg:fix_cache
Open

[fix](olap) Avoid polluting the schema cache in OlapScanner#61510
mrhhsg wants to merge 1 commit intoapache:masterfrom
mrhhsg:fix_cache

Conversation

@mrhhsg
Copy link
Member

@mrhhsg mrhhsg commented Mar 19, 2026

What problem does this PR solve?

*** SIGSEGV address not mapped to object (@0x20) received by PID 2309810 (TID 2312493 OR 0x7f39af4bf640) from PID 32; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/gcp-user/selectdb-core/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /opt/jdk/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /opt/jdk/lib/server/libjvm.so
 3# 0x00007F3E7723FC30 in /lib64/libc.so.6
 4# std::_Hashtable<doris::StringRef, std::pair<doris::StringRef const, int>, std::allocator<std::pair<doris::StringRef const, int> >, std::__detail::_Select1st, std::equal_to<doris::StringRef>, doris::StringRefHash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::find(doris::StringRef const&) const at /home/gcp-user/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/hashtable.h:1587
 5# doris::TabletSchema::field_index(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const at /home/gcp-user/selectdb-core/be/src/olap/tablet_schema.cpp:1304
 6# doris::vectorized::NewOlapScanner::_init_tablet_reader_params(std::vector<doris::OlapScanRange*, std::allocator<doris::OlapScanRange*> > const&, std::vector<doris::TCondition, std::allocator<doris::TCondition> > const&, doris::pipeline::FilterPredicates const&, std::vector<doris::FunctionFilter, std::allocator<doris::FunctionFilter> > const&) at /home/gcp-user/selectdb-core/be/src/vec/exec/scan/new_olap_scanner.cpp:413
 7# doris::vectorized::NewOlapScanner::init() in /opt/selectdb/4.0.7.2025101510/be/lib/doris_be
 8# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at /home/gcp-user/selectdb-core/be/src/vec/exec/scan/scanner_scheduler.cpp:221
 9# std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /home/gcp-user/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
10# doris::ThreadPool::dispatch_thread() at /home/gcp-user/selectdb-core/be/src/util/threadpool.cpp:609
11# doris::Thread::supervise_thread(void*) at /home/gcp-user/selectdb-core/be/src/util/thread.cpp:499
12# start_thread in /lib64/libc.so.6
13# __GI___clone3 in /lib64/libc.so.6

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Mar 19, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@mrhhsg
Copy link
Member Author

mrhhsg commented Mar 19, 2026

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 26855 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a16a5ed38847a266a3a0d4ba0bdca10cb7dc52d7, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17590	4532	4298	4298
q2	q3	10630	801	530	530
q4	4678	353	249	249
q5	7549	1228	1009	1009
q6	178	172	148	148
q7	777	847	684	684
q8	9312	1510	1353	1353
q9	4979	4797	4651	4651
q10	6311	1952	1674	1674
q11	479	278	247	247
q12	724	590	461	461
q13	18063	2974	2197	2197
q14	228	236	214	214
q15	q16	766	762	678	678
q17	739	834	480	480
q18	6042	5274	5250	5250
q19	1111	988	625	625
q20	561	496	374	374
q21	4312	1884	1423	1423
q22	400	474	310	310
Total cold run time: 95429 ms
Total hot run time: 26855 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4820	4867	4556	4556
q2	q3	3899	4387	3848	3848
q4	889	1213	814	814
q5	4074	4369	4385	4369
q6	195	187	150	150
q7	1818	1739	1548	1548
q8	2515	2734	2597	2597
q9	7571	7383	7471	7383
q10	3772	4042	3611	3611
q11	503	432	429	429
q12	515	584	443	443
q13	2697	3296	2424	2424
q14	300	306	304	304
q15	q16	705	821	734	734
q17	1197	1342	1357	1342
q18	7444	7073	6753	6753
q19	929	934	946	934
q20	2070	2227	2058	2058
q21	3957	3513	3310	3310
q22	462	437	381	381
Total cold run time: 50332 ms
Total hot run time: 47988 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169527 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a16a5ed38847a266a3a0d4ba0bdca10cb7dc52d7, data reload: false

query5	4350	635	522	522
query6	343	237	210	210
query7	4217	469	272	272
query8	353	251	233	233
query9	8709	2732	2695	2695
query10	540	404	333	333
query11	7034	5088	4889	4889
query12	193	136	136	136
query13	1288	468	367	367
query14	5712	3707	3496	3496
query14_1	2888	2880	2845	2845
query15	208	193	177	177
query16	1003	483	478	478
query17	1129	742	627	627
query18	2470	456	366	366
query19	229	217	190	190
query20	136	131	128	128
query21	213	133	115	115
query22	13585	15098	14899	14899
query23	16242	15800	15535	15535
query23_1	15662	15733	15360	15360
query24	7148	1652	1223	1223
query24_1	1197	1254	1234	1234
query25	561	501	412	412
query26	1248	268	157	157
query27	2758	476	288	288
query28	4453	1863	1840	1840
query29	838	573	479	479
query30	303	211	188	188
query31	1003	965	877	877
query32	86	76	75	75
query33	525	333	294	294
query34	886	877	530	530
query35	648	666	597	597
query36	1041	1133	965	965
query37	137	98	83	83
query38	2979	2918	2908	2908
query39	880	849	819	819
query39_1	810	805	795	795
query40	238	160	140	140
query41	65	59	59	59
query42	259	258	258	258
query43	244	254	237	237
query44	
query45	205	194	183	183
query46	917	992	614	614
query47	2110	2157	2056	2056
query48	332	317	233	233
query49	637	466	388	388
query50	692	283	212	212
query51	4063	4048	3988	3988
query52	269	267	260	260
query53	297	333	288	288
query54	305	272	266	266
query55	89	87	88	87
query56	323	342	317	317
query57	1944	1848	1788	1788
query58	291	277	267	267
query59	2906	2937	2767	2767
query60	343	346	326	326
query61	156	156	157	156
query62	622	580	539	539
query63	315	278	281	278
query64	5059	1313	1010	1010
query65	
query66	1475	465	353	353
query67	24206	24337	24215	24215
query68	
query69	414	320	286	286
query70	989	983	972	972
query71	358	328	309	309
query72	2881	2898	2661	2661
query73	580	574	327	327
query74	9662	9657	9474	9474
query75	2875	2760	2453	2453
query76	2306	1030	675	675
query77	368	407	325	325
query78	10929	11180	10440	10440
query79	1185	780	572	572
query80	1312	632	561	561
query81	546	262	237	237
query82	998	156	116	116
query83	334	265	250	250
query84	256	120	99	99
query85	910	503	451	451
query86	424	299	283	283
query87	3140	3157	2995	2995
query88	3547	2654	2635	2635
query89	431	373	343	343
query90	2038	175	173	173
query91	167	160	137	137
query92	78	75	70	70
query93	1010	857	504	504
query94	665	328	297	297
query95	589	398	316	316
query96	661	528	225	225
query97	2497	2480	2418	2418
query98	253	229	224	224
query99	1001	998	921	921
Total cold run time: 249900 ms
Total hot run time: 169527 ms

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.69% (19775/37532)
Line Coverage 36.21% (184601/509876)
Region Coverage 32.46% (142901/440264)
Branch Coverage 33.65% (62513/185793)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants