Skip to content

[fix](binlog) Report streams with missing base tables as stale - #68087

Open
luwei16 wants to merge 1 commit into
apache:masterfrom
luwei16:fix/doris-27286-stream-health
Open

luwei16 wants to merge 1 commit into
apache:masterfrom
luwei16:fix/doris-27286-stream-health

Conversation

@luwei16

@luwei16 luwei16 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: #67173

Problem Summary: Dropping a base table leaves its table stream object in metadata, but the table stream metadata function previously reported the retained stream as enabled and non-stale even though its original base table ID could no longer be resolved. This change derives the displayed health from the same base-table lookup used for the identity columns, while preserving the stream configured state so recovering the original table restores its health.

Release note

Table streams whose base table is unavailable are now reported as disabled and stale with an explicit reason.

Check List (For Author)

  • Test: Unit Test
    • DropTableStreamTest
    • TableStreamManagerCleanupTest
    • TableStreamBaseTableInfoTest
  • Behavior changed: Yes. Missing-base streams are reported as disabled and stale.
  • Does this need documentation: No

### What problem does this PR solve?

Issue Number: None

Related PR: apache#67173

Problem Summary: Dropping a base table leaves its table stream object in metadata, but the table stream metadata function previously reported the retained stream as enabled and non-stale even though its original base table ID could no longer be resolved. This change derives the displayed health from the same base-table lookup used for the identity columns, while preserving the stream configured state so recovering the original table restores its health.

### Release note

Table streams whose base table is unavailable are now reported as disabled and stale with an explicit reason.

### Check List (For Author)

- Test: Unit Test
    - DropTableStreamTest
    - TableStreamManagerCleanupTest
    - TableStreamBaseTableInfoTest
- Behavior changed: Yes. Missing-base streams are reported as disabled and stale.
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@luwei16 luwei16 changed the title [fix](fe) Report streams with missing base tables as stale [fix](binlog) Report streams with missing base tables as stale Sep 16, 2026
@luwei16

luwei16 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 71.43% (5/7) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 71.43% (5/7) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16902 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c176e9993b466e2f7bd489209193f9f4d8a5109a, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17564	3036	3021	3021
q2	2099	245	229	229
q3	10257	877	513	513
q4	4669	249	202	202
q5	7677	568	388	388
q6	138	115	92	92
q7	524	504	398	398
q8	9238	858	866	858
q9	3408	2412	2406	2406
q10	6503	829	749	749
q11	393	193	184	184
q12	611	257	197	197
q13	18158	1577	1182	1182
q14	165	155	134	134
q15	q16	429	398	366	366
q17	1396	932	833	833
q18	3107	2248	2261	2248
q19	1265	892	804	804
q20	392	300	199	199
q21	5612	1671	1850	1671
q22	336	275	228	228
Total cold run time: 93941 ms
Total hot run time: 16902 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3405	3335	3333	3333
q2	509	387	378	378
q3	2313	2318	2203	2203
q4	1205	1172	900	900
q5	2193	2127	2133	2127
q6	170	118	85	85
q7	1030	914	817	817
q8	1593	1410	1404	1404
q9	3147	3128	3101	3101
q10	1862	1838	1602	1602
q11	355	270	254	254
q12	453	435	345	345
q13	1490	1537	1138	1138
q14	178	170	159	159
q15	q16	391	407	360	360
q17	3568	3265	3190	3190
q18	4816	4398	4750	4398
q19	889	891	899	891
q20	975	955	826	826
q21	3908	3176	3213	3176
q22	404	355	331	331
Total cold run time: 34854 ms
Total hot run time: 31018 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 82225 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c176e9993b466e2f7bd489209193f9f4d8a5109a, data reload: false

query5	4235	414	331	331
query6	385	133	122	122
query7	4964	410	231	231
query8	290	139	118	118
query9	8689	2908	2927	2908
query10	395	230	184	184
query11	5374	1059	911	911
query12	119	72	68	68
query13	1194	433	312	312
query14	6153	2213	2121	2121
query14_1	2017	1973	2022	1973
query15	175	120	116	116
query16	903	386	353	353
query17	795	441	348	348
query18	2330	327	241	241
query19	172	136	107	107
query20	69	72	70	70
query21	198	103	88	88
query22	5449	5338	5338	5338
query23	6797	6204	6001	6001
query23_1	6159	6155	6070	6070
query24	7254	1093	795	795
query24_1	774	760	772	760
query25	428	297	253	253
query26	1221	239	131	131
query27	2785	414	253	253
query28	4676	1494	1504	1494
query29	928	445	362	362
query30	251	153	134	134
query31	819	397	334	334
query32	131	78	82	78
query33	464	217	182	182
query34	986	843	482	482
query35	418	415	343	343
query36	560	563	553	553
query37	125	85	74	74
query38	1003	841	827	827
query39	477	504	460	460
query39_1	456	490	475	475
query40	203	91	79	79
query41	59	54	57	54
query42	77	75	72	72
query43	241	242	218	218
query44	1038	521	528	521
query45	112	105	103	103
query46	790	883	520	520
query47	765	758	731	731
query48	329	304	218	218
query49	533	241	183	183
query50	791	264	189	189
query51	8340	8105	8251	8105
query52	69	67	57	57
query53	188	196	144	144
query54	217	186	212	186
query55	78	60	59	59
query56	215	192	155	155
query57	691	656	672	656
query58	195	162	154	154
query59	1199	1231	1104	1104
query60	233	186	171	171
query61	106	106	104	104
query62	371	207	178	178
query63	171	141	137	137
query64	2649	729	619	619
query65	1570	1626	1588	1588
query66	1822	256	201	201
query67	9861	9734	9736	9734
query68	3032	1183	761	761
query69	358	226	188	188
query70	687	616	617	616
query71	254	172	169	169
query72	2487	1718	1484	1484
query73	618	629	320	320
query74	2014	1218	1142	1142
query75	1180	1100	945	945
query76	2398	704	531	531
query77	255	251	213	213
query78	3824	3779	3298	3298
query79	2327	834	599	599
query80	1613	332	261	261
query81	482	153	131	131
query82	635	139	100	100
query83	282	202	186	186
query84	293	109	87	87
query85	782	329	286	286
query86	395	180	151	151
query87	1027	978	885	885
query88	2765	2129	2121	2121
query89	283	192	178	178
query90	2002	135	122	122
query91	126	112	93	93
query92	70	69	71	69
query93	1642	1084	741	741
query94	627	297	208	208
query95	545	376	221	221
query96	814	591	269	269
query97	1113	1073	1005	1005
query98	168	140	129	129
query99	421	342	311	311
Total cold run time: 178493 ms
Total hot run time: 82225 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.9 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c176e9993b466e2f7bd489209193f9f4d8a5109a, data reload: false

query1	0.00	0.01	0.01
query2	0.08	0.04	0.03
query3	0.25	0.09	0.11
query4	1.60	0.10	0.10
query5	0.18	0.15	0.16
query6	1.24	0.73	0.70
query7	0.04	0.00	0.01
query8	0.05	0.03	0.03
query9	0.28	0.21	0.22
query10	0.34	0.35	0.35
query11	0.17	0.12	0.12
query12	0.14	0.12	0.12
query13	0.31	0.31	0.32
query14	0.48	0.45	0.45
query15	0.37	0.34	0.35
query16	0.22	0.22	0.23
query17	0.65	0.73	0.72
query18	0.18	0.17	0.18
query19	1.08	1.09	1.13
query20	0.01	0.00	0.00
query21	15.46	0.16	0.11
query22	5.06	0.04	0.05
query23	16.21	0.26	0.10
query24	3.01	0.30	0.27
query25	0.09	0.05	0.04
query26	0.79	0.16	0.12
query27	0.04	0.03	0.03
query28	3.70	0.57	0.27
query29	12.44	3.21	2.64
query30	0.27	0.12	0.12
query31	2.76	0.38	0.17
query32	3.51	0.34	0.23
query33	1.47	1.52	1.53
query34	15.35	2.24	1.78
query35	1.75	1.78	1.74
query36	0.46	0.29	0.29
query37	0.06	0.04	0.03
query38	0.04	0.03	0.03
query39	0.03	0.03	0.02
query40	0.12	0.08	0.07
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 90.43 s
Total hot run time: 14.9 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants