Skip to content

[feature](fe) Integrate OIDC authentication, MySQL login bridge, and role mapping#61819

Merged
morningman merged 28 commits intoapache:masterfrom
CalvinKirs:rich-master-auth-rolemapping-oidc
Apr 2, 2026
Merged

[feature](fe) Integrate OIDC authentication, MySQL login bridge, and role mapping#61819
morningman merged 28 commits intoapache:masterfrom
CalvinKirs:rich-master-auth-rolemapping-oidc

Conversation

@CalvinKirs
Copy link
Copy Markdown
Member

@CalvinKirs CalvinKirs commented Mar 27, 2026

#60361

This PR integrates the Doris FE authentication extension modules and delivers the end-to-end OIDC login path for MySQL clients.

The core changes are:

  1. Add MySQL protocol-layer support for OIDC login

    • bridge OIDC credentials carried in the MySQL authentication packet into Doris authentication requests
    • allow OIDC-based external login to enter the FE authentication pipeline through the MySQL protocol path
  2. Add ROLE MAPPING for built-in authorization

    • support using external identity attributes/groups to drive internal authorization decisions
  3. Integrate the authentication feature modules into FE

    • wire the authentication plugin/integration modules into the FE runtime
    • connect external authentication, JIT/external login, and built-in role authorization into one complete flow

With these changes, Doris can accept OIDC-based external logins from the MySQL protocol entry, authenticate them through the integrated authentication modules, and apply Doris-native authorization
through ROLE MAPPING.

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Move AuthIntegration role mapping into the successful authentication runtime path so granted roles are computed once during authentication and downstream authenticators only propagate the result.

### Release note

None

### Check List (For Author)

- Test: Attempted `./run-fe-ut.sh --run org.apache.doris.authentication.AuthenticationIntegrationRuntimeTest`, `./run-fe-ut.sh --run org.apache.doris.mysql.authenticate.integration.AuthenticationIntegrationAuthenticatorTest`, `./run-fe-ut.sh --run org.apache.doris.mysql.authenticate.AuthenticatorManagerTest`
    - Unit Test (blocked by existing FE build issues: concurrent cache restore collisions, then fe-core pattern generator failure `NoClassDefFoundError: org.apache.doris.nereids.pattern.generator.javaast.TypeBound`)
- Behavior changed: Yes (role mapping is now computed in `AuthenticationIntegrationRuntime` immediately after successful authentication, while `AuthenticationIntegrationAuthenticator` only propagates granted roles)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary: Define the approved design for porting CREATE/DROP ROLE MAPPING into the current OIDC branch, removing property-backed production role mapping, and adding the minimum MySQL OIDC login bridge needed to reach the existing OIDC plugin.

### Release note

None

### Check List (For Author)

- Test: No need to test (design document only)
- Behavior changed: No
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Port CREATE ROLE MAPPING and DROP ROLE MAPPING into FE metadata, switch runtime evaluation to metadata-backed role mappings, advertise the MySQL capability flags required by OIDC clients, and propagate authenticated roles into session privilege checks so OIDC logins receive mapped roles end to end.

### Release note

Support metadata-backed role mappings for authentication integrations and apply mapped roles during OIDC MySQL login.

### Check List (For Author)

- Test: FE unit tests
    - ./run-fe-ut.sh --run org.apache.doris.nereids.parser.RoleMappingParserTest,org.apache.doris.authentication.RoleMappingMgrTest,org.apache.doris.authentication.AuthenticationIntegrationMgrTest,org.apache.doris.authentication.AuthenticationIntegrationRuntimeTest,org.apache.doris.mysql.MysqlCapabilityTest,org.apache.doris.mysql.MysqlHandshakePacketTest,org.apache.doris.mysql.MysqlAuthPacketTest,org.apache.doris.mysql.authenticate.password.ClearPasswordResolverTest,org.apache.doris.mysql.authenticate.AuthenticatorManagerTest,org.apache.doris.mysql.authenticate.integration.AuthenticationIntegrationAuthenticatorTest,org.apache.doris.authentication.handler.AuthenticationServiceTest,org.apache.doris.authentication.handler.OidcAuthenticationServiceTest,org.apache.doris.mysql.privilege.AuthTest
- Behavior changed: Yes (OIDC login now consumes metadata-backed role mappings and session-mapped roles)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Prevent FE from throwing a null pointer when a client closes the MySQL connection during handshake, and preserve the original OIDC token when password auth falls back to authentication_chain so the fallback authenticator can still use the client-provided token.

### Release note

Improve FE MySQL authentication error handling and OIDC fallback behavior.

### Check List (For Author)

- Test: No test run (per user request; added FE unit tests but did not execute them)
- Behavior changed: Yes (handshake failures now return a clear error instead of a null pointer; OIDC fallback reuses the original token)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Re-enable edit log persistence for create, alter, and drop operations in AuthenticationIntegrationMgr so authentication integration metadata changes participate in FE journal replay again.

### Release note

None

### Check List (For Author)

- Test: No need to test (per user request in this session)
- Behavior changed: Yes (authentication integration DDL now writes edit log records again)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Add temporary FE-side logging around MySQL OIDC auth packet decoding and token bridging so we can determine whether the incoming mysqlsh token still contains a length-encoded prefix or is being transformed before OIDC validation.

### Release note

None

### Check List (For Author)

- Test: No need to test (debug-only instrumentation requested by user; user asked not to run tests)
- Behavior changed: Yes (adds temporary INFO logs for OIDC MySQL authentication flow only)
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Move MySQL OIDC auth-packet parsing out of ClearPasswordResolver and make packet-aware credential resolution explicit for integration and plugin authenticators.

### Release note

None

### Check List (For Author)

- Test: No test run (per user request)
    - No test execution requested by the user in this session
- Behavior changed: Yes (direct plugin-based OIDC auth now consumes credentials from the initial MySQL auth packet through an explicit resolver path)
- Does this need documentation: No
… role mapping

 The core changes are:

  1. Add MySQL protocol-layer support for OIDC login
     - bridge OIDC credentials carried in the MySQL authentication packet into Doris authentication requests
     - allow OIDC-based external login to enter the FE authentication pipeline through the MySQL protocol path

  2. Add ROLE MAPPING for built-in authorization
     - introduce role mapping so external authenticated identities can be mapped to Doris built-in roles
     - support using external identity attributes/groups to drive internal authorization decisions

  3. Integrate the authentication feature modules into FE
     - wire the authentication plugin/integration modules into the FE runtime
     - connect external authentication, JIT/external login, and built-in role authorization into one complete flow

  With these changes, Doris can accept OIDC-based external logins from the MySQL protocol entry, authenticate them through the integrated authentication modules, and apply Doris-native authorization
  through ROLE MAPPING.

  ### Release note

  Support OIDC-based external login through the MySQL protocol and add ROLE MAPPING for Doris built-in authorization.
…mapping-oidc

# Conflicts:
#	fe/fe-core/src/main/java/org/apache/doris/persist/meta/PersistMetaModules.java
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Remove the OIDC authentication plugin module and the role-mapping OIDC bridge design spec from this branch so the PR only keeps the MySQL protocol credential bridge and role-mapping related FE integration work.

### Release note

None

### Check List (For Author)

- Test: No test run (per user request)
    - No test execution requested in this session
- Behavior changed: Yes (this branch no longer submits the OIDC plugin module or the design spec; MySQL protocol-layer OIDC credential adaptation remains)
- Does this need documentation: No
@CalvinKirs
Copy link
Copy Markdown
Member Author

/review

@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

Comment thread fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/Auth.java
Comment thread fe/fe-core/src/main/java/org/apache/doris/authentication/RoleMappingMgr.java Outdated
Comment thread fe/fe-core/src/main/java/org/apache/doris/persist/meta/PersistMetaModules.java Outdated
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Address review blockers in the OIDC login and role mapping integration by including the MySQL auth packet OIDC credential extractor in fe-core, fixing role mapping metadata image persistence, aligning lock ordering between authentication integration and role mapping managers, avoiding null role materialization in Auth, using the command context consistently for DROP ROLE MAPPING privilege checks, and making evaluator cache refresh atomic.

### Release note

Fix role mapping metadata persistence and stabilize fe-core OIDC login integration.

### Check List (For Author)

- Test: No test run (per request)
    - No unit test or regression test was executed
- Behavior changed: Yes
    - Role mappings are now included in image save/load and fe-core OIDC credential extraction is committed with the protocol bridge
- Does this need documentation: No
@CalvinKirs
Copy link
Copy Markdown
Member Author

/review

@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary: OIDC Authentication, MySQL Login Bridge, and Role Mapping

This PR adds OIDC authentication support via the MySQL protocol, role mapping based on CEL expressions, and wires these into the FE authentication pipeline. The architecture is well-structured with clear separation between metadata management, runtime plugin management, and role mapping evaluation. The EditLog/persistence paths are correctly symmetric, and test coverage is good.

However, I found several issues that should be addressed before merging:

Critical Checkpoint Conclusions

1. Goal Accomplishment: The code accomplishes the stated goals of OIDC login bridge, role mapping, and integration. Tests cover the main paths.

2. Concurrency:

  • BUG (High): Auth.doesRoleExist() accesses roleManager without Auth's read lock — data race. Called from RoleMappingMgr.normalizeGrantedRoles() which does not hold Auth's lock.
  • BUG (High): Deadlock risk from inconsistent lock ordering between AuthenticationIntegrationMgr (write lock → calls RoleMappingMgr.hasRoleMapping()) and RoleMappingMgr.createRoleMapping() (IntegrationMgr read lock → own write lock). Classic ABBA pattern.
  • Issue (Medium): ThreadLocal-based CEL context passing in UnifiedRoleMappingCelEngine is non-reentrant and fragile if CEL ever delegates to another thread.

3. Lifecycle Management:

  • Issue (Medium): Image load replaces authenticationIntegrationMgr but never resets AuthenticationIntegrationRuntime — stale cached plugins persist.
  • Dead Code: replayUpsertAuthenticationIntegration() and rebuildAuthenticationIntegrations() exist but are never called.

4. EditLog Replay Correctness:

  • BUG (Medium): ALTER/DROP authentication integration replay updates only metadata, not the runtime. Followers retain stale plugins. The master-side DDL path correctly updates the runtime (dirty flag / removal), but the replay path does not.

5. Incompatible Changes:

  • Issue (Medium): MAPPING, CEL, and RULE are not in the nonReserved rule in DorisParser.g4, making them reserved keywords. This breaks any existing queries using these words as unquoted identifiers.

6. Code Quality:

  • Issue (Medium): main(), runMultipleMatchDemo(), and DemoResult class left in production UnifiedRoleMappingCelEngine. Should be in test code only.
  • Issue (Low): Defensive null checks on Env.getCurrentEnv() in AuthenticationIntegrationRuntime should be assertions per coding standard.

7. Configuration: No new configuration items requiring dynamic change support.

8. Parallel Code Paths: N/A for this feature.

9. Test Coverage: Good unit test coverage for the new modules. Role mapping, CEL engine, auth service chain, and parser rules are all tested. Missing: no regression test for the end-to-end SQL flow (CREATE ROLE MAPPING / DROP ROLE MAPPING).

10. Observability: No logging in the CEL evaluation path. When role mapping fails due to CEL errors or misconfiguration, there is no log output for diagnosis.

11. Transaction/Persistence: EditLog writes are correctly covered. The replay asymmetry (runtime not updated) is the main concern.

12. FE-BE Variable Passing: N/A — this is FE-only.

13. Performance: Cache entries in IntegrationPropertyRoleMappingEvaluator are never evicted (minor memory leak). No limit on CEL expression complexity or number of rules per integration.

integrationMgr.readLock();
try {
writeLock();
try {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Deadlock Risk - High] Inconsistent lock ordering between AuthenticationIntegrationMgr and RoleMappingMgr:

  • createRoleMapping (here): acquires IntegrationMgr.readLock → then RoleMappingMgr.writeLock
  • dropAuthenticationIntegration (AuthenticationIntegrationMgr.java:128,136): acquires IntegrationMgr.writeLock → then calls RoleMappingMgr.hasRoleMapping() which acquires RoleMappingMgr.readLock

This is a classic ABBA deadlock pattern. Thread 1 holds IntegrationMgr and waits for RoleMappingMgr; Thread 2 holds RoleMappingMgr and waits for IntegrationMgr.

Fix: Establish a consistent lock ordering. Either always acquire IntegrationMgr lock first, or refactor dropAuthenticationIntegration to check the role mapping constraint before acquiring its own write lock.

public static void main(String[] args) {
DemoResult demoResult = runMultipleMatchDemo();
System.out.println("=== unified role mapping demo ===");
System.out.println("principal: " + demoResult.getContext().getName());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Code Quality - Medium] main() method and runMultipleMatchDemo() along with the DemoResult class (lines 323-345) are left in production code. This is test/demo code that should not ship in a security-sensitive module. The test class RoleMappingCelPlayground already exists for this purpose.

Fix: Remove main(), runMultipleMatchDemo(), and DemoResult from this production class.

private static final ThreadLocal<EvaluationContext> CURRENT_CONTEXT = new ThreadLocal<>();
private static final CelCompiler COMPILER = createCompiler();
private static final CelRuntime RUNTIME = createRuntime();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Thread Safety - Medium] ThreadLocal-based context passing is fragile:

  1. Non-reentrancy: If evaluate() is ever called recursively on the same thread (e.g., via future refactoring), the outer context is silently overwritten and the finally block removes it entirely.
  2. CEL runtime thread delegation: If the CEL runtime internally delegates to a different thread, CURRENT_CONTEXT.get() returns null, causing IllegalStateException.

If the CEL library supports passing context variables through the eval() activation map rather than ThreadLocal, that would be a cleaner and safer pattern.

@@ -350,6 +354,7 @@ supportedDropStatement
| DROP WORKLOAD GROUP (IF EXISTS)? name=identifierOrText (FOR computeGroup=identifierOrText)? #dropWorkloadGroup
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Compatibility - Medium] The tokens MAPPING, CEL, and RULE are used here but are NOT listed in the nonReserved rule (lines 1985-2370). This makes them reserved keywords, meaning any existing user query that uses these words as unquoted identifiers (e.g., SELECT mapping FROM t, CREATE TABLE rule (...)) will break after this change.

Fix: Add MAPPING, CEL, and RULE to the nonReserved rule in DorisParser.g4, following the alphabetical convention used there.

@github-actions
Copy link
Copy Markdown
Contributor

Additional Findings (outside diff hunks)

1. Concurrency Bug (High): Auth.doesRoleExist()Auth.java:160-162

doesRoleExist() accesses roleManager without acquiring Auth's read lock. Per the comment at line 104-106: "There is no concurrency control logic inside roleManager... their methods cannot be directly called outside." Every other public method in Auth acquires the lock.

This is called from RoleMappingMgr.normalizeGrantedRoles() at line 231, which holds RoleMappingMgr's write lock but NOT Auth's lock, creating a data race on roleManager's internal map.

Fix: Wrap the body in readLock()/readUnlock(), or have callers acquire Auth.readLock first.

2. EditLog Replay Bug (Medium): EditLog.java:1095-1104

Replay of OP_ALTER_AUTHENTICATION_INTEGRATION and OP_DROP_AUTHENTICATION_INTEGRATION updates only the metadata manager, not the AuthenticationIntegrationRuntime. On the master-side DDL path, alterAuthenticationIntegrationProperties() calls runtime.markAuthenticationIntegrationDirty() and dropAuthenticationIntegration() calls runtime.removeAuthenticationIntegration(). On followers, after replay, stale plugins remain cached.

Note: AuthenticationIntegrationRuntime.replayUpsertAuthenticationIntegration() (line 155) and rebuildAuthenticationIntegrations() (line 162) exist but are never called — dead code that was intended to solve this.

Fix: Wire ALTER replay to call runtime.replayUpsertAuthenticationIntegration(log) and DROP replay to call runtime.removeAuthenticationIntegration(log.getIntegrationName()).

3. Lifecycle Issue (Medium): Env.java:2522-2531

Image load replaces authenticationIntegrationMgr with a freshly deserialized object, but authenticationIntegrationRuntime (caching plugins, dirty flags, runtime states) is never reset. Stale state persists from prior lifecycle.

Fix: After loading, call authenticationIntegrationRuntime.rebuildAuthenticationIntegrations(...).

4. Coding Standard (Low): AuthenticationIntegrationRuntime.java:280-288,331-333

Defensive null checks on Env.getCurrentEnv() should be assertions per Doris coding standard ("Assert correctness only — never use defensive programming"). These silent fallbacks mask bugs.

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 26434 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a864055c4031dd412198854b30fb37a3d8c336ca, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	16923	4441	4293	4293
q2	q3	10360	766	511	511
q4	4706	348	245	245
q5	7603	1192	1025	1025
q6	184	172	145	145
q7	785	828	670	670
q8	9372	1469	1300	1300
q9	4770	4782	4696	4696
q10	6237	1944	1625	1625
q11	481	256	248	248
q12	694	595	462	462
q13	18020	2716	1963	1963
q14	230	225	207	207
q15	q16	749	736	664	664
q17	738	862	435	435
q18	5994	5432	5236	5236
q19	1110	991	612	612
q20	539	510	377	377
q21	4382	1826	1486	1486
q22	345	294	234	234
Total cold run time: 94222 ms
Total hot run time: 26434 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4938	4737	4641	4641
q2	q3	3943	4386	3897	3897
q4	913	1208	809	809
q5	4083	4374	4332	4332
q6	180	171	136	136
q7	1761	1714	1533	1533
q8	2548	2724	2573	2573
q9	7552	7527	7536	7527
q10	3766	3947	3529	3529
q11	541	440	416	416
q12	488	558	451	451
q13	2443	2891	2250	2250
q14	284	299	269	269
q15	q16	715	775	696	696
q17	1159	1647	1387	1387
q18	7279	6871	6790	6790
q19	904	864	929	864
q20	2062	2166	1960	1960
q21	3943	3510	3353	3353
q22	478	457	392	392
Total cold run time: 49980 ms
Total hot run time: 47805 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 168899 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a864055c4031dd412198854b30fb37a3d8c336ca, data reload: false

query5	4337	632	508	508
query6	339	222	206	206
query7	4220	462	260	260
query8	350	241	231	231
query9	8740	2730	2671	2671
query10	515	369	337	337
query11	6993	5114	4895	4895
query12	185	129	121	121
query13	1249	455	340	340
query14	5677	3700	3430	3430
query14_1	2806	2799	2832	2799
query15	208	195	173	173
query16	968	474	433	433
query17	973	750	625	625
query18	2458	454	357	357
query19	221	214	186	186
query20	140	126	124	124
query21	229	136	110	110
query22	13269	13843	14532	13843
query23	17005	16308	16098	16098
query23_1	16103	16410	15915	15915
query24	7150	1618	1221	1221
query24_1	1214	1242	1247	1242
query25	615	484	441	441
query26	1248	280	157	157
query27	2765	479	295	295
query28	4437	1832	1860	1832
query29	865	583	497	497
query30	308	237	193	193
query31	1008	940	884	884
query32	90	71	75	71
query33	527	350	305	305
query34	896	860	526	526
query35	661	701	606	606
query36	1089	1154	990	990
query37	133	99	90	90
query38	2884	2939	2882	2882
query39	873	843	819	819
query39_1	808	792	838	792
query40	230	143	133	133
query41	61	60	58	58
query42	258	252	258	252
query43	237	247	219	219
query44	
query45	193	188	181	181
query46	873	971	601	601
query47	2107	2090	2084	2084
query48	305	310	224	224
query49	615	447	382	382
query50	696	269	221	221
query51	4200	4111	4045	4045
query52	264	267	254	254
query53	294	341	287	287
query54	301	281	265	265
query55	88	86	83	83
query56	307	315	315	315
query57	1915	1823	1670	1670
query58	288	277	261	261
query59	2769	2940	2736	2736
query60	341	330	322	322
query61	166	155	151	151
query62	684	572	550	550
query63	308	280	271	271
query64	4980	1287	991	991
query65	
query66	1446	474	350	350
query67	24400	24198	24190	24190
query68	
query69	416	313	288	288
query70	966	919	870	870
query71	331	313	292	292
query72	2823	2698	2454	2454
query73	543	533	308	308
query74	9728	9631	9403	9403
query75	2868	2766	2471	2471
query76	2197	1020	694	694
query77	349	381	312	312
query78	11023	11130	10502	10502
query79	1117	764	557	557
query80	1358	608	537	537
query81	563	262	225	225
query82	1009	156	121	121
query83	352	271	249	249
query84	296	123	96	96
query85	908	493	462	462
query86	431	317	302	302
query87	3128	3062	3021	3021
query88	3528	2633	2634	2633
query89	434	364	346	346
query90	2034	177	176	176
query91	170	177	138	138
query92	77	76	75	75
query93	1021	823	502	502
query94	651	325	306	306
query95	596	417	314	314
query96	634	503	227	227
query97	2458	2465	2383	2383
query98	242	229	224	224
query99	1014	998	930	930
Total cold run time: 250196 ms
Total hot run time: 168899 ms

@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 26457 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 908deebd8dcbced5b1803bd88270897d239f5046, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17176	4436	4306	4306
q2	q3	10418	748	523	523
q4	4676	343	251	251
q5	7606	1197	1019	1019
q6	181	178	148	148
q7	784	847	670	670
q8	9372	1508	1309	1309
q9	4698	4761	4733	4733
q10	6241	1937	1638	1638
q11	467	257	233	233
q12	747	582	471	471
q13	18024	2681	1930	1930
q14	233	228	208	208
q15	q16	733	749	660	660
q17	741	853	419	419
q18	6092	5477	5294	5294
q19	1113	982	629	629
q20	565	532	369	369
q21	4389	1796	1404	1404
q22	342	296	243	243
Total cold run time: 94598 ms
Total hot run time: 26457 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4929	4721	4585	4585
q2	q3	3961	4457	3845	3845
q4	885	1197	799	799
q5	4104	4537	4331	4331
q6	181	170	138	138
q7	1762	1664	1556	1556
q8	2491	2663	2566	2566
q9	7572	7362	7647	7362
q10	3790	3943	3578	3578
q11	500	428	419	419
q12	490	605	466	466
q13	2498	3065	2084	2084
q14	284	306	280	280
q15	q16	707	774	721	721
q17	1175	1314	1365	1314
q18	7307	6822	6634	6634
q19	907	889	920	889
q20	2098	2335	2023	2023
q21	3958	3494	3341	3341
q22	472	469	388	388
Total cold run time: 50071 ms
Total hot run time: 47319 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 168728 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 908deebd8dcbced5b1803bd88270897d239f5046, data reload: false

query5	4333	647	533	533
query6	344	222	204	204
query7	4212	465	266	266
query8	331	238	249	238
query9	8696	2709	2721	2709
query10	518	381	363	363
query11	7003	5099	4872	4872
query12	178	129	123	123
query13	1274	464	365	365
query14	5734	3723	3457	3457
query14_1	2874	2836	2831	2831
query15	199	195	177	177
query16	1008	478	428	428
query17	1119	724	612	612
query18	2451	456	360	360
query19	213	214	186	186
query20	137	128	125	125
query21	214	135	109	109
query22	13178	13999	14477	13999
query23	16652	16716	16221	16221
query23_1	16265	16035	15682	15682
query24	7014	1612	1210	1210
query24_1	1226	1227	1241	1227
query25	636	454	402	402
query26	1244	257	159	159
query27	2777	476	292	292
query28	4377	1810	1828	1810
query29	845	569	493	493
query30	298	227	191	191
query31	993	933	852	852
query32	81	72	69	69
query33	509	344	311	311
query34	883	867	523	523
query35	633	688	597	597
query36	1078	1125	1001	1001
query37	140	94	79	79
query38	2952	2973	2886	2886
query39	848	835	797	797
query39_1	801	787	789	787
query40	234	158	137	137
query41	64	64	59	59
query42	265	257	255	255
query43	247	246	224	224
query44	
query45	196	190	183	183
query46	871	984	600	600
query47	3077	2135	2066	2066
query48	309	315	227	227
query49	641	447	382	382
query50	688	285	215	215
query51	4044	4095	3965	3965
query52	264	273	257	257
query53	287	344	286	286
query54	295	268	273	268
query55	100	96	81	81
query56	306	326	305	305
query57	1910	1789	1646	1646
query58	308	272	276	272
query59	2773	2964	2790	2790
query60	345	337	332	332
query61	163	151	155	151
query62	624	586	549	549
query63	319	274	280	274
query64	4983	1280	1026	1026
query65	
query66	1473	449	366	366
query67	24125	24316	24086	24086
query68	
query69	399	309	275	275
query70	948	963	901	901
query71	328	305	292	292
query72	2823	2670	2433	2433
query73	537	540	317	317
query74	9589	9597	9419	9419
query75	2827	2781	2433	2433
query76	2193	1020	673	673
query77	364	388	302	302
query78	10831	11112	10497	10497
query79	1996	773	564	564
query80	1466	624	534	534
query81	550	260	231	231
query82	938	149	119	119
query83	327	262	240	240
query84	249	126	101	101
query85	922	530	449	449
query86	417	317	271	271
query87	3173	3100	3013	3013
query88	3557	2672	2660	2660
query89	428	378	340	340
query90	2008	183	177	177
query91	168	161	136	136
query92	75	70	72	70
query93	1029	824	504	504
query94	642	301	300	300
query95	599	348	317	317
query96	643	522	228	228
query97	2442	2463	2431	2431
query98	244	223	229	223
query99	1009	959	933	933
Total cold run time: 251192 ms
Total hot run time: 168728 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 73.24% (323/441) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Apr 2, 2026
CalvinKirs and others added 2 commits April 2, 2026 14:37
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Add a narrow gitleaks allowlist for the fake OIDC token fixture in AuthenticatorManagerTest so secret scanning does not flag this known false positive or require rewriting historical commits.

### Release note

None

### Check List (For Author)

- Test: Manual test
    - Reviewed the staged gitleaks config diff; gitleaks binary is not available in this environment for an end-to-end scan
- Behavior changed: Yes
    - Gitleaks should ignore the specific false positive in AuthenticatorManagerTest while keeping the generic-api-key rule active elsewhere
- Does this need documentation: No
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Exclude .gitleaks.toml from repository license header checks so the new gitleaks configuration file does not require an Apache license header and does not fail license validation.

### Release note

None

### Check List (For Author)

- Test: Manual test
    - Reviewed the staged diffs in .licenserc.yaml and .rat-excludes; full license-eyes CI is not available in this environment
- Behavior changed: Yes
    - License header checks should ignore .gitleaks.toml
- Does this need documentation: No
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

1 similar comment
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 29162 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ff8f6c433ae1f776d1facba93532edca7b11f588, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17058	4107	3781	3781
q2	q3	10392	858	613	613
q4	4606	461	358	358
q5	7566	1338	1158	1158
q6	195	173	139	139
q7	916	938	775	775
q8	9424	1463	1294	1294
q9	5555	5340	5245	5245
q10	6256	2039	1769	1769
q11	473	277	282	277
q12	810	704	510	510
q13	18048	2796	2184	2184
q14	283	279	262	262
q15	q16	852	853	786	786
q17	918	1103	692	692
q18	6441	5611	5538	5538
q19	1155	1239	1041	1041
q20	595	539	405	405
q21	4934	2470	2011	2011
q22	477	384	324	324
Total cold run time: 96954 ms
Total hot run time: 29162 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4630	4494	4396	4396
q2	q3	4604	4777	4220	4220
q4	2009	2105	1353	1353
q5	4913	4979	5190	4979
q6	209	178	138	138
q7	2083	1844	1684	1684
q8	3290	3008	3053	3008
q9	8492	8284	8477	8284
q10	4467	4449	4240	4240
q11	777	430	383	383
q12	650	703	480	480
q13	2763	3371	2453	2453
q14	299	307	282	282
q15	q16	758	790	705	705
q17	1282	1268	1182	1182
q18	7980	6961	6971	6961
q19	1144	1153	1112	1112
q20	2201	2208	1985	1985
q21	5987	5330	5040	5040
q22	560	519	425	425
Total cold run time: 59098 ms
Total hot run time: 53310 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 180264 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ff8f6c433ae1f776d1facba93532edca7b11f588, data reload: false

query5	4336	670	498	498
query6	337	237	211	211
query7	4219	585	338	338
query8	327	243	224	224
query9	8732	3871	3884	3871
query10	491	397	341	341
query11	6681	5498	5157	5157
query12	186	132	126	126
query13	1314	597	437	437
query14	5595	5161	4751	4751
query14_1	4104	4090	4111	4090
query15	210	205	177	177
query16	1019	458	429	429
query17	1122	784	626	626
query18	2462	500	370	370
query19	249	218	185	185
query20	141	134	128	128
query21	223	144	116	116
query22	14726	14828	14408	14408
query23	18135	17163	16689	16689
query23_1	16736	16813	16751	16751
query24	7320	1748	1353	1353
query24_1	1370	1366	1376	1366
query25	621	524	470	470
query26	1280	318	188	188
query27	2707	678	380	380
query28	4355	1893	1881	1881
query29	1033	701	577	577
query30	309	239	197	197
query31	1099	1046	965	965
query32	103	74	74	74
query33	556	370	304	304
query34	1207	1207	671	671
query35	740	763	669	669
query36	1249	1277	1079	1079
query37	165	105	89	89
query38	3143	3035	2985	2985
query39	922	900	850	850
query39_1	839	841	843	841
query40	239	165	147	147
query41	67	66	64	64
query42	291	282	287	282
query43	318	323	277	277
query44	
query45	207	214	192	192
query46	1135	1244	819	819
query47	2398	2349	2238	2238
query48	399	419	321	321
query49	638	552	429	429
query50	715	283	218	218
query51	4310	4257	4305	4257
query52	279	280	273	273
query53	328	349	273	273
query54	317	283	264	264
query55	97	95	88	88
query56	311	322	323	322
query57	1728	1705	1649	1649
query58	306	275	274	274
query59	2894	3015	2756	2756
query60	341	332	325	325
query61	160	155	154	154
query62	671	617	570	570
query63	309	268	261	261
query64	5327	1441	1124	1124
query65	
query66	1478	478	373	373
query67	24381	24447	24293	24293
query68	
query69	446	350	310	310
query70	969	1013	949	949
query71	359	320	299	299
query72	2972	2764	2503	2503
query73	773	783	468	468
query74	9926	9711	9578	9578
query75	3593	3411	3004	3004
query76	2157	1145	789	789
query77	391	404	331	331
query78	11308	11380	10748	10748
query79	1524	1081	849	849
query80	834	751	685	685
query81	456	278	241	241
query82	1245	161	129	129
query83	361	295	262	262
query84	260	141	116	116
query85	886	510	471	471
query86	399	363	312	312
query87	3328	3260	3072	3072
query88	3557	2659	2668	2659
query89	477	416	371	371
query90	1950	183	168	168
query91	175	172	143	143
query92	81	74	74	74
query93	922	897	506	506
query94	530	338	301	301
query95	685	457	332	332
query96	1017	806	320	320
query97	2701	2685	2619	2619
query98	244	227	217	217
query99	1102	1067	998	998
Total cold run time: 258795 ms
Total hot run time: 180264 ms

kaka11chen
kaka11chen previously approved these changes Apr 2, 2026
Copy link
Copy Markdown
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 2, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 2, 2026

PR approved by at least one committer and no changes requested.

@CalvinKirs CalvinKirs requested a review from morningman April 2, 2026 08:45
Copy link
Copy Markdown
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EditLog / Persistence / Concurrency Review

I did a focused review on the edit log, persistence, and concurrency aspects of this PR. Findings below.


CRITICAL: PersistMetaModules"TableStreamManager" accidentally removed from MODULE_NAMES

File: fe/fe-core/src/main/java/org/apache/doris/persist/meta/PersistMetaModules.java

The PR changes:

// master:
"constraintManager", "TableStreamManager"
// , "authenticationIntegrations"

// PR:
"constraintManager", "authenticationIntegrations", "roleMappings"

"TableStreamManager" is dropped. Since MODULES_MAP is built from MODULE_NAMES, this causes two failures:

  1. Reading old images crashes FE on startup. When MetaReader encounters "TableStreamManager" in the footer's MetaIndex, MODULES_MAP.get("TableStreamManager") returns null. With Config.ignore_unknown_metadata_module = false (the default), this throws IOException: Unknown meta module: TableStreamManager. FE cannot start from any existing image checkpoint.

  2. New image checkpoints silently drop TableStreamManager data. MetaWriter iterates MODULES_IN_ORDER (built from MODULE_NAMES). TableStreamManager is never written → data lost permanently after the next checkpoint.

Fix: append new modules after TableStreamManager:

"constraintManager", "TableStreamManager", "authenticationIntegrations", "roleMappings"

HIGH: putRoleMappingInternal throws unchecked IllegalStateException during replay

File: fe/fe-core/src/main/java/org/apache/doris/authentication/RoleMappingMgr.java

private void putRoleMappingInternal(RoleMappingMeta meta) {
    // ...
    String previousMappingName = integrationToMappingName.put(meta.getIntegrationName(), meta.getName());
    if (previousMappingName != null && !previousMappingName.equals(meta.getName())) {
        throw new IllegalStateException(...);
    }
}

This is called from both createRoleMapping (guarded by pre-checks) and replayCreateRoleMapping (no pre-checks). During replay, if duplicate mappings for the same integration exist in the journal (e.g., corrupted journal, partial write), this throws an unchecked IllegalStateException that propagates up through EditLog.loadJournal and crashes FE startup.

Recommendation: In replayCreateRoleMapping, either catch IllegalStateException and log a warning, or make the method idempotent by removing the throw and overwriting silently (which is what nameToRoleMapping.put() on the line above already does).


HIGH: Cross-class lock ordering undocumented

Files: RoleMappingMgr.java, AuthenticationIntegrationMgr.java

The lock ordering is consistent (integrationMgr → roleMappingMgr) across all paths:

Path Lock 1 Lock 2
RoleMappingMgr.createRoleMapping() integrationMgr.readLock() roleMappingMgr.writeLock()
IntegrationMgr.dropAuthenticationIntegration() integrationMgr.writeLock() roleMappingMgr.readLock() (via hasRoleMapping())

No deadlock exists today. However, readLock()/readUnlock() on AuthenticationIntegrationMgr were widened from private to package-private specifically for this cross-class protocol, with no documentation of the ordering invariant. A future developer adding a reverse-order lock acquisition would introduce a deadlock.

Recommendation: Add a lock ordering comment to both classes:

// Lock ordering: AuthenticationIntegrationMgr.lock → RoleMappingMgr.lock
// Always acquire integration lock before role mapping lock to prevent deadlocks.

MEDIUM: NPE on corrupted journal during replay

File: fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java

case OperationType.OP_CREATE_ROLE_MAPPING: {
    RoleMappingMeta log = (RoleMappingMeta) journal.getData();
    env.getRoleMappingMgr().replayCreateRoleMapping(log);  // NPE if log is null
    break;
}

If getData() returns null (corrupted journal entry), replayCreateRoleMapping(null)putRoleMappingInternal(null)null.getName() → NPE → FE fails to start.

This is consistent with the existing codebase pattern (other replay handlers have the same issue), but adding a null-guard would make role mapping replay more resilient.


Minor notes

  • Edit log write atomicity: createRoleMapping calls putRoleMappingInternal(meta) then logCreateRoleMapping(meta) inside the write lock. If the BDBJE write fails, in-memory state is updated but not persisted. On restart the mapping is lost. This is the standard Doris pattern (same as AuthenticationIntegrationMgr, SqlBlockRuleMgr, etc.) — not a new issue, just noting for awareness.

  • Edit log write holds integrationMgr.readLock(): BDBJE writes can be slow under replication pressure. While pending, DROP/ALTER AUTHENTICATION INTEGRATION operations are blocked. Latency concern only.

  • DropRoleMappingOperationLog degrades gracefully: If GSON deserializes corrupted JSON missing the "n" field, mappingName defaults to "", and removeRoleMappingInternal("") is a no-op. Good defensive design.

  • ReentrantReadWriteLock lock field in RoleMappingMgr: Correctly initialized via field initializer in the default constructor that GSON uses. No NPE after deserialization.

Copy link
Copy Markdown
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EditLog / Persistence / Concurrency Review

I did a focused review on the edit log, persistence, and concurrency aspects of this PR. Findings below.


CRITICAL: PersistMetaModules"TableStreamManager" accidentally removed from MODULE_NAMES

File: fe/fe-core/src/main/java/org/apache/doris/persist/meta/PersistMetaModules.java

The PR changes:

// master:
"constraintManager", "TableStreamManager"
// , "authenticationIntegrations"

// PR:
"constraintManager", "authenticationIntegrations", "roleMappings"

"TableStreamManager" is dropped. Since MODULES_MAP is built from MODULE_NAMES, this causes two failures:

  1. Reading old images crashes FE on startup. When MetaReader encounters "TableStreamManager" in the footer's MetaIndex, MODULES_MAP.get("TableStreamManager") returns null. With Config.ignore_unknown_metadata_module = false (the default), this throws IOException: Unknown meta module: TableStreamManager. FE cannot start from any existing image checkpoint.

  2. New image checkpoints silently drop TableStreamManager data. MetaWriter iterates MODULES_IN_ORDER (built from MODULE_NAMES). TableStreamManager is never written → data lost permanently after the next checkpoint.

Fix: append new modules after TableStreamManager:

"constraintManager", "TableStreamManager", "authenticationIntegrations", "roleMappings"

HIGH: putRoleMappingInternal throws unchecked IllegalStateException during replay

File: fe/fe-core/src/main/java/org/apache/doris/authentication/RoleMappingMgr.java

private void putRoleMappingInternal(RoleMappingMeta meta) {
    // ...
    String previousMappingName = integrationToMappingName.put(meta.getIntegrationName(), meta.getName());
    if (previousMappingName != null && !previousMappingName.equals(meta.getName())) {
        throw new IllegalStateException(...);
    }
}

This is called from both createRoleMapping (guarded by pre-checks) and replayCreateRoleMapping (no pre-checks). During replay, if duplicate mappings for the same integration exist in the journal (e.g., corrupted journal, partial write), this throws an unchecked IllegalStateException that propagates up through EditLog.loadJournal and crashes FE startup.

Recommendation: In replayCreateRoleMapping, either catch IllegalStateException and log a warning, or make the method idempotent by removing the throw and overwriting silently (which is what nameToRoleMapping.put() on the line above already does).


HIGH: Cross-class lock ordering undocumented

Files: RoleMappingMgr.java, AuthenticationIntegrationMgr.java

The lock ordering is consistent (integrationMgr → roleMappingMgr) across all paths:

Path Lock 1 Lock 2
RoleMappingMgr.createRoleMapping() integrationMgr.readLock() roleMappingMgr.writeLock()
IntegrationMgr.dropAuthenticationIntegration() integrationMgr.writeLock() roleMappingMgr.readLock() (via hasRoleMapping())

No deadlock exists today. However, readLock()/readUnlock() on AuthenticationIntegrationMgr were widened from private to package-private specifically for this cross-class protocol, with no documentation of the ordering invariant. A future developer adding a reverse-order lock acquisition would introduce a deadlock.

Recommendation: Add a lock ordering comment to both classes:

// Lock ordering: AuthenticationIntegrationMgr.lock → RoleMappingMgr.lock
// Always acquire integration lock before role mapping lock to prevent deadlocks.

MEDIUM: NPE on corrupted journal during replay

File: fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java

case OperationType.OP_CREATE_ROLE_MAPPING: {
    RoleMappingMeta log = (RoleMappingMeta) journal.getData();
    env.getRoleMappingMgr().replayCreateRoleMapping(log);  // NPE if log is null
    break;
}

If getData() returns null (corrupted journal entry), replayCreateRoleMapping(null)putRoleMappingInternal(null)null.getName() → NPE → FE fails to start.

This is consistent with the existing codebase pattern (other replay handlers have the same issue), but adding a null-guard would make role mapping replay more resilient.


Minor notes

  • Edit log write atomicity: createRoleMapping calls putRoleMappingInternal(meta) then logCreateRoleMapping(meta) inside the write lock. If the BDBJE write fails, in-memory state is updated but not persisted. On restart the mapping is lost. This is the standard Doris pattern (same as AuthenticationIntegrationMgr, SqlBlockRuleMgr, etc.) — not a new issue, just noting for awareness.

  • Edit log write holds integrationMgr.readLock(): BDBJE writes can be slow under replication pressure. While pending, DROP/ALTER AUTHENTICATION INTEGRATION operations are blocked. Latency concern only.

  • DropRoleMappingOperationLog degrades gracefully: If GSON deserializes corrupted JSON missing the "n" field, mappingName defaults to "", and removeRoleMappingInternal("") is a no-op. Good defensive design.

  • ReentrantReadWriteLock lock field in RoleMappingMgr: Correctly initialized via field initializer in the default constructor that GSON uses. No NPE after deserialization.

### What problem does this PR solve?

Issue Number: close apache#60361

Related PR: apache#61819

Problem Summary: Restore the persisted module ordering so TableStreamManager remains readable and writable when authentication integrations and role mappings are persisted, and document the cross-manager lock ordering used by authentication metadata managers.

### Release note

None

### Check List (For Author)

- Test: Checkstyle
- Behavior changed: Yes - preserves metadata compatibility by keeping TableStreamManager in the persisted module list before the new auth metadata modules.
- Does this need documentation: No
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Apr 2, 2026
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@doris-robot
Copy link
Copy Markdown

TPC-H: Total hot run time: 29145 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2eb2ba66651d135d42ef072104bdc7b865bb88a0, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17216	3749	3699	3699
q2	q3	10380	858	609	609
q4	4701	455	362	362
q5	7541	1339	1139	1139
q6	193	169	139	139
q7	924	940	775	775
q8	9430	1397	1327	1327
q9	5464	5300	5267	5267
q10	6240	2025	1785	1785
q11	473	273	282	273
q12	791	696	509	509
q13	18059	2786	2145	2145
q14	284	288	255	255
q15	q16	857	860	787	787
q17	975	1177	740	740
q18	6430	5698	5507	5507
q19	1154	1195	1075	1075
q20	611	518	402	402
q21	4876	2443	1997	1997
q22	497	414	353	353
Total cold run time: 97096 ms
Total hot run time: 29145 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4390	4385	4324	4324
q2	q3	4666	4785	4173	4173
q4	2080	2114	1322	1322
q5	4932	4975	5222	4975
q6	195	160	127	127
q7	1994	1769	1674	1674
q8	3297	3112	3088	3088
q9	8280	8264	8570	8264
q10	4439	4433	4205	4205
q11	574	419	398	398
q12	820	709	472	472
q13	2713	3448	2615	2615
q14	308	311	283	283
q15	q16	762	795	712	712
q17	1304	1244	1195	1195
q18	7894	6878	7011	6878
q19	1144	1133	1107	1107
q20	2190	2252	1949	1949
q21	6095	5270	4768	4768
q22	530	496	417	417
Total cold run time: 58607 ms
Total hot run time: 52946 ms

@doris-robot
Copy link
Copy Markdown

TPC-DS: Total hot run time: 180200 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2eb2ba66651d135d42ef072104bdc7b865bb88a0, data reload: false

query5	4351	666	520	520
query6	347	229	213	213
query7	4225	591	350	350
query8	338	257	225	225
query9	8730	3823	3844	3823
query10	494	398	341	341
query11	6672	5498	5109	5109
query12	183	134	135	134
query13	1296	608	454	454
query14	6563	5203	4771	4771
query14_1	4155	4118	4123	4118
query15	220	222	188	188
query16	1028	485	447	447
query17	1153	786	656	656
query18	2714	502	364	364
query19	245	228	193	193
query20	137	132	130	130
query21	224	144	127	127
query22	14011	15352	14284	14284
query23	17977	17063	16859	16859
query23_1	16791	16707	16904	16707
query24	8097	1932	1407	1407
query24_1	1360	1407	1432	1407
query25	594	503	535	503
query26	1259	316	179	179
query27	2630	666	367	367
query28	4340	1887	1886	1886
query29	956	668	553	553
query30	296	232	197	197
query31	1146	1040	938	938
query32	81	69	71	69
query33	535	347	280	280
query34	1191	1183	654	654
query35	752	782	686	686
query36	1233	1258	1059	1059
query37	158	95	81	81
query38	3070	3018	2979	2979
query39	915	913	860	860
query39_1	830	835	838	835
query40	237	153	138	138
query41	62	60	58	58
query42	277	272	270	270
query43	313	316	281	281
query44	
query45	206	199	189	189
query46	1123	1192	793	793
query47	2339	2327	2225	2225
query48	400	417	305	305
query49	638	531	430	430
query50	703	289	214	214
query51	4360	4255	4269	4255
query52	287	286	274	274
query53	320	338	273	273
query54	326	277	273	273
query55	101	94	88	88
query56	321	323	330	323
query57	1628	1669	1712	1669
query58	301	275	274	274
query59	2881	2995	2737	2737
query60	340	338	320	320
query61	158	154	152	152
query62	694	622	575	575
query63	309	271	261	261
query64	5021	1461	1041	1041
query65	
query66	1392	463	374	374
query67	24258	24229	24136	24136
query68	
query69	453	345	301	301
query70	1039	970	1039	970
query71	358	326	317	317
query72	3066	2872	2644	2644
query73	803	794	471	471
query74	9922	9765	9616	9616
query75	3550	3370	3039	3039
query76	2154	1116	752	752
query77	409	424	350	350
query78	11340	11413	10760	10760
query79	1491	1121	804	804
query80	1365	768	676	676
query81	502	289	243	243
query82	1245	152	123	123
query83	335	289	256	256
query84	270	143	116	116
query85	911	506	449	449
query86	441	335	349	335
query87	3263	3187	3047	3047
query88	3580	2678	2665	2665
query89	464	410	372	372
query90	1935	181	188	181
query91	182	173	152	152
query92	92	75	72	72
query93	909	892	512	512
query94	673	337	302	302
query95	668	356	416	356
query96	949	772	351	351
query97	2695	2666	2571	2571
query98	246	227	225	225
query99	1085	1075	975	975
Total cold run time: 259884 ms
Total hot run time: 180200 ms

Copy link
Copy Markdown
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 2, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 2, 2026

PR approved by at least one committer and no changes requested.

@morningman morningman merged commit c116852 into apache:master Apr 2, 2026
31 of 32 checks passed
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Apr 13, 2026
…role mapping (apache#61819)

apache#60361

This PR integrates the Doris FE authentication extension modules and
delivers the end-to-end OIDC login path for MySQL clients.

  The core changes are:

  1. Add MySQL protocol-layer support for OIDC login
- bridge OIDC credentials carried in the MySQL authentication packet
into Doris authentication requests
- allow OIDC-based external login to enter the FE authentication
pipeline through the MySQL protocol path

  2. Add ROLE MAPPING for built-in authorization
- support using external identity attributes/groups to drive internal
authorization decisions

  3. Integrate the authentication feature modules into FE
- wire the authentication plugin/integration modules into the FE runtime
- connect external authentication, JIT/external login, and built-in role
authorization into one complete flow

With these changes, Doris can accept OIDC-based external logins from the
MySQL protocol entry, authenticate them through the integrated
authentication modules, and apply Doris-native authorization
  through ROLE MAPPING.

(cherry picked from commit c116852)
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Apr 13, 2026
### What problem does this PR solve?

Issue Number: close apache#60361

Related PR: apache#61819, apache#62077, apache#61362

Problem Summary: Backport the missing FE-side authentication chain compatibility pieces required by the branch-4.1 OIDC authentication and role mapping cherry-picks so the FE build succeeds on auth_branch-4.1.

### Release note

None

### Check List (For Author)

- Test: env DORIS_PARALLELISM=16 BUILD_TYPE=ASAN ./build.sh --fe -j16
    - Manual test
- Behavior changed: Yes (restores FE-side authentication chain compatibility for the branch-4.1 auth backport)
- Does this need documentation: No
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/4.1.x dev/4.1.x-conflict reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants