Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 18 additions & 19 deletions conformance-tests/VALIDATION_RESULTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

**Server Tests:** 40/40 passed (100%)
**Client Tests:** 3/4 scenarios passed (9/10 checks passed)
**Auth Tests:** 12/14 scenarios fully passing (178 passed, 1 failed, 1 warning, 85.7% scenarios, 98.9% checks)
**Auth Tests:** 14/15 scenarios fully passing (196 passed, 0 failed, 1 warning, 93.3% scenarios, 99.5% checks)

## Server Test Results

Expand Down Expand Up @@ -37,35 +37,35 @@

## Auth Test Results (Spring HTTP Client)

**Status: 178 passed, 1 failed, 1 warning across 14 scenarios**
**Status: 196 passed, 0 failed, 1 warning across 15 scenarios**

Uses the `client-spring-http-client` module with Spring Security OAuth2 and the [mcp-client-security](https://github.com/springaicommunity/mcp-client-security) library.

### Fully Passing (12/14 scenarios)
### Fully Passing (14/15 scenarios)

- **auth/metadata-default (12/12):** Default metadata discovery
- **auth/metadata-var1 (12/12):** Metadata discovery variant 1
- **auth/metadata-var2 (12/12):** Metadata discovery variant 2
- **auth/metadata-var3 (12/12):** Metadata discovery variant 3
- **auth/scope-from-www-authenticate (13/13):** Scope extraction from WWW-Authenticate header
- **auth/scope-from-scopes-supported (13/13):** Scope extraction from scopes_supported
- **auth/scope-omitted-when-undefined (13/13):** Scope omitted when not defined
- **auth/metadata-default (13/13):** Default metadata discovery
- **auth/metadata-var1 (13/13):** Metadata discovery variant 1
- **auth/metadata-var2 (13/13):** Metadata discovery variant 2
- **auth/metadata-var3 (13/13):** Metadata discovery variant 3
- **auth/scope-from-www-authenticate (14/14):** Scope extraction from WWW-Authenticate header
- **auth/scope-from-scopes-supported (14/14):** Scope extraction from scopes_supported
- **auth/scope-omitted-when-undefined (14/14):** Scope omitted when not defined
- **auth/scope-step-up (16/16):** Scope step-up challenge
- **auth/scope-retry-limit (11/11):** Scope retry limit handling
- **auth/token-endpoint-auth-basic (17/17):** Token endpoint with HTTP Basic auth
- **auth/token-endpoint-auth-post (17/17):** Token endpoint with POST body auth
- **auth/token-endpoint-auth-none (17/17):** Token endpoint with no client auth
- **auth/token-endpoint-auth-basic (18/18):** Token endpoint with HTTP Basic auth
- **auth/token-endpoint-auth-post (18/18):** Token endpoint with POST body auth
- **auth/token-endpoint-auth-none (18/18):** Token endpoint with no client auth
- **auth/resource-mismatch (2/2):** Resource mismatch handling
- **auth/pre-registration (6/6):** Pre-registered client credentials flow

### Partially Passing (2/14 scenarios)
### Partially Passing (1/15 scenarios)

- **auth/basic-cimd (12/12 + 1 warning):** Basic Client-Initiated Metadata Discovery — all checks pass, minor warning
- **auth/scope-step-up (11/12):** Scope step-up challenge — 1 failure, client does not fully handle scope escalation after initial authorization
- **auth/basic-cimd (13/13 + 1 warning):** Basic Client-Initiated Metadata Discovery — all checks pass, minor warning

## Known Limitations

1. **Client SSE Retry:** Client doesn't parse or respect the `retry:` field, reconnects immediately, and doesn't send Last-Event-ID header
2. **Auth Scope Step-Up:** Client does not fully handle scope step-up challenges where the server requests additional scopes after initial authorization
3. **Auth Basic CIMD:** Minor conformance warning in the basic Client-Initiated Metadata Discovery flow
2. **Auth Basic CIMD:** Minor conformance warning in the basic Client-Initiated Metadata Discovery flow

## Running Tests

Expand Down Expand Up @@ -113,4 +113,3 @@ npx @modelcontextprotocol/conformance@0.1.15 client \
### High Priority
1. Fix client SSE retry field handling in `HttpClientStreamableHttpTransport`
2. Implement CIMD
3. Implement scope step up
12 changes: 6 additions & 6 deletions conformance-tests/client-spring-http-client/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Test with @modelcontextprotocol/conformance@0.1.15.
| auth/scope-from-www-authenticate | ✅ Pass | 13/13 |
| auth/scope-from-scopes-supported | ✅ Pass | 13/13 |
| auth/scope-omitted-when-undefined | ✅ Pass | 13/13 |
| auth/scope-step-up | ❌ Fail | 11/12 (1 failed) |
| auth/scope-step-up | ✅ Pass | 12/12 |
| auth/scope-retry-limit | ✅ Pass | 11/11 |
| auth/token-endpoint-auth-basic | ✅ Pass | 17/17 |
| auth/token-endpoint-auth-post | ✅ Pass | 17/17 |
Expand Down Expand Up @@ -67,7 +67,7 @@ cd conformance-tests/client-spring-http-client

This creates an executable JAR at:
```
target/client-spring-http-client-1.1.0-SNAPSHOT.jar
target/client-spring-http-client-2.0.0-SNAPSHOT.jar
```

## Running Tests
Expand All @@ -79,7 +79,7 @@ Run the full auth suite:
```bash
npx @modelcontextprotocol/conformance@0.1.15 client \
--spec-version 2025-11-25 \
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-1.1.0-SNAPSHOT.jar" \
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-2.0.0-SNAPSHOT.jar" \
--suite auth
```

Expand All @@ -88,7 +88,7 @@ Run a single scenario:
```bash
npx @modelcontextprotocol/conformance@0.1.15 client \
--spec-version 2025-11-25 \
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-1.1.0-SNAPSHOT.jar" \
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-2.0.0-SNAPSHOT.jar" \
--scenario auth/metadata-default
```

Expand All @@ -97,7 +97,7 @@ Run with verbose output:
```bash
npx @modelcontextprotocol/conformance@0.1.15 client \
--spec-version 2025-11-25 \
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-1.1.0-SNAPSHOT.jar" \
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-2.0.0-SNAPSHOT.jar" \
--scenario auth/metadata-default \
--verbose
```
Expand All @@ -108,7 +108,7 @@ You can also run the client manually if you have a test server:

```bash
export MCP_CONFORMANCE_SCENARIO=auth/metadata-default
java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-1.1.0-SNAPSHOT.jar http://localhost:3000/mcp
java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-2.0.0-SNAPSHOT.jar http://localhost:3000/mcp
```

## Known Issues
Expand Down
14 changes: 10 additions & 4 deletions conformance-tests/client-spring-http-client/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,9 @@

<properties>
<java.version>17</java.version>
<spring-boot.version>4.0.2</spring-boot.version>
<spring-ai.version>2.0.0-M2</spring-ai.version>
<spring-boot.version>4.0.5</spring-boot.version>
<spring-ai.version>2.0.0-M4</spring-ai.version>
<spring-ai-mcp-security.version>0.1.5</spring-ai-mcp-security.version>
<maven.deploy.skip>true</maven.deploy.skip>
</properties>

Expand Down Expand Up @@ -64,7 +65,12 @@
<dependency>
<groupId>org.springaicommunity</groupId>
<artifactId>mcp-client-security</artifactId>
<version>0.1.2</version>
<version>${spring-ai-mcp-security.version}</version>
</dependency>
<dependency>
<groupId>io.modelcontextprotocol.sdk</groupId>
<artifactId>mcp-core</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>

Expand Down Expand Up @@ -106,4 +112,4 @@
</repository>
</repositories>

</project>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,11 @@

import io.modelcontextprotocol.conformance.client.scenario.Scenario;
import org.springaicommunity.mcp.security.client.sync.oauth2.metadata.McpMetadataDiscoveryService;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.DefaultMcpOAuth2ClientManager;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.DynamicClientRegistrationService;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.InMemoryMcpClientRegistrationRepository;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2ClientManager;

import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
Expand Down Expand Up @@ -49,8 +52,15 @@ McpMetadataDiscoveryService discovery() {
}

@Bean
InMemoryMcpClientRegistrationRepository clientRegistrationRepository(McpMetadataDiscoveryService discovery) {
return new InMemoryMcpClientRegistrationRepository(new DynamicClientRegistrationService(), discovery);
McpClientRegistrationRepository clientRegistrationRepository() {
return new InMemoryMcpClientRegistrationRepository();
}

@Bean
McpOAuth2ClientManager mcpOAuth2ClientManager(McpClientRegistrationRepository mcpClientRegistrationRepository,
McpMetadataDiscoveryService mcpMetadataDiscoveryService) {
return new DefaultMcpOAuth2ClientManager(mcpClientRegistrationRepository,
new DynamicClientRegistrationService(), mcpMetadataDiscoveryService);
}

@Bean
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
package io.modelcontextprotocol.conformance.client;

import io.modelcontextprotocol.conformance.client.scenario.Scenario;
import io.modelcontextprotocol.spec.McpSchema;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
Expand All @@ -27,4 +28,15 @@ public String execute() {
return "OK";
}

@GetMapping("/tools-list")
public String toolsList() {
return "OK";
}

@GetMapping("/tools-call")
public String toolsCall() {
this.scenario.getMcpClient().callTool(McpSchema.CallToolRequest.builder().name("test-tool").build());
return "OK";
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,16 @@
import io.modelcontextprotocol.conformance.client.scenario.DefaultScenario;
import org.springaicommunity.mcp.security.client.sync.config.McpClientOAuth2Configurer;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2ClientManager;

import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
import org.springframework.boot.web.server.servlet.context.ServletWebServerApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.oauth2.client.web.OAuth2AuthorizedClientRepository;
import org.springframework.security.web.SecurityFilterChain;
import static io.modelcontextprotocol.conformance.client.ConformanceSpringClientApplication.REGISTRATION_ID;

@Configuration
@ConditionalOnExpression("#{environment['MCP_CONFORMANCE_SCENARIO'] != 'auth/pre-registration'}")
Expand All @@ -25,15 +26,16 @@ public class DefaultConfiguration {
@Bean
DefaultScenario defaultScenario(McpClientRegistrationRepository clientRegistrationRepository,
ServletWebServerApplicationContext serverCtx,
OAuth2AuthorizedClientRepository oAuth2AuthorizedClientRepository) {
return new DefaultScenario(clientRegistrationRepository, serverCtx, oAuth2AuthorizedClientRepository);
OAuth2AuthorizedClientRepository oAuth2AuthorizedClientRepository,
McpOAuth2ClientManager mcpOAuth2ClientManager) {
return new DefaultScenario(clientRegistrationRepository, serverCtx, oAuth2AuthorizedClientRepository,
mcpOAuth2ClientManager);
}

@Bean
SecurityFilterChain securityFilterChain(HttpSecurity http, ConformanceSpringClientApplication.ServerUrl serverUrl) {
return http.authorizeHttpRequests(authz -> authz.anyRequest().permitAll())
.with(new McpClientOAuth2Configurer(),
mcp -> mcp.registerMcpOAuth2Client(REGISTRATION_ID, serverUrl.value()))
.with(new McpClientOAuth2Configurer(), Customizer.withDefaults())
.build();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,16 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springaicommunity.mcp.security.client.sync.AuthenticationMcpTransportContextProvider;
import org.springaicommunity.mcp.security.client.sync.oauth2.http.client.OAuth2AuthorizationCodeSyncHttpRequestCustomizer;
import org.springaicommunity.mcp.security.client.sync.oauth2.http.client.OAuth2HttpClientTransportCustomizer;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2ClientManager;

import org.springframework.boot.web.server.servlet.context.ServletWebServerApplicationContext;
import org.springframework.http.client.JdkClientHttpRequestFactory;
import org.springframework.security.oauth2.client.web.DefaultOAuth2AuthorizedClientManager;
import org.springframework.security.oauth2.client.web.OAuth2AuthorizedClientRepository;
import org.springframework.web.client.RestClient;
import static io.modelcontextprotocol.conformance.client.ConformanceSpringClientApplication.REGISTRATION_ID;
import org.springframework.web.util.UriComponentsBuilder;

public class DefaultScenario implements Scenario {

Expand All @@ -35,12 +36,19 @@ public class DefaultScenario implements Scenario {

private final DefaultOAuth2AuthorizedClientManager authorizedClientManager;

private final McpClientRegistrationRepository clientRegistrationRepository;

private final McpOAuth2ClientManager mcpOAuth2ClientManager;

private McpSyncClient client;

public DefaultScenario(McpClientRegistrationRepository clientRegistrationRepository,
ServletWebServerApplicationContext serverCtx,
OAuth2AuthorizedClientRepository oAuth2AuthorizedClientRepository) {
OAuth2AuthorizedClientRepository oAuth2AuthorizedClientRepository,
McpOAuth2ClientManager mcpOAuth2ClientManager) {
this.serverCtx = serverCtx;
this.clientRegistrationRepository = clientRegistrationRepository;
this.mcpOAuth2ClientManager = mcpOAuth2ClientManager;
this.authorizedClientManager = new DefaultOAuth2AuthorizedClientManager(clientRegistrationRepository,
oAuth2AuthorizedClientRepository);
}
Expand All @@ -51,10 +59,13 @@ public void execute(String serverUrl) {
var testServerUrl = "http://localhost:" + serverCtx.getWebServer().getPort();
var testClient = buildTestClient(testServerUrl);

var customizer = new OAuth2AuthorizationCodeSyncHttpRequestCustomizer(authorizedClientManager, REGISTRATION_ID);
HttpClientStreamableHttpTransport transport = HttpClientStreamableHttpTransport.builder(serverUrl)
.httpRequestCustomizer(customizer)
.build();
var customizer = new OAuth2HttpClientTransportCustomizer(authorizedClientManager, clientRegistrationRepository,
mcpOAuth2ClientManager);
var baseUri = UriComponentsBuilder.fromUriString(serverUrl).replacePath(null).toUriString();
var path = UriComponentsBuilder.fromUriString(serverUrl).build().getPath();
var transportBuilder = HttpClientStreamableHttpTransport.builder(baseUri).endpoint(path);
customizer.customize("default-transport", transportBuilder);
HttpClientStreamableHttpTransport transport = transportBuilder.build();

this.client = McpClient.sync(transport)
.transportContextProvider(new AuthenticationMcpTransportContextProvider())
Expand All @@ -64,6 +75,8 @@ public void execute(String serverUrl) {

try {
testClient.get().uri("/initialize-mcp-client").retrieve().toBodilessEntity();
testClient.get().uri("/tools-list").retrieve().toBodilessEntity();
testClient.get().uri("/tools-call").retrieve().toBodilessEntity();
}
finally {
// Close the client (which will close the transport)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ private void setClientRegistration(String mcpServerUrl, PreRegistrationContext o
.clientSecret(oauthCredentials.clientSecret())
.authorizationGrantType(AuthorizationGrantType.CLIENT_CREDENTIALS)
.build();
clientRegistrationRepository.addPreRegisteredClient(registration,
clientRegistrationRepository.addClientRegistration(registration,
metadata.protectedResourceMetadata().resource());
}

Expand Down
2 changes: 0 additions & 2 deletions conformance-tests/conformance-baseline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,3 @@ client:
- sse-retry
# CIMD not implemented yet
- auth/basic-cimd
# Scope step up beyond initial authorization request not implemented
- auth/scope-step-up
Loading