[4.0] Prototype and Benchmark Higher-Level Grouped Discovery Mode #3988

Release Tracker: microsoft/mcp-pr#691 (Theme 1.4)
Initiative: #3105 (Product and Platform)
Priority: P1 (Decision Needed for 4.0 Default Mode)

Objective

Prototype and benchmark a higher-level grouped discovery mode (group -> service/namespace -> command) as a candidate for the default server mode in 4.0, balancing initial tool-list footprint against discovery hop overhead.

Scope

• Introduce a high-level grouping layer above service namespaces, establishing a group -> service/namespace -> command discovery path (adding one discovery hop compared with namespace mode).
• Benchmark the grouped mode prototype against namespace mode and consolidated mode (microsoft/mcp#849) using identical models, clients, and representative single-service and cross-service workflows.
• Measure end-to-end metrics across the evaluation matrix:
  • Total end-to-end token consumption (including discovery turns)
  • Tool-selection and parameter-binding accuracy
  • Task-completion success rate
  • End-to-end latency, accounting for additional discovery round trips
• Keep grouped mode opt-in during evaluation while preserving namespace mode as the fallback until empirical results justify any default-mode change.

Dependencies and Related Issues

• Experiment-Driven Tool Evaluation Epic: microsoft/mcp#3035
• Consolidated Mode comparison counterpart: microsoft/mcp#849
• Vally selection accuracy harness: microsoft/mcp-pr#638
• Latency and token budget benchmark harness: microsoft/mcp-pr#637

4.0 Exit Criteria

• Grouped mode prototype implemented behind an opt-in --mode configuration.
• Comparative benchmark report completed for Grouped Mode vs. Namespace Mode vs. Consolidated Mode across single-service and cross-service workflows.
• Default startup mode decision recorded for 4.0 GA with namespace mode retained as a supported fallback.