Skip to content

Testing Requirements

Jump to bottom
Codex edited this page May 22, 2026 · 1 revision

Testing Requirements (MCP Server)

  • TEST-GRAPHRAG-ADHOC-001: GraphEntityEntity/GraphRelationshipEntity persist with all fields, workspace isolation, cascade delete, FK validation, and RemoveVector correctness.
  • TEST-GRAPHRAG-ADHOC-002: IngestTextAsync creates document + chunks, generates embeddings, registers vectors, handles empty content, defaults SourceType/SourceKey, and optionally triggers reindex.
  • TEST-GRAPHRAG-ADHOC-003: ListDocumentsAsync pagination and filtering, GetDocumentChunksAsync ordering, DeleteDocumentAsync cascade and vector cleanup.
  • TEST-GRAPHRAG-ADHOC-004: Create/Get/Update/List/Delete for entities and relationships with ID generation, timestamp management, FK validation, cascade behavior.
  • TEST-GRAPHRAG-ADHOC-005: All CQRS command and query handlers delegate to IGraphRagService and wrap results in Result.
  • TEST-GRAPHRAG-ADHOC-006: All 14 controller actions return correct HTTP status codes, content types, and error responses.
  • TEST-GRAPHRAG-ADHOC-007: MCP tools serialize correctly, REPL workflow delegates to ContextClient, McpAgent tool adapter exposes all 14 tools.
  • TEST-MCP-001: Given configurable RepoRoot/Todo paths, when service starts, then path resolution is correct.
  • TEST-MCP-002: Given TODO API operations, when create/update/delete/query run, then contracts remain stable.
  • TEST-MCP-003: Given multi-tenant workspace configuration, when requests are made with different X-Workspace-Path headers, then data remains isolated per workspace on the single shared port.
  • TEST-MCP-004: Given vector + FTS data, when context search executes, then hybrid results are returned.
  • TEST-MCP-005: Given GitHub sync enabled, when issue sync runs, then ISSUE-* mapping is consistent.
  • TEST-MCP-006: Given STDIO mode, when tool requests are sent, then parity with HTTP behavior is preserved.
  • TEST-MCP-007: Given workspace registration, when a workspace is created, then its directory scaffold is created and an AGENTS-README-FIRST.yaml marker file is written to its root pointing to the shared host port.
  • TEST-MCP-008: Given tool registry with tags, when keyword search runs with a singular or plural term, then matching tools from both global and workspace scopes are returned. Given default buckets in config, when the server starts for the first time, then buckets are seeded and idempotent on subsequent starts.
  • TEST-MCP-009: Given per-workspace auth tokens and X-Workspace-Path header resolution, when a request to any /mcpserver/* endpoint lacks X-Api-Key, then the server returns 401. When a valid token is provided, workspace resolution uses the three-tier chain: X-Workspace-Path header → API key reverse lookup → default workspace.
  • TEST-MCP-010: Given valid pairing credentials, when the /pair login flow completes, then an HttpOnly session cookie is issued and the API key is returned. Given constant-time comparison, when two passwords of the same length differ by one character, then timing side-channel is not exploitable.
  • TEST-MCP-011: Given a configured tunnel provider, when the hosted service starts, then the tunnel process launches and GetStatusAsync returns a public URL. When the service stops, the process is terminated within 5 s.
  • TEST-MCP-012: Given an MCP client connecting to /mcp-transport, when a tool call is made, then the response is semantically equivalent to the corresponding REST endpoint result. Given a request without the required Accept header, then the endpoint returns 406.
  • TEST-MCP-013: Given a workspace, when StartAsync completes, then AGENTS-README-FIRST.yaml exists at the workspace root with the shared host port, endpoint paths, and auth token. When StopAsync completes, then the marker file is removed.
  • TEST-MCP-014: Given a TODO item with a title and description, when RequirementsService.AnalyzeAsync is called, then ExtractRequirementIds correctly parses both JSON-block and regex-fallback response formats and returns distinct, non-empty FR/TR ID lists.
  • TEST-MCP-015: Given a Markdown file with a # Session Log - {title} header, when MarkdownSessionLogParser.TryParse is called, then it returns a UnifiedSessionLogDto with matching title, model, status, and at least one entry. Given a file without the header, then TryParse returns null.
  • TEST-MCP-026: Given a CQRS Dispatcher with a registered command handler, when SendAsync is called with a valid command, then the handler is invoked and Result<T>.IsSuccess is true with the expected value.
  • TEST-MCP-027: Given a CQRS command handler that returns Result.Failure(error), when the Dispatcher processes the result, then it logs at Warning level with the error message and correlation context.
  • TEST-MCP-028: Given a CQRS command handler that throws an exception, when the Dispatcher catches it, then Result<T>.IsFailure is true, Result<T>.Exception is set, and the Dispatcher logs at Error level with exception details.
  • TEST-MCP-029: Given a new CorrelationId, when Next() is called multiple times, then the base ID remains stable and the counter increments sequentially. Given a correlation string "12345678.3", when CorrelationId.Parse is called, then BaseId is 12345678 and the counter is 3.
  • TEST-MCP-030: Given a CallContext used as ILogger, when LogInformation is called, then the log entry is captured in CallContext.LogEntries with the correct level, message, and timestamp. Given the Dispatcher's ILoggerProvider, when a DispatcherLogger emits a log with a correlation scope, then the structured output includes correlationBaseId and correlationStep as separate fields.
  • TEST-MCP-031: Given two pipeline behaviors registered in order, when a command is dispatched, then the first behavior's pre-processing runs before the second, and the second's post-processing runs before the first's. Given a behavior that returns Result.Failure without calling next, then the handler is never invoked.
  • TEST-MCP-032: Given an empty database, when AgentService.SeedBuiltInDefaultsAsync is called, then 7 built-in agent definitions are created. Given a database already containing built-in agents, when SeedBuiltInDefaultsAsync is called again, then no duplicates are created (idempotent).
  • TEST-MCP-033: Given the AgentService, when UpsertDefinitionAsync creates a new definition and GetDefinitionAsync retrieves it, then all fields match. When DeleteDefinitionAsync is called on a non-built-in definition, then it succeeds. When called on a built-in definition, then it returns IsFailure with an appropriate error.
  • TEST-MCP-034: Given a workspace with an agent configured, when BanAgentAsync is called with Global = false, then only that workspace's agent is banned. When called with Global = true, then all workspaces with that agent are banned. When UnbanAgentAsync is called, then the agent is re-enabled.
  • TEST-MCP-035: Given a Director LoginHandler, when the device authorization flow completes successfully, then Result<LoginResult>.IsSuccess is true and the token is cached. When the flow times out, then Result<LoginResult>.IsFailure with an appropriate error.
  • TEST-MCP-036: Given a Director LaunchAgentHandler, when the agent is enabled and not banned, then the agent process is spawned and Result<LaunchResult>.IsSuccess. When the agent is banned, then Result<LaunchResult>.IsFailure with a ban reason.
  • TEST-MCP-037: Given a Director InitWorkspaceHandler, when called on a valid workspace path, then agents.yaml is created and agents are registered via the MCP Server API. When the workspace path doesn't exist, then Result.IsFailure.
  • TEST-MCP-038: Given a Director BanAgentHandler, when called with a valid agent ID and reason, then the MCP Server API is called to ban the agent and a Ban event is logged. When the agent doesn't exist, then Result.IsFailure.
  • TEST-MCP-039: Given seeded canonical requirements docs, when RequirementsDocumentService loads them, then FR/TR/TEST/mapping entries are parsed correctly and generated Markdown preserves the canonical header and entry formats.
  • TEST-MCP-040: Given /mcpserver/requirements CRUD endpoints, when an FR entry is created, generated via /mcpserver/requirements/generate?doc=functional, and deleted, then the generated document reflects each mutation and the deleted entry is no longer returned.
  • TEST-MCP-041: Given /mcpserver/requirements/generate?doc=all, when the endpoint is called, then it writes Functional-Requirements.md, Technical-Requirements.md, Testing-Requirements.md, TR-per-FR-Mapping.md, and Requirements-Matrix.md to the workspace, preserves existing matrix rows, appends missing FR/TR/TEST IDs, and returns export metadata.
  • TEST-MCP-042: Given concurrent requirement mutations, when RequirementsDocumentService persists updates, then writes remain atomic and the resulting Markdown files remain parseable without temp-file residue.
  • TEST-MCP-043: Given MCP STDIO requirements tools (requirements_list, requirements_generate, requirements_create, requirements_update, requirements_delete), when agents invoke them, then results are semantically equivalent to the corresponding REST requirements endpoints.
  • TEST-MCP-044: Given the three-tier workspace resolution chain, when X-Workspace-Path header is present, then it takes priority over API key reverse lookup; when absent, API key resolves workspace; when neither is present, the default workspace is used.
  • TEST-MCP-045: Given EF Core global query filter on WorkspaceId, when entities are inserted for workspace A and queried from workspace B context, then workspace A's entities are not visible from workspace B.
  • TEST-MCP-046: Given the Director client with WorkspacePath set, when API calls are made, then the X-Workspace-Path header is sent on all requests and workspace switching only changes the header, not the base URL.
  • TEST-MCP-047: Given the typed client library with McpServerClientOptions.WorkspacePath set, when requests are sent, then the X-Workspace-Path header is present alongside X-Api-Key.
  • TEST-MCP-048: Given a TODO item in workspace A, when POST /mcpserver/todo/{id}/move is called with targetWorkspacePath pointing to workspace B, then the item is created in workspace B with all fields preserved and deleted from workspace A. Given an invalid target workspace path, then the endpoint returns 400.
  • TEST-MCP-049: Given voice conversation endpoints, when a session is created with a DeviceId, then GET /mcpserver/voice/session?deviceId= returns the active session. When DELETE is called, then the session is destroyed and subsequent status queries return 404.
  • TEST-MCP-050: Given an active voice session, when POST /mcpserver/voice/session/{id}/turn is called with transcript text, then a VoiceTurnResponse is returned with assistant text. When POST /mcpserver/voice/session/{id}/turn/stream is called, then SSE events are streamed with type values of chunk, tool_status, done, or error.
  • TEST-MCP-051: Given a voice session with SessionIdleTimeoutMinutes configured, when the session is idle beyond the timeout, then the IdleShutdownCommand is sent and the session is terminated. Given a device with an active session, when a new session is requested for the same device, then the existing session is returned (one-per-device enforcement).
  • TEST-MCP-052: Given a Windows service running as LocalSystem, when DesktopProcessLauncher.LaunchWithStdio is called, then a process is created on the interactive desktop with redirected stdio pipes via CreateProcessAsUser. Given LaunchVisible, then a visible console window is created.
  • TEST-MCP-053: Given an appsettings.yaml file with configuration overrides, when the server starts, then YAML values override appsettings.json values for matching keys. Given the YAML file is absent, then startup succeeds using JSON configuration only.
  • TEST-MCP-054: Given the template service with a YAML file, when GET /mcpserver/templates is called with optional category, tag, and keyword query parameters, then a filtered list of PromptTemplate items is returned. When no filters are provided, all templates are returned.
  • TEST-MCP-055: Given a valid PromptTemplateCreateRequest, when POST /mcpserver/templates is called, then a new template is persisted to YAML and returned with 201 Created. When a duplicate ID is submitted, then 409 Conflict is returned. When PUT /mcpserver/templates/{id} is called with partial update fields, only specified fields are changed. When DELETE /mcpserver/templates/{id} is called, the template is removed.
  • TEST-MCP-056: Given a stored template with Handlebars content and declared variables, when POST /mcpserver/templates/{id}/test is called with variable values, then PromptTemplateTestResult contains the rendered content. When required variables are missing, then MissingVariables is populated and Success is false. When POST /mcpserver/templates/test is called with inline template content, the inline content is rendered without requiring a stored template.
  • TEST-MCP-057: Given the Director TUI with a Templates tab, when TemplateListViewModel.RefreshCommand executes, then the CQRS ListTemplatesQuery flows through ListTemplatesQueryHandler → ITemplateApiClient → TemplateClient → REST API and populates the table view. When a template is selected and the test action is invoked via TemplateDetailViewModel, then TestTemplateQuery renders the template and displays output.
  • TEST-MCP-058: Given FileMarkerPromptProvider with a valid templates/default-marker-prompt.hbs.yaml file, when GetGlobalPromptTemplateAsync is called, then the template content is returned and cached. When the file is missing, then null is returned and MarkerFileService.DefaultPromptTemplate is used as fallback.
  • TEST-MCP-059: Given TodoPromptProvider with todo prompt templates stored in IPromptTemplateService by well-known IDs (todo-status-prompt, todo-implement-prompt, todo-plan-prompt), when the provider is queried, then file-loaded content is returned. When templates are missing from the store, then TodoPromptDefaults built-in constants are returned as fallback.
  • TEST-MCP-060: Given PairingHtmlRenderer with pairing HTML templates stored in IPromptTemplateService by well-known IDs (pairing-login-page, pairing-key-page, pairing-not-configured-page), when rendering methods are called with substitution parameters, then {errorBanner}, {apiKey}, and {serverUrl} tokens are replaced. When templates are missing, then PairingHtml static method output is returned as fallback.
  • TEST-MCP-061: Given Mcp:AgentPool:Agents configuration, when options bind and validate, then each definition accepts AgentName, AgentPath, AgentModel, AgentSeed, AgentParameters, IsInteractiveDefault, IsTodoPlanDefault, IsTodoStatusDefault, and IsTodoImplementDefault. Duplicate AgentName values (case-insensitive) or ambiguous defaults fail validation.
  • TEST-MCP-062: Given one-shot or interactive requests with no AgentName, when context is Plan, Status, Implement, or AdHoc, then the default agent mapped for that intent is selected. Given an explicit AgentName, explicit assignment overrides default routing.
  • TEST-MCP-063: Given one-shot request payloads, when both promptTemplateId and ad-hoc promptText are supplied or both are missing without resolvable context template, then the API returns 400. Given template-resolved mode, missing id returns 400.
  • TEST-MCP-064: Given one-shot context without template ID, when context is Plan, Status, or Implement, then existing context-based template resolution is used. Given context AdHoc without template ID and no ad-hoc prompt text, then the API returns 400.
  • TEST-MCP-065: Given prompt resolution requests with template ID, caller values, workspace-context values, and id, when rendering executes, then output includes {id} substitution and caller values override workspace-context values on key conflicts.
  • TEST-MCP-066: Given no eligible pooled agent is idle, when one-shot requests are enqueued, then requests remain queued and transition to processing once an eligible agent becomes available, followed by terminal states (completed, failed, or canceled).
  • TEST-MCP-067: Given queue operations, when move up/down is requested for queued items, then order changes correctly; when requested for the currently processing item, the operation is rejected. Cancel/remove semantics correctly update queue state and persisted metadata.
  • TEST-MCP-068: Given pool lifecycle transitions, when queued/processing/completed/failed events occur, then notification SSE emits events in order with payload fields AgentName, LastRequestPrompt, and SessionId.
  • TEST-MCP-069: Given multiple clients connected to a read-only response stream, when one client disconnects, then remaining subscribers continue receiving stream data and active pooled work is unaffected.
  • TEST-MCP-070: Given a pooled agent processing a one-shot request, when an interactive voice connection targets that agent, then interactive linkage is established without canceling or reassigning the one-shot operation.
  • TEST-MCP-071: Given an interactive stream connection, when the client disconnects, then User is AFK. is sent to the agent. When the client reconnects and stream establishment completes, then User is here. is sent. These messages are not sent for one-shot sessions.
  • TEST-MCP-072: Given Director Agent Pool tab actions (connect, recycle, stop/start, queue move up/down, cancel/remove, free-form enqueue), when invoked from UI commands, then the correct REST endpoints are called and UI state refreshes from server snapshots and notifications.
  • TEST-MCP-073: Given McpServer.Support.Mcp stateful services and registries, when architecture validation runs, then each authoritative data source is DI-owned (singleton/scoped), no stateful service is created outside DI, change notifications use INotifyPropertyChanged, and consumers pull current state from the source-of-truth service.
  • TEST-MCP-074: Given TODO create/update and session log submit/append requests, when IDs violate canonical naming formats (TODO persisted ids: ^[A-Z][A-Z0-9]*(?:-[A-Z0-9]+)+-\d{3}$ or ^ISSUE-\d+$, sessionId: <Agent>-<yyyyMMddTHHmmssZ>-<suffix>, requestId: req-<yyyyMMddTHHmmssZ>-<slugOrOrdinal>), then APIs reject with validation errors and no data mutation occurs; valid IDs are accepted across YAML and SQLite TODO backends.
  • TEST-MCP-075: Given ChannelChangeEventBus, when events are published with zero, one, or multiple subscribers, then publish does not throw and each active subscriber receives events independently; canceled subscriptions stop enumeration.
  • TEST-MCP-076: Given TODO, session log, and repo mutation services, when create/update/delete-style operations succeed, then each service publishes one change event with the expected category/action/entityId values.
  • TEST-MCP-077: Given extended mutation services (ToolRegistryService, WorkspaceService, AgentService), when representative create/update operations succeed, then each service publishes the expected category/action event.
  • TEST-MCP-078: Given GET /mcpserver/events, when a client subscribes, then the response content type is text/event-stream.
  • TEST-MCP-079: Given GET /mcpserver/events?category=todo, when a TODO change event is published, then the stream includes an event: todo payload containing the matching entity ID.
  • TEST-MCP-080: Given category filtering on /mcpserver/events, when non-matching categories are published, then filtered subscribers do not receive those non-matching domain events.
  • TEST-MCP-081: Given workspace-scoped GitHub auth endpoints, when a token is set via PUT /mcpserver/gh/auth/token, then GET /mcpserver/gh/auth/status reports hasStoredToken=true; when DELETE /mcpserver/gh/auth/token is called, the token is removed.
  • TEST-MCP-082: Given GitHub OAuth bootstrap endpoints, when GET /mcpserver/gh/oauth/config is called, then effective configuration fields are returned; when OAuth is not fully configured, GET /mcpserver/gh/oauth/authorize-url returns 400 with a clear error.
  • TEST-MCP-083: Given GitHubCliService with a stored workspace token, when GitHub commands are executed, then IProcessRunner receives a ProcessRunRequest containing GitHubTokenOverride; when no token exists and fallback is enabled, standard CLI execution is used.
  • TEST-MCP-084: Given GitHub Actions workflow operations, when list/detail/rerun/cancel paths are invoked, then gh CLI commands and REST/client contracts for /mcpserver/gh/actions/runs* remain consistent and parse expected run/job/step metadata.
  • TEST-MCP-085: Given natural-language workspace policy directives, when POST /mcpserver/workspace/policy or workspace_policy_apply is invoked with valid directives, then targeted workspace ban lists are mutated and invalid directives return structured 400 errors.
  • TEST-MCP-086: Given audited copilot decoration, when invoke and streaming operations execute, then session-log submissions include copilot_invocation actions and completed status records.
  • TEST-MCP-087: Given ingestion options without the required new-project allowlist patterns, when host configuration post-processing runs, then src/McpServer.Cqrs, src/McpServer.Cqrs.Mvvm, src/McpServer.UI.Core, and src/McpServer.Director glob patterns are enforced; marker prompt output includes the Available Capabilities section for these projects.
  • TEST-MCP-088: Given direct website URL ingestion requests, when POST /mcpserver/context/ingest-website or context_ingest_website runs, then valid HTTP/HTTPS pages ingest as external-web sources, URL outcomes are returned, SSRF/private/link-local targets are blocked, redirects are bounded, and source dedup/update behavior is preserved by source key.
  • TEST-MCP-089: Given a .NET 9 host application that registers the hosted Microsoft Agent Framework library against an MCP Server workspace, when the built-in agent workflow runs, then session log turns are created/updated through canonical identifiers, TODO plan/status/implementation operations execute through the existing MCP Server contracts, repository read/list/write tools browse repo-relative paths without host-specific glue code, local desktop process launch reuses the authenticated workspace desktop-launch contract, mcp_powershell_session_* tools execute commands inside a persistent in-process PowerShell session hosted by the agent itself, and host applications can drive the same local runspace interactively through IMcpHostedAgent.PowerShellSessions.
  • TEST-MCP-090: Given representative controller and middleware failure paths across the server, when an unhandled exception produces HTTP 500, then the response body contains a non-empty detailed error description for the failed operation, excludes secrets and raw stack traces, and remains consistent across endpoints through the shared error-handling path.
  • TEST-MCP-091: Given the admin configuration management surface, when configuration values are read or patched through the configuration controller and YAML helper, then effective settings are exposed as flattened key-value pairs, submitted YAML-backed keys are persisted and reloaded, and standard JWT Bearer admin authorization keeps the endpoints unavailable when OIDC is disabled.
  • TEST-MCP-092: Given a TODO create request with id ISSUE-NEW, when GitHub issue creation succeeds, then the server persists the TODO using the canonical ISSUE-{number} id returned by GitHub, includes GitHub correlation metadata in the TODO note, and returns the canonical id from the create surface instead of the temporary alias.
  • TEST-MCP-093: Given workspace-scoped GitHub CLI execution, when gh commands run with either a stored workspace token or fallback authentication, then the process runner receives the resolved workspace root as the working directory.
  • TEST-MCP-094: Given an existing ISSUE-{number} TODO updated through any server TODO update surface, when the local update succeeds, then the server preserves the existing description, syncs title/state/priority metadata back to GitHub using canonical priority: HIGH|MEDIUM|LOW labels, and posts a GitHub issue comment describing the applied change set.
  • TEST-MCP-095: Given an ISSUE-NEW TODO created through the HTTP API, when GitHub-origin comments are added and GitHub-to-TODO sync runs, then the TODO note gains the generated GitHub comment section without altering the TODO description. When the TODO priority changes and a TODO-authored note comment is appended, then the GitHub issue receives the canonical updated priority label and a GitHub comment containing the appended note text. When the GitHub issue is later closed externally and GitHub-to-TODO sync runs again, then the TODO is marked done.
  • TEST-MCP-096: Given an empty authoritative SQLite TODO store and an existing TODO.yaml, when initialization runs and later authoritative mutations project back to YAML, then bootstrap import, deterministic projection ordering, projection-only YAML behavior, and preservation of notes, completed, and code-review-remediation metadata are all verified by automated tests.
  • TEST-MCP-097: Given a TODO item mutated through create/update/delete and queried through storage, REST, typed client, and integration surfaces, when audit history is requested, then append-only ordered states, delete-history retention, not-found behavior, and explicit projection-failure classification are all verified by automated tests.
  • TEST-MCP-098: Given a Parseable-bound log event with more than 250 structured properties and user properties that collide with reserved Parseable field names, when ParseableEventFormatter serializes the event, then the payload contains at most 250 top-level fields, canonical reserved metadata is preserved, and overflow non-reserved properties are deterministically omitted.
  • TEST-MCP-099: Given the repository Azure DevOps pipeline definition, when tracked paths change on main or develop or through a pull request, then azure-pipelines.yml triggers the core CI workflow, runs config validation/build/test/docs/MSIX/package jobs with the documented branch and variable gates, and skips optional feed/docs publication safely when the required Azure DevOps variables are absent.
  • TEST-MCP-100: Given a PowerShell McpSession workspace with an active session cached in .mcpSession/current-session.json, when the legacy .mcpServer/session.yaml wrapper is missing and the module is reinitialized or resolves a session without an explicit Session argument, then it reuses the cached current session object and session ID. When the session is completed, both cache files are removed.
  • TEST-MCP-101: Given trust-bootstrap marker rendering and the public PowerShell bootstrap modules, when the marker signature is valid and /health echoes the submitted nonce exactly, then McpSession, McpTodo, and McpContext initialize successfully and proceed with MCP usage. When the signature is invalid or the nonce does not match, then each bootstrap module emits MCP_UNTRUSTED, does not probe additional endpoints, and aborts MCP usage before session-log or TODO traffic continues.
  • TEST-MCP-102: Given the provider-factory, native encryption, and maintenance-command workstreams, when SQLite, PostgreSQL, and SQL Server are configured for clean-database integration runs, then provider-specific migrations apply successfully and the live encryption state matches the configured state. When encryption is enabled or disabled on an existing database, then the provider-specific transition workflow shall expose a no-data-loss maintenance procedure and dry-run plan before mutation, with automated dry-run tests covering SQLite SEE, PostgreSQL pg_tde, and SQL Server TDE command generation. SQL Server provider and migration integration coverage shall use self-managed SQL Server LocalDB instances that are created and torn down by the test harness, while SQL Server TDE validation shall run against a separate non-LocalDB SQL Server target because LocalDB cannot validate TDE.
  • TEST-MCP-103: Given a Byrd execution TODO, when unit tests are not defined, then the service rejects transition to Implementing; when unit tests are defined through the test-plan API, then the TODO advances to TestReady.
  • TEST-MCP-104: Given a Byrd execution TODO linked to requirements, session turns, and modified files, when bounded execution context or checkpoint delta context is requested, then the server returns only concise snippets, recent turn summaries, relevant files, artifacts, commits, and updated next action for that TODO.
  • TEST-MCP-105: Given the Byrd execution REST controller, STDIO MCP tools, typed client, and adb_step surface, when representative phase creation, active TODO lookup, status progression, and screenshot validation calls are executed, then structured contracts remain stable and Android validation results are returned without arbitrary shell passthrough.
  • TEST-MCP-106: Given requirements export with doc=all and format=wiki, when generation runs, then docs/Project/wiki contains both azure/ and github/ folders, each manifest includes generatedAtUtc, Azure includes .order, and GitHub includes _Sidebar.md and _Footer.md.
  • TEST-MCP-107: Given wiki ingest with Azure and GitHub document folders, when manifest and file modified timestamps identify a newer source, then import selects that source; when the two checks disagree, import fails unless preferredWikiFormat is supplied.
  • TEST-MCP-108: Given the REPL requirements workflow, when wiki export or import is invoked, then export returns format, docType, generatedAtUtc, outputRoot, and written file metadata, and import accepts path-keyed documents with per-document timestamps.
  • TEST-MCP-109: Given Codex, Claude Code, Copilot, and Cline agent plugins, when requirements wiki workflows are used, then each plugin exposes the wiki requirements contract and routes generate/ingest envelopes without expecting archive bytes.
  • TEST-MCP-110: Question CRUD (happy, validation, 404).
  • TEST-MCP-111: Answer CRUD (orphan rejection on deleted question, cascade delete).
  • TEST-MCP-112: Accept-answer flow (single-accept invariant, un-accept clears).
  • TEST-MCP-113: Tag filter AND-semantics + empty-result case.
  • TEST-MCP-114: Vote increment / decrement on Question and Answer; concurrency atomicity.
  • TEST-MCP-115: Comment thread create / list / delete with depth-cap enforcement.
  • TEST-MCP-116: FAQ endpoint projection shape, ordering, deeplink format.
  • TEST-MCP-117: Search: created question/answer text is found via IContextSearchService; removed on delete.
  • TEST-MCP-118: Author resolution precedence (body > API key > JWT > anonymous-rejected).
  • TEST-MCP-119: Workspace isolation (mirror EfTodoService_WorkspaceIsolationTests).
  • TEST-MCP-120: MCP STDIO tool parity for each REST endpoint.
  • TEST-MCP-121: QaClient end-to-end against CustomWebApplicationFactory.
  • TEST-MCP-122: QaWorkflow unit tests in tests/McpServer.Repl.Core.Tests (NSubstitute over QaClient); REPL agent-stdio integration test in tests/McpServer.Repl.IntegrationTests that spawns the host with ReplChildProcessHelper, sends a workflow.qa.* YAML envelope, asserts the response shape.
  • TEST-MCP-123: PowerShell module Pester tests (if a tools/powershell/tests/ pattern exists, otherwise smoke-script invoked from ./build.ps1 Test or a new ValidatePowerShell target).
  • TEST-MCP-124: Skill smoke test: each new qa/SKILL.md is loaded by the plugin packager and its frontmatter passes the standard skill validation script in each plugin repo.
  • TEST-MCP-125: Audit emission tests: one audit row per mutation, correct Action, Version monotonic per (EntityKind, EntityId), Actor populated via IQaAuthorResolver, SnapshotJson round-trips.
  • TEST-MCP-126: Audit query tests: paging contract, filter combinations, empty-result case, workspace isolation (audits from workspace A invisible to workspace B).
  • TEST-MCP-127: Vote audit: an UPDATE ... SET VoteCount = VoteCount + @delta plus an audit row are emitted in a single transaction (both succeed or both rollback). Audit row's Actor is populated from IQaAuthorResolver, so voter identity is captured per event.
  • TEST-MCP-128: Answer-with-sources: round-trip CreateAnswerRequest.Sources -> AnswerEntity.SourcesJson -> AnswerDto.Sources; FAQ projection includes the sources array; deletion of an answer hard-deletes the sources via the existing cascade.
  • TEST-MCP-129: Skill mandate text test: each sibling-plugin skills/qa/SKILL.md contains the exact mandatory rule block (regex match on the callout) and the sources[] schema example. Validation is a small PowerShell or dotnet test content-check (tests/McpServer.Qa.Validation/SkillMandateTests.cs or a tools/plugin-skill-check.ps1 invoked from ./build.ps1 Test).
  • TEST-MCP-130: Close / duplicate flow tests: close-with-reason, reopen, mark-as-duplicate (canonical link both ways), FAQ excludes closed by default, FAQ surfaces duplicate redirect when requested with ?includeClosed=true, audit rows captured per transition.
  • TEST-MCP-131: Sanitization test corpus: XSS-payload corpus validates that every common attack vector is stripped on Question/Answer/Comment write; bodyHtml contains only allow-listed tags/attributes; raw body is preserved verbatim; round-trip Markdown -> HTML matches snapshot.
  • TEST-MCP-132: FAQ wiki page generation test: build target produces deterministic Markdown matching the snapshot fixture, wiki index files updated, generated page renders cleanly in both Azure DevOps and GitHub wiki conventions (e.g., _Sidebar.md / .order references present).
  • TEST-MCP-133: Voter-history endpoint: posting N votes from M distinct actors produces N audit rows; GET /questions/{id}/voters returns exactly those rows projected to { actor, action, createdAt }; same for answers; workspace isolation enforced.
  • TEST-MCP-134: One-vote-per-user enforcement: same actor posts vote_up twice -> second call is no-op (no counter change, no second audit row); actor posts vote_up then vote_down -> counter delta is -2, audit row recorded with action vote_change; actor revokes vote -> counter delta is -1, audit row vote_revoke; unique index prevents duplicate QaVoteEntity rows under concurrent calls (test with parallel writes against in-memory SQLite using Task.WhenAll).
  • TEST-MCP-135: Current vote state endpoint: GET /questions/{id}/votes returns one row per active voter from QaVoteEntity after a sequence of apply / change / revoke calls; revoked voters do not appear; workspace isolation enforced.
  • TEST-MCP-136: Hub-and-spoke federation tests: config role defaults, durable proxy/workspace/operation storage, hub enrollment and status, LocalProxy /mcp-transport routing, operation headers, queued write fallback, replay candidate persistence, stale-version conflict creation, and provider migration compilation. Covered by: FederationMiddlewareTests, FederationTopologyServiceTests, FederationProxyServiceTests, FederationEntityModelTests
  • TEST-MCP-137: Given templates/prompt-templates.yaml, when the marker-template contract tests run, then default-marker-prompt contains the frontier-to-implementation planning guidance, explicit requirements capture guidance, and TDD unit-test planning guidance.
  • TEST-MCP-138: Unit tests must fail red until WorkspaceService is database-authoritative and DbForeignKeyContractTests prove every WorkspaceId entity has a Workspaces FK with non-cascade delete behavior.
  • TEST-MCP-139: Unit tests must fail red until persistent delete paths preserve rows through soft-delete metadata and every mutable entity writes DataAuditLog rows for create, update, and soft-delete operations.
  • TEST-MCP-140: Unit and provider tests must fail red until TODO requirement links and requirement traceability links enforce FKs, missing requirements are backfilled, and SQLite, SQL Server, and PostgreSQL migrations preserve data.
  • TEST-MCP-141: Add or update a documentation contract test proving docs/Development-Process-draft-v3.md captures the plan creation requirements for decision-complete frontier-model handoff plans, FR/TR/TEST traceability, TDD-first red/green behavior, and zero-failure zero-skip Byrd gates.
  • TEST-MCP-142: Bats coverage must prove workflow.requirements.updateFr, updateTr, and updateTest accept priority changes and do not fail inside the Codex plugin wrapper.
  • TEST-MCP-143: Validate that outstanding-session consolidation creates MCP-backed requirements and TODO traceability, inventories dirty workspaces, preserves unrelated changes, blocks unsafe deploys, and records zero-failure zero-skip validation gates before completion.
  • TEST-MCP-REPL-001: ✅ Complete - Given a REPL host process, when a well-formed YAML command envelope is sent to stdin, then a YAML response envelope is emitted to stdout with type: result and the expected result payload. Covered by: Iteration1_IntegrationTests, YamlFramingTests, YamlEnvelopeShapeTests
  • TEST-MCP-REPL-002: ✅ Complete - Given a REPL host process, when malformed YAML is sent to stdin, then a structured error response is emitted with type: error and descriptive error details, without crashing the host process. Covered by: FakeYamlSerializerTests, YamlFramingTests
  • TEST-MCP-REPL-003: ✅ Complete - Given a REPL host with no bootstrap invocation, when an operational command is sent, then the response contains type: error and appropriate error code. Covered by: ProtocolHandshakeTests, TrustBootstrapFlowTests
  • TEST-MCP-REPL-004: ✅ Complete - Given a REPL host, when trust bootstrap is invoked with a valid marker file, then the host verifies the marker signature, performs the health nonce challenge, caches the API key, and returns success. Covered by: TrustBootstrapFlowTests, MarkerFileTrustTests, MockTrustBootstrapServiceTests
  • TEST-MCP-REPL-005: ✅ Complete - Given a REPL host with completed bootstrap, when the API key in the marker file is rotated, then the host detects rotation via marker file watch and emits appropriate notifications. Covered by: AuthRotationTests, AuthKeyAndWorkspaceTests, StubAuthRotationHandlerTests
  • TEST-MCP-REPL-006: ✅ Complete - Given bootstrapped REPL commands for TODO operations (workflow.todo.*), when invoked with valid args, then results match the equivalent client operation semantics. Covered by: TodoWorkflowTests, Iteration3IntegrationTests, TodoWorkflowTestExtensions
  • TEST-MCP-REPL-007: ✅ Complete - Given bootstrapped REPL commands for session log operations (workflow.session.*), when invoked with valid args, then results match the equivalent client operation semantics. Covered by: SessionLogWorkflowTests, SessionLogWorkflowIntegration2Tests, SessionLogWorkflowProductionTests, Iteration2IntegrationTests
  • TEST-MCP-REPL-007-1: Given TryResolveWithDiagnostics with a workspace path containing no marker file, when called, then the error message enumerates every directory walked from the start path to its root. Covered by: MarkerFileClientOptionsResolverTests.TryResolveWithDiagnostics_WhenMarkerMissing_EnumeratesSearchedPaths
  • TEST-MCP-REPL-007-2: Given an explicit workspacePathOverride pointing to a workspace with a valid marker, when TryResolveWithDiagnostics is called, then resolution succeeds and the returned options carry the marker's API key. Covered by: MarkerFileClientOptionsResolverTests.TryResolveWithDiagnostics_AcceptsExplicitWorkspaceArgument
  • TEST-MCP-REPL-007-3: Given a marker file whose canonicalization is tampered, when TryResolveWithDiagnostics is called, then the error names the marker path and identifies "signature" failure. Covered by: MarkerFileClientOptionsResolverTests.TryResolveWithDiagnostics_WhenSignatureFails_ReportsReason
  • TEST-MCP-REPL-007-4: Given a marker whose HMAC payload is signed with LF-only (\n) line endings (matching the production server's MarkerFileService.AppendPayloadLine), when TryResolveWithDiagnostics is called on Windows or any platform where Environment.NewLine differs from \n, then signature verification succeeds. Covered by: MarkerFileClientOptionsResolverTests.TryResolveWithDiagnostics_VerifiesSignatureBuiltWithLfLineEndings
  • TEST-MCP-REPL-008: ✅ Complete - Given bootstrapped REPL commands for context operations (client.context.*), when invoked with valid args, then results match the equivalent client operation semantics. Covered by: GenericClientPassthroughTests, Iteration5IntegrationTests
  • TEST-MCP-REPL-009: ✅ Complete - Given bootstrapped REPL commands for requirements management (workflow.requirements.*), when invoked with valid args, then results match the equivalent client operation semantics. Covered by: RequirementsWorkflowTests, Iteration4IntegrationTests
  • TEST-MCP-REPL-010: ✅ Complete - Given bootstrapped REPL commands for workspace selection, when invoked with valid workspace paths, then workspace context resolution matches expected behavior. Covered by: WorkspaceSelectionTests, AuthKeyAndWorkspaceTests
  • TEST-MCP-REPL-011: ✅ Complete - Given generic client passthrough operations, when invoked with valid method/args, then operations delegate to the correct client type and method without duplicating logic. Covered by: GenericClientPassthroughTests, Iteration5IntegrationTests
  • TEST-MCP-REPL-012: ✅ Complete - Given streaming TODO operations (streamStatus, streamPlan, streamImplement), when invoked, then events stream correctly with proper cancellation handling. Covered by: TodoWorkflowTests (streaming event tests)
  • TEST-MCP-REPL-013: ✅ Complete - Given a REPL host, when EOF is received on stdin, then the host terminates gracefully. Covered by: EndToEndFlowTests
  • TEST-MCP-REPL-014: ✅ Complete - Given a workflow operation that throws an exception, when the command is executed, then a structured error response is emitted and the command loop continues. Covered by: SessionLogWorkflowTests, TodoWorkflowTests (error handling paths)
  • TEST-MCP-REPL-015: ✅ Complete - Given request IDs in command envelopes, when commands are executed, then response envelopes echo the same request ID for request/response matching. Covered by: RequestResponseCorrelationTests, YamlEnvelopeShapeTests
  • TEST-MCP-REPL-016: ✅ Complete - Given workflow implementations, when registered in DI, then all dependencies are resolved from the container and no services are instantiated via new outside DI. Covered by: McpServerClientIntegrationTests, DI registration tests
  • TEST-MCP-REPL-017: ✅ Complete - Given workspace selection via workspace selector, when commands target specific workspaces, then workspace context is properly scoped. Covered by: WorkspaceSelectionTests, AuthKeyAndWorkspaceTests
  • TEST-MCP-REPL-018: ✅ Complete - Given orchestration rules for trust and auth, when workflows execute, then trust-before-auth and nonce-validation rules are enforced. Covered by: OrchestrationRulesTests, TrustBootstrapFlowTests
  • TEST-MCP-REPL-019: ✅ Complete - Given namespace-organized command shapes, when workflows execute, then operations delegate to typed client contracts without duplicating business logic. Covered by: TodoWorkflowTests, SessionLogWorkflowTests, RequirementsWorkflowTests, GenericClientPassthroughTests
  • TEST-MCP-REPL-020: ✅ Complete - Given concurrent REPL operations, when workflows maintain stateful context, then session state and TODO selection are properly isolated per workflow instance. Covered by: SessionLogWorkflowTests (state management), TodoWorkflowTests (selection state)
  • TEST-SUPPORT-010A-1: Given a SessionLogService constructed with a non-null WorkspaceContext, when SubmitAsync persists a session, then SessionLogEntity.WorkspaceId and every child entity's WorkspaceId equal the context's WorkspacePath. Covered by: SessionLogServiceTests.SubmitAsync_StampsWorkspaceIdOnSessionEntity, SessionLogServiceTests.SubmitAsync_StampsWorkspaceIdOnEveryChildEntity
  • TEST-SUPPORT-010A-2: Given a SessionLogService constructed with workspaceContext: null and a DbContext without _workspaceId, when SubmitAsync persists a session, then SessionLogEntity.WorkspaceId remains empty string. Covered by: SessionLogServiceTests.SubmitAsync_WithNullWorkspaceContext_KeepsWorkspaceIdEmpty
  • TEST-SUPPORT-010B-1: Given a malformed POST body that fails JSON deserialization against UnifiedSessionLogDto, when the controller returns 400, then the response content-type is application/problem+json and the errors object contains $.workspace (or the offending field path), never the dto parameter name. Covered by: SessionLogControllerTests.WhenPostingMalformedWorkspaceFieldThenReturnsProblemDetailsWithoutDtoKey
  • TEST-SUPPORT-010B-2: Given a POST body missing sourceType, when domain validation rejects it, then the response is application/problem+json with sourceType cited (not the legacy {"error":"..."} plain shape). Covered by: SessionLogControllerTests.WhenPostingMissingSourceTypeThenReturnsProblemDetails
  • TEST-SUPPORT-010C-1: Given a successful POST to /mcpserver/sessionlog, when GET /mcpserver/sessionlog/{agent}/{sessionId} is called under the same workspace context, then the response is 200 OK with the round-tripped session. Covered by: SessionLogControllerTests.WhenPostingThenGetBySessionIdReturnsRecord
  • TEST-SUPPORT-010C-2: Given a session exists, when POST /mcpserver/sessionlog/{agent}/{sessionId}/turn carries a UnifiedRequestEntryDto, then 201 is returned and the subsequent GET shows the appended turn. Covered by: SessionLogControllerTests.WhenPostingTurnViaRestThenTurnIsRetrievable, SessionLogServiceTests.UpsertTurnAsync_NewTurn_AppendsWithoutDeletingSiblings
  • TEST-SUPPORT-010C-3: Given the turn-append route, when PUT is used instead of POST, then 405 is returned with Allow: POST. Covered by: SessionLogControllerTests.WhenPuttingTurnRouteThenReturns405WithAllowHeader

Generated from MCP requirements wiki export.

Clone this wiki locally