Add MCP project profile advisor#356
Open
ozymandiashh wants to merge 1 commit into
Open
Conversation
Contributor
Author
|
@iamtoruk quick meta question so we do not create noise: are PRs like #356 / #357 in the direction you want for CodeBurn contributions, or would you prefer issues/design notes before feature PRs? Happy to keep changes small, evidence-backed, and wait for feedback before opening more exploratory optimize/features work. |
Member
|
Hey, appreciate you asking. For new features like #356 and #357, please open an issue first so we can discuss the idea before any code. We try to keep CodeBurn focused and add features based on user demand rather than exploring every possible direction. That way nobody wastes time building something we end up not shipping. Bug fixes and small improvements against existing functionality are always welcome as direct PRs. |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
codeburn optimize.Why
Existing MCP optimize findings answer two narrower questions:
This adds a third question: is the MCP useful, but scoped too broadly?
For example, a GitHub MCP server may be useful in the main code repo but unnecessary in docs/playground repos. Removing it globally would hurt the hot workflow; project-scoping it preserves the useful setup while reducing cold-project schema overhead.
What changed
detectMcpProfileAdvisor()tosrc/optimize.ts.session.mcpInventorysession.mcpBreakdownValidation
I validated the behavior by running the real
detectMcpProfileAdvisor()export against controlledProjectSummaryfixtures vianpx tsx --eval. This constructs the actual edge cases the feature is meant to handle and prints detector output.{ "one_hot_project_profile": { "title": "1 MCP server should be project-scoped", "tokensSaved": 4000, "expectedTokensSaved": 4000, "containsHotProject": true, "containsColdProjects": true, "proof": "two cold projects each loaded 4 github MCP tools: 4*400 schema tokens * 1.25 cache-write pricing * 2 cold sessions = 4,000 effective tokens" }, "two_hot_project_profile": { "title": "1 MCP server should be project-scoped", "containsBothHotProjects": true, "containsColdProjects": true, "proof": "top two projects can form the hot profile while cold projects are still recommended for project-scoping" }, "shared_cold_session_cap": { "title": "2 MCP servers should be project-scoped", "tokensSaved": 5000, "expectedTokensSaved": 5000, "proof": "github + slack share the same two cold calls; combined schema budget is capped by each 2,000-token cache bucket before 1.25 cache-write pricing, not summed independently" }, "healthy_even_use_guard": { "finding": null, "proof": "server used evenly across all loaded projects emits no profile finding" }, "low_tool_coverage_suppression": { "finding": null, "proof": "server already eligible for low MCP tool coverage emits no duplicate profile finding" } }What this proves:
4,000effective-token estimate.5,000tokens, not the inflated independent sum.Supporting checks:
./node_modules/.bin/tsc --noEmit --pretty falsenpx vitest run tests/mcp-coverage.test.ts— 29 tests passednpm run buildnpm test -- --run— 62 files / 877 tests passedgit diff --check origin/main...HEADPASSPASSon the amended diffNotes
origin/mainand uses existing MCP inventory/breakdown data.