Add skill for using dotnet format whitespace#358
Add skill for using dotnet format whitespace#358jjonescz wants to merge 5 commits intodotnet:mainfrom
dotnet format whitespace#358Conversation
|
/evaluate |
There was a problem hiding this comment.
Pull request overview
Adds a new .NET plugin skill document to guide users on using dotnet format whitespace for fast, whitespace-only formatting, positioned as a safer alternative to analyzer/style formatting when the goal is indentation/trailing whitespace cleanup.
Changes:
- Introduces a new
dotnet-formatskill focused ondotnet format whitespace. - Documents recommended
--folder-based workflow, key flags, validation checklist, and pitfalls.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a new dotnet plugin skill documenting how to use dotnet format whitespace for fast, safe whitespace-only formatting in C#/.NET codebases.
Changes:
- Introduces a new
dotnet-formatskill with guidance on when to usedotnet format whitespacevsstyle/analyzers - Documents a recommended workflow using
--folder,--include, and--verify-no-changes - Adds validation checklist and common pitfalls for consistent usage
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a new .NET plugin skill document describing how to use dotnet format whitespace for fast, whitespace-only formatting (primarily via --folder mode) to clean up indentation/trailing whitespace after edits or code generation.
Changes:
- Introduces the
dotnet-formatskill with usage guidance, workflow steps, and validation checklist fordotnet format whitespace.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
|
/evaluate |
Skill Validation Results❌ Skill validation errors
Model: claude-opus-4.6 | Judge: claude-opus-4.6 |
|
I'd be very surprised if this skill is better than no skill: probably worse since it's using up context with its tokens. This is basically just restating the docs which models are already trained on? |
|
You'd have to add an eval.yaml then we could see. |
Add two evaluation scenarios: - Format specific files after editing (tests --folder + --include usage) - Verify formatting without modifying files (tests --verify-no-changes) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
I got the feedback that models generally just use
Added, but it seems I cannot run |
|
/evaluate |
There was a problem hiding this comment.
Pull request overview
Adds a new dotnet-format skill to the dotnet plugin focused on using dotnet format whitespace in --folder mode, plus accompanying evaluation scenarios to validate correct agent guidance for whitespace-only formatting without a solution file.
Changes:
- Introduces
plugins/dotnet/skills/dotnet-format/SKILL.mddocumenting a workflow fordotnet format whitespace(including--folder,--include,--verify-no-changes). - Adds
tests/dotnet/dotnet-format/eval.yamlwith scenarios for formatting specific files and verifying formatting without modifying files.
Show a summary per file
| File | Description |
|---|---|
| tests/dotnet/dotnet-format/eval.yaml | Adds eval scenarios/assertions intended to enforce use of dotnet format whitespace with folder-mode targeting. |
| plugins/dotnet/skills/dotnet-format/SKILL.md | Adds the new skill documentation and recommended command patterns for whitespace-only formatting. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (2)
tests/dotnet/dotnet-format/eval.yaml:36
- Scenario 2’s rubric requires
--folderand--include, but the assertions don’t check for either (onlydotnet format whitespaceand--verify-no-changes). Since task completion is gated by assertions/constraints (not the rubric), this scenario can pass without actually using folder mode or targetingsrc/Services/PaymentService.cs. Add assertions for--folderplus the specific--include src/Services/PaymentService.cs.
assertions:
- type: "output_contains"
value: "dotnet format whitespace"
- type: "output_contains"
value: "--verify-no-changes"
rubric:
- "Uses `dotnet format whitespace` with `--verify-no-changes` to verify without modifying the file"
- "Uses --folder mode since there is no solution file available"
- "Uses --include to target the specific file rather than checking the entire directory"
plugins/dotnet/skills/dotnet-format/SKILL.md:35
- The skill doc doesn’t explicitly state prerequisites/inputs (e.g., that
dotnet/the .NET SDK must be available, and thatdotnet formatmust be installed/available in the environment). CONTRIBUTING’s skill checklist calls out documenting inputs/prereqs; adding a short “Prerequisites” or “Inputs” section would make the skill more self-contained and reduce failure modes.
## When to Use
- Fixing indentation or trailing whitespace in files you just created or edited
- Batch-formatting a set of changed files before committing
- Cleaning up generated code that has inconsistent whitespace
## When Not to Use
- You need code-style fixes (naming, `var` vs explicit type, etc.) — use `dotnet format style` instead
- You need analyzer-driven fixes (e.g., SA1200) — use `dotnet format analyzers` instead
## Workflow
### Step 1: Determine files to format
Identify the file paths that need formatting — typically files that were just created or edited.
### Step 2: Run `dotnet format whitespace`
Use `--folder` mode to format without needing a project/solution file:
- Files reviewed: 2/2 changed files
- Comments generated: 2
| assertions: | ||
| - type: "output_contains" | ||
| value: "dotnet format whitespace" | ||
| - type: "output_contains" | ||
| value: "--folder" | ||
| - type: "output_matches" | ||
| pattern: "--include" | ||
| rubric: |
There was a problem hiding this comment.
Scenario 1’s assertions only check that the output mentions --include, but they don’t ensure the agent targets the specific files listed in the prompt. As written, the test can pass even if the agent formats the whole folder or includes unrelated paths. Consider asserting that the output contains both src/Services/OrderService.cs and src/Models/Order.cs (or a regex that requires both --include occurrences).
This issue also appears on line 28 of the same file.
| --- | ||
| name: dotnet-format | ||
| description: > | ||
| Format C#/.NET source files using the `dotnet format whitespace` subcommand. | ||
| USE FOR: quickly fixing whitespace and indentation in one or more files after | ||
| code generation or editing, batch-formatting changed files before a commit. | ||
| DO NOT USE FOR: enforcing code-style analyzers (SA/IDE rules), fixing | ||
| non-whitespace style issues, or formatting entire large repositories at once. | ||
| --- |
There was a problem hiding this comment.
This skill adds a new plugins/dotnet/skills/dotnet-format entry, but the dotnet plugin README’s skill list doesn’t include dotnet-format (it currently lists only csharp-scripts, dotnet-pinvoke, and nuget-trusted-publishing). Please update plugins/dotnet/README.md to keep the published skill inventory accurate.
This issue also appears on line 15 of the same file.
Skill Validation Results
[1]
Model: claude-opus-4.6 | Judge: claude-opus-4.6 🔍 Full Results - additional metrics and failure investigation steps ▶ Sessions Visualisation -- interactive replay of all evaluation sessions |
|
@jjonescz in the eval above, in the first case baseline (no skill) gives the same result but 30% faster (didn't need to read in the skill). In the second case, the skill doesn't activate at all. edit: in the first case, with the skill added it actually didn't even complete apparently in the 120s -- not sure why it got graded anyway If you click on the link it will give
this is handy to have AI look at the actual logs and you can ask it eg why did it not activate? But I think you might need to devise more eval scenario(s) that show why the skill is needed. One way to start might be to try locally -- without the skill -- and give it a task in Copilot CLI with some model or other and see what fails that you want to work. Then put that in an eval. What I've learned working on skills for this repo is it's often surprisingly hard to beat the agent-without-skill results. You start at -100 points because it has to read and interpret the skill - and because it uses up context window too For the ability to run '/evaluate' @ViktorHofer do we have a group to add Jan to? |
|
Just maintainer permissions are needed, no special group. You can reach out to @timheuer to get the required perms. |
No description provided.