[PoC] Code testing agent + tests PoC#433
Conversation
|
/evaluate |
Skill Validation Results
[1] (Isolated) Quality unchanged but weighted score is -6.5% due to: tokens (24237 → 49377), tool calls (2 → 3)
Model: claude-opus-4.6 | Judge: claude-opus-4.6 |
|
/evaluate |
Skill Validation Results
Model: claude-opus-4.6 | Judge: claude-opus-4.6 |
There was a problem hiding this comment.
Pull request overview
Adds a proof-of-concept “code testing agent” skill (with supporting sub-agents and prompts) plus a non-trivial ASP.NET Core MVC sample app (ContosoUniversity) and an evaluation scenario to drive test generation against it.
Changes:
- Introduces
code-testing-agentskill docs/prompts and a suite of RPI (Research → Plan → Implement) sub-agent definitions for test generation. - Adds a ContosoUniversity ASP.NET Core MVC + EF Core sample app used as the real-world target for generated tests.
- Adds a dotnet-test evaluation scenario (
eval.yaml) to validate generated tests compile/pass and produce coverage output.
Reviewed changes
Copilot reviewed 42 out of 42 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/dotnet-test/code-testing-agent/eval.yaml | Evaluation scenario that prompts the agent to generate tests and runs dotnet test with coverage collection. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/appsettings.json | Adds app settings + default SQL Server connection string for the sample app. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Services/NotificationService.cs | In-memory notification queue service used by controllers. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Program.cs | Minimal hosting startup configuring EF Core, MVC, session, and static file serving. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/PaginatedList.cs | Pagination helper used by list views/controllers. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/Student.cs | Student entity model. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/SchoolViewModels/InstructorIndexData.cs | ViewModel for instructor index (instructors/courses/enrollments). |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/SchoolViewModels/EnrollmentDateGroup.cs | ViewModel used by Home/About grouping. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/SchoolViewModels/AssignedCourseData.cs | ViewModel for course assignment UI. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/Person.cs | Base class for TPH inheritance (Student/Instructor). |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/OfficeAssignment.cs | OfficeAssignment entity model. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/Notification.cs | Notification entity + EntityOperation enum. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/Instructor.cs | Instructor entity model. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/ErrorViewModel.cs | Error view model. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/Enrollment.cs | Enrollment entity model + Grade enum. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/Department.cs | Department entity model (includes concurrency token). |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/CourseAssignment.cs | Join entity for instructor/course assignments. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Models/Course.cs | Course entity model (includes uploaded-material path). |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Data/SchoolContextFactory.cs | Context factory wiring SQL Server options from configuration. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Data/SchoolContext.cs | EF Core DbContext with TPH config and relationships. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Data/DbInitializer.cs | Seeds initial data on first run via EnsureCreated(). |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Controllers/StudentsController.cs | CRUD controller for students + notification hooks. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Controllers/NotificationsController.cs | Basic endpoints/UI for retrieving/marking notifications. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Controllers/InstructorsController.cs | CRUD controller for instructors + course assignment management. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Controllers/HomeController.cs | Home/About/Contact/Error actions for the sample app. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Controllers/DepartmentsController.cs | CRUD controller for departments + concurrency handling + notifications. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Controllers/CoursesController.cs | CRUD controller for courses + teaching material upload/delete logic. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/Controllers/BaseController.cs | Shared base controller providing DbContext + notification helper. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/ContosoUniversity.sln | Solution file for the sample app. |
| tests/dotnet-test/code-testing-agent/ContosoUniversity/ContosoUniversity.csproj | SDK-style web project targeting net10.0 with EF Core + UI packages. |
| plugins/dotnet-test/skills/code-testing-agent/unit-test-generation.prompt.md | Cross-language unit test generation prompt used as default guidance. |
| plugins/dotnet-test/skills/code-testing-agent/extensions/dotnet.md | .NET-specific guidance for builds/tests, references, and error handling. |
| plugins/dotnet-test/skills/code-testing-agent/SKILL.md | Skill doc describing the multi-agent RPI pipeline and usage guidance. |
| plugins/dotnet-test/plugin.json | Registers the new agent definitions under the dotnet-test plugin. |
| plugins/dotnet-test/agents/code-testing-generator.agent.md | Orchestrator agent for Research/Plan/Implement + final validation steps. |
| plugins/dotnet-test/agents/code-testing-researcher.agent.md | Sub-agent for repository research and convention discovery. |
| plugins/dotnet-test/agents/code-testing-planner.agent.md | Sub-agent for producing phased test implementation plans. |
| plugins/dotnet-test/agents/code-testing-implementer.agent.md | Sub-agent for implementing plan phases and verifying build/test. |
| plugins/dotnet-test/agents/code-testing-builder.agent.md | Sub-agent for compiling projects and reporting errors. |
| plugins/dotnet-test/agents/code-testing-tester.agent.md | Sub-agent for running tests and summarizing pass/fail output. |
| plugins/dotnet-test/agents/code-testing-fixer.agent.md | Sub-agent for addressing compilation errors based on build output. |
| plugins/dotnet-test/agents/code-testing-linter.agent.md | Sub-agent for formatting/linting fixes (polyglot). |
Comments suppressed due to low confidence (2)
tests/dotnet-test/code-testing-agent/ContosoUniversity/Controllers/CoursesController.cs:176
- Returning
ex.Messageto the client can leak internal details (paths, stack hints). Since you already log the exception, prefer a generic message for the user instead of concatenating the exception text.
_logger.LogError(ex, "Error uploading file");
ModelState.AddModelError("teachingMaterialImage", "Error uploading file: " + ex.Message);
ViewBag.DepartmentID = new SelectList(db.Departments, "DepartmentID", "Name", course.DepartmentID);
tests/dotnet-test/code-testing-agent/ContosoUniversity/Controllers/CoursesController.cs:201
- Same issue here:
.Single()will throw if no course matches the id, so thecourse == nullcheck won't be hit. UseSingleOrDefault()/FirstOrDefault()and return NotFound when null.
Course course = db.Courses.Include(c => c.Department).Where(c => c.CourseID == id).Single();
if (course == null)
{
return NotFound();
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
No description provided.