[DOCS]: Add portable regression receipt example to results-and-reporting#70
[DOCS]: Add portable regression receipt example to results-and-reporting#70sumit1kr wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds documentation describing a "Portable Regression Receipt" pattern for CI gating using JsonFileReportSink, including a sample JSON artifact illustrating run-level and scenario-level metadata.
Changes:
- Adds a new section to
docs/usage/results-and-reporting.mdexplaining how to usereport.metadataandresult.metadatafor stable regression artifacts. - Provides an example JSON receipt with sample fields.
|
@microsoft-github-policy-service agree |
|
Apologies for the slow response, and thanks for taking this on. I agree with the docs-only direction here. I also agree with keeping the broader runtime-evidence / production-receipt idea from #63 out of scope for this PR; this section works best as test-time CI evidence using RAMPART’s existing reporting surfaces. One small thing I noticed while re-checking the current sink behavior: Could you update the docs example to mirror the current If maintainers would rather support run-level metadata in the built-in JSON output, then the alternate path would be a tiny serialization + test change for Once that small alignment is made, I’m happy to take another pass on the diff. Thanks again for moving this forward. |
4ae7e65 to
d9f3146
Compare
|
@gitrishiom Updated — moved the receipt fields under @spencrr tagging again since this is ready for maintainer review/approval whenever you have bandwidth. |
|
Thanks, this addresses the concern. The example now matches the current I’ll leave final review/approval to @spencrr . Thanks again for the quick update. |
Closes #63
Adds a "Portable Regression Receipt" section to
docs/usage/results-and-reporting.mdshowing how to use existing
report.metadataandresult.metadatafields to producea stable, diffable CI artifact tied to
JsonFileReportSink.No new Python types introduced — docs only.