Skip to content

Support raw view output in replay fixtures - #2102

Closed
erik-krogh wants to merge 2 commits into
github:mainfrom
erik-krogh:erik-krogh/raw-view-output-replay
Closed

Support raw view output in replay fixtures#2102
erik-krogh wants to merge 2 commits into
github:mainfrom
erik-krogh:erik-krogh/raw-view-output-replay

Conversation

@erik-krogh

Copy link
Copy Markdown

Summary

Companion to github/copilot-agent-runtime#13802 and github/copilot-agent-runtime#10996.

  • canonicalize shared view tool snapshots to raw file content
  • preserve compatibility with currently published CLIs by trying exact replay matches first, then a narrowly scoped legacy sequential-line-prefix fallback for view results
  • cover naturally numbered raw files, ranges, CRLF, non-view tools, non-sequential output, and cross-snapshot-type match priority

Testing

  • cd test/harness && npm test -- replayingCapiProxy.test.ts (34 passed)
  • cd test/harness && npx tsc --noEmit
  • git diff --check

The fallback can be removed once the minimum supported CLI version always emits raw view output.

Canonicalize view tool snapshots to raw file content and add an exact-first replay fallback for CLI versions that still prefix sequential line numbers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c5c3fb3-1b45-495a-84d6-90521feaff29
Copilot AI review requested due to automatic review settings July 28, 2026 00:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates replay fixtures for raw view output while retaining exact-first compatibility with legacy line-numbered results.

Changes:

  • Converts shared view snapshots to raw content.
  • Adds narrowly scoped legacy replay matching.
  • Adds compatibility and priority tests.
Show a summary per file
File Description
test/harness/replayingCapiProxy.ts Adds exact-first legacy view fallback.
test/harness/replayingCapiProxy.test.ts Tests fallback scope and priority.
test/snapshots/tools/invokes_built_in_tools.yaml Stores raw view output.
test/snapshots/system_message_transform/should_work_with_static_overrides_and_transforms_together.yaml Stores raw view output.
test/snapshots/system_message_transform/should_invoke_transform_callbacks_with_section_content.yaml Stores raw view output.
test/snapshots/system_message_transform/should_apply_transform_modifications_to_section_content.yaml Stores raw view output.
test/snapshots/subagent_hooks/should_invoke_pretooluse_and_posttooluse_hooks_for_sub_agent_tool_calls.yaml Stores raw view output.
test/snapshots/session/should_send_with_file_attachment.yaml Stores raw view output.
test/snapshots/session_config/should_use_workingdirectory_for_tool_execution.yaml Stores raw view output.
test/snapshots/session_config/should_apply_workingdirectory_on_session_resume.yaml Stores raw view output.
test/snapshots/session_config/should_accept_message_attachments.yaml Stores raw view output.
test/snapshots/permissions/should_invoke_permission_handler_for_write_operations.yaml Stores raw view outputs.
test/snapshots/permissions/permission_handler_for_write_operations.yaml Stores raw view outputs.
test/snapshots/multi_turn/should_use_tool_results_from_previous_turns.yaml Stores raw multi-turn outputs.
test/snapshots/multi_turn/should_handle_file_creation_then_reading_across_turns.yaml Stores raw view output.
test/snapshots/hooks/should_invoke_pretooluse_hook_when_model_runs_a_tool.yaml Stores raw view output.
test/snapshots/hooks/should_invoke_posttooluse_hook_after_model_runs_a_tool.yaml Stores raw view output.
test/snapshots/hooks/should_invoke_both_pretooluse_and_posttooluse_hooks_for_single_tool_call.yaml Stores raw view output.
test/snapshots/hooks/should_invoke_both_pretooluse_and_posttooluse_hooks_for_a_single_tool_call.yaml Stores raw view output.
test/snapshots/hooks/invoke_pre_tool_use_hook_when_model_runs_a_tool.yaml Stores raw view output.
test/snapshots/hooks/invoke_post_tool_use_hook_after_model_runs_a_tool.yaml Stores raw view output.
test/snapshots/hooks/invoke_both_hooks_for_single_tool_call.yaml Stores raw view output.
test/snapshots/event_fidelity/should_preserve_message_order_in_getmessages_after_tool_use.yaml Stores raw view output.
test/snapshots/event_fidelity/should_emit_tool_execution_events_with_correct_fields.yaml Stores raw view output.
test/snapshots/event_fidelity/should_emit_events_in_correct_order_for_tool_using_conversation.yaml Stores raw view output.
test/snapshots/client_options/should_use_client_cwd_for_default_workingdirectory.yaml Stores raw view output.
test/snapshots/builtin_tools/should_read_file_with_line_range.yaml Stores raw ranged output.
test/snapshots/builtin_tools/should_edit_a_file_successfully.yaml Stores raw edited-file output.
test/snapshots/builtin_tools/should_create_a_new_file.yaml Stores raw created-file output.

Review details

  • Files reviewed: 29/29 changed files
  • Comments generated: 1
  • Review effort level: Medium

Comment thread test/harness/replayingCapiProxy.ts Outdated
Comment on lines +1580 to +1582
return (
JSON.stringify({ ...requestMessage, content: rawCandidate }) === savedJson
);

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cca1c28: the fallback now deletes the candidate content field when legacy numbering strips to empty, matching canonical normalized messages. Added an end-to-end empty-file replay test; harness tests and typecheck pass.

@erik-krogh
erik-krogh marked this pull request as ready for review July 28, 2026 00:50
@erik-krogh
erik-krogh requested a review from a team as a code owner July 28, 2026 00:50
Match a legacy numbered empty-file result to the canonical content-less tool message and cover the replay edge case.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c5c3fb3-1b45-495a-84d6-90521feaff29
Copilot AI review requested due to automatic review settings July 28, 2026 00:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 29/29 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@erik-krogh erik-krogh closed this Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants