Skip to content

Diff viewer for test outputs — compare results between executions #17

Description

@ThomasMalletCodra

Description

Add a diff viewer that compares the output of a test between two executions (e.g., current vs. previous, or between two selected campaigns). This helps developers quickly identify what changed when a previously passing test starts failing — especially useful for regression debugging.

Current behavior

  • Only the latest test output is displayed.
  • No way to compare outputs between runs.
  • Debugging regressions requires manually saving and diffing outputs externally.

Expected behavior

  • A "Compare" action is available for any test that has been run at least twice.
  • Side-by-side or inline diff view highlighting added/removed/changed lines.
  • Color-coded diff: green for additions, red for deletions, yellow for changes.
  • Option to compare against:
    • Previous execution of the same test
    • A specific campaign (if campaign management is available)
    • A saved "baseline" output
  • Diff summary: "42 lines changed, 3 added, 5 removed".

Implementation ideas

  1. Store the last N outputs per test (configurable, default: 5) in .moduletester/history/.
  2. Use Python's difflib for generating unified or side-by-side diffs.
  3. Render diff in a QTextEdit or QWebEngineView with syntax highlighting.
  4. Add "Compare with previous" context menu action on test items.
  5. Optional: integrate with campaign management (Campaign management/Non regression — execution history and release comparison #9 ) for cross-campaign comparison.

Dependencies

Acceptance criteria

  • Last N test outputs are stored per test
  • Side-by-side or inline diff view is available
  • Diff highlights additions, deletions, and changes
  • "Compare with previous" action accessible from test context menu
  • Diff summary shows number of changed lines
  • Output history storage is configurable and can be cleared

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions