Repository navigation
rebase: improve CI baseline reconciliation - #128
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 85af377f70
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| cur_errors = extract_exception_lines(log_text, spec.extra_exception_names) | ||
| main_errors = extract_exception_lines(main_log, | ||
| spec.extra_exception_names) | ||
| if cur_errors & main_errors: |
There was a problem hiding this comment.
When a multi-test job contains a shared exception line elsewhere in both logs, this global set intersection succeeds even if the current and baseline causes for the matched pytest node differ. For example, a common auxiliary ValueError plus RuntimeError: new regression for the current node is classified as ignored_baseline when the baseline node failed with an AssertionError, allowing a real regression to contribute to clean_pass; correlate the exception evidence with the corresponding node or failure summary instead.
Useful? React with 👍 / 👎.
| for entries in failures.values(): | ||
| if entries[0][0] == 0 or len(entries) >= 2: | ||
| out.extend(failure for _, failure in entries) |
There was a problem hiding this comment.
When one older build contains multiple failed attempts of the same retried job and the newest build is green, list_jobs() includes both the original retried=True job and its failed retry, so len(entries) >= 2 treats a failure seen in only one build as recurring. That resurrects stale baseline evidence and can cause a later matching regression to be ignored; filter retried predecessors or require at least two distinct build indices.
Useful? React with 👍 / 👎.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
85af377 to
4877a30
Compare
Summary
Tests
PYTHONPATH=$PWD/src /data/zhoutaichang/copilot/InferMatrixCopilot/.venv/bin/python -m pytest -qgit diff --check