System tests: Archive fieldcompare diff files on failure#740
Open
PranjalManhgaye wants to merge 3 commits into
Open
System tests: Archive fieldcompare diff files on failure#740PranjalManhgaye wants to merge 3 commits into
PranjalManhgaye wants to merge 3 commits into
Conversation
MakisH
added
GSoC
PranjalManhgaye
changed the title
PranjalManhgaye
changed the title
PranjalManhgaye
force-pushed
the
issue-441-archive-fieldcompare-diffs
branch
from
44fb3ea to
a904f4f
Compare
Collaborator
|
This pull request has been mentioned on preCICE Forum on Discourse. There might be relevant details there: https://precice.discourse.group/t/gsoc-2026-pranjal-manhgaye/2769/6 |
2 tasks
MakisH
changed the title
MakisH
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds automatic archiving of fieldcompare-generated VTK diff files into a dedicated folder to make CI artifact debugging easier when comparisons fail.
Changes:
- Add logic to collect and copy
*diff*.(vtu|vtk|vtp)files intodiff-results/when fieldcompare fails. - Invoke the archiving step during systemtest failure handling.
- Document the new
diff-results/behavior in the systemtests README and add a changelog entry.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| tools/tests/systemtests/Systemtest.py | Copies fieldcompare diff VTK files into diff-results/ on fieldcompare failure. |
| tools/tests/README.md | Documents where to find diff VTK files in CI artifacts for debugging. |
| changelog-entries/441.md | Notes the user-facing behavior change for archiving diff files. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
6
to
11
| from pathlib import Path | ||
| from paths import PRECICE_REL_OUTPUT_DIR, PRECICE_TOOLS_DIR, PRECICE_REL_REFERENCE_DIR, PRECICE_TESTS_DIR, PRECICE_TUTORIAL_DIR | ||
|
|
||
| DIFF_RESULTS_DIR = "diff-results" | ||
|
|
||
| from metadata_parser.metdata import Tutorial, CaseCombination, Case, ReferenceResult |
Comment on lines
+467
to
+471
| dest_dir = self.system_test_dir / DIFF_RESULTS_DIR | ||
| dest_dir.mkdir(exist_ok=True) | ||
| for f in diff_files: | ||
| if f.is_file(): | ||
| shutil.copy2(f, dest_dir / f.name) |
| for f in diff_files: | ||
| if f.is_file(): | ||
| shutil.copy2(f, dest_dir / f.name) | ||
| logging.debug(f"Archived {len(diff_files)} fieldcompare diff file(s) to {dest_dir} for {self}") |
| The easiest way to debug a systemtest run is first to have a look at the output written into the action on GitHub. | ||
| If this does not provide enough hints, the next step is to download the generated `system_tests_run_<run_id>_<run_attempt>` artifact. Note that by default this will only be generated if the systemtests fail. | ||
| Inside the archive, a test-specific subfolder like `flow-over-heated-plate_fluid-openfoam-solid-fenics_2023-11-19-211723` contains two log files: a `stderr.log` and `stdout.log`. This can be a starting point for a further investigation. | ||
| Inside the archive, a test-specific subfolder like `flow-over-heated-plate_fluid-openfoam-solid-fenics_2023-11-19-211723` contains two log files: a `stderr.log` and `stdout.log`. This can be a starting point for a further investigation. When fieldcompare runs with `--diff`, it writes VTK diff files; these are copied into a `diff-results/` subfolder in the same run directory so you can open them (e.g. in ParaView) to see where results differ from the reference. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements #441: archive fieldcompare diff VTK files so they are easy to find in CI artifacts when a comparison fails.
Changes
*diff*.vtu,*diff*.vtk, and*diff*.vtpfiles from precice-exports/ into adiff-results/folder in the same run directory.diff-results/for inspection (e.g. in ParaView) when debugging comparison failures.Checklist