Skip to content

feat: execute tests#489

Merged
a-frantz merged 52 commits intomainfrom
feat/tests-execute
Jan 12, 2026
Merged

feat: execute tests#489
a-frantz merged 52 commits intomainfrom
feat/tests-execute

Conversation

@a-frantz
Copy link
Copy Markdown
Member

@a-frantz a-frantz commented Dec 9, 2025

Describe the problem or feature in addition to a link to the issues.

Execute the test definitions defined by #468 🚂

relevant links:

Before submitting this PR, please make sure:

For external contributors:

  • You have read the contributing guide in its entirety.
  • You have not used AI on any parts of this pull request.

For all contributors:

  • You have added a few sentences describing the PR here.
  • You have added yourself or the appropriate individual as the assignee.
  • You have added at least one relevant code reviewer to the PR.
  • Your code builds clean without any errors or warnings.
  • You have added tests (when appropriate).
  • You have added an entry in the CHANGELOG (when appropriate).
  • You have updated the README or other documentation to account for these changes (when appropriate).
  • You have either (a) made a PR to the next branch in the sprocket.bio repository [if you are a Sprocket team member] or (b) have suggested any changes you think need to be made to sprocket.bio in the description of your PR [if you are an external contributor].

For PRs containing lint rule changes:

  • You have updated any and all effected entries within RULES.md.
  • You have added a test case in crates/wdl-lint/tests/lints that covers every
    possible diagnostic emitted for the rule within the file where the rule
    is implemented.

@github-actions github-actions Bot added the S-awaiting-pass-CI PR is awaiting CI to pass. label Dec 9, 2025
@a-frantz
Copy link
Copy Markdown
Member Author

copying a comment I made on Slack that explains some of the vision implemented in this PR:

doc sets a precedent for a "WDL workspace" concept that is being extended with test. A WDL workspace is not all that different from a git repo and it should be intuitive (I would expect just about every workspace to also be a git repo, although we aren't interacting with git at all). The basic assumption is that there is a "root" of the workspace which is just a local-on-disk directory with arbitrarily nested WDL documents within it.

Relative to this "root" of the workspace is expected to be a test/ directory. Under test/, there will be fixtures/. Test definitions are always loaded with <workspace>/test/fixtures/ loaded as the "origin". So to reference test/fixtures/test.bam in a test, the test definition would simply reference test.bam without any pathing. If users want to set up nesting or structure for their fixtures, that is also possible. Imagine a test/fixtures/bams/test.bam file exists, that would be referenced in test definitions as bams/test.bam.

Tests are also run inside the test directory, specifically at test/runs/<entrypoint>/<test name>/. The <entrypoint>/<test name> directory gets blown away on re-runs of sprocket test (I don't see a rationale for persisting test executions. That just sounds like disk bloat to me).

The "custom" test executables are at test/bin/. PATH=<workspace>/test/bin/:$PATH will be added to the child process so that any custom executables (bash/python scripts for example) are easy to reference.

This PR does not reference or interact with the test/bin/ directory. That's coming in a future PR.

@github-actions github-actions Bot removed the S-awaiting-pass-CI PR is awaiting CI to pass. label Dec 14, 2025
@a-frantz a-frantz marked this pull request as ready for review December 15, 2025 11:56
@a-frantz a-frantz requested a review from a team as a code owner December 15, 2025 11:56
@a-frantz a-frantz requested a review from peterhuene December 15, 2025 11:56
@a-frantz a-frantz changed the title [WIP]feat: execute tests feat: execute tests Dec 15, 2025
this allows users to use the same `run` config without making them duplicate it for `test`
@github-actions github-actions Bot added the S-awaiting-pass-CI PR is awaiting CI to pass. label Dec 16, 2025
@github-actions github-actions Bot removed the S-awaiting-pass-CI PR is awaiting CI to pass. label Dec 29, 2025
@github-actions github-actions Bot added the S-awaiting-pass-CI PR is awaiting CI to pass. label Dec 29, 2025
@github-actions github-actions Bot removed the S-awaiting-pass-CI PR is awaiting CI to pass. label Dec 30, 2025
Copy link
Copy Markdown
Contributor

@peterhuene peterhuene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Just a few minor comments.

Comment thread src/commands/test.rs Outdated
Comment thread src/commands/test.rs Outdated
Comment thread src/commands/test.rs Outdated
@a-frantz a-frantz requested a review from peterhuene January 9, 2026 14:09
@github-actions github-actions Bot added the S-awaiting-pass-CI PR is awaiting CI to pass. label Jan 9, 2026
@github-actions github-actions Bot removed the S-awaiting-pass-CI PR is awaiting CI to pass. label Jan 12, 2026
@github-actions github-actions Bot added the S-awaiting-pass-CI PR is awaiting CI to pass. label Jan 12, 2026
Copy link
Copy Markdown
Contributor

@peterhuene peterhuene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few minor comments.

Comment thread src/commands/test.rs
Comment thread src/commands/test.rs Outdated
Comment thread src/commands/test.rs Outdated
Comment thread src/commands/test.rs
@a-frantz a-frantz requested a review from peterhuene January 12, 2026 17:15
@a-frantz a-frantz merged commit c7b6111 into main Jan 12, 2026
20 checks passed
@a-frantz a-frantz deleted the feat/tests-execute branch January 12, 2026 17:20
@github-actions github-actions Bot removed the S-awaiting-pass-CI PR is awaiting CI to pass. label Jan 12, 2026
bfrazho pushed a commit to bfrazho/sprocket that referenced this pull request Mar 22, 2026
Mahathi-s154 pushed a commit to Mahathi-s154/sprocket that referenced this pull request Mar 26, 2026
Co-authored-by: Peter Huene <peter@huene.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants