The shared workflows reference actions as though they are external rather than in the same repository

[briantist]

For example ansible-community/github-docs-build/actions/ansible-docs-build-init@main

This works but is problematic for several reasons:

• the git ref is hardcoded, so no matter which version (ref) of the workflow you are using, the action will always be loaded from main
• this also makes modifications via PR even more challenging than they already are because of the pull_request_target event, because during development and testing the refs all need to be changed to the PR branch, and then changed once again before merging
• a fork would always be referencing the source repo's version rather than itself, unless permanently changed

Actions can be referenced from the local filesystem, for example the above would change to ./actions/ansible-docs-build-init. The possible challenge to this, is that typically, you must do a checkout on the runner first to ensure the files actually exist. In most repositories, this is pretty straightforward because you're already doing a checkout anyway.

With these shared workflows, a checkout is running in context of the calling repo, not the workflow's repo. I am not sure if there's a way that we can infer from within the workflow, which commit of the shared workflow itself has been referenced. If we can do that, we can also do a checkout of the repo and get the actions that way.

What would be really ideal, is to re-use the checkout that already occurred to get the workflow in the first place. Much like actions, shared workflows are automatically retrieved with a git clone/checkout by the runner system, so the files are somewhere. Composite actions have a variable they can access that gives the filesystem location of the action (so that files in the action can be referenced). If a similar variable exists for shared workflows that would be the ideal way to handle this. I have not seen any documentation that suggests that variable exists though.

In the reusing workflows page there is an example that references a local action:

jobs:
  reusable_workflow_job:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: ./.github/actions/my-action@v1
        with:
          username: ${{ inputs.username }}
          token: ${{ secrets.envPAT }}

The interesting thing about this example is that is clearly does not have a checkout step, implying that local action references will "just work" but I have not tried this yet. The example is not meant to demonstrate local action referencing though, so it's entirely possible that this is a non-working example.

[briantist]

So I've done some research on this and it's.. bleak.

The checkout for the shared workflow does not happen on the runner like I thought (like it does for actions), it seems retrieving the shared workflow happens internal to the orchestration around workflows, before a runner is ever acquired, which does make sense. But it means the workflow(s) being executed are not on the runner.

In addition to that, I've dumped everything I can in such a job and as far a shared workflow "knows" from its runtime environment, its event was the event that triggered the calling workflow, which also kind of makes sense but it means there's no info at all that we could use to figure out which commit's version of the shared workflow was invoked.

As far as I can tell, this leaves us in the unenviable position of hardcoding the commit hashes of the local actions (or hardcoding a commit for a checkout and referencing the actions locally, same difference), which presents some of the same issues as #10 , but with additional complications as it relates to making changes in this repo.

[felixfontein]

Maybe we need a 'build' script which creates copies of the files with main replaced by a tag name, then commits it, and tags that commit with that tag name. That way we can at least create tags that are internally consistent. A subset of that script (without the commit and tag step) can also be used for development, by replacing main and even the repo name with another repo and branch name.

[briantist]

I've come up with a possible implementation strategy of @felixfontein 's idea above, whereby we come up with a script that can replace the references, hereafter referred to as the "script" and then leveraging github workflows to perform the changes. In my mind this looks like so:

The script

• thinking python + ruyaml for round-trip and comment-preservation support
• makes safe updates of references
• can accurately determine and report whether an update was needed or actually made (so that we can avoid unnecessary commits ands pushes)
• takes parameters for target reference at a minimum, perhaps also for search criteria around which with:s to update

Workflow: on: push (all branches, no tags)

• runs the script changing all references to refer to the current branch
  • if a change was made by the script (usually would happen on the first push to the branch), then it will be committed and pushed up to the branch as well; this won't trigger the same workflow again, but if it did it wouldn't matter because:
  • if no change was made, no commit and push will happen
• this ensures that using the workflows from this branch also uses the actions from the same branch
• targeting all branches ensures:
  • any branch in the repo gets this treatment whether it's part of a PR or not (I think this is desirable, perhaps we provide an out via commit message flags or something)
  • the workflow will run on forks (desirable because many/most PRs will come from forks and we want this working there too)
  • regardless of a PR being merged with references to itself, the merge will trigger this push workflow on main, which will set those references back to main, in main, solving a tricky situation in PR branches about when/how to set references back off of the PR branch

How to Release

• What we want: a tagged release where the shared workflows have action references pointing to a specific commit
• Why we want tags (as opposed to branches for release versions): because GitHub releases are bound to tags

There are two similar ways I can think of to achieve this:

Workflow: on: push (tags only [all or probably a pattern matching versions], no branches)

This method uses pushing a tag as a trigger for releasing, even though we're going to change the reference the tag points to.

• Create a temporary branch off the commit in the tag
• Run the script to update the references to point to that commit
• Commit this modification to the branch
• Re-target the current tag to this new commit
• Push the the temporary branch and the new tag
• Delete the branch and push its deletion
• (also handles creating the formal "release" on GitHub)

Workflow: on: workflow_dispatch

(with input asking for the version/tag, and a reference the tag points to, defaulting to main)
This method not directly manipulate pushed tags, it means we trigger a release via running the workflow manually (technically can be triggered via API too)

• This method is exactly the same as the above, with the difference that the tag we're going to create doesn't exist beforehand. All the steps are the same.

The reason for the temporary branch and two pushes:

• We are essentially creating an orphaned commit (almost); it's not technically orphaned since a tag references it.
• But I don't believe we have any way to push the commit to GitHub in that state, without it being in a branch.
• So pushing the branch gets the commit upstream. Then pushing the delete removes the branch we don't need.
• The existence of the tag referencing the commit should prevent it from being deleted/GC'd.

---

Release-wise: I think I prefer the second option (manual workflow).

[briantist]

a couple of afterthoughts on the above, regarding running the reference updater on all branches:

• in PRs, it will require the author to git pull after first pushing, to get the new commit; I think that's ok, it'll be easy to forget but will be obvious as soon as the next push is attempted (it will fail until you pull)
• in forks, pushes to main will set its local references to main on the fork repo; I think this is ok too because:
  • it allows references directly to the fork's shared workflows in its main branch to also reference the actions from that same repo
  • a git pull upstream main should still work cleanly if it's done with --rebase, since it will just reapply the commit (but it will I guess need to be more like git pull --rebase && git pull upstream main --rebase)
  • pushing the result of above will re-run the workflow, but it will not have any changes to make
  • even if you decide to undo fork's modification, and take what's upstream only, and then (force?) push that to the fork, the workflow will run again, and reset the references back to the fork anyway

[felixfontein]

But I don't believe we have any way to push the commit to GitHub in that state, without it being in a branch.

If there's a tag pointing to it, it's not orphaned :) If you push a tag to a commit that's not contained in any branch of the repo, you end up with that tagged commit in the repository. So it should work fine without creating a branch.

Besides that, you plan sounds good! (About rebasing while pulling: I've pull.rebase=true in my global git config, and this is another reason why that's a good thing :) )

[briantist]

But I don't believe we have any way to push the commit to GitHub in that state, without it being in a branch.

If there's a tag pointing to it, it's not orphaned :) If you push a tag to a commit that's not contained in any branch of the repo, you end up with that tagged commit in the repository. So it should work fine without creating a branch.

Well that sounds good, I can't tell if you know for certain it works or just strongly suspect.. but I do hope so, it would simplify it a bit.

Besides that, you plan sounds good!

🙌😀

(About rebasing while pulling: I've pull.rebase=true in my global git config, and this is another reason why that's a good thing :) )

I do the same! I added it explicitly in the example for those who don't, or those who do and forget not everyone does ;)

[felixfontein]

I think I did that at some point in the past. I don't remember that clearly anymore, it has been a longer time ago (more than a year).

[nrgbistro]

Any updates on this? I'm trying to do something very similar. I have a repo for creating reusable workflows and actions; in my workflow, I reference my composite actions with OWNER/REPO/ACTION@v1. However, v1 is my production tag so I can't make a reference to that when testing in my PR. Ideally, I could leave the v1 tag and still allow caller workflows that test my reusable workflow to use this PR's commit hash as the tag. Do you have the script you described available on GitHub?