`refs/pull/%s/head` might not always be there, but if it is missing, buildkite fails the checkout

[romanoid-openai]

Based on our recent observations, github creates refs/pull/%s/head asynchronously and sometimes these creations queue up enough that the ref will not be there by the time buildkite receives the webhook and proceeds.

Unfortunately if the provider is github and we're building the PR, buildkite agent completely fails the job if it is unable to fetch the refs/pull/%s/head even the refspec is not even guaranteed to match the commit we're building (I can update between the time original webhook is issued and when the job runs, for example when retrying old job, or even when developer pushes new commit and the build is not configured to cancel)

The main fetch:

agent/internal/job/checkout.go

Line 578 in e55e764

refspec := fmt.Sprintf("refs/pull/%s/head", e.PullRequest)

Mirror update logic:

agent/internal/job/checkout.go

Line 386 in e55e764

cmd := e.shell.Command("git", "--git-dir", mirrorDir, "fetch", "origin", refspec)

Proposed solution: allow this fetch to fail and proceed as long as we still can fetch the specific refspec here:

agent/internal/job/checkout.go

Line 580 in e55e764

if err := gitFetch(ctx, e.shell, gitFetchFlags, "origin", refspec); err != nil {

Note that mirror logic does not mirror the same dual-fetch, but it also can fail the build if the refs/pull/%s/head is not there yet, so it needs to be expecting this corner case too.

[top-oai]

Heads up there was another flare up from GitHub that triggered this issue again, not sure if anyone's had time to look at this, but it seems like it would make things more robust all around if this was in place.

[123sarahj123]

Hello @top-oai and @romanoid-openai :)

A fix has gone in, and will be going out on the next agent release
#3294