[WIP] Bug 2051180 - Track which pushes are actually backfilled#9662
Open
junngo wants to merge 1 commit into
Open
[WIP] Bug 2051180 - Track which pushes are actually backfilled#9662junngo wants to merge 1 commit into
junngo wants to merge 1 commit into
Conversation
junngo
commented
Jul 3, 2026
| request.save() | ||
|
|
||
| @staticmethod | ||
| def _parse_backfill_started(line: str) -> dict: |
Collaborator
Author
There was a problem hiding this comment.
This helper will be removed after the format is changed to JSON by https://phabricator.services.mozilla.com/D309317
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is a work in progress. Please don't merge it yet.
Bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=2051180
This PR adds tracking for which pushes were actually backfilled when Sherlock triggers a backfill.
How it works
Bktask ID is returned and is saved to a newBackfillRequesttable.BackfillTrackerclass picks up any PENDING requests and parses thelive_backing_logof eachBktask.Bktask takes a few minutes to complete.Backfill started:is logged before backfilling starts, contains the overall backfill planBACKFILL_DATA:is logged per push as it gets backfilled. We use thedecision_task_idhere to look up which push was actually backfilled in Treeherder.Backfill startedformat to JSON is currently under review: https://phabricator.services.mozilla.com/D309317New tables
BackfillRequest: one row per backfill trigger, stores theBktask ID and parsed metadata fromBackfill startedBackfilledPush: one row per actual backfilled push, linked to the TreeherderPushviadecision_task_idThis will serve as a foundation for a follow-up patch to improve Sherlock's outcome check, which will only check the actual backfilled pushes rather than the full push range.
If you have any suggestions or opinions, feel free to share anytime.
[1] example
https://firefoxci.taskcluster-artifacts.net/Rymag17MSy2fMS-egTfdoA/0/public/logs/live_backing.log
https://treeherder.mozilla.org/jobs?repo=autoland&searchStr=bk&fromchange=47b90175fca9b0eb0f34d554e2226857016359eb&selectedTaskRun=Rymag17MSy2fMS-egTfdoA.0