[Android v4 API] Migrate fetch_artifact to use new V4 API by IvanBM18 · Pull Request #5371 · google/clusterfuzz

IvanBM18 · 2026-07-21T07:26:26Z

Overview

Bug: b/422775458

Our previous implementation of the android builv v4 api was wrong, it failed always because the api no longer worked on discovery service url. In other PR we focused on creating the new api to interact with the v4 requirements. In this PR we focus on using this api class in the fetch_artifact.py module,

Changes

Made private all the methods that were only used internally in fetch_artifact.py
Refactors previous usage of the v4 api to now use the current version with minimal changes in the fetch_artifact.py script to avoid breaking anything

Tests performed

Created unit tests for feth_artifact.py
Tested this script using a custom script (test details in this PR description )

PR review

#5370 Add Android Build API V4 REST client and tests
👉 [Android v4 API] Migrate fetch_artifact to use new V4 API #5371 Migrate fetch_artifact to use new V4 API (This PR)
#5373 Creates Feature Flag for Android V4 API

Please only approve, I'll merge this PR's to avoid any possible merge conflict that can appear in the process :D

letitz · 2026-07-21T08:37:53Z

Same as #5370, I'll let @dylanjew handle this review.

dylanjew · 2026-07-21T15:26:40Z


 def get(bid, target, regex, output_directory, output_filename=None):
  """Return artifact for a given build id, target and file regex."""
-  client = get_client()


could we add one more flag that completely disables fetch_artifact? In the chance that we run into an issue with v4 and v3 is throttled? And then add any error handling we need to the flashing and symbolization path to gracefully handle the new flag being disabled

oooh i get it now, ill create a third PR for this

Done! #5373

…id-v4-api

dylanjew · 2026-07-21T19:20:12Z


 def get(bid, target, regex, output_directory, output_filename=None):
  """Return artifact for a given build id, target and file regex."""
-  client = get_client()


javanlacerda · 2026-07-22T12:28:13Z

+      attempt_id=attempt_id,
+      artifact_name=name)
+
+  if _use_v4():


I just saw that the use_v4 logic was implemented at #5335 by using the config of source of truth for the flag. Maybe we could move it to use the structure of https://github.com/google/clusterfuzz/blob/6ff486c47af95d6fa9d86238a65cc15adf5a9393/src/clusterfuzz/_internal/base/feature_flags.py#L23 to avoid the need of PRs and deployments to apply the change.

## Overview As per requested, we created a new feature flag to disable definitely the API build use cases(used trough `fetch_artifact.py`), this feature flag is enabled by default, because that's the current API & ClusterFuzz workflow. But can be disabled by adding the feature flag to the DB and marking it as enabled=false. ## Changes - Creates new feature_flag, and enables it by default in `fetch_artifact.py` - Logs warnings when the feature flag is disabled in `fetch_artifact.py` - Safely exists the related methods in `symbols_downloader` and `flash.py` which consume the `fetch_artifact.py` - Moved up the `adb.bad_state_reach()` method from the `fetch_artifact.py` module to its callers, this way we can now handle better the None values and raised errors from the API being turned off by the feature flag. ## Tests After the initial tests in the PR parents, i tested this changes once more in dev(including all 3 prs), looking specifically for `utils.fetch_url` or errors inherent to the return value from this method. Basically since my changes landed in dev the following error groups where seen: <img width="1735" height="1240" alt="image" src="https://github.com/user-attachments/assets/0591fb5c-d728-406a-b888-4f2534b2daab" /> Discarded the errors following this logic: - 403 Errors are expected to appear as per the changes that were previously there to avoid them are no longer in dev - Some error groups only appeared on previous clusterfuzz versions - Other errors are expected(like the queue size limit or bad revision errors) - There are errors from corpus pruning & fuzzer not found, this have been seen for a while now, not related to my changes - Lastly there are some errors in a cron_job, specifically in [this file ](https://github.com/google/clusterfuzz/blob/master/src/clusterfuzz/_internal/cron/load_bigquery_stats.py)but this one doesn't make usage of `fetch_url` and nor does the method that triggered the error #### PR review 1. [**#5370** Add Android Build API V4 REST client and tests ](#5370) 2. [#5371 Migrate fetch_artifact to use new V4 API](#5371) 3. 👉 [#5373 Creates Feature Flag for Android V4 API](#5373) *(This PR)* Please only approve, I'll merge this PR's to avoid any possible merge conflict that can appear in the process :D

## Changes We previously migrated the v3 api to use v4 in [fetch_artifact.py](https://github.com/google/clusterfuzz/blob/master/src/clusterfuzz/_internal/platforms/android/fetch_artifact.py) but after testing the implementation we discovered that we can't continue the same approach of using a client created by service discovery url , if we want to continue using the android build api we needed to change our strategy and start using raw http requests. - Creates the new android v4 api class - Instantiated trough a static method that recieves service account credentials to avoid silent failures - Updates the `utils.fetch_url` to unify requests and reuse it in this new api while maintaining previous functionality For more info in the v4 api see: go/build-api-v4-clients-migration-guide & b/422775458 ## Tests - Created unit tests for this new class - Tested this class implementation using [a custom script](https://github.com/google/clusterfuzz/compare/android-build-v4-testing) ### Testing script details The [custom script](https://github.com/google/clusterfuzz/compare/android-build-v4-testing) performs various tests that imitate how clusterfuzz would use the `fetch_artifact.py` module in conjunction with our new api. The script test the following: - Correct API Version selection - Correct API instantiation by checking that the credentials in the db are valid - Queries latest artifact info(like build_id) for the given branch and target - branch and target come from a adb device (see where [we got this from](https://paste.googleplex.com/5462351594782720)) - Uses the retrieved build_id to get android builds artifact metadata - Downloads the given artifact Test steps: - Chery-pick the scripot commit into my branch - Run the script using butler: ``` pipenv run python butler.py run -c /usr/local/google/home/ibarba/projects/clusterfuzz-config/configs/chrome-staging test_fetch_artifact \ --script_args \ --branch=git_main \ --target=cf_x86_64_phone-next-userdebug ``` ## Dev verification After the initial tests ^ i tested this changes once more in dev(including all 3 prs), looking specifically for `utils.fetch_url` or errors inherent to the return value from this method. Basically since my changes landed in dev the following error groups where seen: <img width="1735" height="1240" alt="image" src="https://github.com/user-attachments/assets/0591fb5c-d728-406a-b888-4f2534b2daab" /> Discarded the errors following this logic: - 403 Errors are expected to appear as per the changes that were previously there to avoid them are no longer in dev - Some error groups only appeared on previous clusterfuzz versions - Other errors are expected(like the queue size limit or bad revision errors) - There are errors from corpus pruning & fuzzer not found, this have been seen for a while now, not related to my changes - Lastly there are some errors in a cron_job, specifically in [this file ](https://github.com/google/clusterfuzz/blob/master/src/clusterfuzz/_internal/cron/load_bigquery_stats.py)but this one doesn't make usage of `fetch_url` and nor does the method that triggered the error ## Note: ### PR review Stack 1. 👉 **#5370** Add Android Build API V4 REST client and tests *(This PR)* 2. [#5371 Migrate fetch_artifact to use new V4 API](#5371) 3. [#5373 Creates Feature Flag for Android V4 API](#5373) Please only approve, I'll merge this PR's to avoid any possible merge conflict that can appear in the process :D

Migrate Android artifact fetcher to V4 client API

56d6769

IvanBM18 requested review from dylanjew and letitz July 21, 2026 07:26

IvanBM18 requested a review from a team as a code owner July 21, 2026 07:26

IvanBM18 mentioned this pull request Jul 21, 2026

[Adroid v4 API] Add Android Build API V4 REST client and tests #5370

Merged

Uses correct service account credentials at construction

c088acd

letitz removed their request for review July 21, 2026 08:37

dylanjew changed the base branch from master to feature/creates-android-v4-api July 21, 2026 14:54

dylanjew reviewed Jul 21, 2026

View reviewed changes

IvanBM18 added 2 commits July 21, 2026 19:01

Simplify validations & Adds a TODO

f64347f

Merge branch 'feature/creates-android-v4-api' into refactor/use-andro…

f662a8b

…id-v4-api

dylanjew approved these changes Jul 21, 2026

View reviewed changes

IvanBM18 requested a review from javanlacerda July 21, 2026 20:11

IvanBM18 mentioned this pull request Jul 21, 2026

[Android V4 api Migration] Creates Feature Flag #5373

Merged

IvanBM18 changed the title ~~Migrate Android Build V4 API~~ [Android v4 API] Migrate fetch_artifact to use new V4 API Jul 21, 2026

javanlacerda approved these changes Jul 22, 2026

View reviewed changes

IvanBM18 added a commit that referenced this pull request Jul 22, 2026

[#5371] Migrate fetch_artifact to use new V4 API

2c8df7a

IvanBM18 merged commit 1e3a245 into feature/creates-android-v4-api Jul 23, 2026
5 checks passed

IvanBM18 deleted the refactor/use-android-v4-api branch July 23, 2026 16:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Android v4 API] Migrate fetch_artifact to use new V4 API#5371

[Android v4 API] Migrate fetch_artifact to use new V4 API#5371
IvanBM18 merged 5 commits into
feature/creates-android-v4-apifrom
refactor/use-android-v4-api

IvanBM18 commented Jul 21, 2026 •

edited

Loading

Uh oh!

letitz commented Jul 21, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dylanjew Jul 21, 2026

Uh oh!

IvanBM18 Jul 21, 2026

Uh oh!

dylanjew Jul 21, 2026

Uh oh!

IvanBM18 Jul 21, 2026

Uh oh!

Uh oh!

Uh oh!

dylanjew Jul 21, 2026

Uh oh!

javanlacerda Jul 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

IvanBM18 commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Changes

Tests performed

PR review

Uh oh!

letitz commented Jul 21, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dylanjew Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

IvanBM18 Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

dylanjew Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

IvanBM18 Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dylanjew Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

javanlacerda Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

IvanBM18 commented Jul 21, 2026 •

edited

Loading