[android-sdk] strip cross-arch swiftmodule leakage from per-arch overlays#548
Open
ken0nek wants to merge 1 commit into
Open
[android-sdk] strip cross-arch swiftmodule leakage from per-arch overlays#548ken0nek wants to merge 1 commit into
ken0nek wants to merge 1 commit into
Conversation
…lays
The Swift stdlib and sdk-overlay install steps run from a shared
SWIFT_BUILD_ROOT across all cross-compile-host iterations of build.sh.
Each iteration's install pulls in .swiftinterface / .swiftmodule files
from every host that built before it, leaving non-monotonic per-arch
overlays in the bundle:
lib/swift-aarch64/android/Android.swiftmodule/ - aarch64 only
lib/swift-x86_64/android/Android.swiftmodule/ - aarch64 + x86_64
lib/swift-armv7/android/Android.swiftmodule/ - all three
Foundation, Dispatch, XCTest, and swift-testing are built per-arch via
separate CMake invocations and are correctly partitioned, so the leak
is confined to the 16 stdlib + sdk-overlay swiftmodule dirs (Swift,
Android, _Concurrency, Cxx, CxxStdlib, _Builtin_float, _math,
_RegexParser, _StringProcessing, _Volatile, _Differentiation,
Distributed, Observation, RegexBuilder, SwiftOnoneSupport,
Synchronization).
The asymmetry interacts badly with SwiftPM's targetTriples lookup:
when SwiftPM picks the wrong swiftResourcesPath for the requested
--triple, the wrong-arch .swiftinterface load succeeds because of the
leaked files, and the failure surfaces one step later as a Clang
modulemap miss ("missing required module 'SwiftAndroid'") instead of
the cleaner "could not find module 'Foundation' for target ...".
Strip leaked entries during the bundling stage, after the per-arch
rename and before the rsync into swift_res_root, so each
lib/swift{,_static}-<arch>/android/*.swiftmodule/ contains only this
arch's triple - matching the layout already produced by Foundation
and friends.
ken0nek
requested review from
justice-adams-apple and
shahmishal
as code owners
Member
|
Sorry to break it to you after this long exegesis, 😉 but the extra modules issue has long been known, #510, and I fixed it weeks ago: It just didn't make it into the first two 6.3 tags, but will be in the next patch release. |
Member
|
This is fixed in the 6.3.2 release, as I said it would be: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Android Swift SDK artifactbundle produced by
swift-ci/sdks/android/scripts/build.shships non-monotonic per-arch overlays for 16 stdlib + sdk-overlay swiftmodule directories. Each per-arch resource dir (swift-<arch>/android/<Module>.swiftmodule/andswift_static-<arch>/android/<Module>.swiftmodule/) accumulates.swiftinterface/.swiftmodule/.swiftdoc/.abi.json/.private.swiftinterfacefiles from every cross-compile host that built before it, instead of carrying only its own arch's triple. This patch prunes the leaked entries during the bundling stage so each arch's overlay matches the per-arch layout already produced by Foundation, Dispatch, XCTest, and swift-testing.Concrete asymmetry in the current bundle
Inspection of
swift-6.3.1-RELEASE_android.artifactbundle(Apple-built, SBOM dated 2026-04-14, defaultTARGET_ARCHS=aarch64,x86_64,armv7):Android.swiftmodule/entriesswift-aarch64/android/swift-x86_64/android/swift-armv7/android/The same accumulation hits 16 modules:
Swift,SwiftOnoneSupport,Android,_Concurrency,_Differentiation,Distributed,Synchronization,Observation,Cxx,CxxStdlib,_Builtin_float,_math,_Volatile,_RegexParser,_StringProcessing,RegexBuilder— under bothswift-<arch>/andswift_static-<arch>/.Foundation,FoundationEssentials,FoundationInternationalization,FoundationNetworking,FoundationXML,Dispatch,XCTest, andTestingare correctly partitioned (each arch's dir contains only its own triple).Root cause
The discriminator matches the install path used in
build.sh:--swift-install-components='compiler;clang-resource-dir-symlink;license;stdlib;sdk-overlay'runs inside the Swiftbuild-scriptorchestrator from a sharedSWIFT_BUILD_ROOT=${build_dir}/swift-projectacross every--cross-compile-hosts=android-${arch}iteration. The install step copies per-host swiftmodule output from every host that built before it. Leaky.--install-libdispatch --install-foundation --xctest --install-xctest --swift-testing --install-swift-testingare separate per-arch CMake invocations earlier in the arch loop, with strictly per-arch install dirs. Clean.By the time bundling reaches the
mv lib/swift lib/swift-${arch}rename (line 666 ofbuild.sh),${sdk_staging}/${arch}/usr/lib/swift/android/*.swiftmodule/already contains cross-arch artifacts. The rename + subsequentrsyncpropagates the leakage to the final publishedswift-${arch}/android/directory.Why this matters downstream
swift-sdk.jsondeclares 27targetTriples(3 arches × 9 API levels). When SwiftPM'stargetTripleslookup picks the wrongswiftResourcesPathfor the requested--triple(a non-determinism inSwiftSDKBundleStore.selectBundle's artifact-ID match path, fixed in SwiftPM trunk by2f506071ad; regression test inswiftlang/swift-package-manager#9988), the wrong-arch.swiftinterfaceload succeeds because of the leaked files. The failure surfaces one step later as a Clang modulemap miss:instead of the cleaner
Symmetric per-arch overlays don't eliminate the SwiftPM lottery, but they make its failure mode transparent and consistent with how Foundation already behaves.
The fix
Single 24-line addition to
swift-ci/sdks/android/scripts/build.sh, inside the bundling-stage arch loop, after thelib/swift → lib/swift-${arch}rename and before thersyncintoswift_res_root:The triple-prefix
<arch>-unknown-linux-android.is correct for all three arches: filenames in the bundle usearmv7-unknown-linux-android.<ext>even though the linker triple isarm-linux-androideabi. Already-clean directories (Foundation et al.) match the keep pattern entirely, so the prune is a no-op there.Verification (no Linux container, NDK, or
build-scriptrebuild required)Paste-and-run against any installed
swift-6.3.1-RELEASE_android.artifactbundle. The script copies the worst-case arch (armv7, last under default order) to a tmp dir, applies the prune, and asserts symmetry:Expected output:
Locally observed before/after counts:
swift-armv7/android/*.swiftmodule/swift_static-armv7/android/*.swiftmodule/Foundation.swiftmodule/(already clean)lib*.so,lib*.a,lib*.modulemap, etc.)Scope
SWIFT_BUILD_ROOTaccumulation inswiftlang/swift'sutils/build-script-implinstall path is not touched here; addressing it properly risks regressions for other consumers and is out of scope for this packaging-level fix.info.json,swift-sdk.json, thetargetTriplesmap, or theswift-resources/usr/lib/swift-<arch>directory layout. Reviewers can compare bundle directory listings before/after to confirm — only file contents of the 32 leaky.swiftmodule/directories change.Related
swiftlang/swift-package-manager#9988— regression test locking in the fix for thetargetTriplesselection lottery.SwiftSDKBundleStore.selectBundle's artifact-ID match path was iterating over the 27-entry triple map in hash-randomized order, producing a non-deterministicswiftResourcesPaththat sometimes pointed at the wrong arch (the-target aarch64-…-android33paired with-resource-dir …/swift-armv7mismatch observable indescription.json). Fixed in SwiftPM trunk by2f506071ad; the new test synthesizes a multi-triple Android-shaped bundle and asserts selection across all (arch, API-level) combos. This swift-docker PR is independent — symmetric overlays improve bundle correctness whether SwiftPM's selection logic regresses again or not.swiftlang/swift-package-manager#9229— broader in-flight fix (open since 2025-10-08) that covers the sameswift sdk configure --swift-static-resources-pathtypo inSwiftSDK.PathsConfiguration.merge(with:relativeTo:)(assigning toself.swiftResourcesPathinstead ofself.swiftStaticResourcesPath), a sibling typo inSources/SwiftSDKCommand/Configuration/SetConfiguration.swift:97, and a rework ofselectSwiftSDKto surface ambiguous matches. Adjacent multi-resource-path SDK territory, but unrelated to the leakage fixed here. (A parallel single-typo fix was filed as#9987and closed 2026-04-27 as duplicate of #9229.)