fix(sdist): regenerate Cargo.lock when workspace members are removed#3192
fix(sdist): regenerate Cargo.lock when workspace members are removed#3192ckhordiasma wants to merge 5 commits into
Conversation
Fixes PyO3#2609. When maturin builds an sdist from a workspace, `rewrite_cargo_toml` strips workspace members not needed by the package. The Cargo.lock was copied verbatim and still referenced those removed crates, so `cargo build --locked` inside the sdist would fail. After all sdist entries are assembled, materialize them to a temp directory, run `cargo generate-lockfile` to reconcile the lockfile, and replace the tracker entry with the regenerated Cargo.lock. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Exercises the fix for PyO3#2609: builds an sdist from a workspace member while a sibling member (`devtool`, with a unique `rand` dependency) is stripped, then asserts `cargo metadata --frozen` succeeds in the unpacked sdist. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR fixes sdists produced from Cargo workspaces where rewrite_cargo_toml removes unneeded workspace members but the copied Cargo.lock still references them, causing cargo build --locked / cargo metadata --frozen to fail inside the unpacked sdist.
Changes:
- Add
VirtualWriter<SDistWriter>::materialize_toandreplace_byteshelpers to support post-processing tracked sdist entries. - After assembling workspace-related sdist entries, materialize the sdist to a temp dir, run
cargo generate-lockfile, and replace the sdist’sCargo.lockwith the regenerated version. - Add a regression test that constructs a two-member workspace, builds an sdist for one member, unpacks it, and asserts
cargo metadata --frozensucceeds.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
tests/run/sdist.rs |
Adds regression coverage ensuring workspace-member sdists have a usable Cargo.lock. |
src/source_distribution/mod.rs |
Adds regenerate_cargo_lock step in the sdist build pipeline to reconcile lockfiles after workspace-member stripping. |
src/module_writer/virtual_writer.rs |
Adds helpers to materialize tracked sdist entries to disk and replace a tracked file’s bytes in-memory. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Use `workspace_members.len() > 1` instead of path-inequality heuristic so workspace-root packages are also covered. - Make `replace_bytes` return `Result` and error if the target entry does not exist in the tracker. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Set file permissions based on the tracked executable flag after writing each entry, matching the existing default_permission pattern. Unix-only via #[cfg(unix)], consistent with the rest of the codebase. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Preserve original executable flag and source path when replacing a tracker entry in replace_bytes. - Include stdout and working directory in cargo generate-lockfile error messages. - Use output() instead of status() in test for better failure diagnostics. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Thanks for the PR — the problem is real and reconciling the lockfile against the rewritten manifests is the right general approach. A few things need to change before this can merge though: 1. Per the cargo docs, if the lockfile already exists it "will be rebuilt with the latest available version of every package" — it's effectively a full Please use 2. Trigger condition is too coarse
3. Offline / cargo config This adds a hard network dependency to sdist builds for multi-member workspaces. At minimum, propagate Smaller points
The |
Summary
Fixes #2609.
When maturin builds an sdist from a workspace,
rewrite_cargo_tomlstrips workspace membersnot needed by the package. The
Cargo.lockwas copied verbatim and still referenced thoseremoved crates, so
cargo build --lockedinside the sdist would fail.After all sdist entries are assembled in the virtual writer, materialize them to a temp
directory, run
cargo generate-lockfileto reconcile the lockfile, and replace the trackerentry with the regenerated
Cargo.lock. This only runs for workspace crates that have aCargo.lockentry in the sdist.Changes
materialize_toandreplace_byteshelpers onVirtualWriter<SDistWriter>regenerate_cargo_lockstep afteradd_workspace_manifestin the sdist buildera unique dependency), unpacks it, and asserts
cargo metadata --frozensucceedsTest plan
cargo test --test run -- sdist_workspace_removed_members_cargo_lockpasses with the fixmainwithout the fix (verified on a separate branch)cargo fmtandcargo clippyclean🤖 Generated with Claude Code — ~40 back-and-forth messages covering issue analysis, implementation, line-by-line code walkthrough, test authoring, failure verification, style review, and PR creation.