Add generic environment-creation utilities (PEP 723 PR 5a/16) by StellaHuang95 · Pull Request #1651 · microsoft/vscode-python-environments

StellaHuang95 · 2026-07-23T22:19:38Z

Part of #1602 (PEP 723 inline script env support). Design doc: #1601.

Split for review (3 PRs). Reviewers flagged the original PR 5 as too large, so it is split into three stacked PRs grouped by dependency layer:

5a — generic env-creation utilities — this PR (Add generic environment-creation utilities (PEP 723 PR 5a/16) #1651). Based on main; independent; merges first.

5b — inline-script cache + interpreter utilities — Add inline-script cache and interpreter utilities (PEP 723 PR 5b/16) #1655. Stacked on 5a.

5c — create() happy path (manager + wiring) — Add PEP 723 inline script environment creation (PEP 723 PR 5c/16) #1656. Stacked on 5b.

Applied together the three PRs are byte-for-byte identical to the original single change. Merge order: 5a → 5b → 5c.

Roadmap context

This is the first slice of PR 5 of 16 in the PEP 723 inline-script roadmap. The full plan lives in #1602.

Phase	PR	Status
Phase 1: Foundation	PR 1: cache key hash utility	merged (#1634)
	PR 2: cache layout + `meta.json` sidecar	merged (#1635)
	PR 3: `requires-python` to interpreter selection	merged (#1636)
Phase 2: Manager	PR 4: `InlineScriptEnvManager` skeleton	merged (#1610)
	PR 5a: generic env-creation utilities	this PR (#1651)
	PR 5b: inline-script cache + interpreter utilities	#1655
	PR 5c: `create()` happy path (manager + wiring)	#1656
	PR 6: `create()` uv-install fallback	not started (needs 3, 5)
	PR 7: persistence with `get`, `set`, and Memento	not started (needs 4)
	PR 8: activation-time discovery	not started (needs 2, 4, 7)
Phase 3: Routing	PR 9: route PEP 723 scripts to the inline manager	not started (needs 4, 7)
	PR 10: per-script project registration	not started (needs 9)
Phase 4+: UX / lifecycle	PRs 11-16	not started

Why this PR

PR 5c implements InlineScriptEnvManager.create(). Before touching the manager, this PR lands the generic, reusable primitives it relies on — a cross-process file lock, a venv Python-path helper, a cancellation-hardened process runner, and two small createWithProgress options. None of this code is inline-script-specific, so it is reviewed on its own.

What this PR adds

Cross-process file lock (src/common/lockfile.apis.ts, new): acquireFileLock uses an atomic mkdir of a <path>.lock directory plus a per-owner marker file, returning AcquiredFileLock { release, retain }. retain() writes a retained marker so a later acquirer fails fast with ELOCKRETAINED instead of waiting out the 5-minute timeout — used when a build is cancelled mid-flight. Distinct error codes (ELOCKED, ELOCKRETAINED, ELOCKORPHANED, ECOMPROMISED, ERETAINFAILED) separate contention from corruption.

Shared getVenvPythonPath (src/common/utils/virtualEnvironment.ts, new): returns Scripts\python.exe on Windows, else bin/python. Replaces an inline copy in venvUtils and is reused by 5b/5c.

Hardened process helper (src/managers/builtin/helpers.ts): runUV and runPython now share one runProcess implementation whose cancellation guards kill() in try/catch and still emits a clean CancellationError if the process errors after a cancel. Per-caller options preserve existing behavior (collectStderr, logPrefix).

venvUtils.ts: createWithProgress gains CreateWithProgressOptions { trackUvEnvironment }, and CreateEnvironmentResult gains pkgInstallationCancelled so a caller can tell cancellation apart from a real install failure. Existing callers are unaffected (both are optional / additive).

Tests

lockfile.apis.unit.test.ts — 9 tests: contention, retain/fail-fast, orphaned and compromised locks, and timeout.
virtualEnvironment.unit.test.ts — 2 tests for getVenvPythonPath on Windows and POSIX.
helpers.cancellation.unit.test.ts — 4 tests for runProcess cancellation safety.
venvUtils.createWithProgress.unit.test.ts — 3 tests for trackUvEnvironment and pkgInstallationCancelled.

On this branch alone npm run compile-tests is clean and npm run unittest reports 1447 passing, 0 failing, 4 pending.

User impact

None. These are internal primitives with no new user-visible behavior. The refactors to helpers.ts and venvUtils.ts are behavior-preserving for existing callers.

Cross-process file lock, venv Python-path helper, cancellation-safe process runner, and createWithProgress tracking options that inline-script environment creation builds on. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 39dcc6a3-0fbd-4f36-9d0f-68677de49c27

edvilme

LGTM:)

edvilme · 2026-07-24T22:20:52Z

+async function delay(milliseconds: number): Promise<void> {
+    return new Promise((resolve) => setTimeout(resolve, milliseconds));
+}


Maybe this could be moved to a more generic location? But it doesn't bother me too much.

eleanorjboyd · 2026-07-27T18:12:30Z

        });

        proc.on('error', (err) => {
+            if (cancellationRequested) {


is cleanup required by this error stage? you went through and confirmed we aren't leaving stuff behind in people's file systems right?

This change I made only catches proc.kill() throwing so the operation still reports CancellationError because it didn't catch the exception before, these 2 functions don't do cleanups, cleanup belongs to the caller. If you're asking about inline script manager in particular, that will be handled at the inline manager level later.

…1655) > Part of #1602 (PEP 723 inline script env support). Design doc: #1601. > **Split for review (3 PRs).** Reviewers flagged the original PR 5 as too large, so it is split into three PRs grouped by dependency layer: > - **5a — generic env-creation utilities — #1651.** Merged. > - **5b — inline-script cache + interpreter utilities — this PR (#1655).** Rebased on `main`. > - **5c — `create()` happy path (manager + wiring) — #1656.** Stacked on 5b. > > #1651 has merged and this branch has been rebased. The diff now contains only this PR's seven files. **Remaining merge order: 5b → 5c.** ### Roadmap context This is the second slice of **PR 5 of 16**. See #1651 for the full roadmap table. | Phase 2: Manager | PR | Status | |---|---|---| | | PR 4: `InlineScriptEnvManager` skeleton | merged (#1610) | | | PR 5a: generic env-creation utilities | merged (#1651) | | | **PR 5b: inline-script cache + interpreter utilities** | **this PR (#1655)** | | | PR 5c: `create()` happy path (manager + wiring) | #1656 | ### Why this PR With the generic primitives from 5a in place, this PR lands the **inline-script-specific utilities** that `create()` (5c) composes: a normalized dependency cache key, cache-layout ownership/status checks, and interpreter-constraint handling. These are pure functions with no manager wiring yet, so they are reviewed on their own. ### What this PR adds **Cache-key tail normalization** (`src/common/inlineScriptCacheKey.ts`): adds `normalizeRequirementTail`, a quote-aware scanner that collapses whitespace and tightens comparator spacing (`>= 1.0` → `>=1.0`) in a requirement's version/marker tail while **preserving quoted PEP 508 marker literals verbatim** (e.g. `python_version >= "3.11"`). Direct-reference requirements (`pkg @ https://…`) are kept verbatim after the name and extras. The effect is that semantically identical dependency strings normalize to the same cache key, so they reuse the same cached environment. **Cache-layout additions** (`src/common/inlineScriptCacheLayout.ts`): `resolveCacheEntryPath` (containment under the cache root), `inspectOwnedCacheEntry` (realpath ownership), `getBaseInterpreterStatus` (`available | missing | unavailable`), `inspectMetaJson` (typed sidecar read), and a stricter `validateMeta`. The `.meta.json` sidecar schema is `{ schemaVersion, baseInterpreterPath, baseInterpreterVersion, lastUsedAt }`. Uses `getVenvPythonPath` from merged PR #1651. **Interpreter-constraint trimming** (`src/common/inlineScriptInterpreter.ts`): `pickCompatibleInterpreter` now trims `requires-python`, so a whitespace-only constraint is treated as no constraint. **Manager ID constants** (`src/common/constants.ts`): centralizes the conda and inline-script manager IDs used by interpreter filtering. ### Tests - **`inlineScriptCacheKey.unit.test.ts`** — canonicalization cases including marker literals and direct references. - **`inlineScriptCacheLayout.unit.test.ts`** — the new containment, ownership, base-interpreter-status, and typed sidecar-read helpers. - **`inlineScriptInterpreter.unit.test.ts`** — constraint trimming / selection. On this rebased branch `npm run compile-tests` is clean and `npm run unittest` reports **1467 passing, 0 failing, 5 pending**. ### User impact **None.** These are pure utilities. Nothing calls the new code paths until the manager lands in 5c (#1656). ### Merge order #1651 has merged. Merge this PR next, then #1656. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 39dcc6a3-0fbd-4f36-9d0f-68677de49c27

StellaHuang95 marked this pull request as ready for review July 23, 2026 22:21

StellaHuang95 added the feature-request Request for new features or functionality label Jul 23, 2026

StellaHuang95 requested review from edvilme and eleanorjboyd July 23, 2026 22:22

StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from b1e9b95 to f005e16 Compare July 23, 2026 23:06

StellaHuang95 mentioned this pull request Jul 23, 2026

Add inline-script environment creation tests (PEP 723 PR 5/16 - tests) StellaHuang95/vscode-python-environments#2

Closed

StellaHuang95 changed the title ~~Add inline-script environment creation happy path (PEP 723 PR 5/16)~~ Add inline-script environment creation happy path - implementation (PEP 723 PR 5/16) Jul 23, 2026

StellaHuang95 marked this pull request as draft July 23, 2026 23:09

StellaHuang95 mentioned this pull request Jul 23, 2026

Add inline-script environment creation tests (PEP 723 PR 5/16 - tests) #1654

Closed

StellaHuang95 marked this pull request as ready for review July 23, 2026 23:14

StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from f005e16 to d36561b Compare July 23, 2026 23:30

StellaHuang95 changed the title ~~Add inline-script environment creation happy path - implementation (PEP 723 PR 5/16)~~ Add generic environment-creation utilities (PEP 723 PR 5a/16) Jul 23, 2026

This was referenced Jul 23, 2026

Add inline-script cache and interpreter utilities (PEP 723 PR 5b/16) #1655

Merged

Add PEP 723 inline script environment creation (PEP 723 PR 5c/16) #1656

Open

StellaHuang95 marked this pull request as draft July 23, 2026 23:34

StellaHuang95 marked this pull request as ready for review July 23, 2026 23:35

edvilme reviewed Jul 23, 2026

View reviewed changes

Comment thread src/common/utils/virtualEnvironment.ts

Comment thread src/managers/builtin/helpers.ts Outdated

edvilme reviewed Jul 24, 2026

View reviewed changes

Comment thread src/common/lockfile.apis.ts Outdated

Comment thread src/common/lockfile.apis.ts Outdated

Comment thread src/common/lockfile.apis.ts

Comment thread src/managers/builtin/venvUtils.ts Outdated

StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from d36561b to 104166d Compare July 24, 2026 00:13

StellaHuang95 and others added 3 commits July 24, 2026 10:28

address feedback 1

25acae1

address feedback 2

e78bda6

StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from 0706167 to e78bda6 Compare July 24, 2026 17:28

StellaHuang95 requested a review from edvilme July 24, 2026 17:28

edvilme approved these changes Jul 24, 2026

View reviewed changes

eleanorjboyd reviewed Jul 27, 2026

View reviewed changes

eleanorjboyd approved these changes Jul 27, 2026

View reviewed changes

StellaHuang95 merged commit dda5f69 into microsoft:main Jul 27, 2026
46 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add generic environment-creation utilities (PEP 723 PR 5a/16) - #1651

Add generic environment-creation utilities (PEP 723 PR 5a/16)#1651
StellaHuang95 merged 3 commits into
microsoft:mainfrom
StellaHuang95:pep723-pr5-create-happy-path

StellaHuang95 commented Jul 23, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

edvilme left a comment

Uh oh!

edvilme Jul 24, 2026

Uh oh!

eleanorjboyd Jul 27, 2026

Uh oh!

StellaHuang95 Jul 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

StellaHuang95 commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Roadmap context

Why this PR

What this PR adds

Tests

User impact

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

edvilme left a comment

Choose a reason for hiding this comment

Uh oh!

edvilme Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

eleanorjboyd Jul 27, 2026

Choose a reason for hiding this comment

Uh oh!

StellaHuang95 Jul 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

StellaHuang95 commented Jul 23, 2026 •

edited

Loading