feat(turbopack): add async-to-promise transpilation bounds for old browser capability matching

[xusd320]

This pull request makes several significant changes to the turbopack-core crate, focusing on simplifying minification options, improving version comparison logic, updating asset identifier hashing, and cleaning up unused or redundant code. The most notable updates include the removal of the CompressType and CompressOptions structs, the introduction of a macro for easier version comparisons, a switch to base38 encoding for asset hashes, and the removal of the unused Umd variant from ExternalType.

Minification and Compression Refactoring:

• Removed the CompressType enum and CompressOptions struct from the minification pipeline, simplifying the MinifyType definition to only include the mangle option. This reduces complexity and unused configuration. [1] [2]

Environment Version Comparison Improvements:

• Introduced the version_at_least! macro for concise and consistent browser version checks, and refactored feature support methods (e.g., for arrow functions and async/await) to use this macro. Added a new method to check for async/await support. [1] [2] [3]

Asset Identifier Hashing Updates:

• Changed asset identifier hash encoding from hexadecimal to base38, reducing the length of hash strings while maintaining collision resistance. Also adjusted the hash truncation logic and ensured extension disambiguation is handled correctly. [1] [2] [3]
• Simplified the file path escaping logic to only replace a narrower set of separator characters.

Module Graph API Simplification:

• Consolidated and simplified ModuleGraph construction methods by merging variants that handled unused reference removal into a single method with an optional binding usage parameter. Updated tests accordingly. [1] [2] [3]

Resolve and External Type Cleanups:

• Removed the unused Umd variant from the ExternalType enum and all related code paths. [1] [2] [3]
• Simplified Windows path resolution logic by removing unimplemented or redundant code and clarifying error messages for unsupported imports.
• Cleaned up unused imports in the resolve module.

[gemini-code-assist]

Code Review

This pull request refactors Turbopack's core by implementing AST-level async module wrapping, switching asset identification to base38 encoding, and simplifying minification and path resolution logic. Feedback indicates that removing chunk item sorting may break build determinism, and using global instead of globalThis in external modules will cause browser compatibility issues. Additionally, the simplified path escape regex might lead to invalid filenames, the removal of tilde prefix handling is a breaking change for existing projects, and the async-to-yield transformation needs to account for for await...of loops.

I am having trouble creating individual review comments. Click here to see my feedback.

turbopack/crates/turbopack-ecmascript/src/chunk/chunk_type.rs (39-64)

The removal of the sorting logic for chunk items by their module ID is a significant change that likely breaks build determinism. Sorting ensures that chunks containing the same modules produce identical content regardless of the order in which they were discovered or processed. Without this sort, the generated chunk content may vary between builds, leading to cache misses and non-reproducible artifacts.

turbopack/crates/turbopack-ecmascript/src/references/external_module.rs (226)

The use of global here appears to be a mistake. In browser environments, the global object is accessed via globalThis or window. While global is standard in Node.js, it will be undefined in most browsers. This is also inconsistent with the use of globalThis elsewhere in this file (e.g., line 204).

  if (typeof globalThis[${variable_name}] === 'undefined') {

turbopack/crates/turbopack-ecmascript/src/references/external_module.rs (237)

As noted previously, global should be replaced with globalThis to ensure compatibility with browser environments.

  mod = globalThis[${variable_name}];

turbopack/crates/turbopack-core/src/ident.rs (440)

The simplification of the SEPARATOR_REGEX from r"[/#?:\[\]<>@\s()]" to r"[/#?:]" removes several characters that are problematic in file paths across different operating systems. Specifically, characters like <, >, [, ], @, and spaces are often reserved or require special handling in shells and certain file systems (e.g., Windows reserved characters). Reducing the set of escaped characters might lead to invalid filenames or issues with downstream tools.

turbopack/crates/turbopack-core/src/resolve/parse.rs (166-178)

Removing the webpack-style tilde prefix (~) handling is a breaking change. This prefix is widely used in the ecosystem, particularly in CSS preprocessors (like sass-loader or less-loader), to signal that a path should be resolved from node_modules. Removing this will break existing projects that rely on this resolution behavior.

turbopack/crates/turbopack-ecmascript/src/references/async_module.rs (398-424)

The replace_await_with_yield function correctly transforms AwaitExpr into YieldExpr, but it misses for await...of loops. If the environment does not support async/await, it will also not support for await syntax. These loops should also be transpiled (e.g., to a standard for...of loop that yields the iteration result) to avoid syntax errors in older browsers.