feat: bundle local (non-pip) modules for serverless endpoints#352
Open
deanq wants to merge 13 commits into
Open
feat: bundle local (non-pip) modules for serverless endpoints#352deanq wants to merge 13 commits into
deanq wants to merge 13 commits into
Conversation
|
Capy auto-review is paused for this organization because the usage-cycle auto-review limit has been reached. Increase the limit or turn it off in billing settings to resume automatic reviews. |
|
Promptless prepared a documentation update related to this change. Triggered by PR #352: feat: bundle local (non-pip) modules for serverless endpoints Documents Flash's new first-class support for bundling local (non-pip) Python modules that endpoints import: a new "Import local modules" section in the Flash endpoints guide (supported import forms, transitive resolution, the 8 MiB live-execution cap, and the parent-directory absolute-import constraint), a force-include note in the build CLI reference, and two troubleshooting entries for the new resolution and payload-size errors. Review: Document Flash local (non-pip) module bundling for endpoints |
Add a shared materialized_modules context manager in runpod_flash.runtime
that writes a {relative_path: source} map to a temp dir, prepends it to
sys.path, and cleans up on exit (including on exception). Wire it into
the /execute LB handler so function_code exec has access to local
modules shipped in the request body's "modules" field.
The LB /execute endpoint parses the request as a plain JSON dict (not a
FunctionRequest model), so modules are read via body.get("modules", {})
rather than an attribute on the FastAPI Request object.
deanq
force-pushed
the
deanquinanola/sls-360-local-module-bundling
branch
from
724bc98 to
0c8c90c
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Adds first-class support for bundling/shipping local (non-pip) Python modules that serverless endpoints import, across both live execution (inline module shipping) and build/deploy (force-include local imports in artifacts), with new protocol support via FunctionRequest.modules.
Changes:
- Added a shared local-import resolver (
resolve_local_modules) to compute a transitive closure of local dependencies for endpoint code. - Live execution paths now populate
modules(subject to an 8 MiB cap) and worker runtime materializes shipped modules ontosys.pathfor the duration of the function call. - Build path now force-includes locally imported modules that would otherwise be ignored, and fails builds for unresolved endpoint-local imports.
Reviewed changes
Copilot reviewed 20 out of 22 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/test_stub_live_serverless.py | Verifies live stub request includes sibling local modules. |
| tests/unit/test_load_balancer_sls_stub.py | Verifies LB stub request includes sibling local modules. |
| tests/unit/stubs/test_local_modules_resolve.py | Unit tests for transitive local module resolution behavior and error cases. |
| tests/unit/stubs/test_local_modules_classify.py | Unit tests for stdlib detection, local path resolution, and size cap constant. |
| tests/unit/stubs/test_live_serverless_modules.py | Tests build_modules_map behavior and size cap enforcement. |
| tests/unit/runtime/test_module_loader.py | Tests temp materialization behavior and sys.path restoration. |
| tests/unit/runtime/test_lb_handler_extended.py | Regression test ensuring shipped modules remain importable during function execution. |
| tests/unit/protos/test_remote_execution.py | Verifies FunctionRequest.modules defaults and round-trips. |
| tests/unit/core/test_exceptions.py | Tests new exception types’ message behavior. |
| tests/unit/cli/commands/test_build_local_modules.py | Tests build-time force-include and endpoint-only strictness behavior. |
| tests/unit/_live_serverless_prepare_request_sibling.py | Sibling module fixture for live-serverless stub tests. |
| tests/unit/_lb_prepare_request_sibling.py | Sibling module fixture for LB stub tests. |
| src/runpod_flash/stubs/local_modules.py | New AST-based resolver for local module import closure and warnings/errors. |
| src/runpod_flash/stubs/load_balancer_sls.py | Populates modules in LB request using build_modules_map. |
| src/runpod_flash/stubs/live_serverless.py | Adds build_modules_map and populates modules in live request with cap enforcement. |
| src/runpod_flash/runtime/module_loader.py | New temp-dir module materializer that prepends to sys.path during execution. |
| src/runpod_flash/runtime/lb_handler.py | Keeps module materialization active through exec + function call/await. |
| src/runpod_flash/protos/remote_execution.py | Adds FunctionRequest.modules protocol field with default empty dict. |
| src/runpod_flash/core/exceptions.py | Adds LocalModuleResolutionError and LocalModulePayloadTooLargeError. |
| src/runpod_flash/cli/commands/build.py | Adds endpoint detection and build-time augmentation to force-include local imports. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+42
to
+47
| tmpdir = tempfile.mkdtemp(prefix="flash_modules_") | ||
| root = Path(tmpdir) | ||
| for rel_path, source in modules.items(): | ||
| dest = root / rel_path | ||
| dest.parent.mkdir(parents=True, exist_ok=True) | ||
| dest.write_text(source, encoding="utf-8") |
Comment on lines
+95
to
+101
| except (LocalModuleResolutionError, UnicodeDecodeError, SyntaxError) as e: | ||
| if _defines_endpoint(py_file): | ||
| raise | ||
| print_warning( | ||
| console, f"Skipping local-module resolution for {py_file}: {e}" | ||
| ) | ||
| continue |
deanq
added a commit
to runpod-workers/flash
that referenced
this pull request
Jul 11, 2026
Worker uses runpod_flash.runtime.module_loader, which is not in a published runpod-flash yet. Temporary git pin so CI installs a flash with the module. Revert to a released constraint once runpod/flash#352 ships.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Flash could not reliably include local (non-pip) Python modules an endpoint imports — a sibling
utils.py, or ahelpers/package next to the endpoint, imported viaimport utils/from helpers import x. This adds first-class support across both code paths that ship user code to a worker.Resolves SLS-360.
@Endpoint(...).run(...),flash devLB dispatch): the endpoint's transitive local-import closure is now shipped inline alongside the function source.flash build/flash deploy): local modules an endpoint imports are force-included in the artifact even when the ignore filter would drop them, and the build fails loudly (clean error, not a traceback) if an endpoint's local import can't be resolved.What changed
FunctionRequest.modules: dict[str, str]— additive protocol field (POSIX relative path → source text), default empty. Backward compatible: old workers ignore it, old SDKs send nothing. Shared contract with the worker repo — see the companion worker PR: feat: SLS-360 import shipped local modules before executing user code runpod-workers/flash#100.stubs/local_modules.py— a shared, pureresolve_local_modules(...)that walks the transitive local-import closure (absolute, relative, andimportlib.import_module("literal")imports, at module level and inside function bodies), classifies local vs stdlib/installed, pulls in package__init__.py, terminates on cycles, warns on non-literal dynamic imports, and raisesLocalModuleResolutionErroron unresolved relative imports / out-of-root files.stubs/live_serverless.pyandstubs/load_balancer_sls.pypopulatemodulesfrom the resolved closure, with an 8 MiB inline cap raisingLocalModulePayloadTooLargeError(points users toflash deployfor larger local dependencies).runtime/module_loader.py::materialized_moduleswrites shipped modules to a temp dir onsys.pathand cleans up; wired intoruntime/lb_handler.py's/executeso the temp dir stays live through the function call (not justexec).augment_with_local_modulesincli/commands/build.pyforce-includes resolved local files past the ignore filter. Strictness is scoped to endpoint files (@Endpoint/@remote): an endpoint with an unresolved local import fails the build loudly; an incidental non-endpoint file that fails resolution is warned and skipped (no regression on unrelated files).Test plan
sys.pathon exception, no-op when empty), both live senders populatemodules, LB/executedef-now/call-later import regression, build force-include + endpoint-only-strict A/B.make quality-checkgreen (86%+ coverage).flash build: an ignoredtest_*.pysibling imported by an endpoint is force-included intoartifact.tar.gz; an endpoint with an unresolvable relative import fails the build with a clean error (exit 1, no traceback).Coordinated release
The worker-side materializer requires the companion worker PR (runpod-workers/flash#100) to be released for the live inline-shipping path to work end-to-end. The deploy path is independent (bundled files sit on the worker filesystem regardless of image version). Live-path E2E against real endpoints is a gating step once the worker image is rebuilt/published.
Follow-ups (non-blocking)
flash deploy.sys.pathmaterialization assumes one invocation at a time per worker; isolate per-invocation if concurrent execution is enabled.ep = Endpoint(...)assignment form is not yet detected (fails safe toward warn).