Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions 01_getting_started/05_local_modules/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.flash/
.venv/
__pycache__/
*.pyc
140 changes: 140 additions & 0 deletions 01_getting_started/05_local_modules/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Local (non-pip) modules

Factor a Flash endpoint across local Python files that are **not** pip-installable —
a sibling module and a package that live next to the endpoint. This is the
capability added by **SLS-360**.

```
05_local_modules/
├── cpu_worker.py # the @Endpoint — imports the local modules below
├── text_utils.py # local sibling module
└── greetings/ # local package
├── __init__.py # re-exports render() from .messages (transitive import)
└── messages.py # greeting templates
```

`cpu_worker.py` imports `text_utils` and `greetings` **inside the function body**.
That is required on the live path (`flash dev` / `.run()`), where only the
function source plus its local-module closure are shipped to the worker — so
imports must resolve at call time. The same code works unchanged for
`flash deploy`, where the whole project tree is bundled.

## Run it

```bash
# Live dev server (see the live-path note below re: worker image)
flash dev
# then: curl -s localhost:8888/cpu_worker/runsync -d '{"input": {"name": "Ada", "lang": "es"}}'

# Or deploy to Runpod
flash deploy
```

Expected result shape:

```json
{"status": "success", "greeting": "HOLA, ADA!", "timestamp": "..."}
```

`greeting` is produced by the local modules: `greetings.render()` builds
`"Hola, Ada"`, then `text_utils.shout()` upper-cases and adds `!`.

## Verifying SLS-360

Three tiers, in increasing cost. Tier 1 needs no infra; Tier 2 uses a real
Runpod endpoint; Tier 3 exercises the live inline-shipping path.

### Tier 1 — build path (local, no infra)

The build resolves each endpoint's local-import closure, force-includes those
files even when the ignore filter would drop them, and fails loudly if an
endpoint's local import can't be resolved.

```bash
flash build

# (a) sibling + package (init AND submodule) are all bundled:
tar tzf .flash/artifact.tar.gz | grep -E 'text_utils|greetings|cpu_worker'
# ./cpu_worker.py
# ./greetings/__init__.py
# ./greetings/messages.py
# ./text_utils.py
```

Force-include of an **ignore-dropped** module (this is the specific SLS-360 fix —
a module the ignore rules would silently drop is still bundled because an
endpoint imports it):

```bash
# a helper whose name matches the built-in test_*.py ignore rule
printf 'SAMPLE_NAMES = ["Ada", "Alan", "Grace"]\n' > test_samples.py
# make the endpoint import it (add `import test_samples` inside greet())
flash build
tar tzf .flash/artifact.tar.gz | grep test_samples.py # -> ./test_samples.py (rescued)
rm test_samples.py # revert the endpoint edit too
```

Loud failure on an unresolved endpoint import (clean error, exit 1, no traceback):

```bash
# temporarily add `from . import missing_sibling` to cpu_worker.py, then:
flash build ; echo "exit=$?"
# ✗ .../cpu_worker.py: relative import (level=1, module='missing_sibling')
# could not be resolved to a local file under ...
# exit=1
```

### Tier 2 — deploy path (real endpoint) — verified

The deploy path works with **any** worker image: the local modules are physically
bundled into the artifact and unpacked onto the worker filesystem, so the
endpoint imports them at runtime. This validates SLS-360's build/deploy half
end-to-end on real infra today.

```bash
flash deploy
curl -X POST https://api.runpod.ai/v2/<endpoint-id>/runsync \
-H "Authorization: Bearer $RUNPOD_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input": {"name": "Ada", "lang": "es"}}'
# -> {"output": {"greeting": "HOLA, ADA!", "status": "success", ...}, "status": "COMPLETED"}
flash undeploy <name> --force # tear down when done
```

Verified on a real CPU endpoint: the worker imported the local sibling
(`text_utils`) and package (`greetings`) and returned `"HOLA, ADA!"`.

> **Payload shape:** the generated queue handler calls the endpoint as
> `greet(**job_input)`, so the function takes typed params (`name`, `lang`) and
> the request nests them under `input`. A single `def greet(input_data: dict)`
> would instead require `{"input": {"input_data": {...}}}`.
>
> **Worker turnover:** a warm worker keeps serving the previous build for a short
> while after `flash deploy`. If a call returns stale behavior, retry until a
> fresh worker (new `workerId`) picks up the new build.

### Tier 3 — live path (`flash dev` / `.run()`)

This is the genuinely new capability: local modules are shipped **inline** with
the function and materialized on the worker's `sys.path` before the function
runs. It requires the worker runtime that materializes `FunctionRequest.modules`
(flash-worker PR runpod-workers/flash#100).

- **Against stock Runpod workers** (current published image): the inline modules
are ignored by the worker, so `import text_utils` fails on the worker. This is
expected until #100 is released.
- **To test now:** build the flash-worker image locally with #100
(`make build` in the worker repo) and exercise it via the worker's
`make smoketest` with a request carrying `modules`, or via
`flash deploy --preview` (docker-compose) pointed at the local image.
- **After #100 ships:** `flash dev` + a POST to `/cpu_worker/runsync` works
directly, and the greeting is computed by the shipped local modules on the
worker.

## Why in-function imports

On the live path only the function's own source is extracted and shipped. Module
top-level imports/constants are not sent (that is the older AE-2308 behavior).
Keeping `import text_utils` / `from greetings import render` inside `greet()`
ensures the resolver discovers them and the worker can import them at call time —
and it stays correct for `flash deploy` too.
34 changes: 34 additions & 0 deletions 01_getting_started/05_local_modules/cpu_worker.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# CPU serverless worker that factors its logic across local (non-pip) modules.
# run locally with: flash dev | deploy with: flash deploy
from runpod_flash import CpuInstanceType, Endpoint


@Endpoint(
name="01_05_local_modules",
cpu=CpuInstanceType.CPU3C_1_2,
)
async def greet(name: str = "world", lang: str = "en") -> dict:
"""Greeting endpoint whose logic lives in local sibling + package modules.

The imports are inside the function body on purpose. On the live path
(`flash dev` / `.run()`) only the function source plus its local-module
closure are shipped to the worker, so imports must resolve at call time.
The same code works unchanged for `flash deploy` (the whole tree is bundled).
"""
from datetime import datetime

import text_utils
from greetings import render

return {
"status": "success",
"greeting": text_utils.shout(render(name, lang)),
"timestamp": datetime.now().isoformat(),
}


if __name__ == "__main__":
import asyncio

result = asyncio.run(greet(name="Flash", lang="es"))
print(result)
10 changes: 10 additions & 0 deletions 01_getting_started/05_local_modules/greetings/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
"""Local package whose __init__ re-exports from a submodule.

Exercises transitive local-import resolution: the endpoint imports ``greetings``,
whose ``__init__`` imports ``greetings.messages`` — the resolver must pull in both
``greetings/__init__.py`` and ``greetings/messages.py``.
"""

from .messages import render

__all__ = ["render"]
12 changes: 12 additions & 0 deletions 01_getting_started/05_local_modules/greetings/messages.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
"""Package submodule with greeting templates."""

TEMPLATES = {
"en": "Hello, {name}",
"es": "Hola, {name}",
"fr": "Bonjour, {name}",
}


def render(name: str, lang: str = "en") -> str:
template = TEMPLATES.get(lang, TEMPLATES["en"])
return template.format(name=name)
8 changes: 8 additions & 0 deletions 01_getting_started/05_local_modules/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[project]
name = "05_local_modules"
version = "0.1.0"
description = "Factor a Flash endpoint across local (non-pip) modules (SLS-360)"
requires-python = ">=3.10"
dependencies = [
"runpod-flash",
]
10 changes: 10 additions & 0 deletions 01_getting_started/05_local_modules/text_utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
"""Local sibling module (not pip-installable) that lives next to the endpoint.

Demonstrates SLS-360: a Flash endpoint can factor logic into local files and
have them ship to the worker.
"""


def shout(text: str) -> str:
"""Uppercase with emphasis."""
return f"{text.upper()}!"
Loading