Skip to content

[Draft] chore: test Hyperdrive#152

Draft
ryanking13 wants to merge 3 commits into
gyeongjae/experimental-python314from
gyeongjae/hyperdrive
Draft

[Draft] chore: test Hyperdrive#152
ryanking13 wants to merge 3 commits into
gyeongjae/experimental-python314from
gyeongjae/hyperdrive

Conversation

@ryanking13

Copy link
Copy Markdown
Contributor

not ready to be reviewed yet. It seems there is an issue with tls connection

@ask-bonk ask-bonk Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm Bonk, and I've done a quick review of your PR.

This PR adds Hyperdrive integration tests for MySQL (pymysql) and PostgreSQL (pg8000) using the same two-layer test pattern as the bindings tests. I see the PR description mentions it's not ready yet, but here's early feedback on issues that should be fixed before merging.

Issues (highest severity first)

  1. test_hyperdrive.py imports COMPAT_DATES which does not exist in conftest.py — The shared conftest exports COMPAT_CONFIGS (a list of CompatConfig objects), not COMPAT_DATES. The bindings test file (test_bindings.py) is the up-to-date reference. This will cause an ImportError at test collection time.

  2. test_hyperdrive.py doesn't call inject_compat_flags() — The compat_date fixture only passes a bare string, not a CompatConfig. Without injecting the extra compat flags (e.g. enable_python_external_sdk, python_workers_20260610), the tests will fail for all Python versions since each requires specific flags.

  3. test_hyperdrive.py uv run command is missing --no-project — The bindings test uses uv run --no-project --with ... to avoid uv resolving against a parent pyproject.toml. Without --no-project, uv run may pick up the wrong project context.

  4. worker.py is missing the event loop save/restore around pytest.main() — The bindings worker saves and restores the running event loop (asyncio.events._get_running_loop()) around pytest.main() calls to work around a Pyodide WebLoop issue. The hyperdrive worker lacks this, which will cause subsequent requests to hang on a dead loop.

  5. AssertionError typo in worker.py — Line 100 uses excinfo.errisinstance(AssertionError) which references AssertionError (the correct Python name), but note this same pattern exists in the bindings worker — just flagging for awareness that this is a deliberate pytest internal API call, not a typo. (No action needed.)

Items 1-4 are all patterns already solved in test_bindings.py and its worker — aligning with that file should fix everything.

because the database setup is non-trivial. In CI, databases are provided via Docker services
in the hyperdrive-tests.yml workflow.

To run locally:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import will fail: conftest.py does not export COMPAT_DATES. It exports COMPAT_CONFIGS (a list of CompatConfig objects). The fixture and imports should be updated to match the pattern in test_bindings.py.

Suggested change
To run locally:
from conftest import (
COMPAT_CONFIGS,
CompatConfig,
inject_compat_flags,
replace_compat_date,
)

Comment on lines +87 to +90
f"stdout: {stdout}"
)
try:
resp = requests.get(f"{base_url}/health", timeout=2)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fixture should use COMPAT_CONFIGS and yield a CompatConfig object (not a bare string), matching the bindings test pattern. This also enables inject_compat_flags() support.

Suggested change
f"stdout: {stdout}"
)
try:
resp = requests.get(f"{base_url}/health", timeout=2)
@pytest.fixture(
scope="module",
params=COMPAT_CONFIGS,
ids=[c.python_version for c in COMPAT_CONFIGS],
)
def compat_config(request: pytest.FixtureRequest) -> CompatConfig:
return request.param

Comment on lines +93 to +107
except (requests.ConnectionError, requests.Timeout):
time.sleep(DEV_POLL_INTERVAL)

process.kill()
process.wait()
pytest.fail(f"pywrangler dev did not become ready within {DEV_STARTUP_TIMEOUT}s")


@pytest.fixture(scope="module", params=COMPAT_DATES)
def compat_date(request: pytest.FixtureRequest) -> str:
return request.param


@pytest.fixture(scope="module")
def dev_server(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two issues here:

  1. The fixture parameter should be compat_config: CompatConfig instead of compat_date: str.
  2. inject_compat_flags() must be called after replace_compat_date() — without it, tests will fail for Python versions that require extra compat flags.
Suggested change
except (requests.ConnectionError, requests.Timeout):
time.sleep(DEV_POLL_INTERVAL)
process.kill()
process.wait()
pytest.fail(f"pywrangler dev did not become ready within {DEV_STARTUP_TIMEOUT}s")
@pytest.fixture(scope="module", params=COMPAT_DATES)
def compat_date(request: pytest.FixtureRequest) -> str:
return request.param
@pytest.fixture(scope="module")
def dev_server(
def dev_server(
tmp_path_factory: pytest.TempPathFactory, compat_config: CompatConfig
) -> Generator[str]:
tmp_path = tmp_path_factory.mktemp("hyperdrive_test")
target = tmp_path / "hyperdrive-test"
shutil.copytree(HYPERDRIVE_TEST_DIR, target)
env = os.environ | {"_PYODIDE_EXTRA_MOUNTS": str(tmp_path)}
wrangler_jsonc = target / "wrangler.jsonc"
replace_compat_date(wrangler_jsonc, compat_config.compat_date)
inject_compat_flags(wrangler_jsonc, compat_config.extra_compat_flags)
pywrangler_cmd = ["uv", "run", "--no-project", "--with", WORKERS_PY, "pywrangler"]
subprocess.run(

Comment thread packages/cli/tests/test_hyperdrive.py Outdated
Comment on lines +108 to +113
tmp_path_factory: pytest.TempPathFactory, compat_date: str
) -> Generator[str]:
tmp_path = tmp_path_factory.mktemp("hyperdrive_test")
target = tmp_path / "hyperdrive-test"
shutil.copytree(HYPERDRIVE_TEST_DIR, target)
env = os.environ | {"_PYODIDE_EXTRA_MOUNTS": str(tmp_path)}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use the pywrangler_cmd variable (with --no-project) like in test_bindings.py, to avoid uv resolving against a parent pyproject.toml.

Suggested change
tmp_path_factory: pytest.TempPathFactory, compat_date: str
) -> Generator[str]:
tmp_path = tmp_path_factory.mktemp("hyperdrive_test")
target = tmp_path / "hyperdrive-test"
shutil.copytree(HYPERDRIVE_TEST_DIR, target)
env = os.environ | {"_PYODIDE_EXTRA_MOUNTS": str(tmp_path)}
subprocess.run(
[*pywrangler_cmd, "sync"],
cwd=target,
check=True,
env=env,
)

Comment on lines +118 to +132
["uv", "run", "--with", WORKERS_PY, "pywrangler", "sync"],
cwd=target,
check=True,
env=env,
)

shutil.copytree(WORKERS_RUNTIME_SDK, target / "python_modules", dirs_exist_ok=True)

port: int = _get_free_port()
base_url: str = f"http://127.0.0.1:{port}"

process = subprocess.Popen(
[
"uv",
"run",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue — should use the shared pywrangler_cmd list with --no-project.

Suggested change
["uv", "run", "--with", WORKERS_PY, "pywrangler", "sync"],
cwd=target,
check=True,
env=env,
)
shutil.copytree(WORKERS_RUNTIME_SDK, target / "python_modules", dirs_exist_ok=True)
port: int = _get_free_port()
base_url: str = f"http://127.0.0.1:{port}"
process = subprocess.Popen(
[
"uv",
"run",
process = subprocess.Popen(
[
*pywrangler_cmd,
"dev",
"--port",
str(port),
"--persist-to",
str(tmp_path / "state"),
],
cwd=target,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
env=env,
)

Comment on lines +143 to +154
if importlib.util.find_spec(module) is None:
return Response.json(
{"error": f"Unknown suite '{suite_name}' (no module '{module}')"},
status=404,
)

collector = ResultCollector()
pytest.main(
["--pyargs", module, "-p", "no:cacheprovider"],
plugins=[collector, EnvPlugin(self.env)],
)
return Response.json(collector.results)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing the event loop save/restore workaround that bindings-test/src/worker.py has. Without this, after the first pytest.main() call, the running loop will point at an abandoned WebLoop, causing subsequent requests to hang.

Suggested change
if importlib.util.find_spec(module) is None:
return Response.json(
{"error": f"Unknown suite '{suite_name}' (no module '{module}')"},
status=404,
)
collector = ResultCollector()
pytest.main(
["--pyargs", module, "-p", "no:cacheprovider"],
plugins=[collector, EnvPlugin(self.env)],
)
return Response.json(collector.results)
def _run_suite(self, suite_name):
module = f"test_{suite_name}"
if importlib.util.find_spec(module) is None:
return Response.json(
{"error": f"Unknown suite '{suite_name}' (no module '{module}')"},
status=404,
)
collector = ResultCollector()
saved_loop = asyncio.events._get_running_loop()
try:
pytest.main(
["--pyargs", module, "-p", "no:cacheprovider"],
plugins=[collector, EnvPlugin(self.env)],
)
finally:
asyncio.events._set_running_loop(saved_loop)
return Response.json(collector.results)

@ask-bonk

ask-bonk Bot commented Jul 9, 2026

Copy link
Copy Markdown

I'm Bonk, and I've done a quick review of your PR.

This PR adds Hyperdrive integration tests for MySQL and PostgreSQL. I posted a batch review on PR #152 with 4 actionable issues (all with concrete suggestion comments):

  1. COMPAT_DATES import will failconftest.py exports COMPAT_CONFIGS, not COMPAT_DATES. This is a breaking import error.
  2. Missing inject_compat_flags() call — Without injecting extra compat flags, tests will fail for all Python versions that need them.
  3. Missing --no-project in uv run — Can cause uv to resolve against the wrong pyproject.toml.
  4. Missing event loop save/restore in worker.py — Will cause subsequent requests to hang after the first pytest.main() call.

All four are already solved in test_bindings.py and its worker — aligning with that file should fix everything.

github run

@ryanking13 ryanking13 marked this pull request as draft July 9, 2026 07:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant