Migrate from Poetry to uv for dependency management (PP-4076) by jonathangreen · Pull Request #3221 · ThePalaceProject/circulation

jonathangreen · 2026-04-10T14:27:05Z

Description

Migrate the entire project from Poetry to uv for dependency management, packaging, and virtualenv orchestration.

Motivation and Context

Poetry lacks native workspace/monorepo support, which is needed for future project restructuring. uv also provides faster dependency resolution, built-in Python version management, and a simpler CI setup via astral-sh/setup-uv.

How Has This Been Tested?

uv lock resolves 213 packages successfully
uv build produces a clean sdist + wheel (597 files, correct namespace)
uv sync --all-groups installs all dependencies
uv run mypy passes (1123 source files, no issues)
uv run pytest --no-cov tests/manager/util/test_log.py passes (23 tests)
All pre-commit hooks pass
Tests pass in CI

Checklist

I have updated the documentation accordingly.
All new and existing tests passed.

codecov · 2026-04-10T17:50:32Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.30%. Comparing base (c555189) to head (c9d32ed).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3221   +/-   ##
=======================================
  Coverage   93.30%   93.30%           
=======================================
  Files         497      497           
  Lines       46144    46144           
  Branches     6318     6318           
=======================================
+ Hits        43054    43055    +1     
  Misses       2004     2004           
+ Partials     1086     1085    -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Black can auto-detect target versions from pyproject.toml, which will bring in unexpected formatting changes when python-requires is updated (see #3221). Explicitly set -t py312/py313/py314 in pre-commit config so these formatting changes are introduced separately.

## Description Explicitly set `-t py312`, `-t py313`, and `-t py314` target version flags for black in the pre-commit configuration, and apply the resulting formatting changes. ## Motivation and Context In #3221, a change to `python-requires` in `pyproject.toml` allows black to auto-detect the supported Python versions. This auto-detection brings in unexpected formatting changes (parenthesized `with` statements, trailing commas on `**kwargs`, etc.). By explicitly setting the target versions in pre-commit and applying the formatting changes here, we can land #3221 without mixing unrelated formatting noise into that PR. Reference: https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#t-target-version ## How Has This Been Tested? - `pre-commit run -a` passes cleanly with the updated configuration. ## Checklist - [x] I have updated the documentation accordingly. - [x] All new and existing tests passed.

tdilauro

✔️

Replace Poetry with uv across the entire project: pyproject.toml converted to PEP 621 with hatchling build backend, dependency groups (PEP 735) for dev/ci, pg moved to an optional-dependencies extra, and tox-uv added for tox integration. All four GitHub workflows now use astral-sh/setup-uv@v4. Docker images use COPY --from=ghcr.io/astral-sh/uv:0.5 instead of the Poetry curl installer. README and CLAUDE.md updated to reflect the new tooling. Dependabot switched from pip to uv ecosystem. uv.lock is excluded from the check-added-large-files pre-commit hook since lockfiles must be checked in regardless of size. This is PR 1 of the monorepo restructure plan. The .github/actions/poetry composite action is intentionally left on disk for transitive consumers in other repositories.

…ault Add a recommended direnv setup section so developers can auto-activate the virtualenv on cd without needing the uv run prefix. Change the default install command from uv sync to uv sync --all-groups so both dev and ci groups are installed. Fix broken #pyenv anchor reference.

Move psycopg2-binary from the dev dependency group into the main project dependencies and drop the pg optional extra entirely. In Docker containers where the environment is fully controlled, psycopg2-binary works fine in production and eliminates the need to maintain separate binary/source variants. Remove --extra pg from Dockerfiles, workflows, and README.

Add a 4-day cooldown for uv dependency updates as recommended by the uv dependabot integration docs, to avoid PRs that uv cannot resolve due to exclude-newer constraints.

The commands_pre section with punkt_tab and textblob corpus downloads was accidentally dropped during the Poetry-to-uv migration. Without these corpora, the summary evaluator tests fail with MissingCorpusError.

The uv-venv-runner doesn't install project dependencies (it's a simple venv creator). Switch to uv-venv-lock-runner which runs `uv sync --locked` to install the project and all dependencies from the lockfile. Also remove `skipsdist = true` which prevented tox from installing the project, and use `package = skip` for the report env instead of `skip_install`.

- Pin setup-uv to v8.0.0 and use activate-environment to avoid uv run wrappers in all workflows - Fix test.yml: pass matrix python-version to setup-python so tox actually tests against 3.12, 3.13, and 3.14 - Add --frozen to lint.yml install step for CI consistency - Move UV_PROJECT_ENVIRONMENT to ENV in Dockerfile.baseimage so it propagates to child images automatically - Bump uv Docker image from 0.5 to 0.11

The Poetry to uv migration in #3221 switched the container install from editable (Poetry's default) to non-editable (uv sync --no-editable). That change broke palace.manager.scripts.startup's discovery of the startup_tasks/ directory — the script resolves STARTUP_TASKS_DIR via Path(__file__).parents[4], which assumes startup.py is at ${root}/src/palace/manager/scripts/startup.py (editable install). Under --no-editable, startup.py lives inside site-packages and parents[4] resolves to env/lib/python3.12/ instead of the project root, so startup_tasks/ is never found. Drop --no-editable to restore the working layout.

Switch dependency management and build tooling from Poetry to uv, matching the configuration pattern used in the circulation repo (PR ThePalaceProject/circulation#3221). - Rewrite pyproject.toml using PEP 621 [project] + hatchling build backend; move dev/ci deps to PEP 735 [dependency-groups]; git subdirectory deps move to [tool.uv.sources] (palace-util now resolves transitively via palace-opds). - Replace poetry.lock with uv.lock. - tox.ini uses the uv-venv-lock-runner (tox-uv). - GitHub workflows use astral-sh/setup-uv; lint workflow gets a pre-commit cache. - Dependabot switches from pip to uv ecosystem. - README documents uv-based setup. - .python-version now pins a real version (was a pyenv-virtualenv name).

## Description Drop `--no-editable` from the final `uv sync` in `docker/Dockerfile` so `palace-manager` is installed editably in the container — same behavior we had under Poetry. ## Motivation and Context Found while checking the uv changes that rolled out to Minotaur last night. The Poetry → uv migration in #3221 switched the in-container install from editable (Poetry's default) to non-editable (`uv sync --no-editable`). That change quietly broke `palace.manager.scripts.startup`'s discovery of the `startup_tasks/` directory: ``` Startup tasks directory /var/www/circulation/env/lib/python3.12/startup_tasks does not exist; skipping. ``` The script resolves `STARTUP_TASKS_DIR` via `Path(__file__).parents[4]`, which assumes `startup.py` is at `${root}/src/palace/manager/scripts/startup.py` (editable install). Under `--no-editable`, `startup.py` lives inside `site-packages` and `parents[4]` resolves to `env/lib/python3.12/` instead of the project root, so `startup_tasks/` is never found. Restoring the editable install fixes the immediate breakage and matches the previous behavior. It also has the minor side benefit of making in-container debugging easier — you can edit files in place without rebuilding the image. ## Future work Eventually we'd like to build `palace-manager` as a proper non-editable package that can be published the same way `palace-util` and `palace-opds` will be. Several path-resolution issues need to be sorted out first (alembic migrations, startup scripts, and similar file-layout assumptions) before that's feasible. For now, editable install is the simpler and safer choice. ## How Has This Been Tested? - Built the image locally and confirmed `startup.py` now resolves `STARTUP_TASKS_DIR` to `/var/www/circulation/startup_tasks`. - Existing startup task (`2026_02_17_force_re_harvest_all_opds_for_distributors.py`) is picked up. ## Checklist - [x] I have updated the documentation accordingly. - [x] All new and existing tests passed.

Switch dependency management and build tooling from Poetry to uv, matching the configuration pattern used in the circulation repo (PR ThePalaceProject/circulation#3221). - Rewrite pyproject.toml using PEP 621 [project] + hatchling build backend; move dev/ci deps to PEP 735 [dependency-groups]; git subdirectory deps move to [tool.uv.sources] (palace-util now resolves transitively via palace-opds). - Replace poetry.lock with uv.lock. - tox.ini uses the uv-venv-lock-runner (tox-uv). - GitHub workflows use astral-sh/setup-uv; lint workflow gets a pre-commit cache. - Dependabot switches from pip to uv ecosystem. - README documents uv-based setup. - .python-version now pins a real version (was a pyenv-virtualenv name).

Switch dependency management and build tooling from Poetry to uv, matching the configuration pattern used in the circulation repo (PR ThePalaceProject/circulation#3221). - Rewrite pyproject.toml using PEP 621 [project] + hatchling build backend; move dev/ci deps to PEP 735 [dependency-groups]; git subdirectory deps move to [tool.uv.sources] (palace-util now resolves transitively via palace-opds). - Replace poetry.lock with uv.lock. - tox.ini uses the uv-venv-lock-runner (tox-uv). - GitHub workflows use astral-sh/setup-uv; lint workflow gets a pre-commit cache. - Dependabot switches from pip to uv ecosystem. - README documents uv-based setup.

## Summary - Switch dependency management and build tooling from Poetry to [uv](https://docs.astral.sh/uv/), matching the pattern introduced in [circulation#3221](ThePalaceProject/circulation#3221). - Rewrite \`pyproject.toml\` using PEP 621 \`[project]\` + \`hatchling\` build backend; dev/ci deps move to PEP 735 \`[dependency-groups]\`; git subdirectory deps move to \`[tool.uv.sources]\` (with \`palace-util\` now resolved transitively via palace-opds). - Replace \`poetry.lock\` with \`uv.lock\`. - \`tox.ini\` uses the \`uv-venv-lock-runner\` from \`tox-uv\`. - GitHub workflows use \`astral-sh/setup-uv\`; lint workflow gains a pre-commit cache. - Dependabot switches from \`pip\` to \`uv\` ecosystem. - README documents uv-based setup; \`.python-version\` now pins a real version (was a pyenv-virtualenv name). > Stacked on top of #242. Please review/merge that one first. ## Test plan - [x] \`uv run mypy\` clean - [x] \`uv run pytest\` (154 tests) passes - [x] \`tox\` (py312), \`tox -e mypy\`, \`tox -e lint\` all green

jonathangreen force-pushed the chore/migrate-poetry-to-uv branch from eae2bdf to f9c8666 Compare April 10, 2026 17:06

jonathangreen mentioned this pull request Apr 10, 2026

Set explicit target Python versions for black #3229

Merged

2 tasks

jonathangreen force-pushed the chore/migrate-poetry-to-uv branch from 4094ed6 to 27a80ed Compare April 10, 2026 20:14

jonathangreen changed the base branch from main to chore/set-python-version-for-black April 10, 2026 20:15

jonathangreen requested a review from a team April 13, 2026 12:23

Base automatically changed from chore/set-python-version-for-black to main April 13, 2026 13:14

tdilauro approved these changes Apr 13, 2026

View reviewed changes

Comment thread docker/Dockerfile.baseimage

This was referenced Apr 13, 2026

Extract palace.opds into a uv workspace package (PP-4074) #3230

Merged

Extract palace.util into a uv workspace package (PP-4076) #3231

Merged

Publish palace-util and palace-opds to PyPI on release (PP-4076) #3238

Merged

jonathangreen changed the title ~~Migrate from Poetry to uv for dependency management~~ Migrate from Poetry to uv for dependency management (PP-4076) Apr 14, 2026

jonathangreen marked this pull request as ready for review April 14, 2026 18:45

jonathangreen added 11 commits April 14, 2026 15:47

Add direnv to gitignore

a1f91ae

Fix issue with tox.ini

87be300

Add cooldown to dependabot uv config

2c1d545

Add a 4-day cooldown for uv dependency updates as recommended by the uv dependabot integration docs, to avoid PRs that uv cannot resolve due to exclude-newer constraints.

Restore NLTK/TextBlob corpus downloads in tox commands_pre

1f72a6c

The commands_pre section with punkt_tab and textblob corpus downloads was accidentally dropped during the Poetry-to-uv migration. Without these corpora, the summary evaluator tests fail with MissingCorpusError.

Remove explicit version for black, let it read from pyproject.toml

339c7db

A couple of small updates

c9d32ed

jonathangreen force-pushed the chore/migrate-poetry-to-uv branch from 88c618a to c9d32ed Compare April 14, 2026 18:47

jonathangreen enabled auto-merge (squash) April 14, 2026 18:51

jonathangreen merged commit 5b11aa7 into main Apr 14, 2026
20 checks passed

jonathangreen deleted the chore/migrate-poetry-to-uv branch April 14, 2026 18:58

jonathangreen mentioned this pull request Apr 15, 2026

Install palace-manager editable in the container (PP-4076) #3245

Merged

2 tasks

jonathangreen mentioned this pull request Apr 15, 2026

Migrate from Poetry to uv (PP-3973) ThePalaceProject/palace-tools#243

Merged

3 tasks

jonathangreen added the incompatible changes Changes that require a new major version label Apr 15, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate from Poetry to uv for dependency management (PP-4076)#3221

Migrate from Poetry to uv for dependency management (PP-4076)#3221
jonathangreen merged 11 commits intomainfrom
chore/migrate-poetry-to-uv

jonathangreen commented Apr 10, 2026 •

edited

Loading

Uh oh!

codecov bot commented Apr 10, 2026 •

edited

Loading

Uh oh!

tdilauro left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jonathangreen commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

How Has This Been Tested?

Checklist

Uh oh!

codecov bot commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tdilauro left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jonathangreen commented Apr 10, 2026 •

edited

Loading

codecov bot commented Apr 10, 2026 •

edited

Loading