Skip to content

Commit 09e49f3

Browse files
AdrienVannsonAdrienVannson
authored
Bring back the compiler (still in a different Python package), switch to uv (#106)
* Move betterproto * Add back the compiler * Switch to uv * Add lock file * Add uv lock file * Update pyproject * Fix CI * Temporary fix * Fix workflow name * Fix test skipping on windows * Fix code quality * Fix Python version * Fix action * Remove comment * Compile tests in CI * Move documentation * Fix documentation workflow * Fix Python version * Update lockfile * Bump version * Update pyprojects * Update README * Fix release script for the lib
1 parent 00acbd1 commit 09e49f3

276 files changed

Lines changed: 12291 additions & 2673 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 20 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -13,63 +13,45 @@ permissions:
1313

1414
jobs:
1515
tests:
16-
name: ${{ matrix.os }} / ${{ matrix.python-version }} / Rust codec ${{ matrix.rust-codec }}
16+
name: ${{ matrix.os }} / ${{ matrix.python-version }}
1717
runs-on: ${{ matrix.os }}-latest
1818
strategy:
1919
fail-fast: false
2020
matrix:
2121
os: [Ubuntu, MacOS, Windows]
2222
python-version: ['3.10', '3.11', '3.12', '3.13']
23-
rust-codec: ['disabled'] # TODO add 'enabled' when the Rust codec is ready
2423
steps:
2524
- uses: actions/checkout@v4
2625

27-
- name: Set up Python ${{ matrix.python-version }}
28-
uses: actions/setup-python@v5
26+
- name: Install uv with Python ${{ matrix.python-version }}
27+
uses: astral-sh/setup-uv@v5
2928
with:
29+
version: "0.7.5"
3030
python-version: ${{ matrix.python-version }}
3131

32-
- name: Get full Python version
33-
id: full-python-version
34-
shell: bash
35-
run: echo "version=$(python -c "import sys; print('-'.join(str(v) for v in sys.version_info))")" >> "$GITHUB_OUTPUT"
36-
37-
- name: Install poetry
38-
shell: bash
39-
run: |
40-
python -m pip install poetry
41-
echo "$HOME/.poetry/bin" >> $GITHUB_PATH
42-
43-
- name: Configure poetry
44-
shell: bash
45-
run: poetry config virtualenvs.in-project true
46-
47-
- name: Set up cache
48-
uses: actions/cache@v4
49-
id: cache
50-
with:
51-
path: .venv
52-
key: venv-${{ runner.os }}-${{ steps.full-python-version.outputs.version }}-rust-codec-${{ matrix.rust-codec }}-${{ hashFiles('**/poetry.lock') }}
32+
- name: Install betterproto2
33+
working-directory: ./betterproto2
34+
run: uv sync --locked --all-extras --all-groups
5335

54-
- name: Ensure cache is healthy
55-
if: steps.cache.outputs.cache-hit == 'true'
56-
shell: bash
57-
run: poetry run pip --version >/dev/null 2>&1 || rm -rf .venv
36+
- name: Install betterproto2_compiler
37+
working-directory: ./betterproto2_compiler
38+
run: uv sync --locked --all-extras --all-groups
5839

59-
- name: Install dependencies
60-
if: matrix.rust-codec == 'disabled'
40+
- name: Test betterproto2_compiler
41+
working-directory: ./betterproto2_compiler
6142
shell: bash
62-
run: poetry install -E all
43+
run: uv run poe test
6344

64-
- name: Install dependencies with the Rust codec
65-
if: matrix.rust-codec == 'enabled'
45+
- name: Compile test files
46+
working-directory: ./betterproto2_compiler
6647
shell: bash
67-
run: poetry install -E all -E rust-codec
48+
run: uv run poe generate
6849

69-
- name: Download compiled test files
50+
- name: Move compiled files to betterproto2
7051
shell: bash
71-
run: poetry run poe pull-compiled-tests
52+
run: mv betterproto2_compiler/tests/output_betterproto betterproto2_compiler/tests/output_betterproto_pydantic betterproto2_compiler/tests/output_reference betterproto2/tests
7253

7354
- name: Execute test suite
55+
working-directory: ./betterproto2
7456
shell: bash
75-
run: poetry run poe test
57+
run: uv run poe test

‎.github/workflows/code-quality.yml‎

Lines changed: 10 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -12,42 +12,24 @@ permissions:
1212
contents: read
1313

1414
jobs:
15-
check-formatting:
15+
code-quality:
1616
name: Check code/doc formatting
1717
runs-on: ubuntu-latest
1818
steps:
1919
- uses: actions/checkout@v4
20-
- uses: actions/setup-python@v5
2120
- uses: pre-commit/action@v3.0.1
2221

23-
- name: Install poetry
24-
shell: bash
25-
run: |
26-
python -m pip install poetry
27-
echo "$HOME/.poetry/bin" >> $GITHUB_PATH
28-
29-
- name: Configure poetry
30-
shell: bash
31-
run: poetry config virtualenvs.in-project true
32-
33-
- name: Set up cache
34-
uses: actions/cache@v3
35-
id: cache
22+
- name: Install uv
23+
uses: astral-sh/setup-uv@v5
3624
with:
37-
path: .venv
38-
key: venv-quality-${{ hashFiles('**/poetry.lock') }}
25+
version: "0.7.5"
26+
python-version: "3.10"
3927

40-
- name: Ensure cache is healthy
41-
if: steps.cache.outputs.cache-hit == 'true'
42-
shell: bash
43-
run: poetry run pip --version >/dev/null 2>&1 || rm -rf .venv
44-
45-
- name: Install dependencies
46-
shell: bash
47-
run: poetry install -E all
48-
49-
- uses: pre-commit/action@v3.0.1
28+
- name: Install the project
29+
working-directory: ./betterproto2
30+
run: uv sync --locked --all-extras --all-groups
5031

5132
- name: Pyright
33+
working-directory: ./betterproto2
5234
shell: bash
53-
run: poetry run poe typecheck
35+
run: uv run poe typecheck

‎.github/workflows/documentation.yml‎

Lines changed: 10 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -16,35 +16,16 @@ jobs:
1616
git config user.name github-actions[bot]
1717
git config user.email 41898282+github-actions[bot]@users.noreply.github.com
1818
19-
- name: Set up Python 3.12
20-
uses: actions/setup-python@v5
19+
- name: Install uv
20+
uses: astral-sh/setup-uv@v5
2121
with:
22-
python-version: 3.12
22+
version: "0.7.5"
23+
python-version: "3.10"
2324

24-
- name: Install poetry
25-
shell: bash
26-
run: |
27-
python -m pip install poetry
28-
echo "$HOME/.poetry/bin" >> $GITHUB_PATH
29-
30-
- name: Configure poetry
31-
shell: bash
32-
run: poetry config virtualenvs.in-project true
33-
34-
- name: Set up cache
35-
uses: actions/cache@v3
36-
id: cache
37-
with:
38-
path: .venv
39-
key: venv-documentation-${{ hashFiles('**/poetry.lock') }}
40-
41-
- name: Ensure cache is healthy
42-
if: steps.cache.outputs.cache-hit == 'true'
43-
shell: bash
44-
run: poetry run pip --version >/dev/null 2>&1 || rm -rf .venv
45-
46-
- name: Install dependencies
47-
shell: bash
48-
run: poetry install
25+
- name: Install betterproto2
26+
working-directory: ./betterproto2
27+
run: uv sync --locked --all-extras --all-groups
4928

50-
- run: poetry run mkdocs gh-deploy --force
29+
- name: Build and deploy documentation
30+
working-directory: ./betterproto2
31+
run: uv run mkdocs gh-deploy --force

‎.github/workflows/release.yml‎

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,9 @@ on:
88
- '**'
99

1010
jobs:
11-
packaging:
12-
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
13-
name: Distribution
11+
publish_betterproto2:
12+
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/lib-v')
13+
name: Publish betterproto2 to PyPI
1414
runs-on: ubuntu-latest
1515

1616
permissions:
@@ -23,16 +23,17 @@ jobs:
2323
steps:
2424
- uses: actions/checkout@v4
2525

26-
- name: Set up Python 3.10
27-
uses: actions/setup-python@v5
26+
- name: Install uv
27+
uses: astral-sh/setup-uv@v5
2828
with:
29+
version: "0.7.5"
2930
python-version: "3.10"
3031

31-
- name: Install poetry
32-
run: python -m pip install poetry
33-
3432
- name: Build package
35-
run: poetry build
33+
working-directory: ./betterproto2
34+
run: uv build
3635

3736
- name: Publish package distributions to PyPI
3837
uses: pypa/gh-action-pypi-publish@release/v1.12
38+
with:
39+
packages-dir: betterproto2/dist

‎.pre-commit-config.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,6 @@ repos:
66
rev: v0.9.3
77
hooks:
88
- id: ruff-format
9-
args: ["--diff", "src", "tests"]
9+
args: ["--diff", "betterproto2/src", "betterproto2/tests", "betterproto2_compiler/src", "betterproto2_compiler/tests"]
1010
- id: ruff
11-
args: ["src", "tests"]
11+
args: ["betterproto2/src", "betterproto2/tests", "betterproto2_compiler/src", "betterproto2_compiler/tests"]

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
> :warning: `betterproto2` is a fork of the original [`betterproto`](https://github.com/danielgtaylor/python-betterproto) repository. It is a major redesign of the library, allowing to fix several bugs and to support new features.
66
>
7-
> However, it is still in active developement. The documentation is not complete and a few breaking changes still are likely to be done. There is still work to do and the project is still subject to breaking changes.
7+
> However, it is still in active developement. The documentation is not complete, there is still work to do and the project is still subject to breaking changes.
88
99
This project aims to provide an improved experience when using Protobuf / gRPC in a modern Python environment by making use of modern language features and generating readable, understandable, idiomatic Python code. It will not support legacy features or environments (e.g. Protobuf 2). The following are supported:
1010

docs/api.md renamed to betterproto2/docs/api.md

File renamed without changes.

‎betterproto2/docs/development.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Development
2+
3+
This page targets the betterproto maintainers.
4+
5+
## Recompiling the lib proto files
6+
7+
After some updates in the compiler, it might be useful to recompile the standard Google proto files used by the
8+
compiler. As the proto files are distributed with `protoc`, their path might depend on your installation.
9+
10+
```bash
11+
mkdir lib
12+
protoc \
13+
--python_betterproto2_out=lib \
14+
-I /usr/include/ \
15+
/usr/include/google/protobuf/*.proto
16+
```
17+
18+
!!! warning
19+
These proto files are written with the `proto2` syntax, which is not supported by betterproto. For the compiler to
20+
work, you need to manually patch the generated file to mark the field `oneof_index` in `Field` and
21+
`FieldDescriptorProto` optional.
22+
23+
In the compiler, you also need to compile the [plugin.proto](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/plugin.proto)
24+
file in `src/betterproto2_compiler/lib/google.protobug/compiler/__init__.py`.

docs/getting-started.md renamed to betterproto2/docs/getting-started.md

Lines changed: 90 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,111 @@
11
Getting Started
22
===============
33

4-
## Installation
4+
## Compilation
5+
6+
7+
### Install protoc
58

6-
The package `betterproto2` can be simply installed from PyPI using `pip`:
9+
The betterproto2 compiler is a plugin of `protoc`, you first need to [install](https://grpc.io/docs/protoc-installation/) it.
10+
11+
You can also use it from `grpcio-tools`:
712

813
```sh
9-
pip install betterproto2[all]
14+
pip install grpcio-tools
15+
```
16+
17+
### Install `betterproto2_compiler`
18+
19+
It is possible to install `betterproto2_compiler` using pip:
20+
21+
```
22+
pip install betterproto2_compiler
23+
```
24+
25+
### Compile a proto file
26+
27+
Create the following `example.proto` file.
28+
29+
```proto
30+
syntax = "proto3";
31+
32+
package helloworld;
33+
34+
message HelloWorld {
35+
string message = 1;
36+
}
37+
38+
service HelloService {
39+
rpc SayHello (HelloWorld) returns (HelloWorld);
40+
}
41+
```
42+
43+
You should be able to compile it using:
44+
45+
```
46+
mkdir lib
47+
protoc -I . --python_betterproto2_out=lib example.proto
48+
```
49+
50+
If you installed `protoc` with `grpc-tools`, the command will be:
51+
1052
```
53+
mkdir lib
54+
python -m grpc.tools.protoc -I . --python_betterproto2_out=lib example.proto
55+
```
56+
57+
#### Service compilation
58+
59+
##### Clients
60+
61+
By default, for each service, betterproto will generate a synchronous client. Both synchronous and asynchronous clients
62+
are supported.
63+
64+
- Synchronous clients rely on the `grpcio` package. Make sure to enable the `grpcio` extra package when installing
65+
betterproto2 to use them.
66+
- Asynchronous clients rely on the `grpclib` package. Make sure to enable the `grpclib` extra package when installing
67+
betterproto2 to use them.
1168

12-
!!! info
13-
The package is compatible with all Python versions from 3.10 to 3.13.
69+
To choose which clients to generate, use the `client_generation` option of betterproto. It supports the following
70+
values:
1471

72+
- `none`: Clients are not generated.
73+
- `sync`: Only synchronous clients are generated.
74+
- `async`: Only asynchronous clients are generated.
75+
- `sync_async`: Both synchronous and asynchronous clients are generated.
76+
Asynchronous clients are generated with the Async suffix.
77+
- `async_sync`: Both synchronous and asynchronous clients are generated.
78+
Synchronous clients are generated with the Sync suffix.
79+
- `sync_async_no_default`: Both synchronous and asynchronous clients are generated.
80+
Synchronous clients are generated with the Sync suffix, and asynchronous clients are generated with the Async
81+
suffix.
1582

16-
## Compiling proto files
83+
For example, `protoc -I . --python_betterproto2_out=lib example.proto --python_betterproto2_opt=client_generation=async`
84+
will only generate asynchronous clients.
1785

18-
Follow the documentation of [betterproto2 compiler](https://betterproto.github.io/python-betterproto2-compiler/getting-started/) to compile your proto files.
86+
##### Servers
87+
88+
By default, betterproto will not generate server base classes. To enable them, set the `server_generation` option to
89+
`async` with `--python_betterproto2_opt=server_generation=async`.
90+
91+
These base classes will be asynchronous and rely on `grpclib`. To use them, make sure to install `betterproto2` with the
92+
`grpclib` extra package.
93+
94+
95+
## Installation
96+
97+
The package `betterproto2` can be installed from PyPI using `pip`:
98+
99+
```sh
100+
pip install betterproto2[all]
101+
```
19102

20103
!!! warning
21104
Make sure that the proto files were generated with a version of `betterproto2_compiler` that is compatible with your
22105
version of `betterproto2`.
23106

24107
The version `0.x.y` of `betterproto` is compatible with the version `0.a.b` of the compiler if and only if `a=b`.
25108

26-
27109
## Basic usage
28110

29111
If you successfuly compiled the `example.proto` file from the compiler documentation, you should now be able to use it!

docs/index.md renamed to betterproto2/docs/index.md

File renamed without changes.

0 commit comments

Comments
 (0)