Skip to content

Commit c02deea

Browse files
AdrienVannsonAdrienVannson
committed
Add back the compiler
1 parent 3303b73 commit c02deea

139 files changed

Lines changed: 9065 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎betterproto2_compiler/docs/development.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Development
2+
3+
## Recompiling the lib proto files
4+
5+
After some updates in the compiler, it might be useful to recompile the standard Google proto files used by the
6+
compiler. As the proto files are distributed with `protoc`, their path might depend on your installation.
7+
8+
```bash
9+
mkdir lib
10+
protoc \
11+
--python_betterproto2_out=lib \
12+
-I /usr/include/ \
13+
/usr/include/google/protobuf/*.proto
14+
```
15+
16+
!!! warning
17+
These proto files are written with the `proto2` syntax, which is not supported by betterproto. For the compiler to
18+
work, you need to manually patch the generated file to mark the field `oneof_index` in `Field` and
19+
`FieldDescriptorProto` optional.
20+
21+
In the compiler, you also need to compile the [plugin.proto](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/plugin.proto)
22+
file in `src/betterproto2_compiler/lib/google.protobug/compiler/__init__.py`.

‎betterproto2_compiler/docs/getting-started.md‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Getting started
2+
3+
## Install protoc
4+
5+
The betterproto2 compiler is a plugin of `protoc`, you first need to [install](https://grpc.io/docs/protoc-installation/) it.
6+
7+
You can also use it from `grpcio-tools`:
8+
9+
```sh
10+
pip install grpcio-tools
11+
```
12+
13+
## Install `betterproto2_compiler`
14+
15+
It is possible to install `betterproto2_compiler` using pip:
16+
17+
```
18+
pip install betterproto2_compiler
19+
```
20+
21+
## Compile a proto file
22+
23+
Create the following `example.proto` file.
24+
25+
```proto
26+
syntax = "proto3";
27+
28+
package helloworld;
29+
30+
message HelloWorld {
31+
string message = 1;
32+
}
33+
34+
service HelloService {
35+
rpc SayHello (HelloWorld) returns (HelloWorld);
36+
}
37+
```
38+
39+
You should now be able to compile it using:
40+
41+
```
42+
mkdir lib
43+
protoc -I . --python_betterproto2_out=lib example.proto
44+
```
45+
46+
If you installed `protoc` with `grpc-tools`, the command will be:
47+
48+
```
49+
mkdir lib
50+
python -m grpc.tools.protoc -I . --python_betterproto2_out=lib example.proto
51+
```
52+
53+
### Service compilation
54+
55+
#### Clients
56+
57+
By default, for each service, betterproto will generate a synchronous client. Both synchronous and asynchronous clients
58+
are supported.
59+
60+
- Synchronous clients rely on the `grpcio` package. Make sure to enable the `grpcio` extra package when installing
61+
betterproto2 to use them.
62+
- Asynchronous clients rely on the `grpclib` package. Make sure to enable the `grpclib` extra package when installing
63+
betterproto2 to use them.
64+
65+
To choose which clients to generate, use the `client_generation` option of betterproto. It supports the following
66+
values:
67+
68+
- `none`: Clients are not generated.
69+
- `sync`: Only synchronous clients are generated.
70+
- `async`: Only asynchronous clients are generated.
71+
- `sync_async`: Both synchronous and asynchronous clients are generated.
72+
Asynchronous clients are generated with the Async suffix.
73+
- `async_sync`: Both synchronous and asynchronous clients are generated.
74+
Synchronous clients are generated with the Sync suffix.
75+
- `sync_async_no_default`: Both synchronous and asynchronous clients are generated.
76+
Synchronous clients are generated with the Sync suffix, and asynchronous clients are generated with the Async
77+
suffix.
78+
79+
For example, `protoc -I . --python_betterproto2_out=lib example.proto --python_betterproto2_opt=client_generation=async`
80+
will only generate asynchronous clients.
81+
82+
#### Servers
83+
84+
By default, betterproto will not generate server base classes. To enable them, set the `server_generation` option to
85+
`async` with `--python_betterproto2_opt=server_generation=async`.
86+
87+
These base classes will be asynchronous and rely on `grpclib`. To use them, make sure to install `betterproto2` with the
88+
`grpclib` extra package.

‎betterproto2_compiler/docs/index.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Home
2+
====
3+
4+
Welcome to `betterproto2_compiler`'s documentation! This site will guide you through the steps needed to install and use the betterproto2 compiler.

‎betterproto2_compiler/pyproject.toml‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
[project]
2+
name = "betterproto2_compiler"
3+
version = "0.5.1"
4+
description = "Compiler for betterproto2"
5+
authors = [
6+
{ name = "Adrien Vannson", email = "adrien.vannson@protonmail.com" },
7+
{ name = "Daniel G. Taylor", email = "danielgtaylor@gmail.com" },
8+
]
9+
readme = "README.md"
10+
keywords = ["protobuf", "gRPC", "compiler"]
11+
license = "MIT"
12+
requires-python = ">=3.10,<4.0"
13+
dynamic = ["dependencies"]
14+
15+
[project.urls]
16+
Documentation = "https://betterproto.github.io/python-betterproto2-compiler/"
17+
Repository = "https://github.com/betterproto/python-betterproto2-compiler"
18+
19+
[project.scripts]
20+
protoc-gen-python_betterproto2 = "betterproto2_compiler.plugin:main"
21+
22+
[tool.poetry]
23+
packages = [
24+
{ include = "betterproto2_compiler", from = "src" }
25+
]
26+
27+
[tool.poetry.dependencies]
28+
python = "^3.10"
29+
betterproto2 = { version = "^0.5.1", extras = ["grpclib"] }
30+
# betterproto2 = { git="https://github.com/betterproto/python-betterproto2" }
31+
# betterproto2 = { path = "../python-betterproto2", extras = ["grpclib"] }
32+
# The Ruff version is pinned. To update it, also update it in .pre-commit-config.yaml
33+
ruff = "~0.9.3"
34+
jinja2 = ">=3.0.3"
35+
typing-extensions = "^4.7.1"
36+
strenum = [
37+
{version = "^0.4.15", python = "=3.10"},
38+
]
39+
40+
[tool.poetry.group.dev.dependencies]
41+
pre-commit = "^2.17.0"
42+
grpcio-tools = "^1.54.2"
43+
mkdocs-material = {version = "^9.5.49"}
44+
mkdocstrings = {version = "^0.27.0", extras = ["python"]}
45+
poethepoet = "^0.32.2"
46+
pyright = "^1.1.391"
47+
ipykernel = "^6.29.5"
48+
49+
[tool.poetry.group.test.dependencies]
50+
pytest = "^8.3.4"
51+
protobuf = "^5.29.3"
52+
53+
[tool.ruff]
54+
extend-exclude = ["tests/output_*", "src/betterproto2_compiler/lib"]
55+
target-version = "py310"
56+
line-length = 120
57+
58+
[tool.ruff.lint]
59+
select = [
60+
"F401", # Unused imports
61+
"F841", # Unused local variables
62+
"F821", # Undefined names
63+
"E501", # Line length violations
64+
65+
"SIM101", # Simplify unnecessary if-else blocks
66+
"SIM102", # Simplify return or yield statements
67+
"SIM103", # Simplify list/set/dict comprehensions
68+
69+
"UP",
70+
71+
"I",
72+
]
73+
74+
75+
[tool.ruff.lint.isort]
76+
combine-as-imports = true
77+
78+
# Dev workflow tasks
79+
80+
[tool.poe.tasks.test]
81+
cmd = "pytest"
82+
help = "Run tests"
83+
84+
[tool.poe.tasks.generate]
85+
sequence = ["_generate_tests", "_generate_tests_lib"]
86+
help = "Generate test cases"
87+
88+
[tool.poe.tasks._generate_tests]
89+
script = "tests.generate:main"
90+
91+
[tool.poe.tasks._generate_tests_lib]
92+
shell = """
93+
python -m grpc.tools.protoc \
94+
--python_betterproto2_out=tests/output_betterproto \
95+
google/protobuf/any.proto \
96+
google/protobuf/api.proto \
97+
google/protobuf/duration.proto \
98+
google/protobuf/empty.proto \
99+
google/protobuf/field_mask.proto \
100+
google/protobuf/source_context.proto \
101+
google/protobuf/struct.proto \
102+
google/protobuf/timestamp.proto \
103+
google/protobuf/type.proto \
104+
google/protobuf/wrappers.proto
105+
106+
python -m grpc.tools.protoc \
107+
--python_betterproto2_out=tests/output_betterproto_pydantic \
108+
--python_betterproto2_opt=pydantic_dataclasses \
109+
google/protobuf/any.proto \
110+
google/protobuf/api.proto \
111+
google/protobuf/duration.proto \
112+
google/protobuf/empty.proto \
113+
google/protobuf/field_mask.proto \
114+
google/protobuf/source_context.proto \
115+
google/protobuf/struct.proto \
116+
google/protobuf/timestamp.proto \
117+
google/protobuf/type.proto \
118+
google/protobuf/wrappers.proto
119+
"""
120+
121+
[tool.poe.tasks.typecheck]
122+
cmd = "pyright src"
123+
help = "Typecheck the code with Pyright"
124+
125+
[tool.poe.tasks.format]
126+
sequence = ["_format", "_sort-imports"]
127+
help = "Format the source code, and sort the imports"
128+
129+
[tool.poe.tasks.check]
130+
sequence = ["_check-format", "_check-ruff-lint"]
131+
help = "Check that the source code is formatted and the code passes the linter"
132+
133+
[tool.poe.tasks._format]
134+
cmd = "ruff format src tests"
135+
help = "Format the source code without sorting the imports"
136+
137+
[tool.poe.tasks._sort-imports]
138+
cmd = "ruff check --select I --fix src tests"
139+
help = "Sort the imports"
140+
141+
[tool.poe.tasks._check-format]
142+
cmd = "ruff format --diff src tests"
143+
help = "Check that the source code is formatted"
144+
145+
[tool.poe.tasks._check-ruff-lint]
146+
cmd = "ruff check src tests"
147+
help = "Check the code with the Ruff linter"
148+
149+
[tool.poe.tasks.serve-docs]
150+
cmd = "mkdocs serve"
151+
help = "Serve the documentation locally"
152+
153+
[build-system]
154+
requires = ["poetry-core>=2.0.0,<3"]
155+
build-backend = "poetry.core.masonry.api"
156+
157+
# python -m grpc.tools.protoc \
158+
# --python_betterproto2_out=src/lib2 \
159+
# google/protobuf/any.proto \
160+
# google/protobuf/api.proto \
161+
# google/protobuf/duration.proto \
162+
# google/protobuf/empty.proto \
163+
# google/protobuf/field_mask.proto \
164+
# google/protobuf/source_context.proto \
165+
# google/protobuf/struct.proto \
166+
# google/protobuf/timestamp.proto \
167+
# google/protobuf/type.proto \
168+
# google/protobuf/wrappers.proto \
169+
# google/protobuf/compiler/plugin.proto

‎betterproto2_compiler/pytest.ini‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
[pytest]
2+
python_files = test_*.py
3+
python_classes =
4+
norecursedirs = **/output_*
5+
addopts = -p no:warnings

‎betterproto2_compiler/src/betterproto2_compiler/__init__.py‎

Whitespace-only changes.

0 commit comments

Comments
 (0)