diff --git a/CHANGELOG.md b/CHANGELOG.md index 0400244..2ec57a2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -## [4.0.1] - 2026-10-19 +### Changed +- Improved CLI error message when `libclang` is unavailable: now lists platform + package manager commands (including Alpine Linux `apk add clang-dev`), details + the `autopxd2[treesitter]` alternative, and explicitly summarizes tree-sitter's + limitations (parses single file only, no `#include` recursion, no `#define` macros). + +### Documentation +- Added Alpine Linux (`apk add clang-dev`) to system package manager installation + instructions in `README.md` and `docs/getting-started/installation.md`. +- Added a dedicated "Headless CI & Build Containers (cibuildwheel, Docker)" guide + to `README.md` and `docs/getting-started/installation.md` for projects using + autopxd2 in automated container builds. +- Expanded the documentation of the `tree-sitter` backend's limitations and + trade-offs versus `libclang` in `README.md`, `installation.md`, and + `docs/user-guide/backends.md`. ### Changed - Updated test snapshot fixtures (`issue_039_opaque_struct.test`, `keywords.test`, `array_dimensions.test`, `simple_c.expected.pxd`, and `json_lib.expected.pxd`) to reflect HeaderKit 0.46.2 struct/variable ordering, `ctypedef struct` opaque struct typedef semantics, and topological dependency sort order. diff --git a/README.md b/README.md index 10ccf04..5c63b5b 100644 --- a/README.md +++ b/README.md @@ -69,6 +69,7 @@ You only need the system `libclang` shared library (`libclang.so`, `libclang.dyl If `libclang` is not already installed on your system, install it via your system package manager: - **macOS**: `brew install llvm` - **Ubuntu/Debian**: `sudo apt-get install libclang-dev` +- **Alpine Linux (musl)**: `apk add clang-dev` - **Fedora/RHEL**: `sudo dnf install clang-devel` - **Windows**: `winget install LLVM.LLVM` @@ -93,12 +94,37 @@ Then specify `--backend tree-sitter`: autopxd --backend tree-sitter myheader.h myheader.pxd ``` -The tree-sitter backend parses the given file only. Compared to libclang it does not: - -- follow `#include` directives, so declarations from included headers are absent -- extract `#define` macros as constants -- detect system include paths -- accept `--std` or `--clang-arg` (both are rejected with an error) +> **Limitations:** The tree-sitter backend is more limited than `libclang`. It operates purely on the syntax tree of the given file. Compared to libclang, it: +> - **Does not follow `#include` directives**: Declarations from included headers are absent. +> - **Does not extract `#define` macros**: Macro constants and macro functions are not emitted. +> - **Does not detect system include paths**: Local/system header lookups are skipped. +> - **Rejects compiler driver options**: Options like `--std` or `--clang-arg` are rejected with an error. + +Use `tree-sitter` for self-contained headers where installing system LLVM is not feasible. For umbrella headers and full macro extraction, use `libclang`. + +### Headless CI & Build Containers (cibuildwheel, Docker) + +In minimal Docker containers or CI build environments (such as Alpine/`musllinux`, `manylinux`, or slim Debian images) where `libclang` is not present, you can: + +1. **Install `libclang` in the container:** + - **Alpine / musllinux**: `apk add clang-dev` + - **Debian / Ubuntu**: `apt-get install -y libclang-dev` + - **RHEL / manylinux**: `dnf install -y clang-devel` + + *(For example, in `cibuildwheel` config in `pyproject.toml`:)* + ```toml + [[tool.cibuildwheel.overrides]] + select = ["*-musllinux_*"] + before-all = "apk add clang-dev" + ``` + +2. **Or use the Tree-sitter backend (no system packages required):** + Declare `autopxd2[treesitter]` in your build dependencies: + ```toml + [build-system] + requires = ["meson-python", "autopxd2[treesitter]"] + ``` + and pass `--backend tree-sitter` in your build script (e.g. `meson.build`). ## Quick Start diff --git a/autopxd/__init__.py b/autopxd/__init__.py index c6e070f..d935d2d 100644 --- a/autopxd/__init__.py +++ b/autopxd/__init__.py @@ -346,11 +346,26 @@ def _print_backends_json() -> None: DOCKER_DOCS_URL = "https://elijahr.github.io/python-autopxd2/getting-started/docker/" +INSTALL_DOCS_URL = "https://elijahr.github.io/python-autopxd2/getting-started/installation/" LIBCLANG_REQUIRED_ERROR = f"""Error: libclang backend not available. -Install libclang with headerkit: python -m headerkit.install_libclang -Or install LLVM/Clang manually (e.g., apt install libclang-dev, brew install llvm) -Or use Docker: {DOCKER_DOCS_URL} +To use libclang, install it via your system package manager: + - Ubuntu/Debian: sudo apt-get install libclang-dev + - Alpine Linux (musl): apk add clang-dev + - Fedora/RHEL: sudo dnf install clang-devel + - macOS: brew install llvm + - Automated installer: python -m headerkit.install_libclang + +Alternatively, use the tree-sitter backend (no LLVM / system library required): + pip install "autopxd2[treesitter]" + autopxd --backend tree-sitter [] + +Note: The tree-sitter backend is more limited than libclang: it parses the input +file only (does not follow #include directives), does not extract #define macros, +and cannot detect system include paths. + +See installation docs: {INSTALL_DOCS_URL} +Or Docker usage: {DOCKER_DOCS_URL} """ diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index 8929edd..847a1c1 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -35,8 +35,8 @@ autopxd --backend tree-sitter myheader.h myheader.pxd ``` This backend parses only the file you give it: it does not follow `#include` -directives, does not extract `#define` macros, and rejects `--std` and -`--clang-arg`. See [Parser Backends](../user-guide/backends.md). +directives, does not extract `#define` macros, does not detect system include +paths, and rejects `--std` and `--clang-arg`. See [Parser Backends](../user-guide/backends.md). ## System Package Managers @@ -54,6 +54,12 @@ brew install llvm sudo apt-get install libclang-dev llvm ``` +### Alpine Linux / musl + +```bash +apk add clang-dev +``` + ### Fedora / RHEL ```bash @@ -65,6 +71,30 @@ sudo dnf install clang-devel llvm 1. Install LLVM from [releases.llvm.org](https://releases.llvm.org/) or via `winget install LLVM.LLVM` 2. Ensure the LLVM `bin` directory is in your `PATH` +## Headless CI & Build Containers (cibuildwheel, Docker) + +In minimal Docker containers or CI build environments (such as Alpine/`musllinux`, `manylinux`, or slim Debian images) where `libclang` is not present, you can: + +1. **Install `libclang` in the container:** + - **Alpine / musllinux**: `apk add clang-dev` + - **Debian / Ubuntu**: `apt-get install -y libclang-dev` + - **RHEL / manylinux**: `dnf install -y clang-devel` + + *(For example, in `cibuildwheel` config in `pyproject.toml`:)* + ```toml + [[tool.cibuildwheel.overrides]] + select = ["*-musllinux_*"] + before-all = "apk add clang-dev" + ``` + +2. **Or use the Tree-sitter backend (no system packages required):** + Declare `autopxd2[treesitter]` in your build dependencies: + ```toml + [build-system] + requires = ["meson-python", "autopxd2[treesitter]"] + ``` + and pass `--backend tree-sitter` in your build script (e.g. `meson.build`). + ## Development Installation For development, clone the repository and install with development dependencies: diff --git a/docs/user-guide/backends.md b/docs/user-guide/backends.md index 487c493..fbc2337 100644 --- a/docs/user-guide/backends.md +++ b/docs/user-guide/backends.md @@ -23,7 +23,7 @@ Default: libclang | `--std` / `--clang-arg` | ✅ | ❌ (rejected with an error) | Use libclang unless you cannot install LLVM. Use tree-sitter for self-contained -headers in environments where a system `libclang` is unavailable. +headers in environments (such as minimal CI or Docker containers) where a system `libclang` is unavailable. ## libclang @@ -34,9 +34,10 @@ Uses LLVM's clang library for parsing. Provides the same parser used by producti - Full C++ support (classes, templates, namespaces) - Extracts macros as constants (integers, floats, strings, expressions) - Automatic system include path detection +- Follows and recursively expands `#include` directives across translation units - Handles comments and preprocessor directives directly - Better error messages -- Handles complex headers reliably +- Handles complex and umbrella headers reliably **Limitations:** @@ -55,7 +56,7 @@ autopxd myheader.hpp ## tree-sitter Uses [tree-sitter](https://tree-sitter.github.io/) grammars for C and C++. It needs no -system LLVM, so it installs and runs anywhere Python does. +system LLVM, so it installs and runs anywhere Python does using pre-built PyPI wheels. **Install:** @@ -74,11 +75,12 @@ autopxd --backend tree-sitter --cpp myclass.hpp **Limitations:** -- Does not follow `#include` directives. Given `#include "inner.h"`, declarations from - `inner.h` do not appear in the output; only the parsed file's own declarations do. -- Does not extract `#define` macros as constants. -- Does not detect system include paths. -- Rejects `--std` and `--clang-arg`: +The tree-sitter backend is substantially more limited than libclang: + +- **No `#include` recursion**: It parses only the file given to it. Given `#include "inner.h"` or `#include `, declarations from the included headers do not appear in the output; only the parsed file's own declarations do. If the header relies on types defined in included files, those type definitions will not be present. +- **No `#define` macro extraction**: Constants and macro functions defined via `#define` are not extracted into the `.pxd`. +- **No system include paths**: It does not query or search compiler system include directories. +- **Rejects compiler driver options**: Options that configure the compiler frontend (`--std` and `--clang-arg`) are rejected with an explicit error: ```console $ autopxd --backend tree-sitter --std c11 myheader.h @@ -86,6 +88,11 @@ autopxd --backend tree-sitter --cpp myclass.hpp Use --backend libclang, or drop the option. ``` +**When to use tree-sitter:** +- Single, self-contained C or C++ header files. +- Headless CI pipelines, Docker containers, or minimal environments (e.g. Alpine/musllinux) where installing LLVM/Clang is impractical. +- When you do not need macro constants or included header declarations in the generated `.pxd`. + ## System Include Paths The libclang backend automatically detects system include directories by querying the system clang compiler. This means headers like `` and `` work without requiring manual `-I` flags. diff --git a/test/test_cli.py b/test/test_cli.py index a321848..710cdc0 100644 --- a/test/test_cli.py +++ b/test/test_cli.py @@ -30,6 +30,19 @@ def test_unknown_backend_not_available(self) -> None: """Unknown backends should not be available.""" assert is_backend_available("nonexistent") is False + def test_libclang_unavailable_error_message(self, monkeypatch: pytest.MonkeyPatch, simple_header_file: str) -> None: + """When libclang is requested but unavailable, print helpful guidance.""" + import autopxd + + monkeypatch.setattr(autopxd, "is_backend_available", lambda name: False) + runner = CliRunner() + result = runner.invoke(cli, ["--backend", "libclang", simple_header_file]) + assert result.exit_code != 0 + assert "Error: libclang backend not available" in result.output + assert "apk add clang-dev" in result.output + assert "tree-sitter" in result.output + assert 'pip install "autopxd2[treesitter]"' in result.output + class TestBackendInfo: """Tests for backend info retrieval."""