Skip to content

Commit 8d157a0

Browse files
Mazyodclaude
andcommitted
feat: add ty backend support for Astral's Python type checker
- Add lsp_types/ty/ module with TyBackend, config schema, and KNOWN_LIMITATIONS.md - Support nested TOML config sections (environment, src, rules, etc.) - Implement recursive snake_case to kebab-case key conversion for ty.toml - Add parametrized tests for ty backend in test_session.py and test_pool.py - Add ty-specific tests for config serialization and extra_paths - Mark tests requiring virtual documents as xfail for ty (requires files on disk) - Add KNOWN_LIMITATIONS.md for Pyrefly backend - Add INTEGRATION_NOTES.md documenting frictions and enhancement opportunities - Update README.md and CLAUDE.md with ty documentation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent f898cfa commit 8d157a0

10 files changed

Lines changed: 730 additions & 15 deletions

File tree

CLAUDE.md

Lines changed: 16 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,14 @@ This is a minimal-dependency Python library providing typed LSP (Language Server
8383
- **Key Design**: Uses consolidated `Session` class with `PyreflyBackend` for specialization
8484
- **Config Flexibility**: Supports arbitrary configuration fields via TOML serialization (using `tomli-w`)
8585

86+
**ty Integration (`lsp_types/ty/`)**
87+
- `backend.py`: `TyBackend` implementation for ty LSP server (Astral's Rust-based type checker)
88+
- `config_schema.py`: ty configuration types with nested sections (environment, src, rules, etc.)
89+
- **Key Design**: Uses consolidated `Session` class with `TyBackend` for specialization
90+
- **Config Format**: TOML (`ty.toml`) with nested sections and kebab-case keys
91+
- **Known Limitation**: ty requires files to exist on disk (virtual documents not fully supported)
92+
- **Documentation**: See `KNOWN_LIMITATIONS.md` in the ty package for details
93+
8694
### Type Generation Pipeline
8795

8896
**Schema Sources:**
@@ -97,33 +105,34 @@ This is a minimal-dependency Python library providing typed LSP (Language Server
97105

98106
### Testing Strategy
99107

100-
**Tests are parametrized to run against multiple backends (Pyright and Pyrefly).**
108+
**Tests are parametrized to run against multiple backends (Pyright, Pyrefly, and ty).**
101109

102110
**Process Pool Tests (`tests/test_pool.py`)**
103111
- Direct `LSPProcessPool` testing with generic interface
104-
- Parametrized fixtures for testing both Pyright and Pyrefly backends
112+
- Parametrized fixtures for testing Pyright, Pyrefly, and ty backends
105113
- Comprehensive pool behavior testing (creation, recycling, limits, cleanup)
106114
- Performance benchmarks comparing pooled vs non-pooled sessions
107115
- Concurrent usage scenarios and idle process management
108116

109117
**Session Tests (`tests/test_session.py`)**
110118
- Core consolidated Session class functionality
111-
- Parametrized fixtures for testing both Pyright and Pyrefly backends
119+
- Parametrized fixtures for testing Pyright, Pyrefly, and ty backends
112120
- Integration testing with actual language servers (diagnostics, hover, completion)
113121
- Dynamic environment testing with temporary directories
114122
- Backend-agnostic tests that validate common LSP operations
123+
- Backend-specific tests for unique configuration options (e.g., ty's nested config, Pyrefly's search_path)
115124

116125
### Dependencies
117126

118127
**Runtime:**
119-
- `tomli-w>=1.0.0` - TOML writing support for Pyrefly configuration serialization
128+
- `tomli-w>=1.0.0` - TOML writing support for Pyrefly and ty configuration serialization
120129

121130
**Development:** uv-managed dependencies in `pyproject.toml`
122131
- `pytest` with async support for testing
123132
- `datamodel-code-generator` for type generation
124133
- `httpx` for schema downloading
125134

126-
**Note:** Previously a zero-dependency library. Added `tomli-w` to support arbitrary configuration fields in Pyrefly backend.
135+
**Note:** Previously a zero-dependency library. Added `tomli-w` to support TOML configuration for Pyrefly and ty backends.
127136

128137
### Examples
129138

@@ -135,9 +144,10 @@ The `examples/` directory contains demo scripts showing library usage:
135144

136145
- Always prefix test commands with `uv run`
137146
- **Before committing**: Run tests (`uv run pytest`), type checking (`uvx pyright`), and linting (`uvx ruff check .`) - CI will fail if any have errors
138-
- Pool tests require `pyright-langserver` and/or `pyrefly` binaries available in PATH
147+
- Pool tests require `pyright-langserver`, `pyrefly`, and/or `ty` binaries available in PATH
139148
- Type generation requires Python 3.12+ for modern TypedDict features
140149
- Generated types should not be manually edited - regenerate from schemas
150+
- Each backend has a `KNOWN_LIMITATIONS.md` file documenting backend-specific behaviors
141151

142152
### Architecture Design Patterns
143153

INTEGRATION_NOTES.md

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# LSP Backend Integration Notes
2+
3+
This document captures frictions and enhancement opportunities discovered while integrating new LSP backends into lsp-python-types.
4+
5+
## ty Backend Integration (January 2026)
6+
7+
### Frictions Encountered
8+
9+
#### 1. Virtual Document Support
10+
11+
**Issue**: ty requires files to exist on disk before it can provide diagnostics, completion, and other features. Pyright and Pyrefly work with "virtual documents" opened via `didOpen` without requiring the file to exist on disk.
12+
13+
**Impact**: Tests that don't write files to disk fail for ty. The following parametrized tests required `xfail` markers for ty:
14+
- `test_session_diagnostics`
15+
- `test_session_rename`
16+
- `test_session_completion`
17+
- `test_session_recycling_with_diagnostics`
18+
- `test_session_warmup_on_recycle` (in test_pool.py)
19+
20+
**Workaround**: ty-specific tests must write files to disk using `tmp_path`:
21+
```python
22+
(tmp_path / "new.py").write_text(code)
23+
session = await Session.create(backend, base_path=tmp_path, initial_code=code)
24+
```
25+
26+
**Potential Enhancement**: Consider adding a `requires_file_on_disk` property to the `LSPBackend` protocol, allowing the Session class to automatically write files when needed.
27+
28+
#### 2. `workspace/didChangeConfiguration` Not Supported
29+
30+
**Issue**: ty logs `Received notification workspace/didChangeConfiguration which does not have a handler.` The Session class sends this notification after initialization to apply workspace settings.
31+
32+
**Impact**: Runtime configuration changes via `didChangeConfiguration` don't work with ty. However, configuration written to `ty.toml` is respected.
33+
34+
**Current Behavior**: The notification is sent but ignored by ty. No functional impact since config file is written first.
35+
36+
**Potential Enhancement**: Add an optional `supports_did_change_configuration` flag to backends so the Session can skip sending this notification when unsupported.
37+
38+
#### 3. Nested Configuration Structure
39+
40+
**Issue**: ty uses nested TOML sections (`[environment]`, `[src]`, `[rules]`) unlike Pyrefly's flat structure. This required implementing recursive key conversion.
41+
42+
**Solution**: Created `_convert_keys_to_kebab()` function in `lsp_types/ty/backend.py`:
43+
```python
44+
def _convert_keys_to_kebab(obj: t.Mapping[str, t.Any]) -> dict[str, t.Any]:
45+
"""Recursively convert dict keys from snake_case to kebab-case."""
46+
result: dict[str, t.Any] = {}
47+
for key, value in obj.items():
48+
kebab_key = key.replace("_", "-")
49+
if isinstance(value, dict):
50+
result[kebab_key] = _convert_keys_to_kebab(value)
51+
elif isinstance(value, list):
52+
result[kebab_key] = [
53+
_convert_keys_to_kebab(v) if isinstance(v, dict) else v
54+
for v in value
55+
]
56+
else:
57+
result[kebab_key] = value
58+
return result
59+
```
60+
61+
**Potential Enhancement**: Extract this utility to a shared module (`lsp_types/utils.py`) since Pyrefly also uses TOML with kebab-case keys (though currently with flat structure).
62+
63+
#### 4. Hover Information Format Differences
64+
65+
**Issue**: ty's hover response shows just the type (`str`) rather than `variable_name: type` format used by Pyright and Pyrefly.
66+
67+
**Impact**: Test assertions checking for variable names in hover text fail for ty.
68+
69+
**Workaround**: Added backend-specific assertion in `test_session_hover`:
70+
```python
71+
if backend_name != "ty":
72+
assert "result" in hover_text
73+
assert "str" in hover_text
74+
```
75+
76+
#### 5. No CLI Flags for LSP Server
77+
78+
**Issue**: Unlike Pyrefly which accepts `--verbose`, `--threads`, and `--indexing-mode` CLI flags, ty's `server` command accepts no configuration flags.
79+
80+
**Impact**: All configuration must be done via `ty.toml`. This is actually simpler than Pyrefly's hybrid approach.
81+
82+
**Solution**: `create_process_launch_info()` simply returns `["ty", "server"]` without any conditional flag building.
83+
84+
---
85+
86+
## Enhancement Opportunities
87+
88+
### 1. Shared TOML Key Conversion Utility
89+
90+
Both Pyrefly and ty use TOML with kebab-case keys but Python code uses snake_case. Consider creating:
91+
92+
```python
93+
# lsp_types/utils.py
94+
def snake_to_kebab_recursive(obj: Mapping[str, Any]) -> dict[str, Any]:
95+
"""Recursively convert dict keys from snake_case to kebab-case."""
96+
# ... implementation
97+
```
98+
99+
Then refactor both backends to use this shared utility.
100+
101+
### 2. Backend Capability Flags
102+
103+
Add optional flags to the `LSPBackend` protocol for:
104+
- `requires_file_on_disk: bool` - Whether backend needs files to exist on disk
105+
- `supports_did_change_configuration: bool` - Whether backend handles `workspace/didChangeConfiguration`
106+
107+
This would allow the Session class to adapt its behavior automatically.
108+
109+
### 3. Common LSP Capabilities Base
110+
111+
Create a helper function for shared capabilities:
112+
113+
```python
114+
def get_base_python_capabilities() -> types.ClientCapabilities:
115+
"""Common LSP capabilities for Python type checkers."""
116+
return {
117+
"textDocument": {
118+
"publishDiagnostics": {...},
119+
"hover": {...},
120+
"signatureHelp": {},
121+
}
122+
}
123+
```
124+
125+
Backends could extend this base instead of duplicating the boilerplate.
126+
127+
### 4. Backend Registry Pattern
128+
129+
For easier discovery and testing:
130+
131+
```python
132+
_BACKENDS: dict[str, type[LSPBackend]] = {}
133+
134+
def register_backend(name: str):
135+
def decorator(cls):
136+
_BACKENDS[name] = cls
137+
return cls
138+
return decorator
139+
140+
@register_backend("ty")
141+
class TyBackend(LSPBackend):
142+
...
143+
```
144+
145+
---
146+
147+
## Summary
148+
149+
The ty backend integration revealed that different LSP servers have varying requirements around file handling and configuration. The current abstraction works but could benefit from:
150+
151+
1. Optional capability flags on backends
152+
2. Shared utilities for common patterns (TOML conversion, base capabilities)
153+
3. Better documentation of backend-specific behaviors
154+
155+
The core `LSPBackend` protocol and `Session` class work well across all three backends (Pyright, Pyrefly, ty) with minimal backend-specific handling needed in tests.

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ The following LSPs are available out of the box:
6262

6363
- [Pyright](https://github.com/microsoft/pyright)
6464
- [Pyrefly](https://github.com/facebook/pyrefly)
65+
- [ty](https://github.com/astral-sh/ty) - Astral's fast Python type checker
6566

6667
### Pyright Example
6768

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Pyrefly Backend - Known Limitations
2+
3+
This document describes known limitations and behavioral differences when using the Pyrefly backend compared to other LSP backends (Pyright, ty).
4+
5+
## 1. Rename Operations Disabled for External Files
6+
7+
**Limitation**: Pyrefly detects files as "external" and disables rename edit functionality.
8+
9+
**Behavior**: Calling `get_rename_edits()` returns `None` or empty edits for files that Pyrefly considers external to the project.
10+
11+
**Impact**: Symbol renaming may not work in certain project configurations.
12+
13+
**Status**: Marked as xfail in tests pending investigation.
14+
15+
## 2. Completion Item Resolution Not Supported
16+
17+
**Limitation**: Pyrefly does not support the `completionItem/resolve` LSP request.
18+
19+
**Behavior**: Calling `resolve_completion()` may return the item unchanged without additional details.
20+
21+
**Impact**: Completion items won't have extended documentation or additional metadata that resolution typically provides.
22+
23+
## 3. Configuration Key Format
24+
25+
**Note**: Pyrefly uses TOML configuration (`pyrefly.toml`) with kebab-case keys (e.g., `python-version`, `search-path`). The backend automatically converts snake_case Python keys to kebab-case when writing the config file.
26+
27+
---
28+
29+
## Version Information
30+
31+
These limitations were documented based on Pyrefly version 0.32.0+. Future versions may address some of these limitations.

lsp_types/ty/KNOWN_LIMITATIONS.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# ty Backend - Known Limitations
2+
3+
This document describes known limitations and behavioral differences when using the ty backend compared to other LSP backends (Pyright, Pyrefly).
4+
5+
## 1. Files Must Exist on Disk
6+
7+
**Limitation**: ty requires Python files to exist on disk before it can provide diagnostics, completion, rename, and other analysis features.
8+
9+
**Behavior**: Opening a "virtual document" via `didOpen` without a corresponding file on disk results in:
10+
- Empty diagnostics (no errors reported even for invalid code)
11+
- Empty or limited completion results
12+
- Rename operations may fail
13+
14+
**Workaround**: Always write files to disk before creating a session:
15+
```python
16+
from pathlib import Path
17+
18+
base_path = Path("/tmp/my_project")
19+
base_path.mkdir(exist_ok=True)
20+
21+
code = "x: int = 'not an int'"
22+
(base_path / "new.py").write_text(code)
23+
24+
session = await Session.create(
25+
TyBackend(),
26+
base_path=base_path,
27+
initial_code=code,
28+
)
29+
diagnostics = await session.get_diagnostics() # Now returns errors
30+
```
31+
32+
## 2. `workspace/didChangeConfiguration` Not Supported
33+
34+
**Limitation**: ty does not handle the `workspace/didChangeConfiguration` notification.
35+
36+
**Behavior**: ty logs a warning:
37+
```
38+
WARN Received notification workspace/didChangeConfiguration which does not have a handler.
39+
```
40+
41+
**Impact**: Runtime configuration changes via LSP notifications are ignored. However, configuration written to `ty.toml` before session creation is respected.
42+
43+
**Workaround**: Ensure all configuration is set in `ty.toml` via the `options` parameter when creating a session. If configuration needs to change, create a new session.
44+
45+
## 3. Hover Format Differs
46+
47+
**Limitation**: ty's hover information shows only the type, not the variable name.
48+
49+
**Behavior**:
50+
- Pyright/Pyrefly hover: `result: str` or `(variable) result: str`
51+
- ty hover: `str`
52+
53+
**Impact**: Code that parses hover text expecting variable names will not find them with ty.
54+
55+
## 4. No CLI Configuration Flags
56+
57+
**Limitation**: The `ty server` command accepts no configuration flags.
58+
59+
**Behavior**: Unlike Pyrefly which supports `--verbose`, `--threads`, etc., ty's LSP server is configured entirely via `ty.toml`.
60+
61+
**Impact**: No impact on functionality - all configuration works via the config file.
62+
63+
## 5. Workspace Folders Warning
64+
65+
**Limitation**: ty expects `workspaceFolders` in the initialization parameters.
66+
67+
**Behavior**: ty logs a warning when workspaceFolders is not provided:
68+
```
69+
WARN No workspace(s) were provided during initialization. Using the current working directory from the fallback system as a default workspace
70+
```
71+
72+
**Impact**: ty falls back to using the working directory. This typically works correctly but may affect multi-root workspace scenarios.
73+
74+
## 6. File Watching Not Supported by Client
75+
76+
**Limitation**: The current LSP client implementation doesn't support file watching.
77+
78+
**Behavior**: ty logs:
79+
```
80+
WARN Your LSP client doesn't support file watching: You may see stale results when files change outside the editor
81+
```
82+
83+
**Impact**: If files are modified outside the LSP session (e.g., by external tools), ty may not detect the changes until the session is recreated.
84+
85+
---
86+
87+
## Version Information
88+
89+
These limitations were documented based on ty version 0.0.11 (January 2026). Future versions may address some of these limitations.

lsp_types/ty/__init__.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
from .backend import TyBackend
2+
from .config_schema import Model as TyConfig
3+
4+
__all__ = ["TyBackend", "TyConfig"]

0 commit comments

Comments
 (0)