You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Update docs for simplified API, add parallel execution guide, bump to 3.0.0
Documentation updates after API simplification:
- Remove WSGI/ASGI references from buffer.md, getting-started.md
- Remove py_asgi examples from asyncio.md
- Mark subinterp/OWN_GIL as internal in scalability.md, process-bound-envs.md
- Update migration.md with simplified mode descriptions
- Update source comments in py_buffer.erl, py_nif.erl, py_reactor_context.erl
- Remove missing owngil_internals.md from rebar.config
New documentation:
- Add docs/parallel-execution.md explaining parallel pool architecture
Version bump:
- Bump version to 3.0.0 in app.src and getting-started.md
Copy file name to clipboardExpand all lines: docs/buffer.md
+15-28Lines changed: 15 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,20 +1,20 @@
1
1
# Buffer API
2
2
3
-
The Buffer API provides a zero-copy WSGI input buffer for streaming HTTP request bodies from Erlang to Python. Buffers use shared memory with GIL-released blocking reads for efficient data transfer.
3
+
The Buffer API provides a zero-copy streaming buffer for transferring data from Erlang to Python. Buffers use shared memory with GIL-released blocking reads for efficient data transfer.
4
4
5
5
## Overview
6
6
7
-
Buffers are designed for WSGI/ASGI `wsgi.input`scenarios where Erlang receives HTTP body chunks and Python needs to consume them:
7
+
Buffers are designed for scenarios where Erlang produces data chunks and Python needs to consume them:
8
8
9
9
- Zero-copy access via Python's buffer protocol (`memoryview`)
| 3.12+ | Multi-executor or Parallel | With `ENABLE_PARALLEL_PYTHON=ON`, uses internal parallel pool|
23
23
| 3.13t | Free-threaded | No GIL, `py:execution_mode()` returns `free_threaded`|
24
-
| 3.14+ | SHARED_GIL subinterpreters | Subinterpreters with shared GIL for C extension compatibility |
24
+
25
+
**Note:** Subinterpreter/OWN_GIL modes are now internal implementation details used by the parallel pool build. Users don't need to select these modes directly.
25
26
26
27
**Python 3.14 Support**: Full support for Python 3.14 including:
27
-
-SHARED_GIL subinterpreter mode for C extension compatibility
28
-
- Proper `sys.path` initialization in subinterpreters
28
+
-Parallel pool build for true parallelism
29
+
- Proper `sys.path` initialization
29
30
- All asyncio features work correctly
30
31
31
32
**FreeBSD Support**: Improved fd handling on FreeBSD/kqueue platforms:
@@ -34,18 +35,18 @@ This guide covers breaking changes and migration steps when upgrading from erlan
34
35
35
36
## Architecture Changes
36
37
37
-
### OWN_GIL Subinterpreter Thread Pool (Python 3.12+)
38
+
### Parallel Pool Build
38
39
39
-
The most significant change in v2.0 is the new execution model. On Python 3.12+, erlang_python now uses **OWN_GIL subinterpreters** for true parallelism:
40
+
The most significant change in v2.0 is the new execution model. When built with `ENABLE_PARALLEL_PYTHON=ON`, erlang_python uses an internal parallel pool for true parallelism:
40
41
41
42
**v1.8.x Architecture:**
42
43
- Single Python interpreter with shared GIL
43
44
- Worker pool with round-robin dispatch
44
45
- All workers share global state
45
46
46
-
**v2.0 Architecture:**
47
-
-N subinterpreters, each in its own thread with its own GIL
48
-
- Each subinterpreter has isolated state (no shared globals)
47
+
**v2.0 Architecture (with parallel build):**
48
+
-Internal pool provides true parallelism
49
+
- Each context has isolated state (no shared globals)
49
50
- True parallel execution without GIL contention
50
51
- 25-30% faster cast operations
51
52
@@ -85,11 +86,10 @@ Check which mode is active:
85
86
```erlang
86
87
%% Check execution mode
87
88
py:execution_mode().
88
-
%% => subinterp (Python 3.12+ with OWN_GIL)
89
-
%% => free_threaded (Python 3.13t with --disable-gil)
90
-
%% => multi_executor (Python < 3.12)
89
+
%% => multi_executor (default worker mode)
90
+
%% => free_threaded (Python 3.13t with --disable-gil)
0 commit comments