Skip to content

Commit 7a4cc6a

Browse files
committed
Add reentrant callbacks for Python→Erlang→Python chains
Implement suspension/resume mechanism that allows Python code to call Erlang functions which call back into Python without deadlocking. Key changes: - Add SuspensionRequired exception for clean Python execution interruption - Spawn separate processes for callback handling to prevent worker exhaustion - Support arbitrarily deep nesting (tested up to 10+ levels) - Fully transparent - erlang.call() works identically from user perspective New files: - test/py_reentrant_SUITE.erl: Test suite for reentrant callbacks - examples/reentrant_demo.erl: Erlang demo with Fibonacci, nested callbacks - examples/reentrant_demo.py: Python module demonstrating transparent reentrant calls
1 parent be9b433 commit 7a4cc6a

10 files changed

Lines changed: 1653 additions & 83 deletions

File tree

CHANGELOG.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,22 @@
11
# Changelog
22

3+
## 1.2.0 (unreleased)
4+
5+
### Added
6+
7+
- **Reentrant Callbacks** - Python→Erlang→Python callback chains without deadlocks
8+
- Exception-based suspension mechanism interrupts Python execution cleanly
9+
- Callbacks execute in separate processes to prevent worker pool exhaustion
10+
- Supports arbitrarily deep nesting (tested up to 10+ levels)
11+
- Transparent to users - `erlang.call()` works the same, just without deadlocks
12+
- New test suite: `test/py_reentrant_SUITE.erl`
13+
- New examples: `examples/reentrant_demo.erl` and `examples/reentrant_demo.py`
14+
15+
### Changed
16+
17+
- Callback handlers now spawn separate processes for execution, allowing workers
18+
to remain available for nested `py:eval`/`py:call` operations
19+
320
## 1.1.0 (2026-02-15)
421

522
### Added

README.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,40 @@ result = erlang.call('my_func', 10, 20)
155155
All three methods are equivalent. The import and attribute syntaxes provide
156156
a more natural Python experience.
157157

158+
### Reentrant Callbacks
159+
160+
Python→Erlang→Python callbacks are fully supported. When Python code calls
161+
an Erlang function that in turn calls back into Python, the system handles
162+
this transparently without deadlocking:
163+
164+
```erlang
165+
%% Register an Erlang function that calls Python
166+
py:register_function(double_via_python, fun([X]) ->
167+
{ok, Result} = py:call('__main__', double, [X]),
168+
Result
169+
end).
170+
171+
%% Define Python functions
172+
py:exec(<<"
173+
def double(x):
174+
return x * 2
175+
176+
def process(x):
177+
from erlang import call
178+
# This calls Erlang, which calls Python's double()
179+
doubled = call('double_via_python', x)
180+
return doubled + 1
181+
">>).
182+
183+
%% Test the full round-trip
184+
{ok, 21} = py:call('__main__', process, [10]).
185+
%% 10 → double_via_python → double(10)=20 → +1 = 21
186+
```
187+
188+
The implementation uses a suspension/resume mechanism that frees the dirty
189+
scheduler while the Erlang callback executes, preventing deadlocks even with
190+
multiple levels of nesting.
191+
158192
## Shared State Between Workers
159193

160194
Python workers don't share namespace state, but you can share data via the

0 commit comments

Comments
 (0)