Commit 51cbdbd
authored
fix: time travel when going back to interrupt node (#7498)
# Fix: Create fork checkpoint on subgraph time travel
## Problem
When time-traveling to a subgraph checkpoint that has an interrupt, and
then resuming, the resume would load the **wrong state** — it would pick
up the original execution's latest checkpoint instead of the
time-traveled one.
This happened because replaying from a subgraph checkpoint never created
a new parent checkpoint. If the replay hit an interrupt before
`after_tick()` ran, no checkpoint was written at all, so the parent's
"latest" checkpoint was still the old one from the original execution.
## Fix
When the loop detects a time-travel replay (not an `update_state` fork),
it now **eagerly writes a fork checkpoint** at the start of the tick.
This ensures:
1. The parent thread's latest checkpoint points to the replayed state
2. Subsequent `Command(resume=...)` calls find the correct checkpoint
3. Stale `INTERRUPT` pending writes from the old checkpoint are cleared
(they reference old task IDs)
Additionally, the subgraph replay logic now uses the **parent checkpoint
ID** (from `prev_checkpoint_config`) when resolving subgraph checkpoints
during time-travel, matching the existing behavior for `update_state`
forks.
## Checkpoint flow diagrams
### Before fix: time travel leaves no fork
```
Original execution:
C0 (start) --> C1 (step_a) --> C2 (ask_1 interrupt) --> C3 (resume) --> C4 (ask_2 interrupt) --> C5 (done)
Time travel to C2 (subgraph config):
Replay runs... hits interrupt... no new checkpoint written.
Parent "latest" is still C5.
Command(resume="new_answer"):
Loads C5 (wrong!) instead of the replayed C2 state.
```
### After fix: time travel creates a fork
```
Original execution:
C0 --> C1 --> C2 --> C3 --> C4 --> C5 (done)
Time travel to C2 (subgraph config):
C0 --> C1 --> C2 --> C3 --> C4 --> C5
\
F1 (fork, source="fork") <-- new latest
Command(resume="new_answer"):
Loads F1 (correct!) --> resumes from the right state.
After full resume:
C0 --> C1 --> C2 --> C3 --> C4 --> C5
\
F1 --> F2 (ask_1 result) --> F3 (ask_2 interrupt) --> F4 (done)
```
### Manual fork via `update_state` (unchanged)
```
C0 --> C1 --> C2 --> C3
\
U1 (source="update") <-- created by update_state()
This path already worked. The fix skips update/fork sources
so existing behavior is preserved.
```
## Changes
- **`libs/langgraph/langgraph/pregel/_loop.py`**:
- Extract `is_time_traveling` flag from the existing replay detection
logic for reuse
- Write a fork checkpoint (`source="fork"`) eagerly at the start of a
time-travel tick, before execution begins
- Clear stale `INTERRUPT` pending writes when creating the fork (they
reference old task IDs that won't match the new checkpoint)
- Unify subgraph replay ID resolution: check `source in ("update",
"fork")` instead of a separate `is_time_traveling` condition, since the
new fork checkpoint now has `source="fork"`
- **`libs/langgraph/tests/test_time_travel.py`** and
**`test_time_travel_async.py`**: Added 4 new test cases (sync + async):
- `test_replay_from_before_interrupt_then_resume` — replays from a
checkpoint before an interrupt, resumes with a new answer, and verifies
the full checkpoint history (source, next, values) at each stage
- `test_subgraph_time_travel_resume_from_first_interrupt` — time-travels
to a subgraph's first interrupt, resumes both interrupts with new
answers, and verifies the fork creates a new branch while preserving the
original
- `test_subgraph_time_travel_resume_from_second_interrupt` —
time-travels to a subgraph's second interrupt, resumes with a new
answer, and verifies the first interrupt's original answer is preserved
- `test_subgraph_time_travel_checkpoint_pattern` — verifies the fork
checkpoint branches from the correct replay point and that the full
checkpoint tree is correct after resume
- **`libs/langgraph/tests/test_pregel.py`** /
**`test_pregel_async.py`**: Updated existing
`test_weather_subgraph_state` to account for the new fork checkpoint
appearing in history (history length increases by 1)1 parent 4d64227 commit 51cbdbd
5 files changed
Lines changed: 1171 additions & 24 deletions
File tree
- libs/langgraph
- langgraph/pregel
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
692 | 692 | | |
693 | 693 | | |
694 | 694 | | |
695 | | - | |
| 695 | + | |
696 | 696 | | |
697 | 697 | | |
698 | 698 | | |
| |||
710 | 710 | | |
711 | 711 | | |
712 | 712 | | |
713 | | - | |
| 713 | + | |
| 714 | + | |
714 | 715 | | |
715 | 716 | | |
716 | 717 | | |
| |||
765 | 766 | | |
766 | 767 | | |
767 | 768 | | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
| 778 | + | |
| 779 | + | |
| 780 | + | |
| 781 | + | |
| 782 | + | |
| 783 | + | |
| 784 | + | |
| 785 | + | |
| 786 | + | |
| 787 | + | |
| 788 | + | |
768 | 789 | | |
769 | 790 | | |
770 | 791 | | |
| |||
807 | 828 | | |
808 | 829 | | |
809 | 830 | | |
810 | | - | |
| 831 | + | |
811 | 832 | | |
812 | 833 | | |
813 | 834 | | |
814 | 835 | | |
815 | 836 | | |
816 | 837 | | |
817 | | - | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
818 | 843 | | |
819 | 844 | | |
820 | 845 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
615 | 615 | | |
616 | 616 | | |
617 | 617 | | |
618 | | - | |
619 | | - | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
620 | 623 | | |
621 | 624 | | |
622 | 625 | | |
623 | 626 | | |
624 | 627 | | |
625 | 628 | | |
626 | 629 | | |
627 | | - | |
| 630 | + | |
628 | 631 | | |
629 | 632 | | |
630 | 633 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2086 | 2086 | | |
2087 | 2087 | | |
2088 | 2088 | | |
2089 | | - | |
2090 | | - | |
| 2089 | + | |
| 2090 | + | |
| 2091 | + | |
| 2092 | + | |
| 2093 | + | |
2091 | 2094 | | |
2092 | 2095 | | |
2093 | 2096 | | |
2094 | 2097 | | |
2095 | 2098 | | |
2096 | 2099 | | |
2097 | 2100 | | |
2098 | | - | |
| 2101 | + | |
2099 | 2102 | | |
2100 | 2103 | | |
2101 | 2104 | | |
| |||
0 commit comments