sbcl-generations saves a running SBCL image, keeps the saved images as
numbered generations, and lets a process select one to be booted next. It is
the whole-heap counterpart to sbcl-workers: that library runs work in a
fresh child; this one preserves the exact live state of the process you are
already in.
(defparameter *store*
(sbcl-generations:make-generation-store
:root #p"/var/lib/example/generations/"
:current-pathname #p"/var/lib/example/current-generation.sexp"))
(defparameter *backend*
(sbcl-generations:make-checkpoint-backend
:store *store*
:toplevel-function #'example-main))
(sbcl-generations:checkpoint-create *backend*)
;; => #<GENERATION "..." :PENDING>, and this process keeps runningsave-lisp-and-die ends the process that calls it. A checkpoint uses two
forks so the caller can keep running:
- The caller forks a coordinator and returns immediately. The returned
generation is
:pending, and a watcher thread in the caller updates it. - The coordinator forks a saver, which is the process that calls
save-lisp-and-die. - The coordinator waits for the saver, publishes the result, and exits.
Only step 1 is exclusive, and only briefly. Everything expensive happens in processes that were always going to exit.
A forked child inherits only the forking thread, so a checkpoint requires the
caller to be the only live Lisp thread. This is checked. Hosts with other
fork-and-save paths can use checkpoint-single-threaded-p for the same
exact-Boolean preflight while holding their own exclusion across the check and
fork. The predicate is a snapshot.
On Windows, or when a host explicitly chooses a process restart, construct
make-restart-checkpoint-backend with the same host hooks. Call
checkpoint-restart-create to reserve a pending generation. Deliver the current
command result durably before calling checkpoint-restart-save-and-exit.
At that safe boundary the library reruns prechecks, validates and refreshes
metadata under the host guard, then invokes :ready-function with the final
generation. In that function, atomically record the generation identifier,
creation time and exact metadata for the supervisor. The prepare hook must clear
secrets, detach resources and stop other Lisp threads. Saving terminates the
process; it preserves the exact heap, including mutations after reservation.
After a zero exit, the supervisor reconstructs the unpublished generation with
generation-recreate-pending and calls generation-publish. Only then should it
boot the selected core with the saved session arguments. If saving or probing
fails, recover through the host’s prior image and durable session data. Failed
saves record failure.sexp and exit one. The library does not own the supervisor
or promise a restart duration.
A core is an opaque heap written by a process that no longer exists. Before publication the unpublished core is booted with a private argument and must print the exact identity it was saved with.
That is also why the library owns the saved core’s toplevel:
checkpoint-resume-toplevel answers the probe, and otherwise calls the host
entry point you supplied with the command-line arguments.
The core is renamed over its final name, then described by its manifest, then named by the selection pointer, each written atomically. An interruption at any point leaves the previous selection intact.
The library owns the envelope:
- the identifier
- the core path
- the creation time
- the runtime identity that decides whether a core can be booted here
(
:sbcl-version,:operating-system,:operating-system-version,:architecture)
generation-compatible-p refuses a core saved by a different SBCL build, and
generation-select will not choose it.
Everything else belongs to the host. :metadata is spliced into the manifest
flat rather than nested, and :manifest-version is the host’s number, so a
host replacing its own reader with this library keeps reading the manifests it
has already written. Supply :manifest-validator to check your own fields and
:accepted-manifest-versions to keep loading older ones.
Checkpointing a live image means the host has to be given control at four moments. Each hook may be omitted.
| Hook | Runs | For |
|---|---|---|
:around-function | wrapping creation after the precheck | holding the dynamic context for validation, saving, and parent restoration |
:precheck-function | before the host context and guard | slow validation, such as checking a source tree |
:fork-guard-function | wrapping the checks and fork | making that region exclusive |
:validate-function | inside that region | re-checking, and suspending what must not be saved |
:metadata-function | inside that region | the manifest properties, and state that must not drift before the fork |
:prepare-function | inside the saver child | detaching descriptors and clearing secrets |
:resume-function | in the parent afterwards | restoring what :validate-function suspended |
:prepare-function is the security-relevant one. Anything the process is
holding when the saver runs is written into the core, so clear credentials and
detach inherited descriptors there.
Whatever :validate-function returns is handed to :resume-function, which
runs whether or not the fork succeeded. If the fork succeeded but resuming
failed, the library signals a checkpoint-resume-warning rather than an
error: the coordinator is already publishing, and failing would report a
checkpoint that did in fact happen.
generation-request-rollback selects a generation durably and then signals
rollback-requested. Establish a handler where exiting is safe.
Every refusal signals checkpoint-error with a stable stage: :backend,
:validation, :fork, :save, :saver-exit, :coordinator, :probe,
:publish, :manifest, or :selection. A failure inside the coordinator or
saver cannot signal into the caller, so it is recorded as failure.sexp
beside the generation instead.
A prebuilt image is a core saved from a process that loaded a source tree, so a host can boot it instead of loading the tree again:
(sbcl-generations:image-install
#P"/src/app/" #P"/var/lib/app/active/app.core"
:inputs (lambda (root) (list "app.asd" "src/main.lisp"))
:toplevel #'app-main)
(sbcl-generations:image-current-core
#P"/var/lib/app/active/app.core" #P"/src/app/"
:inputs (lambda (root) (list "app.asd" "src/main.lisp")))image-build-record names every input by its Git content hash together with
the runtime identity a generation records. image-install saves the image by
fork where the host can fork and one Lisp thread is alive, and otherwise runs
the :fresh-process-command argv, whose process loads the source and calls
image-save. The saved core is booted with the source root and
the :probe-argument it was saved with (default *image-probe-argument*),
checks its embedded record against that tree, and
must answer image-probe-record exactly. Only then, with the inputs unchanged,
does :publish-function move the core into place and :write-function write
its manifest. image-current-core compares the manifest with the tree without
booting anything, and image-running-build-record returns the record inside a
booted image. :prepare clears host state in the saver before the save.
Use the same source protocol for application-specific build records:
(let ((source (sbcl-generations:make-source-provenance #P"/src/app/")))
(list :revision (sbcl-generations:source-revision source)
:clean-p (sbcl-generations:source-clean-p source :paths '("src/main.lisp"))
:files (sbcl-generations:source-file-identities source '("src/main.lisp"))))make-source-provenance selects the nearest ancestor containing .jj or .git,
including Git’s linked-worktree files. In a colocated checkout it selects jj.
source-root is the canonical input root, and source-kind returns :git,
:jj or :directory. A failed selected backend signals checkpoint-error
at stage :source with a retry restart; it never substitutes another
backend’s revision.
Git revisions are HEAD. Jujutsu revisions are the parents of the working-copy
commit @, joined with + for merges. Cleanliness compares @ against its
merged parents, not Git’s possibly stale HEAD. jj snapshots current files
before querying them; requested files excluded by its snapshot policy count as
dirty. source-clean-p accepts literal relative filenames, or omitted :paths
for the whole tree. source-file-identities returns ordered (PATH BLOB) pairs
for current file bytes and signals on missing inputs. It retains Git’s object
format in Git trees and uses streaming SHA1 Git blob hashes elsewhere.
An unversioned directory has revision "unversioned" and is never clean.
Its image records still compare exact input bytes. Deterministic Git metadata
provided by build systems, including stat-less read-only Nix indexes, uses the
Git backend without an index refresh. Build-record and probe formats are
unchanged. Fresh savers and booted-core probes select provenance directly from
the supplied source root; hosts need no dynamic backend bindings.
(asdf:test-system :sbcl-generations)The suite includes one real checkpoint: it forks, saves a genuine image, boots
it to confirm its identity, and publishes it. That test needs a
single-threaded image and an sbcl on PATH able to boot the core it just
wrote; set SBCL_GENERATIONS_SBCL when the right runtime is somewhere else.
The source tests run actual Git repositories, linked worktrees, shared object
stores, binary hashing, and fresh-process image savers and probes. Install jj
on PATH to include standalone and colocated Jujutsu integration cases;
otherwise those cases are reported as skipped.
Licensed under COLL-Attribution. See LICENSE.lisp for the authoritative
terms.
Part of the Lambda Symbolics library shelf.