From 46d063fcedec45eefdd6cec568eff42e1859292f Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 09:19:05 +0200 Subject: [PATCH 01/34] Add ADRs 001-003: projects/runs model, notebook integration, UI shell Co-Authored-By: Claude Fable 5 --- docs/adr/001-projects-runs-and-storage.md | 282 ++++++++++++++++++ docs/adr/002-notebook-first-class.md | 187 ++++++++++++ ...003-ui-shell-and-simulations-experience.md | 137 +++++++++ docs/adr/README.md | 10 + 4 files changed, 616 insertions(+) create mode 100644 docs/adr/001-projects-runs-and-storage.md create mode 100644 docs/adr/002-notebook-first-class.md create mode 100644 docs/adr/003-ui-shell-and-simulations-experience.md create mode 100644 docs/adr/README.md diff --git a/docs/adr/001-projects-runs-and-storage.md b/docs/adr/001-projects-runs-and-storage.md new file mode 100644 index 00000000..d17f174f --- /dev/null +++ b/docs/adr/001-projects-runs-and-storage.md @@ -0,0 +1,282 @@ +# ADR-001: Projects, runs, and a single project filesystem + +- **Status**: Proposed +- **Date**: 2026-07-11 +- **Deciders**: Anders Hafreager, Claude + +## Context + +Today a "simulation" in Atomify is a single in-memory object +(`Simulation` in `src/store/simulation.ts`): + +```ts +export interface Simulation { + id: string; + files: SimulationFile[]; + inputScript: string; + analysisDescription?: string; + analysisScript?: string; + start: boolean; + vars?: Record; + showSimulationBox?: boolean; + showWalls?: boolean; +} +``` + +Its problems, in order of user pain: + +1. **Nothing persists.** The New Simulation modal literally warns + *"these files are not stored across Atomify sessions yet"* + (`src/containers/NewSimulation.tsx`). Closing the tab loses all edits. +2. **There is no history.** Every run overwrites the outputs of the previous + one in the WASM filesystem and in the notebook storage. Comparing "before" + and "after" a parameter change — the single most common workflow in + MD work — is impossible without manually copying files out. +3. **One simulation at a time, keyed by a collision-prone `id`.** The `id` + doubles as the WASM FS directory and the JupyterLite folder name. Loading + the "diffusion" example twice silently merges/overwrites state. +4. **Edits bypass the store.** The Monaco editor mutates `file.content` in + place (`src/containers/Edit.tsx`), invisible to easy-peasy/immer, so there + is no reliable "content changed" signal to hook persistence onto. +5. **The notebook is a one-way dump.** `syncFilesJupyterLite` copies run + outputs into the JupyterLite contents IndexedDB after a run; nothing is + ever read back, and generated notebooks are never updated once they exist. + +The goal is a product-grade story: a **library of projects** that persists +across sessions, where each project accumulates **runs** with full +provenance, and where the Jupyter notebook operates on the *same files* the +app does (see ADR-002). + +## Decision + +### 1. Domain model: Project = working tree + runs + +A **project** is a named directory containing the user's *source files* +(input scripts, data files, notebooks) — the working tree — plus a `runs/` +directory that accumulates immutable **run records**. + +A **run** is created every time a simulation is executed. It contains a +snapshot of the source files as they were at launch time, all outputs the +run produced, and a metadata record. Runs are append-only history; the +working tree is the only thing the user edits. + +### 2. Canonical filesystem layout + +The layout is storage-agnostic — the same tree works in IndexedDB today and +on a server filesystem behind an API later: + +``` +diffusion/ # project directory (dirName = identity) + .atomify/ + project.json # project metadata (see below) + in.diffusion # working tree: source files + data.lammps + analysis.ipynb # project notebook (see ADR-002) + runs/ + 0001-k3f9a2/ # run directory: seq number + short id + .atomify/ + run.json # run metadata (see below) + in.diffusion # snapshot of sources at launch + log.lammps # outputs + msd.dat + 0002-8xc1p0/ + ... +``` + +`project.json`: + +```jsonc +{ + "schemaVersion": 1, + "displayName": "Diffusion", // free-form, renameable + "description": "LJ binary diffusion", + "createdAt": "2026-07-11T09:00:00Z", + "inputScript": "in.diffusion", // default script to run + "source": { "type": "example", "exampleId": "diffusion/simple_diffusion" } + // source.type: "blank" | "upload" | "example" | "shared" +} +``` + +`run.json`: + +```jsonc +{ + "schemaVersion": 1, + "id": "0002-8xc1p0", + "inputScript": "in.diffusion", + "vars": { "T": 3.0 }, + "status": "completed", // running | completed | failed | canceled + "startedAt": "2026-07-11T09:05:00Z", + "finishedAt": "2026-07-11T09:06:31Z", + "stats": { "timesteps": 5000, "numAtoms": 6750, "wallSeconds": 91.2 }, + "origin": "app" // app | notebook (ADR-002) +} +``` + +Key naming decisions: + +- **`dirName` is the project identity.** It is a slug derived from the + display name at creation time (`"Diffusion"` → `diffusion`), unique-ified + with a numeric suffix on collision (`diffusion-2`). It is what the user + sees in the Jupyter file browser, so it must be human-readable — a UUID + here would wreck the notebook experience. Renaming a project changes only + `displayName`; `dirName` is immutable after creation so that notebook + cells referencing paths never break. +- **Run directories sort chronologically by name** (zero-padded sequence + number) and carry a short random suffix so concurrent creators (app and + notebook kernel) cannot collide. +- **Snapshot scope**: all working-tree files except notebooks (`*.ipynb`) + are copied into the run directory at launch. Notebooks are analysis, not + simulation input. Data files are included because they affect results; + this trades some storage for complete provenance (see Consequences). + +### 3. The project filesystem IS the JupyterLite contents store + +There is **one** persistent filesystem, not two stores kept in sync. +Projects are stored in the IndexedDB database that JupyterLite's contents +manager already reads and writes (`"JupyterLite Storage"`, store `files` — +configured in `jupyter-lite.json` and already targeted by +`src/store/simulation.ts:25-30`). Records use JupyterLite's contents schema +(`{name, path, content, format, mimetype, type, last_modified, created, size, +writable}`), one record per file/directory, keyed by path. + +Consequences of this choice do the heavy lifting for the whole feature: + +- "Sync with the notebook" stops being a mechanism and becomes a + **non-event**: the notebook file browser shows a project's directory + because the project *is* that directory. +- Files created by the pyodide kernel (via its DriveFS mount, ADR-002) are + project files with zero extra code in the app. +- The future API backend is a natural fit: the Jupyter contents schema is + already a REST API shape (Jupyter Server's contents API), so the swap is + an implementation change, not a redesign. + +### 4. Swappable storage interface + +All persistence goes through a `ProjectStorage` interface. Nothing in the +app touches localforage/IndexedDB directly. + +```ts +// src/storage/types.ts +export interface FileStat { + path: string; // project-relative, e.g. "runs/0001-k3f9a2/log.lammps" + type: "file" | "notebook" | "directory"; + lastModified: string; // ISO 8601 + size: number; +} + +export interface ProjectStorage { + // Project level + listProjects(): Promise; // reads .atomify/project.json of each top-level dir + createProject(meta: NewProjectMeta): Promise; // allocates dirName + updateProjectMeta(dirName: string, patch: Partial): Promise; + deleteProject(dirName: string): Promise; // recursive + + // File level (paths are project-relative) + list(dirName: string, subdir?: string): Promise; + read(dirName: string, path: string): Promise; + write(dirName: string, path: string, content: string | Uint8Array): Promise; + rename(dirName: string, from: string, to: string): Promise; + remove(dirName: string, path: string): Promise; + stat(dirName: string, path: string): Promise; +} +``` + +Implementations: + +- **`JupyterContentsStorage`** (now): backed by a dedicated `localforage` + instance on `"JupyterLite Storage"/files`. Maintains directory records and + parent listings the way JupyterLite expects. +- **`MemoryProjectStorage`** (now): in-memory map, used in **embedded mode** + (iframe embeds and share-link previews must not write into the visitor's + library) and in unit tests. +- **`ApiProjectStorage`** (later): same interface over HTTP. + +The active implementation is chosen once at startup (embedded mode check) +and injected into the easy-peasy store via `createStore`'s `injections`, so +thunks receive it as a dependency rather than importing a singleton. + +Run management (snapshotting sources, allocating run dirs, writing +`run.json`) is a thin `runs.ts` module implemented **on top of** +`ProjectStorage` file operations — it is layout convention, not a storage +concern, so the swappable surface stays minimal. + +### 5. Write-through editing, single source of truth + +The store's in-memory project state and Monaco buffers become **caches** of +the project filesystem: + +- Editor changes dispatch a real action (fixing the in-place mutation bug) + and persist via a debounced (~500 ms) `storage.write`. The save state is + surfaced in the UI ("Saved" indicator) — think product. +- When the app regains focus (`visibilitychange`) or the user switches into + the Edit/Files view, the working tree is re-`stat`ed; files whose + `lastModified` is newer than the cached copy (e.g. edited in the Jupyter + editor) are reloaded. Conflict policy is **per-file last-writer-wins** — + acceptable for a single-user, single-browser tool, and revisit-able when + an API backend introduces multi-device access. +- Runs write outputs to the WASM FS during execution (unchanged — the hot + path stays off IndexedDB), then outputs are copied to + `runs//` at completion, plus a throttled copy every few seconds + during the run so notebooks can watch progress (ADR-002). + +### 6. Product rules for project creation + +Every way of loading files into Atomify lands in the same model: + +| Entry point | Behavior | +| --- | --- | +| **New simulation** | Creates a project (blank template or uploaded files). | +| **Examples → open** | Instantiates a project from the example (name = example name, suffixed on repeat). Examples are templates; the gallery itself stays read-only. | +| **Share link / embed** | Embedded mode: `MemoryProjectStorage`, nothing persisted. Normal mode: offer "Save to your simulations" which materializes a project. | + +There is no "unsaved scratch simulation" state — removing it is what makes +the persistence story simple and trustworthy. + +### 7. Migration + +No migration is needed: nothing today persists projects. Existing +`"JupyterLite Storage"` content from the old one-way sync (flat +`analyze.ipynb`, `/` directories without `.atomify/project.json`) is +simply not listed as projects; users can delete it via the Jupyter file +browser. The legacy `syncFilesJupyterLite` thunk is removed. + +## Alternatives considered + +- **Separate Atomify IndexedDB + two-way sync with JupyterLite storage.** + Rejected: perpetual sync code, conflict windows, duplicate storage of + potentially large dump files, and a second source of truth to corrupt. + Single-store was chosen precisely to delete this problem class. +- **OPFS (Origin Private File System).** Attractive perf, but JupyterLite's + contents manager reads IndexedDB; using OPFS reintroduces the two-store + sync problem. +- **File System Access API (real local folder).** Great for power users but + permission UX is heavy, unsupported in Firefox/Safari for writable + handles, and orthogonal — it can become another `ProjectStorage` + implementation later. +- **UUID project identity + display-name mapping everywhere.** Rejected + because the directory name is user-visible in Jupyter; readable paths in + notebooks (`runs/0002-.../log.lammps` under `diffusion/`) beat opaque ones. +- **Storing runs as diffs/refs instead of full snapshots.** Premature; + snapshot-copy is trivially correct and the files are small in the common + case. Content-addressed dedup can be added inside the storage layer + without changing the layout contract. + +## Consequences + +- (+) Projects survive sessions; runs are comparable, reproducible history. +- (+) Notebook integration is structural, not synchronized (ADR-002 builds + directly on this). +- (+) Storage backend swap is one class, as requested. +- (−) IndexedDB quota is finite (Chrome ~60 % of free disk per origin, but + Safari is stricter). Large dump files multiplied by run history will hit + it eventually. Mitigations, in order: surface per-project storage size in + the UI, "delete run" affordance, `navigator.storage.persist()`, and later + a size-capped output policy. Quota errors must surface as a visible + warning, not silent data loss. +- (−) Writing run outputs through the contents schema stores file bodies as + JSON strings (text) — binary dump formats need base64 records, matching + JupyterLite's own convention for binary files. +- (−) The app takes on responsibility for keeping directory records + consistent with what JupyterLite expects; covered by unit tests against + recorded JupyterLite fixtures. diff --git a/docs/adr/002-notebook-first-class.md b/docs/adr/002-notebook-first-class.md new file mode 100644 index 00000000..a7576ef6 --- /dev/null +++ b/docs/adr/002-notebook-first-class.md @@ -0,0 +1,187 @@ +# ADR-002: The notebook as a first-class analysis and scripting surface + +- **Status**: Proposed +- **Date**: 2026-07-11 +- **Deciders**: Anders Hafreager, Claude +- **Depends on**: ADR-001 (projects, runs, single project filesystem) + +## Context + +Atomify embeds JupyterLite (pyodide kernel) in an iframe +(`src/containers/Notebook.tsx`) and today pushes run outputs into the +JupyterLite contents IndexedDB after each run (`syncFilesJupyterLite`). +The integration is shallow: + +- One-way: notebook edits never reach the app; the generated + `analyze.ipynb` is written once and never regenerated. +- Flat: a single global `analyze.ipynb` regardless of simulation; outputs + of successive runs overwrite each other. +- Passive: the notebook can only *look at* results. It cannot run LAMMPS. + +Meanwhile, the upstream **lammps.js** repo (which Atomify already depends +on for its WASM engine) ships a proven, deeper integration +(`lammps/lammps.js`, deployed at editor.lammps.org/notebook): + +- A Python package **`lammps-js`** (source `python/lammps/__init__.py`) + whose wheel is bundled into the JupyterLite build (`pypi/`), installable + with `%pip install lammps-js`. It mirrors the official LAMMPS Python API + (`lmp = await lammps()`, `command`, `get_thermo`, `extract_atom`, …). +- Inside the pyodide worker it dynamically imports the lammps.js ES module + from `{site}/lammps/client.js` — the engine runs **in the same worker as + the kernel**, no message passing. +- A proxy Emscripten filesystem (`_MOUNT_DRIVEFS_JS`) mounts JupyterLite's + DriveFS onto the engine's `/work`, so every file LAMMPS writes lands + **next to the notebook** in the contents store, instantly visible in the + file browser. +- Cross-origin isolation on static hosting via a build-time service-worker + patch (`examples/notebook/coi_patch.py`) — JupyterLite's own service + worker must keep owning the scope (the DriveFS contents API rides on it), + so the generic `coi-serviceworker` shim cannot be used inside the + notebook's scope. + +Atomify pins the **same** JupyterLite versions as lammps.js +(`jupyterlite-core==0.8.0`, `jupyterlite-pyodide-kernel==0.8.1` in +`jupyterlite/requirements.txt`), so these pieces port directly. + +With ADR-001, projects and runs live *in* the JupyterLite contents store. +This ADR decides what the notebook experience is on top of that. + +## Decision + +### 1. The notebook operates on the project directory + +The Notebook pane opens JupyterLite at the current project's notebook: +`lab/index.html?path=/analysis.ipynb`. The kernel's working +directory is therefore the project directory, and by ADR-001's layout the +notebook sees: + +``` +in.diffusion # the working tree +runs/0001-k3f9a2/log.lammps +runs/0002-8xc1p0/log.lammps +``` + +**Cross-run analysis is a glob, not a feature**: + +```python +import glob, lammps_logfile +for path in sorted(glob.glob("runs/*/log.lammps")): + log = lammps_logfile.File(path) + ... +``` + +### 2. A per-project `analysis.ipynb`, generated once + +At project creation, Atomify generates `/analysis.ipynb` from a +template (evolving `src/utils/AnalyzeNotebook.ts`). The template contains: + +1. A markdown header naming the project. +2. A "latest run" cell: loads `sorted(glob("runs/*/log.lammps"))[-1]` and + plots thermo output with `lammps_logfile` + matplotlib. +3. A "compare runs" cell iterating all runs (as above). +4. A commented "drive LAMMPS from Python" cell demonstrating `lammps-js` + (see §3). + +The file is **owned by the user after creation** — Atomify never overwrites +an existing notebook (preserving the guarantee the current code already +makes at `src/store/simulation.ts:443-447`). Because runs are discovered by +glob at execution time, the notebook does not need regeneration when new +runs appear — this is what makes "generate once" viable. + +### 3. Bundle the `lammps-js` Python package: script runs from Python + +The Atomify JupyterLite build adopts the lammps.js recipe: + +- Add the `lammps-js` wheel to the build (piplite index in `pypi/`), and + copy the lammps.js `dist/` to `public/jupyter/lammps/` so the package's + client-URL derivation (strip `lab/…` from the kernel URL, append + `lammps/client.js`) resolves. Both steps go into the existing + `jupyter lite build` step in `.github/workflows/deploy.yaml` and the + local `Makefile` equivalent. +- Notebooks can then run *entire simulations in the kernel*: + +```python +%pip install lammps-js +from lammps import lammps +lmp = await lammps() +for T in [1.0, 2.0, 3.0]: + lmp.command(f"variable T equal {T}") + lmp.file("in.diffusion") # reads the project's working tree +``` + +- Thanks to the DriveFS mount, files the kernel-side engine writes land in + the project directory. The template's scripting cell demonstrates the + convention of writing into `runs//` so scripted sweeps appear in + the app's run history. Run records created this way carry + `"origin": "notebook"`; a `run.json` is optional for notebook runs — the + app lists any `runs/*` directory, and fills in what metadata exists. + +This is the "script and run multiple things" capability: the app's Run +button is one producer of runs; Python loops in the notebook are another. +Both write the same layout into the same store. + +### 4. App ⇄ notebook freshness + +Because there is one store (ADR-001 §3), there is no sync protocol — only +cache refresh: + +- **App → notebook**: JupyterLite's file browser polls the contents store; + newly written run outputs appear on its next refresh. During a run the + app copies outputs from the WASM FS into `runs//` on a throttle + (every ~3 s) and at completion, so a notebook can analyze a run in + flight. +- **Notebook → app**: the app re-stats the working tree on + `visibilitychange` and on pane switches (ADR-001 §5), picking up files + created or edited from the notebook side. The runs list refreshes on the + same triggers, discovering notebook-created runs. + +No postMessage bridge is required for v1. If richer signaling is ever +needed (e.g. push-refresh of the runs list), a `BroadcastChannel` on the +same origin is the natural upgrade path; it is explicitly out of scope now. + +### 5. Cross-origin isolation strategy + +KOKKOS multithreading needs `crossOriginIsolated`, and the kernel-side +engine wants it too (`lammps-js` falls back to serial otherwise). The main +app uses the `coi-serviceworker` shim, but **inside the notebook's scope +JupyterLite's service worker must stay in control** (its contents API and +DriveFS depend on it). Therefore Atomify adopts lammps.js's approach: patch +the built JupyterLite `service-worker.js` to add +COOP/COEP/CORP headers to every response (port of +`examples/notebook/coi_patch.py`), applied as a post-build step. This also +resolves the known conflict noted in project memory (JupyterLite's SW +shadowing the coi shim). Serial fallback keeps notebooks functional if +isolation fails. + +## Alternatives considered + +- **Keep app-side runs as the only producer and treat the notebook as a + viewer.** Rejected: parameter sweeps are the core scientific workflow the + user asked for ("script and run multiple things"), and lammps.js already + paid the engineering cost. +- **postMessage/custom JupyterLite extension for file sync.** Heavy, + version-pinned against JupyterLite internals, and unnecessary once the + store is shared (ADR-001). +- **Running app-triggered simulations inside the kernel too (one engine).** + Tempting unification, but the app's live 3D visualization is wired to its + own worker pipeline (`LammpsWorkerProxy`, per-timestep heap streaming, + ASYNCIFY constraints per project memory). Moving it into the kernel + would couple rendering to notebook lifecycle for no user benefit. +- **Regenerating `analysis.ipynb` after every run.** Rejected: clobbers + user work or forks endless copies; glob-based discovery makes it moot. + +## Consequences + +- (+) "Always synced with the notebook" holds by construction; the demo + story ("run, then analyze across runs in Python, then script a sweep + from Python") requires no manual file plumbing. +- (+) Atomify inherits upstream lammps.js maintenance of the Python + package instead of building its own. +- (−) The JupyterLite build grows by the lammps.js dist (~40 MB with the + KOKKOS wasm). Acceptable for a static site; can be trimmed to the serial + build if size becomes a problem. +- (−) The SW patch pins against jupyterlite-core 0.8.0 internals (same + trade lammps.js accepted; tracked upstream at jupyterlite#1409). +- (−) Two engines (app worker + kernel) can write the same project + concurrently. Run-directory naming (unique suffixes) makes collisions + structurally unlikely; per-file LWW covers the rest (ADR-001 §5). diff --git a/docs/adr/003-ui-shell-and-simulations-experience.md b/docs/adr/003-ui-shell-and-simulations-experience.md new file mode 100644 index 00000000..3995b496 --- /dev/null +++ b/docs/adr/003-ui-shell-and-simulations-experience.md @@ -0,0 +1,137 @@ +# ADR-003: UI shell redesign and the "Your simulations" experience + +- **Status**: Proposed +- **Date**: 2026-07-11 +- **Deciders**: Anders Hafreager, Claude +- **Depends on**: ADR-001 (projects & runs), ADR-002 (notebook) +- **Design source**: claude.ai Design project `f50a64cb-1082-404c-b41f-728c9d24e37f` + (`Atomify.dc.html`). The design is being actively iterated by the user; the + latest version is re-fetched immediately before UI implementation. This ADR + fixes the *structure*; pixels follow the design file. + +## Context + +The current shell is an antd `Layout` with a `Sider` menu driving an +invisible antd `Tabs` (`src/App.tsx`, `src/containers/Main.tsx`, +`src/hooks/useMenuItems.tsx`). There is no notion of a simulation library: +the menu shows View / Console / Notebook / "Edit ``" / New simulation / +Examples / Share / Run controls / Settings, all scoped to the single +in-memory simulation. Files of the loaded simulation appear only as +submenu entries; there is no add/rename/delete, no list of past work, and +no visual identity beyond stock antd dark theme. + +The Claude Design draft establishes: a collapsible custom sidebar (brand, +grouped nav: *Workspace* [View / Console / Notebook / Edit], *Simulations* +[New simulation / Examples / Share], *Run* [Run-Stop primary button / Run +in cloud], footer with dark/light toggle + Settings), an Examples gallery +as a card grid with filter chips and search, a viewport HUD (Dynamics +panel: stats, speed sliders, modifier checkboxes, computes), custom-styled +modals, and a full design-token system (CSS variables, Manrope + +JetBrains Mono, light/dark themes, accent-configurable). + +ADR-001/002 introduce concepts the current UI cannot express: a project +library, a working tree per project, and a run history. + +## Decision + +### 1. Information architecture + +``` +Home ("Your simulations") ← default screen, replaces Examples as landing + ├─ New simulation (modal) ← creates project (blank / upload) + ├─ Examples gallery ← templates; opening one instantiates a project + └─ Project workspace (per project) + ├─ View (3D viewport + Dynamics HUD) + ├─ Console (LAMMPS log of current/last run) + ├─ Notebook (JupyterLite at /analysis.ipynb) + ├─ Files (working tree: open-in-editor, add, upload, rename, delete) + │ └─ Editor (Monaco, per file, autosave with saved-state indicator) + └─ Runs (history list: status, started, duration, atoms/steps, + open outputs in Files/Notebook, delete run) +``` + +- **"Your simulations" is the landing screen** (except embedded mode, which + keeps landing on View). It shows a card grid of projects — name, + description, updated-at, run count, thumbnail — plus prominent "New + simulation" and "Browse examples" entries and an empty state that points + first-time users at Examples. +- **The sidebar becomes project-aware.** The *Workspace* group (View / + Console / Notebook / Files / Runs) renders only when a project is open, + under the project's name; a "Your simulations" item above it returns + home. The *Run* group's primary button runs the open project's + `inputScript`. +- **Thumbnails**: captured from the WebGL canvas at run completion + (`canvas.toBlob` downscaled to ~320 px) and stored via `ProjectStorage` + under `.atomify/thumbnail.png`, so library cards show real results. + +### 2. Navigation state + +The app keeps its no-router architecture (it must keep working under +`/atomify/` on GitHub Pages and in embeds), but the ad-hoc +`selectedMenu` string is replaced by a typed navigation state in the store: + +```ts +type Screen = + | { name: "home" } + | { name: "examples" } + | { name: "project"; dirName: string; + pane: "view" | "console" | "notebook" | "files" | "runs"; + filePath?: string } // files pane with a file open in Monaco +``` + +Deep links (`?project=diffusion&pane=view`) map 1:1 onto this state so a +browser refresh restores where you were. Legacy query params (`script=`, +embed codec URLs) keep working and resolve into the same state. + +### 3. Visual implementation strategy + +- **The shell (sidebar, screens, cards, modals, HUD) is bespoke**, built + from the design file's token system: CSS custom properties + (`--bg/--surface/--text/--accent/…`) defined once with `[data-theme]` + dark/light variants, Manrope for UI, JetBrains Mono for code/paths. + antd's `Menu`-driven shell and its invisible `Tabs` are removed. +- **antd stays for commodity inner components** (upload dragger, tooltips, + notifications) restyled through its `ConfigProvider` token bridge to the + same CSS variables, until replaced opportunistically. This avoids a + big-bang rewrite of working panes (Monaco editor, dygraphs plots, omovi + canvas are untouched). +- **Theme**: a `theme` field in the persisted settings model; `data-theme` + attribute on the root; the design's light palette becomes the first + supported light mode in Atomify. The JupyterLite iframe follows via its + own theme setting (jupyterlab-night for dark) — acceptable to leave + manual in v1. + +### 4. What ships in v1 vs later + +| In v1 | Deferred | +| --- | --- | +| Home grid, create/open/rename/duplicate/delete project | Project search/sort/tags | +| Files pane with full file CRUD + Monaco autosave | Drag-drop reorder, folders in working tree | +| Runs pane (list, open outputs, delete run) | Run diffing/comparison UI | +| Design tokens, dark + light themes, collapsible sidebar | Accent color picker | +| Examples gallery restyled per design | Example authoring from a project | + +## Alternatives considered + +- **Adopt a router (react-router).** The screen graph is small and the + deep-link mapping is trivial; a router adds a dependency and base-path + friction on GitHub Pages for no structural gain at this size. +- **Restyle antd in place instead of a bespoke shell.** The design's + sidebar, cards, and HUD do not map onto antd `Menu`/`Tabs` idioms; + fighting the component library costs more than owning ~600 lines of + layout primitives. +- **Keep Examples as the landing page.** Rejected: the product story is + "your work lives here"; examples are an on-ramp, reachable in one click + and via the empty state. + +## Consequences + +- (+) The app reads as a product: work persists visibly, history is + browsable, files are manageable, and the visual language is owned. +- (+) Typed navigation state kills the `"file"+fileName` string-key hack + and makes project/pane deep-linking testable. +- (−) `useMenuItems.tsx`, `Main.tsx`, and `App.tsx` are substantially + rewritten; embed mode and the AutoStart flow must be regression-tested + (they bypass the home screen). +- (−) Two styling systems (tokens + antd bridge) coexist during the + transition; contained by keeping antd usage inside panes. diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 00000000..9bd2e960 --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1,10 @@ +# Architecture Decision Records + +ADRs document significant architectural decisions for Atomify. Each record states +context, the decision, alternatives considered, and consequences. + +| ADR | Title | Status | +| --- | --- | --- | +| [ADR-001](001-projects-runs-and-storage.md) | Projects, runs, and a single project filesystem | Proposed | +| [ADR-002](002-notebook-first-class.md) | The notebook as a first-class analysis and scripting surface | Proposed | +| [ADR-003](003-ui-shell-and-simulations-experience.md) | UI shell redesign and the "Your simulations" experience | Proposed | From de20f62e7406257f6346466d49fa5ba1306eb6c8 Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 09:36:06 +0200 Subject: [PATCH 02/34] Revise ADRs after agent review round 1 - ADR-001: run-outputs worker protocol, snapshot contract, quota ordering, checkpoints cleanup, quick-run/embed storage contexts, input script selection, parameter sweeps, reconciliation rules - ADR-002: lammps-js wheel supply chain, kernel capability boundary, put-on-close visibility, lammps-logfile wheel, COI two-SW plan - ADR-003: rewritten around Atomify Projects design (Projects noun, Home, quick runs, Files/Runs/Notebook tabs, run detail, run lifecycle, sweeps) Co-Authored-By: Claude Fable 5 --- docs/adr/001-projects-runs-and-storage.md | 449 +++++++++++------- docs/adr/002-notebook-first-class.md | 303 ++++++------ ...003-ui-shell-and-simulations-experience.md | 287 ++++++----- docs/adr/README.md | 2 +- 4 files changed, 632 insertions(+), 409 deletions(-) diff --git a/docs/adr/001-projects-runs-and-storage.md b/docs/adr/001-projects-runs-and-storage.md index d17f174f..547290ec 100644 --- a/docs/adr/001-projects-runs-and-storage.md +++ b/docs/adr/001-projects-runs-and-storage.md @@ -1,6 +1,6 @@ # ADR-001: Projects, runs, and a single project filesystem -- **Status**: Proposed +- **Status**: Proposed (rev 2 — after architecture + product review) - **Date**: 2026-07-11 - **Deciders**: Anders Hafreager, Claude @@ -28,24 +28,27 @@ Its problems, in order of user pain: 1. **Nothing persists.** The New Simulation modal literally warns *"these files are not stored across Atomify sessions yet"* (`src/containers/NewSimulation.tsx`). Closing the tab loses all edits. -2. **There is no history.** Every run overwrites the outputs of the previous - one in the WASM filesystem and in the notebook storage. Comparing "before" - and "after" a parameter change — the single most common workflow in - MD work — is impossible without manually copying files out. -3. **One simulation at a time, keyed by a collision-prone `id`.** The `id` - doubles as the WASM FS directory and the JupyterLite folder name. Loading - the "diffusion" example twice silently merges/overwrites state. -4. **Edits bypass the store.** The Monaco editor mutates `file.content` in - place (`src/containers/Edit.tsx`), invisible to easy-peasy/immer, so there - is no reliable "content changed" signal to hook persistence onto. -5. **The notebook is a one-way dump.** `syncFilesJupyterLite` copies run - outputs into the JupyterLite contents IndexedDB after a run; nothing is - ever read back, and generated notebooks are never updated once they exist. +2. **There is no history.** Every run overwrites the previous one's outputs. + Comparing runs — the single most common workflow in MD work — requires + manually copying files out of the browser. +3. **One simulation at a time, keyed by a collision-prone `id`** that + doubles as the WASM FS directory and the JupyterLite folder name. +4. **Edits bypass the store.** Monaco mutates `file.content` in place + (`src/containers/Edit.tsx`), invisible to easy-peasy/immer — no reliable + "content changed" signal exists to hook persistence onto. +5. **The notebook integration is one-way and stale.** `syncFilesJupyterLite` + was written when the WASM FS lived on the main thread. Since the worker + migration, the main thread's `getWasm().FS` is a **write-only bridge** + (a JS `Map` mirroring only files the main thread itself wrote — + `src/wasm/LammpsWorkerProxy.ts:300-362`), and the worker protocol has no + file-read command (`src/wasm/workerMessages.ts`). So today's "sync" only + round-trips input files; **run outputs never reach the notebook at all.** The goal is a product-grade story: a **library of projects** that persists across sessions, where each project accumulates **runs** with full provenance, and where the Jupyter notebook operates on the *same files* the -app does (see ADR-002). +app does (ADR-002). The user-facing noun is **project** (per the design, +ADR-003); "simulation" describes what a run executes. ## Decision @@ -55,10 +58,19 @@ A **project** is a named directory containing the user's *source files* (input scripts, data files, notebooks) — the working tree — plus a `runs/` directory that accumulates immutable **run records**. -A **run** is created every time a simulation is executed. It contains a -snapshot of the source files as they were at launch time, all outputs the -run produced, and a metadata record. Runs are append-only history; the -working tree is the only thing the user edits. +A **run** is created every time a simulation is executed — from the app's +Run button, from a parameter sweep (§7), or from Python in the notebook +(ADR-002). It contains a snapshot of the source files as they were at +launch, all outputs the run produced, and a metadata record. Runs are +append-only history; the working tree is the only thing the user edits. + +Each project has **at most one designated input script** +(`project.json.inputScript`). It is set automatically when unambiguous +(project created from an example; or exactly one `in.*`/`*.in` file in the +working tree) and is otherwise unset. **Run behavior when unset**: the Run +button prompts the user to choose from the project's candidate scripts and +persists the choice (ADR-003 §5). Any file can also be run directly +("Run this file") without changing the default. ### 2. Canonical filesystem layout @@ -68,18 +80,19 @@ on a server filesystem behind an API later: ``` diffusion/ # project directory (dirName = identity) .atomify/ - project.json # project metadata (see below) + project.json # project metadata in.diffusion # working tree: source files data.lammps - analysis.ipynb # project notebook (see ADR-002) + analysis.ipynb # project notebook (ADR-002) runs/ - 0001-k3f9a2/ # run directory: seq number + short id + run-001/ # run directory .atomify/ - run.json # run metadata (see below) + run.json # run metadata + frame.png # final viewport capture (ADR-003) in.diffusion # snapshot of sources at launch log.lammps # outputs msd.dat - 0002-8xc1p0/ + run-002/ ... ``` @@ -88,11 +101,12 @@ diffusion/ # project directory (dirName = identity) ```jsonc { "schemaVersion": 1, - "displayName": "Diffusion", // free-form, renameable + "displayName": "Diffusion coefficients", // free-form, renameable "description": "LJ binary diffusion", "createdAt": "2026-07-11T09:00:00Z", - "inputScript": "in.diffusion", // default script to run - "source": { "type": "example", "exampleId": "diffusion/simple_diffusion" } + "inputScript": "in.diffusion", // designated script; may be absent + "color": "#3F6EFF", // library identity dot (ADR-003) + "source": { "type": "example", "exampleId": "diffusion" } // source.type: "blank" | "upload" | "example" | "shared" } ``` @@ -102,181 +116,292 @@ diffusion/ # project directory (dirName = identity) ```jsonc { "schemaVersion": 1, - "id": "0002-8xc1p0", + "id": "run-002", "inputScript": "in.diffusion", - "vars": { "T": 3.0 }, - "status": "completed", // running | completed | failed | canceled + "vars": { "T": 3.0 }, // LAMMPS variables injected (§7) + "sweepId": "sweep-yj2k", // present when part of a sweep (§7) + "status": "completed", // running | completed | failed | canceled | interrupted "startedAt": "2026-07-11T09:05:00Z", "finishedAt": "2026-07-11T09:06:31Z", "stats": { "timesteps": 5000, "numAtoms": 6750, "wallSeconds": 91.2 }, - "origin": "app" // app | notebook (ADR-002) + "error": null, // LAMMPS error message when failed + "origin": "app" // app | sweep | notebook (ADR-002) } ``` -Key naming decisions: - -- **`dirName` is the project identity.** It is a slug derived from the - display name at creation time (`"Diffusion"` → `diffusion`), unique-ified - with a numeric suffix on collision (`diffusion-2`). It is what the user - sees in the Jupyter file browser, so it must be human-readable — a UUID - here would wreck the notebook experience. Renaming a project changes only - `displayName`; `dirName` is immutable after creation so that notebook - cells referencing paths never break. -- **Run directories sort chronologically by name** (zero-padded sequence - number) and carry a short random suffix so concurrent creators (app and - notebook kernel) cannot collide. -- **Snapshot scope**: all working-tree files except notebooks (`*.ipynb`) - are copied into the run directory at launch. Notebooks are analysis, not - simulation input. Data files are included because they affect results; - this trades some storage for complete provenance (see Consequences). +Naming rules: + +- **`dirName` is the project identity.** Slug of the display name at + creation (`"Diffusion coefficients"` → `diffusion-coefficients`), + alphabet `[a-z0-9-]`, non-empty (fallback `project`), unique-ified with a + numeric suffix. It is what the user sees in the Jupyter file browser, so + it must be human-readable. Renaming a project changes only `displayName`; + `dirName` is immutable so notebook paths never break. The rename dialog + and the workspace header show the dirName (mono font) so the mapping is + never a mystery. `%` and `/` are rejected in all file names (JupyterLite's + drive `decodeURIComponent`s paths; `%` throws in its UI). +- **Run directories are `run-NNN`** (zero-padded sequence, per the design). + Allocation takes the max existing number + 1; if the allocated directory + already exists (concurrent creation from the notebook side), allocation + retries with the next number. Runs created by external tools with other + names are still listed (§6). + +**Snapshot contract** (explicit allow/deny — review finding): a run +snapshot copies the working tree **excluding** `runs/`, `.atomify/`, and +`*.ipynb`. Notebooks are analysis, not simulation input; `runs/` excluded +prevents recursive snapshot growth. To keep snapshots cheap, a file is +**skipped when byte-identical to the same path in the previous run's +snapshot** — the record is written as a copy of the previous record +(storage-level dedup can improve this later without changing the layout). +Files larger than a threshold (default 64 MB) are not snapshotted; the run +records their names + hashes in `run.json` so provenance gaps are explicit. ### 3. The project filesystem IS the JupyterLite contents store There is **one** persistent filesystem, not two stores kept in sync. -Projects are stored in the IndexedDB database that JupyterLite's contents -manager already reads and writes (`"JupyterLite Storage"`, store `files` — -configured in `jupyter-lite.json` and already targeted by -`src/store/simulation.ts:25-30`). Records use JupyterLite's contents schema -(`{name, path, content, format, mimetype, type, last_modified, created, size, -writable}`), one record per file/directory, keyed by path. - -Consequences of this choice do the heavy lifting for the whole feature: - -- "Sync with the notebook" stops being a mechanism and becomes a - **non-event**: the notebook file browser shows a project's directory - because the project *is* that directory. -- Files created by the pyodide kernel (via its DriveFS mount, ADR-002) are - project files with zero extra code in the app. -- The future API backend is a natural fit: the Jupyter contents schema is - already a REST API shape (Jupyter Server's contents API), so the swap is - an implementation change, not a redesign. +Projects live in the IndexedDB database JupyterLite's contents manager +already uses (`"JupyterLite Storage"`, per `jupyter-lite.json`), in +JupyterLite's contents schema: one record per file/directory keyed by path, +value `{name, path, content, format, mimetype, type, last_modified, +created, size, writable}`. + +This single decision does the heavy lifting: "sync with the notebook" stops +being a mechanism and becomes a non-event; files created by the pyodide +kernel are project files with zero extra code; and the future API backend +maps 1:1 onto Jupyter Server's contents REST API. + +**Facts we design around** (verified against `@jupyterlite/contents`' +`BrowserStorageDrive`, jupyterlite-core 0.8): + +- The DB has **three object stores**: `files`, `counters`, and + `checkpoints`. JupyterLite saves up to 5 full checkpoint copies of files + edited in its UI. Our `remove`/`rename`/`deleteProject` must clear + matching `checkpoints` entries (mirroring JupyterLite's own + `forgetPath`), or deleted projects leak quota and can be "restored" from + the checkpoint UI. +- Listing a directory in JupyterLite **iterates every record in the store, + materializing values**. Browsing cost therefore scales with total bytes + stored, which is another reason the size caps in §2/§8 exist. +- `get()` on a path with no record **throws**; every ancestor directory + needs an explicit directory record. `ProjectStorage.write()` auto-creates + missing parent directory records. +- Notebooks are stored as **parsed JSON** (`format: "json"`), binary files + as base64 strings, text as strings. The storage implementation owns the + extension→`{type, format, mimetype}` table and it must match JupyterLite's + file registry. +- Dot-directories (`.atomify/`) get records like any other path; JupyterLab's + file browser hides dotfiles by default (`showHiddenFiles` off), which is + exactly the behavior the layout wants. +- The kernel-side DriveFS proxy flushes writes on **file close** — a file + LAMMPS holds open all run (e.g. `log.lammps`) reaches the store at + completion, not incrementally (consequences in ADR-002 §4). + +**App-side listing strategy** (avoids the O(store bytes) trap): the app +never bulk-reads file bodies to render lists. `listProjects()` scans +**keys only** (`localforage.keys()`), finds top-level dirs, and reads just +their `.atomify/project.json` records. The Runs tab likewise scans keys +under `/runs/` and reads only `run.json` records lazily. `stat()` +of a working-tree file does read its record (localforage has no partial +reads), which is acceptable because working trees are small; the app never +`stat`s `runs/` contents wholesale. ### 4. Swappable storage interface -All persistence goes through a `ProjectStorage` interface. Nothing in the -app touches localforage/IndexedDB directly. +All persistence goes through a `ProjectStorage` interface; nothing else +touches localforage/IndexedDB. ```ts // src/storage/types.ts +export type FileFormat = "text" | "json" | "base64"; + export interface FileStat { - path: string; // project-relative, e.g. "runs/0001-k3f9a2/log.lammps" + path: string; // project-relative type: "file" | "notebook" | "directory"; - lastModified: string; // ISO 8601 + format: FileFormat; + mimetype: string; + lastModified: string; // ISO 8601 size: number; } export interface ProjectStorage { - // Project level - listProjects(): Promise; // reads .atomify/project.json of each top-level dir - createProject(meta: NewProjectMeta): Promise; // allocates dirName + // Projects + listProjects(): Promise; + createProject(meta: NewProjectMeta): Promise; // allocates dirName updateProjectMeta(dirName: string, patch: Partial): Promise; - deleteProject(dirName: string): Promise; // recursive + deleteProject(dirName: string): Promise; // recursive, incl. checkpoints - // File level (paths are project-relative) - list(dirName: string, subdir?: string): Promise; - read(dirName: string, path: string): Promise; + // Files (project-relative paths; parents auto-created on write) + list(dirName: string, subdir?: string): Promise; // key-scan, no body reads + read(dirName: string, path: string): Promise; // Uint8Array iff base64 write(dirName: string, path: string, content: string | Uint8Array): Promise; - rename(dirName: string, from: string, to: string): Promise; - remove(dirName: string, path: string): Promise; + rename(dirName: string, from: string, to: string): Promise; // incl. checkpoints + remove(dirName: string, path: string): Promise; // incl. checkpoints stat(dirName: string, path: string): Promise; } ``` +`read` returns `Uint8Array` exactly when the stored format is `base64`; +notebooks come back as their JSON text. Binary uploads are supported by +reading `File.arrayBuffer()` (the current `originFile.text()` upload path +is replaced). + Implementations: -- **`JupyterContentsStorage`** (now): backed by a dedicated `localforage` - instance on `"JupyterLite Storage"/files`. Maintains directory records and - parent listings the way JupyterLite expects. -- **`MemoryProjectStorage`** (now): in-memory map, used in **embedded mode** - (iframe embeds and share-link previews must not write into the visitor's - library) and in unit tests. +- **`JupyterContentsStorage`** (now): the IndexedDB implementation above. +- **`MemoryProjectStorage`** (now): in-memory map for **embedded mode**, + **quick runs** (§6), and unit tests. - **`ApiProjectStorage`** (later): same interface over HTTP. -The active implementation is chosen once at startup (embedded mode check) -and injected into the easy-peasy store via `createStore`'s `injections`, so -thunks receive it as a dependency rather than importing a singleton. - -Run management (snapshotting sources, allocating run dirs, writing -`run.json`) is a thin `runs.ts` module implemented **on top of** -`ProjectStorage` file operations — it is layout convention, not a storage -concern, so the swappable surface stays minimal. - -### 5. Write-through editing, single source of truth - -The store's in-memory project state and Monaco buffers become **caches** of -the project filesystem: - -- Editor changes dispatch a real action (fixing the in-place mutation bug) - and persist via a debounced (~500 ms) `storage.write`. The save state is - surfaced in the UI ("Saved" indicator) — think product. -- When the app regains focus (`visibilitychange`) or the user switches into - the Edit/Files view, the working tree is re-`stat`ed; files whose - `lastModified` is newer than the cached copy (e.g. edited in the Jupyter - editor) are reloaded. Conflict policy is **per-file last-writer-wins** — - acceptable for a single-user, single-browser tool, and revisit-able when - an API backend introduces multi-device access. -- Runs write outputs to the WASM FS during execution (unchanged — the hot - path stays off IndexedDB), then outputs are copied to - `runs//` at completion, plus a throttled copy every few seconds - during the run so notebooks can watch progress (ADR-002). - -### 6. Product rules for project creation - -Every way of loading files into Atomify lands in the same model: +The storage a given project uses is resolved by context (persistent library +vs. quick-run/embed scratch space) — not a single global chosen once. Both +implementations are injected into the easy-peasy store via `createStore` +`injections` so thunks receive them as dependencies. + +Run management (snapshotting, run-dir allocation, `run.json`, sweeps) is a +`src/storage/runs.ts` module implemented **on top of** `ProjectStorage` — +layout convention, not a storage concern, so the swap surface stays minimal. + +### 5. Run outputs data path (new worker protocol) + +The runs mechanism requires reading files back out of the worker, which is +impossible today (Context #5). The worker protocol gains: + +- `snapshotWorkdir` request → worker responds with `{path, bytes}[]` for + files under the run's working directory, **collected inside the + step-callback window** (the WASM module sits suspended in ASYNCIFY sleep + between steps and must only be touched synchronously inside the step + callback — established project constraint). Buffers are transferred, not + copied. +- The collection is **size-aware**: per file, only files ≤ the copy + threshold are included mid-run; everything (subject to §2 caps) is + included in the final end-of-run snapshot. `log.lammps` and small series + files stream live; a growing multi-hundred-MB dump does not stall the + simulation loop. + +Flow per run: + +1. Flush pending editor debounce (§8), snapshot working tree → + `runs/run-NNN/` (from the store's buffers, so what runs is what is + recorded — no debounce race). +2. Write `run.json` (`status: "running"`), sync sources into the worker's + WASM FS under `//runs/run-NNN/`, run there (cwd = run dir, so + LAMMPS relative writes land in the run by construction). +3. Every ~3 s of wall clock, the step callback triggers `snapshotWorkdir` + (thresholded) → `ProjectStorage.write` the changed outputs. +4. On finish: final snapshot (all sizes, subject to caps), update + `run.json` (`completed`/`failed`/`canceled` + stats + error), capture + `frame.png`. + +**Reconciliation** (crash/zombie handling — review finding): on project +open and on library listing, any run with `status: "running"` that this +session does not own is rewritten to `"interrupted"` (partial outputs +kept, badge in UI). A top-level directory without `.atomify/project.json` +is not listed; the library offers "recover as project" for such orphans. +If the project directory disappears mid-run (deleted from the Jupyter +side), the run is canceled and its output copies stop. + +### 6. Product rules: where files come from | Entry point | Behavior | | --- | --- | -| **New simulation** | Creates a project (blank template or uploaded files). | -| **Examples → open** | Instantiates a project from the example (name = example name, suffixed on repeat). Examples are templates; the gallery itself stays read-only. | -| **Share link / embed** | Embedded mode: `MemoryProjectStorage`, nothing persisted. Normal mode: offer "Save to your simulations" which materializes a project. | - -There is no "unsaved scratch simulation" state — removing it is what makes -the persistence story simple and trustworthy. - -### 7. Migration - -No migration is needed: nothing today persists projects. Existing -`"JupyterLite Storage"` content from the old one-way sync (flat -`analyze.ipynb`, `/` directories without `.atomify/project.json`) is -simply not listed as projects; users can delete it via the Jupyter file -browser. The legacy `syncFilesJupyterLite` thunk is removed. +| **New project** | Modal: name + start blank / from example / drop files. Creates a persisted project. | +| **Example → "Use as project"** | Instantiates a project from the example's files. | +| **Example → "Quick run"** | Runs immediately in a **scratch project on `MemoryProjectStorage`** — nothing persisted, banner offers **"Save as project"**, which materializes the working tree *and completed runs* into the library. Closing the tab discards it. This keeps tire-kicking from littering the library (review finding) while "everything in the library persists" stays absolutely true. | +| **Share link** | Normal mode: opens as a quick run (same banner/save flow). Embedded mode: `MemoryProjectStorage`, no save affordance, Notebook hidden (ADR-002 §6). | + +### 7. Parameter sweeps + +A script that uses LAMMPS variables (`variable T equal 3.0` / +`${T}`-style references, already injectable via the existing +`prepareVarsScript` mechanism, `src/store/simulation.ts:37-59`) can be +**parameterized at run time**: the Run flow lets the user define values or +ranges for one or more variables (e.g. `T = 1.0, 1.5, … 3.0`), producing +the cartesian product as a **sweep** — one run per combination: + +- Each combination becomes an ordinary run (`run-014`, `run-015`, …) with + `vars` and a shared `sweepId` in `run.json`. There is no separate sweep + entity in storage — a sweep is a label over runs, so cross-run analysis + in the notebook (`glob("runs/*/log.lammps")` + reading `vars` from + `run.json`) works identically for swept and manual runs. +- The app executes sweep runs **sequentially** (one WASM engine); the Runs + tab shows queued entries (`status: "queued"` in memory only — never + persisted, so no zombie queue states) with progress per run and + cancel-remaining. +- Variable values are injected per run via the vars-wrapper mechanism, and + recorded in `run.json.vars` — the Runs list surfaces them (ADR-003), which + is what makes ten near-identical rows distinguishable. +- Sweeps from Python in the notebook remain a plain `for` loop (ADR-002); + the app sweep is the no-code path over the same storage layout. + +### 8. Write-through editing, single source of truth + +Store state and Monaco buffers are **caches** of the project filesystem: + +- Editor changes dispatch a store action (fixing the in-place mutation) and + persist via debounced (~500 ms) `storage.write`. The editor shows + three save states: *saving / saved / couldn't save (retry)* — a saved + indicator that can silently lie is worse than none (review finding). +- Debounce is **flushed** on run start (§5), on `visibilitychange: hidden`, + and on pane switches. +- On focus regain and pane switches, the working tree is re-`stat`ed; + files newer in storage than the cache (edited in the Jupyter UI) reload — + **unless the local buffer is dirty**, in which case the local buffer wins + until saved (per-file last-writer-wins with a dirty-buffer carve-out). +- Editing is **no longer locked during runs** — runs execute from their + snapshot, so the working tree is free (product win the old design left + on the table; the current read-only editor lock is removed). + +### 9. Migration + +Nothing today persists projects, but the legacy one-way sync did write +`analyze.ipynb` and `/…` entries that may hold real user analysis +work. Policy: leave legacy records untouched; they remain reachable from +the Jupyter file browser. The Notebook pane's flat-path logic +(`src/containers/Notebook.tsx`) and `syncFilesJupyterLite` are removed in +the **same change** that introduces project-scoped notebooks (they are +load-bearing for each other — review finding). Examples that ship a curated +`analysisScript` notebook copy it into the project working tree at +instantiation (taking precedence over the generated template, ADR-002 §2). ## Alternatives considered -- **Separate Atomify IndexedDB + two-way sync with JupyterLite storage.** - Rejected: perpetual sync code, conflict windows, duplicate storage of - potentially large dump files, and a second source of truth to corrupt. - Single-store was chosen precisely to delete this problem class. -- **OPFS (Origin Private File System).** Attractive perf, but JupyterLite's - contents manager reads IndexedDB; using OPFS reintroduces the two-store - sync problem. -- **File System Access API (real local folder).** Great for power users but - permission UX is heavy, unsupported in Firefox/Safari for writable - handles, and orthogonal — it can become another `ProjectStorage` - implementation later. -- **UUID project identity + display-name mapping everywhere.** Rejected - because the directory name is user-visible in Jupyter; readable paths in - notebooks (`runs/0002-.../log.lammps` under `diffusion/`) beat opaque ones. -- **Storing runs as diffs/refs instead of full snapshots.** Premature; - snapshot-copy is trivially correct and the files are small in the common - case. Content-addressed dedup can be added inside the storage layer - without changing the layout contract. +- **Separate Atomify IndexedDB + two-way sync into JupyterLite storage.** + Rejected: perpetual sync code, conflict windows, duplicated large files, + a second source of truth to corrupt. +- **OPFS / File System Access API.** Not readable by JupyterLite's contents + manager → reintroduces two-store sync. FSA additionally has heavy + permission UX and no writable-handle support in Firefox/Safari. Either + can become a `ProjectStorage` implementation later. +- **UUID project identity.** Rejected: the directory name is user-visible + in Jupyter; readable notebook paths beat opaque ones. +- **A custom JupyterLite drive plugin** to avoid the contents-schema + coupling. Deferred: version-pinned against JupyterLite internals and not + needed while the schema-fidelity tests pass. +- **Run snapshots as diffs.** The skip-unchanged rule (§2) captures the + dominant saving (large data files rarely change between runs) without a + diff format. ## Consequences -- (+) Projects survive sessions; runs are comparable, reproducible history. -- (+) Notebook integration is structural, not synchronized (ADR-002 builds - directly on this). -- (+) Storage backend swap is one class, as requested. -- (−) IndexedDB quota is finite (Chrome ~60 % of free disk per origin, but - Safari is stricter). Large dump files multiplied by run history will hit - it eventually. Mitigations, in order: surface per-project storage size in - the UI, "delete run" affordance, `navigator.storage.persist()`, and later - a size-capped output policy. Quota errors must surface as a visible - warning, not silent data loss. -- (−) Writing run outputs through the contents schema stores file bodies as - JSON strings (text) — binary dump formats need base64 records, matching - JupyterLite's own convention for binary files. -- (−) The app takes on responsibility for keeping directory records - consistent with what JupyterLite expects; covered by unit tests against - recorded JupyterLite fixtures. +- (+) Projects survive sessions; runs are comparable, reproducible history + with explicit provenance (including what was *not* snapshotted). +- (+) Notebook integration is structural (ADR-002 builds on §3). +- (+) Storage backend swap is one class per backend, as requested. +- (−) IndexedDB quota is finite and JupyterLite browsing cost scales with + stored bytes. Mitigations **in priority order**: (1) size caps on + snapshot/copy with explicit gaps recorded (§2, §5); (2) skip-unchanged + snapshot dedup (§2); (3) per-project size surfaced in the UI + delete-run + affordance (ADR-003); (4) `navigator.storage.persist()` (prevents + eviction; adds no quota). A quota error during output copy marks the run + `failed` with reason `storage full` (partial outputs kept) and raises a + visible warning naming the project — never silent loss. +- (−) Base64 inflates binary outputs by ~33 % and giant single-record + strings are the known fragile spot in browsers; the size caps above are + the guard. Blob-valued records are a possible future optimization if + JupyterLite compatibility allows. +- (−) The app owns contents-schema fidelity (directory records, checkpoint + cleanup, format table), verified by unit tests against fixtures recorded + from a real JupyterLite session. +- (−) A new worker protocol (`snapshotWorkdir`) is required; its + step-callback scheduling is constrained by ASYNCIFY re-entry rules and + must be reviewed against them. diff --git a/docs/adr/002-notebook-first-class.md b/docs/adr/002-notebook-first-class.md index a7576ef6..56cee5b9 100644 --- a/docs/adr/002-notebook-first-class.md +++ b/docs/adr/002-notebook-first-class.md @@ -1,6 +1,6 @@ # ADR-002: The notebook as a first-class analysis and scripting surface -- **Status**: Proposed +- **Status**: Proposed (rev 2 — after integration review) - **Date**: 2026-07-11 - **Deciders**: Anders Hafreager, Claude - **Depends on**: ADR-001 (projects, runs, single project filesystem) @@ -8,180 +8,203 @@ ## Context Atomify embeds JupyterLite (pyodide kernel) in an iframe -(`src/containers/Notebook.tsx`) and today pushes run outputs into the -JupyterLite contents IndexedDB after each run (`syncFilesJupyterLite`). -The integration is shallow: - -- One-way: notebook edits never reach the app; the generated - `analyze.ipynb` is written once and never regenerated. -- Flat: a single global `analyze.ipynb` regardless of simulation; outputs - of successive runs overwrite each other. -- Passive: the notebook can only *look at* results. It cannot run LAMMPS. - -Meanwhile, the upstream **lammps.js** repo (which Atomify already depends -on for its WASM engine) ships a proven, deeper integration -(`lammps/lammps.js`, deployed at editor.lammps.org/notebook): - -- A Python package **`lammps-js`** (source `python/lammps/__init__.py`) - whose wheel is bundled into the JupyterLite build (`pypi/`), installable - with `%pip install lammps-js`. It mirrors the official LAMMPS Python API - (`lmp = await lammps()`, `command`, `get_thermo`, `extract_atom`, …). +(`src/containers/Notebook.tsx`). The intended integration — push run +outputs into the notebook — is shallower than it looks: since the engine +moved into a Web Worker, `syncFilesJupyterLite` only round-trips **input** +files (the main-thread FS bridge is write-only; see ADR-001 Context #5). +Run outputs (`log.lammps`, dumps) never reach the notebook today. Beyond +that: the generated `analyze.ipynb` is flat and global, successive runs +overwrite each other, and the notebook can only look at results — it +cannot run LAMMPS. + +The upstream **lammps.js** repo ships a proven deeper integration +(deployed at editor.lammps.org/notebook): + +- A Python package **`lammps-js`** (`python/lammps/__init__.py`) mirroring + the official LAMMPS Python API (`lmp = await lammps()`, `command`, + `get_thermo`, `extract_atom`, …), built as a wheel into the JupyterLite + piplite index. - Inside the pyodide worker it dynamically imports the lammps.js ES module - from `{site}/lammps/client.js` — the engine runs **in the same worker as - the kernel**, no message passing. -- A proxy Emscripten filesystem (`_MOUNT_DRIVEFS_JS`) mounts JupyterLite's - DriveFS onto the engine's `/work`, so every file LAMMPS writes lands - **next to the notebook** in the contents store, instantly visible in the - file browser. + from `{site}/lammps/client.js` — engine and kernel share one thread, no + message passing. URL derivation from the kernel URL is verified to work + under Atomify's `/atomify/jupyter/` base. +- A proxy Emscripten FS (`_MOUNT_DRIVEFS_JS`) mounts JupyterLite's DriveFS + onto the engine's workdir, mapped to **the kernel's cwd** (re-evaluated + per operation, so `os.chdir` moves it): files LAMMPS writes land next to + the notebook. - Cross-origin isolation on static hosting via a build-time service-worker - patch (`examples/notebook/coi_patch.py`) — JupyterLite's own service - worker must keep owning the scope (the DriveFS contents API rides on it), - so the generic `coi-serviceworker` shim cannot be used inside the - notebook's scope. + patch (`examples/notebook/coi_patch.py`) — JupyterLite's own SW must + keep owning its scope, so the generic `coi-serviceworker` shim cannot be + used inside it. -Atomify pins the **same** JupyterLite versions as lammps.js -(`jupyterlite-core==0.8.0`, `jupyterlite-pyodide-kernel==0.8.1` in -`jupyterlite/requirements.txt`), so these pieces port directly. +Atomify pins the same JupyterLite versions (`jupyterlite-core==0.8.0`, +`jupyterlite-pyodide-kernel==0.8.1`), so the pieces port — but review +established the port is **not** turnkey; §3 states the real supply chain. With ADR-001, projects and runs live *in* the JupyterLite contents store. -This ADR decides what the notebook experience is on top of that. +This ADR decides the notebook experience on top of that. ## Decision ### 1. The notebook operates on the project directory -The Notebook pane opens JupyterLite at the current project's notebook: -`lab/index.html?path=/analysis.ipynb`. The kernel's working -directory is therefore the project directory, and by ADR-001's layout the -notebook sees: - -``` -in.diffusion # the working tree -runs/0001-k3f9a2/log.lammps -runs/0002-8xc1p0/log.lammps -``` - -**Cross-run analysis is a glob, not a feature**: +The Notebook tab opens JupyterLite at the current project's notebook: +`lab/index.html?path=/analysis.ipynb`. The kernel cwd is the +project directory; by ADR-001's layout the notebook sees the working tree +and `runs/`. **Cross-run analysis is a glob, not a feature**: ```python -import glob, lammps_logfile +import glob, json, lammps_logfile for path in sorted(glob.glob("runs/*/log.lammps")): - log = lammps_logfile.File(path) - ... + meta = json.load(open(path.rsplit("/", 1)[0] + "/.atomify/run.json")) + log = lammps_logfile.File(path) # plot D vs meta["vars"]["T"], etc. ``` -### 2. A per-project `analysis.ipynb`, generated once - -At project creation, Atomify generates `/analysis.ipynb` from a -template (evolving `src/utils/AnalyzeNotebook.ts`). The template contains: +The Notebook tab is available only for **library projects**. Quick runs +and embeds live on `MemoryProjectStorage` (ADR-001 §6), which JupyterLite +cannot see — the tab is hidden there and appears after "Save as project". +(A scoped in-memory contents drive is possible future work.) -1. A markdown header naming the project. -2. A "latest run" cell: loads `sorted(glob("runs/*/log.lammps"))[-1]` and - plots thermo output with `lammps_logfile` + matplotlib. -3. A "compare runs" cell iterating all runs (as above). -4. A commented "drive LAMMPS from Python" cell demonstrating `lammps-js` - (see §3). +### 2. A per-project `analysis.ipynb`, generated once -The file is **owned by the user after creation** — Atomify never overwrites -an existing notebook (preserving the guarantee the current code already -makes at `src/store/simulation.ts:443-447`). Because runs are discovered by -glob at execution time, the notebook does not need regeneration when new -runs appear — this is what makes "generate once" viable. +At project creation Atomify generates `/analysis.ipynb` from a +template (evolving `src/utils/AnalyzeNotebook.ts`): a header, a +latest-run plot cell, the compare-runs glob cell above, and a commented +scripting cell (§4). Examples that ship a curated notebook copy that in +instead. The file is **owned by the user afterwards** — never overwritten +(preserving today's guarantee, `src/store/simulation.ts:443-447`). Because +runs are discovered by glob at execution time, the notebook needs no +regeneration when runs appear — that is what makes generate-once viable. -### 3. Bundle the `lammps-js` Python package: script runs from Python +`lammps_logfile` moves from a source-directory copy at the contents root +(which becomes unimportable once the notebook cwd is `/`) to the +**`lammps-logfile` wheel from PyPI in the piplite index**; the content-dir +copy is deleted. Templates start with +`%pip install lammps-logfile lammps-js`. -The Atomify JupyterLite build adopts the lammps.js recipe: +### 3. Bundle the `lammps-js` Python package — with an explicit supply chain -- Add the `lammps-js` wheel to the build (piplite index in `pypi/`), and - copy the lammps.js `dist/` to `public/jupyter/lammps/` so the package's - client-URL derivation (strip `lab/…` from the kernel URL, append - `lammps/client.js`) resolves. Both steps go into the existing - `jupyter lite build` step in `.github/workflows/deploy.yaml` and the - local `Makefile` equivalent. -- Notebooks can then run *entire simulations in the kernel*: +Goal: notebooks can run entire simulations in the kernel: ```python %pip install lammps-js from lammps import lammps -lmp = await lammps() +import os for T in [1.0, 2.0, 3.0]: + rundir = f"runs/run-T{T}" + os.makedirs(rundir, exist_ok=True) # mkdir BEFORE LAMMPS writes there + lmp = await lammps() + lmp.command(f"log {rundir}/log.lammps") lmp.command(f"variable T equal {T}") - lmp.file("in.diffusion") # reads the project's working tree + lmp.file("in.diffusion") # reads the project working tree + lmp.close() ``` -- Thanks to the DriveFS mount, files the kernel-side engine writes land in - the project directory. The template's scripting cell demonstrates the - convention of writing into `runs//` so scripted sweeps appear in - the app's run history. Run records created this way carry - `"origin": "notebook"`; a `run.json` is optional for notebook runs — the - app lists any `runs/*` directory, and fills in what metadata exists. - -This is the "script and run multiple things" capability: the app's Run -button is one producer of runs; Python loops in the notebook are another. -Both write the same layout into the same store. - -### 4. App ⇄ notebook freshness - -Because there is one store (ADR-001 §3), there is no sync protocol — only -cache refresh: - -- **App → notebook**: JupyterLite's file browser polls the contents store; - newly written run outputs appear on its next refresh. During a run the - app copies outputs from the WASM FS into `runs//` on a throttle - (every ~3 s) and at completion, so a notebook can analyze a run in - flight. -- **Notebook → app**: the app re-stats the working tree on - `visibilitychange` and on pane switches (ADR-001 §5), picking up files - created or edited from the notebook side. The runs list refreshes on the - same triggers, discovering notebook-created runs. - -No postMessage bridge is required for v1. If richer signaling is ever -needed (e.g. push-refresh of the runs list), a `BroadcastChannel` on the -same origin is the natural upgrade path; it is explicitly out of scope now. +**Supply chain** (review blocker — the wheel exists nowhere consumable +today: PyPI 404s, and the npm tarball excludes `python/`): + +- **Upstream first** (we control lammps.js): PR to publish the `lammps-js` + wheel — either to PyPI in `release.yml`, or by adding `python/` to the + npm `files` so consumers build it from `node_modules`. Same PR bumps + `python/pyproject.toml` to track the npm version (already skewed: + 1.5.0 vs 1.5.1). +- **Interim**: vendor the prebuilt wheel in-repo at + `pypi/lammps_js--py3-none-any.whl`, pinned to the npm lammps.js + version, with a comment pointing at the upstream issue. +- **Build mechanics** (verified specifics): Atomify's lite dir is the repo + root, so wheels go in `/pypi/` for auto-indexing; the built + lammps.js client is copied `node_modules/lammps.js/dist/` → + `public/jupyter/lammps/` **after** `npm install` — `deploy.yaml` + currently builds the Jupyter site before npm install, so the workflow + steps are reordered. A `make jupyter` target gives the same build + locally (none exists today; `public/jupyter` is CI-only). + +**Capability boundary** (stated honestly): the kernel client loads only +the *serial* or *KOKKOS* wasm variants (MOLECULE package set + KOKKOS); +the app's engine is the full-package `lammps-atomify` build. Projects +using packages outside that set (ReaxFF, KSPACE/water, Vashishta/silica…) +run in the app but **error in the kernel** until an upstream +`variant: "atomify"` client/python option lands (small change — the wasm +already ships in the npm dist as `./wasm-atomify`; it also means the dist +copy ships all three variants, ~55 MB, not ~40). The template's scripting +cell names this limitation. + +Notebook-created runs carry `"origin": "notebook"` when the script writes +`run.json`; the app lists any `runs/*` directory regardless, rendering +metadata-less ones as "external run" rows with mtime-derived timestamps +(ADR-001 §5). The app's Run button and sweeps (ADR-001 §7) are one +producer of runs; Python loops are another; both write the same layout. + +### 4. App ⇄ notebook freshness — corrected semantics + +One store (ADR-001 §3) means no sync protocol, only cache refresh. Review +verified `BrowserStorageDrive` holds **no in-memory cache** — but two +timing realities must be designed around, not wished away: + +- **App → notebook**: app-side writes are visible to the kernel and file + browser on their next operation (JupyterLab's browser auto-refreshes + ~10 s while visible). The app copies run outputs into `runs//` + during a run (ADR-001 §5 data path), so a notebook **can** analyze an + app-driven run in flight. +- **Notebook → app**: the DriveFS proxy flushes on **file close**. LAMMPS + holds `log.lammps` open for the whole run, so kernel-driven runs become + visible **at completion**, not in flight. Consequently: the Runs tab + discovers notebook runs on its normal refresh triggers after they + finish; and a file held open by the kernel clobbers concurrent app-side + edits at close (the per-file LWW window is the open duration — + acceptable, documented). +- The app re-stats the working tree on `visibilitychange` and pane + switches (ADR-001 §8) to pick up notebook-side edits. + +No postMessage bridge in v1; a same-origin `BroadcastChannel` is the +upgrade path if push-refresh is ever wanted. ### 5. Cross-origin isolation strategy -KOKKOS multithreading needs `crossOriginIsolated`, and the kernel-side -engine wants it too (`lammps-js` falls back to serial otherwise). The main -app uses the `coi-serviceworker` shim, but **inside the notebook's scope -JupyterLite's service worker must stay in control** (its contents API and -DriveFS depend on it). Therefore Atomify adopts lammps.js's approach: patch -the built JupyterLite `service-worker.js` to add -COOP/COEP/CORP headers to every response (port of -`examples/notebook/coi_patch.py`), applied as a post-build step. This also -resolves the known conflict noted in project memory (JupyterLite's SW -shadowing the coi shim). Serial fallback keeps notebooks functional if +KOKKOS in the kernel needs `crossOriginIsolated`. Inside the Jupyter +scope, JupyterLite's own service worker must stay in control (DriveFS's +contents API rides on it), so Atomify ports lammps.js's `coi_patch.py`: +patch the built `service-worker.js` to add COOP/COEP/CORP headers, with +the one-shot reload bootstrap injected into the app pages. Two nested +service workers then coexist: the parent app's `coi-serviceworker` +(scope `/atomify/`, COEP `credentialless`) and JupyterLite's patched SW +(scope `/atomify/jupyter/`, `require-corp`). This combination is legal +(same-origin child with its own COEP) but the first-load sequencing — +two independent SW installs, two reload bootstraps — gets an **explicit +manual test matrix** (cold load, hard refresh, SW update) on the deployed +site before release; per project memory, dev/preview cannot reproduce +GitHub Pages SW behavior. Serial fallback keeps notebooks functional if isolation fails. ## Alternatives considered -- **Keep app-side runs as the only producer and treat the notebook as a - viewer.** Rejected: parameter sweeps are the core scientific workflow the - user asked for ("script and run multiple things"), and lammps.js already - paid the engineering cost. -- **postMessage/custom JupyterLite extension for file sync.** Heavy, - version-pinned against JupyterLite internals, and unnecessary once the - store is shared (ADR-001). -- **Running app-triggered simulations inside the kernel too (one engine).** - Tempting unification, but the app's live 3D visualization is wired to its - own worker pipeline (`LammpsWorkerProxy`, per-timestep heap streaming, - ASYNCIFY constraints per project memory). Moving it into the kernel - would couple rendering to notebook lifecycle for no user benefit. -- **Regenerating `analysis.ipynb` after every run.** Rejected: clobbers - user work or forks endless copies; glob-based discovery makes it moot. +- **Notebook as viewer only.** Rejected: scripted parameter sweeps are the + core scientific workflow requested, and lammps.js already paid the cost. +- **postMessage/custom JupyterLite extension for sync.** Heavy, + version-pinned, unnecessary with a shared store. +- **One engine (run app simulations in the kernel too).** Couples the 3D + visualization pipeline (worker heap streaming, ASYNCIFY constraints) to + notebook lifecycle for no user benefit. +- **Regenerating `analysis.ipynb` per run.** Clobbers user work; glob + discovery makes it moot. +- **Bundling `lammps_logfile` as content.** That is the status quo; it + breaks under project-scoped cwds. Wheel via piplite is strictly better. ## Consequences -- (+) "Always synced with the notebook" holds by construction; the demo - story ("run, then analyze across runs in Python, then script a sweep - from Python") requires no manual file plumbing. -- (+) Atomify inherits upstream lammps.js maintenance of the Python - package instead of building its own. -- (−) The JupyterLite build grows by the lammps.js dist (~40 MB with the - KOKKOS wasm). Acceptable for a static site; can be trimmed to the serial - build if size becomes a problem. -- (−) The SW patch pins against jupyterlite-core 0.8.0 internals (same - trade lammps.js accepted; tracked upstream at jupyterlite#1409). -- (−) Two engines (app worker + kernel) can write the same project - concurrently. Run-directory naming (unique suffixes) makes collisions - structurally unlikely; per-file LWW covers the rest (ADR-001 §5). +- (+) "Always synced with the notebook" holds by construction, with the + honest caveat that kernel-side writes surface at file close. +- (+) Scripted sweeps and app sweeps produce the same on-disk shape; + analysis code never cares who ran the simulation. +- (−) Two upstream lammps.js PRs are on the critical path for the full + vision (wheel publication; `variant: "atomify"`). Both are small and we + control the repo; the vendored wheel and the stated capability boundary + keep Atomify shippable meanwhile. +- (−) The Jupyter site grows by the lammps.js dist (all wasm variants, + ~55 MB). Trim to serial+kokkos if size bites. +- (−) The SW patch pins jupyterlite-core 0.8.0 internals (tracked + upstream, jupyterlite#1409); version bumps require re-validating the + patch asserts. +- (−) JupyterLite directory listings deserialize the entire store — + ADR-001's size caps are what keep the notebook file browser fast; this + coupling is a standing constraint on how much run output we persist. diff --git a/docs/adr/003-ui-shell-and-simulations-experience.md b/docs/adr/003-ui-shell-and-simulations-experience.md index 3995b496..40bdb9c7 100644 --- a/docs/adr/003-ui-shell-and-simulations-experience.md +++ b/docs/adr/003-ui-shell-and-simulations-experience.md @@ -1,137 +1,212 @@ -# ADR-003: UI shell redesign and the "Your simulations" experience +# ADR-003: UI shell redesign and the projects experience -- **Status**: Proposed +- **Status**: Proposed (rev 2 — after product review + updated design) - **Date**: 2026-07-11 - **Deciders**: Anders Hafreager, Claude - **Depends on**: ADR-001 (projects & runs), ADR-002 (notebook) -- **Design source**: claude.ai Design project `f50a64cb-1082-404c-b41f-728c9d24e37f` - (`Atomify.dc.html`). The design is being actively iterated by the user; the - latest version is re-fetched immediately before UI implementation. This ADR - fixes the *structure*; pixels follow the design file. +- **Design source**: claude.ai Design project `f50a64cb-1082-404c-b41f-728c9d24e37f`, + file **`Atomify Projects.dc.html`** (supersedes `Atomify.dc.html`). The + design is still being iterated; the latest version is re-fetched + immediately before UI implementation. This ADR fixes structure and + behavior; pixels follow the design file. ## Context -The current shell is an antd `Layout` with a `Sider` menu driving an +The current shell is an antd `Layout` whose `Sider` menu drives an invisible antd `Tabs` (`src/App.tsx`, `src/containers/Main.tsx`, -`src/hooks/useMenuItems.tsx`). There is no notion of a simulation library: -the menu shows View / Console / Notebook / "Edit ``" / New simulation / -Examples / Share / Run controls / Settings, all scoped to the single -in-memory simulation. Files of the loaded simulation appear only as -submenu entries; there is no add/rename/delete, no list of past work, and -no visual identity beyond stock antd dark theme. - -The Claude Design draft establishes: a collapsible custom sidebar (brand, -grouped nav: *Workspace* [View / Console / Notebook / Edit], *Simulations* -[New simulation / Examples / Share], *Run* [Run-Stop primary button / Run -in cloud], footer with dark/light toggle + Settings), an Examples gallery -as a card grid with filter chips and search, a viewport HUD (Dynamics -panel: stats, speed sliders, modifier checkboxes, computes), custom-styled -modals, and a full design-token system (CSS variables, Manrope + -JetBrains Mono, light/dark themes, accent-configurable). - -ADR-001/002 introduce concepts the current UI cannot express: a project -library, a working tree per project, and a run history. +`src/hooks/useMenuItems.tsx`). There is no library, no run history, no +file management, and the run-completion moment is a Console *modal* +popping over everything (`handleRunResult` → `setShowConsole(true)`). + +The design establishes a different information architecture than the old +menu — and different from rev 1 of this ADR: there are no global +View/Console panes; those fold into a per-run **Run detail** screen. ## Decision -### 1. Information architecture +### 1. Vocabulary (glossary — binding for all UI copy) + +| Term | Meaning | UI usage | +| --- | --- | --- | +| **Project** | Named directory: working tree + runs (ADR-001) | "Projects", "New project", breadcrumb `Projects /` | +| **Run** | One execution, immutable record | "Run #005", "Runs" tab | +| **Simulation** | What a run executes | verbs/marketing only: "Run simulation" button | +| **Quick run** | Ephemeral scratch project (ADR-001 §6) | banner: "This is a quick run — files are temporary" | + +"Your simulations" from the original brief is realized as the **Projects** +library. The word "project" is the user-facing noun (per the design); +runs are numbered `#NNN`. The internal `dirName` appears as a mono-font +subtitle in the workspace header and rename dialog ("folder name stays +`diffusion`"), so the Jupyter file browser never shows a mystery name. + +### 2. Information architecture (per `Atomify Projects.dc.html`) ``` -Home ("Your simulations") ← default screen, replaces Examples as landing - ├─ New simulation (modal) ← creates project (blank / upload) - ├─ Examples gallery ← templates; opening one instantiates a project - └─ Project workspace (per project) - ├─ View (3D viewport + Dynamics HUD) - ├─ Console (LAMMPS log of current/last run) - ├─ Notebook (JupyterLite at /analysis.ipynb) - ├─ Files (working tree: open-in-editor, add, upload, rename, delete) - │ └─ Editor (Monaco, per file, autosave with saved-state indicator) - └─ Runs (history list: status, started, duration, atoms/steps, - open outputs in Files/Notebook, delete run) +Sidebar (global): Home · Example library · PROJECTS (list, + New project) + footer: theme toggle · Settings + +Home: greeting · "Continue where you left off" card (live run + status) · Create new project · Quick-run an example · + Recent projects grid · "Start from an example" row + +Example library: card grid; per card: "Use as project" | "Quick run" + +Project workspace: + header: breadcrumb · displayName · Running pill · Share · Run simulation + tabs: Files | Runs | Notebook + Files: table (name/size/modified/actions) · search · New file · + Upload · drop-anywhere · runs/ folder row (tagged) · + input-script chip on the designated script + Editor: sub-screen of Files (back button, filename, type chip, + save state, Save & run) + Runs: run rows: status icon, #NNN, when, script (mono), summary, + vars, live progress bar for running runs + Run detail: sub-screen of Runs: 3D viewport + following console strip + + right panel (status stats, speed slider, output files) · + Stop (while running) · "Analyze in notebook" + Notebook: JupyterLite iframe at /analysis.ipynb + (design shows a project-files rail; v1 relies on + JupyterLite's own file browser) ``` -- **"Your simulations" is the landing screen** (except embedded mode, which - keeps landing on View). It shows a card grid of projects — name, - description, updated-at, run count, thumbnail — plus prominent "New - simulation" and "Browse examples" entries and an empty state that points - first-time users at Examples. -- **The sidebar becomes project-aware.** The *Workspace* group (View / - Console / Notebook / Files / Runs) renders only when a project is open, - under the project's name; a "Your simulations" item above it returns - home. The *Run* group's primary button runs the open project's - `inputScript`. -- **Thumbnails**: captured from the WebGL canvas at run completion - (`canvas.toBlob` downscaled to ~320 px) and stored via `ProjectStorage` - under `.atomify/thumbnail.png`, so library cards show real results. - -### 2. Navigation state - -The app keeps its no-router architecture (it must keep working under -`/atomify/` on GitHub Pages and in embeds), but the ad-hoc -`selectedMenu` string is replaced by a typed navigation state in the store: +Sidebar projects show a **color identity dot** (from `project.json.color`, +assigned from the design palette at creation) and a pulsing dot when +running. **No thumbnails in v1** — the design uses color dots for cards; +the only image is the Home "continue" card, which uses the last run's +`frame.png` (captured from the WebGL canvas at run completion, +ADR-001 §5) when present, else a gradient. + +### 3. Navigation state + +No router (GitHub Pages subpath + embeds); the ad-hoc `selectedMenu` +string is replaced by typed state: ```ts type Screen = | { name: "home" } | { name: "examples" } - | { name: "project"; dirName: string; - pane: "view" | "console" | "notebook" | "files" | "runs"; - filePath?: string } // files pane with a file open in Monaco + | { name: "project"; dirName: string; tab: "files" | "runs" | "notebook"; + filePath?: string; // files tab: editor sub-screen + runId?: string } // runs tab: run-detail sub-screen ``` -Deep links (`?project=diffusion&pane=view`) map 1:1 onto this state so a -browser refresh restores where you were. Legacy query params (`script=`, -embed codec URLs) keep working and resolve into the same state. - -### 3. Visual implementation strategy - -- **The shell (sidebar, screens, cards, modals, HUD) is bespoke**, built - from the design file's token system: CSS custom properties - (`--bg/--surface/--text/--accent/…`) defined once with `[data-theme]` - dark/light variants, Manrope for UI, JetBrains Mono for code/paths. - antd's `Menu`-driven shell and its invisible `Tabs` are removed. -- **antd stays for commodity inner components** (upload dragger, tooltips, - notifications) restyled through its `ConfigProvider` token bridge to the - same CSS variables, until replaced opportunistically. This avoids a - big-bang rewrite of working panes (Monaco editor, dygraphs plots, omovi - canvas are untouched). -- **Theme**: a `theme` field in the persisted settings model; `data-theme` - attribute on the root; the design's light palette becomes the first - supported light mode in Atomify. The JupyterLite iframe follows via its - own theme setting (jupyterlab-night for dark) — acceptable to leave - manual in v1. - -### 4. What ships in v1 vs later +Deep links (`?project=diffusion&tab=runs&run=run-005`) map 1:1 so refresh +restores position. Legacy params (`script=`, embed codec URLs) resolve +into quick-run state. Embedded mode lands on the run detail of its +auto-started run and hides sidebar/library (as today) and the Notebook tab +(ADR-002 §1). + +### 4. The run lifecycle (the core loop, specified end-to-end) + +1. **Run simulation** (header) or **Save & run** (editor): flush editor + saves, create the run (ADR-001 §5), navigate to **Run detail** of the + new run — live viewport, following console, stats. The old + console-completion modal is **removed**. +2. **While running**: sidebar project row and header show the running + pulse; the Runs tab shows the live row with progress. Speed/UI sliders + live in the run-detail right panel. Editing files stays enabled + (runs execute from snapshots). +3. **Completion**: run-detail status pill flips (completed/failed — + failed shows the LAMMPS error from `run.json.error` prominently); + output files panel fills in; `frame.png` captured. If the user is + elsewhere in the app, a toast: "Run #005 finished — View · Analyze in + notebook". No modal. +4. **Run button with no input script** (user requirement): if + `project.json.inputScript` is unset and more than one candidate + exists, the Run button opens a "Choose input script" dialog listing + the working tree's scripts; the choice persists to `project.json`. + Exactly one candidate → auto-set silently. Zero candidates → dialog + explains and offers New file/Upload. The Files tab shows an + "input script" chip on the designated file, and each script row has + "Set as input script" + one-off "Run this file" actions. +5. **Parameter sweep** (user requirement): a dropdown on the Run button — + "Run simulation" / "Sweep variables…". The sweep dialog detects + `variable X equal …` definitions in the input script, lets the user + enter a list or range per variable (`1.0, 1.5 … 3.0`), previews the + run count, and enqueues the cartesian product as sequential runs + (ADR-001 §7). The Runs tab groups rows sharing a `sweepId` under a + collapsible header with "cancel remaining"; each row shows its `vars`. +6. **Opening a previously-run project**: workspace opens on **Files** + for never-run projects, **Runs** otherwise. Run detail of a finished + run shows `frame.png` in the viewport area (with "Run again" overlay) + plus the persisted console log and outputs — never a silent black + canvas (review blocker). Trajectories are not replayed in v1. +7. **Interrupted runs** (ADR-001 §5 reconciliation) render with a + distinct badge and keep partial outputs. + +### 5. Files, editing, deletion + +- Files tab lists the working tree plus the `runs/` folder row (tagged + "one directory per run", navigating to the Runs tab). `.atomify/` is + hidden. Run outputs open **read-only** in Monaco with a "snapshot of + run #NNN" banner; immutability of runs is a UI convention (the kernel + is not blocked from writing there — documented, not enforced). +- Editor autosaves (debounced) with visible *saving / saved / couldn't + save (retry)* states (ADR-001 §8); the design's Save button is the + immediate-flush affordance. `Save & run` = flush + run. +- **Delete project**: confirm modal states run count and total size + ("Deletes 14 runs, 320 MB — cannot be undone"); blocked while that + project has a live run. **Delete run**: plain confirm with size. No + undo/trash in v1 (decided, not deferred-by-silence). +- **Duplicate project**: copies working tree + notebook, **not** runs; + fresh dirName. (Explicitly: the copied notebook's glob sees zero runs + until the copy is run.) +- Rename edits `displayName` only (§1). +- **Storage pressure**: Settings shows a storage meter (via + `navigator.storage.estimate()`) with per-project sizes; quota failures + surface as a toast naming the project (ADR-001 consequences). + +### 6. Visual implementation strategy + +- **The shell is bespoke**: design tokens as CSS custom properties + (`--bg/--surface/--text/--accent/…`) with `[data-theme]` dark/light + variants; Manrope UI font, JetBrains Mono for code/paths. The antd + `Menu`/`Tabs` shell is removed. +- **antd remains for commodity inner components** (Upload dragger, + notifications/toasts, tooltips), bridged to the same tokens via + `ConfigProvider`; Monaco, dygraphs, omovi untouched. +- **Theme** is a persisted setting (`data-theme` on root); the design's + light palette becomes Atomify's first light mode. The JupyterLite + iframe keeps its own theme setting in v1. +- The WASM-loading progress must not block Home/library browsing (today a + modal blocks everything until the engine loads) — the engine is needed + only to *run*, not to browse, edit, or read notebooks. + +### 7. v1 vs deferred | In v1 | Deferred | | --- | --- | -| Home grid, create/open/rename/duplicate/delete project | Project search/sort/tags | -| Files pane with full file CRUD + Monaco autosave | Drag-drop reorder, folders in working tree | -| Runs pane (list, open outputs, delete run) | Run diffing/comparison UI | -| Design tokens, dark + light themes, collapsible sidebar | Accent color picker | -| Examples gallery restyled per design | Example authoring from a project | +| Home, library, create (blank/example/upload), open, rename, duplicate, delete | Project tags/search/sort beyond updated-at | +| Files CRUD + Monaco autosave + input-script selection | Working-tree subfolders UI, drag-reorder | +| Runs list + run detail + interrupted reconciliation + delete run | Run comparison/diff UI, run notes | +| Variable sweeps (list/range, sequential queue) | Parallel sweep execution, sweep charts | +| Quick run + Save as project; share links open as quick runs | Publishing a project as a shareable example | +| Design tokens, dark + light, storage meter | Accent color picker, thumbnails on cards | ## Alternatives considered -- **Adopt a router (react-router).** The screen graph is small and the - deep-link mapping is trivial; a router adds a dependency and base-path - friction on GitHub Pages for no structural gain at this size. -- **Restyle antd in place instead of a bespoke shell.** The design's - sidebar, cards, and HUD do not map onto antd `Menu`/`Tabs` idioms; - fighting the component library costs more than owning ~600 lines of - layout primitives. -- **Keep Examples as the landing page.** Rejected: the product story is - "your work lives here"; examples are an on-ramp, reachable in one click - and via the empty state. +- **react-router**: small screen graph, trivial deep-link mapping; a + router adds base-path friction on Pages for no structural gain. +- **Restyle antd in place**: the sidebar/cards/run-rows don't map onto + antd idioms; owning ~600 lines of layout primitives is cheaper than + fighting the library. +- **Global View/Console panes (rev 1 of this ADR)**: superseded by the + design — run-scoped detail makes "which run am I looking at?" + unaskable, and kills the completion-modal pattern naturally. +- **Thumbnails on all cards (rev 1)**: dropped in favor of the design's + color-dot identity; `frame.png` capture is kept only where it earns + its cost (continue card, run detail). ## Consequences -- (+) The app reads as a product: work persists visibly, history is - browsable, files are manageable, and the visual language is owned. -- (+) Typed navigation state kills the `"file"+fileName` string-key hack - and makes project/pane deep-linking testable. -- (−) `useMenuItems.tsx`, `Main.tsx`, and `App.tsx` are substantially - rewritten; embed mode and the AutoStart flow must be regression-tested - (they bypass the home screen). -- (−) Two styling systems (tokens + antd bridge) coexist during the - transition; contained by keeping antd usage inside panes. +- (+) The core loop is fully specified: create → edit (autosave) → run + (live detail) → analyze (notebook glob) → iterate (sweeps, history). +- (+) Typed navigation kills the `"file"+fileName` string-key hack; + every screen is deep-linkable and testable. +- (−) `App.tsx`, `Main.tsx`, `useMenuItems.tsx` are substantially + rewritten; embed + AutoStart flows need regression tests (they bypass + Home and land on run detail). +- (−) Two styling systems coexist during transition (tokens + antd + bridge), contained inside panes. +- (−) Run detail for historical runs is a still image + logs in v1; + trajectory replay stays future work. diff --git a/docs/adr/README.md b/docs/adr/README.md index 9bd2e960..e1161e79 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -7,4 +7,4 @@ context, the decision, alternatives considered, and consequences. | --- | --- | --- | | [ADR-001](001-projects-runs-and-storage.md) | Projects, runs, and a single project filesystem | Proposed | | [ADR-002](002-notebook-first-class.md) | The notebook as a first-class analysis and scripting surface | Proposed | -| [ADR-003](003-ui-shell-and-simulations-experience.md) | UI shell redesign and the "Your simulations" experience | Proposed | +| [ADR-003](003-ui-shell-and-simulations-experience.md) | UI shell redesign and the projects experience | Proposed | From daeeb31879a9f4621bd0f80bbd6cca490b3d9f3a Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 09:41:40 +0200 Subject: [PATCH 03/34] Apply verification-pass fixes to ADRs (final documentation round) Co-Authored-By: Claude Fable 5 --- docs/adr/001-projects-runs-and-storage.md | 34 +++++++++++++------ docs/adr/002-notebook-first-class.md | 14 +++++--- ...003-ui-shell-and-simulations-experience.md | 5 +-- 3 files changed, 36 insertions(+), 17 deletions(-) diff --git a/docs/adr/001-projects-runs-and-storage.md b/docs/adr/001-projects-runs-and-storage.md index 547290ec..0d4788c2 100644 --- a/docs/adr/001-projects-runs-and-storage.md +++ b/docs/adr/001-projects-runs-and-storage.md @@ -143,16 +143,21 @@ Naming rules: - **Run directories are `run-NNN`** (zero-padded sequence, per the design). Allocation takes the max existing number + 1; if the allocated directory already exists (concurrent creation from the notebook side), allocation - retries with the next number. Runs created by external tools with other - names are still listed (§6). + retries with the next number. The app **claims the directory by writing + its directory record before anything else**, and the notebook template + recommends non-numeric run names (`run-T3.0`), so simultaneous + allocation cannot silently merge two runs. Runs created by external + tools with other names are still listed (§5). **Snapshot contract** (explicit allow/deny — review finding): a run snapshot copies the working tree **excluding** `runs/`, `.atomify/`, and `*.ipynb`. Notebooks are analysis, not simulation input; `runs/` excluded -prevents recursive snapshot growth. To keep snapshots cheap, a file is -**skipped when byte-identical to the same path in the previous run's -snapshot** — the record is written as a copy of the previous record -(storage-level dedup can improve this later without changing the layout). +prevents recursive snapshot growth. When a file is byte-identical to the +same path in the previous run's snapshot, the copy skips re-reading and +re-encoding the working-tree body — but **the bytes are still duplicated +per run** (the contents schema stores full content per path-keyed record). +True dedup (content-addressed records behind the storage interface) is +future work; the size caps below are what actually bound quota today. Files larger than a threshold (default 64 MB) are not snapshotted; the run records their names + hashes in `run.json` so provenance gaps are explicit. @@ -293,9 +298,15 @@ Flow per run: `frame.png`. **Reconciliation** (crash/zombie handling — review finding): on project -open and on library listing, any run with `status: "running"` that this +open and on library listing, a run with `status: "running"` that this session does not own is rewritten to `"interrupted"` (partial outputs -kept, badge in UI). A top-level directory without `.atomify/project.json` +kept, badge in UI) — with an **ownership/grace rule** so live notebook +runs are not killed: app/sweep-origin runs reconcile immediately (one app +session owns the engine), while `origin: "notebook"` runs reconcile only +when their `run.json` `last_modified` is older than a grace window +(10 min). Runs in `runs/*` without any `run.json` (external/scripted) +are listed as "external run" rows with mtime-derived timestamps, never +reconciled. A top-level directory without `.atomify/project.json` is not listed; the library offers "recover as project" for such orphans. If the project directory disappears mid-run (deleted from the Jupyter side), the run is canceled and its output copies stop. @@ -307,7 +318,7 @@ side), the run is canceled and its output copies stop. | **New project** | Modal: name + start blank / from example / drop files. Creates a persisted project. | | **Example → "Use as project"** | Instantiates a project from the example's files. | | **Example → "Quick run"** | Runs immediately in a **scratch project on `MemoryProjectStorage`** — nothing persisted, banner offers **"Save as project"**, which materializes the working tree *and completed runs* into the library. Closing the tab discards it. This keeps tire-kicking from littering the library (review finding) while "everything in the library persists" stays absolutely true. | -| **Share link** | Normal mode: opens as a quick run (same banner/save flow). Embedded mode: `MemoryProjectStorage`, no save affordance, Notebook hidden (ADR-002 §6). | +| **Share link** | Normal mode: opens as a quick run (same banner/save flow). Embedded mode: `MemoryProjectStorage`, no save affordance, Notebook hidden (ADR-002 §1). | ### 7. Parameter sweeps @@ -326,7 +337,10 @@ the cartesian product as a **sweep** — one run per combination: - The app executes sweep runs **sequentially** (one WASM engine); the Runs tab shows queued entries (`status: "queued"` in memory only — never persisted, so no zombie queue states) with progress per run and - cancel-remaining. + cancel-remaining. Consequence of in-memory queuing: a page refresh + mid-sweep drops the not-yet-started combinations; on reload the app + shows "sweep interrupted — N runs not started" with a re-enqueue + affordance (completed runs are untouched). - Variable values are injected per run via the vars-wrapper mechanism, and recorded in `run.json.vars` — the Runs list surfaces them (ADR-003), which is what makes ten near-identical rows distinguishable. diff --git a/docs/adr/002-notebook-first-class.md b/docs/adr/002-notebook-first-class.md index 56cee5b9..7a5e26b6 100644 --- a/docs/adr/002-notebook-first-class.md +++ b/docs/adr/002-notebook-first-class.md @@ -90,14 +90,18 @@ Goal: notebooks can run entire simulations in the kernel: %pip install lammps-js from lammps import lammps import os +home = os.getcwd() for T in [1.0, 2.0, 3.0]: rundir = f"runs/run-T{T}" os.makedirs(rundir, exist_ok=True) # mkdir BEFORE LAMMPS writes there - lmp = await lammps() - lmp.command(f"log {rundir}/log.lammps") - lmp.command(f"variable T equal {T}") - lmp.file("in.diffusion") # reads the project working tree - lmp.close() + os.chdir(rundir) # dumps/write_data land in the run, + try: # not the working tree (DriveFS + lmp = await lammps() # follows the kernel cwd) + lmp.command(f"variable T equal {T}") + lmp.file("../../in.diffusion") # script from the working tree + lmp.close() + finally: + os.chdir(home) ``` **Supply chain** (review blocker — the wheel exists nowhere consumable diff --git a/docs/adr/003-ui-shell-and-simulations-experience.md b/docs/adr/003-ui-shell-and-simulations-experience.md index 40bdb9c7..d12195ad 100644 --- a/docs/adr/003-ui-shell-and-simulations-experience.md +++ b/docs/adr/003-ui-shell-and-simulations-experience.md @@ -129,8 +129,9 @@ auto-started run and hides sidebar/library (as today) and the Notebook tab 6. **Opening a previously-run project**: workspace opens on **Files** for never-run projects, **Runs** otherwise. Run detail of a finished run shows `frame.png` in the viewport area (with "Run again" overlay) - plus the persisted console log and outputs — never a silent black - canvas (review blocker). Trajectories are not replayed in v1. + plus the console (rendered from the run's persisted `log.lammps`) and + outputs — never a silent black canvas (review blocker). Trajectories + are not replayed in v1. 7. **Interrupted runs** (ADR-001 §5 reconciliation) render with a distinct badge and keep partial outputs. From 0c3a48e193916ac63c4faf08207a3fbe4272b9fb Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 09:55:59 +0200 Subject: [PATCH 04/34] Add project storage layer and worker run-outputs protocol (ADR-001) - ProjectStorage interface with ContentsProjectStorage over the JupyterLite contents schema (IndexedDB via localforage, or in-memory for quick runs/embeds/tests); checkpoint-store cleanup on delete/rename - runs module: run-NNN allocation, snapshot contract, run.json metadata, zombie reconciliation with notebook grace window, sweep expansion - worker protocol: snapshotWorkdir command reads run outputs out of the worker wasm FS (MEMFS reads are asyncify-safe), size-aware - vitest: exclude .claude/ worktrees from test discovery Co-Authored-By: Claude Fable 5 --- src/storage/contentsProjectStorage.test.ts | 222 +++++++++++++ src/storage/contentsProjectStorage.ts | 358 +++++++++++++++++++++ src/storage/contentsSchema.test.ts | 87 +++++ src/storage/contentsSchema.ts | 224 +++++++++++++ src/storage/index.ts | 22 ++ src/storage/kv.ts | 81 +++++ src/storage/runs.test.ts | 203 ++++++++++++ src/storage/runs.ts | 294 +++++++++++++++++ src/storage/slug.ts | 32 ++ src/storage/types.ts | 130 ++++++++ src/wasm/LammpsWorkerProxy.ts | 46 +++ src/wasm/lammps.worker.ts | 52 +++ src/wasm/workerMessages.ts | 25 +- vitest.config.mts | 3 + 14 files changed, 1777 insertions(+), 2 deletions(-) create mode 100644 src/storage/contentsProjectStorage.test.ts create mode 100644 src/storage/contentsProjectStorage.ts create mode 100644 src/storage/contentsSchema.test.ts create mode 100644 src/storage/contentsSchema.ts create mode 100644 src/storage/index.ts create mode 100644 src/storage/kv.ts create mode 100644 src/storage/runs.test.ts create mode 100644 src/storage/runs.ts create mode 100644 src/storage/slug.ts create mode 100644 src/storage/types.ts diff --git a/src/storage/contentsProjectStorage.test.ts b/src/storage/contentsProjectStorage.test.ts new file mode 100644 index 00000000..7331f932 --- /dev/null +++ b/src/storage/contentsProjectStorage.test.ts @@ -0,0 +1,222 @@ +import { describe, expect, it, beforeEach } from "vitest"; +import { ContentsProjectStorage } from "./contentsProjectStorage"; +import type { ContentsRecord } from "./contentsSchema"; +import { MemoryStore } from "./kv"; +import type { ProjectMeta, ProjectStorage } from "./types"; + +describe("ContentsProjectStorage", () => { + let files: MemoryStore; + let checkpoints: MemoryStore; + let storage: ProjectStorage; + + beforeEach(() => { + files = new MemoryStore(); + checkpoints = new MemoryStore(); + storage = new ContentsProjectStorage(files, checkpoints); + }); + + describe("projects", () => { + it("creates a project with a slugged dirName and skeleton records", async () => { + const meta = await storage.createProject({ + displayName: "Diffusion coefficients", + }); + + expect(meta.dirName).toBe("diffusion-coefficients"); + expect(meta.displayName).toBe("Diffusion coefficients"); + expect(meta.color).toBeDefined(); + + // Skeleton: project dir, .atomify dir, runs dir, project.json. + const dir = await files.getItem("diffusion-coefficients"); + expect(dir?.type).toBe("directory"); + const runs = await files.getItem( + "diffusion-coefficients/runs", + ); + expect(runs?.type).toBe("directory"); + const metaRecord = await files.getItem( + "diffusion-coefficients/.atomify/project.json", + ); + expect(metaRecord?.format).toBe("json"); + }); + + it("unique-ifies colliding dirNames", async () => { + const first = await storage.createProject({ displayName: "Diffusion" }); + const second = await storage.createProject({ displayName: "Diffusion" }); + expect(first.dirName).toBe("diffusion"); + expect(second.dirName).toBe("diffusion-2"); + }); + + it("lists only directories holding a project.json", async () => { + await storage.createProject({ displayName: "Real project" }); + // Legacy junk from the pre-projects one-way sync. + await files.setItem("analyze.ipynb", { + name: "analyze.ipynb", + path: "analyze.ipynb", + type: "notebook", + } as ContentsRecord); + await files.setItem("legacy-sim/log.lammps", { + name: "log.lammps", + path: "legacy-sim/log.lammps", + type: "file", + } as ContentsRecord); + + const projects = await storage.listProjects(); + expect(projects.map((p) => p.dirName)).toEqual(["real-project"]); + }); + + it("updates meta but refuses dirName changes", async () => { + const created = await storage.createProject({ displayName: "Original" }); + const updated = await storage.updateProjectMeta(created.dirName, { + displayName: "Renamed study", + }); + expect(updated.displayName).toBe("Renamed study"); + expect(updated.dirName).toBe("original"); + + await expect( + storage.updateProjectMeta(created.dirName, { + dirName: "hijack", + } as Partial), + ).rejects.toThrow(/immutable/); + }); + + it("deletes recursively including checkpoints", async () => { + const { dirName } = await storage.createProject({ displayName: "Doomed" }); + await storage.write(dirName, "in.script", "run 100"); + await checkpoints.setItem(`${dirName}/in.script`, [{ id: "0" }]); + + await storage.deleteProject(dirName); + + expect(await files.keys()).toEqual([]); + expect(await checkpoints.keys()).toEqual([]); + }); + }); + + describe("files", () => { + let dirName: string; + + beforeEach(async () => { + const meta = await storage.createProject({ displayName: "Proj" }); + dirName = meta.dirName; + }); + + it("round-trips text files and preserves created on rewrite", async () => { + const first = await storage.write(dirName, "in.diffusion", "units lj"); + const record1 = await files.getItem( + `${dirName}/in.diffusion`, + ); + await storage.write(dirName, "in.diffusion", "units metal"); + const record2 = await files.getItem( + `${dirName}/in.diffusion`, + ); + + expect(await storage.read(dirName, "in.diffusion")).toBe("units metal"); + expect(first.format).toBe("text"); + expect(record2?.created).toBe(record1?.created); + }); + + it("stores notebooks as parsed JSON and reads them back as JSON text", async () => { + const notebook = JSON.stringify({ cells: [], nbformat: 4 }); + await storage.write(dirName, "analysis.ipynb", notebook); + + const record = await files.getItem( + `${dirName}/analysis.ipynb`, + ); + expect(record?.type).toBe("notebook"); + expect(record?.format).toBe("json"); + expect(typeof record?.content).toBe("object"); + + const read = await storage.read(dirName, "analysis.ipynb"); + expect(JSON.parse(read as string)).toEqual({ cells: [], nbformat: 4 }); + }); + + it("round-trips binary files as base64", async () => { + const bytes = new Uint8Array([137, 80, 78, 71, 0, 255]); + await storage.write(dirName, ".atomify/frame.png", bytes); + + const record = await files.getItem( + `${dirName}/.atomify/frame.png`, + ); + expect(record?.format).toBe("base64"); + + const read = await storage.read(dirName, ".atomify/frame.png"); + expect(read).toBeInstanceOf(Uint8Array); + expect([...(read as Uint8Array)]).toEqual([137, 80, 78, 71, 0, 255]); + }); + + it("auto-creates parent directory records on deep writes", async () => { + await storage.write(dirName, "runs/run-001/log.lammps", "LAMMPS log"); + const runDir = await files.getItem( + `${dirName}/runs/run-001`, + ); + expect(runDir?.type).toBe("directory"); + }); + + it("lists direct children only, synthesizing implied directories", async () => { + await storage.write(dirName, "in.diffusion", "units lj"); + await storage.write(dirName, "runs/run-001/log.lammps", "log"); + // Simulate an externally-written deep key with no parent record. + await files.removeItem(`${dirName}/runs/run-001`); + + const root = await storage.list(dirName); + const names = root.map((stat) => stat.path); + expect(names).toContain("in.diffusion"); + expect(names).toContain("runs"); + expect(names).toContain(".atomify"); + expect(names).not.toContain("runs/run-001/log.lammps"); + + const runs = await storage.list(dirName, "runs"); + expect(runs).toHaveLength(1); + expect(runs[0].path).toBe("runs/run-001"); + expect(runs[0].type).toBe("directory"); + }); + + it("renames files, carrying checkpoints along", async () => { + await storage.write(dirName, "in.old", "run 1"); + await checkpoints.setItem(`${dirName}/in.old`, [{ id: "0" }]); + + await storage.rename(dirName, "in.old", "in.new"); + + expect(await storage.stat(dirName, "in.old")).toBeNull(); + expect(await storage.read(dirName, "in.new")).toBe("run 1"); + expect(await checkpoints.getItem(`${dirName}/in.old`)).toBeNull(); + expect(await checkpoints.getItem(`${dirName}/in.new`)).not.toBeNull(); + }); + + it("renames directories recursively", async () => { + await storage.write(dirName, "data/one.txt", "1"); + await storage.write(dirName, "data/deep/two.txt", "2"); + + await storage.rename(dirName, "data", "inputs"); + + expect(await storage.read(dirName, "inputs/one.txt")).toBe("1"); + expect(await storage.read(dirName, "inputs/deep/two.txt")).toBe("2"); + expect(await storage.stat(dirName, "data")).toBeNull(); + }); + + it("removes directories recursively including checkpoints", async () => { + await storage.write(dirName, "runs/run-001/log.lammps", "log"); + await checkpoints.setItem(`${dirName}/runs/run-001/log.lammps`, [ + { id: "0" }, + ]); + + await storage.remove(dirName, "runs/run-001"); + + expect(await storage.stat(dirName, "runs/run-001")).toBeNull(); + expect(await storage.stat(dirName, "runs/run-001/log.lammps")).toBeNull(); + expect( + await checkpoints.getItem(`${dirName}/runs/run-001/log.lammps`), + ).toBeNull(); + }); + + it("rejects paths JupyterLite cannot handle", async () => { + await expect(storage.write(dirName, "bad%name", "x")).rejects.toThrow( + /%/, + ); + await expect(storage.write(dirName, "../escape", "x")).rejects.toThrow( + /segment/, + ); + await expect(storage.write(dirName, "/absolute", "x")).rejects.toThrow( + /slash/, + ); + }); + }); +}); diff --git a/src/storage/contentsProjectStorage.ts b/src/storage/contentsProjectStorage.ts new file mode 100644 index 00000000..058b4bcc --- /dev/null +++ b/src/storage/contentsProjectStorage.ts @@ -0,0 +1,358 @@ +/** + * ProjectStorage over a JupyterLite-contents-schema key-value store + * (ADR-001 §3-§4). + * + * One implementation, two backends: localforage on the "JupyterLite Storage" + * IndexedDB (the persistent library — shared with the notebook by + * construction) or an in-memory store (quick runs, embeds, tests). Listing + * never bulk-reads file bodies beyond the directory being listed: project + * and run discovery scan keys and read only the small `.atomify/*.json` + * records. + */ + +import { + buildDirectoryRecord, + buildFileRecord, + classifyPath, + recordToReadResult, + recordToStat, + validateRelativePath, + type ContentsRecord, +} from "./contentsSchema"; +import { createJupyterCheckpointsStore, createJupyterFilesStore, MemoryStore } from "./kv"; +import type { KeyValueStore } from "./kv"; +import { uniqueSlug } from "./slug"; +import type { + FileStat, + NewProjectMeta, + ProjectMeta, + ProjectStorage, +} from "./types"; + +export const PROJECT_META_PATH = ".atomify/project.json"; + +/** Colors assigned round-robin to new projects (design palette). */ +const PROJECT_COLORS = [ + "#3F6EFF", + "#7C3AED", + "#10B981", + "#06B6D4", + "#F43F5E", + "#FFB020", +]; + +export class ContentsProjectStorage implements ProjectStorage { + constructor( + private readonly files: KeyValueStore, + private readonly checkpoints: KeyValueStore, + ) {} + + // --- Projects ----------------------------------------------------------- + + async listProjects(): Promise { + const keys = await this.files.keys(); + const topLevel = new Set(); + for (const key of keys) { + const slash = key.indexOf("/"); + topLevel.add(slash === -1 ? key : key.slice(0, slash)); + } + const projects: ProjectMeta[] = []; + for (const dirName of topLevel) { + const meta = await this.readProjectMeta(dirName); + if (meta) { + projects.push(meta); + } + } + return projects.sort((a, b) => a.displayName.localeCompare(b.displayName)); + } + + async createProject(meta: NewProjectMeta): Promise { + const keys = await this.files.keys(); + const taken = new Set( + keys.map((key) => { + const slash = key.indexOf("/"); + return slash === -1 ? key : key.slice(0, slash); + }), + ); + const dirName = uniqueSlug(meta.displayName, taken); + const color = + meta.color ?? + PROJECT_COLORS[taken.size % PROJECT_COLORS.length]; + + const projectMeta: ProjectMeta = { + schemaVersion: 1, + dirName, + displayName: meta.displayName, + description: meta.description, + createdAt: new Date().toISOString(), + inputScript: meta.inputScript, + color, + source: meta.source, + }; + + await this.writeDirectoryRecord(dirName); + await this.writeDirectoryRecord(`${dirName}/.atomify`); + await this.writeDirectoryRecord(`${dirName}/runs`); + await this.write( + dirName, + PROJECT_META_PATH, + JSON.stringify(projectMeta, null, 2), + ); + return projectMeta; + } + + async updateProjectMeta( + dirName: string, + patch: Partial>, + ): Promise { + if ("dirName" in patch) { + throw new Error("dirName is immutable (ADR-001 §2)"); + } + const existing = await this.readProjectMeta(dirName); + if (!existing) { + throw new Error(`Not a project: ${dirName}`); + } + const updated: ProjectMeta = { ...existing, ...patch, dirName }; + await this.write( + dirName, + PROJECT_META_PATH, + JSON.stringify(updated, null, 2), + ); + return updated; + } + + async deleteProject(dirName: string): Promise { + validateRelativePath(dirName); + const prefix = `${dirName}/`; + const keys = await this.files.keys(); + for (const key of keys) { + if (key === dirName || key.startsWith(prefix)) { + await this.files.removeItem(key); + await this.checkpoints.removeItem(key); + } + } + } + + // --- Files ---------------------------------------------------------------- + + async list(dirName: string, subdir?: string): Promise { + validateRelativePath(dirName); + if (subdir !== undefined) { + validateRelativePath(subdir); + } + const base = subdir ? `${dirName}/${subdir}` : dirName; + const prefix = `${base}/`; + const keys = await this.files.keys(); + + // Direct children, plus directories implied by deeper keys whose own + // record is missing (defensive against externally-written trees). + const children = new Set(); + for (const key of keys) { + if (!key.startsWith(prefix)) { + continue; + } + const rest = key.slice(prefix.length); + const slash = rest.indexOf("/"); + children.add(slash === -1 ? rest : rest.slice(0, slash)); + } + + const stats: FileStat[] = []; + for (const child of children) { + const fullPath = `${prefix}${child}`; + const relative = fullPath.slice(dirName.length + 1); + const record = await this.files.getItem(fullPath); + if (record) { + stats.push(recordToStat(record, relative)); + } else { + // Implied directory: a deeper key exists but no record for this level. + stats.push({ + path: relative, + type: "directory", + format: "json", + mimetype: "application/json", + lastModified: new Date(0).toISOString(), + size: 0, + }); + } + } + return stats.sort((a, b) => a.path.localeCompare(b.path)); + } + + async read(dirName: string, path: string): Promise { + const record = await this.requireRecord(dirName, path); + return recordToReadResult(record); + } + + async write( + dirName: string, + path: string, + content: string | Uint8Array, + ): Promise { + validateRelativePath(dirName); + validateRelativePath(path); + const fullPath = `${dirName}/${path}`; + await this.ensureParents(fullPath); + const existing = await this.files.getItem(fullPath); + const record = buildFileRecord(fullPath, content, existing?.created); + await this.files.setItem(fullPath, record); + return recordToStat(record, path); + } + + async rename(dirName: string, from: string, to: string): Promise { + validateRelativePath(dirName); + validateRelativePath(from); + validateRelativePath(to); + const fromFull = `${dirName}/${from}`; + const toFull = `${dirName}/${to}`; + const record = await this.requireRecord(dirName, from); + + if (record.type === "directory") { + const prefix = `${fromFull}/`; + const keys = await this.files.keys(); + for (const key of keys) { + if (key === fromFull || key.startsWith(prefix)) { + const target = toFull + key.slice(fromFull.length); + await this.moveKey(key, target); + } + } + return; + } + await this.ensureParents(toFull); + await this.moveKey(fromFull, toFull); + } + + async remove(dirName: string, path: string): Promise { + validateRelativePath(dirName); + validateRelativePath(path); + const fullPath = `${dirName}/${path}`; + const record = await this.files.getItem(fullPath); + if (record?.type === "directory") { + const prefix = `${fullPath}/`; + const keys = await this.files.keys(); + for (const key of keys) { + if (key.startsWith(prefix)) { + await this.files.removeItem(key); + await this.checkpoints.removeItem(key); + } + } + } + await this.files.removeItem(fullPath); + await this.checkpoints.removeItem(fullPath); + } + + async stat(dirName: string, path: string): Promise { + validateRelativePath(dirName); + validateRelativePath(path); + const record = await this.files.getItem( + `${dirName}/${path}`, + ); + return record ? recordToStat(record, path) : null; + } + + // --- Internals ------------------------------------------------------------ + + private async readProjectMeta(dirName: string): Promise { + const record = await this.files.getItem( + `${dirName}/${PROJECT_META_PATH}`, + ); + if (!record || record.type === "directory") { + return null; + } + try { + const raw = recordToReadResult(record); + const meta = JSON.parse( + typeof raw === "string" ? raw : new TextDecoder().decode(raw), + ) as ProjectMeta; + // The path is the identity; a copied/moved project.json must not + // impersonate another dirName. + return { ...meta, dirName }; + } catch { + return null; + } + } + + /** Create directory records for every missing ancestor of fullPath. */ + private async ensureParents(fullPath: string): Promise { + const segments = fullPath.split("/"); + let current = ""; + for (let i = 0; i < segments.length - 1; i++) { + current = current ? `${current}/${segments[i]}` : segments[i]; + const existing = await this.files.getItem(current); + if (!existing) { + await this.files.setItem(current, buildDirectoryRecord(current)); + } + } + } + + private async writeDirectoryRecord(fullPath: string): Promise { + const existing = await this.files.getItem(fullPath); + if (!existing) { + await this.files.setItem( + fullPath, + buildDirectoryRecord(fullPath), + ); + } + } + + private async requireRecord( + dirName: string, + path: string, + ): Promise { + validateRelativePath(dirName); + validateRelativePath(path); + const record = await this.files.getItem( + `${dirName}/${path}`, + ); + if (!record) { + throw new Error(`File not found: ${dirName}/${path}`); + } + return record; + } + + private async moveKey(from: string, to: string): Promise { + const record = await this.files.getItem(from); + if (!record) { + return; + } + const moved: ContentsRecord = { + ...record, + path: to, + name: to.slice(to.lastIndexOf("/") + 1), + last_modified: new Date().toISOString(), + }; + // A rename that changes the extension changes how the record should be + // classified; rebuild file records so format/mimetype stay truthful. + if (moved.type !== "directory") { + const kind = classifyPath(to); + if (kind.format !== (record.format ?? "text")) { + const raw = recordToReadResult(record); + await this.files.setItem(to, buildFileRecord(to, raw, record.created)); + } else { + moved.type = kind.type; + await this.files.setItem(to, moved); + } + } else { + await this.files.setItem(to, moved); + } + await this.files.removeItem(from); + // Checkpoints follow the file (mirrors BrowserStorageDrive's own rename). + const checkpoint = await this.checkpoints.getItem(from); + if (checkpoint !== null) { + await this.checkpoints.setItem(to, checkpoint); + } + await this.checkpoints.removeItem(from); + } +} + +/** The persistent library: JupyterLite's own contents database. */ +export function createIndexedDbProjectStorage(): ProjectStorage { + return new ContentsProjectStorage( + createJupyterFilesStore(), + createJupyterCheckpointsStore(), + ); +} + +/** Scratch space for quick runs, embeds, and tests — invisible to Jupyter. */ +export function createMemoryProjectStorage(): ProjectStorage { + return new ContentsProjectStorage(new MemoryStore(), new MemoryStore()); +} diff --git a/src/storage/contentsSchema.test.ts b/src/storage/contentsSchema.test.ts new file mode 100644 index 00000000..64c8c61a --- /dev/null +++ b/src/storage/contentsSchema.test.ts @@ -0,0 +1,87 @@ +import { describe, expect, it } from "vitest"; +import { + base64ToBytes, + bytesToBase64, + bytesToWriteContent, + classifyPath, + validateRelativePath, +} from "./contentsSchema"; +import { slugify, uniqueSlug } from "./slug"; + +describe("classifyPath", () => { + it("classifies notebooks, json, binaries, and defaults to text", () => { + expect(classifyPath("analysis.ipynb").type).toBe("notebook"); + expect(classifyPath("analysis.ipynb").format).toBe("json"); + expect(classifyPath("config.json").format).toBe("json"); + expect(classifyPath(".atomify/frame.png").format).toBe("base64"); + expect(classifyPath("frame.png").mimetype).toBe("image/png"); + // LAMMPS files have arbitrary "extensions" — they must default to text. + expect(classifyPath("in.diffusion").format).toBe("text"); + expect(classifyPath("data.spce").format).toBe("text"); + expect(classifyPath("log.lammps").format).toBe("text"); + expect(classifyPath("dump.custom").format).toBe("text"); + }); +}); + +describe("validateRelativePath", () => { + it("accepts normal project paths", () => { + expect(() => validateRelativePath("in.diffusion")).not.toThrow(); + expect(() => + validateRelativePath("runs/run-001/.atomify/run.json"), + ).not.toThrow(); + }); + + it("rejects %, dot-segments, and leading/trailing slashes", () => { + expect(() => validateRelativePath("50%done.txt")).toThrow(/%/); + expect(() => validateRelativePath("a/../b")).toThrow(/segment/); + expect(() => validateRelativePath("./a")).toThrow(/segment/); + expect(() => validateRelativePath("/a")).toThrow(/slash/); + expect(() => validateRelativePath("a/")).toThrow(/slash/); + expect(() => validateRelativePath("a//b")).toThrow(/segment/); + }); +}); + +describe("base64 round-trip", () => { + it("round-trips bytes of every value", () => { + const bytes = new Uint8Array(256).map((_, i) => i); + expect([...base64ToBytes(bytesToBase64(bytes))]).toEqual([...bytes]); + }); + + it("handles buffers larger than the chunk size", () => { + const bytes = new Uint8Array(0x8000 * 2 + 17).fill(42); + const decoded = base64ToBytes(bytesToBase64(bytes)); + expect(decoded.length).toBe(bytes.length); + expect(decoded[decoded.length - 1]).toBe(42); + }); +}); + +describe("bytesToWriteContent", () => { + it("decodes text files to strings and keeps binaries as bytes", () => { + const text = bytesToWriteContent( + "log.lammps", + new TextEncoder().encode("Step Temp"), + ); + expect(text).toBe("Step Temp"); + + const binary = bytesToWriteContent("frame.png", new Uint8Array([1, 2])); + expect(binary).toBeInstanceOf(Uint8Array); + }); +}); + +describe("slug", () => { + it("slugifies display names to [a-z0-9-]", () => { + expect(slugify("Diffusion coefficients")).toBe("diffusion-coefficients"); + expect(slugify("Nanoporous SiO₂!")).toBe("nanoporous-sio2"); + expect(slugify("Étude thermique")).toBe("etude-thermique"); + }); + + it("falls back for names with no usable characters", () => { + expect(slugify("🔥🔥🔥")).toBe("project"); + }); + + it("unique-ifies against taken slugs", () => { + const taken = new Set(["diffusion", "diffusion-2"]); + expect(uniqueSlug("Diffusion", taken)).toBe("diffusion-3"); + expect(uniqueSlug("Fresh", taken)).toBe("fresh"); + }); +}); diff --git a/src/storage/contentsSchema.ts b/src/storage/contentsSchema.ts new file mode 100644 index 00000000..1ec0a6f4 --- /dev/null +++ b/src/storage/contentsSchema.ts @@ -0,0 +1,224 @@ +/** + * JupyterLite contents-schema helpers (ADR-001 §3). + * + * The project filesystem is stored in the same IndexedDB database + * JupyterLite's contents manager reads ("JupyterLite Storage"), one record + * per file/directory keyed by full path. Records must match what + * @jupyterlite/contents' BrowserStorageDrive expects, so the app owns this + * extension → {type, format, mimetype} table and the record builders. + */ + +import type { FileFormat, FileStat, FileType } from "./types"; + +/** The record shape BrowserStorageDrive stores per path. */ +export interface ContentsRecord { + name: string; + path: string; + last_modified: string; + created: string; + format: FileFormat | null; + mimetype: string; + /** Parsed JSON for json-format records; base64 string for binary; text + * string otherwise; null for directories. */ + content: string | object | null; + size: number; + writable: boolean; + type: FileType; +} + +/** + * Extensions stored as base64 binary. Everything else defaults to text — + * LAMMPS files have arbitrary "extensions" (in.lj, data.spce) and are text. + */ +const BINARY_MIMETYPES: Record = { + png: "image/png", + jpg: "image/jpeg", + jpeg: "image/jpeg", + gif: "image/gif", + webp: "image/webp", + pdf: "application/pdf", + zip: "application/zip", + gz: "application/gzip", + npy: "application/octet-stream", + npz: "application/octet-stream", + bin: "application/octet-stream", +}; + +function extensionOf(path: string): string { + const name = path.slice(path.lastIndexOf("/") + 1); + const dot = name.lastIndexOf("."); + return dot > 0 ? name.slice(dot + 1).toLowerCase() : ""; +} + +/** File kind by extension: {type, format, mimetype}. */ +export function classifyPath(path: string): { + type: FileType; + format: FileFormat; + mimetype: string; +} { + const ext = extensionOf(path); + if (ext === "ipynb") { + return { type: "notebook", format: "json", mimetype: "application/json" }; + } + if (ext === "json") { + return { type: "file", format: "json", mimetype: "application/json" }; + } + const binary = BINARY_MIMETYPES[ext]; + if (binary) { + return { type: "file", format: "base64", mimetype: binary }; + } + return { type: "file", format: "text", mimetype: "text/plain" }; +} + +/** + * Validate a project-relative path (ADR-001 §2): no empty/./.. segments, no + * leading slash, and no "%" anywhere — BrowserStorageDrive decodeURIComponents + * paths, so a "%" throws inside the Jupyter UI. + */ +export function validateRelativePath(path: string): void { + if (path.startsWith("/") || path.endsWith("/")) { + throw new Error(`Invalid path (leading/trailing slash): ${path}`); + } + if (path.includes("%")) { + throw new Error(`Invalid path ("%" is not allowed): ${path}`); + } + const segments = path.split("/"); + for (const segment of segments) { + if (segment === "" || segment === "." || segment === "..") { + throw new Error(`Invalid path segment in: ${path}`); + } + } +} + +const CHUNK = 0x8000; + +export function bytesToBase64(bytes: Uint8Array): string { + let binary = ""; + for (let i = 0; i < bytes.length; i += CHUNK) { + binary += String.fromCharCode(...bytes.subarray(i, i + CHUNK)); + } + return btoa(binary); +} + +export function base64ToBytes(base64: string): Uint8Array { + const binary = atob(base64); + const bytes = new Uint8Array(binary.length); + for (let i = 0; i < binary.length; i++) { + bytes[i] = binary.charCodeAt(i); + } + return bytes; +} + +/** + * Build the contents record for a file write. String content is stored per + * the extension table (json records hold parsed objects); Uint8Array content + * is always stored base64. + */ +export function buildFileRecord( + path: string, + content: string | Uint8Array, + existingCreated?: string, +): ContentsRecord { + const now = new Date().toISOString(); + const name = path.slice(path.lastIndexOf("/") + 1); + const kind = classifyPath(path); + + if (content instanceof Uint8Array) { + return { + name, + path, + last_modified: now, + created: existingCreated ?? now, + format: "base64", + mimetype: + kind.format === "base64" ? kind.mimetype : "application/octet-stream", + content: bytesToBase64(content), + size: content.length, + writable: true, + type: "file", + }; + } + + let stored: string | object = content; + if (kind.format === "json") { + // Notebooks/.json are stored parsed (matching BrowserStorageDrive). + stored = JSON.parse(content) as object; + } + return { + name, + path, + last_modified: now, + created: existingCreated ?? now, + format: kind.format, + mimetype: kind.mimetype, + content: stored, + size: content.length, + writable: true, + type: kind.type, + }; +} + +export function buildDirectoryRecord( + path: string, + existingCreated?: string, +): ContentsRecord { + const now = new Date().toISOString(); + return { + name: path.slice(path.lastIndexOf("/") + 1), + path, + last_modified: now, + created: existingCreated ?? now, + format: "json", + mimetype: "application/json", + content: null, + size: 0, + writable: true, + type: "directory", + }; +} + +/** The FileStat view of a record, with the path made project-relative. */ +export function recordToStat( + record: ContentsRecord, + relativePath: string, +): FileStat { + return { + path: relativePath, + type: record.type, + format: record.format ?? "text", + mimetype: record.mimetype, + lastModified: record.last_modified, + size: record.size, + }; +} + +/** Record content -> what ProjectStorage.read returns (ADR-001 §4). */ +export function recordToReadResult( + record: ContentsRecord, +): string | Uint8Array { + if (record.type === "directory") { + throw new Error(`Cannot read a directory: ${record.path}`); + } + if (record.format === "base64") { + return base64ToBytes(record.content as string); + } + if (record.format === "json") { + return JSON.stringify(record.content); + } + return (record.content as string) ?? ""; +} + +/** + * Decode worker-snapshot bytes into what write() should store: text files + * (the LAMMPS common case) become strings so they stay readable in the + * notebook; known-binary extensions stay bytes. + */ +export function bytesToWriteContent( + path: string, + bytes: Uint8Array, +): string | Uint8Array { + if (classifyPath(path).format === "base64") { + return bytes; + } + return new TextDecoder().decode(bytes); +} diff --git a/src/storage/index.ts b/src/storage/index.ts new file mode 100644 index 00000000..7ec67d09 --- /dev/null +++ b/src/storage/index.ts @@ -0,0 +1,22 @@ +export * from "./types"; +export { + ContentsProjectStorage, + createIndexedDbProjectStorage, + createMemoryProjectStorage, + PROJECT_META_PATH, +} from "./contentsProjectStorage"; +export { + allocateRunDir, + expandSweep, + listRuns, + parseSweepValues, + readRunMeta, + reconcileRuns, + RUN_META_PATH, + RUNS_DIR, + SNAPSHOT_SIZE_CAP, + snapshotWorkingTree, + writeRunMeta, +} from "./runs"; +export { classifyPath, bytesToWriteContent } from "./contentsSchema"; +export { slugify, uniqueSlug } from "./slug"; diff --git a/src/storage/kv.ts b/src/storage/kv.ts new file mode 100644 index 00000000..c5dd20be --- /dev/null +++ b/src/storage/kv.ts @@ -0,0 +1,81 @@ +/** + * Minimal async key-value contract under ContentsProjectStorage. + * + * The IndexedDB backend is localforage over the "JupyterLite Storage" + * database (the same one JupyterLite's contents manager uses — that sharing + * is the whole point, ADR-001 §3). The memory backend serves quick runs, + * embeds, and tests, where nothing may touch the visitor's real library. + */ + +import localforage from "localforage"; + +export interface KeyValueStore { + getItem(key: string): Promise; + setItem(key: string, value: T): Promise; + removeItem(key: string): Promise; + keys(): Promise; +} + +class LocalForageStore implements KeyValueStore { + constructor(private readonly instance: LocalForage) {} + + async getItem(key: string): Promise { + return this.instance.getItem(key); + } + async setItem(key: string, value: T): Promise { + await this.instance.setItem(key, value); + } + async removeItem(key: string): Promise { + await this.instance.removeItem(key); + } + async keys(): Promise { + return this.instance.keys(); + } +} + +export class MemoryStore implements KeyValueStore { + private readonly map = new Map(); + + async getItem(key: string): Promise { + return (this.map.get(key) as T | undefined) ?? null; + } + async setItem(key: string, value: T): Promise { + // Deep-copy so callers can't mutate stored records in place — matches + // the structured-clone semantics of the IndexedDB backend. + this.map.set(key, structuredClone(value)); + } + async removeItem(key: string): Promise { + this.map.delete(key); + } + async keys(): Promise { + return [...this.map.keys()]; + } +} + +const JUPYTERLITE_DB = "JupyterLite Storage"; + +/** The `files` object store JupyterLite's contents manager reads. */ +export function createJupyterFilesStore(): KeyValueStore { + return new LocalForageStore( + localforage.createInstance({ + driver: localforage.INDEXEDDB, + name: JUPYTERLITE_DB, + storeName: "files", + }), + ); +} + +/** + * JupyterLite's `checkpoints` object store (up to 5 full copies per edited + * file). Delete/rename must clear these too or deleted projects leak quota + * and can be resurrected from the checkpoint UI (ADR-001 §3). + */ +export function createJupyterCheckpointsStore(): KeyValueStore { + return new LocalForageStore( + localforage.createInstance({ + driver: localforage.INDEXEDDB, + name: JUPYTERLITE_DB, + storeName: "checkpoints", + }), + ); +} diff --git a/src/storage/runs.test.ts b/src/storage/runs.test.ts new file mode 100644 index 00000000..9f6f90be --- /dev/null +++ b/src/storage/runs.test.ts @@ -0,0 +1,203 @@ +import { describe, expect, it, beforeEach } from "vitest"; +import { createMemoryProjectStorage } from "./contentsProjectStorage"; +import { + allocateRunDir, + expandSweep, + listRuns, + NOTEBOOK_RUN_GRACE_MS, + parseSweepValues, + readRunMeta, + reconcileRuns, + snapshotWorkingTree, + writeRunMeta, +} from "./runs"; +import type { ProjectStorage, RunMeta } from "./types"; + +function runMeta(id: string, patch?: Partial): RunMeta { + return { + schemaVersion: 1, + id, + inputScript: "in.diffusion", + status: "completed", + startedAt: "2026-07-11T09:00:00.000Z", + finishedAt: "2026-07-11T09:05:00.000Z", + origin: "app", + ...patch, + }; +} + +describe("runs", () => { + let storage: ProjectStorage; + let dirName: string; + + beforeEach(async () => { + storage = createMemoryProjectStorage(); + const meta = await storage.createProject({ displayName: "Diffusion" }); + dirName = meta.dirName; + await storage.write(dirName, "in.diffusion", "units lj\nrun 100"); + }); + + describe("allocateRunDir", () => { + it("allocates run-001 first and increments past existing runs", async () => { + const first = await allocateRunDir(storage, dirName); + expect(first).toBe("run-001"); + + const second = await allocateRunDir(storage, dirName); + expect(second).toBe("run-002"); + }); + + it("skips past non-numeric external run directories", async () => { + await storage.write(dirName, "runs/run-T3.5/log.lammps", "external"); + const allocated = await allocateRunDir(storage, dirName); + expect(allocated).toBe("run-001"); + }); + + it("claims the directory immediately (run.json placeholder exists)", async () => { + const runId = await allocateRunDir(storage, dirName); + const meta = await readRunMeta(storage, dirName, runId); + expect(meta?.status).toBe("running"); + }); + }); + + describe("snapshotWorkingTree", () => { + it("copies sources but excludes runs/, .atomify/ and notebooks", async () => { + await storage.write(dirName, "data.lammps", "1 atoms"); + await storage.write(dirName, "analysis.ipynb", '{"cells":[]}'); + await storage.write(dirName, "runs/run-000/old.log", "old"); + const runId = await allocateRunDir(storage, dirName); + + const { copied } = await snapshotWorkingTree(storage, dirName, runId); + + expect(copied.sort()).toEqual(["data.lammps", "in.diffusion"]); + expect( + await storage.read(dirName, `runs/${runId}/in.diffusion`), + ).toContain("units lj"); + expect( + await storage.stat(dirName, `runs/${runId}/analysis.ipynb`), + ).toBeNull(); + expect( + await storage.stat(dirName, `runs/${runId}/runs/run-000/old.log`), + ).toBeNull(); + }); + + it("records oversized files as provenance gaps instead of copying", async () => { + await storage.write(dirName, "data.big", "x".repeat(100)); + const runId = await allocateRunDir(storage, dirName); + + const { copied, gaps } = await snapshotWorkingTree( + storage, + dirName, + runId, + { sizeCap: 50 }, + ); + + expect(copied).toEqual(["in.diffusion"]); + expect(gaps).toHaveLength(1); + expect(gaps![0].path).toBe("data.big"); + expect(gaps![0].size).toBe(100); + expect(gaps![0].sha256).toMatch(/^[0-9a-f]{64}$/); + expect(await storage.stat(dirName, `runs/${runId}/data.big`)).toBeNull(); + }); + }); + + describe("listRuns", () => { + it("lists runs newest-first and includes metadata-less external runs", async () => { + await writeRunMeta(storage, dirName, runMeta("run-001")); + await writeRunMeta( + storage, + dirName, + runMeta("run-002", { finishedAt: "2026-07-11T10:00:00.000Z" }), + ); + // External run: directory with outputs but no run.json. + await storage.write(dirName, "runs/run-T3.5/log.lammps", "external"); + + const runs = await listRuns(storage, dirName); + + expect(runs.map((run) => run.runId)).toContain("run-T3.5"); + const external = runs.find((run) => run.runId === "run-T3.5"); + expect(external?.meta).toBeNull(); + const ordered = runs.filter((run) => run.meta); + expect(ordered[0].runId).toBe("run-002"); + }); + }); + + describe("reconcileRuns", () => { + it("interrupts unowned app runs but leaves owned ones", async () => { + await writeRunMeta( + storage, + dirName, + runMeta("run-001", { status: "running", finishedAt: undefined }), + ); + await writeRunMeta( + storage, + dirName, + runMeta("run-002", { status: "running", finishedAt: undefined }), + ); + + const interrupted = await reconcileRuns( + storage, + dirName, + new Set(["run-002"]), + ); + + expect(interrupted).toEqual(["run-001"]); + expect((await readRunMeta(storage, dirName, "run-001"))?.status).toBe( + "interrupted", + ); + expect((await readRunMeta(storage, dirName, "run-002"))?.status).toBe( + "running", + ); + }); + + it("gives live notebook runs a grace window", async () => { + await writeRunMeta( + storage, + dirName, + runMeta("run-001", { + status: "running", + finishedAt: undefined, + origin: "notebook", + }), + ); + + // Fresh run.json (just written): inside the grace window. + const early = await reconcileRuns(storage, dirName, new Set()); + expect(early).toEqual([]); + + // Well past the grace window. + const later = new Date(Date.now() + NOTEBOOK_RUN_GRACE_MS + 60_000); + const interrupted = await reconcileRuns( + storage, + dirName, + new Set(), + later, + ); + expect(interrupted).toEqual(["run-001"]); + }); + }); + + describe("sweeps", () => { + it("expands the cartesian product in stable order", () => { + const combos = expandSweep({ T: [1, 2], rho: [0.5, 0.8] }); + expect(combos).toEqual([ + { T: 1, rho: 0.5 }, + { T: 1, rho: 0.8 }, + { T: 2, rho: 0.5 }, + { T: 2, rho: 0.8 }, + ]); + }); + + it("returns empty for empty input or an empty value list", () => { + expect(expandSweep({})).toEqual([]); + expect(expandSweep({ T: [] })).toEqual([]); + }); + + it("parses comma lists and start:end:step ranges", () => { + expect(parseSweepValues("1, 1.5, 2")).toEqual([1, 1.5, 2]); + expect(parseSweepValues("1:3:0.5")).toEqual([1, 1.5, 2, 2.5, 3]); + expect(parseSweepValues("3:1:-1")).toEqual([3, 2, 1]); + expect(() => parseSweepValues("1:3:0")).toThrow(/never reaches/); + expect(() => parseSweepValues("1, banana")).toThrow(/Invalid number/); + }); + }); +}); diff --git a/src/storage/runs.ts b/src/storage/runs.ts new file mode 100644 index 00000000..c145aa54 --- /dev/null +++ b/src/storage/runs.ts @@ -0,0 +1,294 @@ +/** + * Run management (ADR-001 §2, §5, §7): allocation of run directories, + * working-tree snapshots, run metadata, zombie reconciliation, and sweep + * expansion. Pure layout convention over ProjectStorage — deliberately not + * part of the storage interface so the swappable surface stays minimal. + */ + +import type { + FileStat, + ProjectStorage, + RunListEntry, + RunMeta, + RunStatus, +} from "./types"; + +export const RUNS_DIR = "runs"; +export const RUN_META_PATH = ".atomify/run.json"; + +/** Files above this many bytes are recorded in snapshotGaps, not copied. */ +export const SNAPSHOT_SIZE_CAP = 64 * 1024 * 1024; + +/** Notebook-origin "running" runs younger than this are left alone. */ +export const NOTEBOOK_RUN_GRACE_MS = 10 * 60 * 1000; + +const RUN_ID_PATTERN = /^run-(\d+)$/; + +/** + * Allocate the next run-NNN directory. The directory record is written + * immediately — that write is the claim (ADR-001 §2) — so a concurrent + * creator that observes the tree afterwards allocates the next number. + */ +export async function allocateRunDir( + storage: ProjectStorage, + dirName: string, +): Promise { + const existing = await storage.list(dirName, RUNS_DIR).catch(() => []); + let next = 1; + for (const entry of existing) { + const match = RUN_ID_PATTERN.exec(entry.path.slice(RUNS_DIR.length + 1)); + if (match) { + next = Math.max(next, Number(match[1]) + 1); + } + } + for (;;) { + const runId = `run-${String(next).padStart(3, "0")}`; + const runPath = `${RUNS_DIR}/${runId}`; + if (!(await storage.stat(dirName, runPath))) { + // Claim by writing the run.json parent chain (creates the dir records). + await storage.write( + dirName, + `${runPath}/${RUN_META_PATH}`, + JSON.stringify(placeholderMeta(runId), null, 2), + ); + return runId; + } + next++; + } +} + +function placeholderMeta(runId: string): RunMeta { + return { + schemaVersion: 1, + id: runId, + inputScript: "", + status: "running", + startedAt: new Date().toISOString(), + origin: "app", + }; +} + +/** + * Snapshot the working tree into the run directory (ADR-001 §2 contract): + * excludes runs/, .atomify/, and *.ipynb; files above the size cap are + * recorded as explicit provenance gaps instead of copied. + */ +export async function snapshotWorkingTree( + storage: ProjectStorage, + dirName: string, + runId: string, + options?: { sizeCap?: number }, +): Promise<{ copied: string[]; gaps: RunMeta["snapshotGaps"] }> { + const sizeCap = options?.sizeCap ?? SNAPSHOT_SIZE_CAP; + const entries = await storage.list(dirName); + const copied: string[] = []; + const gaps: NonNullable = []; + + for (const entry of entries) { + if (!isSnapshotted(entry)) { + continue; + } + if (entry.size > sizeCap) { + const content = await storage.read(dirName, entry.path); + gaps.push({ + path: entry.path, + size: entry.size, + sha256: await sha256Hex(content), + }); + continue; + } + const content = await storage.read(dirName, entry.path); + await storage.write( + dirName, + `${RUNS_DIR}/${runId}/${entry.path}`, + content, + ); + copied.push(entry.path); + } + return { copied, gaps }; +} + +function isSnapshotted(entry: FileStat): boolean { + if (entry.type === "directory") { + // Working-tree subdirectories are not snapshotted in v1 (the Files UI + // doesn't create them); runs/ and .atomify/ are excluded by contract. + return false; + } + if (entry.path.endsWith(".ipynb")) { + return false; + } + return true; +} + +export async function writeRunMeta( + storage: ProjectStorage, + dirName: string, + meta: RunMeta, +): Promise { + await storage.write( + dirName, + `${RUNS_DIR}/${meta.id}/${RUN_META_PATH}`, + JSON.stringify(meta, null, 2), + ); +} + +export async function readRunMeta( + storage: ProjectStorage, + dirName: string, + runId: string, +): Promise { + try { + const raw = await storage.read( + dirName, + `${RUNS_DIR}/${runId}/${RUN_META_PATH}`, + ); + const meta = JSON.parse( + typeof raw === "string" ? raw : new TextDecoder().decode(raw), + ) as RunMeta; + return { ...meta, id: runId }; + } catch { + return null; + } +} + +/** + * List runs, newest first. Directories without run.json (scripted/external + * runs, ADR-001 §5) are listed with meta null and the directory mtime. + */ +export async function listRuns( + storage: ProjectStorage, + dirName: string, +): Promise { + const entries = await storage.list(dirName, RUNS_DIR).catch(() => []); + const runs: RunListEntry[] = []; + for (const entry of entries) { + if (entry.type !== "directory") { + continue; + } + const runId = entry.path.slice(RUNS_DIR.length + 1); + const meta = await readRunMeta(storage, dirName, runId); + runs.push({ + runId, + meta, + lastModified: meta?.finishedAt ?? meta?.startedAt ?? entry.lastModified, + }); + } + return runs.sort((a, b) => b.lastModified.localeCompare(a.lastModified)); +} + +/** + * Zombie reconciliation (ADR-001 §5): a "running" run not owned by this + * session becomes "interrupted" — immediately for app/sweep origins (one app + * session owns the engine), and only after a grace window for notebook + * origins (the kernel may genuinely still be executing it). + */ +export async function reconcileRuns( + storage: ProjectStorage, + dirName: string, + ownedRunIds: ReadonlySet, + now: Date = new Date(), +): Promise { + const runs = await listRuns(storage, dirName); + const interrupted: string[] = []; + for (const run of runs) { + if (!run.meta || run.meta.status !== "running") { + continue; + } + if (ownedRunIds.has(run.runId)) { + continue; + } + if (run.meta.origin === "notebook") { + const metaStat = await storage.stat( + dirName, + `${RUNS_DIR}/${run.runId}/${RUN_META_PATH}`, + ); + const age = + now.getTime() - new Date(metaStat?.lastModified ?? 0).getTime(); + if (age < NOTEBOOK_RUN_GRACE_MS) { + continue; + } + } + const meta: RunMeta = { + ...run.meta, + status: "interrupted" as RunStatus, + finishedAt: run.meta.finishedAt ?? now.toISOString(), + }; + await writeRunMeta(storage, dirName, meta); + interrupted.push(run.runId); + } + return interrupted; +} + +/** + * Sweep expansion (ADR-001 §7): the cartesian product of per-variable value + * lists, in stable order (first variable varies slowest). + */ +export function expandSweep( + variables: Record, +): Record[] { + const names = Object.keys(variables); + if (names.length === 0) { + return []; + } + let combinations: Record[] = [{}]; + for (const name of names) { + const values = variables[name]; + if (values.length === 0) { + return []; + } + combinations = combinations.flatMap((combo) => + values.map((value) => ({ ...combo, [name]: value })), + ); + } + return combinations; +} + +/** + * Parse sweep values: a comma list ("1, 1.5, 2") or a range with step + * ("1:3:0.5" = start:end:step, end inclusive within float tolerance). + */ +export function parseSweepValues(input: string): number[] { + const trimmed = input.trim(); + if (trimmed === "") { + return []; + } + if (trimmed.includes(":")) { + const parts = trimmed.split(":").map((part) => Number(part.trim())); + if (parts.length !== 3 || parts.some((value) => !Number.isFinite(value))) { + throw new Error(`Invalid range (expected start:end:step): ${input}`); + } + const [start, end, step] = parts; + if (step === 0 || Math.sign(end - start) * Math.sign(step) < 0) { + throw new Error(`Range never reaches its end: ${input}`); + } + const values: number[] = []; + const count = Math.floor((end - start) / step + 1e-9); + for (let i = 0; i <= count; i++) { + // Multiply instead of accumulating so float error doesn't drift. + values.push(Number((start + i * step).toPrecision(12))); + } + return values; + } + return trimmed.split(",").map((part) => { + const value = Number(part.trim()); + if (!Number.isFinite(value)) { + throw new Error(`Invalid number in sweep values: "${part.trim()}"`); + } + return value; + }); +} + +async function sha256Hex(content: string | Uint8Array): Promise { + const bytes = + typeof content === "string" ? new TextEncoder().encode(content) : content; + const digest = await crypto.subtle.digest( + "SHA-256", + bytes.buffer instanceof ArrayBuffer && bytes.byteOffset === 0 && + bytes.byteLength === bytes.buffer.byteLength + ? bytes.buffer + : bytes.slice().buffer, + ); + return [...new Uint8Array(digest)] + .map((byte) => byte.toString(16).padStart(2, "0")) + .join(""); +} diff --git a/src/storage/slug.ts b/src/storage/slug.ts new file mode 100644 index 00000000..becdae6f --- /dev/null +++ b/src/storage/slug.ts @@ -0,0 +1,32 @@ +/** + * Project dirName slugs (ADR-001 §2): alphabet [a-z0-9-], non-empty, + * human-readable because the dirName is what the Jupyter file browser shows. + */ + +const FALLBACK = "project"; + +/** "Diffusion coefficients!" -> "diffusion-coefficients". */ +export function slugify(name: string): string { + const slug = name + .toLowerCase() + .normalize("NFKD") + // Strip combining marks left by NFKD (é -> e). + .replace(/[̀-ͯ]/g, "") + .replace(/[^a-z0-9]+/g, "-") + .replace(/^-+|-+$/g, ""); + return slug || FALLBACK; +} + +/** First of slug, slug-2, slug-3, … not in `taken`. */ +export function uniqueSlug(name: string, taken: Set): string { + const base = slugify(name); + if (!taken.has(base)) { + return base; + } + for (let i = 2; ; i++) { + const candidate = `${base}-${i}`; + if (!taken.has(candidate)) { + return candidate; + } + } +} diff --git a/src/storage/types.ts b/src/storage/types.ts new file mode 100644 index 00000000..f53eadeb --- /dev/null +++ b/src/storage/types.ts @@ -0,0 +1,130 @@ +/** + * Project storage contracts (ADR-001). + * + * A project is a directory in the shared project filesystem: a working tree + * of source files plus a `runs/` directory of immutable run records. The + * filesystem is stored in JupyterLite's contents database, so everything a + * project contains is visible to the embedded notebook with no sync layer. + * + * All persistence goes through {@link ProjectStorage}; implementations are + * swappable (IndexedDB-backed today, HTTP API later, in-memory for quick + * runs/embeds/tests). + */ + +export type FileFormat = "text" | "json" | "base64"; +export type FileType = "file" | "notebook" | "directory"; + +export interface FileStat { + /** Project-relative path, no leading slash (e.g. "runs/run-001/log.lammps"). */ + path: string; + type: FileType; + format: FileFormat; + mimetype: string; + /** ISO 8601. */ + lastModified: string; + size: number; +} + +export interface ProjectSource { + type: "blank" | "upload" | "example" | "shared"; + /** Example id from examples.json when type is "example". */ + exampleId?: string; +} + +/** Contents of `/.atomify/project.json`. */ +export interface ProjectMeta { + schemaVersion: 1; + /** + * The project's identity AND its directory name in the shared filesystem. + * Immutable after creation so notebook paths never break; renames touch + * only displayName. + */ + dirName: string; + displayName: string; + description?: string; + createdAt: string; + /** Designated input script (project-relative path). Absent until chosen. */ + inputScript?: string; + /** Library identity dot color. */ + color?: string; + source?: ProjectSource; +} + +export interface NewProjectMeta { + displayName: string; + description?: string; + inputScript?: string; + color?: string; + source?: ProjectSource; +} + +export type RunStatus = + | "running" + | "completed" + | "failed" + | "canceled" + | "interrupted"; + +/** Contents of `/runs//.atomify/run.json`. */ +export interface RunMeta { + schemaVersion: 1; + id: string; + inputScript: string; + /** LAMMPS variables injected for this run (sweeps, URL vars). */ + vars?: Record; + /** Groups the runs of one sweep. */ + sweepId?: string; + status: RunStatus; + startedAt: string; + finishedAt?: string; + stats?: { + timesteps?: number; + numAtoms?: number; + wallSeconds?: number; + }; + /** LAMMPS error message when status is "failed". */ + error?: string; + /** Files excluded from the snapshot by the size cap (name + size + sha256). */ + snapshotGaps?: { path: string; size: number; sha256: string }[]; + origin: "app" | "sweep" | "notebook"; +} + +/** + * A run row as listed by the app: either a real run.json, or an "external" + * directory created by scripts/tools that wrote no metadata (ADR-001 §5) — + * still listed, with what can be derived. + */ +export interface RunListEntry { + runId: string; + meta: RunMeta | null; + /** Directory record mtime — the only timestamp an external run has. */ + lastModified: string; +} + +export interface ProjectStorage { + // --- Projects --------------------------------------------------------- + listProjects(): Promise; + /** Allocates a unique dirName from displayName and creates the skeleton. */ + createProject(meta: NewProjectMeta): Promise; + /** dirName is immutable; a patch attempting to change it throws. */ + updateProjectMeta( + dirName: string, + patch: Partial>, + ): Promise; + /** Recursive; also clears matching checkpoint records. */ + deleteProject(dirName: string): Promise; + + // --- Files (project-relative paths; parents auto-created on write) ----- + /** Direct children of subdir (default: project root). Never recursive. */ + list(dirName: string, subdir?: string): Promise; + /** Returns Uint8Array iff the stored format is base64; else string. */ + read(dirName: string, path: string): Promise; + write( + dirName: string, + path: string, + content: string | Uint8Array, + ): Promise; + rename(dirName: string, from: string, to: string): Promise; + remove(dirName: string, path: string): Promise; + stat(dirName: string, path: string): Promise; +} diff --git a/src/wasm/LammpsWorkerProxy.ts b/src/wasm/LammpsWorkerProxy.ts index 8ab14203..777c28fd 100644 --- a/src/wasm/LammpsWorkerProxy.ts +++ b/src/wasm/LammpsWorkerProxy.ts @@ -7,8 +7,22 @@ import type { WorkerEvent, WorkerStepData, WorkerModifierData, + WorkdirFile, } from "./workerMessages"; +/** One file from a workdir snapshot, as delivered to callers. */ +export interface WorkdirSnapshotFile { + /** Path relative to the requested dir. */ + path: string; + bytes: Uint8Array; +} + +export interface WorkdirSnapshot { + files: WorkdirSnapshotFile[]; + /** Files present but over the requested maxBytes. */ + skipped: { path: string; size: number }[]; +} + /** Byte offsets of one streamed series' x/y arrays in the bridge heap. */ interface SeriesPointers { name: string; @@ -96,6 +110,10 @@ export class LammpsWorkerProxy implements LammpsWeb { private nextRequestId = 1; private readonly pendingRuns = new Map(); + private readonly pendingSnapshots = new Map< + number, + (snapshot: WorkdirSnapshot) => void + >(); private readyPromise?: Promise; private stepListener?: () => void; @@ -157,6 +175,20 @@ export class LammpsWorkerProxy implements LammpsWeb { } break; } + case "workdirSnapshot": { + const resolve = this.pendingSnapshots.get(event.requestId); + if (resolve) { + this.pendingSnapshots.delete(event.requestId); + resolve({ + files: event.files.map((file: WorkdirFile) => ({ + path: file.path, + bytes: new Uint8Array(file.bytes), + })), + skipped: event.skipped, + }); + } + break; + } } } @@ -475,6 +507,20 @@ export class LammpsWorkerProxy implements LammpsWeb { }); } + /** + * Read the files under `dir` out of the worker's wasm FS — the run-outputs + * data path (ADR-001 §5). Safe to call mid-run: MEMFS reads are pure JS and + * never re-enter the suspended module. Pass maxBytes for the throttled + * mid-run copy; omit it for the complete end-of-run snapshot. + */ + snapshotWorkdir(dir: string, maxBytes?: number): Promise { + const requestId = this.nextRequestId++; + return new Promise((resolve) => { + this.pendingSnapshots.set(requestId, resolve); + this.send({ type: "snapshotWorkdir", requestId, dir, maxBytes }); + }); + } + setSyncFrequency(every: number) { if (every <= 0 || every === this.syncFrequency) return; this.syncFrequency = every; diff --git a/src/wasm/lammps.worker.ts b/src/wasm/lammps.worker.ts index 50490651..867be5fa 100644 --- a/src/wasm/lammps.worker.ts +++ b/src/wasm/lammps.worker.ts @@ -81,6 +81,50 @@ ctx.addEventListener("unhandledrejection", (e: PromiseRejectionEvent) => { } }); +/** + * Read every file under `dir` (recursively) out of the wasm FS. MEMFS reads + * are pure JS — they never call into compiled wasm — so unlike the deferred + * native calls above this is safe even while the module is asyncify-suspended + * mid-run. Files above maxBytes are reported in `skipped` instead of read + * (the mid-run throttle passes a cap; the end-of-run snapshot passes none). + */ +function snapshotWorkdir(dir: string, maxBytes?: number) { + const files: { path: string; bytes: ArrayBuffer }[] = []; + const skipped: { path: string; size: number }[] = []; + if (!module || !module.FS.analyzePath(dir).exists) { + return { files, skipped }; + } + const base = dir.replace(/\/+$/, ""); + const walk = (current: string) => { + for (const entry of module!.FS.readdir(current)) { + if (entry === "." || entry === "..") continue; + const fullPath = `${current}/${entry}`; + const stat = module!.FS.stat(fullPath); + if (module!.FS.isDir(stat.mode)) { + walk(fullPath); + continue; + } + const relative = fullPath.slice(base.length + 1); + if (maxBytes !== undefined && stat.size > maxBytes) { + skipped.push({ path: relative, size: stat.size }); + continue; + } + // Without an encoding option Emscripten returns the raw bytes; the + // declared type only covers the utf8 overload. + const data = module!.FS.readFile(fullPath) as unknown as + | Uint8Array + | string; + const bytes = + typeof data === "string" + ? new TextEncoder().encode(data) + : new Uint8Array(data); // copy: heap subarrays must not be transferred + files.push({ path: relative, bytes: bytes.buffer as ArrayBuffer }); + } + }; + walk(base); + return { files, skipped }; +} + /** mkdir -p: create every missing parent, ignoring "already exists". */ function mkdirp(path: string) { if (!module) return; @@ -327,6 +371,14 @@ ctx.onmessage = async (ev: MessageEvent) => { case "runCommand": callNativeSafely(() => adapter?.runCommand(cmd.command)); break; + case "snapshotWorkdir": { + const { files, skipped } = snapshotWorkdir(cmd.dir, cmd.maxBytes); + post( + { type: "workdirSnapshot", requestId: cmd.requestId, files, skipped }, + files.map((file) => file.bytes), + ); + break; + } } } catch (error) { post({ diff --git a/src/wasm/workerMessages.ts b/src/wasm/workerMessages.ts index 6c564589..a8f9c3a3 100644 --- a/src/wasm/workerMessages.ts +++ b/src/wasm/workerMessages.ts @@ -36,7 +36,13 @@ export type WorkerCommand = // streams that compute's per-atom values each step so color-by-compute works; // null streams none (the common case), avoiding needless per-atom invocation. | { type: "setPerAtomModifier"; category: ModifierCategory; name: string | null } - | { type: "runCommand"; command: string }; + | { type: "runCommand"; command: string } + // Read the files under `dir` out of the worker's wasm FS (the run-outputs + // data path, ADR-001 §5). MEMFS reads are pure JS, so this is safe even + // while the module is asyncify-suspended mid-run; files larger than + // maxBytes are listed in `skipped` instead of transferred (the caller sends + // maxBytes: undefined for the final end-of-run snapshot). + | { type: "snapshotWorkdir"; requestId: number; dir: string; maxBytes?: number }; /** One compute/fix/variable's streamed snapshot (scalar + 1D series). */ export interface WorkerModifierData { @@ -105,10 +111,25 @@ export interface WorkerStepData { walls: WallInfo[]; } +/** One file read out of the worker FS by snapshotWorkdir. */ +export interface WorkdirFile { + /** Path relative to the snapshot's `dir`. */ + path: string; + /** Raw bytes; the ArrayBuffer is transferred to the main thread. */ + bytes: ArrayBuffer; +} + /** Events sent from the worker to the main thread. */ export type WorkerEvent = | { type: "ready" } | WorkerStepData | { type: "printed"; text: string } | { type: "error"; message: string } - | { type: "runFinished"; requestId: number; error: string }; + | { type: "runFinished"; requestId: number; error: string } + | { + type: "workdirSnapshot"; + requestId: number; + files: WorkdirFile[]; + /** Files present but over maxBytes: relative path + size. */ + skipped: { path: string; size: number }[]; + }; diff --git a/vitest.config.mts b/vitest.config.mts index 3e424583..577ca8b2 100644 --- a/vitest.config.mts +++ b/vitest.config.mts @@ -11,6 +11,9 @@ export default defineConfig({ // even though they pass in isolation. Real failures are assertion errors, // which are immediate; give slow-under-load tests comfortable headroom. testTimeout: 20000, + // Stale git worktrees under .claude/ carry their own src/ + node_modules; + // without this exclude their tests (and broken deps) pollute every run. + exclude: ["**/node_modules/**", "**/dist/**", "**/.claude/**"], environmentOptions: { jsdom: { resources: "usable", From 2df4ed0a438e5ae8f8396de431a950efe50c7b46 Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 10:04:25 +0200 Subject: [PATCH 05/34] Vendor lammps-js and lammps-logfile wheels into pypi/ for piplite MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per ADR-002 §2-3: jupyter lite build auto-indexes wheels from /pypi/ (the lite dir is the repo root), so notebooks can `%pip install lammps-js lammps-logfile`. - pypi/lammps_js-1.5.0-py3-none-any.whl is built from the lammps.js repo's python/ package (interim vendoring until the wheel is published upstream — PyPI 404s and the npm tarball excludes python/). Note: the wheel version (1.5.0, from python/pyproject.toml) skews from the npm lammps.js dependency (1.5.1); upstream PR should bump pyproject.toml to track the npm version. - pypi/lammps_logfile-1.1.3-py3-none-any.whl is the pure-python wheel from PyPI (deps numpy/matplotlib ship with pyodide). - Delete jupyterlite/content/lammps_logfile/: the source-directory copy at the contents root becomes unimportable once the notebook cwd is a project directory; the piplite wheel replaces it. Co-Authored-By: Claude Fable 5 --- jupyterlite/content/lammps_logfile/File.py | 125 ------------------ .../content/lammps_logfile/__init__.py | 3 - .../content/lammps_logfile/cmd_interface.py | 30 ----- jupyterlite/content/lammps_logfile/utils.py | 63 --------- pypi/lammps_js-1.5.0-py3-none-any.whl | Bin 0 -> 12106 bytes pypi/lammps_logfile-1.1.3-py3-none-any.whl | Bin 0 -> 25742 bytes 6 files changed, 221 deletions(-) delete mode 100644 jupyterlite/content/lammps_logfile/File.py delete mode 100644 jupyterlite/content/lammps_logfile/__init__.py delete mode 100644 jupyterlite/content/lammps_logfile/cmd_interface.py delete mode 100644 jupyterlite/content/lammps_logfile/utils.py create mode 100644 pypi/lammps_js-1.5.0-py3-none-any.whl create mode 100644 pypi/lammps_logfile-1.1.3-py3-none-any.whl diff --git a/jupyterlite/content/lammps_logfile/File.py b/jupyterlite/content/lammps_logfile/File.py deleted file mode 100644 index 5f8f21ba..00000000 --- a/jupyterlite/content/lammps_logfile/File.py +++ /dev/null @@ -1,125 +0,0 @@ -import numpy as np -import pandas as pd -from io import BytesIO, StringIO - -class File: - """Class for handling lammps log files. - - Parameters - ---------------------- - :param ifile: path to lammps log file - :type ifile: string or file - - """ - def __init__(self, ifile): - # Identifiers for places in the log file - self.start_thermo_strings = ["Memory usage per processor", "Per MPI rank memory allocation"] - self.stop_thermo_strings = ["Loop time", "ERROR"] - self.data_dict = {} - self.keywords = [] - self.output_before_first_run = "" - self.partial_logs = [] - if hasattr(ifile, "read"): - self.logfile = ifile - else: - self.logfile = open(ifile, 'r') - self.read_file_to_dict() - - def read_file_to_dict(self): - contents = self.logfile.readlines() - keyword_flag = False - before_first_run_flag = True - i = 0 - while i < len(contents): - line = contents[i] - if before_first_run_flag: - self.output_before_first_run += line - - if keyword_flag: - keywords = line.split() - tmpString = "" - # Check wheter any of the thermo stop strigs are in the present line - while not sum([string in line for string in self.stop_thermo_strings]) >= 1: - if "\n" in line: - tmpString+=line - i+=1 - if i= 1: - keyword_flag = True - before_first_run_flag = False - i += 1 - - def flush_dict_and_set_new_keyword(self, keywords): - self.data_dict = {} - for entry in keywords: - self.data_dict[entry] = np.asarray([]) - self.keywords = keywords - - def get(self, entry_name, run_num=-1): - """Get time-series from log file by name. - - Paramerers - -------------------- - :param entry_name: Name of the entry, for example "Temp" - :type entry_name: str - :param run_num: Lammps simulations commonly involve several run-commands. Here you may choose what run you want the log data from. Default of :code:`-1` returns data from all runs concatenated - :type run_num: int - - If the rows in the log file changes between runs, the logs are being flushed. - """ - - if run_num == -1: - if entry_name in self.data_dict.keys(): - return self.data_dict[entry_name] - else: - return None - else: - if len(self.partial_logs) > run_num: - partial_log = self.partial_logs[run_num] - if entry_name in partial_log.keys(): - return partial_log[entry_name] - else: - return None - else: - return None - - def get_keywords(self, run_num=-1): - """Return list of available data columns in the log file.""" - if run_num == -1: - return sorted(self.keywords) - else: - if len(self.partial_logs) > run_num: - return sorted(list(self.partial_logs[run_num].keys())) - else: - return None - - def to_exdir_group(self, name, exdirfile): - group = exdirfile.require_group(name) - for i, log in enumerate(self.partial_logs): - subgroup = group.require_group(str(i)) - for key, value in log.items(): - key = key.replace("/", ".") - subgroup.create_dataset(key, data=value) - - - - def get_num_partial_logs(self): - return len(self.partial_logs) diff --git a/jupyterlite/content/lammps_logfile/__init__.py b/jupyterlite/content/lammps_logfile/__init__.py deleted file mode 100644 index 5a7b7bb7..00000000 --- a/jupyterlite/content/lammps_logfile/__init__.py +++ /dev/null @@ -1,3 +0,0 @@ -from .File import File -from .utils import running_mean, get_color_value, get_matlab_color -from .cmd_interface import run \ No newline at end of file diff --git a/jupyterlite/content/lammps_logfile/cmd_interface.py b/jupyterlite/content/lammps_logfile/cmd_interface.py deleted file mode 100644 index 30b1069f..00000000 --- a/jupyterlite/content/lammps_logfile/cmd_interface.py +++ /dev/null @@ -1,30 +0,0 @@ -from lammps_logfile import File -from lammps_logfile import running_mean -import argparse -import matplotlib.pyplot as plt - -def get_parser(): - parser = argparse.ArgumentParser(description="Plot contents from lammps log files") - parser.add_argument("input_file", type=str, help="Lammps log file containing thermo output from lammps simulation.") - parser.add_argument("-x", type=str, default="Time", help="Data to plot on the first axis") - parser.add_argument("-y", type=str, nargs="+", help="Data to plot on the second axis. You can supply several names to get several plot lines in the same figure.") - parser.add_argument("-a", "--running_average", type=int, default=1, help="Optionally average over this many log entries with a running average. Some thermo properties fluctuate wildly, and often we are interested in te running average of properties like temperature and pressure.") - return parser - -def run(): - args = get_parser().parse_args() - log = File(args.input_file) - x = log.get(args.x) - print(x) - if args.running_average >= 2: - x = running_mean(x, args.running_average) - for y in args.y: - data = log.get(y) - print(data) - if args.running_average >= 2: - data = running_mean(data, args.running_average) - - plt.plot(x, data, label=y) - plt.legend() - plt.show() - diff --git a/jupyterlite/content/lammps_logfile/utils.py b/jupyterlite/content/lammps_logfile/utils.py deleted file mode 100644 index a62c9262..00000000 --- a/jupyterlite/content/lammps_logfile/utils.py +++ /dev/null @@ -1,63 +0,0 @@ -import numpy as np -import matplotlib - -def running_mean(data, N): - """Calculate running mean of an array-like dataset. - - Parameters - -------------------- - :param data: The array - :type data: 1d array-like - :param N: Width of the averaging window - :type N: int - - """ - - data = np.asarray(data) - if N == 1: - return data - else: - retArray = np.zeros(data.size)*np.nan - padL = int(N/2) - padR = N-padL-1 - retArray[padL:-padR] = np.convolve(data, np.ones((N,))/N, mode='valid') - return retArray - -def get_color_value(value, minValue, maxValue, cmap='viridis'): - """Get color from colormap. - - Parameters - ----------------- - :param value: Value used tpo get color from colormap - :param minValue: Minimum value in colormap. Values below this value will saturate on the lower color of the colormap. - :param maxValue: Maximum value in colormap. Values above this value will saturate on the upper color of the colormap. - - :returns: 4-vector containing colormap values. - - This is useful if you are plotting data from several simulations, and want to color them based on some parameters changing between the simulations. For example, you may want the color to gradually change along a clormap as the temperature increases. - - """ - diff = maxValue-minValue - cmap = matplotlib.cm.get_cmap(cmap) - rgba = cmap((value-minValue)/diff) - return rgba - -def get_matlab_color(i): - """Get colors from matlabs standard color order. - - Parameters - ------------- - :param i: Index. Cycles with a period of 7, so calling with 1 returns the same color as calling with 8. - :type i: int - - :returns: color as 3-vector - - """ - colors = np.asarray([ [0, 0.447000000000000, 0.741000000000000], - [0.850000000000000, 0.325000000000000, 0.098000000000000], - [0.929000000000000, 0.694000000000000, 0.125000000000000], - [0.494000000000000, 0.184000000000000, 0.556000000000000], - [0.466000000000000, 0.674000000000000, 0.188000000000000], - [0.301000000000000, 0.745000000000000, 0.933000000000000], - [0.635000000000000, 0.078000000000000, 0.184000000000000]]) - return colors[ i%len(colors) ] diff --git a/pypi/lammps_js-1.5.0-py3-none-any.whl b/pypi/lammps_js-1.5.0-py3-none-any.whl new file mode 100644 index 0000000000000000000000000000000000000000..2244e56eac916ce3201522b8c2169acd04bf636c GIT binary patch literal 12106 zcmaL71FUd8yRE%!o6oXs+qTVT*|u%lwr$(CZF`+}pMA8yfB)ZXnx-?kuQcuG-J~7z zQotZ60000G05RI<+O#M|b-H8#09b(l07(D*wKA}_wsWM@(=)d*chb|NwR2zgwshPS zZP|1EiGcI6gc(D5s!L>FP-%#uk;|&5sXj7xQB{aXSWnXT2LJ~knQy6h5!=#p^BvC- z;{%LK;7Q-mR3QPw-Y)Hw%XQ7Q0TH+}{k`5SM7D1#ql?teDt%|4rlVw_kW|o@6d@$*eAwpO_L{3GSPU9hQJWVjp=(cwL4DvN^hmj;++E`GvOy`p=^Vd+0m`$Q$ za!{PzH`Z;h-M>e7`iMN(J_uvCIMnE{mBJ=m|rK{H5zrj$wL7mYr$_Sjm7w9vzEdSH(= z>uYltPHyO(dwRq> zFkV4Jd6_c~cdk&pl2FQm1Fi8$P}ESD0rs`Rm@+uy%p)WyGYS2%e9BTA2e1&p0ey;v zOUm4Y8CR&$;+9#x7wUtM+~LbN`qwk$SF4uvZFX734s6iZoV>)aLwG}xhr#-F_#Nm3 z5VfL77eaqVAzp8i+l2A5IuboXQs2;G`dR&eJtl;3x)N&+xQjI8CK6) zf5=K#s@5uD$D_A4kx^!bDV$k#!bA7u=0k-L2enaI6_Hw4AyT0`t)Ka?NJbz6;LhZw zV7`*{aT7T@E>S{jBM|!831c}V#}I?4`u=6Z4kEvh-;$GbTZl~nMu<}w3b|=tK7`bl zW6lvj6+J!%9`%iLf+@QPfEL3m$OUN|HZ$X&vhwa_6S!L9rO%JU*GSldxl{G&=F|W1ak-ppuu?Wp<E7BdH(4J}CoVjiq}rs~+wFhmg{J*G6m5s`SNF>^v= z1}Iqz%IUBhLCvBtjQ^3vQ6Nv6=54ZnMS{-58VU=tAGQ&y-|O!0X)j8O=h#n^@VxBR5hJ+NCH{VRKF2sFsD325EZ>J1a1fVxeo4ez*1raf&9t_in|yeT9Q^_pNR#>?JEeM zmtjOdPRfcWG~XAGK)@+v{@ZY+gjc{>!n4O(wx>$~SS%e7K~1<56VnPmeRlWXST^v$ z-X4xG_g|S#91+pZr|Es@-iJ?L99$ULJiFgEm9NkK$k)*8gYUz`Z{N0J@_e|TQ(Inr znwGq)s#Q;7S$P1PJY^vnp(5xZS!m~MV@HXJx#sOAnqv(c`&tf1{sYt5Tvin8XDr8% zhHKqg2BXE5^cr)kQ;W@v^o$gbjHRkqGcg?9$p+zdAZ@{*lg;`<7-p;K2Q-*4_h8DS z)Ye=k4zXVV z_uuQix9$GS{`sFzJ+M)5d0k27XVbEnFu@6YkZJI&(5olRrQ^RebrrM3w!EvwNx4!S zle|RDDC)$c=HKbmb!J4mpRsx1Av1f|`<<`7S^Vg^X-_^f5Bg$O{yjbA^c2>;$a0dwQVSOoPz#@JV%MCTZGHPAs^1aGq`vYFL!yUahfH6}>%Sp# zYc3|u&mnG)KJ3nr!r;Hlg8vtqCx-{zE6UJVZ4O3$^Nh0!No>Kq$M-2i+m8pDGNeI= zGU%kXpV2*k3ZV)+23o)Q_j!`xp|*CS@Xz7ZUt$6M9qR73DqG^8U6$|}e?8*5JQufS ziixQxGZT8F^zlfLYA8rTU}?KL^7TE{mK7|hQ5IJxhAFhBbI@>LlR|pxgpR!T!j>RB-xcV}X>n zcI6PJxY754>@Fj?CrpI1du$3*6Z_~U7MVvUC_=vJaRH$1bcv@p_1%qnZvY}+Y%p}v zyD(V=O))3Hb_VjDesbk&CE%_m01;U6cU}{xhCj5oF z5Rof1M^5y%tP&;!wubMV!xCknkjhgttel@_Z&21i_mfJp6;E#j)0i=glQ7Zw#$BUn zI*TCm=jH91<{x0EJAk=Ty<2mkG4RBXYig%iyojPQMJm#v=uaj+pABXSOO~UY7!(Tj z_iVuQ39X>Ifr0wkf!N4D12s>H)*XeM%j4tZ2u&&lYOKZ()q)RR)3s zOU4+z&ujKFCCePNt(qcHCd!$voTQJP4^+TC-Sx8@Bid85sF8H-%!uRy zw+ko4A`;3$*>@#RG*jVD4=^-2ERahmIQpYO4^2g!2jbTfhFGDI*<9Gh#>48=+^ zlopVAha)GPI@MS?MXNWd>Ot)Et}mxZ20v;ti8zCf8T6Y7_>DbwUfLrn;>N&snESS7 z@s0Nn{KZCLBL2%FrgXFntR+vnC@Q1Xc3Ifx5Rj1&(ryIiLm-g7LSnyW;4T<<)r5uY zNHza9B**6-+!^hH&lDyyeh}fTC(0O-F`C|AIMz~{a=-KSjyR< zY`~xAy_I3Fr$3*|V4IWz5g8g2V3HFE4E{37-tgGRUQ@F)-y&Oz1jd)`3{+=}ilQ_J zSRu*B8ba{%fL=7$G%%-W7J|~=_!k>X`6$eKR(=GXVXcM5Orlj{bnUpEL!78cBN0j7eY40f&peX^JekE-Anz8?6j} zL?U|+Xaf5l+wSwxI|@~6%I(=Z>l@;rZ!oNu{o4R_eXZCbre~v8kPo;epo=`1T=VUAP~J2F!Q!wV+n~y^NBY3u>u_J)nyn~wN%Y7LAJ={BK3Fl z%_Qi8f#^7H?9HL6X*staLOGBUwjSp4fsv+r_m&g#){+IqwU2P%e7xm3BD30`COKTv zA?ftuZc}d3{+c4>2#og-XqQbFYr}}W z8Z@gg)92Y14JBjF?L^5Ab}%%-5qwt*i*w*n9pE&$##e0b3DcN#swf;NUvCg9r?bhyC%J(|fJch`!t!q>mM*gT= zzH5+-#H#&yxsIm_v&6eYUS3K`wqPPSx`; zaqYzvO0Miv{IJ9t<=_&{h2rk(mfv>CpCqDf{J6(KHD|~T-D!*B9*Dqt9;#NS!(lj{t%5YO`AqhUYy|0# z>1LC3Ljh)gN27p3gFM0(w<}?OS7P0z7_jv6IqI6rtrUft@`we99K{_k%4@JzE>$$` zysTBy<((x5cunu&Rr(IORD~ov1@%4yvt((4k8oB$l*;QfZ{*NJ&-7ZxM1>^n)QSOx zyBLG%9sb~OUea$?X)#Wjca*$9aQY^QMzZf?e!)1$I3)H$HWN-U_V1jv-~1n~XICUfL5-F_*f#Dk-{BSq?hBe~k&GU{~5| z6Lg736LDe@|23_lw-QyJw_!4W;iCATG}Y+Mzy9#i4#mn71kqx*Fb++1>7oUUE^eN5 z5t(zk{Y@&g6%`z{MY$h^D?3L<721nyf)Q>M0Ioc?&7Sb;n=aYQRO1ru5gUBU-k>GB zI;j%M#Sdpzaqj8CczxWOzlbJ~N)%%-Wbq{wa930wjycK+E-z=RZ5Rq*3`uA5Trp&G z#{#*E_JrH9T;)kMfzZ`jmrww}o*&n5&0q%LB_Z-0%SI~cTnU&?$I;+S5DKc6 z`ZmAWetgRHHpqG@4g-EvJEWFJsQD4lzyQ(L)5?h4%ixrQzRNZzu-B$Je}~^?_ol)z z$CFigt9$Zf^n=DV)L9{+nX1Xfz!HUxkMEM?ZW8g8)0vWJpCQ3NW!%B7>Iq~8RgR($c#3)c`{R@-$Y9p*_=7Q4?vTV7njftcgDBc#}17OWhF+;LO*ij#MPR zAS6wwxFAQ*6!mZDfpOpp)n_w*M9?tIG#KuJhy9#~bVC^rXV~TJY}1MDQ(zlP3W|M<3I;f5ht<06FJ`(bPrvTfkv*IB^2x$eyE8PyE3( zH_x@DG;N-UISLv(`BYSG#sci$K;!ybElD4g2WR)C-u;QN3@)iuZNUOxhUdHjtd5s; zuAgZ3UQazseEl)_o(<-;PzDvp+2280R=PB;@k)Ltr;*re^ul~zv&=KyM|;bU7Y`dV zQ0y^tXhJzUg)Tn|%$x>xDv>kEc;u_jil0B2*8V6anEMOe>WlFC2wmUZ{tZLDzH5qa zO9D{N-!s zTV-&LcPKaOXD|=cckjm9*EEcMX%!TE53z;V`~E#@&;l=1fG+ruiSLZ zR!)RS+Fd<%+kBmDQ=F)i;CyPgX1^9SV-IUoe~YutAV!G@BqBb+VIgqJti_IVARao% z>!%W6%DyFntDQPXKrYS=CT1uDNf7`K1vD(+5g*9F6f;DfsdgS4 zqyGSe={dMhHYm0yye+_|uVuDsXaWB;US9y?ToxPaqo8OTR&m5ReinyfFRdh>X-=UN zYXnGV&01?u?dBtr{Vi#i>OnkyT7XPtvEJ6u$F0T^C2#H~y^@>EuMeS(V{kf1;6ONA z1VjW3j1ANGVsj0I7;*08fk`PW;%Rj+Y1|2)v0wHHAKQ704`i+25zxpFBnC z{Ro}9s_EIHohv&uzoDr9EDBlUTWB`T`;fu9^XT*n!2pNPbRL5Yp*uzZ&QM2Br6}q~NcqCMBA5HX-M6=AT1qoU8Tg@v973~+7_ev>- z_buy<_JRVmqsIo$Z17R-r=41+0M{+U!`xitkAtAD1IBPm2Ugr6R+%I$`UHmd}@u~}e8yR?8_S$PfbU)llm&`>dKIym^44t$*hgb7MT zaIv!B0Vy_V>F86FGoaHE^r>|@(N{1x`V}NQ6kBkN9&!EcQTZ!1C$tpI&f!b^MAi3^ zH*ncZ2J4KN>P5hFG*Yf4d^l7}cbFAhl~{v@_+2oR%kSwt#U^2XZa~O=ryW~AEi;ca(svvhHE$gC)e`l-JxRg_Qb3PQ*IYiHo(_+$1V@QjUN9*H>Df`&6w$0j zM{@M7K_}zZ%pGo$iW+Xu}&Xou!|8u zopyP_?rwM$FY(hKL*))|cO4+_Cv2SkA4L@3UH)XI-bMa9ZwNAiH(Ly6AylTnKr==u z)pTnXCaO@`&1apD&jKD&z7{FpRRoNNHT459_IpE3UP&v4E3VEu>drdbRULSs^M;PI zmQ%8TZWi{mwgbsy7Wt4`>~$X{v17^DUVj!LEqQCE+$FdNB}yAA25egWf13V6?Lj4Hr>{?*Pu%Bq3wU=fF78#whOjWE^VHBw43PoIEJ5WMt_w}3 zsSh?+d3!dA=L4#%h^0hu0yy$~#ibuybVHx}?93XOf+#J1Czv>OfXA@{C*Hm-l;)Vg zqDhIQpT=RGqi{LB$GI4uMv}9e>JKu8on~+fp>E(JV}3vsQsZr5EJnYb(bQ#}%R$dq z6oIW|VG*`?nrknI^aogG!^a&F;`O@8VXl1r($f5ib=D-z4!r&1w+;09#-y9E2SrJB zNRv7HsLrWjuUjwa6(%aWC*!oRxC$OKtg7cG%S^VNrD1p~K1rLrzL;IWkht=-W+%(2(R?-GHc$VcnELerZj1^o{|?kW=7oRHx%N*};n%dX5mfF?f-qtcTD zrQ!%Yqrac=A#_pJc(mTG@PFWmBQa_NH0dMkoNJSRidN7 z3>0b@ciA7J3l~OR5#`=8g*zVuq6cKqv#~~U)~>XFz0fAlt4bPWn2&k5?29Db_yA+8 zes0oFYw9_AC+s|jfD0py?@aTx$0$z1As`o-K{~1KXgSFNM-7M<7*s*|*h(r*s{|AQ ziPCrgWE#MD)1H+1&Ng{*;XqG4OlL|jdS$$=rM@Qnf95MoH+wj`j(h*1|9WhCZd-jh zjn*Zzz5`XgZZjq7DI`q-Y-K+fTMI);ca=|&d5huLZb#su`q3Y zudmVjRb`8<&{Nwh+QShROqdcQ<@E&oeLytQM<-q+7hRnvP?Ng1IhCc5_Ly8GpXtN{ zJpO~^ZguyI!EMw9L9e}g!9}B40bIcu6M0+-)G$W80|TBE7$qJH7os-SiDvjMq8+TT z@eD|Csz-lJltNt6YuZ=hF&<;wu7|#eN_Bb=uKzXx?CT73f$b{qg}Tx`vPf`puVXyy zo)90#=8AzN-g8u)G3zu?yh;VB9SnI2RV}+Om%y`R7OPR$CQz3!c&MS-zFgKec-nVR zn`aUl2Mc-WiPtaDT|%4#izJby`P2qV3~_I`01~B@t}n?BHe|se#0Gm(0YqiU=UQzz!&BlzFD&eJ)0MR&19_JI_J%rdvb9?+H(|oOWR=A zCV9nj%(%mOiEo!Pa(<%Wg`Mrp>eWzX(zR#Ju6P488agM^78TS|TH9FOwzyc`NV_m| zzo1zKFk{Nt0i|NyxfDC}=|s7@ys@`&@66>vl6EDZ@?gXB=R*9hhiI#1INA_A0SL(f zKP87Dd4jxh>_B5X|5YVyLC7dshVtX-!^!(n8q;IHxdHU9XOLaa0mwNzuWhq&IiUc0 zA8UQfSxJw%-pgZqzSw!pnLE-@QtvdWm9_KhO?faEK% z`PoeGKHh=>X8{x@iIE{+9fvQ$lWVBb_4>OMYy(tRd!-|L2`{f>&Pr!nrs7twc-q#H z3Hg@>`nLR%zxz|-U2}D%yNE!~p%MaxPH|AXfLL+-2S4yHz67~zaZwrpq#C$Dtr;6I zo$}!oI7kG@iSGMIOlF7WbZ>pcd4GZ_$w_;6PAW;oc#cB)%a6{{jE9zu{R_i_{n~!z zTT5AaOnS`pFPPWlpDA|vV2Oqqn%s;~4ca@VW432(~{Y!oN9({$af z6LPNwvz08P?8srwcN(}bdlq@wOc;>j_WY{3uL8i#M$X5I{XbXmqcWDbDW4e6Jv|TF z^~Dya?dvtm;BMaiDnOiBVeNS~Yr|V1GcK_=xW3NLF~?;nUptj9uhUv9fqV~swZ#Pj zbw_yR(x?~Jmhu(#5m5VRFO*YRe?jd`Je{hl7I1{Bmdm97y0IVt#eCU6QpLoc&uo<7 zEjv6c8W>KuF#-cY1M1Ogr65-XR|>++CWkbo|Ns=6*f6vxyg{ZB-8tacRCMw-O57>xkK)D#7bB&{-c<-Q01*Y^&PyG4M zGx_Jsu@s`>$=2GApR|9`yYyc`*Rj9IqhK8dy9!bmJ6C41B~0h*+XD_MQhNKdKF*Hg zPEUwsVN$6NB2fLYw`j!jmbvUVosQp8%Byn@IG^mQjOp}v z5+Qk=)S9lP!W5U{YiKHb$+J6cJ)D$m@<|&4^N9i!-7legUs7Z!eMyP42Rf$io9c=*??IF$Hd^@^hlCR9dru zXRlPxdC~N|&k)faa5X@DN0PKFU#BPgM1@+j;jv&)>Gaj-tubMt(5dRAWN~cwuD5== z4?6&BD>k#4yQBx<@Jc2ko)0~lO>T>Etw8N#dwtUnvQ@i(eyxYGETZW?zqGO5r?9vZ zft^`Wygij~|J-lm=z3*Urp~5Xe{xT(2yo^B?(1lyVO(qkuj&0B#%uK)XT6?0)2*3} ztO<e3 z0{-$v!S>FCcZUU>@^5{2V{z|jB#E@!i3ji}U^`ur0N2xG5mwpkX0zZmM$$+YuC1VN zZ_-h!x>gLFYsou#X6$)cNr57$%INO9z8TR`jaxKwAPyp6A?LwKbXX^5Yhz@C)8&L7 zYi?a9SI0%96jrLY0jY~Y4~__F zmUUOYW-F&AWz17`B7uVi8=Y%o&#@ZY|7HFMjBsUdAdhF^9d} z=E_VHbm6_X?i8EM&jL_A2t794?O8);7t@)27~I+R!@=Eo_55WJ*+s{f7^y7&;o_?6yYYtZM`G{nn00R=61HoQ z61B$lZJw>>nkA?d!DGQ{YLpE>D#R2~Tj~tRBT?=`*cLcu?}j^Y+S)3S_9{*OW+bS5 zdQBiQ?(9e$gJ@9$(ED1MZBAyTeR@nklo2x_a8Pa?6hh>#0FltHH5DS~9c2hLS2hG_ z4a-AnyH0p1MOIWyN-6a0ZSFd*!d!aC!ChWyvb)F>^+!~=Jg1tL#CbCk?RDx!=}84} z`VGA!lZ)mVtndkYF<7o6Bibxh^W36oi=w8%;+0FpPd{JQfjnH7WVyZ4$;Kz+shW9H z(WO6TZ>m&cWrLwDB>}iL+hL-0GtZ*`+=>M|Bg;ouX^rUOo`B!K!crPs^&-|B)nMyE zcx~-|7=7VZORW8J8Na+;Q{Y96JDbLKZUs+w)-^dbNRmd;pha@BufaE$36Nj9so>IZ zLyNe~&RNU-4ylt^wt@8Q8~%+%Uh~!M57Clb=6?NLsi+w&h+?uI=M2{^bHw1`AYHHgq^&k`)&Mq8=4I!M=&(h7x%}p5Gi^%I3E8ZDs zSXjpzv1|EMlx9ab-GL?Jz0g0DmFi@Zv~ev>xDFDBzl#bY*S@sw(a|ByDmH3!bM&|m z5wxI%S>8puFWXR9VP8mGL&Cdzd z`}vVRyikYEG7;6{Dsm?C@vF-amCn=k)4jT2L!(RNz%`=7E8OSA;x9`6b^bZs`h~IH z|E*e`AMLrbR6sG4H*s=McbKj1>Ck0tCrBry$j2)Cb9;-Mqiw*Z(d8x(ftNp25z`qu zWJ%>o9vmzaE^Pf#Y6H<&N-dM1&hNBl*4UEcr3>c4f$>J<-sU;?N3@~I)A{YBqYzzu zy0JNyh>(U?V`+085SmI*cL_*C9q3%nNE8(mG*)r(?U4z?-Z4{e62d>&OdG z7MwElk!wj7tFA%3l_{;E_P-{8^fcdWt$#NDe^k7Gn*h|rMMR`Q`x&4H>0vgXNIu72}QxT5mmfam^n5Uk97+h)g^L_F1N&U5D5QtDJjyN|Tg& z77fuJZW!UkxO{>K!Og0*_8&8ZWe8O03i$r(xFnFEpdkMl&p&wy;2*c}|FQ}HcZtc# z)=tmL#Kpvl*2&FjQetYFhWrEw;J?y%<0Pv90RaHK|Ec)@mZm5oB&R6sJtgH3Ko0;g z*ZmiAjINwo8zesuP2`x!hVe~6Fi;S3-r-}$DTL-0Al5`hrBzArEYLyl=mWh;q(p$( z_n4hU0o>em_o=Qve*Z!w!^(ly-6TwPvv^3hK%J};ea<NjW^lUdAC_$j;I zVCPEQLD6?1UrXiEM!QUN*6{vj`+`yj;>&q*8hRx?$$^o61re3+O3?OD^!V3w=r`%H zK)?*mXgI2p!S_*>p^amG;h+L!5dNm1S4gxW0}QX|LwDoajWqzNue=l>5DM`Bzia!S zIsBhrk^g@*|6dRH{~Q0`oiqO#4FFK&AM?-Z{%8Dux48fN)cIH5zsmOi@?xp}9q)gN z_+MfFsYdGe}(?5Mp(;erYhpHN*p4Q76bAGEz zMIIai0|W#F24qW##$W{z=aw281Vk4H1O)59TYFOn2WK~9dnYSP8+!|8ssA=6XPR1IkOcqaJeH5rxgqoa12y2miD~b2DEY!LM3iVLMf?A?d!yT{6vzzb=bPgr^b9AJYSD$vR{gW3M|-lbnQ&92i@x zhGFzCMkqTnO5BfDNTHq#_W_4J*+4)>Hpkxuuf<9QvQE@8SWeNoJ5Q^cDNojCgwf@$ha8ExQeW@cYbZ`B4Q+=hsVQ|90#N_*Y`EzVi!rSM zyzxjm7LB=v`$gwaWt%KrC- z|8rT{wR>RTWURj*g>cV0d-q|Z>5xYwWZCz~zKtU2y6sa#eU|G<&BCK3mDyh4;rQMK zLK5TI$8jyMV73yJ+)yVEp!)7qQH}5(-8*0x#hY;^W3ilTSOi-}VtJ0BOzN5y{+vAR zLo0~`j#%Ya*zqJ?t0dS2f}cjZol+IOX{SD{AjC4t;(R$}sMkg8l}~mI!D8`ActQ?I z;uHj{8XO`Q=Ws>DMtQrq3-~ZfTX1qz-5s;SJs2N*Mddmd(0W7s)_FznoMAdVbFi=0 zTfu}&0p`s_=U-_Q5o_2lg!P=eqh3Fm{7iiuj_Unm!`6}h(lR0$j|WD+_Ry+N$G&}+rRv`Ba6T9VW`PQJ_;lrG*+#oL8Cs5 z0{E~&9iOdV7`IXX0e<5eJ9h+hbl~U-pOdWf>EZoo@`R2%{Ue)P4kIU9kqnk6lsRq+ z!2($Nx?>hMTO%8pDEJM{f@ZQQfLQYthLVgZ;uQCie>DaiqRNV3l!_z6rkOx$-9x=cR zmy`IS5IPJLMr)u}e3`ftK@HZQByeU9v-?y9=0DbOXr|*tK3b^oWS=4Q?==5Hs&aEh zI0FRYO79-xG|&>__g`$#pxt6YBZpY1+`QSTb5}@8odXhF0%@~{;y~h&)}Hl*16VoA zb%v|@DS$((k9&N3VYB5(RxLlNzcX%=z}aTHia}2~cTNms@UX#F=4^vTVUmh7(V`HO zh&G6o_4dxR>YK^^!CVk}(CW$!hrX>kOr>2}>r}(^>b-iqxyNxGF01~$RTvrjBJoD+ zOu{Bmt_EO)xv+Uw1kttAk@kF!2h^RGeg}HfnPzx=a~aNNhXxORignfp$s6}J;Oo_1 zMW-H{u&5|d|A49d542u>vhSPxht%}{7T*6xtFf_-qm8?<@qc(#oZUnmVLGy#N1VuD7uc2{-NH~g!D4<@6#mMPk5&^0 z?XL=TQCphZL^?E>^;#s1~$*AtvER062&`2h+Tl)5J80%jh;!=GT_QO znvD;9-U4_%r*l|;jjqWC{d0d4E^n!TqP*;Jp||Kr*MMnpjDK-fylcHY>2he#@a?Tp zJ=Y~b_uPiLa{k*8;gt3u*YH$v#EpW4cVg9wM!!*F9C5+bt1}y1@C_eZs`9<$xQ*9t z{Q4y4j!%!zoWPVq3GW#His8i~3MmF@>A|~mUh#4eCK-lRU%fPhM{kZw)Vz(aPgL4Q z)t$9?T*9C&&LYo*JVoHYZgS9<51udHHjOf!z! z=5BS;i|(U{s!h?jK*wK3ktWI)bypFQu(-cfp-MPbum~q9<`2-5krhUQLZ#%C1f-SN zNc1=`o?5X&qEcpNhQUI>JGz);ZY5Uxur~!EdZ5QP9s098%j3&SYb_a8;a}*>O88xP zkwE1l(z|w59A2OvhAD5;UD{rMgVfV&zY;5nvbE ztz1`^q$IYXu01mfP*T8fQS5)G6HXQAH=ogvs+3|Y4`W@-?LKagTz;Is%JbE`6y|NK zpFtVJdDqe-9Qww>uIHvE{6*Ok}oa&y%wUPC^*Qw=5r92lbqY};tnXH7vv95gz znJ{>8tuo^JCJfoG-lVO0a67y|6D(J7FOFArtO12asoXs-1w2lsx3Q?**Tr}VKr*!= zsgUTqt!Ftrh1{0cOr7F1cnMKI^rK>h47hg;+&2~OiUs;us%Cus8)Q%&N8+WQn!8+( zbs>udz!*v5BP&gp>*RzawH`w>%;qwygfe^|#DC;S=YsI6+3{wxssG49xIFM@U{&OL zh+uZJ<6)aK2zVz6^~PF5lO@IqejOY_yz71rM}hre@*a+%sa0_Lyk_~idNj(N1tlzO zJR^9Q8|aZLa1{9cb0CqAPDbPhd)bxs?d|$X_4Ec`g;fOa=n-pGyD#9_?yj3b1hf zzb38c;Iz$y_FZ5+^;rx!W0c_YlyZUcPb=ndg75wYT@!{cPPQp$9Vz(;? z&JupB#zo#XZgLlzXE}VCs>^$feqeqtbnDPrY>~BQji>7Dl>XCYP6PF+$q!$u&g$7@tx?twyUYMhZc-y0TNjMzkDzY zwJ)4*e&j(X6c|m$iHJu4Qz$9GTPXE#b!Yo&{H)I`pRdVX@+~y?kdV%x`Ec zTvX*yL}*gu?ESIf&TK4;adXfuBE`+<1TkK@HYf&%*Exl3S$HYOS_cU%G2~)J!#Uml zTcOdAVFth>!r9_lHUruN0dFY!Yb4=|Nk|f5Mb6feu~L1p()V&EPKyOw*c$P%hkb_l$X-|48Jy z3KF4himtQ)e_9nVl;Xg9td)@Jf}u-LmlYni5HOd1O~8?fGtNnS0BoUQycUGwI9Vho z1_P@V?S5adFU%+qwBAh@V0ksY?i;su{_6XT2*`-=bLRLwYy9=y`{oLS6ASiV>uyl;Acd|@OOoe3Dfi3HW;NuS}CHi_V+t1ym1E=#g~ zaJWkmofvzNsX@45*z3D#b`b;<4mnzqC*ogTFMW(3XN}Y)4LZ*SNtoO0ZCJ68LfD;j zV?pOWC*P7``OF6auT7V%&GtYQUy~v7L5&MZl8`3iz1y*(W*RX`TIgzK!VO6nZRJ>d z5Xf?jTtQ%5O4|kXoaM3+9A->~v$U3>i&NNTL;*_B@~D8y>(LzP+gLFDK;x?S610&O zyCu{2TQ)V1Hl}@c51=DVWuXXy{Gf4(u0fUC&u52LeZSN@2eY-NY=35_{+iPimUo!dE_kJkQu#utOtbTCW_rlx6<0e9>UI1*HL; zukCr9WMUZIb|0+S2GIv-2oY^kXJqSrgP;tf45FYjpi)6;0Me1XFIaH$g2snrxz%rS zyO8(iVS)?cT=N&dHTc-Y1SFyo41O!}l?$t3f|k$i!ZJgaQ5hA*!p=G(9QohY($Vc; z-9!4_LYinUVoxz9i0g%^?aSRqWB5-z67|qxW2qUU*cUbN@FvD)gwezsf(G=wTl_jBI0=P z_JmVoXP%+Z#zyh!g*0o-E&)sR6Q4V0k{+?k<)BoRQHob;Yn`;8FBF8r$k^l3Wy$mb zT*Pt~V4Hclp%GSpr&j))hRy7D-3iv;^5BASQgzLj6+sL&hf+X7!rPR9c|)6C!VVOn zAH#5gmA`^OmO%JdItQjHCy(4h4(}tu|Abh*S}EupG)G=7-VucEC?Hfi8Z-R$(rLKm zH7t>rF+(1qur(a{Nt$J)<`?V*TdR?KXJsBbW4$xwL!^O8_mR^Fe$z;;MApmZn(BS4 z*Ev20AaG_r@lvccgc&(gIzJmJtDRgCRCkN<|5eR>DfFz#ndwmOKHE01CEuGNIw4-B zyht9d?NxlMT+nip`CM&)Jw2Z?lTAkE^jKM2bP1Tf=>Ruyq^vcmjq2_|-l)7J0^wRR zc_|$k5toXJh@H@k%d(yg$Z!O@NPD+%Gzdn}`{}W~6ax(^2^ze-es?bj#pqCmnsP^O znG7)O@{1XO0TFl$C?zQH8tr}W3m?*872++08a{B28YU>CJALt-a?uTLqyiWRi35n@ zbZBI)(u_X?m}Q;DH9zppZPWvZbnI-c3(Njts^`mA=iO6()M7g)Vg6MWDw0nc)HeNH zafJ=A+w5EjEWM1zizOz_`eS;pPzE)dTc1sr_k?&$tqzh`^hm*#n5@Ar26?temMA_b zEtTdb_&w4mioU}7K1?fd~>)l60*Zpge!4w!q$z2i~o+HM8%R6~2ZYYU|WvD+& zcD9$*^iZbm`-V|58D*RSCa(c}(a59NE6H)7n`as3_|*UXq~q7gN+LPt)+sVtAWR`- zo5fp#0aeo^KLa8ethF8zfVW&6V2jtDZp~IDLp1s!djUwc{9;OKGOXZWQzUqUh<~EU^%N#nRNE?Ri|5(grL3iSHo*BBK+#q(CF6dE?&8n#h3Y8X& zPzkd0AQy$6={T+_r3$=|Z7`acJ=)mb3**BNN8^&a49UEJZb8I6%#U`}m5^3(E>#p9 zPMuP5;s@-XPB`SV8Nd13Mqem;ReSM7ydIv4+DDl4&$^%?AKdJ!o!PCN0%_gsdI)aU zw)!OQsjB&f@#X?)bgwDwTV43kO2j*gNvX}TmsUmjZI$3q<~*P8=_rnLyajVyK0T#4 zc#I@Cc>?h^_s_*rhfs_Xca12VE~piCUTpa{?!}_2Pgoe4a=)a%yg?g~|Jq+P-^6>v zgs#E>V0;aR1ilH1K0rb1+!OQ|CZ6{bA!SAku)OC)z-Rwm5B3+3v^+0pzl?&!6pi3r zm)8}^$vmV(b8!n6fooK-BwNS;b84G(YAG?89TXtw38)68Dj*MBFKrs&TIn9BF`F11 z@NVnZ(LPPWISG-P%3o5#=uVgfTn^m!`(y45VWG@g$jOE>q_wH!m$8GXdwP(`uFf#u zs)V5_0;F-Evi*Lb|EJ5*VUNQV{uhuS;6OmI|F_F|xZBvf{ikyInhthHJShLE94kh~ zc+?TqoF6H129p=TPGmVjFR>U+tf1rAR7&!3*+t`C@-a`0ddD{UP_uNYm1ThsDzjWU z<75rZ+aSUu8lS)ZoDqm2wTq;NPTT>tsm9097`Ou zptEn0YZ3KI&W&H_Pt-mVUYqO$@z*#9{bC8%ck4RC&xQK|Q(mBZW&dh@P$6qHI-a8Z4mKAa=~Gz*qK=rwLNtOiRn3CA^HP z`=1FAtWn$H_=%@J$-u<3<5x4#u-@_LSEhG$qsZa?&0DJ|R1jlT`lAE15o6YmFhI(b zR4-stxJ#!OhH;ITIwr-sXV-#2l6wXR zF8$!pcLrjiq12NhVWl{-U{R<-j^>}t2d3x0RFbYQqhHOLAB>DTvP}k0@wr###s4C( z1R##}Fb~o-f@+KUcx%!PXguq%oKWb(aCJ@${PFjyi3s`!|4vgf*OYV&o;(-?b2o_= zdup)ntjh5-LEuPnbT_CjucKcSHQcOIu00k}oiBsGJ_4cbOFRBHKx^uLZ-Hmxv^drQ zMbk6rrzDQXQZ}53-RjVKVf}~lG>5@>7b#x<8>?v20v&`e9a!6^yQvwE2JrFeg{ahz z7}Lkl+my~t*%X8yDI3G>2K#HQ<4L+itq8N6bPBz~9PLpACFl}3W0yxcrr9=FZ44zs zNMl=r5#8Fwzi=g&G8JLPK4!}b1@PPWM@8;o&A@`tpaz;kOXBdz0Uy*h$$vY_ z6B_)&iPn|zxwrZ_+FZiWcX{C3^#q}0K6yZ444M>iAPm2y>~fRBima+uvvMaki|ITf z`9k(=y#Mszd}rVeP&ofmhU6Rtt}qS-!yM7MY4Rt~MkJvv9rP64h!Tnwt#>^l8KE3l z#kT%LjsGqL_(vqmFnauGDoTVpm1`LA5t)x1A$*Oz1+%gQS^0Eu zPhOOIV4+pf+EFz&=^L;XR-C3aC=8pndyhHP^MwBYdivnMcMks-FH3D^5D@to5Rm_l z|1Y00vNEwUu`>Z|+}s&$94(!g?QP609NjG3m=$EjC6&}A#|-t|w#QQf3ynh#_WB!gp;9B=DRLA!$3!DT61nTj`S%@igTm6dng z6jKL5K?Xr`-&uwWw-wq`H04lV6-73X}p)Q0&CHZK&lj}**X4YXJRR_MN-qE`<06T=-; z)!MS172GpsHnnS1KK<9+9hb}D1y0yZ|ru6o0vY0Y+AF}JAW|jP=58;V=?jR z2Nl|G*N<5;gLTm&l}YLJ3mv%3sk>2I`F9Z$SJSG=^!B`Ar6?u5{Qx{QkroqbKsg>g zTR7)=F;8_c8Yqk40_nfG!0c%lcWenQ7ex=HO7L{F%cHM-^~Dz<4$<4HgJctl5&37ffMOZY*?>FV=uybf z8mm3L*92!Szp%HAdYMCeFtepK1w?6X-{cMkNAUJm`+_~QyPoDkw9aLznJe@Aa9_G1o*N&RwTWj;@y- z!D^6u|9K{$_A=j?yRn&S1>r#Zy>+}c-_TR+gyB_hAYmE~sWTyfY>GS$beJ;lq+{y{ z=nSK)ZUm91`>FUdy0giRs8e(BG^viwfPG$JXCOc{TAo$zz;}o!X533DMTaG+>)wSR z`Lv`T+ni?hX$A>LQL+$+Df75K@&d!~g%kyZ1!Wkv0?hVRwqL*5w=&x7o{?C8p71#f z6gxUJ#}*g%gt89rTnjcFdtkkDEO0O<$I(*jh`UNNf#9bTB4as}3Cvgp!Xg;qe+=h8$V2YVx zs4N1XbOd;N=x@kjq1SpCue>#TZT!R9cIY#Jax~v-&33oSFQIRm$#27iQ76pAslYA@ z-p!=1*1b;3@Lm`0B^}iT3IY7{6v0$5als@e?3RbnN##*tC)=+I#$A_;z4J z_Akih8f_7Hd5I;lKZIoZem{leHKUWlUw@d9_m*FF;R4hSj0drOzBcrU{75M|Zr7C$l@E zr&>}#E{WH_CaG#aFtd?IAre$&*Vh;P$r%le8rGb6RA2C74?b%H4`PgQLt=Rea4<Icoc&3t&1%TPukL>bCXT5OJjNasz8i$ba2j4fHd2AVOTeX%i<@sXJ9$Cxi*fTH-oXBt~0how=Vd)@A|3=)A;~6`vuumHx_64OL~M0Q^+M z8IE9#9@ATE{uVOq=T6#_(Pd+yZa}EcyIFpsLunQ`WZ&G6aM|&BA$w1t%%J33-Qa0c zz@d`~>R=D?$Y73yzsUWD#c#Slq7$duSF1xye|Ene0Q?&q#4LJgfuO>;mx)otpaTCa zat_4Q5BqAI3jIef5S?&TiG#ZWXsNSd^o9DBOG9U041(^Z8K~TUZS7;UTSZtN^eO}8 zWtBME5am6et*{}F@^4a)lw256__tuQPk2A!j0hHq8Tm@PxHB3-J)}*FXA$vCXf@)1 zx@x^MeuiWT3c*Yzr7-GkbXLgdv)4J-2XXH;e}Yf&Sps@W4EN!^HqPOgIu#b*?X}hd ztQg6h@oO1EKLIN^uj_bJ`N?yZEQ7}6koxdsK}!ihWz3sf|J0LJCFWjlLySU;UJsWE z%oywWBDITnl#O3MN=U{7}0y(g_qR-M8ql90g0L zWljaPhmFhV#E5+@PQ|~eT;UWGnRXRva57;e^ky>9!PLvmTRh$dQ4sbk9JhFr?)nK} z0salCYmMcaYkw>Vn{c7^4Bv1Wd-HOIMVe zw}ArVTrZRER|)eSv8OwX8F(k+hipEg&XR;KGuxrIv|p#LwRfyGk#@^1c{6+QhQ5hMu`E!Ts+9Z8j&h~9s8KF>B;&anaHKz+!P*)dwIoWr%kVwZ0x|Iz(rMy1IX}*r zg9vdasPMaPDz#f&&s$Ke?7kG{IP^#A)&`VcA>ObqX{3VWd@5~rFz1E*Lf~UlP=rZ8 z{|>cZhLA2ZTPG&4RBM($<2jVV^3%fDQfR9K6i=?@`{NpQC`oRRJ^Bx7ak?Mtc#ERq z0wi)~;#`X6Se!oD&XQm+w}eU@RhUX{w5LWj!7R*)YNL=wCtjoeJ8cqLH>-zD`r|^2 zTg<@YcDJ~gB(}J#=v~zD&ZGV-jo7#Ot8sX^Jes!0$;ja!-B1lUDo}^P#N2b2(}yu$ zo2paT&^9!Ed5GGab5IixG}v_L&(N2cRTL^Tcwt@&f&ms#AOqY_-DTOUNHIl;ibQ5O z3fX;dbV7D~Q5MZ+@QLl#GyhO|ncQrMv}tX#`8ZmEF`L!iq z{IuCe9_WbegC#*~y539r;BX&6CW%(@}yH#Q$7-74KA>1nZd-;*+51xfR7B)BM>u z_!D*$Z~rF1Ag)C?BN(%g1R@0x$h)41J9b!00rb2e#dIaBKf-fvd`;^1eMN(gNnCti z$McL56*(Y*Ep;*tEg8cs&A=Lx@}=d{c!o@Hces(tb_#=y1Ly}G=O%4M z7Q;r412nO6!R!#sH1?QdY!Xdw1Xj4VqgxnvtiFYQOt7ke7z^nv&qt(wC2u z07O zd0K@tsyRu?L0(yyRKn}hY5PJxZ9+SzjMOoUSoYPmY!)z_tJhgvKSLn4#fg$8iV}t? z<2;sBmQw6A8Q{*_kUsjmmL!AI8AG*ZhBrk?${<^d1qi;?j1F4y3?zy&Bk8wgJ|8>SnIfVzo8fAFWkydI{Rg1e|LnbD z61C9E9A~-dv=2QvOso+bEM?Msgh)OK%P|cWsRX34O4(`cmgyXo`YIwiMl@roQ!@G) z6#awM0bWetbBvX22=aFY@;G|N&p)Lvis!^@xalsWfYzoqGZTUBa8Tz(Zu6ms$bd+L zHhB`vH^+W=r#3)GfoW7Bv)Mq~LswWVJIN{A1&@871`ZB&o4R&@!oi4bcQJ%Go&VW~ zM;s8|A9$m&nE8IANWsGg@n$oZx1$H_rWF28>Lcf{$9n#_3i>*-x=*u_y&o<=6saRe zm9D{CK9e~FLCm>wv9cB)3uD2Nk0`E7Gq`60Cv(=B(_JoamKGiTMI#dx--)}=;POd1 zk_4-d6Q+Ft4tFS7l(M3-DIwIQ@U7?;n7CXy} zWd9pM{G~Z}EU$DYU_hZm?g z(ErO&!vAygx0Rus2;-WFZ*M6YlNZbH1?H^~S@P+zE*3&L5tL@CW7JlRf2Unp4O@)7 zUnyxPU(PV*q=w+Miw&ZiSP#DS?0#GuFXR5Lo^kl%Y5b`gEN9N7W__eQ962W68h&9g zvFyqP{_|AYKZ5#9SL4}-bfJN;vWTSRU$`I`L7Slh>*bi6&2Oxo+z8|3?8x0ts_&x@ zI5>iuz1FWhc(=@IHGWNq>v$mzHF`gY78r|o#&8I*M>wEYG67=SPMw_LM(AJ>#oi>V z;Vdvgi7_{&tv9P|-vrcK(i+&(AIJ!r?Ofv_taonMwzNn_%Nud4BANKDjx6rO}5jDMCpyW$J zoW0fp(ZTOgxzZawe@gimLkMaJh`fEMHbCy;4U6NeZ)z*a1ykq(+VvU(kL8{9T8z1MbXH6?W!4kRsIn>T_`nKmUVz~ zx;PLV9Hp0`59T(3O$LS-h$4wnW75V(rA8h$89~kK7Q$Yk`Q??LZuE$(Q?^vL;b=-guk*d<{&vTk$FTb zLaN4p3Akq!R6=$|k@{;~8MSI5v+1hMI8a@VCsHak>g!-=GZaHHxLa^vewY`z#ZLNh zGG|snmqroQqUC{HP#T!ElU>`Cgg~J#tyv3?#=uj4Cewq|KlAspt%SU%N!)i8Zi+fV zj}VfjMqx*_Y&CgE#6*;i;h@wE6I&Z7zmXWm%(u#( z_(pz0@i;n+8!gNnYUm#NfPI<3%0EL$`$5S*;^JGej3!&&>M}R5Gtr%u5juoY4s`st z39p?f$A*m?Z0~=;_M^sS1-GxM6+pfuS`4MC@kdE*!eo){Ui;kf7*yIiCB;ZdfSuyf zw2LyTq)o7B_nCqg_DJ@mu*nOQz%^QK@>_vSoD5m#59kePPOI=4r;c}3lid;hqF4jX zBjJUy%j5wW=Anm6abPr3spDV2GsM{QG>7q8xoS>?9x=5@>SlSGKKw3Rd)y1sbqbR$ zD`Vw!ZjgF-$Jk`4mZeNqNM;VX)t0S$I`ZRseU%nK?sf?GCf)i*s=_E^W$)An_=_tqTRB6du=j| zLFobhOFKDhaxFb65ttn|St-q598efxP}x|XYgKC~DCpORlp_YHQyhvtSvcQu!@m-q zLbDYF(Zf{;utVN|O&5#Np(vLZ4fr=+@m<*dqMn`qm7`uM)=pDaTptE<2(ZidVPek~ zEmN)4-kL6#Z$@%B#&VT-CdN zX%XdteI}~gDN!3Z61V*0OVf7kb_69;BSMD$qeb& z^B#voIJ)&+lvVU~TS-bSXTN0to5*TW! z4vU2kp}W6kcjTa5P`ZdI6%=dBb7P6o0 zZ){TsdY8}Y<)|n`HFk^iRtHikeglyc2nj{f6w;r}g2mT1Nh|1mQaZpEQF*Lqoh8TS z2Bs#m>7q7+^zVB_P>UuWc*dhL3OqGCTEkT>R8(*@x)e8A8>C7u?2O$nQz4kNR*~!( zsui-OciSY1S|kYzf>;kRV?iN%YMcHpI;-;CwnM1#4a9cQD zPa+n9zMDqGRCm;aLzD&*C6oR28{Wf9s=7cV)|U~wjEls!v@cJE`6O?ZM)N^KtsdH7 zGNFu<*A4=eEGdrPXI7^aN44U*nGZ6{e1zY?4g*pJUIo9Qiu=Uw+=QEP2gV5TyJ{sc zP^UOEdkGCVb9j;`kG!2cvt;-h4coIv7S7O)n)7K|fM|d8uqYwDr2puiPDh&!{_Jhb z^lh(719t~vG5!u=dwZ5KLe|?}kCdW%QJMTt8dgBYZ6gdDs7*4Znsem33 z(FQ~P2eoMR5c;JJNB@@r#Y$49PZLuSUrZoA8i6&moxOoCINrMNMc93X9Y6S-tsu}{ zLk68>9bZc;ClK#y)kvn>FzZNUaz5L#h-3YDnu>z@beYgzDup8{2i3+E{$?D&CC{#C zP)7va0@aQ)fNg-Lgm%S$?jxp&Z`X58b1g--*B~d@Q2gSf@9<{NHxXh#9p#_m0!DbN z*w#^23p5#0r1J@vCA)z?EfRnjb1MnnX-+KM2Z+W))B@AT_i?i45tFnq&h~LOtx5;o}9)ScNhr1$Vz$hbtpgPXGlCX&jUQj zQ&33FS?K%XN>wTfoL+S6r zYnMNwk<(pdJgdzpWzrzeos))Owmi@l7$%|8HazasLmJEt(_qwiU1wdb$FIFHDpxP1P3VO2B+5slnKaaG8R?Yw)cd+O&8Ul^D3a&lyQmW((`r=GMvgo!ag(>|Y{~YZ%?UM^ zc&?Vt0@dakZ(%Di_~o1eq$*_xE-p;J(Ewj zt*p2*oZ3xxT{yx7lEHUKD74&kX878Fl?j$-i0n_^ zM>wnZ+g*|UH!%+dZ6;?JUoP>XLzwJuAm>*8;6jbb%_>bu{-_1aCULphYD~DK825C`olusS=DOxsy2Wj`bhB!7oE2& zm-YqwaDpGB9Xa)WLNX4B49C3PxeF&yL@w;$Unjh$O03)=s>?3)Tj@J6nFZd~-EC`e zjzt8w1EAn1$HxGET&%}pcoErQxXMkYG_W`KcS?OqKvzpTUt_!52butRZk5Hk2FvP5bM1_s4G*tH#o4Q~2Q=mw|k8zmwCy#h@{1m_j1 z!|p=Ec~XenZ&W%C<{S&mmX4s`yT`v5b;%KT9B}p>6P~qmk(j^Bx4V$UHujJJwOM`5 z_~PDFzRp|=6qB*=8Ud|upd0dLxlBfVW^IQ&~_ zdqdGwFBfY#KIePHVd#=`*ysJak^%1%jKR$K$FK=&ukaGxuJFxMEA) z;SFSSralbdUQ+9M0U8WDrx1j?1dapSh~QF?@f6eQ*${Je^>FAGkqhv7riHx_3SF8nu(h%iUCjDeb zj}cpi`_Ie1CTl6;dSfOYqoLcgtRednm-3ZgW*A7lA!t{{9Jq9ik8>r8@jmo1XE;@E z{60r8TV?I7si)%=$j|7OPpL{J&7=~d4|jGSYyO)ghmLXf0j<&rj)nNR^MWtaAenOW zi7k-Kdd4INJK~&@+6RNS+|2`%RXdT~W#agLkp1V7?5X=2G}Dt7_(`Wkb$Lkc?J5Y0 zK8wr#&5!-!RXQ}f^?0V)Z&S4lF`wMZ(Z8XBx@KkXdA>h_+H3d0+A>ghO;bO=d z#sf~VFKAy!Vcp>}K+CqG4k&kC=0`us7J0O4UmoCZrjQE~RXVG1{u?1Sg17luZHwgO zyPGUM=(G%3PtGq~E=kDGU8yey!6SeN=2`$Csen zM2;8{D!!NPl{5M^Amg~JBhT&HkWyv7t6*?2Hwl1Hevf8)gCKA@l9uDcQ{c|Ra>J)` zcE!)7Au9b^>onyuwGMZtgoC4UF-K@8{&nUj1Em~k%EP+FDG&M1^|zgL%>2n3PdZIG z0ZU1INsabNv9%nJ*PUOb%EE5KXgT+F<2d${F*20XBEuHHp~XJ`}+W%<-3u_TVd-{ z{iJ4jiePXn^NbvG4}mz`#lp56de!2bIeo5nGjG7eKr}uusS#_XfVOFf92m<&k#y1{ zta`yH79KUm=P|ZaIdz{zMp4rbpNQkTYU?hyd?!i$zFaH1vD6yYJ)>?i%|*P=<4koVObX>|1cC?VkYXfVE|^tXUIFzNFis=G_H4qli< z-Ks&p{v&3EC~qcNYG-!-im28VFS`+^R7}N9+k}?O+XJM=y*i&kDFG3Ro5D;-YL*t{ z$g2D)%VbrtXW;xxSw96$d%FNvdj-XT&DQ5$E~ggZo+2|9gLKq{uTVwOnS(CC1JR6n zUd~En7Ih`r<4L6|7O-E+Lxj~THYR-#iXpi)tVNOJZ|@<25*Y4>5%G{ZGavZKU)C(a z79>hWn1TBDeZ%p&jn=$#%1@#g69gUkhnEVAyK7O}k|Xsn>WPAF7c+J!0z|4T{Tw5$ z3b08K-Z^$^Gy3hyiVdoO2c7t$m2;3=63{n-m+6aPeMZ4aPPxNpt(RINg)QU1kUwr{ zG6lz*77@@=3?tsnB#GB_!%jPi)|w5}-5ko&pjmg-(UA64VAccZb0CspsHCs+>d z)Wau6`E`H@`pa6H)SlD=@!f<+``ifEku;JRdYD)JMjF3rh>wswvg5zGpp1HxTy?IX z*EkV=GPMsM?yGcVTuoFNo4^aY@mBLES%(LHq;G@|ycsTpw3SA|aebyrC$&DWD2@g*tY$z z1HMW=3JmwCAq_Y}#iPSoJ_={%h<7?IDfW-Uf8%y>4jDbkr$!o4gS9ef*p>YINZ8(r z-HfzUh0TL!7iCdurW-NbC^iGpT5(?UWq}R*O9S1=%u`iCUmpf(&?zVo(49PTF;?ls z+gq(wKY`&{L=B3)9&E|2t{j)4tv<=|Z zWjcJ&am#01PRfvbC$g2iR0!=~J|^H1I(yy%8Dn}-}O96zTOcv1V&BPDHjTi!S2euD&rXB81w^7N89h(~y5%UAz$g&C_ zH=(GlBk2a}J^DiI813Q|jXkqJmEtun2^CK9!Xb8|G1h4#Wgv+K`zkI^NSKJDf-pq< zN8+OjsVK+?JYVu(yuzdcggx7F~rbpi^&l(P^ zEBq7~`>94*#7{dyrTxvFE`yWol2qutWqp4FTdyqmp_^beq}-OTV9$Ym>Kd-w_;4Sw z1WnM!ed-O%R(W_t_HPIRj%oFQ(H&ZBt*b{*+b+QR_9tSce4P4E;Gu6(xkH&fGv4W4 zKhvF=EUG9({h{R5M0u~=`H2(uRDL-=RhRJg-);HQdzeM{OZ{-Q%^L7YJVQm{rU(Rc z1C~=@z-3&-td-0ei1UXrk+TdBQC-+mvDkLP)kt`m!xF|ou68X}z-nARz6h;kxHqY^ zwCvR0V)8=vJ=S-rlo>ksu4=(lGWCHJRcXrui0$RYc^CGZ#BcMaqQ)<_fi43(k$lc` z`RJ1fQ|Wb>|I^r6hsCjM{Tc}nT!STOfWd;hyGwAlV8Ly03r+$AcXyZI?gaOsgS)#2 zxs!A5KFQ|X`|YpishMZ~sJE(ns%urP)$2EECjP~r10Ba-pl78$HhtxR=C`HtBZ>X- zrYYnD4}h;B8>Nn>z#a`BHPZv-1Cmd+KN=;$SQtFeK^=G7%BcghWRXNHwa_`qU+8w$ zUZ;sRK+At!o{-N0sWrGP)jXVpr(Pf{;n@AQlk@?`8g}r-tx>FNW!G^cukf9^5aP8C zFMpiZxH*Hd`W3585iCm&Igvu>QX#h>eEHFmEuVLZSlNwGYQrWfWC@@75LsDVN%H$t zi?zn50GfzUtHp4X2X(ud zU%Z>z@Bnmnci?Z5cmehtklb5)J&pRyWduKj%PX~^8N^TDaJ00h7?ke}1(w10%C;_f z_~_uCkiO^% zj&Bnlnbb|*5>xvI?9)%`?NtUz8OsC(t41@zN&5 z9z$7Yc~Zl!<^oZNEOCOs5C$!OA^TP&6auQ*$zRxg-I=k%)Zkb+OQy{FNYH0%@xma9 zq5GoE#?SN5Oz7TK^wMBo@LsHhE0u_#awUZgFjFWrMQtPp8LskvT zSSqR&zmvY*(f`4#U5q~M3zP4I#1L6!hY-nb20JASHWE7ed@4D#+k3CL6Y&s9P(kQC@GceGkyIo zXYtag)294fC!5OJ9rDh`^#UcksI~v7({lna(HR$xwP&#%7YTH_Acp6Jk9yFvf^VhqAbb7rK<}^yNiguy0_4Z@BBi@F5f$`iDaaNn%8Y2=H4$o4m(jj_*)S9%apj&bw2`7I9HvqM0yM<$hx&hs7Y(B|PSAsET)l*V z!1zyvU)A*fuW(Urtb$GQFY$s~EC9bx42k~!U=E*3F$+~uR_PRB9=yh?#Db5kRW{qQ zmnjq(+I%D#>bEeDwx_3&t-c!S%(zZX7Hkm>Dy!AVE$}(blo{)&a<9M|P4OziHe2hS zRF=qS7Rb^eTr@j&gqaavS3i;;I2E#4?>Vom6vJH@hBg3B8bzB#kuTuzT`4xc3mm1` zF*vHcf?_7-;o!3kt5DTkpAS5N)-%r>XZiN9WI_bv?`}aNkUtZOx1~SEr_$O%YBOA) zt`#V^;}oMjJF7-fQok5DD%Ix|3IERZvsY4Mm>E9pC*uWsvGoopM5yu^ry9A!0xGFu zm?X*9%ECXcK+?mAZRfA7ZHd(FtYGL@u$EHh?4B1}YN=p7LZL-Zpd!o0{LY?ZVzTqSNdJ5ujeiMz?RAGmtHJ)rd%V(0J z;vuF=VCv*{+Oyk(sT!R*Rg93XeL&lgakAZAEgq2q#CHy8QjIQ)t!CkGD6T|UG%;@} z75f4x-QJ1St%oRUSjZrHXPrfvHzD5Zyicpo1A#%OiFPRUk?9i`I7-X;KFHX_f%Cw85g{OcDQ5k9JpOBV{U@J4Ruw$JBz`Gl;cgNl6@^3b(iG)t6WQa`C}N40 zg4}~it9?3jVbmL&$i!5@+5EJ#9bX9maaQ`ySIIV}TgjqvH51rLQko&(zZzX@hfrg} z?r522YZqf6D8$uBBGmL5`;39iu<*je#LP%SeK-v~a=%H}oJ=P`qNXAGJzBoF=x2Z< zl%Eda^%Gr^KMNvI&JyA9!(?MQW;WCcE>uM@Npt;iMfhB1fT zEr*d>HA(;s@#GWyaX{}D&2c@D+{*pkPx38no-dP~bOw7w5FxsdPNm}s?E2q93B3g{ zf)L?!f$mV5_*(37Tgd7I;D9+e?&IO3h+h^|s_apSV)zDOJL%mxu6j!GnVjmzK2cF0 z;;`rd&vx=!k|0(ZikgQKLxA(q_2~tscB9))(Q09aAgzB1@hklEJj!{Jb~U;NyxbY; z8=9OCELi+!7Qg=ma>(I<#wRsVfQ%Q0NF1 zvOo>8;hLXTs`>Q=Cd;cBf2;IF>w6&ZHDf{X9PXErS|7qv*m%_S*1p6waf|;z^ zr^g(x9^cc-lq7V#<9&}p>ZD3HbNBkSt3Ztj`G{maL+ni#D8snMrYN3UDJaMIT1T8X zp5+VkO|5x1Mb^!mYra5-#f-Xr27UM?ylTRyN}iowPj8|J!l#o*?ucaMNYE#_wDgUc z{DgZ1Dmu(y?_<9$K6;15Mxg*rN%JA@qL&@CDoRH0sidbKKJ62KauyfIXHCU%8_coM z@2TBMRlYa<8b^lsgW0yIDh7^=d=P(c3Slp9LOq&HN;TLyw{fc}1f8I(z;z03mwDHb zUbz|C(ZB!2&Q}XlnxSp$-Uqz-!aB9_XznEBLMSR9El&R`=9e~lbZ)v*RB2+59TUaO z)U{F^lLFY(E$}urcSuWjaD8IJ_T!ND<97Dr2KG2v3|a?SYE>aY2W!v=YwHIeOsK!~OIl43ZU@6@z zH;*grc@4u30@#O`EbdR=^HhK;~S4(3(D@RM9890Mvst z27=b|Qw`~dpK<$pZJ<1iy?V|r=`5R;Wtgt-C;eOn8Glk0Yj>^ZWNcXR^BGA|_zOeUJA$|hdiTNN7&PnH~cK&!ILN)OyM&8zz_S_d^o z)y$PtC%v2wtzFWexfwOEI5o8K>kncGIMDsr*#p8m$?wEB(h-djvxK1mR9}vW@HOXX z60q5#;PLI|)kv5c&FaFf6Qq{v7B|wW1lwVbBp!F+abr^*TC!!e_)3d=2BC%f$Ol4& zC1W(KIOyX{>x4Jt3z6dV(-JXh&}NSC-8A{HdVHF{E`uUiajkIe$Nrh2GTe`ps3j(vzx)Y5Xkf;Vo>-3tD^h%T!Kqq2p@%EP93=kXOL)HZkmeH*nV zU_{Lw6`;PR6k^Gi* zDQR2!_ZE2q4CA`DY5pdZ3`b~g^g+iG_4b^{^oeU9;tR31Nl{i4<8?C_MZVkBi1fY{ zpY)SId0b)NanQV)M(9!3gmcdkxj9Aczp&7fvSW zi_lhGZiu-1#UigN=C!vw98F)gCXO&G`yHepf@3SbEawGv>g~#`trto$Xfg}Ejx5=) zg!~|Tl}@S!a#t!PFdT+|4kk*t8g0$fmP-$7-YPdeS{arW+j!7i2uFl^L!&a)J4YDh z$>@tQs|ug}w?T=sZSEc7Dhp47&G@yAA&dTU(NE&C)`VSTFOIHjQ9~qCEQn6tHFsg_ zkWZahyIc@O>~|@peJjx^>{29t?NIkQtcNiKehd!je8*_-T{o6E%twnlrAeCJQIt00 zh1Efw8dq#)+7)VnO%1x4rxHL34!|tl{OB`#j7!}W##~vA}ne}TL~;~12L*ik5xH)w=iH*=I`79~|Jn?0qa7QL{B@3B@DqBg8HCl||F zVG1$<;3Qf9)C(Yf=m-c6U}AXK-9x?Fz4I+KStWBxcQD*gL{P6U18R2ED^oSCP?xS! zB^xg4!M2%{$D(?BUVdKI_55(RXW>7&yC43h<+0DYc^+wOX*=1$Ni!~1_p^VRj+{Tn ztI;{fmOkU=NZny|qFv2lSndJ7c(huhhdf4AEL|FII36h~Upt%Gzx zOa!yRnp7P27-7rIfDjM}iV+BWh3bS9x(skD01{C@5IM5`(;!kYu%I!EbMc6f=VIbpPEtu9mO?^qct7- zSQatED&?9MQgmD9;uD-Dr(iM_Ui8&*j@I0w=58FbzE#FsJxQ(5VW;6{UANJ4>~7O8Sm4j_fLryJwM`gzSO7v7eP!&m$86MEvGrr8CM3B-D%IE&JJo549P$cv>*b>&pGOoe1 z&#@$j8QQPoPGd8qr&lsx7MVt3=FsftYJC=RN|prjLT4szBUdHrGxkC@0Nr8)SSa7u zbi$h=o44yE77!1y7b^g!8mrF^ad-xNGs2)kBhGGoL6(;U#+hnuF->SBKI*P9|OWgjiKWZCi{8;7{KcMbDIj z{K$z4I}YGHJcT!|xxfQG6h5LkL_cx8W11!QEI!rZ&^wS%PqhMh@7ww2E|V7~D&K;- zxT5U>4Iy%w{8h#84v;=R1QP~(_z~3u>j34HH34!Sv2O31$i@jVWSFDm751S&eYL3W zIV5{XJ#H+D5-p?|}{dnQU=%TWP4QZga6D1!8%&k4e7eZ-V^T8p%g0x?k zyIA|udu>E{;dRmMn_H$L)OtPt)&v}b`rbmUQ5hW;^x)@70O!|^hc((|qWcov7v)p? zB4OImn7n($$Uf8?NOO@43$sc#Gtu{|_1gRzfp`bka3eohd)8I4tmPn*D`E}URbwEb zu;`4xz&`FlJ98~m3>RAFa4-1f zZlVmct&f-&}|X zlLCEo`Ox~ek&As)d)8&STc(a-%1lcgT9twoV6+GI>Lvi5A}x6V80!%i;}qsJoE?wB12Rr$RI5 zxA8*cNflb(f1Wd9%4`##v^}fhzX;*KnAWgcia?)A3qtBAAmUuj)=R_Z;z_BLKc0oD zp44}Bk)Vq_n>%q~!t2E1z4r)(-T-t1a8D$WWZUmZH`VF1am8WirH#f>Eh6^l-#fKF zC{pRU=edpgOXWgc`axlHK1EZT!gDZYyn0Dl{{?3JO@=GIh$LtU0tur$s}yscICH_# zbE4EjYr6Juu06_T28VRh&aVu)fXm=I%YD&n| zfT3I~ToWk_XeC@EnVlEk|lhFOXWn=t5 z;a*77vT<7!SYg}~-db~PK>BqvW7x(t#g)xX*hX^VhmM!L;%lWh_)$W37K)c`1!%P@ zVWt*!W2qG^s*p#0*WBdMCvrJv#i&7S_f?dUD=y(xWA|o^3nkK=E2U79(#IO+`K!Ew zcdVvVLi;Q-yK?VN$PYAE-YwKI@7w%1yUjav(G<6D()t8si2ekp?{*uNC_Kw~5Q?O6 zfyPLjAS!#Y&}lt6t1x;q(YU+U;|_Et9%VbtsKTwrWw;O2Q540L=Rced)SS*szTzE& zgQQdBK?nGD?^9-(-QUz_FuY1gltfjL5)TueF#o8ojFxo@b((*@DjL?&N_%d?9dkp7 zo2iXIbR-$yAN+|kSZuN2%$xEHiA5@BgIgh=Q|(i)B4>WR(y3w6BMNry$G1H7W7?v; z0_j3H==oR+qjZm{M!~t74ci7V<_jh&uFDnRJ4shZ+gPn=$o~GJ zG_2$YJ0H`r?@L&F0`0zk`mcU1G=N(J2io8Th8DPx{qsUYO+r*u2Bw1%0WAFqGp3V2 zHjBc9qJyuO0i+?Ru0l*_(a#|s6t-Tz=Ht)kf4$^9*{L*e@u|}y0p}>#&R=8SqhXsR z-y^q&c4Kan^^MgNHVRQjjdeiyIfXewp&P~n@H2jY8>uA3`eP2I4mE=3c3KfCi zT;r{rJ@W0mZqqHu+zE_mrP41_=kY*DH6iSlIKM*vTi)bAmde@*6hBlh7Pd~OotE0K>|>y zP2xq~#yZ}1T8UkEPGer&rBw%SdtVJ}y_Yucr#-Li4XNa;OILA)l106f?^Sf9A8WHO zdwswn92$_q{PXkj@hdm>VM+)fQ7_E8&O+5Hv^Fw#ygtX5S9}aRc2DI+>VlKbC)7z& z5$`6(iq>IFi}hzApHv;v5Dz&t7)2Ry(FGqz?_DSB-Z6=cqjltLdAMBGRYmTIc_0uK z2uWOU!WE%RmePeOy(ANy#P})Yxo^zXJ!GRcJ zOkojpLFR>@u%0x1Q^Fe2!ncFVmM5(*R29ZzHHF0(E2UY1Mi*J3Ab6Z%;J>qkLtNm;8Av*m2E zF`)nd!vEl#k~hRJy&A=4U+)-ai5*ofBd-5DbF>)ep9f(lh!lJUy5LVLViti&qarRLw3L#v44mB z*FN;Q1ks=HU%`)Gus@?Ezu}=0fBCBahim>kcJjM=%YUl>8A*BG{B!5{@A_|~|7!m4 zuJY%E=iaj41X=mN5&oaPvp;FSuJ!X>{%_j4;y+&NzxMpkq0cS)Z>S8|$p6C+_%HkZ zocP=d|0eGJGx51iehz Date: Sat, 11 Jul 2026 10:04:38 +0200 Subject: [PATCH 06/34] Add make jupyter: local JupyterLite build with COI service-worker patch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per ADR-002 §3/§5 the Jupyter site gains a local build path (it was CI-only). `make jupyter`: - installs jupyterlite/requirements.txt + build into a gitignored .venv-jupyterlite - runs `jupyter lite build --contents jupyterlite/content --output-dir public/jupyter` (pypi/ wheels auto-indexed) - copies node_modules/lammps.js/dist -> public/jupyter/lammps/ so notebooks can load the engine from {site}/lammps/client.js (fails early with a clear message if npm install hasn't run) - runs scripts/jupyter_coi_patch.py public/jupyter scripts/jupyter_coi_patch.py is ported from lammps.js examples/notebook/coi_patch.py: it folds the coi-serviceworker trick into JupyterLite's own service worker (COOP/COEP/CORP on every response it serves) and injects a one-shot reload bootstrap into the app pages, so SharedArrayBuffer/KOKKOS work on static hosting. The asserts stay pinned to jupyterlite-core 0.8.0 internals and fail the build loudly on a version bump. The recipe uses single-line commands: macOS ships GNU Make 3.81, which ignores .ONESHELL. Also gitignore the build outputs/caches: public/jupyter, .venv-jupyterlite, .jupyterlite.doit.db, .cache/ (piplite wheel cache written to the lite dir, i.e. the repo root). Co-Authored-By: Claude Fable 5 --- .gitignore | 6 +- Makefile | 19 ++++++ scripts/jupyter_coi_patch.py | 109 +++++++++++++++++++++++++++++++++++ 3 files changed, 133 insertions(+), 1 deletion(-) create mode 100644 scripts/jupyter_coi_patch.py diff --git a/.gitignore b/.gitignore index d242e5c4..4f0653c5 100644 --- a/.gitignore +++ b/.gitignore @@ -6,4 +6,8 @@ cpp/obj cpp/lammps.mjs cpp/lammps.wasm cpp/lammps.worker.js -build/ \ No newline at end of file +build/ +/public/jupyter +/.venv-jupyterlite +.jupyterlite.doit.db +/.cache/ \ No newline at end of file diff --git a/Makefile b/Makefile index 64734257..90c9cc1a 100644 --- a/Makefile +++ b/Makefile @@ -34,6 +34,25 @@ frontend-lint: ## Run frontend linter (prettier) frontend-lint-fix: ## Fix frontend linting issues cd . && npx prettier --write . +## JUPYTER + +JUPYTER_VENV := .venv-jupyterlite + +jupyter: ## Build the JupyterLite site into public/jupyter (piplite wheels, lammps.js client, COI patch) + test -d node_modules/lammps.js/dist || { echo "error: node_modules/lammps.js/dist is missing — run 'make frontend-install' (npm install) first" >&2; exit 1; } + test -x $(JUPYTER_VENV)/bin/pip || python3 -m venv $(JUPYTER_VENV) + $(JUPYTER_VENV)/bin/pip install --quiet -r jupyterlite/requirements.txt build + rm -rf public/jupyter .jupyterlite.doit.db + # Wheels in /pypi/ (lammps-js, lammps-logfile) are auto-indexed + # into the piplite index because the lite dir is the repo root. + $(JUPYTER_VENV)/bin/jupyter lite build --contents jupyterlite/content --output-dir public/jupyter + # Notebooks import the engine from {site}/lammps/client.js; ship the whole + # built package so its relative imports (worker, wasm modules) resolve. + mkdir -p public/jupyter/lammps + cp -R node_modules/lammps.js/dist/. public/jupyter/lammps/ + # Cross-origin isolation on static hosting (ADR-002 §5). + $(JUPYTER_VENV)/bin/python scripts/jupyter_coi_patch.py public/jupyter + ## BACKEND backend-install: ## Install backend dependencies diff --git a/scripts/jupyter_coi_patch.py b/scripts/jupyter_coi_patch.py new file mode 100644 index 00000000..af21afee --- /dev/null +++ b/scripts/jupyter_coi_patch.py @@ -0,0 +1,109 @@ +#!/usr/bin/env python3 +"""Make the built JupyterLite site cross-origin isolated on static hosting. + +Ported from lammps.js `examples/notebook/coi_patch.py` (ADR-002 §5) and run +against Atomify's built site (`public/jupyter`). + +GitHub Pages cannot send the COOP/COEP headers that SharedArrayBuffer (and +therefore the KOKKOS multithreaded wasm build) requires. The standard +workaround is coi-serviceworker: a service worker that re-serves every +response with the headers added, plus a one-time page reload so the document +itself goes through the service worker. JupyterLite already owns the Jupyter +scope's service worker (its contents API — and the DriveFS mount — depend on +it), so instead of a second worker this script folds the header logic into +JupyterLite's own service worker: + +1. service-worker.js: wrap ``maybeFromCache`` — the function that answers + every plain GET the worker intercepts (documents, assets, and cross-origin + fetches alike) — so its responses carry COOP/COEP, plus CORP so the + assets remain embeddable under the policy they themselves impose. +2. every app page (lab/, notebooks/, tree/, …): inject a bootstrap that + reloads the page once, the first time the service worker takes control. + First visit: plain page registers the worker and reloads into an isolated + page. Later visits are isolated from the start. If isolation still fails + (e.g. an old worker version), the sessionStorage guard stops the loop and + the site simply runs single-threaded, as before. + +The upstream request for this is jupyterlite/jupyterlite#1409; if JupyterLite +grows a supported flag for it, this script can be replaced by that flag. +Pinned to the service-worker internals of jupyterlite-core 0.8.0 — the +asserts below fail the build loudly if a version bump changes them. +""" +import argparse +from pathlib import Path + +parser = argparse.ArgumentParser( + description="Patch a built JupyterLite site for cross-origin isolation." +) +parser.add_argument( + "site_dir", + type=Path, + help="the built JupyterLite site directory (e.g. public/jupyter)", +) +out = parser.parse_args().site_dir + +# --- 1. teach the service worker to add the isolation headers ------------- + +sw_path = out / "service-worker.js" +sw = sw_path.read_text() + +HOOK = "async function maybeFromCache(e){" +assert sw.count(HOOK) == 1, ( + "jupyterlite service-worker.js changed shape (expected maybeFromCache); " + "update jupyter_coi_patch.py for this jupyterlite-core version" +) +assert "_lammpsCoiHeaders" not in sw, "service worker already patched" + +sw = sw.replace( + HOOK, + "async function maybeFromCache(e){" + "return _lammpsCoiHeaders(await _lammpsMaybeFromCache(e))}" + "async function _lammpsMaybeFromCache(e){", + 1, +) +sw += ( + "\nfunction _lammpsCoiHeaders(r){" + "if(!r||r.status===0){return r}" + "const h=new Headers(r.headers);" + 'h.set("Cross-Origin-Embedder-Policy","require-corp");' + 'h.set("Cross-Origin-Opener-Policy","same-origin");' + 'h.set("Cross-Origin-Resource-Policy","cross-origin");' + "return new Response(r.body,{status:r.status,statusText:r.statusText,headers:h})" + "}\n" +) +sw_path.write_text(sw) + +# --- 2. reload app pages once, when the service worker takes control ------ + +BOOTSTRAP = """""" + +patched = 0 +for page in sorted(out.glob("*/index.html")): + html = page.read_text() + if "jupyter-config-data" not in html: + continue # not an app page + assert "lammps-coi-reload" not in html, f"{page} already patched" + page.write_text(html.replace("", "\n " + BOOTSTRAP, 1)) + patched += 1 + +assert patched >= 1, "no app pages found to patch" +print(f"jupyter_coi_patch: service worker patched, {patched} app pages bootstrapped") From 353d11e160a944e8f86a922953f8ea0c30f883ef Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 10:04:47 +0200 Subject: [PATCH 07/34] CI: build the Jupyter site via make jupyter, after npm install MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Jupyter site build now needs node_modules/lammps.js/dist (copied into public/jupyter/lammps/), so npm install moves ahead of the Python setup and the raw `jupyter lite build` step becomes `make jupyter` — the same steps as the local build (deps install, lite build with pypi/ wheel indexing, lammps.js dist copy, COI service-worker patch). Co-Authored-By: Claude Fable 5 --- .github/workflows/deploy.yaml | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/.github/workflows/deploy.yaml b/.github/workflows/deploy.yaml index 47000ea3..5bbd6c5b 100644 --- a/.github/workflows/deploy.yaml +++ b/.github/workflows/deploy.yaml @@ -12,22 +12,20 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - - name: Set up Python - uses: actions/setup-python@v5 - with: - python-version: "3.12" - - name: Install JupyterLite dependencies - run: | - python -m pip install -r jupyterlite/requirements.txt - - name: Build JupyterLite site - run: | - jupyter lite build --contents jupyterlite/content --output-dir public/jupyter - name: Use Node.js 20.x uses: actions/setup-node@v4 with: node-version: "20.x" + # npm install must come before the JupyterLite build: `make jupyter` + # copies node_modules/lammps.js/dist into public/jupyter/lammps. - name: Install Packages run: npm install + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + - name: Build JupyterLite site + run: make jupyter - name: Run Typecheck run: npm run typecheck - name: Run Tests From e9a766576280095e2184934e0591222cfe8258d2 Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 10:07:22 +0200 Subject: [PATCH 08/34] Add projects store model and project-run orchestration (ADR-001/002) - projects model: library, active project (files/runs), typed navigation state, run queue with sweeps, quick-run scratch projects, debounced editor saves with pre-snapshot flush - runs execute through the unchanged engine pipeline: each run materializes a Simulation with id /runs/ so all engine writes land inside the run directory; outputs stream to project storage on a 3s throttle and fully at completion; run.json records outcome - simulation model: remove the stale one-way syncFilesJupyterLite (its outputs never left the worker since the worker migration); run() now returns {stopReason, errorMessage} for the orchestrator - AnalyzeNotebook: project-scoped glob-based template (latest run, cross-run comparison reading run.json vars, lammps-js scripting cell) - scriptVariables: detect overridable 'variable X equal ' params Co-Authored-By: Claude Fable 5 --- src/store/index.ts | 12 +- src/store/model.ts | 3 + src/store/projects.ts | 857 ++++++++++++++++++++++++++++++ src/store/simulation.ts | 160 +----- src/types.ts | 12 + src/utils/AnalyzeNotebook.test.ts | 194 ++----- src/utils/AnalyzeNotebook.ts | 176 ++++-- src/utils/scriptVariables.ts | 62 +++ 8 files changed, 1128 insertions(+), 348 deletions(-) create mode 100644 src/store/projects.ts create mode 100644 src/utils/scriptVariables.ts diff --git a/src/store/index.ts b/src/store/index.ts index 6ccf6ada..448ac7a2 100644 --- a/src/store/index.ts +++ b/src/store/index.ts @@ -1,7 +1,17 @@ import { createStore } from "easy-peasy"; import { storeModel, StoreModel } from "./model"; +import type { StoreInjections } from "./projects"; +import { + createIndexedDbProjectStorage, + createMemoryProjectStorage, +} from "../storage"; -const store = createStore(storeModel); +const injections: StoreInjections = { + libraryStorage: createIndexedDbProjectStorage(), + scratchStorage: createMemoryProjectStorage(), +}; + +const store = createStore(storeModel, { injections }); // Added support for hot reloading // Vite uses import.meta.hot instead of module.hot diff --git a/src/store/model.ts b/src/store/model.ts index 9195c1a4..9ccccb03 100644 --- a/src/store/model.ts +++ b/src/store/model.ts @@ -7,6 +7,7 @@ import { import { ProcessingModel, processingModel } from "./processing"; import { RenderModel, renderModel } from "./render"; import { AppModel, appModel } from "./app"; +import { ProjectsModel, projectsModel } from "./projects"; import { persist } from "easy-peasy"; export interface StoreModel { @@ -16,6 +17,7 @@ export interface StoreModel { processing: ProcessingModel; app: AppModel; render: RenderModel; + projects: ProjectsModel; } export const storeModel: StoreModel = { @@ -25,4 +27,5 @@ export const storeModel: StoreModel = { processing: processingModel, app: appModel, render: renderModel, + projects: projectsModel, }; diff --git a/src/store/projects.ts b/src/store/projects.ts new file mode 100644 index 00000000..20dd8c59 --- /dev/null +++ b/src/store/projects.ts @@ -0,0 +1,857 @@ +/** + * The projects model (ADR-001/003): the persisted library, the active + * project's working tree + run history, navigation state, and run + * orchestration (single runs and sweeps) on top of the storage layer. + * + * Runs execute through the existing simulation model unchanged: each run + * materializes a legacy `Simulation` whose id is `/runs/`, + * so the engine pipeline writes into the run directory by construction. + * This model does the work around that: snapshot, run.json, output copies + * into project storage, and queue advancement. + */ + +import { action, Action, thunk, Thunk, Actions, State } from "easy-peasy"; +import { StoreModel } from "./model"; +import type { Simulation } from "./simulation"; +import type { + FileStat, + ProjectMeta, + ProjectStorage, + RunListEntry, + RunMeta, +} from "../storage"; +import { + allocateRunDir, + bytesToWriteContent, + listRuns, + readRunMeta, + reconcileRuns, + RUNS_DIR, + snapshotWorkingTree, + writeRunMeta, +} from "../storage"; +import { projectAnalysisNotebook } from "../utils/AnalyzeNotebook"; +import { track } from "../utils/metrics"; + +export interface StoreInjections { + /** The persistent library (JupyterLite contents IndexedDB). */ + libraryStorage: ProjectStorage; + /** Scratch space for quick runs and embeds — never visible to Jupyter. */ + scratchStorage: ProjectStorage; +} + +export type ProjectTab = "files" | "runs" | "notebook"; + +export type Screen = + | { name: "home" } + | { name: "examples" } + | { + name: "project"; + dirName: string; + tab: ProjectTab; + /** files tab: file open in the editor sub-screen. */ + filePath?: string; + /** runs tab: run open in the run-detail sub-screen. */ + runId?: string; + }; + +export interface ActiveProject { + meta: ProjectMeta; + /** True when the project lives on scratch storage (quick run / embed). */ + quick: boolean; + /** Working-tree root listing (directories included). */ + files: FileStat[]; + runs: RunListEntry[]; +} + +export interface NewProjectFile { + fileName: string; + content?: string; + url?: string; +} + +export interface CreateProjectPayload { + displayName: string; + color?: string; + source?: ProjectMeta["source"]; + files: NewProjectFile[]; + inputScript?: string; + /** Create on scratch storage (quick run) instead of the library. */ + quick?: boolean; + /** Start a run immediately after creation. */ + autoStart?: boolean; +} + +export interface RunRequest { + dirName: string; + quick: boolean; + inputScript: string; + vars: Record; + useKokkos: boolean; + threads: number; + sweepId?: string; +} + +export type SaveState = "saving" | "saved" | "error"; + +/** Files ≤ this size stream to storage during the run; larger wait for the end. */ +const MID_RUN_COPY_CAP = 4 * 1024 * 1024; +const MID_RUN_COPY_INTERVAL_MS = 3000; + +/** Wrapper artifacts from the vars-injection mechanism — engine detail. */ +const RUN_INTERNAL_FILE = /^(_vars_|_wrapper_)/; + +const SCRIPT_FILE = /(^|\/)in\.[^/]*$|\.in$/; + +export function isScriptFile(fileName: string): boolean { + return SCRIPT_FILE.test(fileName); +} + +// Debounced editor saves live outside the (immer) store: content buffers and +// timers are not state. flushPendingSaves is awaited before every snapshot so +// what runs is always what is recorded (ADR-001 §8). +interface PendingSave { + dirName: string; + quick: boolean; + path: string; + content: string; + timer: ReturnType; +} +const pendingSaves = new Map(); + +export interface ProjectsModel { + screen: Screen; + projects: ProjectMeta[]; + active?: ActiveProject; + /** Editor cache: project-relative path -> content (active project only). */ + fileContents: Record; + saveStates: Record; + runQueue: RunRequest[]; + /** The run this session is executing right now. */ + activeRun?: { dirName: string; runId: string; quick: boolean }; + /** One-shot notice (toasts): run finished, sweep interrupted, quota… */ + notice?: string; + + setScreen: Action; + setProjects: Action; + setActive: Action; + patchActiveMeta: Action; + setFileContent: Action; + clearFileContents: Action; + setSaveState: Action; + setRunQueue: Action; + shiftRunQueue: Action; + setActiveRun: Action< + ProjectsModel, + { dirName: string; runId: string; quick: boolean } | undefined + >; + setNotice: Action; + + initialize: Thunk; + openProject: Thunk< + ProjectsModel, + { dirName: string; quick?: boolean; tab?: ProjectTab }, + StoreInjections, + StoreModel + >; + refreshActive: Thunk; + createProject: Thunk< + ProjectsModel, + CreateProjectPayload, + StoreInjections, + StoreModel, + Promise + >; + renameProject: Thunk; + duplicateProject: Thunk; + deleteProject: Thunk; + saveQuickAsProject: Thunk; + + readFile: Thunk< + ProjectsModel, + string, + StoreInjections, + StoreModel, + Promise + >; + saveFileDebounced: Thunk< + ProjectsModel, + { path: string; content: string }, + StoreInjections, + StoreModel + >; + flushPendingSaves: Thunk; + writeFile: Thunk< + ProjectsModel, + { path: string; content: string | Uint8Array }, + StoreInjections, + StoreModel + >; + removeFile: Thunk; + renameFile: Thunk< + ProjectsModel, + { from: string; to: string }, + StoreInjections, + StoreModel + >; + setInputScript: Thunk; + + startRuns: Thunk< + ProjectsModel, + Omit[] , + StoreInjections, + StoreModel + >; + cancelQueuedRuns: Thunk; + deleteRun: Thunk; + executeNextRun: Thunk; +} + +function storageFor( + injections: StoreInjections, + quick: boolean, +): ProjectStorage { + return quick ? injections.scratchStorage : injections.libraryStorage; +} + +/** Recursively copy a directory subtree between storages (same layout). */ +async function copyTree( + from: ProjectStorage, + fromDir: string, + to: ProjectStorage, + toDir: string, + subdir?: string, +) { + const entries = await from.list(fromDir, subdir); + for (const entry of entries) { + if (entry.type === "directory") { + await copyTree(from, fromDir, to, toDir, entry.path); + } else { + await to.write(toDir, entry.path, await from.read(fromDir, entry.path)); + } + } +} + +/** Best-effort viewport capture for `frame.png` (ADR-003 §2). */ +async function captureViewport(): Promise { + try { + const canvas = document.querySelector("canvas"); + if (!canvas || canvas.width === 0) { + return null; + } + const blob = await new Promise((resolve) => + canvas.toBlob(resolve, "image/png"), + ); + if (!blob || blob.size < 200) { + // WebGL without preserveDrawingBuffer yields empty frames; skip those. + return null; + } + return new Uint8Array(await blob.arrayBuffer()); + } catch { + return null; + } +} + +export const projectsModel: ProjectsModel = { + screen: { name: "home" }, + projects: [], + fileContents: {}, + saveStates: {}, + runQueue: [], + + setScreen: action((state, screen) => { + state.screen = screen; + }), + setProjects: action((state, projects) => { + state.projects = projects; + }), + setActive: action((state, active) => { + state.active = active; + }), + patchActiveMeta: action((state, meta) => { + if (state.active) { + state.active.meta = meta; + } + state.projects = state.projects.map((p) => + p.dirName === meta.dirName ? meta : p, + ); + }), + setFileContent: action((state, { path, content }) => { + state.fileContents[path] = content; + }), + clearFileContents: action((state) => { + state.fileContents = {}; + state.saveStates = {}; + }), + setSaveState: action((state, { path, state: saveState }) => { + state.saveStates[path] = saveState; + }), + setRunQueue: action((state, queue) => { + state.runQueue = queue; + }), + shiftRunQueue: action((state) => { + state.runQueue = state.runQueue.slice(1); + }), + setActiveRun: action((state, run) => { + state.activeRun = run; + }), + setNotice: action((state, notice) => { + state.notice = notice; + }), + + initialize: thunk(async (actions, _, { injections }) => { + const projects = await injections.libraryStorage.listProjects(); + actions.setProjects(projects); + }), + + openProject: thunk( + async (actions, { dirName, quick = false, tab }, helpers) => { + const storage = storageFor(helpers.injections, quick); + const metas = await storage.listProjects(); + const meta = metas.find((m) => m.dirName === dirName); + if (!meta) { + actions.setNotice(`Project ${dirName} was not found.`); + return; + } + const owned = new Set( + helpers.getState().activeRun ? [helpers.getState().activeRun!.runId] : [], + ); + const interrupted = await reconcileRuns(storage, dirName, owned); + if (interrupted.length > 0) { + actions.setNotice( + `${interrupted.length} interrupted ${interrupted.length === 1 ? "run" : "runs"} found in ${meta.displayName}.`, + ); + } + const [files, runs] = await Promise.all([ + storage.list(dirName), + listRuns(storage, dirName), + ]); + actions.clearFileContents(); + actions.setActive({ meta, quick, files, runs }); + const hasRuns = runs.length > 0; + actions.setScreen({ + name: "project", + dirName, + tab: tab ?? (hasRuns ? "runs" : "files"), + }); + }, + ), + + refreshActive: thunk(async (actions, _, { getState, injections }) => { + const active = getState().active; + if (!active) { + return; + } + const storage = storageFor(injections, active.quick); + const owned = new Set( + getState().activeRun ? [getState().activeRun!.runId] : [], + ); + await reconcileRuns(storage, active.meta.dirName, owned); + const [files, runs] = await Promise.all([ + storage.list(active.meta.dirName), + listRuns(storage, active.meta.dirName), + ]); + actions.setActive({ ...active, files, runs }); + }), + + createProject: thunk(async (actions, payload, helpers) => { + const storage = storageFor(helpers.injections, payload.quick ?? false); + + // Fetch lazy (url-only) files first so creation is all-or-nothing. + const files: { fileName: string; content: string }[] = []; + for (const file of payload.files) { + let content = file.content; + if (content === undefined && file.url) { + const response = await fetch(file.url); + if (!response.ok) { + throw new Error( + `Could not download ${file.fileName} (HTTP ${response.status}).`, + ); + } + content = await response.text(); + } + files.push({ fileName: file.fileName, content: content ?? "" }); + } + + const scripts = files.filter((f) => isScriptFile(f.fileName)); + const inputScript = + payload.inputScript ?? (scripts.length === 1 ? scripts[0].fileName : undefined); + + const meta = await storage.createProject({ + displayName: payload.displayName, + color: payload.color, + source: payload.source, + inputScript, + }); + for (const file of files) { + await storage.write(meta.dirName, file.fileName, file.content); + } + // A starter notebook that shares the project filesystem — generated once, + // owned by the user afterwards. Curated notebooks (uploaded/example) win. + const hasNotebook = files.some((f) => f.fileName.endsWith(".ipynb")); + if (!hasNotebook) { + await storage.write( + meta.dirName, + "analysis.ipynb", + JSON.stringify(projectAnalysisNotebook(meta)), + ); + } + + if (!payload.quick) { + const projects = await helpers.injections.libraryStorage.listProjects(); + actions.setProjects(projects); + } + track("Project.New", { source: payload.source?.type }); + await actions.openProject({ + dirName: meta.dirName, + quick: payload.quick ?? false, + tab: "files", + }); + if (payload.autoStart && inputScript) { + await actions.startRuns([ + { inputScript, vars: {}, useKokkos: false, threads: 1 }, + ]); + } + return meta; + }), + + renameProject: thunk(async (actions, displayName, { getState, injections }) => { + const active = getState().active; + if (!active) { + return; + } + const storage = storageFor(injections, active.quick); + const meta = await storage.updateProjectMeta(active.meta.dirName, { + displayName, + }); + actions.patchActiveMeta(meta); + }), + + duplicateProject: thunk(async (actions, _, { getState, injections }) => { + const active = getState().active; + if (!active) { + return; + } + const storage = storageFor(injections, active.quick); + // Working tree + notebook only, no runs (ADR-003 §5). + const files: NewProjectFile[] = []; + for (const entry of active.files) { + if (entry.type === "directory") { + continue; + } + const content = await storage.read(active.meta.dirName, entry.path); + files.push({ + fileName: entry.path, + content: typeof content === "string" ? content : undefined, + }); + } + await actions.createProject({ + displayName: `${active.meta.displayName} (copy)`, + color: active.meta.color, + source: active.meta.source, + inputScript: active.meta.inputScript, + files, + }); + }), + + deleteProject: thunk(async (actions, dirName, { getState, injections }) => { + const activeRun = getState().activeRun; + if (activeRun && activeRun.dirName === dirName) { + actions.setNotice("This project has a run in progress — stop it first."); + return; + } + await injections.libraryStorage.deleteProject(dirName); + const projects = await injections.libraryStorage.listProjects(); + actions.setProjects(projects); + if (getState().active?.meta.dirName === dirName) { + actions.setActive(undefined); + actions.setScreen({ name: "home" }); + } + }), + + saveQuickAsProject: thunk(async (actions, _, helpers) => { + const active = helpers.getState().active; + if (!active || !active.quick) { + return; + } + await actions.flushPendingSaves(); + const { scratchStorage, libraryStorage } = helpers.injections; + const meta = await libraryStorage.createProject({ + displayName: active.meta.displayName.replace(/^Quick run — /, ""), + color: active.meta.color, + source: active.meta.source, + inputScript: active.meta.inputScript, + }); + // Everything materializes: working tree AND completed runs (ADR-003). + const entries = await scratchStorage.list(active.meta.dirName); + for (const entry of entries) { + if (entry.path.startsWith(".atomify")) { + continue; // fresh identity: keep the new project.json + } + if (entry.type === "directory") { + await copyTree( + scratchStorage, + active.meta.dirName, + libraryStorage, + meta.dirName, + entry.path, + ); + } else { + await libraryStorage.write( + meta.dirName, + entry.path, + await scratchStorage.read(active.meta.dirName, entry.path), + ); + } + } + const projects = await libraryStorage.listProjects(); + actions.setProjects(projects); + track("Project.SavedFromQuickRun", {}); + await actions.openProject({ dirName: meta.dirName }); + }), + + readFile: thunk(async (actions, path, { getState, injections }) => { + const active = getState().active; + if (!active) { + return ""; + } + const cached = getState().fileContents[path]; + if (cached !== undefined) { + return cached; + } + const storage = storageFor(injections, active.quick); + const content = await storage.read(active.meta.dirName, path); + const text = + typeof content === "string" ? content : new TextDecoder().decode(content); + actions.setFileContent({ path, content: text }); + return text; + }), + + saveFileDebounced: thunk( + (actions, { path, content }, { getState, injections }) => { + const active = getState().active; + if (!active) { + return; + } + actions.setFileContent({ path, content }); + actions.setSaveState({ path, state: "saving" }); + const key = `${active.meta.dirName}/${path}`; + const existing = pendingSaves.get(key); + if (existing) { + clearTimeout(existing.timer); + } + const save: PendingSave = { + dirName: active.meta.dirName, + quick: active.quick, + path, + content, + timer: setTimeout(() => { + pendingSaves.delete(key); + storageFor(injections, save.quick) + .write(save.dirName, save.path, save.content) + .then(() => actions.setSaveState({ path, state: "saved" })) + .catch(() => actions.setSaveState({ path, state: "error" })); + }, 500), + }; + pendingSaves.set(key, save); + }, + ), + + flushPendingSaves: thunk(async (actions, _, { injections }) => { + const saves = [...pendingSaves.values()]; + pendingSaves.clear(); + for (const save of saves) { + clearTimeout(save.timer); + try { + await storageFor(injections, save.quick).write( + save.dirName, + save.path, + save.content, + ); + actions.setSaveState({ path: save.path, state: "saved" }); + } catch { + actions.setSaveState({ path: save.path, state: "error" }); + } + } + }), + + writeFile: thunk(async (actions, { path, content }, { getState, injections }) => { + const active = getState().active; + if (!active) { + return; + } + await storageFor(injections, active.quick).write( + active.meta.dirName, + path, + content, + ); + if (typeof content === "string") { + actions.setFileContent({ path, content }); + } + await actions.refreshActive(); + }), + + removeFile: thunk(async (actions, path, { getState, injections }) => { + const active = getState().active; + if (!active) { + return; + } + await storageFor(injections, active.quick).remove(active.meta.dirName, path); + if (active.meta.inputScript === path) { + const meta = await storageFor(injections, active.quick).updateProjectMeta( + active.meta.dirName, + { inputScript: undefined }, + ); + actions.patchActiveMeta(meta); + } + await actions.refreshActive(); + }), + + renameFile: thunk(async (actions, { from, to }, { getState, injections }) => { + const active = getState().active; + if (!active) { + return; + } + const storage = storageFor(injections, active.quick); + await storage.rename(active.meta.dirName, from, to); + if (active.meta.inputScript === from) { + const meta = await storage.updateProjectMeta(active.meta.dirName, { + inputScript: to, + }); + actions.patchActiveMeta(meta); + } + await actions.refreshActive(); + }), + + setInputScript: thunk(async (actions, path, { getState, injections }) => { + const active = getState().active; + if (!active) { + return; + } + const meta = await storageFor(injections, active.quick).updateProjectMeta( + active.meta.dirName, + { inputScript: path }, + ); + actions.patchActiveMeta(meta); + }), + + startRuns: thunk(async (actions, requests, { getState }) => { + const active = getState().active; + if (!active || requests.length === 0) { + return; + } + const sweepId = + requests.length > 1 + ? `sweep-${Math.random().toString(36).slice(2, 8)}` + : undefined; + const queue: RunRequest[] = requests.map((request) => ({ + ...request, + dirName: active.meta.dirName, + quick: active.quick, + sweepId, + })); + actions.setRunQueue([...getState().runQueue, ...queue]); + await actions.executeNextRun(); + }), + + cancelQueuedRuns: thunk((actions) => { + actions.setRunQueue([]); + }), + + deleteRun: thunk(async (actions, runId, { getState, injections }) => { + const active = getState().active; + if (!active) { + return; + } + const activeRun = getState().activeRun; + if (activeRun && activeRun.runId === runId) { + actions.setNotice("This run is in progress — stop it first."); + return; + } + await storageFor(injections, active.quick).remove( + active.meta.dirName, + `${RUNS_DIR}/${runId}`, + ); + await actions.refreshActive(); + }), + + executeNextRun: thunk(async (actions, _, helpers) => { + const { getState, getStoreState, getStoreActions, injections } = helpers; + if (getState().activeRun) { + return; + } + const request = getState().runQueue[0]; + if (!request) { + return; + } + actions.shiftRunQueue(); + + const storage = storageFor(injections, request.quick); + const storeActions = getStoreActions() as Actions; + const lammps = (getStoreState() as State).simulation.lammps; + if (!lammps) { + actions.setNotice("The simulation engine is still loading."); + actions.setRunQueue([]); + return; + } + + await actions.flushPendingSaves(); + + // 1. Claim the run directory and snapshot the working tree (ADR-001 §5). + const runId = await allocateRunDir(storage, request.dirName); + const { gaps } = await snapshotWorkingTree(storage, request.dirName, runId); + let runMeta: RunMeta = { + schemaVersion: 1, + id: runId, + inputScript: request.inputScript, + vars: Object.keys(request.vars).length ? request.vars : undefined, + sweepId: request.sweepId, + status: "running", + startedAt: new Date().toISOString(), + snapshotGaps: gaps && gaps.length ? gaps : undefined, + origin: request.sweepId ? "sweep" : "app", + }; + await writeRunMeta(storage, request.dirName, runMeta); + actions.setActiveRun({ + dirName: request.dirName, + runId, + quick: request.quick, + }); + actions.setScreen({ + name: "project", + dirName: request.dirName, + tab: "runs", + runId, + }); + await actions.refreshActive(); + + // 2. Materialize the legacy Simulation from the snapshot: its id is the + // run directory, so the whole engine pipeline (kokkos preprocessing, + // vars wrapper, wasm FS writes, streaming) lands inside the run. + const snapshotEntries = await storage.list( + request.dirName, + `${RUNS_DIR}/${runId}`, + ); + const files = []; + for (const entry of snapshotEntries) { + if (entry.type === "directory") { + continue; + } + const relative = entry.path.slice(`${RUNS_DIR}/${runId}/`.length); + if (relative.startsWith(".atomify")) { + continue; + } + const content = await storage.read(request.dirName, entry.path); + files.push({ + fileName: relative, + content: + typeof content === "string" + ? content + : new TextDecoder().decode(content), + }); + } + const simulation: Simulation = { + id: `${request.dirName}/${RUNS_DIR}/${runId}`, + files, + inputScript: request.inputScript, + start: false, + vars: Object.keys(request.vars).length ? request.vars : undefined, + }; + + // 3. Copy outputs out of the worker FS into project storage on a + // throttle, so the notebook can analyze a run in flight (ADR-002 §4). + const workdir = `/${simulation.id}`; + const copyOutputs = async (maxBytes?: number) => { + if (!lammps.snapshotWorkdir) { + return; + } + try { + const snapshot = await lammps.snapshotWorkdir(workdir, maxBytes); + for (const file of snapshot.files) { + if (RUN_INTERNAL_FILE.test(file.path)) { + continue; + } + await storage.write( + request.dirName, + `${RUNS_DIR}/${runId}/${file.path}`, + bytesToWriteContent(file.path, file.bytes), + ); + } + } catch (error) { + actions.setNotice( + `Could not copy run outputs to storage: ${ + error instanceof Error ? error.message : String(error) + }`, + ); + } + }; + + let stopReason: string = "failed"; + let errorMessage: string | undefined; + const interval = setInterval(() => { + void copyOutputs(MID_RUN_COPY_CAP); + }, MID_RUN_COPY_INTERVAL_MS); + try { + await storeActions.simulation.newSimulation(simulation); + const result = (await storeActions.simulation.run()) as + | { stopReason: string; errorMessage?: string } + | undefined; + stopReason = result?.stopReason ?? "completed"; + errorMessage = result?.errorMessage; + } catch (error) { + errorMessage = error instanceof Error ? error.message : String(error); + } finally { + clearInterval(interval); + } + + // 4. Final full output copy + run.json + best-effort frame capture. + await copyOutputs(undefined); + const lammpsState = (getStoreState() as State).simulation; + const existing = await readRunMeta(storage, request.dirName, runId); + runMeta = { + ...(existing ?? runMeta), + status: + stopReason === "canceled" + ? "canceled" + : stopReason === "failed" + ? "failed" + : "completed", + finishedAt: new Date().toISOString(), + error: stopReason === "failed" ? errorMessage : undefined, + stats: { + timesteps: lammps.getTimesteps(), + numAtoms: lammps.getNumAtoms(), + wallSeconds: + (Date.now() - new Date(runMeta.startedAt).getTime()) / 1000, + }, + }; + await writeRunMeta(storage, request.dirName, runMeta); + const frame = await captureViewport(); + if (frame) { + try { + await storage.write( + request.dirName, + `${RUNS_DIR}/${runId}/.atomify/frame.png`, + frame, + ); + } catch { + // Frame capture is decorative; never fail the run over it. + } + } + + actions.setActiveRun(undefined); + await actions.refreshActive(); + const state = getState(); + if (state.screen.name !== "project" || state.screen.runId !== runId) { + actions.setNotice( + `Run #${runId.replace(/^run-0*/, "")} ${runMeta.status}.`, + ); + } + if (lammpsState.running) { + // Defensive: the engine reported running after run() settled. + return; + } + // 5. Advance the sweep queue. + await actions.executeNextRun(); + }), +}; diff --git a/src/store/simulation.ts b/src/store/simulation.ts index 7a91404d..8a0d714b 100644 --- a/src/store/simulation.ts +++ b/src/store/simulation.ts @@ -2,14 +2,12 @@ import { action, Action, thunk, Thunk, Actions, State } from "easy-peasy"; import { StoreModel } from "./model"; import { LammpsWeb } from "../types"; import { AtomTypes, AtomType, hexToRgb } from "../utils/atomtypes"; -import AnalyzeNotebook from "../utils/AnalyzeNotebook"; import { track, time_event, getEmbeddingParams } from "../utils/metrics"; import { scriptOptsIntoKokkos, prepareScriptForSerialStyles, } from "../utils/kokkos"; import * as THREE from "three"; -import localforage from "localforage"; import ColorModifier from "../modifiers/colormodifier"; import Modifier from "../modifiers/modifier"; import { SimulationFile } from "./app"; @@ -22,13 +20,6 @@ import { } from "../utils/parsers"; import { getWasm } from "../wasm/wasmInstance"; -localforage.config({ - driver: localforage.INDEXEDDB, - name: "JupyterLite Storage", - storeName: "files", // Should be alphanumeric, with underscores. - description: "some description", -}); - /** * Builds a LAMMPS variable-injection script from a vars map, writes * both the vars file and the wrapper file to the WASM filesystem, @@ -177,8 +168,15 @@ export interface SimulationModel { setLammps: Action; extractAndApplyAtomifyCommands: Thunk; syncFilesWasm: Thunk; - syncFilesJupyterLite: Thunk; - run: Thunk; + run: Thunk< + SimulationModel, + undefined | void, + unknown, + StoreModel, + Promise< + { stopReason: StopReason; errorMessage: string | undefined } | undefined + > + >; newSimulation: Thunk; lammps?: LammpsWeb; lastError?: string; @@ -330,137 +328,6 @@ export const simulationModel: SimulationModel = { } }, ), - syncFilesJupyterLite: thunk( - async ( - actions, - dummy: undefined, - { getStoreState }: { getStoreState: () => State }, - ) => { - const simulation = getStoreState().simulation.simulation; - if (!simulation) { - return; - } - const wasm = getWasm(); - const fileNames: string[] = wasm.FS.readdir(`/${simulation.id}`); - const files: { [key: string]: SimulationFile } = {}; - fileNames.forEach((fileName: string) => { - if ([".", ".."].includes(fileName)) { - return; - } - - const filePath = `/${simulation.id}/${fileName}`; - files[fileName] = { - content: wasm.FS.readFile(filePath, { encoding: "utf8" }), - fileName, - }; - }); - - type JupyterFileType = "directory" | "file" | "notebook"; - - const createLocalForageObject = ( - name: string, - path: string, - type: JupyterFileType, - content?: string | object, - existingCreated?: string, - ) => { - let mimetype = "text/plain"; - let format = "text"; - let size = 0; - const now = new Date().toISOString(); - - if ( - type === "directory" || - type === "notebook" || - typeof content === "object" - ) { - mimetype = "application/json"; - format = "json"; - } - - if (content) { - if (typeof content === "string") { - size = content.length; - } else { - size = JSON.stringify(content).length; - } - } - - return { - name, - path, - last_modified: now, - created: existingCreated ?? now, - format, - mimetype: mimetype, - content: content ? content : [], - size, - writable: true, - type, - }; - }; - - interface LocalForageEntry { - created?: string; - [key: string]: unknown; - } - - // Add an example analysis file - const analyzeFileName = "analyze.ipynb"; - const existingAnalyze = - await localforage.getItem(analyzeFileName); - await localforage.setItem( - analyzeFileName, - createLocalForageObject( - analyzeFileName, - analyzeFileName, - "notebook", - AnalyzeNotebook(simulation), - existingAnalyze?.created, - ), - ); - - const existingDir = - await localforage.getItem(simulation.id); - await localforage.setItem( - simulation.id, - createLocalForageObject( - simulation.id, - simulation.id, - "directory", - undefined, - existingDir?.created, - ), - ); - for (const file of Object.values(files)) { - let type: JupyterFileType = "file"; - let content: string | object | undefined = file.content; - const filePath = `${simulation.id}/${file.fileName}`; - - if (file.fileName.endsWith("ipynb") && typeof content === "string") { - type = "notebook"; - content = JSON.parse(content) as object; - const existingNotebook = await localforage.getItem(filePath); - if (existingNotebook) { - // We don't want to overwrite the content of the notebook - continue; - } - } - const existingFile = - await localforage.getItem(filePath); - await localforage.setItem( - filePath, - createLocalForageObject( - file.fileName, - filePath, - type, - content, - existingFile?.created, - ), - ); - } - }, - ), run: thunk( async ( actions, @@ -565,15 +432,17 @@ export const simulationModel: SimulationModel = { ); actions.setPaused(false); - handleRunResult({ + const stopReason = handleRunResult({ errorMessage, simulationId: simulation.id, metricsData, actions, allActions, }); - actions.syncFilesJupyterLite(); allActions.simulationStatus.setLastCommand(undefined); + // Callers orchestrating project runs (store/projects.ts) record this + // outcome in the run's metadata. + return { stopReason, errorMessage }; }, ), newSimulation: thunk( @@ -658,9 +527,6 @@ export const simulationModel: SimulationModel = { actions.setSimulation(simulation); // Set it again now that files are updated wasm.FS.chdir(`/${simulation.id}`); - // Sync files to JupyterLite storage now that they're available in WASM filesystem - await actions.syncFilesJupyterLite(); - await allActions.app.setStatus(undefined); if (simulation.start) { actions.run(); diff --git a/src/types.ts b/src/types.ts index c524cdd4..66e93b96 100644 --- a/src/types.ts +++ b/src/types.ts @@ -33,6 +33,18 @@ export type LammpsWeb = { category: "compute" | "fix" | "variable", name: string | null, ) => void; + /** + * Read the files under a wasm-FS directory (the run-outputs data path, + * ADR-001 §5). Only implemented by the worker proxy; safe mid-run because + * MEMFS reads never re-enter the suspended module. + */ + snapshotWorkdir?: ( + dir: string, + maxBytes?: number, + ) => Promise<{ + files: { path: string; bytes: Uint8Array }[]; + skipped: { path: string; size: number }[]; + }>; getMemoryUsage: () => number; getPositionsPointer: () => number; diff --git a/src/utils/AnalyzeNotebook.test.ts b/src/utils/AnalyzeNotebook.test.ts index c44fb013..8369e02f 100644 --- a/src/utils/AnalyzeNotebook.test.ts +++ b/src/utils/AnalyzeNotebook.test.ts @@ -1,163 +1,63 @@ -import { describe, it, expect } from "vitest"; -import AnalyzeNotebook from "./AnalyzeNotebook"; -import { Simulation } from "../store/simulation"; +import { describe, expect, it } from "vitest"; +import { projectAnalysisNotebook } from "./AnalyzeNotebook"; -// Helper function to create mock simulation with defaults -const createMockSimulation = ( - overrides: Partial = {}, -): Simulation => ({ - id: "test-sim", - inputScript: "", - files: [], - start: false, - ...overrides, -}); - -describe("AnalyzeNotebook", () => { - it("should generate notebook with simulation ID replaced", () => { - // Arrange - const simulation = createMockSimulation({ - id: "test-simulation-123", - inputScript: "run 1000", - }); - - // Act - const notebook = AnalyzeNotebook(simulation); +const cellSource = (source: string | string[]): string => + Array.isArray(source) ? source.join("") : source; - // Assert - expect(notebook.cells[1].source).toContain("test-simulation-123"); - expect(notebook.cells[1].source).not.toContain("###SIMULATIONID###"); - }); - - it("should have correct notebook structure", () => { - // Arrange - const simulation = createMockSimulation({ id: "sim-001" }); +describe("projectAnalysisNotebook", () => { + const meta = { displayName: "Diffusion coefficients", dirName: "diffusion" }; - // Act - const notebook = AnalyzeNotebook(simulation); - - // Assert + it("produces a valid pyodide notebook", () => { + const notebook = projectAnalysisNotebook(meta); expect(notebook.nbformat).toBe(4); - expect(notebook.nbformat_minor).toBe(4); - expect(notebook.metadata).toBeDefined(); - expect(notebook.metadata.language_info?.name).toBe("python"); + expect(notebook.metadata.kernelspec).toMatchObject({ name: "python" }); + expect(notebook.cells.length).toBeGreaterThan(3); }); - it("should have three cells by default (pip install, plot code, empty)", () => { - // Arrange - const simulation = createMockSimulation({ id: "sim-002" }); - - // Act - const notebook = AnalyzeNotebook(simulation); - - // Assert - expect(notebook.cells).toHaveLength(3); - expect(notebook.cells[0].cell_type).toBe("code"); - expect(notebook.cells[0].source).toContain("%pip install pandas"); - expect(notebook.cells[1].cell_type).toBe("code"); - expect(notebook.cells[1].source).toContain("lammps_logfile"); - expect(notebook.cells[2].cell_type).toBe("code"); - expect(notebook.cells[2].source).toBe(""); + it("titles the notebook after the project", () => { + const notebook = projectAnalysisNotebook(meta); + const header = notebook.cells[0]; + expect(header.cell_type).toBe("markdown"); + expect(cellSource(header.source)).toContain("Diffusion coefficients"); }); - it("should add markdown cell when analysisDescription is provided", () => { - // Arrange - const simulation = createMockSimulation({ - id: "sim-003", - analysisDescription: "# Analysis\nThis is a test analysis description", + it("includes the project description when present", () => { + const notebook = projectAnalysisNotebook({ + ...meta, + description: "LJ binary diffusion", }); - - // Act - const notebook = AnalyzeNotebook(simulation); - - // Assert - expect(notebook.cells).toHaveLength(4); // markdown + 3 default cells - expect(notebook.cells[0].cell_type).toBe("markdown"); - expect(notebook.cells[0].source).toBe( - "# Analysis\nThis is a test analysis description", + expect(cellSource(notebook.cells[0].source)).toContain( + "LJ binary diffusion", ); }); - it("should place markdown cell before pip install cell", () => { - // Arrange - const simulation = createMockSimulation({ - id: "sim-004", - analysisDescription: "Test description", - }); - - // Act - const notebook = AnalyzeNotebook(simulation); - - // Assert - expect(notebook.cells[0].cell_type).toBe("markdown"); - expect(notebook.cells[1].source).toContain("%pip install"); + it("discovers runs by glob rather than hardcoding paths", () => { + const notebook = projectAnalysisNotebook(meta); + const allCode = notebook.cells + .filter((cell) => cell.cell_type === "code") + .map((cell) => cellSource(cell.source)) + .join("\n"); + expect(allCode).toContain('glob.glob("runs/*'); + // The notebook cwd is the project dir; no dirName-prefixed paths. + expect(allCode).not.toContain("diffusion/"); }); - it("should contain lammps_logfile import and plotting code", () => { - // Arrange - const simulation = createMockSimulation({ id: "sim-005" }); - - // Act - const notebook = AnalyzeNotebook(simulation); - - // Assert - const plotCell = notebook.cells[1].source; - expect(plotCell).toContain("import lammps_logfile"); - expect(plotCell).toContain("import matplotlib.pyplot as plt"); - expect(plotCell).toContain("log.lammps"); - expect(plotCell).toContain("plt.plot"); - }); - - it("should have correct metadata for Python kernel", () => { - // Arrange - const simulation = createMockSimulation({ id: "sim-006" }); - - // Act - const notebook = AnalyzeNotebook(simulation); - - // Assert - expect(notebook.metadata.kernelspec?.name).toBe("python"); - expect(notebook.metadata.kernelspec?.display_name).toBe("Python (Pyodide)"); - // language is an extra field not in the official types but allowed via PartialJSONObject - expect( - (notebook.metadata.kernelspec as { language?: string })?.language, - ).toBe("python"); - expect(notebook.metadata.language_info?.name).toBe("python"); - // version is an extra field not in the official types but allowed via PartialJSONObject - expect( - (notebook.metadata.language_info as { version?: string })?.version, - ).toBe("3.12"); - }); - - it("should handle special characters in simulation ID", () => { - // Arrange - const simulation = createMockSimulation({ - id: "sim-with-special-chars_123", - }); - - // Act - const notebook = AnalyzeNotebook(simulation); - - // Assert - expect(notebook.cells[1].source).toContain("sim-with-special-chars_123"); - }); - - it("should not modify other cells when adding markdown", () => { - // Arrange - const simulation = createMockSimulation({ - id: "sim-007", - analysisDescription: "Description", - }); - - // Act - const notebook = AnalyzeNotebook(simulation); - - // Assert - // The pip install cell should be at index 1 (after markdown) - expect(notebook.cells[1].source).toContain("%pip install pandas"); - // The plot cell should be at index 2 - expect(notebook.cells[2].source).toContain("lammps_logfile"); - // Empty cell should be at index 3 - expect(notebook.cells[3].source).toBe(""); + it("installs analysis deps via piplite wheels", () => { + const notebook = projectAnalysisNotebook(meta); + const pipCell = notebook.cells.find((cell) => + cellSource(cell.source).includes("%pip install"), + ); + expect(pipCell).toBeDefined(); + expect(cellSource(pipCell!.source)).toContain("lammps-logfile"); + }); + + it("reads sweep variables from run metadata", () => { + const notebook = projectAnalysisNotebook(meta); + const allCode = notebook.cells + .map((cell) => cellSource(cell.source)) + .join("\n"); + expect(allCode).toContain(".atomify"); + expect(allCode).toContain("run.json"); + expect(allCode).toContain('"vars"'); }); }); diff --git a/src/utils/AnalyzeNotebook.ts b/src/utils/AnalyzeNotebook.ts index fbeecb62..7feb44a5 100644 --- a/src/utils/AnalyzeNotebook.ts +++ b/src/utils/AnalyzeNotebook.ts @@ -1,47 +1,134 @@ -import { Simulation } from "../store/simulation"; +import type { ProjectMeta } from "../storage/types"; import type { INotebookContent, + ICell, ICodeCell, IMarkdownCell, } from "@jupyterlab/nbformat"; -const AnalyzeNotebook = (simulation: Simulation): INotebookContent => { - const cells: ICodeCell[] = [ - { - cell_type: "code", - source: "%pip install pandas", - metadata: { - trusted: true, - }, - execution_count: null, - outputs: [], - }, - { - cell_type: "code", - source: - 'import lammps_logfile\nimport matplotlib.pyplot as plt\n\nlog = lammps_logfile.File("###SIMULATIONID###/log.lammps")\nstep = log.get("Step")\n\nfor keyword in log.keywords:\n plt.figure()\n plt.plot(step, log.get(keyword), label=keyword)\n plt.xlabel(\'Timestep\')\n plt.title(keyword)\n plt.show()', - metadata: { - trusted: true, - }, - execution_count: null, - outputs: [], - }, - { - cell_type: "code", - source: "", - metadata: {}, - execution_count: null, - outputs: [], - }, +/** + * The starter `analysis.ipynb` generated once per project (ADR-002 §2). The + * notebook's cwd is the project directory, so runs are discovered by glob at + * execution time — the file never needs regeneration and is owned by the + * user after creation. + */ + +const code = (source: string): ICodeCell => ({ + cell_type: "code", + source, + metadata: { trusted: true }, + execution_count: null, + outputs: [], +}); + +const markdown = (source: string): IMarkdownCell => ({ + cell_type: "markdown", + source, + metadata: {}, +}); + +export function projectAnalysisNotebook( + meta: Pick & { + description?: string; + }, +): INotebookContent { + const cells: ICell[] = [ + markdown( + `# ${meta.displayName} — analysis\n\n` + + (meta.description ? `${meta.description}\n\n` : "") + + `This notebook shares the project's files: every run of *${meta.displayName}* ` + + "appears under `runs/` with its input snapshot, `log.lammps`, and outputs. " + + "Re-run the cells below after new runs — they discover runs by glob.", + ), + code("%pip install -q lammps-logfile pandas"), + code( + [ + "import glob, json, os", + "import lammps_logfile", + "import matplotlib.pyplot as plt", + "", + "# Every run, oldest first. Each entry: (run directory, metadata dict or None).", + "runs = []", + 'for rundir in sorted(glob.glob("runs/*")):', + ' meta_path = os.path.join(rundir, ".atomify", "run.json")', + " meta = json.load(open(meta_path)) if os.path.exists(meta_path) else None", + " runs.append((rundir, meta))", + "", + 'print(f"{len(runs)} runs")', + "for rundir, meta in runs:", + ' vars_ = (meta or {}).get("vars") or {}', + ' status = (meta or {}).get("status", "?")', + ' print(f" {rundir} [{status}] {vars_}")', + ].join("\n"), + ), + code( + [ + "# Thermo output of the latest run.", + 'logs = sorted(glob.glob("runs/*/log.lammps"))', + "if logs:", + " log = lammps_logfile.File(logs[-1])", + ' step = log.get("Step")', + " for keyword in log.keywords:", + ' if keyword == "Step":', + " continue", + " plt.figure()", + " plt.plot(step, log.get(keyword))", + ' plt.xlabel("Timestep")', + " plt.title(f\"{keyword} — {logs[-1].split('/')[1]}\")", + " plt.show()", + "else:", + ' print("No runs yet — press Run simulation in Atomify first.")', + ].join("\n"), + ), + code( + [ + "# Compare a thermo quantity across every run (great for sweeps: the", + "# swept variable values live in each run's metadata).", + 'quantity = "Temp" # any column from log.keywords', + "plt.figure()", + 'for path in sorted(glob.glob("runs/*/log.lammps")):', + " log = lammps_logfile.File(path)", + " if quantity not in log.keywords:", + " continue", + " rundir = os.path.dirname(path)", + ' meta_path = os.path.join(rundir, ".atomify", "run.json")', + " meta = json.load(open(meta_path)) if os.path.exists(meta_path) else {}", + ' label = ", ".join(f"{k}={v}" for k, v in (meta.get("vars") or {}).items()) or os.path.basename(rundir)', + ' plt.plot(log.get("Step"), log.get(quantity), label=label)', + 'plt.xlabel("Timestep"); plt.ylabel(quantity); plt.legend(); plt.show()', + ].join("\n"), + ), + code( + [ + "# Optional: drive LAMMPS from Python (runs in this browser tab).", + "# Files written below land in this project — the app lists runs/", + "# directories automatically. Note: the in-notebook engine covers the", + "# MOLECULE package set (+KOKKOS); packages beyond that run in the app only.", + "#", + "# %pip install -q lammps-js", + "# from lammps import lammps", + "# import os", + "# home = os.getcwd()", + "# for T in [1.0, 2.0, 3.0]:", + '# rundir = f"runs/run-T{T}"', + "# os.makedirs(rundir, exist_ok=True)", + "# os.chdir(rundir) # outputs land in the run, not the working tree", + "# try:", + "# lmp = await lammps()", + '# lmp.command(f"variable T equal {T}")', + '# lmp.file("../../in.script") # your input script', + "# lmp.close()", + "# finally:", + "# os.chdir(home)", + ].join("\n"), + ), + code(""), ]; - const notebook: INotebookContent = { + return { metadata: { language_info: { - codemirror_mode: { - name: "python", - version: 3, - }, + codemirror_mode: { name: "python", version: 3 }, file_extension: ".py", mimetype: "text/x-python", name: "python", @@ -59,23 +146,6 @@ const AnalyzeNotebook = (simulation: Simulation): INotebookContent => { nbformat: 4, cells, }; +} - const cell = notebook.cells[1]; - if (cell.cell_type === "code") { - const source = Array.isArray(cell.source) - ? cell.source.join("") - : cell.source; - cell.source = source.replace("###SIMULATIONID###", simulation.id); - } - if (simulation.analysisDescription) { - const markdownCell: IMarkdownCell = { - cell_type: "markdown", - source: simulation.analysisDescription, - metadata: {}, - }; - notebook.cells.splice(0, 0, markdownCell); - } - return notebook; -}; - -export default AnalyzeNotebook; +export default projectAnalysisNotebook; diff --git a/src/utils/scriptVariables.ts b/src/utils/scriptVariables.ts new file mode 100644 index 00000000..46192c0d --- /dev/null +++ b/src/utils/scriptVariables.ts @@ -0,0 +1,62 @@ +/** + * Detect overridable LAMMPS variables in an input script (ADR-003 §4.5). + * + * A variable is a run parameter when it is defined with a plain numeric + * literal: `variable T equal 3.0`. Derived expressions + * (`variable Lhalf equal $(0.5*v_L)`) recompute from the overrides and are + * not editable themselves. Overrides work because LAMMPS ignores a + * `variable` command whose name is already defined — the documented pattern + * for command-line/-var-style defaults — so injected definitions win. + */ + +export interface ScriptVariable { + name: string; + /** The literal value in the script. */ + value: number; + /** 1-based line number of the definition. */ + line: number; + /** Human hint from a trailing `# comment` on the line, if any. */ + label?: string; +} + +const VARIABLE_LINE = + /^\s*variable\s+(\w+)\s+equal\s+(-?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?)\s*(?:#\s*(.*))?$/; + +export function parseScriptVariables(script: string): ScriptVariable[] { + const seen = new Set(); + const variables: ScriptVariable[] = []; + script.split("\n").forEach((text, index) => { + const match = VARIABLE_LINE.exec(text); + if (!match) { + return; + } + const [, name, literal, comment] = match; + // Only the first definition is overridable — LAMMPS ignores later ones. + if (seen.has(name)) { + return; + } + seen.add(name); + variables.push({ + name, + value: Number(literal), + line: index + 1, + label: comment?.trim() || undefined, + }); + }); + return variables; +} + +/** Evenly spaced values from `from` to `to` inclusive (design's sweep rows). */ +export function linspace(from: number, to: number, count: number): number[] { + const n = Math.max(1, Math.floor(count)); + if (!Number.isFinite(from) || !Number.isFinite(to)) { + return []; + } + if (n === 1) { + return [from]; + } + return Array.from( + { length: n }, + (_, i) => Number((from + ((to - from) * i) / (n - 1)).toPrecision(12)), + ); +} From 4710498f22c0867132ef6b272ff8748d8af52391 Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 10:55:01 +0200 Subject: [PATCH 09/34] Mark ADRs 001-003 as Accepted Co-Authored-By: Claude Fable 5 --- docs/adr/001-projects-runs-and-storage.md | 2 +- docs/adr/002-notebook-first-class.md | 2 +- docs/adr/003-ui-shell-and-simulations-experience.md | 2 +- docs/adr/README.md | 6 +++--- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/adr/001-projects-runs-and-storage.md b/docs/adr/001-projects-runs-and-storage.md index 0d4788c2..c554089c 100644 --- a/docs/adr/001-projects-runs-and-storage.md +++ b/docs/adr/001-projects-runs-and-storage.md @@ -1,6 +1,6 @@ # ADR-001: Projects, runs, and a single project filesystem -- **Status**: Proposed (rev 2 — after architecture + product review) +- **Status**: Accepted (2026-07-11) - **Date**: 2026-07-11 - **Deciders**: Anders Hafreager, Claude diff --git a/docs/adr/002-notebook-first-class.md b/docs/adr/002-notebook-first-class.md index 7a5e26b6..62864b7b 100644 --- a/docs/adr/002-notebook-first-class.md +++ b/docs/adr/002-notebook-first-class.md @@ -1,6 +1,6 @@ # ADR-002: The notebook as a first-class analysis and scripting surface -- **Status**: Proposed (rev 2 — after integration review) +- **Status**: Accepted (2026-07-11) - **Date**: 2026-07-11 - **Deciders**: Anders Hafreager, Claude - **Depends on**: ADR-001 (projects, runs, single project filesystem) diff --git a/docs/adr/003-ui-shell-and-simulations-experience.md b/docs/adr/003-ui-shell-and-simulations-experience.md index d12195ad..9b520d9f 100644 --- a/docs/adr/003-ui-shell-and-simulations-experience.md +++ b/docs/adr/003-ui-shell-and-simulations-experience.md @@ -1,6 +1,6 @@ # ADR-003: UI shell redesign and the projects experience -- **Status**: Proposed (rev 2 — after product review + updated design) +- **Status**: Accepted (2026-07-11) - **Date**: 2026-07-11 - **Deciders**: Anders Hafreager, Claude - **Depends on**: ADR-001 (projects & runs), ADR-002 (notebook) diff --git a/docs/adr/README.md b/docs/adr/README.md index e1161e79..71c2ea8f 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -5,6 +5,6 @@ context, the decision, alternatives considered, and consequences. | ADR | Title | Status | | --- | --- | --- | -| [ADR-001](001-projects-runs-and-storage.md) | Projects, runs, and a single project filesystem | Proposed | -| [ADR-002](002-notebook-first-class.md) | The notebook as a first-class analysis and scripting surface | Proposed | -| [ADR-003](003-ui-shell-and-simulations-experience.md) | UI shell redesign and the projects experience | Proposed | +| [ADR-001](001-projects-runs-and-storage.md) | Projects, runs, and a single project filesystem | Accepted | +| [ADR-002](002-notebook-first-class.md) | The notebook as a first-class analysis and scripting surface | Accepted | +| [ADR-003](003-ui-shell-and-simulations-experience.md) | UI shell redesign and the projects experience | Accepted | From b04312a509b4e126106f193a86e735db8be112e1 Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 11:27:19 +0200 Subject: [PATCH 10/34] Add integration tests for the projects store model Real easy-peasy store (full model composition) over real in-memory project storage; only the engine boundary is faked (LammpsWeb + minimal wasm FS). Covers project creation (notebook template, inputScript inference), the full run loop (snapshot, run.json, output copies, activeRun lifecycle), sweeps and mid-sweep cancellation, failed runs, debounced saves + flush-before-snapshot, quick runs and saveQuickAsProject, deleteProject guarding, zombie-run reconciliation, duplicateProject, and removeFile clearing the designated input script. Co-Authored-By: Claude Fable 5 --- src/store/projects.integration.test.ts | 829 +++++++++++++++++++++++++ 1 file changed, 829 insertions(+) create mode 100644 src/store/projects.integration.test.ts diff --git a/src/store/projects.integration.test.ts b/src/store/projects.integration.test.ts new file mode 100644 index 00000000..3736f047 --- /dev/null +++ b/src/store/projects.integration.test.ts @@ -0,0 +1,829 @@ +/** + * Integration tests for the projects store model (ADR-001/003). + * + * These tests build the REAL store (full model composition) on REAL + * in-memory project storage. Only the engine boundary is faked: a + * LammpsWeb whose runFile resolves immediately (or on demand) and a + * minimal wasm module for the FS writes the simulation model performs. + * Every scenario asserts against the actual storage contents — the + * persisted project.json / run.json / snapshots — not just store state. + */ + +import { describe, it, expect, vi, afterEach } from "vitest"; +import { createStore, type Store } from "easy-peasy"; + +// Mock metrics — the store models call track/time_event on most transitions; +// mocking keeps tests quiet and avoids mixpanel/localStorage side effects. +vi.mock(import("../utils/metrics"), () => ({ + track: vi.fn(), + time_event: vi.fn(), + getEmbeddingParams: vi.fn(() => ({ + embedMode: false as const, + embedFullscreen: false, + embedAutoStart: false, + })), +})); + +import { storeModel, type StoreModel } from "./model"; +import type { RunRequest, StoreInjections } from "./projects"; +import { + createMemoryProjectStorage, + PROJECT_META_PATH, + readRunMeta, + writeRunMeta, + RUNS_DIR, + RUN_META_PATH, + type ProjectMeta, + type ProjectStorage, + type RunMeta, +} from "../storage"; +import { setWasm } from "../wasm/wasmInstance"; +import type { AtomifyWasmModule } from "../wasm/types"; +import type { LammpsWeb, LMPModifier, Wall } from "../types"; + +const SCRIPT = "units lj\nrun 100\n"; +const FAKE_LOG = "fake LAMMPS log output"; + +afterEach(() => { + vi.useRealTimers(); +}); + +describe("projects store integration", () => { + describe("createProject", () => { + it("creates a blank project: project.json persisted, analysis notebook generated, listed in library", async () => { + const { store, libraryStorage } = await createTestStore(createFakeEngine()); + + const meta = await store + .getActions() + .projects.createProject({ displayName: "Blank", files: [] }); + + // project.json exists in real storage with the created identity + const persisted = await readJson( + libraryStorage, + meta.dirName, + PROJECT_META_PATH, + ); + expect(persisted.dirName).toBe(meta.dirName); + expect(persisted.displayName).toBe("Blank"); + expect(persisted.inputScript).toBeUndefined(); + + // starter notebook generated (template mentions the project) + const notebook = await libraryStorage.read(meta.dirName, "analysis.ipynb"); + expect(typeof notebook).toBe("string"); + expect(notebook as string).toContain("Blank"); + + // listed in the library (storage) and in store state + const listed = await libraryStorage.listProjects(); + expect(listed.map((p) => p.dirName)).toContain(meta.dirName); + expect( + store.getState().projects.projects.map((p) => p.dirName), + ).toContain(meta.dirName); + expect(store.getState().projects.active?.meta.dirName).toBe(meta.dirName); + }); + + it("does not overwrite a curated notebook with the template", async () => { + const { store, libraryStorage } = await createTestStore(createFakeEngine()); + const curated = JSON.stringify({ + cells: [], + metadata: { curated: true }, + nbformat: 4, + nbformat_minor: 5, + }); + + const meta = await store.getActions().projects.createProject({ + displayName: "Curated", + files: [{ fileName: "analysis.ipynb", content: curated }], + }); + + const stored = await libraryStorage.read(meta.dirName, "analysis.ipynb"); + expect(JSON.parse(stored as string)).toEqual(JSON.parse(curated)); + }); + + it("auto-sets inputScript when exactly one script file is provided", async () => { + const { store, libraryStorage } = await createTestStore(createFakeEngine()); + + const meta = await store.getActions().projects.createProject({ + displayName: "One Script", + files: [ + { fileName: "in.lmp", content: SCRIPT }, + { fileName: "data.spce", content: "some data" }, + ], + }); + + expect(meta.inputScript).toBe("in.lmp"); + const persisted = await readJson( + libraryStorage, + meta.dirName, + PROJECT_META_PATH, + ); + expect(persisted.inputScript).toBe("in.lmp"); + }); + + it("leaves inputScript undefined when two script files are provided", async () => { + const { store, libraryStorage } = await createTestStore(createFakeEngine()); + + const meta = await store.getActions().projects.createProject({ + displayName: "Two Scripts", + files: [ + { fileName: "in.melt", content: SCRIPT }, + { fileName: "equilibrate.in", content: SCRIPT }, + ], + }); + + expect(meta.inputScript).toBeUndefined(); + const persisted = await readJson( + libraryStorage, + meta.dirName, + PROJECT_META_PATH, + ); + expect(persisted.inputScript).toBeUndefined(); + }); + }); + + describe("full run loop", () => { + it("records snapshot, run.json, outputs, and clears activeRun after a completed run", async () => { + const engine = createFakeEngine(); + const { store, libraryStorage } = await createTestStore(engine); + await store.getActions().projects.createProject({ + displayName: "Melt", + files: [ + { fileName: "in.lmp", content: SCRIPT }, + { fileName: "data.spce", content: "some data" }, + ], + }); + const dirName = activeDirName(store); + + await store + .getActions() + .projects.startRuns([runRequest({ vars: { T: 2 } })]); + + // Source snapshot: the script (and data file) yes, notebook and + // project-level .atomify NO. + expect( + await libraryStorage.read(dirName, `${RUNS_DIR}/run-001/in.lmp`), + ).toBe(SCRIPT); + expect( + await libraryStorage.read(dirName, `${RUNS_DIR}/run-001/data.spce`), + ).toBe("some data"); + expect( + await libraryStorage.stat(dirName, `${RUNS_DIR}/run-001/analysis.ipynb`), + ).toBeNull(); + expect( + await libraryStorage.stat( + dirName, + `${RUNS_DIR}/run-001/.atomify/project.json`, + ), + ).toBeNull(); + + // run.json: completed with vars, stats, and both timestamps. + const runMeta = await readRunMeta(libraryStorage, dirName, "run-001"); + expect(runMeta).not.toBeNull(); + expect(runMeta?.status).toBe("completed"); + expect(runMeta?.vars).toEqual({ T: 2 }); + expect(runMeta?.origin).toBe("app"); + expect(runMeta?.sweepId).toBeUndefined(); + expect(runMeta?.stats?.timesteps).toBe(500); + expect(runMeta?.stats?.numAtoms).toBe(100); + expect(typeof runMeta?.stats?.wallSeconds).toBe("number"); + expect(runMeta?.startedAt).toBeTruthy(); + expect(runMeta?.finishedAt).toBeTruthy(); + + // Outputs copied from the fake engine's workdir snapshot. + expect( + await libraryStorage.read(dirName, `${RUNS_DIR}/run-001/log.lammps`), + ).toBe(FAKE_LOG); + + // The materialized Simulation id is the run directory: the engine got + // a script path inside /runs/run-001. + expect(engine.runFilePaths).toHaveLength(1); + expect(engine.runFilePaths[0].startsWith(`/${dirName}/${RUNS_DIR}/run-001/`)).toBe( + true, + ); + + // Store state settled: activeRun cleared, runs list refreshed. + expect(store.getState().projects.activeRun).toBeUndefined(); + const runs = store.getState().projects.active?.runs ?? []; + expect(runs.map((r) => r.runId)).toEqual(["run-001"]); + expect(runs[0].meta?.status).toBe("completed"); + }); + + it("records a failed run with the engine's error message", async () => { + const engine = createFakeEngine(); + const { store, libraryStorage } = await createTestStore(engine); + await store.getActions().projects.createProject({ + displayName: "Broken", + files: [{ fileName: "in.lmp", content: SCRIPT }], + }); + const dirName = activeDirName(store); + engine.setErrorMessage("ERROR: Unknown command: frobnicate"); + + await store.getActions().projects.startRuns([runRequest()]); + + const runMeta = await readRunMeta(libraryStorage, dirName, "run-001"); + expect(runMeta?.status).toBe("failed"); + expect(runMeta?.error).toBe("ERROR: Unknown command: frobnicate"); + expect(runMeta?.finishedAt).toBeTruthy(); + expect(store.getState().projects.activeRun).toBeUndefined(); + }); + }); + + describe("sweeps", () => { + it("executes a 3-run sweep sequentially with shared sweepId and per-run vars", async () => { + const engine = createFakeEngine(); + const { store, libraryStorage } = await createTestStore(engine); + await store.getActions().projects.createProject({ + displayName: "Sweep", + files: [{ fileName: "in.lmp", content: SCRIPT }], + }); + const dirName = activeDirName(store); + + await store + .getActions() + .projects.startRuns([ + runRequest({ vars: { T: 1 } }), + runRequest({ vars: { T: 2 } }), + runRequest({ vars: { T: 3 } }), + ]); + + // Sequential execution: the engine saw run-001, run-002, run-003 in order. + expect(engine.runFilePaths).toHaveLength(3); + expect(engine.runFilePaths[0]).toContain(`/${RUNS_DIR}/run-001/`); + expect(engine.runFilePaths[1]).toContain(`/${RUNS_DIR}/run-002/`); + expect(engine.runFilePaths[2]).toContain(`/${RUNS_DIR}/run-003/`); + + const metas: RunMeta[] = []; + for (const runId of ["run-001", "run-002", "run-003"]) { + const meta = await readRunMeta(libraryStorage, dirName, runId); + expect(meta).not.toBeNull(); + metas.push(meta as RunMeta); + } + expect(metas.map((m) => m.status)).toEqual([ + "completed", + "completed", + "completed", + ]); + expect(metas.map((m) => m.vars)).toEqual([{ T: 1 }, { T: 2 }, { T: 3 }]); + expect(metas.map((m) => m.origin)).toEqual(["sweep", "sweep", "sweep"]); + expect(metas[0].sweepId).toMatch(/^sweep-/); + expect(new Set(metas.map((m) => m.sweepId)).size).toBe(1); + expect(store.getState().projects.runQueue).toEqual([]); + }); + + it("cancelQueuedRuns mid-sweep: queued runs are never created", async () => { + const engine = createFakeEngine({ manual: true }); + const { store, libraryStorage } = await createTestStore(engine); + await store.getActions().projects.createProject({ + displayName: "Canceled Sweep", + files: [{ fileName: "in.lmp", content: SCRIPT }], + }); + const dirName = activeDirName(store); + + const sweepPromise = store + .getActions() + .projects.startRuns([ + runRequest({ vars: { T: 1 } }), + runRequest({ vars: { T: 2 } }), + runRequest({ vars: { T: 3 } }), + ]); + // First run is now in flight (engine holds it); cancel the queue + // before it resolves, then let the first run finish. + await waitForCondition( + () => engine.runFilePaths.length === 1, + "first runFile call", + ); + store.getActions().projects.cancelQueuedRuns(); + engine.resolveRun(); + await sweepPromise; + + // Only run-001 exists — run-002/003 were never allocated. + const runDirs = await libraryStorage.list(dirName, RUNS_DIR); + expect(runDirs.map((e) => e.path)).toEqual([`${RUNS_DIR}/run-001`]); + expect(engine.runFilePaths).toHaveLength(1); + const meta = await readRunMeta(libraryStorage, dirName, "run-001"); + expect(meta?.status).toBe("completed"); + expect(store.getState().projects.activeRun).toBeUndefined(); + expect(store.getState().projects.runQueue).toEqual([]); + }); + }); + + describe("debounced saves", () => { + it("saveFileDebounced does not persist immediately; flushPendingSaves persists", async () => { + const { store, libraryStorage } = await createTestStore(createFakeEngine()); + await store.getActions().projects.createProject({ + displayName: "Editing", + files: [{ fileName: "in.lmp", content: SCRIPT }], + }); + const dirName = activeDirName(store); + const edited = "units metal\nrun 42\n"; + + store + .getActions() + .projects.saveFileDebounced({ path: "in.lmp", content: edited }); + + // Debounce pending: storage still holds the original content. + expect(await libraryStorage.read(dirName, "in.lmp")).toBe(SCRIPT); + expect(store.getState().projects.saveStates["in.lmp"]).toBe("saving"); + + await store.getActions().projects.flushPendingSaves(); + + expect(await libraryStorage.read(dirName, "in.lmp")).toBe(edited); + expect(store.getState().projects.saveStates["in.lmp"]).toBe("saved"); + }); + + it("the debounced save lands on its own after 500ms", async () => { + const { store, libraryStorage } = await createTestStore(createFakeEngine()); + await store.getActions().projects.createProject({ + displayName: "Debounce Timer", + files: [{ fileName: "in.lmp", content: SCRIPT }], + }); + const dirName = activeDirName(store); + const edited = "# edited via timer\n"; + + vi.useFakeTimers(); + store + .getActions() + .projects.saveFileDebounced({ path: "in.lmp", content: edited }); + expect(await libraryStorage.read(dirName, "in.lmp")).toBe(SCRIPT); + + await vi.advanceTimersByTimeAsync(500); + vi.useRealTimers(); + + expect(await libraryStorage.read(dirName, "in.lmp")).toBe(edited); + expect(store.getState().projects.saveStates["in.lmp"]).toBe("saved"); + }); + + it("startRuns flushes pending saves so the snapshot has the edited content", async () => { + const engine = createFakeEngine(); + const { store, libraryStorage } = await createTestStore(engine); + await store.getActions().projects.createProject({ + displayName: "Edit Then Run", + files: [{ fileName: "in.lmp", content: SCRIPT }], + }); + const dirName = activeDirName(store); + const edited = "units real\nrun 7\n"; + + store + .getActions() + .projects.saveFileDebounced({ path: "in.lmp", content: edited }); + await store.getActions().projects.startRuns([runRequest()]); + + expect( + await libraryStorage.read(dirName, `${RUNS_DIR}/run-001/in.lmp`), + ).toBe(edited); + expect(await libraryStorage.read(dirName, "in.lmp")).toBe(edited); + }); + }); + + describe("quick runs", () => { + it("quick projects live on scratch storage; saveQuickAsProject copies tree and runs into the library with a fresh identity", async () => { + const engine = createFakeEngine(); + const { store, libraryStorage, scratchStorage } = + await createTestStore(engine); + + const quickMeta = await store.getActions().projects.createProject({ + displayName: "Quick run — Melt", + files: [{ fileName: "in.lmp", content: SCRIPT }], + quick: true, + }); + + // Invisible to the library while quick. + expect(await libraryStorage.listProjects()).toEqual([]); + expect( + (await scratchStorage.listProjects()).map((p) => p.dirName), + ).toEqual([quickMeta.dirName]); + + await store.getActions().projects.startRuns([runRequest()]); + expect(store.getState().projects.activeRun).toBeUndefined(); + // The run landed on scratch storage, not the library. + expect( + await scratchStorage.read( + quickMeta.dirName, + `${RUNS_DIR}/run-001/log.lammps`, + ), + ).toBe(FAKE_LOG); + expect(await libraryStorage.listProjects()).toEqual([]); + + await store.getActions().projects.saveQuickAsProject(); + + const libraryProjects = await libraryStorage.listProjects(); + expect(libraryProjects).toHaveLength(1); + const saved = libraryProjects[0]; + expect(saved.displayName).toBe("Melt"); + expect(saved.dirName).not.toBe(quickMeta.dirName); + expect(saved.inputScript).toBe("in.lmp"); + + // Working tree AND completed runs materialized. + expect(await libraryStorage.read(saved.dirName, "in.lmp")).toBe(SCRIPT); + expect( + await libraryStorage.stat(saved.dirName, "analysis.ipynb"), + ).not.toBeNull(); + expect( + await libraryStorage.read( + saved.dirName, + `${RUNS_DIR}/run-001/log.lammps`, + ), + ).toBe(FAKE_LOG); + const runMeta = await readRunMeta(libraryStorage, saved.dirName, "run-001"); + expect(runMeta?.status).toBe("completed"); + + // Fresh identity: the library project.json carries the new dirName, + // not a copy of the scratch project's. + const persisted = await readJson( + libraryStorage, + saved.dirName, + PROJECT_META_PATH, + ); + expect(persisted.dirName).toBe(saved.dirName); + expect(persisted.displayName).toBe("Melt"); + expect(store.getState().projects.active?.quick).toBe(false); + expect(store.getState().projects.active?.meta.dirName).toBe( + saved.dirName, + ); + }); + }); + + describe("deleteProject", () => { + it("is blocked while the project has an active run and allowed afterwards", async () => { + const engine = createFakeEngine({ manual: true }); + const { store, libraryStorage } = await createTestStore(engine); + await store.getActions().projects.createProject({ + displayName: "Busy", + files: [{ fileName: "in.lmp", content: SCRIPT }], + }); + const dirName = activeDirName(store); + + const runPromise = store.getActions().projects.startRuns([runRequest()]); + await waitForCondition( + () => store.getState().projects.activeRun !== undefined, + "activeRun to be set", + ); + + await store.getActions().projects.deleteProject(dirName); + + // Blocked: still in storage, notice explains why. + expect( + (await libraryStorage.listProjects()).map((p) => p.dirName), + ).toContain(dirName); + expect(store.getState().projects.notice).toContain("run in progress"); + + engine.resolveRun(); + await runPromise; + + await store.getActions().projects.deleteProject(dirName); + + expect(await libraryStorage.listProjects()).toEqual([]); + expect(await libraryStorage.stat(dirName, PROJECT_META_PATH)).toBeNull(); + expect(store.getState().projects.active).toBeUndefined(); + expect(store.getState().projects.screen).toEqual({ name: "home" }); + }); + }); + + describe("run reconciliation on openProject", () => { + it("marks unowned app-origin running runs interrupted; fresh notebook runs survive", async () => { + const { store, libraryStorage } = await createTestStore(createFakeEngine()); + await store.getActions().projects.createProject({ + displayName: "Zombies", + files: [{ fileName: "in.lmp", content: SCRIPT }], + }); + const dirName = activeDirName(store); + + // A zombie from a previous app session… + await writeRunMeta(libraryStorage, dirName, { + schemaVersion: 1, + id: "run-001", + inputScript: "in.lmp", + status: "running", + startedAt: new Date(Date.now() - 60 * 60 * 1000).toISOString(), + origin: "app", + }); + // …and a notebook-origin run whose run.json was just written (fresh + // mtime — the kernel may genuinely still be executing it). + await writeRunMeta(libraryStorage, dirName, { + schemaVersion: 1, + id: "run-002", + inputScript: "in.lmp", + status: "running", + startedAt: new Date().toISOString(), + origin: "notebook", + }); + + await store.getActions().projects.openProject({ dirName }); + + const appRun = await readRunMeta(libraryStorage, dirName, "run-001"); + expect(appRun?.status).toBe("interrupted"); + expect(appRun?.finishedAt).toBeTruthy(); + + const notebookRun = await readRunMeta(libraryStorage, dirName, "run-002"); + expect(notebookRun?.status).toBe("running"); + expect(store.getState().projects.notice).toContain("interrupted"); + }); + }); + + describe("duplicateProject", () => { + it("copies the working tree and notebook, keeps inputScript, and starts with zero runs", async () => { + const engine = createFakeEngine(); + const { store, libraryStorage } = await createTestStore(engine); + await store.getActions().projects.createProject({ + displayName: "Original", + files: [ + { fileName: "in.lmp", content: SCRIPT }, + { fileName: "data.spce", content: "some data" }, + ], + }); + const originalDirName = activeDirName(store); + // Give the original a run so "zero runs in the copy" is meaningful. + await store.getActions().projects.startRuns([runRequest()]); + + await store.getActions().projects.duplicateProject(); + + const copyDirName = activeDirName(store); + expect(copyDirName).not.toBe(originalDirName); + const copyMeta = await readJson( + libraryStorage, + copyDirName, + PROJECT_META_PATH, + ); + expect(copyMeta.displayName).toBe("Original (copy)"); + expect(copyMeta.inputScript).toBe("in.lmp"); + + // Working tree + notebook copied (the original's notebook, not a + // regenerated template for the copy's name). + expect(await libraryStorage.read(copyDirName, "in.lmp")).toBe(SCRIPT); + expect(await libraryStorage.read(copyDirName, "data.spce")).toBe( + "some data", + ); + const notebook = (await libraryStorage.read( + copyDirName, + "analysis.ipynb", + )) as string; + expect(notebook).toContain("Original"); + expect(notebook).not.toContain("Original (copy)"); + + // Zero runs in the copy; the original keeps its run. + expect(await libraryStorage.list(copyDirName, RUNS_DIR)).toEqual([]); + expect(store.getState().projects.active?.runs).toEqual([]); + expect( + await libraryStorage.stat( + originalDirName, + `${RUNS_DIR}/run-001/${RUN_META_PATH}`, + ), + ).not.toBeNull(); + }); + }); + + describe("removeFile", () => { + it("clears meta.inputScript when the designated input script is removed", async () => { + const { store, libraryStorage } = await createTestStore(createFakeEngine()); + const meta = await store.getActions().projects.createProject({ + displayName: "Remove Input", + files: [{ fileName: "in.lmp", content: SCRIPT }], + }); + expect(meta.inputScript).toBe("in.lmp"); + + await store.getActions().projects.removeFile("in.lmp"); + + expect(await libraryStorage.stat(meta.dirName, "in.lmp")).toBeNull(); + const persisted = await readJson( + libraryStorage, + meta.dirName, + PROJECT_META_PATH, + ); + expect(persisted.inputScript).toBeUndefined(); + expect( + store.getState().projects.active?.meta.inputScript, + ).toBeUndefined(); + }); + }); +}); + +// --- Helpers --------------------------------------------------------------- + +interface FakeEngine { + lammps: LammpsWeb; + /** Every path passed to runFile, in call order. */ + runFilePaths: string[]; + /** Resolve the oldest pending manual runFile (manual mode only). */ + resolveRun: () => void; + /** Value returned by getErrorMessage after runFile settles. */ + setErrorMessage: (message: string) => void; +} + +interface FakeEngineOptions { + /** When true, runFile returns a promise resolved via resolveRun(). */ + manual?: boolean; + /** Files snapshotWorkdir reports (defaults to a log.lammps). */ + outputs?: { path: string; content: string }[]; +} + +function createFakeEngine(options: FakeEngineOptions = {}): FakeEngine { + const outputs = options.outputs ?? [ + { path: "log.lammps", content: FAKE_LOG }, + ]; + const runFilePaths: string[] = []; + const pendingRuns: (() => void)[] = []; + let errorMessage = ""; + + const emptyStringArray = () => ({ + get: () => "", + size: () => 0, + delete: () => {}, + }); + const noModifier = (): LMPModifier => { + throw new Error("LMP modifiers are not exercised by these tests"); + }; + const emptyWalls = () => ({ + get: (): Wall => ({ which: 0, style: 0, position: 0, cutoff: 0 }), + size: () => 0, + delete: () => {}, + }); + + const lammps: LammpsWeb = { + getNumAtoms: () => 100, + setSyncFrequency: () => {}, + setBuildNeighborlist: () => {}, + setBondDistance: () => {}, + clearBondDistances: () => {}, + getIsRunning: () => false, + getErrorMessage: () => errorMessage, + getLastCommand: () => "", + getTimesteps: () => 500, + getRunTimesteps: () => 0, + getRunTotalTimesteps: () => 0, + getTimestepsPerSecond: () => 0, + getCPURemain: () => 0, + getWhichFlag: () => 0, + getCompute: noModifier, + getComputeNames: emptyStringArray, + getFix: noModifier, + getFixNames: emptyStringArray, + getVariable: noModifier, + getVariableNames: emptyStringArray, + syncComputes: () => {}, + syncFixes: () => {}, + syncVariables: () => {}, + snapshotWorkdir: async () => ({ + files: outputs.map((file) => ({ + path: file.path, + bytes: new TextEncoder().encode(file.content), + })), + skipped: [], + }), + getMemoryUsage: () => 0, + getPositionsPointer: () => 0, + getIdPointer: () => 0, + getTypePointer: () => 0, + getCellMatrixPointer: () => 0, + getOrigoPointer: () => 0, + getBondsPosition1Pointer: () => 0, + getBondsPosition2Pointer: () => 0, + getExceptionMessage: (address: number) => `exception at ${address}`, + step: () => {}, + stop: () => true, + start: () => true, + cancel: () => {}, + runCommand: () => {}, + runFile: (path: string) => { + runFilePaths.push(path); + if (!options.manual) { + return Promise.resolve(); + } + return new Promise((resolve) => pendingRuns.push(resolve)); + }, + computeBonds: () => 0, + computeParticles: () => 0, + getDimension: () => 3, + getWalls: emptyWalls, + }; + + return { + lammps, + runFilePaths, + resolveRun: () => { + const resolve = pendingRuns.shift(); + if (!resolve) { + throw new Error("No pending runFile to resolve"); + } + resolve(); + }, + setErrorMessage: (message: string) => { + errorMessage = message; + }, + }; +} + +/** A minimal wasm module: only the FS + heaps the simulation model touches. */ +function createFakeWasm(): AtomifyWasmModule { + const files = new Map(); + const dirs = new Set(["/"]); + const fs: AtomifyWasmModule["FS"] = { + mkdir: (path) => { + dirs.add(path); + }, + rmdir: (path) => { + dirs.delete(path); + }, + chdir: () => {}, + cwd: () => "/", + writeFile: (path, data) => { + files.set(path, data); + }, + unlink: (path) => { + files.delete(path); + }, + readFile: (path) => { + const data = files.get(path); + if (data === undefined) { + throw new Error(`No such fake file: ${path}`); + } + return typeof data === "string" ? data : new TextDecoder().decode(data); + }, + readdir: () => [], + analyzePath: (path) => ({ + exists: dirs.has(path) || files.has(path), + isRoot: path === "/", + path, + name: path.slice(path.lastIndexOf("/") + 1), + error: 0, + }), + stat: () => ({ size: 0, mode: 0, mtime: new Date() }), + isDir: () => false, + isFile: () => true, + }; + // LAMMPSWeb/ScalarType (the embind class handles) are never touched by the + // store models; Partial + cast per docs/testing_patterns.md §9. + const wasm: Partial = { + HEAPF32: new Float32Array(1024), + HEAPF64: new Float64Array(64), + HEAP32: new Int32Array(1024), + HEAP64: new BigInt64Array(16), + FS: fs, + }; + return wasm as AtomifyWasmModule; +} + +async function createTestStore(engine: FakeEngine): Promise<{ + store: Store; + libraryStorage: ProjectStorage; + scratchStorage: ProjectStorage; +}> { + const libraryStorage = createMemoryProjectStorage(); + const scratchStorage = createMemoryProjectStorage(); + const injections: StoreInjections = { libraryStorage, scratchStorage }; + const store = createStore(storeModel, { injections }); + // The settings branch is persist()-wrapped; rehydration is async and + // REPLACES store state when it lands. Await it so it cannot clobber the + // setup dispatches below (it does, once earlier tests have populated + // sessionStorage). + await store.persist.resolveRehydration(); + setWasm(createFakeWasm()); + // The omovi/three post-timestep render pipeline is not under test here; + // clearing the modifiers (via the real action) keeps completed-run + // processing to plain engine getters. + store.getActions().processing.setPostTimestepModifiers([]); + store.getActions().simulation.setLammps(engine.lammps); + return { store, libraryStorage, scratchStorage }; +} + +function runRequest( + overrides: Partial> = {}, +): Omit { + return { + inputScript: "in.lmp", + vars: {}, + useKokkos: false, + threads: 1, + ...overrides, + }; +} + +function activeDirName(store: Store): string { + const active = store.getState().projects.active; + if (!active) { + throw new Error("Expected an active project"); + } + return active.meta.dirName; +} + +async function readJson( + storage: ProjectStorage, + dirName: string, + path: string, +): Promise { + const raw = await storage.read(dirName, path); + return JSON.parse( + typeof raw === "string" ? raw : new TextDecoder().decode(raw), + ) as T; +} + +async function waitForCondition( + condition: () => boolean, + what: string, +): Promise { + const deadline = Date.now() + 5000; + while (!condition()) { + if (Date.now() > deadline) { + throw new Error(`Timed out waiting for ${what}`); + } + await new Promise((resolve) => setTimeout(resolve, 5)); + } +} From 4d4b1509ea47da4e6926ce6ea3096483c4a96ae2 Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 11:31:32 +0200 Subject: [PATCH 11/34] Add theme setting, antd shell bridge, and projects read/size thunks Foundation for the ADR-003 shell: a persisted dark/light theme in the settings model, an antd ConfigProvider config mirroring the design tokens, and three surgical projects thunks (readFileRaw, listFiles, projectSizes) the run-detail/home/storage screens need. Also brings the projects store integration tests into the tree. Co-Authored-By: Claude Fable 5 --- src/store/projects.ts | 93 +++++++++++++++++++++++++++++++++++++++++++ src/store/settings.ts | 32 +++++++++++++++ src/theme.ts | 44 ++++++++++++++++++++ 3 files changed, 169 insertions(+) diff --git a/src/store/projects.ts b/src/store/projects.ts index 20dd8c59..29141c95 100644 --- a/src/store/projects.ts +++ b/src/store/projects.ts @@ -196,6 +196,35 @@ export interface ProjectsModel { >; setInputScript: Thunk; + /** + * Raw read of any project-relative path (binary stays Uint8Array). + * Defaults to the active project; pass dirName for other projects + * (e.g. the Home continue card's frame.png). Returns null when missing. + */ + readFileRaw: Thunk< + ProjectsModel, + { path: string; dirName?: string; quick?: boolean }, + StoreInjections, + StoreModel, + Promise + >; + /** Direct children of a subdir (run output lists, run-frame lookups). */ + listFiles: Thunk< + ProjectsModel, + { subdir: string; dirName?: string; quick?: boolean }, + StoreInjections, + StoreModel, + Promise + >; + /** Per-project storage footprint for the Settings storage tab (ADR-003 §5). */ + projectSizes: Thunk< + ProjectsModel, + void, + StoreInjections, + StoreModel, + Promise> + >; + startRuns: Thunk< ProjectsModel, Omit[] , @@ -635,6 +664,70 @@ export const projectsModel: ProjectsModel = { actions.patchActiveMeta(meta); }), + readFileRaw: thunk( + async (_actions, { path, dirName, quick }, { getState, injections }) => { + const active = getState().active; + const dir = dirName ?? active?.meta.dirName; + if (!dir) { + return null; + } + const isActive = active?.meta.dirName === dir; + const storage = storageFor( + injections, + quick ?? (isActive ? active!.quick : false), + ); + try { + return await storage.read(dir, path); + } catch { + return null; + } + }, + ), + + listFiles: thunk( + async (_actions, { subdir, dirName, quick }, { getState, injections }) => { + const active = getState().active; + const dir = dirName ?? active?.meta.dirName; + if (!dir) { + return []; + } + const isActive = active?.meta.dirName === dir; + const storage = storageFor( + injections, + quick ?? (isActive ? active!.quick : false), + ); + try { + return await storage.list(dir, subdir); + } catch { + return []; + } + }, + ), + + projectSizes: thunk(async (_actions, _payload, { getState, injections }) => { + const storage = injections.libraryStorage; + const sizes: Record = {}; + for (const meta of getState().projects) { + let bytes = 0; + let runs = 0; + const walk = async (subdir?: string): Promise => { + const entries = await storage.list(meta.dirName, subdir).catch(() => []); + for (const entry of entries) { + if (entry.type === "directory") { + await walk(entry.path); + } else { + bytes += entry.size; + } + } + }; + await walk(); + const runDirs = await storage.list(meta.dirName, RUNS_DIR).catch(() => []); + runs = runDirs.filter((entry) => entry.type === "directory").length; + sizes[meta.dirName] = { bytes, runs }; + } + return sizes; + }), + startRuns: thunk(async (actions, requests, { getState }) => { const active = getState().active; if (!active || requests.length === 0) { diff --git a/src/store/settings.ts b/src/store/settings.ts index b3a1aca0..7e456ca8 100644 --- a/src/store/settings.ts +++ b/src/store/settings.ts @@ -42,11 +42,38 @@ export interface SimulationSettings { uiUpdateFrequency: number; } +/** UI theme (ADR-003 §6): drives the shell tokens via [data-theme]. */ +export type ThemeName = "dark" | "light"; + +const THEME_STORAGE_KEY = "atomify_theme"; + +export const loadThemeFromStorage = (): ThemeName => { + try { + const stored = localStorage.getItem(THEME_STORAGE_KEY); + if (stored === "dark" || stored === "light") { + return stored; + } + } catch (e) { + console.warn("Failed to load theme from localStorage", e); + } + return "dark"; +}; + +export const saveThemeToStorage = (theme: ThemeName) => { + try { + localStorage.setItem(THEME_STORAGE_KEY, theme); + } catch (e) { + console.warn("Failed to save theme to localStorage", e); + } +}; + export interface SettingsModel { render: RenderSettings; simulation: SimulationSettings; + theme: ThemeName; setRender: Action; setSimulation: Action; + setTheme: Action; } const defaultRenderSettings: RenderSettings = { @@ -73,6 +100,7 @@ export const settingsModel: SettingsModel = { uiUpdateFrequency: 15, }, render: initialRenderSettings, + theme: loadThemeFromStorage(), setRender: action((state, render: RenderSettings) => { state.render = render; saveRenderSettingsToStorage(render); @@ -81,4 +109,8 @@ export const settingsModel: SettingsModel = { state.simulation = simulation; setSyncFrequency(simulation.speed); }), + setTheme: action((state, theme: ThemeName) => { + state.theme = theme; + saveThemeToStorage(theme); + }), }; diff --git a/src/theme.ts b/src/theme.ts index d9aef724..7e744c96 100644 --- a/src/theme.ts +++ b/src/theme.ts @@ -1,5 +1,49 @@ import { theme } from "antd"; +import type { ThemeConfig } from "antd"; +import type { ThemeName } from "./store/settings"; +/** + * antd bridge for the new shell (ADR-003 §6): antd remains for commodity + * inner components (Upload dragger, notifications, tooltips) and is themed + * from the same values as the CSS custom-property tokens in + * src/shell/tokens.css. Keep the two in sync when changing tokens. + */ +export const shellThemeConfig = (mode: ThemeName): ThemeConfig => + mode === "dark" + ? { + algorithm: theme.darkAlgorithm, + token: { + colorPrimary: "#3F6EFF", + borderRadius: 10, + colorBgBase: "#0B0D12", + colorBgContainer: "#14171E", + colorBgElevated: "#1A1E27", + colorText: "#EAECEF", + colorTextSecondary: "#9AA1AE", + colorBorder: "rgba(255,255,255,0.13)", + colorSplit: "rgba(255,255,255,0.07)", + fontFamily: + "Manrope, -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif", + }, + } + : { + algorithm: theme.defaultAlgorithm, + token: { + colorPrimary: "#2E5BFF", + borderRadius: 10, + colorBgBase: "#F3F5F9", + colorBgContainer: "#FFFFFF", + colorBgElevated: "#FFFFFF", + colorText: "#10131A", + colorTextSecondary: "#545C6B", + colorBorder: "rgba(17,20,28,0.16)", + colorSplit: "rgba(17,20,28,0.09)", + fontFamily: + "Manrope, -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif", + }, + }; + +/** Legacy config for the embedded-mode shell (unchanged). */ export const darkThemeConfig = { algorithm: theme.darkAlgorithm, token: { From 56449b1b02ac93a1587d412d06ef28294fbd15c2 Mon Sep 17 00:00:00 2001 From: Anders Hafreager Date: Sat, 11 Jul 2026 11:54:26 +0200 Subject: [PATCH 12/34] Add the projects shell: sidebar, home, library, workspace, run detail (ADR-003) Implements the new non-embedded experience per the final design: - Bespoke token-based shell (dark + light via [data-theme]) with Manrope / JetBrains Mono, global sidebar (Home, Example library, projects list with running pulse, New project, theme toggle + Settings + engine chip). - Home (greeting, continue card with last frame.png, create/quick-run cards, recent projects, examples row) and the Example library screen. - Project workspace: header (breadcrumb, displayName + dirName/, Running pill, Share, Run split-button with dropdown, project menu), Files | Runs | Notebook tabs; quick-run banner with Save as project. - Files tab (search, new file, upload + drop-anywhere, input-script chip, run/set-input/edit/download/rename/delete row actions); Monaco editor sub-screen with debounced autosave states and read-only run snapshots. - Runs tab (status rows, vars chips, live progress, sweep group headers with cancel-remaining, queued entries) and run detail (live View pane / frame.png / run-again placeholder, following console, status panel, speed slider, output files). - Modals: New run (input-script chooser, parameter rows with per-variable sweeps, KOKKOS toggle, command preview), New project (blank / example / upload + colors), Settings (theme, rendering, storage meter + persist + per-project sizes), rename, delete (blocked while running). - Deep links (?project&tab&run&file) with history.replaceState sync, autosave flush on hide + refresh on focus, notices and run-completion as toasts (no console modal in the shell). - Embedded mode keeps the legacy path verbatim via src/EmbeddedApp.tsx; View gains an opt-in pane mode; Notebook opens the project's notebook path directly; Examples data loading extracted to useExamples. Co-Authored-By: Claude Fable 5 --- index.html | 7 + src/App.tsx | 167 +----- src/EmbeddedApp.tsx | 164 ++++++ src/containers/Examples.tsx | 69 +-- src/containers/Notebook.tsx | 95 ++-- src/containers/View.tsx | 92 +-- src/hooks/index.ts | 1 + src/hooks/useExamples.ts | 122 ++++ src/shell/EditorScreen.tsx | 249 ++++++++ src/shell/ExamplesScreen.tsx | 178 ++++++ src/shell/FilesTab.tsx | 541 ++++++++++++++++++ src/shell/HomeScreen.tsx | 667 ++++++++++++++++++++++ src/shell/NotebookTab.tsx | 83 +++ src/shell/ProjectWorkspace.tsx | 424 ++++++++++++++ src/shell/RunDetail.tsx | 720 ++++++++++++++++++++++++ src/shell/RunsTab.tsx | 500 ++++++++++++++++ src/shell/Shell.tsx | 455 +++++++++++++++ src/shell/ShellContext.tsx | 40 ++ src/shell/Sidebar.tsx | 346 ++++++++++++ src/shell/deepLink.ts | 63 +++ src/shell/format.ts | 89 +++ src/shell/icons.tsx | 313 ++++++++++ src/shell/modals/DeleteProjectModal.tsx | 129 +++++ src/shell/modals/NewProjectModal.tsx | 547 ++++++++++++++++++ src/shell/modals/NewRunModal.tsx | 570 +++++++++++++++++++ src/shell/modals/RenameModal.tsx | 45 ++ src/shell/modals/SettingsModal.tsx | 532 +++++++++++++++++ src/shell/runPlan.ts | 147 +++++ src/shell/tokens.css | 237 ++++++++ src/shell/ui.tsx | 703 +++++++++++++++++++++++ src/store/render.ts | 5 +- 31 files changed, 7990 insertions(+), 310 deletions(-) create mode 100644 src/EmbeddedApp.tsx create mode 100644 src/hooks/useExamples.ts create mode 100644 src/shell/EditorScreen.tsx create mode 100644 src/shell/ExamplesScreen.tsx create mode 100644 src/shell/FilesTab.tsx create mode 100644 src/shell/HomeScreen.tsx create mode 100644 src/shell/NotebookTab.tsx create mode 100644 src/shell/ProjectWorkspace.tsx create mode 100644 src/shell/RunDetail.tsx create mode 100644 src/shell/RunsTab.tsx create mode 100644 src/shell/Shell.tsx create mode 100644 src/shell/ShellContext.tsx create mode 100644 src/shell/Sidebar.tsx create mode 100644 src/shell/deepLink.ts create mode 100644 src/shell/format.ts create mode 100644 src/shell/icons.tsx create mode 100644 src/shell/modals/DeleteProjectModal.tsx create mode 100644 src/shell/modals/NewProjectModal.tsx create mode 100644 src/shell/modals/NewRunModal.tsx create mode 100644 src/shell/modals/RenameModal.tsx create mode 100644 src/shell/modals/SettingsModal.tsx create mode 100644 src/shell/runPlan.ts create mode 100644 src/shell/tokens.css create mode 100644 src/shell/ui.tsx diff --git a/index.html b/index.html index f2a7762d..f744ef05 100644 --- a/index.html +++ b/index.html @@ -9,6 +9,13 @@ Atomify LAMMPS - a real time simulator in the browser + + + + +