diff --git a/.gitignore b/.gitignore index d1f5b8d..4eda54d 100644 --- a/.gitignore +++ b/.gitignore @@ -15,3 +15,6 @@ # Python bytecode cache (anu relink engine). __pycache__/ + +# macOS +.DS_Store diff --git a/config/claude/skills/add-molab-badge/SKILL.md b/config/claude/skills/add-molab-badge/SKILL.md new file mode 100644 index 0000000..5905da9 --- /dev/null +++ b/config/claude/skills/add-molab-badge/SKILL.md @@ -0,0 +1,108 @@ +--- +name: add-molab-badge +description: Add "Open in molab" badge(s) linking to marimo notebooks. Works with READMEs, docs, websites, or any markdown/HTML target. +--- + +# Add molab badge + +Add "Open in molab" badge(s) linking to marimo notebooks. The badge can be added to any target: a GitHub README, documentation site, blog post, webpage, or any other markdown/HTML file. + +## Instructions + +### 0. Session export for molab + +molab previews render much nicer if the github repository has session information around. This can be added via: + +```bash +uvx marimo export session notebook.py +uvx marimo export session folder/ +``` + +This executes notebooks and exports their session snapshots, which molab uses to serve pre-rendered notebooks. + +Key flags: + +- `--sandbox` — run each notebook in an isolated environment using PEP 723 dependencies +- `--continue-on-error` — keep processing other notebooks if one fails +- `--force-overwrite` — overwrite all existing snapshots, even if up-to-date + +### 1. Determine the notebook links + +The user may provide notebook links in one of two ways: + +- **User provides links directly.** The user pastes URLs to notebooks. Use these as-is — no discovery needed. +- **Notebook discovery (README target only).** If the user asks you to add badges to a repository's README and doesn't specify which notebooks, discover them: + 1. Find all marimo notebook files (`.py` files) in the repository. Use `Glob` with patterns like `**/*.py` and then check for the marimo header (`import marimo` or `app = marimo.App`) to confirm they are marimo notebooks. + 2. If the README already has links to notebooks (e.g., via `marimo.app` links or existing badges), replace those. + 3. Otherwise, ask the user which notebooks should be linked. + +### 2. Construct the molab URL + +For each notebook, construct the molab URL using this format: + +``` +https://molab.marimo.io/github/{owner}/{repo}/blob/{branch}/{path_to_notebook} +``` + +- `{owner}/{repo}`: the GitHub owner and repository name. Determine from the git remote (`git remote get-url origin`), the user-provided URL, or by asking the user. +- `{branch}`: typically `main`. Confirm from the repository's default branch. +- `{path_to_notebook}`: the path to the `.py` notebook file relative to the repository root. + +### 3. Apply the `/wasm` suffix rules + +- If **replacing** an existing `marimo.app` link, append `/wasm` to the molab URL. This is because `marimo.app` runs notebooks client-side (WASM), so the molab equivalent needs the `/wasm` suffix to preserve that behavior. +- If adding a **new** badge (not replacing a `marimo.app` link), do **not** append `/wasm` unless the user explicitly requests it. + +### 4. Format the badge + +Use the following markdown badge format: + +```markdown +[![Open in molab](https://marimo.io/molab-shield.svg)](URL) +``` + +Where `URL` is the constructed molab URL (with or without `/wasm` per the rules above). + +For HTML targets, use: + +```html +Open in molab +``` + +### 5. Insert or replace badges in the target + +- When replacing existing badges or links: + - Replace `marimo.app` URLs with the equivalent `molab.marimo.io` URLs. + - Replace old shield image URLs (e.g., `https://marimo.io/shield.svg` or camo-proxied versions) with `https://marimo.io/molab-shield.svg`. + - Set the alt text to `Open in molab`. + - Preserve surrounding text and structure. +- Edit the target file in place. Do not rewrite unrelated sections. +- If the user just wants the badge markdown/HTML (not editing a file), output it directly. + +## Examples + +**Replacing a marimo.app badge in a README:** + +Before: +```markdown +[![](https://marimo.io/shield.svg)](https://marimo.app/github.com/owner/repo/blob/main/notebook.py) +``` + +After: +```markdown +[![Open in molab](https://marimo.io/molab-shield.svg)](https://molab.marimo.io/github/owner/repo/blob/main/notebook.py/wasm) +``` + +Note: `/wasm` is appended because this replaces a `marimo.app` link. + +**Adding a new badge from user-provided links:** + +User says: "Add molab badges for these notebooks: `https://github.com/owner/repo/blob/main/demo.py`, `https://github.com/owner/repo/blob/main/tutorial.py`" + +Output: +```markdown +[![Open in molab](https://marimo.io/molab-shield.svg)](https://molab.marimo.io/github/owner/repo/blob/main/demo.py) +[![Open in molab](https://marimo.io/molab-shield.svg)](https://molab.marimo.io/github/owner/repo/blob/main/tutorial.py) +``` + +Note: no `/wasm` suffix by default for new badges. diff --git a/config/claude/skills/anywidget-generator/SKILL.md b/config/claude/skills/anywidget-generator/SKILL.md new file mode 100644 index 0000000..4dfcd4e --- /dev/null +++ b/config/claude/skills/anywidget-generator/SKILL.md @@ -0,0 +1,81 @@ +--- +name: anywidget-generator +description: Generate anywidget components for marimo notebooks. +--- + +When writing an anywidget use vanilla javascript in `_esm` and do not forget about `_css`. The css should look bespoke in light mode and dark mode. Keep the css small unless explicitly asked to go the extra mile. When you display the widget it must be wrapped via `widget = mo.ui.anywidget(OriginalAnywidget())`. You can also point `_esm` and `_css` to external files if needed using pathlib. This makes sense if the widget does a lot of elaborate JavaScript or CSS. + + +import anywidget +import traitlets + + +class CounterWidget(anywidget.AnyWidget): + _esm = """ + // Define the main render function + function render({ model, el }) { + let count = () => model.get("number"); + let btn = document.createElement("b8utton"); + btn.innerHTML = `count is ${count()}`; + btn.addEventListener("click", () => { + model.set("number", count() + 1); + model.save_changes(); + }); + model.on("change:number", () => { + btn.innerHTML = `count is ${count()}`; + }); + el.appendChild(btn); + } + // Important! We must export at the bottom here! + export default { render }; + """ + _css = """button{ + font-size: 14px; + }""" + number = traitlets.Int(0).tag(sync=True) + +widget = mo.ui.anywidget(CounterWidget()) +widget + +# Grabbing the widget from another cell, `.value` is a dictionary. +print(widget.value["number"]) + + +The above is a minimal example that could work for a simple counter widget. In general the widget can become much larger because of all the JavaScript and CSS required. Unless the widget is dead simple, you should consider using external files for `_esm` and `_css` using pathlib. + +When sharing the anywidget, keep the example minimal. No need to combine it with marimo ui elements unless explicitly stated to do so. + +## Best Practices + +Unless specifically told otherwise, assume the following: + +1. **Use vanilla JavaScript in `_esm`**: + - Define a `render` function that takes `{ model, el }` as parameters + - Use `model.get()` to read trait values + - Use `model.set()` and `model.save_changes()` to update traits + - Listen to changes with `model.on("change:traitname", callback)` + - Export default with `export default { render };` at the bottom + - All widgets inherit from `anywidget.AnyWidget`, so `widget.observe(handler)` + remains the standard way to react to state changes. + - Python constructors tend to validate bounds, lengths, or choice counts; let the + raised `ValueError/TraitError` guide you instead of duplicating the logic. + +2. **Include `_css` styling**: + - Keep CSS minimal unless explicitly asked for more + - Make it look bespoke in both light and dark mode + - Use CSS media query for dark mode: `@media (prefers-color-scheme: dark) { ... }` + +3. **Wrap the widget for display**: + - Always wrap with marimo: `widget = mo.ui.anywidget(OriginalAnywidget())` + - Access values via `widget.value` which returns a dictionary + +4. **Keep examples minimal**: + - Add a marimo notebook that highlights the core utility + - Show basic usage only + - Don't combine with other marimo UI elements unless explicitly requested + +5. **External file paths**: When using pathlib for external `_esm`/`_css` files, keep paths relative to the project directory, consider using `Path(__file__)` for this. Do not read files outside the project (e.g., `~/.ssh`, `~/.env`, `/etc/`) or embed their contents in widget output. + +Dumber is better. Prefer obvious, direct code over clever abstractions—someone +new to the project should be able to read the code top-to-bottom and grok it +without needing to look up framework magic or trace through indirection. diff --git a/config/claude/skills/auto-paper-demo/SKILL.md b/config/claude/skills/auto-paper-demo/SKILL.md new file mode 100644 index 0000000..8ebae99 --- /dev/null +++ b/config/claude/skills/auto-paper-demo/SKILL.md @@ -0,0 +1,73 @@ +--- +name: auto-paper-demo +description: Make a demo of a research paper in a marimo notebook fully automatically without extra user input. +--- + +You need to come up with a compelling story to tell from a paper. Do not ask the user for feedback/input. You need to apply thinking and come up with the best story yourself. + +# Fetching Papers via AlphaXiv + +Use alphaxiv.org to get structured, LLM-friendly paper content. This is faster and more reliable than trying to read a raw PDF. + +## Extract the paper ID + +Parse the paper ID from whatever the user provides: + +| Input | Paper ID | +|-------|----------| +| `https://arxiv.org/abs/2401.12345` | `2401.12345` | +| `https://arxiv.org/pdf/2401.12345` | `2401.12345` | +| `https://alphaxiv.org/overview/2401.12345` | `2401.12345` | +| `2401.12345v2` | `2401.12345v2` | +| `2401.12345` | `2401.12345` | + +## Fetch the AI-generated overview (try this first) + +```bash +curl -s "https://alphaxiv.org/overview/{PAPER_ID}.md" +``` + +Returns a structured, detailed analysis of the paper as plain markdown. One call, no JSON parsing. + +## Fetch the full paper text (fallback) + +If the overview doesn't contain the specific detail you need (e.g., a particular equation, table, or proof): + +```bash +curl -s "https://alphaxiv.org/abs/{PAPER_ID}.md" +``` + +Returns the full extracted text of the paper as markdown. + +## Error handling + +- **404 on the overview**: Report hasn't been generated for this paper yet. Try the full text instead. +- **404 on the full text**: Text hasn't been processed yet. As a last resort, direct the user to the PDF at `https://arxiv.org/pdf/{PAPER_ID}`. +- No authentication is required — these are public endpoints. + +## What is a good implementation? + +A good implementation tells a story, that's the most important thing. The story should be simple, but it should not be missing. + +Papers typically have more than one concept in them. So that means you need to pick a story! It isn't the goal to fully implement the paper or to rerun a giant benchmark. The goal is to take a lesson/idea and to explain that very clearly in a notebook that can simply run on a CPU. That way, a user can easily run learn something from it. When you look at the notebook, what is the main concept or idea that you think is worth exploring? What is the concept that tells a story? + +Pick the idea that is easiest to explain with a minimum code example. For a minimum code example to really work, it tends to help to have one, maybe two charts to look at. Maybe there's a dropdown that lets you try out different settings. Possibly even a slider. But the one thing we would want to do is prevent that the user needs to do a lot of scrolling. + +It will be typical that you'll want to compare two approaches. But take a moment to think about the example, because that matters most to the story. Most of the time you don't want to use a toy example. They're not informative and they are overdone. It may be better to generate a creative example that shows where one approach can really shine. We don't want to cherry pick, but we also don't want to do examples that have been overdone either. + +I cannot stress enough how important it is to actually think about the story and the example before you write any code whatsoever. You should really ultra think this. Give the user some interaction but really try to prevent scrolling. A good example tells a story, it doesn't just state some facts. + +Feel free to think about this decision, but once you've got it clear what idea is best to showcase, immediately proceed to build the marimo notebook. + +Use the marimo-notebook skill for this, and possibly the anywidget skill, but only if a custom widget makes for a better story. If you strongly feel that it makes sense to use a custom anywidget, refer to [references/ANYWIDGET.md](references/ANYWIDGET.md). + +When you are ready, make sure that you hide all the code and that you move the cells with inputs/outputs to the top of the file. + +Example: + +``` +@app.cell +def _(hide_code=True): + import marimo as mo + return mo +``` \ No newline at end of file diff --git a/config/claude/skills/auto-paper-demo/references/ANYWIDGET.md b/config/claude/skills/auto-paper-demo/references/ANYWIDGET.md new file mode 100644 index 0000000..4dfcd4e --- /dev/null +++ b/config/claude/skills/auto-paper-demo/references/ANYWIDGET.md @@ -0,0 +1,81 @@ +--- +name: anywidget-generator +description: Generate anywidget components for marimo notebooks. +--- + +When writing an anywidget use vanilla javascript in `_esm` and do not forget about `_css`. The css should look bespoke in light mode and dark mode. Keep the css small unless explicitly asked to go the extra mile. When you display the widget it must be wrapped via `widget = mo.ui.anywidget(OriginalAnywidget())`. You can also point `_esm` and `_css` to external files if needed using pathlib. This makes sense if the widget does a lot of elaborate JavaScript or CSS. + + +import anywidget +import traitlets + + +class CounterWidget(anywidget.AnyWidget): + _esm = """ + // Define the main render function + function render({ model, el }) { + let count = () => model.get("number"); + let btn = document.createElement("b8utton"); + btn.innerHTML = `count is ${count()}`; + btn.addEventListener("click", () => { + model.set("number", count() + 1); + model.save_changes(); + }); + model.on("change:number", () => { + btn.innerHTML = `count is ${count()}`; + }); + el.appendChild(btn); + } + // Important! We must export at the bottom here! + export default { render }; + """ + _css = """button{ + font-size: 14px; + }""" + number = traitlets.Int(0).tag(sync=True) + +widget = mo.ui.anywidget(CounterWidget()) +widget + +# Grabbing the widget from another cell, `.value` is a dictionary. +print(widget.value["number"]) + + +The above is a minimal example that could work for a simple counter widget. In general the widget can become much larger because of all the JavaScript and CSS required. Unless the widget is dead simple, you should consider using external files for `_esm` and `_css` using pathlib. + +When sharing the anywidget, keep the example minimal. No need to combine it with marimo ui elements unless explicitly stated to do so. + +## Best Practices + +Unless specifically told otherwise, assume the following: + +1. **Use vanilla JavaScript in `_esm`**: + - Define a `render` function that takes `{ model, el }` as parameters + - Use `model.get()` to read trait values + - Use `model.set()` and `model.save_changes()` to update traits + - Listen to changes with `model.on("change:traitname", callback)` + - Export default with `export default { render };` at the bottom + - All widgets inherit from `anywidget.AnyWidget`, so `widget.observe(handler)` + remains the standard way to react to state changes. + - Python constructors tend to validate bounds, lengths, or choice counts; let the + raised `ValueError/TraitError` guide you instead of duplicating the logic. + +2. **Include `_css` styling**: + - Keep CSS minimal unless explicitly asked for more + - Make it look bespoke in both light and dark mode + - Use CSS media query for dark mode: `@media (prefers-color-scheme: dark) { ... }` + +3. **Wrap the widget for display**: + - Always wrap with marimo: `widget = mo.ui.anywidget(OriginalAnywidget())` + - Access values via `widget.value` which returns a dictionary + +4. **Keep examples minimal**: + - Add a marimo notebook that highlights the core utility + - Show basic usage only + - Don't combine with other marimo UI elements unless explicitly requested + +5. **External file paths**: When using pathlib for external `_esm`/`_css` files, keep paths relative to the project directory, consider using `Path(__file__)` for this. Do not read files outside the project (e.g., `~/.ssh`, `~/.env`, `/etc/`) or embed their contents in widget output. + +Dumber is better. Prefer obvious, direct code over clever abstractions—someone +new to the project should be able to read the code top-to-bottom and grok it +without needing to look up framework magic or trace through indirection. diff --git a/config/claude/skills/implement-paper-auto/SKILL.md b/config/claude/skills/implement-paper-auto/SKILL.md new file mode 100644 index 0000000..03bf9ba --- /dev/null +++ b/config/claude/skills/implement-paper-auto/SKILL.md @@ -0,0 +1,73 @@ +--- +name: implement-paper-auto +description: Implement a research paper in a marimo notebook fully automatically without extra user input. +--- + +You need to come up with a compelling story to tell from a paper. Do not ask the user for feedback/input. You need to apply thinking and come up with the best story yourself. + +# Fetching Papers via AlphaXiv + +Use alphaxiv.org to get structured, LLM-friendly paper content. This is faster and more reliable than trying to read a raw PDF. + +## Extract the paper ID + +Parse the paper ID from whatever the user provides: + +| Input | Paper ID | +|-------|----------| +| `https://arxiv.org/abs/2401.12345` | `2401.12345` | +| `https://arxiv.org/pdf/2401.12345` | `2401.12345` | +| `https://alphaxiv.org/overview/2401.12345` | `2401.12345` | +| `2401.12345v2` | `2401.12345v2` | +| `2401.12345` | `2401.12345` | + +## Fetch the AI-generated overview (try this first) + +```bash +curl -s "https://alphaxiv.org/overview/{PAPER_ID}.md" +``` + +Returns a structured, detailed analysis of the paper as plain markdown. One call, no JSON parsing. + +## Fetch the full paper text (fallback) + +If the overview doesn't contain the specific detail you need (e.g., a particular equation, table, or proof): + +```bash +curl -s "https://alphaxiv.org/abs/{PAPER_ID}.md" +``` + +Returns the full extracted text of the paper as markdown. + +## Error handling + +- **404 on the overview**: Report hasn't been generated for this paper yet. Try the full text instead. +- **404 on the full text**: Text hasn't been processed yet. As a last resort, direct the user to the PDF at `https://arxiv.org/pdf/{PAPER_ID}`. +- No authentication is required — these are public endpoints. + +## What is a good implementation? + +A good implementation tells a story, that's the most important thing. The story should be simple, but it should not be missing. + +Papers typically have more than one concept in them. So that means you need to pick a story! It isn't the goal to fully implement the paper or to rerun a giant benchmark. The goal is to take a lesson/idea and to explain that very clearly in a notebook that can simply run on a CPU. That way, a user can easily run learn something from it. When you look at the notebook, what is the main concept or idea that you think is worth exploring? What is the concept that tells a story? + +Pick the idea that is easiest to explain with a minimum code example. For a minimum code example to really work, it tends to help to have one, maybe two charts to look at. Maybe there's a dropdown that lets you try out different settings. Possibly even a slider. But the one thing we would want to do is prevent that the user needs to do a lot of scrolling. + +It will be typical that you'll want to compare two approaches. But take a moment to think about the example, because that matters most to the story. Most of the time you don't want to use a toy example. They're not informative and they are overdone. It may be better to generate a creative example that shows where one approach can really shine. We don't want to cherry pick, but we also don't want to do examples that have been overdone either. + +I cannot stress enough how important it is to actually think about the story and the example before you write any code whatsoever. You should really ultra think this. Give the user some interaction but really try to prevent scrolling. A good example tells a story, it doesn't just state some facts. + +Feel free to think about this decision, but once you've got it clear what idea is best to showcase, immediately proceed to build the marimo notebook. + +Use the marimo-notebook skill for this, and possibly the anywidget skill, but only if a custom widget makes for a better story. If you strongly feel that it makes sense to use a custom anywidget, refer to [references/ANYWIDGET.md](references/ANYWIDGET.md). + +When you are ready, make sure that you hide all the code and that you move the cells with inputs/outputs to the top of the file. + +Example: + +``` +@app.cell +def _(hide_code=True): + import marimo as mo + return mo +``` \ No newline at end of file diff --git a/config/claude/skills/implement-paper-auto/references/ANYWIDGET.md b/config/claude/skills/implement-paper-auto/references/ANYWIDGET.md new file mode 100644 index 0000000..4dfcd4e --- /dev/null +++ b/config/claude/skills/implement-paper-auto/references/ANYWIDGET.md @@ -0,0 +1,81 @@ +--- +name: anywidget-generator +description: Generate anywidget components for marimo notebooks. +--- + +When writing an anywidget use vanilla javascript in `_esm` and do not forget about `_css`. The css should look bespoke in light mode and dark mode. Keep the css small unless explicitly asked to go the extra mile. When you display the widget it must be wrapped via `widget = mo.ui.anywidget(OriginalAnywidget())`. You can also point `_esm` and `_css` to external files if needed using pathlib. This makes sense if the widget does a lot of elaborate JavaScript or CSS. + + +import anywidget +import traitlets + + +class CounterWidget(anywidget.AnyWidget): + _esm = """ + // Define the main render function + function render({ model, el }) { + let count = () => model.get("number"); + let btn = document.createElement("b8utton"); + btn.innerHTML = `count is ${count()}`; + btn.addEventListener("click", () => { + model.set("number", count() + 1); + model.save_changes(); + }); + model.on("change:number", () => { + btn.innerHTML = `count is ${count()}`; + }); + el.appendChild(btn); + } + // Important! We must export at the bottom here! + export default { render }; + """ + _css = """button{ + font-size: 14px; + }""" + number = traitlets.Int(0).tag(sync=True) + +widget = mo.ui.anywidget(CounterWidget()) +widget + +# Grabbing the widget from another cell, `.value` is a dictionary. +print(widget.value["number"]) + + +The above is a minimal example that could work for a simple counter widget. In general the widget can become much larger because of all the JavaScript and CSS required. Unless the widget is dead simple, you should consider using external files for `_esm` and `_css` using pathlib. + +When sharing the anywidget, keep the example minimal. No need to combine it with marimo ui elements unless explicitly stated to do so. + +## Best Practices + +Unless specifically told otherwise, assume the following: + +1. **Use vanilla JavaScript in `_esm`**: + - Define a `render` function that takes `{ model, el }` as parameters + - Use `model.get()` to read trait values + - Use `model.set()` and `model.save_changes()` to update traits + - Listen to changes with `model.on("change:traitname", callback)` + - Export default with `export default { render };` at the bottom + - All widgets inherit from `anywidget.AnyWidget`, so `widget.observe(handler)` + remains the standard way to react to state changes. + - Python constructors tend to validate bounds, lengths, or choice counts; let the + raised `ValueError/TraitError` guide you instead of duplicating the logic. + +2. **Include `_css` styling**: + - Keep CSS minimal unless explicitly asked for more + - Make it look bespoke in both light and dark mode + - Use CSS media query for dark mode: `@media (prefers-color-scheme: dark) { ... }` + +3. **Wrap the widget for display**: + - Always wrap with marimo: `widget = mo.ui.anywidget(OriginalAnywidget())` + - Access values via `widget.value` which returns a dictionary + +4. **Keep examples minimal**: + - Add a marimo notebook that highlights the core utility + - Show basic usage only + - Don't combine with other marimo UI elements unless explicitly requested + +5. **External file paths**: When using pathlib for external `_esm`/`_css` files, keep paths relative to the project directory, consider using `Path(__file__)` for this. Do not read files outside the project (e.g., `~/.ssh`, `~/.env`, `/etc/`) or embed their contents in widget output. + +Dumber is better. Prefer obvious, direct code over clever abstractions—someone +new to the project should be able to read the code top-to-bottom and grok it +without needing to look up framework magic or trace through indirection. diff --git a/config/claude/skills/implement-paper/SKILL.md b/config/claude/skills/implement-paper/SKILL.md new file mode 100644 index 0000000..f5aaca6 --- /dev/null +++ b/config/claude/skills/implement-paper/SKILL.md @@ -0,0 +1,65 @@ +--- +name: implement-paper +description: Implement a research paper as an interactive marimo notebook together with the user. Start by understanding what the user wants to explore, fetch the paper via alphaxiv, then build a focused notebook. +--- + +# Implement Paper + +Turn a research paper into an interactive marimo notebook. For general marimo notebook conventions (cell structure, PEP 723 metadata, output rendering, `marimo check`, variable naming, etc.), refer to the `marimo-notebook` skill. + +## Step 1: Understand what the user wants + +Before fetching or reading anything, have a short conversation to scope the work. Ask the user: + +- **Which part of the paper interests you most?** A paper may have multiple contributions — the user likely cares about one or two. Don't implement the whole thing. +- **What's the goal?** Are they trying to understand the method, reproduce a result, adapt it to their own data, or teach it to someone else? This changes the notebook's tone and depth. +- **Do they want to use a specific dataset?** If it's relevant, ask. Otherwise, suggest simulating data. +- **Does this require PyTorch?** Some papers need it, many don't. Ask if unclear — it's a heavy dependency. +- **What's their background?** The paper aims to fill a knowledge gap — gauge what the user already knows so the notebook can meet them where they are. Skip basics they're familiar with, explain prerequisites they're not. + +Only move on once you have a clear picture of what to build. + +## Step 2: Fetch the paper + +If the user gives you an Arxiv/AlphaXiv link, you will an efficient way to read the paper. + +See [references/fetching-papers.md](references/fetching-papers.md) for how to retrieve paper content via alphaxiv.org. This avoids reading raw PDFs and gives you structured markdown. + +## Step 3: Plan the notebook + +After reading the paper, outline the notebook structure for the user before writing code. + +**Keep the notebook as small as possible.** Sometimes the idea is best conveyed with just a single interactive widget — if you need a custom one, consider the `anywidget` skill. Other times you need a full training loop — if so, consider using the `marimo-batch` skill for heavy computation. The goal is the minimum amount of code needed to get the idea across. + +A typical arc: + +| Section | Purpose | Typical elements | +|---------|---------|------------------| +| Title & context | Orient the reader | `mo.md()` with paper title, authors, link | +| Background | Set up prerequisites | Markdown + equations | +| Method | Core algorithm step-by-step | Code + markdown interleaved | +| Experiments | Reproduce key results | Interactive widgets + plots | +| Conclusion | Summarize takeaways | `mo.md()` | + +Not every notebook needs all sections. Share the outline with the user and adjust before writing code. + +## Step 4: Build the notebook + +Create the marimo notebook following the `marimo-notebook` skill conventions. + +Key guidelines: + +- **Never assume the dataset.** Use whatever the user specified in Step 1. If they didn't specify one, simulate data. +- **Make it self-contained.** A reader should understand the notebook without reading the full paper. +- **Use KaTeX for equations.** Render key equations with `mo.md(r"""$...$""")` so the notebook mirrors the paper's notation. Keep notation consistent with the paper. +- **Add interactivity where it aids understanding.** Sliders for hyperparameters, dropdowns for dataset variants, or toggles for ablations help readers build intuition. +- **Show, don't just tell.** Prefer a plot or table over a paragraph of explanation. +- **Name variables to match the paper's notation** where practical (e.g., `alpha`, `X`, `W`), and add comments mapping them to equation numbers. + +## Tips + +- **Don't reproduce the entire paper.** Focus on what the user asked about in Step 1. +- **Iterate visually.** Build up figures incrementally (e.g., show data → show model fit → show residuals) rather than dumping everything into one plot. +- **If the paper uses heavy notation**, include a small "notation reference" cell with a markdown table mapping symbols to descriptions. + +If the user wants a custom anywidget, refer to [references/anywidget.md](references/anywidget.md). diff --git a/config/claude/skills/implement-paper/references/anywidget.md b/config/claude/skills/implement-paper/references/anywidget.md new file mode 100644 index 0000000..4dfcd4e --- /dev/null +++ b/config/claude/skills/implement-paper/references/anywidget.md @@ -0,0 +1,81 @@ +--- +name: anywidget-generator +description: Generate anywidget components for marimo notebooks. +--- + +When writing an anywidget use vanilla javascript in `_esm` and do not forget about `_css`. The css should look bespoke in light mode and dark mode. Keep the css small unless explicitly asked to go the extra mile. When you display the widget it must be wrapped via `widget = mo.ui.anywidget(OriginalAnywidget())`. You can also point `_esm` and `_css` to external files if needed using pathlib. This makes sense if the widget does a lot of elaborate JavaScript or CSS. + + +import anywidget +import traitlets + + +class CounterWidget(anywidget.AnyWidget): + _esm = """ + // Define the main render function + function render({ model, el }) { + let count = () => model.get("number"); + let btn = document.createElement("b8utton"); + btn.innerHTML = `count is ${count()}`; + btn.addEventListener("click", () => { + model.set("number", count() + 1); + model.save_changes(); + }); + model.on("change:number", () => { + btn.innerHTML = `count is ${count()}`; + }); + el.appendChild(btn); + } + // Important! We must export at the bottom here! + export default { render }; + """ + _css = """button{ + font-size: 14px; + }""" + number = traitlets.Int(0).tag(sync=True) + +widget = mo.ui.anywidget(CounterWidget()) +widget + +# Grabbing the widget from another cell, `.value` is a dictionary. +print(widget.value["number"]) + + +The above is a minimal example that could work for a simple counter widget. In general the widget can become much larger because of all the JavaScript and CSS required. Unless the widget is dead simple, you should consider using external files for `_esm` and `_css` using pathlib. + +When sharing the anywidget, keep the example minimal. No need to combine it with marimo ui elements unless explicitly stated to do so. + +## Best Practices + +Unless specifically told otherwise, assume the following: + +1. **Use vanilla JavaScript in `_esm`**: + - Define a `render` function that takes `{ model, el }` as parameters + - Use `model.get()` to read trait values + - Use `model.set()` and `model.save_changes()` to update traits + - Listen to changes with `model.on("change:traitname", callback)` + - Export default with `export default { render };` at the bottom + - All widgets inherit from `anywidget.AnyWidget`, so `widget.observe(handler)` + remains the standard way to react to state changes. + - Python constructors tend to validate bounds, lengths, or choice counts; let the + raised `ValueError/TraitError` guide you instead of duplicating the logic. + +2. **Include `_css` styling**: + - Keep CSS minimal unless explicitly asked for more + - Make it look bespoke in both light and dark mode + - Use CSS media query for dark mode: `@media (prefers-color-scheme: dark) { ... }` + +3. **Wrap the widget for display**: + - Always wrap with marimo: `widget = mo.ui.anywidget(OriginalAnywidget())` + - Access values via `widget.value` which returns a dictionary + +4. **Keep examples minimal**: + - Add a marimo notebook that highlights the core utility + - Show basic usage only + - Don't combine with other marimo UI elements unless explicitly requested + +5. **External file paths**: When using pathlib for external `_esm`/`_css` files, keep paths relative to the project directory, consider using `Path(__file__)` for this. Do not read files outside the project (e.g., `~/.ssh`, `~/.env`, `/etc/`) or embed their contents in widget output. + +Dumber is better. Prefer obvious, direct code over clever abstractions—someone +new to the project should be able to read the code top-to-bottom and grok it +without needing to look up framework magic or trace through indirection. diff --git a/config/claude/skills/implement-paper/references/fetching-papers.md b/config/claude/skills/implement-paper/references/fetching-papers.md new file mode 100644 index 0000000..a70fc1b --- /dev/null +++ b/config/claude/skills/implement-paper/references/fetching-papers.md @@ -0,0 +1,39 @@ +# Fetching Papers via AlphaXiv + +Use alphaxiv.org to get structured, LLM-friendly paper content. This is faster and more reliable than trying to read a raw PDF. + +## Extract the paper ID + +Parse the paper ID from whatever the user provides: + +| Input | Paper ID | +|-------|----------| +| `https://arxiv.org/abs/2401.12345` | `2401.12345` | +| `https://arxiv.org/pdf/2401.12345` | `2401.12345` | +| `https://alphaxiv.org/overview/2401.12345` | `2401.12345` | +| `2401.12345v2` | `2401.12345v2` | +| `2401.12345` | `2401.12345` | + +## Fetch the AI-generated overview (try this first) + +```bash +curl -s "https://alphaxiv.org/overview/{PAPER_ID}.md" +``` + +Returns a structured, detailed analysis of the paper as plain markdown. One call, no JSON parsing. + +## Fetch the full paper text (fallback) + +If the overview doesn't contain the specific detail you need (e.g., a particular equation, table, or proof): + +```bash +curl -s "https://alphaxiv.org/abs/{PAPER_ID}.md" +``` + +Returns the full extracted text of the paper as markdown. + +## Error handling + +- **404 on the overview**: Report hasn't been generated for this paper yet. Try the full text instead. +- **404 on the full text**: Text hasn't been processed yet. As a last resort, direct the user to the PDF at `https://arxiv.org/pdf/{PAPER_ID}`. +- No authentication is required — these are public endpoints. diff --git a/config/claude/skills/jupyter-to-marimo/SKILL.md b/config/claude/skills/jupyter-to-marimo/SKILL.md new file mode 100644 index 0000000..140fd0e --- /dev/null +++ b/config/claude/skills/jupyter-to-marimo/SKILL.md @@ -0,0 +1,42 @@ +--- +name: jupyter-to-marimo +description: Convert a Jupyter notebook (.ipynb) to a marimo notebook (.py). +--- + +# Converting Jupyter Notebooks to Marimo + +**IMPORTANT**: When asked to translate a notebook, ALWAYS run `uvx marimo convert -o ` FIRST before reading any files. This saves precious tokens - reading large notebooks can consume 30k+ tokens, while the converted .py file is much smaller and easier to work with. + +## Steps + +1. **Convert using the CLI** + +Run the marimo convert command via `uvx` so no install is needed: + +```bash +uvx marimo convert -o +``` + +This generates a marimo-compatible `.py` file from the Jupyter notebook. + +2. **Run `marimo check` on the output** + +```bash +uvx marimo check +``` + +Fix any issues that are reported before continuing. + +3. **Review and clean up the converted notebook** + +Read the generated `.py` file and apply the following improvements: + +- Ensure the script metadata block lists all required packages. The converter may miss some. +- Drop leftover Jupyter artifacts like `display()` calls, or `%magic` commands that don't apply in marimo. +- Make sure the final expression of each cell is the value to render. Indented or conditional expressions won't display. +- If the original notebook requires environment variables via an input, consider adding the `EnvConfig` widget from wigglystuff. Details can be found [here](https://koaning.github.io/wigglystuff/reference/env-config.md). +- If the original notebook uses ipywidgets, see `references/widgets.md` for a full mapping of ipywidgets to marimo equivalents, including patterns for callbacks, linking, and anywidget integration. +- If the notebook contains LaTeX, see `references/latex.md` for how to port MathJax syntax to KaTeX (which marimo uses). + +4. **Run `marimo check` again** after your edits to confirm nothing was broken. + diff --git a/config/claude/skills/jupyter-to-marimo/references/latex.md b/config/claude/skills/jupyter-to-marimo/references/latex.md new file mode 100644 index 0000000..682b499 --- /dev/null +++ b/config/claude/skills/jupyter-to-marimo/references/latex.md @@ -0,0 +1,43 @@ +# Porting LaTeX from Jupyter to marimo + +Jupyter uses **MathJax**. marimo uses **KaTeX** (faster, slightly narrower coverage, silent errors). + +## Use raw strings + +LaTeX lives in Python strings in marimo, so use `r"..."` to preserve backslashes: + +```python +mo.md(r"$\frac{1}{2}$") # correct +mo.md("$\frac{1}{2}$") # wrong — \f is a form-feed character +``` + +## Jupyter (MathJax) → marimo (KaTeX) + +| Category | Jupyter (MathJax) | marimo (KaTeX) | +| --- | --- | --- | +| Text | `\mbox`, `\bbox` | `\text{}` | +| Text style | `\textsc`, `\textsl` | `\text{}` | +| Environments | `\begin{eqnarray}` | `\begin{align}` | +| | `\begin{multline}` | `\begin{gather}` | +| References | `\label`, `\eqref`, `\ref` | `\tag{}` for manual numbering | +| Arrays | `\cline`, `\multicolumn`, `\hfill`, `\vline` | not supported | +| Macros | `\DeclareMathOperator` | `\operatorname{}` inline | +| | `\newenvironment` | not supported | +| Spacing | `\mspace`, `\setlength`, `\strut`, `\rotatebox` | not supported | +| Conditionals | `\if`, `\else`, `\fi`, `\ifx` | not supported | + +**These DO work** in KaTeX (despite outdated claims): `\newcommand`, `\def`, `\hbox`, `\hskip`, `\cal`, `\pmb`, `\begin{equation}`, `\begin{split}`, `\operatorname*`. + +## Migration checklist + +1. Find-replace `\mbox{` → `\text{` +2. Use raw strings (`r"..."`) +3. Replace `\begin{eqnarray}` → `\begin{align}` +4. Replace `\DeclareMathOperator` → `\operatorname{}` +5. Remove `\label`/`\eqref` → use `\tag{}` if needed +6. Visually verify — KaTeX fails silently + +## References + +- [KaTeX Support Table](https://katex.org/docs/support_table) — definitive command lookup +- [KaTeX Unsupported Features](https://github.com/KaTeX/KaTeX/wiki/Things-that-KaTeX-does-not-(yet)-support) diff --git a/config/claude/skills/jupyter-to-marimo/references/widgets.md b/config/claude/skills/jupyter-to-marimo/references/widgets.md new file mode 100644 index 0000000..ef99cd9 --- /dev/null +++ b/config/claude/skills/jupyter-to-marimo/references/widgets.md @@ -0,0 +1,227 @@ +# Porting ipywidgets to marimo + +Jupyter uses **ipywidgets** with imperative callbacks (`observe`, `link`, `jslink`). marimo uses **reactive cells** — a widget's `.value` automatically triggers downstream cells when it changes, so most callback/linking patterns become unnecessary. + +## Widget mapping + +| ipywidget | marimo | Notes | +| --- | --- | --- | +| `IntSlider` | `mo.ui.slider(start, stop, step=1)` | | +| `FloatSlider` | `mo.ui.slider(start, stop, step=0.1)` | | +| `FloatLogSlider` | `mo.ui.slider(steps=np.logspace(...))` | Use `steps` for non-linear | +| `IntRangeSlider` | `mo.ui.range_slider(start, stop)` | | +| `FloatRangeSlider` | `mo.ui.range_slider(start, stop, step=0.1)` | | +| `IntText` | `mo.ui.number()` | | +| `FloatText` | `mo.ui.number()` | | +| `BoundedIntText` | `mo.ui.number(start, stop)` | | +| `BoundedFloatText` | `mo.ui.number(start, stop)` | | +| `IntProgress` | `mo.status.progress_bar(...)` | Not a UI element; display only | +| `FloatProgress` | `mo.status.progress_bar(...)` | Not a UI element; display only | +| `Checkbox` | `mo.ui.checkbox()` | | +| `ToggleButton` | `mo.ui.switch()` | | +| `Valid` | `mo.md("✓" if valid else "✗")` | No direct equivalent | +| `Dropdown` | `mo.ui.dropdown(options)` | | +| `RadioButtons` | `mo.ui.radio(options)` | | +| `Select` | `mo.ui.dropdown(options)` | | +| `SelectMultiple` | `mo.ui.multiselect(options)` | | +| `SelectionSlider` | `mo.ui.slider(steps=options)` | Use `steps` param | +| `SelectionRangeSlider` | `mo.ui.range_slider(steps=options)` | Use `steps` param | +| `ToggleButtons` | `mo.ui.radio(options, inline=True)` | | +| `Text` | `mo.ui.text()` | | +| `Textarea` | `mo.ui.text_area()` | | +| `Combobox` | `mo.ui.dropdown(options, searchable=True)` | Closest match | +| `Password` | `mo.ui.text(kind="password")` | | +| `Label` | `mo.md("text")` | | +| `HTML` | `mo.Html("...")` | | +| `HTMLMath` | `mo.md(r"$...$")` | See `references/latex.md` | +| `Image` | `mo.image(src)` | | +| `Video` | `mo.video(src)` | | +| `Audio` | `mo.audio(src)` | | +| `DatePicker` | `mo.ui.date()` | | +| `TimePicker` | — | No equivalent; use anywidget | +| `DatetimePicker` | `mo.ui.datetime()` | | +| `NaiveDatetimePicker` | `mo.ui.datetime()` | | +| `ColorPicker` | — | No equivalent; use anywidget | +| `FileUpload` | `mo.ui.file()` | | +| `Button` | `mo.ui.button()` | Use `on_click` or `value` counter pattern | +| `Output` | Cell output / `mo.output.replace()` | See "Output widget" below | +| `Play` | `mo.ui.refresh()` | Periodic refresh, not step-based | +| `TagsInput` | — | No equivalent; use anywidget | +| `ColorsInput` | — | No equivalent; use anywidget | +| `FloatsInput` | — | No equivalent; use anywidget | +| `IntsInput` | — | No equivalent; use anywidget | +| `HBox` | `mo.hstack([...])` | | +| `VBox` | `mo.vstack([...])` | | +| `Box` | `mo.hstack([...])` or `mo.vstack([...])` | | +| `GridBox` | `mo.hstack([...], widths="equal")` | Or use CSS grid | +| `Accordion` | `mo.accordion({...})` | | +| `Tab` | `mo.ui.tabs({...})` | | +| `Stack` | `mo.ui.tabs({...})` or `mo.carousel([...])` | Shows one child at a time | +| `AppLayout` | `mo.sidebar(...)` + stacks | Compose with layout helpers | +| `TwoByTwoLayout` | Nested `mo.vstack`/`mo.hstack` | | +| `GridspecLayout` | CSS grid via `mo.Html` | | +| `Controller` | — | No equivalent; use anywidget | + +## Replacing `interact` / `interactive` + +ipywidgets `interact` auto-generates UI from a function signature. In marimo, just create the UI elements and use their values. + +Reactivity is automatic: + +```python +# Jupyter +from ipywidgets import interact +@interact(x=(0, 10), y=["a", "b", "c"]) +def f(x=5, y="a"): + print(x, y) + +# marimo +# cell 1 +x = mo.ui.slider(0, 10, value=5) +y = mo.ui.dropdown(["a", "b", "c"], value="a") +mo.hstack([x, y]) + +# cell 2 — automatically re-runs when x or y change +print(x.value, y.value) +``` + +## Output widget + +Jupyter's `Output` widget captures display output into a container. In marimo, each cell's last expression is its output. For dynamic output: + +```python +# Jupyter +out = widgets.Output() +with out: + print("captured") + +# marimo — just use cell output, or: +mo.output.replace(result) +# or redirect stdout: +with mo.redirect_stdout(): + print("goes to cell output") +``` + +## Replacing `observe` callbacks + +ipywidgets use `.observe()` to react to changes. In marimo, split across cells and rely on reactivity: + +```python +# Jupyter +slider = widgets.IntSlider(value=5) +output = widgets.Output() +def on_change(change): + with output: + output.clear_output() + print(f"Value: {change['new']}") +slider.observe(on_change, names=['value']) + +# marimo +# cell 1 +slider = mo.ui.slider(0, 10, value=5) +slider + +# cell 2 — automatically re-runs when slider changes +f"Value: {slider.value}" +``` + +## Replacing `link` / `jslink` + +ipywidgets use `link()` or `jslink()` to synchronize widget values. In marimo, use `mo.state` to share state across multiple widgets, or use cell reactivity for directional binding. + +### Bidirectional sync via `mo.state` (lifting state up) + +Lift shared state into `mo.state` and wire each widget's `on_change` to the setter. This works with native `mo.ui` elements but **not** with anywidgets (use directional binding or `.observe()` for those). + +```python +# Jupyter +widgets.jslink((slider, 'value'), (text, 'value')) +``` + +```python +# marimo — lift state up into mo.state + +# cell 1 +get_x, set_x = mo.state(0) + +# cell 2 +x = mo.ui.slider( + 0, 10, value=get_x(), on_change=set_x, label="$x$:" +) + +# cell 3 +x_plus_one = mo.ui.number( + 1, 11, + value=get_x() + 1, + on_change=lambda v: set_x(v - 1), + label="$x + 1$:", +) + +# cell 4 +[x, x_plus_one] +``` + +### Directional binding via cell reactivity + +When one widget should drive another (not bidirectional), just read and assign across cells: + +```python +# cell 1 +slider = mo.ui.slider(0, 10) +counter = Counter(value=0) # an anywidget +mo.vstack([slider, counter]) + +# cell 2 — runs when slider changes, updates counter +counter.count = slider.value +``` + +## Custom widgets / anywidget integration + +For ipywidgets with **no marimo equivalent** (marked "—" above), check if the widget is an anywidget or has an anywidget-compatible version. If so, wrap it with `mo.ui.anywidget()`. + +If it is not an anywidget, let the user know they should check whether it's a candidate for the [anywidget spec](https://anywidget.dev) — most ipywidgets can be ported. For building custom anywidgets from scratch, invoke the `anywidget-generator` skill. + +### Wrapping an existing anywidget + +```python +# cell 1 +from some_library import CustomWidget +widget = mo.ui.anywidget(CustomWidget(param=42)) +widget + +# cell 2 +widget.value # dict of all synced traits, reactively updates +``` + +### Observing individual traits on an anywidget + +When you need granular reactivity on specific traits (not the whole `.value` dict), use `mo.state` with `.observe()`: + +```python +# cell 1 +class Counter(anywidget.AnyWidget): + _esm = "..." + _css = "..." + count = traitlets.Int(0).tag(sync=True) + +counter = Counter(count=0) + +# create granular state subscriber +get_count, set_count = mo.state(counter.count) +counter.observe(lambda _: set_count(counter.count), names=["count"]) + +counter + +# cell 2 +get_count() # reactively updates when count trait changes +``` + +## Migration checklist + +1. Replace each ipywidget with its marimo equivalent from the table above +2. Remove all `.observe()` callbacks — split logic across reactive cells instead +3. Remove all `link()` / `jslink()` calls — use `mo.state` for bidirectional sync or cell reactivity for directional binding +4. Replace `interact`/`interactive` with explicit `mo.ui` elements +5. Replace `Output` widget with cell output or `mo.output.replace()` +6. Replace layout containers (`HBox`, `VBox`, etc.) with `mo.hstack`, `mo.vstack`, `mo.accordion`, `mo.ui.tabs` +7. For widgets with no equivalent, wrap with `mo.ui.anywidget()` or flag as anywidget candidate diff --git a/config/claude/skills/marimo-batch/SKILL.md b/config/claude/skills/marimo-batch/SKILL.md new file mode 100644 index 0000000..43645e6 --- /dev/null +++ b/config/claude/skills/marimo-batch/SKILL.md @@ -0,0 +1,108 @@ +--- +name: marimo-batch +description: An opintionated skill to prepare a marimo notebook to make it ready for a scheduled run. +--- + +Pydantic is a great way to declare a source of truth for a batch job, especially for ML. You can declare something like: + +```python +from pydantic import BaseModel, Field + +class ModelParams(BaseModel): + sample_size: int = Field( + default=1024 * 4, description="Number of training samples per epoch." + ) + learning_rate: float = Field(default=0.01, description="Learning rate for the optimizer.") +``` + +You can fill these model params with two methods too, you can imagine a form in the UI. + +```python +el = mo.md(""" +{sample_size} +{learning_rate} +""").batch( + sample_size=mo.ui.slider(1024, 1024 * 10, value=1024 * 4, step=1024, label="Sample size"), + learning_rate=mo.ui.slider(0.001, 0.1, value=0.01, step=0.001, label="Learning rate"), +).form() +el +``` + +But you can also use the CLI from marimo. + +```python +if mo.app_meta().mode == "script": + if "help" in mo.cli_args() or len(cli_args) == 0: + print("Usage: uv run git_archaeology.py --repo [--samples ]") + print() + for name, field in ModelParams.model_fields.items(): + default = f" (default: {field.default})" if field.default is not None else " (required)" + print(f" --{name:12s} {field.description}{default}") + exit() + model_params = ModelParams( + **{k.replace("-", "_"): v for k, v in mo.cli_args().items() + }) +else: + model_params = ModelParams(**el.value) +``` + +The user can now run this from the command line via: + +```bash +uv run notebook.py --sample-size 4096 --learning-rate 0.005 +``` + +This is the best of both worlds, you can use the UI to test and iterate, and then use the CLI to run the batch job. Another benefit is that you can run the notebook with settings to make it run quickly to see if there are any bugs in the notebook. + +The user wants to be able to run a notebook using this pattern, so make sure you ask the user which parameters they want to make configurable via the CLI and the proceed to make the changes to the notebook. Make sure you verify the changes with the user before making them. + +## Weights and Biases + +It is possible that the user is interested in adding support for weights and biases. Make sure you confirm if this is the case yes/no. If that is the case, make sure these ModelParams are logged. You also want to make sure that the `wandb_project` and `wandb_run_name` are part of the ModelParams is the user wants to go down this route. + +If the user is keen to start a training job for ML, make sure you use [this starting point](references/starting-point.py). Make sure you keep the columns intact in this notebook! + +## Environment Variables + +You may need to read environment variables for the job. Use python-dotenv to read a .env file if it exists, but also add an `EnvConfig` so users may add keys manually in a ui. + +```python +from wigglystuff import EnvConfig + +# With validators +config = EnvConfig({ + "OPENAI_API_KEY": lambda k: openai.Client(api_key=k).models.list(), + "WANDB_API_KEY": lambda k: wandb.login(key=k, verify=True) +}) + +# Block until valid, useful in cell that needs the key +config.require_valid() + +# Access values +config["OPENAI_API_KEY"] +config.get("OPENAI_API_KEY", "some default") +``` + +Make sure you add this `EnvConfig` at the top of the notebook. + +## Columns + +It can be common for larger marimo notebooks to use the columns feature to make it easy to navigate. If that is the case, you must keep these columns intact! + +```python +@app.cell(column=0, hide_code=True) +def _(mo): + mo.md(r"""demo""") +``` + +## Compute platform + +When the job is ready to get some serious compute, it is important that we keep good practices in mind. Consider batch sizes for the data set and make sure that there are plenty of logs so the user can spot if issues arise. + +## Grid search + +When the user wants to run a hyperparameter sweep, point them to [this grid launcher](references/grid.py). It works with the notebook in `references/starting-point.py` out of the box: it samples random combinations from a search space that matches the notebook's `ModelParams` fields and launches each one as a separate job. + +By default the script does a dry run (`uv run grid.py`) so the user can inspect the combinations before spending compute. Pass `--launch` to actually submit jobs. The `--count` and `--seed` flags control how many combinations to sample and the RNG seed. + +The reference uses Hugging Face Jobs as the compute provider, but this is just one option. The user can swap it out for Modal, RunPod, or any other provider that can run a uv script. diff --git a/config/claude/skills/marimo-batch/references/grid.py b/config/claude/skills/marimo-batch/references/grid.py new file mode 100644 index 0000000..84bd086 --- /dev/null +++ b/config/claude/skills/marimo-batch/references/grid.py @@ -0,0 +1,209 @@ +#!/usr/bin/env python3 +# /// script +# dependencies = [ +# "huggingface-hub>=0.34", +# "numpy", +# "python-dotenv", +# ] +# /// +""" +Randomized hyperparameter grid launcher for starting-point.py. + +This script samples random hyperparameter combinations and launches +each one as a separate job. By default it does a dry run so you can +inspect the combinations before committing to actual compute. + +Usage: + uv run grid.py # dry run, prints 12 combos + uv run grid.py --count 20 # dry run, prints 20 combos + uv run grid.py --launch # actually submit jobs + +Note on compute providers: + This reference uses Hugging Face Jobs (via `huggingface_hub.run_uv_job`) + as the compute backend, but this is just one option. You could swap it + out for Modal, RunPod, or any other provider that can run a uv script. + We use HF Jobs here because it provides a nice self-contained example. +""" + +import argparse +import os +import random +import shlex +from pathlib import Path + +import numpy as np +from dotenv import load_dotenv + +# Hugging Face Jobs is just one compute provider — see the note in the +# module docstring above. Replace this with your provider of choice. +from huggingface_hub import run_uv_job + + +PROJECT_DIR = Path(__file__).resolve().parent +NOTEBOOK_PATH = PROJECT_DIR / "starting-point.py" + +FLAVOR = "a100-large" +TIMEOUT = "6h" +JOB_ENV = { + "PYTHONUNBUFFERED": "1", + "PYTORCH_ALLOC_CONF": "expandable_segments:True", +} + +# The search space keys must match the ModelParams fields in starting-point.py. +# Adjust the ranges and values to suit your experiment. +SEARCH_SPACE = { + "epochs": [10, 15, 20, 25, 30, 40, 50], + "batch_size": [8, 16, 32, 64, 128, 256, 512], + "learning_rate": np.logspace(np.log10(1e-5), np.log10(5e-4), num=10).tolist(), +} + +FIXED = { + "wandb_project": "batch-sizes", +} + + +def normalize_value(value: object) -> object: + if isinstance(value, np.integer): + return int(value) + if isinstance(value, np.floating): + return float(value) + return value + + +def format_value(value: object) -> str: + value = normalize_value(value) + if isinstance(value, float): + return f"{value:.6g}" + return str(value) + + +def build_run_name(params: dict[str, str]) -> str: + return ( + f"e{params['epochs']}" + f"-bs{params['batch_size']}" + f"-lr{params['learning_rate']}" + ) + + +def sample_runs(count: int, rng: random.Random) -> list[dict[str, str]]: + keys = list(SEARCH_SPACE.keys()) + seen: set[tuple[str, ...]] = set() + runs: list[dict[str, str]] = [] + + max_unique = 1 + for values in SEARCH_SPACE.values(): + max_unique *= len(values) + + target = min(count, max_unique) + while len(runs) < target: + params = {key: normalize_value(rng.choice(SEARCH_SPACE[key])) for key in keys} + signature = tuple(format_value(params[key]) for key in keys) + if signature in seen: + continue + seen.add(signature) + + run = {key: format_value(value) for key, value in params.items()} + run.update(FIXED) + run["wandb_run_name"] = build_run_name(run) + runs.append(run) + + return runs + + +def params_to_cli_args(params: dict[str, str]) -> list[str]: + args: list[str] = [] + for key, value in params.items(): + args.extend([f"--{key.replace('_', '-')}", str(value)]) + return args + + +def print_run(index: int, total: int, params: dict[str, str]) -> None: + cli_args = params_to_cli_args(params) + print(f"[{index}/{total}] {params['wandb_run_name']}") + print(f" flavor: {FLAVOR}") + print(f" timeout: {TIMEOUT}") + print(f" args: {' '.join(shlex.quote(arg) for arg in cli_args)}") + + +def load_secrets() -> dict[str, str]: + load_dotenv(PROJECT_DIR / ".env") + secrets: dict[str, str] = {} + for key in ("HF_TOKEN", "WANDB_API_KEY"): + value = os.environ.get(key) + if not value: + raise SystemExit(f"Missing required secret: {key}") + secrets[key] = value + return secrets + + +# This function uses huggingface_hub.run_uv_job to launch jobs. +# Swap this out if you use a different compute provider. +def launch_run(index: int, total: int, params: dict[str, str], secrets: dict[str, str]) -> None: + print_run(index, total, params) + job = run_uv_job( + str(NOTEBOOK_PATH), + script_args=params_to_cli_args(params), + flavor=FLAVOR, + timeout=TIMEOUT, + env=JOB_ENV, + secrets=secrets, + ) + print(f" launched: {job.url}") + print() + + +def main() -> None: + parser = argparse.ArgumentParser( + description="Launch randomized hyperparameter sweeps for starting-point.py" + ) + parser.add_argument( + "--count", + type=int, + default=12, + help="Number of randomized parameter combinations to sample.", + ) + parser.add_argument( + "--seed", + type=int, + default=0, + help="RNG seed used to sample combinations.", + ) + parser.add_argument( + "--launch", + action="store_true", + help="Actually submit the sampled runs. Without this flag, only a dry run is printed.", + ) + args = parser.parse_args() + + if args.count <= 0: + raise SystemExit("--count must be positive") + + rng = random.Random(args.seed) + runs = sample_runs(args.count, rng) + secrets = load_secrets() if args.launch else None + + print( + f"Randomized grid search with {len(runs)} runs " + f"(requested={args.count}, seed={args.seed})" + ) + print(f"Notebook: {NOTEBOOK_PATH.name}") + print(f"Flavor: {FLAVOR}") + print(f"Timeout: {TIMEOUT}") + print(f"Fixed params: {FIXED}") + print(f"Job env: {JOB_ENV}") + print() + + for index, params in enumerate(runs, start=1): + if args.launch: + launch_run(index, len(runs), params, secrets) + else: + print_run(index, len(runs), params) + print(" (dry run)") + print() + + if not args.launch: + print("Dry run complete. Pass --launch to submit jobs.") + + +if __name__ == "__main__": + main() diff --git a/config/claude/skills/marimo-batch/references/starting-point.py b/config/claude/skills/marimo-batch/references/starting-point.py new file mode 100644 index 0000000..f74ae82 --- /dev/null +++ b/config/claude/skills/marimo-batch/references/starting-point.py @@ -0,0 +1,271 @@ +# /// script +# dependencies = [ +# "marimo", +# "pydantic==2.12.5", +# "python-dotenv==1.2.1", +# "rich==14.3.2", +# "wigglystuff==0.2.30", +# "torch==2.11.0", +# "wandb==0.25.1", +# ] +# requires-python = ">=3.14" +# /// + +import marimo + +__generated_with = "0.21.1" +app = marimo.App(width="columns") + + +@app.cell(column=0, hide_code=True) +def _(mo): + mo.md(r""" + ## Notebook Description + """) + return + + +@app.cell(hide_code=True) +def _(mo): + mo.md(r""" + ## Environment Keys + """) + return + + +@app.cell +def _(): + import marimo as mo + from dotenv import load_dotenv + + load_dotenv(".env") + return (mo,) + + +@app.cell +def _(env_config, is_script_mode, wandb): + env_config if not is_script_mode and not wandb.login() else None + return + + +@app.cell +def _(ModelParams, mo, wandb): + from wigglystuff import EnvConfig + import sys + + is_script_mode = mo.app_meta().mode == "script" + + env_config = mo.ui.anywidget( + EnvConfig( + { + "WANDB_API_KEY": lambda k: wandb.login(key=k, verify=True), + } + ) + ) + + if is_script_mode and not mo.cli_args(): + from rich.console import Console + from rich.table import Table + + table = Table(title="CLI Options") + table.add_column("Flag", style="cyan") + table.add_column("Type", style="green") + table.add_column("Default", style="yellow") + table.add_column("Description") + + for name, field in ModelParams.model_fields.items(): + flag = f"--{name.replace('_', '-')}" + type_name = ( + field.annotation.__name__ + if hasattr(field.annotation, "__name__") + else str(field.annotation) + ) + table.add_row(flag, type_name, str(field.default), field.description or "") + + Console().print(table) + sys.exit(0) + return env_config, is_script_mode + + +@app.cell +def _(): + import wandb + + return (wandb,) + + +@app.cell(column=1, hide_code=True) +def _(mo): + mo.md(r""" + ## Training Parameters + """) + return + + +@app.cell +def _(params_form): + params_form + return + + +@app.cell +def _(): + import hashlib + import json + from pydantic import computed_field, BaseModel, Field + + + class ModelParams(BaseModel): + epochs: int = Field(default=25, description="Number of training epochs.") + batch_size: int = Field(default=32, description="Training batch size.") + learning_rate: float = Field(default=1e-4, description="Learning rate for AdamW.") + wandb_project: str = Field( + default="batch-sizes", description="W&B project name (empty to skip)." + ) + + @computed_field + @property + def run_name(self) -> str: + parts = [ + f"e{self.epochs}", + f"bs{self.batch_size}", + f"lr{self.learning_rate:.0e}", + ] + params_dict = { + "epochs": self.epochs, + "batch_size": self.batch_size, + "learning_rate": self.learning_rate, + } + h = hashlib.md5(json.dumps(params_dict, sort_keys=True).encode()).hexdigest()[:6] + return "-".join(parts) + f"-{h}" + + return (ModelParams,) + + +@app.cell +def _(mo): + params_form = ( + mo.md(""" + ## Model parameters + + {epochs} + {batch_size} + {learning_rate} + """) + .batch( + epochs=mo.ui.slider(10, 50, value=50, step=1, label="epochs"), + batch_size=mo.ui.slider(8, 512, value=32, step=8, label="batch size"), + learning_rate=mo.ui.slider(1e-5, 5e-4, value=1e-4, step=1e-5, label="learning rate"), + ) + .form() + ) + return (params_form,) + + +@app.cell +def _(ModelParams, is_script_mode, mo, params_form): + mo.stop( + not is_script_mode and params_form.value is None, + mo.md("*Submit the form to start training.*"), + ) + + if is_script_mode: + model_params = ModelParams(**{k.replace("-", "_"): v for k, v in mo.cli_args().items()}) + else: + model_params = ModelParams(**params_form.value) + return (model_params,) + + +@app.cell +def _(): + import torch + import torch.nn as nn + + return nn, torch + + +@app.cell(column=2, hide_code=True) +def _(mo): + mo.md(r""" + ## Data Setup + """) + return + + +@app.cell +def _(model_params, torch): + X = torch.randn(1000, 10) + w_true = torch.randn(10, 1) + y = X @ w_true + 0.1 * torch.randn(1000, 1) + + dataset = torch.utils.data.TensorDataset(X, y) + train_loader = torch.utils.data.DataLoader( + dataset, batch_size=model_params.batch_size, shuffle=True + ) + return (train_loader,) + + +@app.cell(column=3, hide_code=True) +def _(mo): + mo.md(r""" + ## Model Setup + """) + return + + +@app.cell +def _(nn): + model = nn.Sequential( + nn.Linear(10, 32), + nn.ReLU(), + nn.Linear(32, 1), + ) + + model + return (model,) + + +@app.cell(column=4, hide_code=True) +def _(mo): + mo.md(r""" + ## Training Loop + """) + return + + +@app.cell +def _(mo, model, model_params, nn, torch, train_loader, wandb): + if model_params.wandb_project: + wandb.init( + project=model_params.wandb_project, + name=model_params.run_name, + config=model_params.model_dump(), + ) + + optimizer = torch.optim.AdamW(model.parameters(), lr=model_params.learning_rate) + loss_fn = nn.MSELoss() + + with mo.status.progress_bar(total=model_params.epochs) as bar: + for epoch in range(model_params.epochs): + epoch_loss = 0.0 + for xb, yb in train_loader: + pred = model(xb) + loss = loss_fn(pred, yb) + optimizer.zero_grad() + loss.backward() + optimizer.step() + epoch_loss += loss.item() + avg_loss = epoch_loss / len(train_loader) + if model_params.wandb_project: + wandb.log({"epoch": epoch, "loss": avg_loss}) + bar.update() + + if model_params.wandb_project: + wandb.finish() + + mo.md(f"**Training complete.** Final loss: `{avg_loss:.6f}`") + return + + +if __name__ == "__main__": + app.run() diff --git a/config/claude/skills/marimo-notebook/SKILL.md b/config/claude/skills/marimo-notebook/SKILL.md new file mode 100644 index 0000000..20bc48f --- /dev/null +++ b/config/claude/skills/marimo-notebook/SKILL.md @@ -0,0 +1,282 @@ +--- +name: marimo-notebook +description: Write a marimo notebook in a Python file in the right format. +--- + +# Notes for marimo Notebooks + +marimo uses Python to create notebooks, unlike Jupyter which uses JSON. Here's an example notebook: + +```python +# /// script +# dependencies = [ +# "marimo", +# "numpy==2.4.3", +# ] +# requires-python = ">=3.14" +# /// + +import marimo + +__generated_with = "0.20.4" +app = marimo.App(width="medium") + + +@app.cell +def _(): + import marimo as mo + import numpy as np + + return mo, np + + +@app.cell +def _(): + print("hello world") + return + + +@app.cell +def _(np, slider): + np.array([1,2,3]) + slider.value + return + + +@app.cell +def _(mo): + slider = mo.ui.slider(1, 10, 1, label="number to add") + slider + return (slider,) + + +@app.cell +def _(): + return + + +if __name__ == "__main__": + app.run() + +``` + +Notice how the notebook is structured with functions can represent cell contents. Each cell is defined with the `@app.cell` decorator and the inputs/outputs of the function are the inputs/outputs of the cell. marimo usually takes care of the dependencies between cells automatically. + +## Running Marimo Notebooks + +```bash +# Run as script (non-interactive, for testing) +uv run + +# Run interactively in browser +uv run marimo run + +# Edit interactively +uv run marimo edit +``` + +## Script Mode Detection + +Use `mo.app_meta().mode == "script"` to detect CLI vs interactive: + +```python +@app.cell +def _(mo): + is_script_mode = mo.app_meta().mode == "script" + return (is_script_mode,) +``` + +## Key Principle: Keep It Simple + +**Show all UI elements always.** Only change the data source in script mode. + +- Sliders, buttons, widgets should always be created and displayed +- In script mode, just use synthetic/default data instead of waiting for user input +- Don't wrap everything in `if not is_script_mode` conditionals +- Don't use try/except for normal control flow + +### Good Pattern + +```python +# Always show the widget +@app.cell +def _(ScatterWidget, mo): + scatter_widget = mo.ui.anywidget(ScatterWidget()) + scatter_widget + return (scatter_widget,) + +# Only change data source based on mode +@app.cell +def _(is_script_mode, make_moons, scatter_widget, np, torch): + if is_script_mode: + # Use synthetic data for testing + X, y = make_moons(n_samples=200, noise=0.2) + X_data = torch.tensor(X, dtype=torch.float32) + y_data = torch.tensor(y) + data_error = None + else: + # Use widget data in interactive mode + X, y = scatter_widget.widget.data_as_X_y + # ... process data ... + return X_data, y_data, data_error + +# Always show sliders - use their .value in both modes +@app.cell +def _(mo): + lr_slider = mo.ui.slider(start=0.001, stop=0.1, value=0.01) + lr_slider + return (lr_slider,) + +# Auto-run in script mode, wait for button in interactive +@app.cell +def _(is_script_mode, train_button, lr_slider, run_training, X_data, y_data): + if is_script_mode: + # Auto-run with slider defaults + results = run_training(X_data, y_data, lr=lr_slider.value) + else: + # Wait for button click + if train_button.value: + results = run_training(X_data, y_data, lr=lr_slider.value) + return (results,) +``` + +## State and Reactivity + +Variables between cells define the reactivity of the notebook for 99% of the use-cases out there. No special state management needed. Don't mutate objects across cells (e.g., `my_list.append()`); create new objects instead. Avoid `mo.state()` unless you need bidirectional UI sync or accumulated callback state. See [STATE.md](references/STATE.md) for details. + +## Don't Guard Cells with `if` Statements + +Marimo's reactivity means cells only run when their dependencies are ready. Don't add unnecessary guards: + +```python +# BAD - the if statement prevents the chart from showing +@app.cell +def _(plt, training_results): + if training_results: # WRONG - don't do this + fig, ax = plt.subplots() + ax.plot(training_results['losses']) + fig + return + +# GOOD - let marimo handle the dependency +@app.cell +def _(plt, training_results): + fig, ax = plt.subplots() + ax.plot(training_results['losses']) + fig + return +``` + +The cell won't run until `training_results` has a value anyway. + +## Don't Use try/except for Control Flow + +Don't wrap code in try/except blocks unless you're handling a specific, expected exception. Let errors surface naturally. + +```python +# BAD - hiding errors behind try/except +@app.cell +def _(scatter_widget, np, torch): + try: + X, y = scatter_widget.widget.data_as_X_y + X = np.array(X, dtype=np.float32) + # ... + except Exception as e: + return None, None, f"Error: {e}" + +# GOOD - let it fail if something is wrong +@app.cell +def _(scatter_widget, np, torch): + X, y = scatter_widget.widget.data_as_X_y + X = np.array(X, dtype=np.float32) + # ... +``` + +Only use try/except when: +- You're handling a specific, known exception type +- The exception is expected in normal operation (e.g., file not found) +- You have a meaningful recovery action + +## Cell Output Rendering + +Marimo only renders the **final expression** of a cell. Indented or conditional expressions won't render: + +```python +# BAD - indented expression won't render +@app.cell +def _(mo, condition): + if condition: + mo.md("This won't show!") # WRONG - indented + return + +# GOOD - final expression renders +@app.cell +def _(mo, condition): + result = mo.md("Shown!") if condition else mo.md("Also shown!") + result # This renders because it's the final expression + return +``` + +## PEP 723 Dependencies + +Notebooks created via `marimo edit --sandbox` have these dependencies added to the top of the file automatically but it is a good practice to make sure these exist when creating a notebook too: + +```python +# /// script +# requires-python = ">=3.12" +# dependencies = [ +# "marimo", +# "torch>=2.0.0", +# ] +# /// +``` + +## marimo check + +When working on a notebook it is important to check if the notebook can run. That's why marimo provides a `check` command that acts as a linter to find common mistakes. + +```bash +uvx marimo check +``` + +Make sure these are checked before handing a notebook back to the user. + +**Important**: you have a tendency to over-do variables with an underscore prefix. You should only apply this to one or two variables at most. Consider creating a new variable instead of prefixing entire cells in marimo. + +## api docs + +If the user specifically wants you to use a marimo function, you can locally check the docs via: + +``` +uv --with marimo run python -c "import marimo as mo; help(mo.ui.form)" +``` + +## tests + +By default, marimo discovers and executes tests inside your notebook. +When the optional `pytest` dependency is present, marimo runs `pytest` on cells that +consist exclusively of test code - i.e. functions whose names start with `test_`. +If the user asks you to add tests, make sure to add the `pytest` dependency is added and that +there is a cell that contains only test code. + +For more information on testing with pytest see [PYTEST.md](references/PYTEST.md) + +Once tests are added, you can run pytest from the commandline on the notebook to run pytest. + +``` +pytest +``` + +## Additional resources + +- For marimo notebooks that run in width=columns [SQL.md](references/COLUMNS.md) +- For SQL use in marimo see [SQL.md](references/SQL.md) +- For UI elements in marimo [UI.md](references/UI.md) +- For exposing functions/classes as top level imports [TOP-LEVEL-IMPORTS.md](references/TOP-LEVEL-IMPORTS.md) +- For exporting notebooks (PDF, HTML, markdown, etc.) [EXPORTS.md](references/EXPORTS.md) +- For state management and reactivity [STATE.md](references/STATE.md) +- For deployment of marimo notebooks [DEPLOYMENT.md](references/DEPLOYMENT.md) +- For custom interactive widgets with anywidget [ANYWIDGET.md](references/ANYWIDGET.md) +- For external editing and `--watch` mode [WATCHING.md](references/WATCHING.md) +- For expensive notebooks (caching, lazy eval, mo.stop) [EXPENSIVE.md](references/EXPENSIVE.md) +- For configuration (pyproject.toml, marimo.toml) [CONFIGURATION.md](references/CONFIGURATION.md) +- For reactivity model (DAG, variable scoping, mutations) [REACTIVITY.md](references/REACTIVITY.md) diff --git a/config/claude/skills/marimo-notebook/references/ANYWIDGET.md b/config/claude/skills/marimo-notebook/references/ANYWIDGET.md new file mode 100644 index 0000000..4dfcd4e --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/ANYWIDGET.md @@ -0,0 +1,81 @@ +--- +name: anywidget-generator +description: Generate anywidget components for marimo notebooks. +--- + +When writing an anywidget use vanilla javascript in `_esm` and do not forget about `_css`. The css should look bespoke in light mode and dark mode. Keep the css small unless explicitly asked to go the extra mile. When you display the widget it must be wrapped via `widget = mo.ui.anywidget(OriginalAnywidget())`. You can also point `_esm` and `_css` to external files if needed using pathlib. This makes sense if the widget does a lot of elaborate JavaScript or CSS. + + +import anywidget +import traitlets + + +class CounterWidget(anywidget.AnyWidget): + _esm = """ + // Define the main render function + function render({ model, el }) { + let count = () => model.get("number"); + let btn = document.createElement("b8utton"); + btn.innerHTML = `count is ${count()}`; + btn.addEventListener("click", () => { + model.set("number", count() + 1); + model.save_changes(); + }); + model.on("change:number", () => { + btn.innerHTML = `count is ${count()}`; + }); + el.appendChild(btn); + } + // Important! We must export at the bottom here! + export default { render }; + """ + _css = """button{ + font-size: 14px; + }""" + number = traitlets.Int(0).tag(sync=True) + +widget = mo.ui.anywidget(CounterWidget()) +widget + +# Grabbing the widget from another cell, `.value` is a dictionary. +print(widget.value["number"]) + + +The above is a minimal example that could work for a simple counter widget. In general the widget can become much larger because of all the JavaScript and CSS required. Unless the widget is dead simple, you should consider using external files for `_esm` and `_css` using pathlib. + +When sharing the anywidget, keep the example minimal. No need to combine it with marimo ui elements unless explicitly stated to do so. + +## Best Practices + +Unless specifically told otherwise, assume the following: + +1. **Use vanilla JavaScript in `_esm`**: + - Define a `render` function that takes `{ model, el }` as parameters + - Use `model.get()` to read trait values + - Use `model.set()` and `model.save_changes()` to update traits + - Listen to changes with `model.on("change:traitname", callback)` + - Export default with `export default { render };` at the bottom + - All widgets inherit from `anywidget.AnyWidget`, so `widget.observe(handler)` + remains the standard way to react to state changes. + - Python constructors tend to validate bounds, lengths, or choice counts; let the + raised `ValueError/TraitError` guide you instead of duplicating the logic. + +2. **Include `_css` styling**: + - Keep CSS minimal unless explicitly asked for more + - Make it look bespoke in both light and dark mode + - Use CSS media query for dark mode: `@media (prefers-color-scheme: dark) { ... }` + +3. **Wrap the widget for display**: + - Always wrap with marimo: `widget = mo.ui.anywidget(OriginalAnywidget())` + - Access values via `widget.value` which returns a dictionary + +4. **Keep examples minimal**: + - Add a marimo notebook that highlights the core utility + - Show basic usage only + - Don't combine with other marimo UI elements unless explicitly requested + +5. **External file paths**: When using pathlib for external `_esm`/`_css` files, keep paths relative to the project directory, consider using `Path(__file__)` for this. Do not read files outside the project (e.g., `~/.ssh`, `~/.env`, `/etc/`) or embed their contents in widget output. + +Dumber is better. Prefer obvious, direct code over clever abstractions—someone +new to the project should be able to read the code top-to-bottom and grok it +without needing to look up framework magic or trace through indirection. diff --git a/config/claude/skills/marimo-notebook/references/COLUMNS.md b/config/claude/skills/marimo-notebook/references/COLUMNS.md new file mode 100644 index 0000000..15d5652 --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/COLUMNS.md @@ -0,0 +1,58 @@ +A user may specify that they want to have a notebook with multiple columns. Below is an example of a notebook that does just that. + +```python +# /// script +# dependencies = ["marimo"] +# requires-python = ">=3.14" +# /// + +import marimo + +__generated_with = "0.23.9" +app = marimo.App(width="columns") + + +@app.cell(column=0, hide_code=True) +def _(mo): + mo.md(r""" + ## Column 1: Cool stuff + + This is where the user will first look. Put plots/inputs here typically. + """) + return + + +@app.cell +def _(): + # This cell is in column 1 + return + + +@app.cell +def _(mo): + # This cell is in column 1 as well + mo.ui.slider(1, 10, 1) + return + + +@app.cell(column=1, hide_code=True) +def _(mo): + mo.md(r""" + ## Column 2: Boilerplate + """) + return + + +@app.cell +def _(): + # This cell is in column 2 + import marimo as mo + + return (mo,) + + +if __name__ == "__main__": + app.run() +``` + +Notice the `@app.cell(column=0)` decorator? Every cell that follows sits in that column. Then, when we see `@app.cell(column=1)` the cells no longer fall into column 0 but they go into column 1. diff --git a/config/claude/skills/marimo-notebook/references/CONFIGURATION.md b/config/claude/skills/marimo-notebook/references/CONFIGURATION.md new file mode 100644 index 0000000..5b92613 --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/CONFIGURATION.md @@ -0,0 +1,37 @@ +# Configuration + +## Two Scopes + +1. **App config** — per-notebook, stored in the `.py` file header. Configure via the gear icon (top-right): notebook width, title, custom CSS, custom HTML head. +2. **User config** — global, typically stored in `~/.config/marimo/marimo.toml`. Runtime, display, hotkeys, autosave, formatting, server settings. + +## Priority (Highest → Lowest) + +1. PEP 723 script metadata block in the notebook file +2. `pyproject.toml` — project-level overrides +3. User config (`marimo.toml`) — global defaults + +## pyproject.toml + +```toml +[tool.marimo.formatting] +line_length = 120 + +[tool.marimo.display] +default_width = "full" + +[tool.marimo.runtime] +default_sql_output = "native" +watcher_on_save = "autorun" +``` + +## Config Discovery + +marimo searches for `.marimo.toml` in: current directory → parent directories → home directory → XDG config directory. + +## Useful Commands + +```bash +marimo config show # view current config and file location +marimo config describe # list all available config options +``` diff --git a/config/claude/skills/marimo-notebook/references/DEPLOYMENT.md b/config/claude/skills/marimo-notebook/references/DEPLOYMENT.md new file mode 100644 index 0000000..ab0b786 --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/DEPLOYMENT.md @@ -0,0 +1,41 @@ +## Running notebooks + +You can deploy a single marimo notebook as a web app: + +```bash +uvx marimo run --sandbox notebook.py +``` + +The `--sandbox` flag makes sure the notebook runs in an isolated UV environment. + +Or deploy a folder of notebooks as a web app with multiple notebooks. Also here, you can use the `--sandbox` flag to run each notebook in its own isolated environment, using the PEP 723 dependencies declared in each notebook: + +```bash +uvx marimo run --sandbox +``` + +### Thumbnails + +When you host multiple notebooks you may want to generate thumbnails. You can generate OpenGraph thumbnails for notebooks using: + +```bash +uvx marimo export thumbnail notebook.py +uvx marimo export thumbnail folder/ +``` + +Thumbnails are stored at `__marimo__/assets//opengraph.png`. The user may also put screenshots there manually. + +Besides images, you can also add metadata to the notebooks by adding to the PEP 723 Dependencies on top of the file. These will appear in an overview if the user deploys a folder of notebooks. + +``` +# /// script +# requires-python = ">=3.12" +# dependencies = [ +# "marimo", +# "polars==1.37.1", +# ] +# [tool.marimo.opengraph] +# title = "My dashboard" +# description = "Tracking my portfolio over time" +# /// +``` diff --git a/config/claude/skills/marimo-notebook/references/EXPENSIVE.md b/config/claude/skills/marimo-notebook/references/EXPENSIVE.md new file mode 100644 index 0000000..6c56086 --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/EXPENSIVE.md @@ -0,0 +1,88 @@ +# Expensive Notebooks + +## mo.stop() + +Halt cell execution when a condition is met: + +```python +@app.cell +def _(mo, data): + mo.stop(data is None, mo.md("Waiting for data...")) + # Only runs if data is not None + result = process(data) + return (result,) +``` + +Pair with `mo.ui.run_button()` for manual triggers: + +```python +@app.cell +def _(mo): + run_btn = mo.ui.run_button(label="Run analysis") + run_btn + return (run_btn,) + +@app.cell +def _(mo, run_btn, data): + mo.stop(not run_btn.value) + result = expensive_analysis(data) + return (result,) +``` + +## mo.cache + +In-memory cache for the current session. Results are reused when inputs match: + +```python +@mo.cache +def fetch_data(query: str): + return db.execute(query) +``` + +Works as a decorator or context manager. + +## mo.persistent_cache + +Disk-based cache that persists across notebook restarts: + +```python +@mo.persistent_cache +def train_model(params): + return heavy_training(params) +``` + +## mo.lazy() + +Defer rendering and computation until needed: + +```python +# Only render table when it scrolls into view +mo.lazy(mo.ui.table(large_df)) + +# Only compute when tab is selected +mo.ui.tabs({ + "Summary": summary, + "Details": mo.lazy(lambda: expensive_query()), +}) +``` + +## Runtime Configuration + +- Disable autorun on cell changes for long-running notebooks +- Disable startup autorun to prevent automatic execution on open +- Disable individual cells temporarily during editing + +## Memory Management + +Wrap intermediate computations in functions so local variables get freed: + +```python +@app.cell +def _(): + def _compute(): + large_data = load_everything() + result = summarize(large_data) + return result # large_data is freed here + output = _compute() + return (output,) +``` diff --git a/config/claude/skills/marimo-notebook/references/EXPORTS.md b/config/claude/skills/marimo-notebook/references/EXPORTS.md new file mode 100644 index 0000000..df29a42 --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/EXPORTS.md @@ -0,0 +1,66 @@ +marimo can export notebooks to several formats via the CLI. + +``` +> uvx marimo export --help +Usage: marimo export [OPTIONS] + COMMAND [ARGS]... + + Export a notebook to various formats. + +Options: + -h, --help Show this message and exit. + +Commands: + html Run a notebook and export it as an HTML file. + html-wasm Export a notebook as a WASM- powered marimo notebook. + ipynb Export a marimo notebook as a Jupyter notebook + md Export a marimo notebook as a code fenced markdown file + pdf Export a marimo notebook as a PDF file. + script Export a marimo notebook as a flat script + session Execute a notebook or directory of notebooks and export session snapshots. + thumbnail Generate OpenGraph thumbnails for notebooks. +``` + +You can learn more about each option by calling the command with the `--help` flag. + +## PDF Export + +Many people may be interested in exporting to a PDF. + +```bash +uvx marimo export pdf notebook.py -o notebook.pdf +``` + +PDF export uses `nbformat` and `nbconvert` under the hood. By default it uses the WebPDF exporter which requires Chromium. Install the dependencies: + +```bash +uv pip install nbformat nbconvert +playwright install chromium +``` + +Useful flags: + +- `--no-include-inputs` — hide code cells, show only outputs +- `--no-include-outputs` — include only code, skip outputs +- `--as=slides` — export as a slide deck PDF (uses reveal.js slide boundaries) +- `--raster-scale 4.0` — controls output sharpness (1.0–4.0, default 4.0) +- `--raster-server=live` — use when a widget needs a running Python kernel to render (recommended for slides) + +## Script Export + +```bash +uvx marimo export script notebook.py -o notebook.script.py +``` + +Flattens the notebook into a plain Python script in topological order. + +## Common Flags + +These flags work across most export subcommands: + +- `-o`, `--output` — output file path +- `--watch` — re-export automatically when the notebook file changes +- `--sandbox` — run in an isolated `uv` environment +- `-f`, `--force` — overwrite if output file already exists +- `--` — pass CLI arguments to the notebook, e.g. `uvx marimo export html notebook.py -o out.html -- --arg value` +- `-y` automatic yes to prompts on the terminal `uvx marimo -y CMD ...` diff --git a/config/claude/skills/marimo-notebook/references/PYTEST.md b/config/claude/skills/marimo-notebook/references/PYTEST.md new file mode 100644 index 0000000..2c70992 --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/PYTEST.md @@ -0,0 +1,212 @@ +# Testing with pytest + +## Testing in notebook + +When `pytest` is present, marimo runs `pytest` on cells that +consist exclusively of test code - i.e. functions whose names start with `test_`, +classes whose names start with `Test`, or functions decorated with `@pytest.fixture`. +If a cell mixes in anything else (helper functions, constants, variables, imports, etc.), +that cell is skipped by the test runner (we recommend you move helpers to another cell). + +For example: + +```python +@app.cell +def __(): + import pytest + def inc(x): + return x + 1 + return inc, pytest + +@app.cell +def __(inc, pytest): + class TestBlock: + @staticmethod + def test_fails(): + assert inc(3) == 5, "This test fails" + + @staticmethod + def test_sanity(): + assert inc(3) == 4, "This test passes" + + @pytest.mark.parametrize(("x", "y"), [(3, 4), (4, 5)]) + def test_parameterized(x, y): + assert inc(x) == y + return +``` + +Reactive tests can be disabled. You can disable this behavior with the `runtime.reactive_test` option in the +configuration file. + +## Testing at the command-line + +```bash +pytest +``` + +runs and tests all notebook cells whose names start with `test_`, or cells that +contain only `test_` functions and `Test` classes (just like in notebook tests). + +## Example + +Running `pytest` on + +```python +# content of test_notebook.py +import marimo + +__generated_with = "0.10.6" +app = marimo.App() + + +@app.cell +def _(): + def inc(x): + return x + 1 + return (inc,) + + +@app.cell +def test_fails(inc): + assert inc(3) == 5, "This test fails" + + +@app.cell +def test_sanity(inc): + assert inc(3) == 4, "This test passes" + +@app.cell +def collection_of_tests(inc, pytest): + @pytest.mark.parametrize(("x", "y"), [(3, 4), (4, 5)]) + def test_answer(x, y): + assert inc(x) == y, "These tests should pass." + +@app.cell +def imports(): + import pytest + return pytest +``` + +prints + +```pytest +============================= test session starts ============================== +platform linux -- Python 3.12.9, pytest-8.3.5, pluggy-1.5.0 +rootdir: /notebooks +configfile: pyproject.toml +collected 4 items + +test_notebook.py::test_fails FAILED [ 25%] +test_notebook.py::test_sanity PASSED [ 50%] +test_notebook.py::MarimoTestBlock_0::test_parameterized[3-4] PASSED [ 75%] +test_notebook.py::MarimoTestBlock_0::test_parameterized[4-5] PASSED [100%] + +=================================== FAILURES =================================== +__________________________________ test_fails __________________________________ + + # content of test_notebook.py + import marimo + + __generated_with = "0.10.6" + app = marimo.App() + + + @app.cell + def _(): + def inc(x): + return x + 1 + return (inc,) + + + @app.cell + def test_fails(inc): +> assert inc(3) == 5, "This test fails" +E AssertionError: This test fails +E assert 4 == 5 +E + where 4 = (3) + +test_notebook.py:17: AssertionError +=========================== short test summary info ============================ +FAILED test_notebook.py::test_fails - AssertionError: This test fails +========================= 1 failed, 3 passed in 0.82s ========================== +``` + +## Using Pytest Fixtures + + +marimo supports pytest fixtures, with one limitation: fixtures defined in one cell cannot be used in another cell, unless the fixtures were defined in the setup cell. + +**Fixtures defined in the setup cell**: + +```python +# test_notebook.py +import marimo +app = marimo.App() + +with app.setup: + from fixtures import db_connection, sample_data + +@app.cell +def _(sample_data): + def test_data_loaded(sample_data): + assert len(sample_data) > 0 +``` + +**Fixtures in the same cell as tests**: + +```python +@app.cell +def _(): + import pytest + return pytest + + +@app.cell +def _(pytest): + @pytest.fixture + def temp_file(): + import tempfile + with tempfile.NamedTemporaryFile() as f: + yield f + + def test_writes_to_file(temp_file): + temp_file.write(b"hello") + temp_file.seek(0) + assert temp_file.read() == b"hello" +``` + +**Class fixtures**: + +```python +@app.cell +def _(): + import pytest + return pytest + + +@app.cell +def _(pytest): + class TestDatabase: + @pytest.fixture(scope="class") + def connection(self): + return create_connection() + + def test_query(self, connection): + result = connection.query("SELECT 1") + assert result == 1 +``` + +**`conftest.py` fixtures** work as expected - pytest discovers them automatically. + +### Fixture Limitations + +Fixtures defined in one cell **cannot** be used by tests in a different cell. +This is because pytest collects tests **statically** by parsing the notebook +file without executing it. During collection, pytest can see module-level +fixtures (from `conftest.py` or imported modules) and fixtures defined in the +same scope as the test, but it cannot see fixtures defined in other cells. + +**Why?** Running the entire notebook just for fixture discovery would be +expensive, and static analysis cannot determine which fixtures will be +available after cell execution since cell order is determined at runtime by +marimo's dependency graph. \ No newline at end of file diff --git a/config/claude/skills/marimo-notebook/references/REACTIVITY.md b/config/claude/skills/marimo-notebook/references/REACTIVITY.md new file mode 100644 index 0000000..88ac57f --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/REACTIVITY.md @@ -0,0 +1,68 @@ +# Reactivity + +## The DAG + +marimo statically analyzes each cell to build a directed acyclic graph: + +- **References** = global variables the cell reads (function parameters) +- **Definitions** = global variables the cell creates (return tuple) + +When a cell runs, all cells that reference its definitions automatically run. Execution order is determined by the DAG, not cell position on the page. + +## Variable Uniqueness + +Every global variable must be defined by exactly one cell. This prevents ambiguity in the dependency graph. + +If you need the same name in multiple cells, use underscore-prefixed cell-local variables: + +```python +@app.cell +def _(): + _temp = expensive_computation() + result_a = summarize(_temp) + return (result_a,) + +@app.cell +def _(): + _temp = different_computation() # no conflict, _temp is cell-local + result_b = summarize(_temp) + return (result_b,) +``` + +## Mutations Are Not Tracked + +marimo does **not** detect mutations like `.append()`, attribute assignment, or in-place DataFrame operations across cells. + +```python +# BAD — mutation in another cell, marimo won't re-run dependents +# Cell 1 +items = [1, 2, 3] +# Cell 2 +items.append(4) # invisible to the DAG + +# GOOD — create a new variable +# Cell 2 +extended = items + [4] +``` + +Mutations within the same cell that defines the variable are fine: + +```python +@app.cell +def _(pd): + df = pd.DataFrame({"a": [1, 2]}) + df["b"] = [3, 4] # same cell, fine + return (df,) +``` + +## Deleting Cells + +Deleting a cell removes its global variables from memory. Cells that referenced those variables become invalidated. + +## Disabling Cells + +Disable a cell to prevent it and its dependents from running. Re-enabling triggers a re-run if upstream cells changed while it was disabled. + +## Lazy Evaluation + +Instead of auto-running dependents, mark them stale for manual execution. Configure in runtime settings or use `mo.lazy()` for specific elements. diff --git a/config/claude/skills/marimo-notebook/references/SQL.md b/config/claude/skills/marimo-notebook/references/SQL.md new file mode 100644 index 0000000..7a64775 --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/SQL.md @@ -0,0 +1,65 @@ +There are multiple ways to use SQL in marimo. Under the hood, a SQL cell is just a function call to `marimo.sql`. A cell looks like this: + +```python +@app.cell(hide_code=True) +def _(df, mo): + grouped = mo.sql( + f""" + SELECT category, AVG(value) as mean FROM df GROUP BY category ORDER BY mean; + """, + output=False + ) + return (grouped,) +``` + +`grouped` is a polars dataframe. By defauly marimo uses DuckDB in memory and can refer to dataframe variables that are in scope. + +This is what the signature is of `mo.sql`: + +```python +def sql(query: str, *, output: bool=True, engine: Optional[DBAPIConnection]=None) -> Any +``` + +Typically a `sql` call returns a polars dataframe, but the user can configure pandas as an alternative. + +Notice how a query string goes in with SQL and how you can pass a specific database engine. Be aware that different SQL engines may have different SQL dialects. + +## SQLAlchemy + +One possible engine is SQLAlchemy. + +```python +import sqlalchemy + +# Create an in-memory SQLite database with SQLAlchemy +sqlite_engine = sqlalchemy.create_engine("sqlite:///:memory:") +``` + +You can also use `SQLModel` with a similar connection string. + +## DuckDB + +You can also use DuckDB with a connection string. + +```python +import duckdb + +# Create a DuckDB connection +duckdb_conn = duckdb.connect("file.db", read_only=True) +``` + +## PyIceberg + +marimo supports data catalogs as well. + +```python +from pyiceberg.catalog.rest import RestCatalog + +catalog = RestCatalog( + name="catalog", + warehouse="1234567890", + uri="https://example.com", + token="my-token", +) +``` + diff --git a/config/claude/skills/marimo-notebook/references/STATE.md b/config/claude/skills/marimo-notebook/references/STATE.md new file mode 100644 index 0000000..82994f6 --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/STATE.md @@ -0,0 +1,94 @@ +# State in marimo + +## Reactivity IS State Management + +In marimo, regular Python variables between cells are your state. When a cell assigns a variable, all cells that read it re-run automatically. Widget values (`widget.value`) work the same way — interact with a widget and dependent cells re-execute. No store, no session_state, no hooks needed. + +## Don't Mutate Objects Across Cells + +marimo does **not** track mutations like `my_list.append(42)` or `obj.value = 42`. + +```python +# BAD - mutation in another cell won't trigger re-runs +# Cell 1 +items = [1, 2, 3] + +# Cell 2 +items.append(4) # marimo won't know this happened + +# GOOD - create new objects instead +# Cell 1 +items = [1, 2, 3] + +# Cell 2 +extended_items = items + [4] +``` + +## You Probably Don't Need `mo.state()` + +In 99% of cases, built-in reactivity is enough: + +- **Reading widget values** — just use `widget.value` in another cell +- **Combining multiple inputs** — use `.batch().form()` +- **Conditional data** — use `if`/`else` in one cell + +## When You Do Need `mo.state()` + +Use it when you need **accumulated state from callbacks** or **bidirectional sync** between UI elements. + +```python +get_val, set_val = mo.state(initial_value) +``` + +- Read: `get_val()` +- Update: `set_val(new_value)` or `set_val(lambda d: d + [new_item])` +- The cell calling the setter does NOT re-run (unless `allow_self_loops=True`) + +### Example: todo list with accumulated state + +```python +# Cell 1 — declare state +@app.cell +def _(mo): + get_items, set_items = mo.state([]) + return get_items, set_items + +# Cell 2 — input form +@app.cell +def _(mo, set_items): + task = mo.ui.text(label="New task") + add = mo.ui.button( + label="Add", + on_click=lambda _: set_items(lambda d: d + [task.value]) + ) + mo.hstack([task, add]) + return + +# Cell 3 — display (re-runs when state changes) +@app.cell +def _(mo, get_items): + mo.md("\n".join(f"- {t}" for t in get_items())) + return +``` + +### Example: syncing two UI elements + +```python +@app.cell +def _(mo): + get_n, set_n = mo.state(50) + return get_n, set_n + +@app.cell +def _(mo, get_n, set_n): + slider = mo.ui.slider(0, 100, value=get_n(), on_change=set_n) + number = mo.ui.number(0, 100, value=get_n(), on_change=set_n) + mo.hstack([slider, number]) + return +``` + +## Warnings + +- Don't store `mo.ui` elements inside state — causes hard-to-diagnose bugs. +- Don't use `on_change` when you can just read `.value` from another cell. +- Write idempotent cells — same inputs should produce same outputs. diff --git a/config/claude/skills/marimo-notebook/references/TOP-LEVEL-IMPORTS.md b/config/claude/skills/marimo-notebook/references/TOP-LEVEL-IMPORTS.md new file mode 100644 index 0000000..0eaebae --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/TOP-LEVEL-IMPORTS.md @@ -0,0 +1,57 @@ +You can import top-level functions and classes defined in a marimo notebook into other Python scripts or notebooks using normal Python syntax, as long as your definitions satisfy the simple criteria described on this page. This makes your notebook code reusable, testable, and easier to edit in text editors of your choice. + +For a function or class to be saved at the top level of the notebook file, it must meet the following criteria: + +The cell must define just a single function or class. +The defined function or class can only refer to symbols defined in the setup cell, or to other top-level symbols. + +```python +# /// script +# dependencies = [ +# "marimo", +# "numpy==2.4.2", +# ] +# requires-python = ">=3.14" +# /// + +import marimo + +__generated_with = "0.19.11" +app = marimo.App(width="medium") + +# Define setup cell +with app.setup: + import numpy as np + + +# Define function cell +@app.function +def calculate_statistics(data): + """Calculate basic statistics for a dataset""" + return { + "mean": np.mean(data), + "median": np.median(data), + "std": np.std(data) + } + + +@app.cell +def _(): + import marimo as mo + + return + +if __name__ == "__main__": + app.run() +``` + +In this example, the setup cell is represented as a context manager `app.setup` and the cell that contains `calculate_statistics` is represented as a function decorator `@app.function`. You can now import `calculate_statistics` from other Python scripts or notebooks. There can be no more than one setup cell per notebook. + +```python +# In another_script.py +from my_notebook import calculate_statistics + +data = [1, 2, 3, 4, 5] +stats = calculate_statistics(data) +print(stats) +``` \ No newline at end of file diff --git a/config/claude/skills/marimo-notebook/references/UI.md b/config/claude/skills/marimo-notebook/references/UI.md new file mode 100644 index 0000000..b2e6ea1 --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/UI.md @@ -0,0 +1,92 @@ +marimo has a rich set of UI components. + +* `mo.ui.altair_chart(altair_chart)` - create a reactive Altair chart +* `mo.ui.button(value=None, kind='primary')` - create a clickable button +* `mo.ui.run_button(label=None, tooltip=None, kind='primary')` - create a button that runs code +* `mo.ui.checkbox(label='', value=False)` - create a checkbox +* `mo.ui.chat(placeholder='', value=None)` - create a chat interface +* `mo.ui.date(value=None, label=None, full_width=False)` - create a date picker +* `mo.ui.dropdown(options, value=None, label=None, full_width=False)` - create a dropdown menu +* `mo.ui.file(label='', multiple=False, full_width=False)` - create a file upload element +* `mo.ui.number(value=None, label=None, full_width=False)` - create a number input +* `mo.ui.radio(options, value=None, label=None, full_width=False)` - create radio buttons +* `mo.ui.refresh(options: List[str], default_interval: str)` - create a refresh control +* `mo.ui.slider(start, stop, value=None, label=None, full_width=False, step=None)` - create a slider +* `mo.ui.range_slider(start, stop, value=None, label=None, full_width=False, step=None)` - create a range slider +* `mo.ui.table(data, columns=None, on_select=None, sortable=True, filterable=True)` - create an interactive table +* `mo.ui.text(value='', label=None, full_width=False)` - create a text input +* `mo.ui.text_area(value='', label=None, full_width=False)` - create a multi-line text input +* `mo.ui.data_explorer(df)` - create an interactive dataframe explorer +* `mo.ui.dataframe(df)` - display a dataframe with search, filter, and sort capabilities +* `mo.ui.plotly(plotly_figure)` - create a reactive Plotly chart (supports scatter, treemap, and sunburst) +* `mo.ui.tabs(elements: dict[str, mo.ui.Element])` - create a tabbed interface from a dictionary +* `mo.ui.array(elements: list[mo.ui.Element])` - create an array of UI elements +* `mo.ui.form(element: mo.ui.Element, label='', bordered=True)` - wrap an element in a form + +As always, you can learn more about the available inputs to all these components via `uv --with marimo run python -c "import marimo as mo; help(mo.ui.form)"` + +## Forms + +You can compose multiple UI elements into a single form using `.batch().form()`. The `.batch()` method binds named UI elements into a markdown template, and `.form()` adds a submit button so values are only sent on submit. + +```python +form = ( + mo.md( + """ + **Choose an option** + + {choice} + + **Enter some text** + + {text} + + **Enable feature** + + {flag} + """ + ) + .batch( + choice=mo.ui.dropdown(options=["A", "B", "C"]), + text=mo.ui.text(), + flag=mo.ui.checkbox(), + ) + .form( + submit_button_label="Submit", + show_clear_button=True, # optional + clear_on_submit=False, # keep values after submit + ) +) + +form +``` + +You can also add validation to a form using the `validate` parameter. Return an error string to block submission, or `None` to allow it. + +```python +group_by_form = mo.ui.dropdown( + options=df_columns, + label="Select column to filter for duplicate analyzis", + allow_select_none=True, + value=None, # start with nothing selected + searchable=True, +).form( + submit_button_label="Apply", + validate=lambda v: ( + "Please select a column and press Apply." + if v is None else None + ), +) +``` + +However, the user may also want to use other components. Popular alternatives include the `ScatterWidget` from the `drawdata` library, `moutils`, and `wigglystuff`. + +For custom classes and static HTML representations you can also use the `_display_` method. + +```python +class Dice: + def _display_(self): + import random + + return f"You rolled {random.randint(0, 7)}" +``` diff --git a/config/claude/skills/marimo-notebook/references/WATCHING.md b/config/claude/skills/marimo-notebook/references/WATCHING.md new file mode 100644 index 0000000..5da008b --- /dev/null +++ b/config/claude/skills/marimo-notebook/references/WATCHING.md @@ -0,0 +1,55 @@ +# External Editing and Watch Mode + +## The Problem + +marimo loads the notebook file into memory at startup. After that, it works from its in-memory state and does **not** watch the file for external changes. If you edit the `.py` file externally (vim, VSCode, another agent), marimo won't see it. When any cell is saved in the marimo UI, it writes its in-memory version back to disk, **overwriting your external edits**. + +## Solution: --watch + +```bash +marimo edit --watch notebook.py +``` + +This monitors the file for changes and streams them to the browser editor. By default, synced code appears as "stale" — the user manually runs cells via the "Run" button or the `runStale` hotkey. + +For apps: + +```bash +marimo run --watch notebook.py +``` + +This auto-refreshes when file changes are detected. + +## Auto-Execute After External Edits + +Add to `pyproject.toml` so affected cells run automatically when the file changes: + +```toml +[tool.marimo.runtime] +watcher_on_save = "autorun" +``` + +## Install watchdog for Performance + +Without `watchdog`, marimo falls back to polling: + +```bash +pip install watchdog +``` + +## Module Autoreloading + +Watch imported `.py` modules for changes (not just the notebook file): + +1. Enable in notebook settings → Runtime → Module Autoreloading +2. Two modes: + - **Autorun**: automatically executes cells affected by module changes + - **Lazy**: marks affected cells as stale for manual execution + +The reloader tracks changes recursively through the import chain. + +Use case: develop logic in Python modules, use the notebook as an orchestrating DAG. + +## Responding to other files + +marimo has `mo.watch.file` and `mo.watch.file` utilities that can cause cells to update when a file/folder updates. diff --git a/config/claude/skills/streamlit-to-marimo/SKILL.md b/config/claude/skills/streamlit-to-marimo/SKILL.md new file mode 100644 index 0000000..a12f191 --- /dev/null +++ b/config/claude/skills/streamlit-to-marimo/SKILL.md @@ -0,0 +1,124 @@ +--- +name: streamlit-to-marimo +description: Convert a Streamlit app to a marimo notebook +--- + +# Converting Streamlit Apps to Marimo + +For general marimo notebook conventions (cell structure, PEP 723 metadata, output rendering, `marimo check`, variable naming, etc.), refer to the `marimo-notebook` skill. This skill focuses specifically on **mapping Streamlit concepts to marimo equivalents**. + +## Steps + +1. **Read the Streamlit app** to understand its widgets, layout, and state management. + +2. **Create a new marimo notebook** following the `marimo-notebook` skill conventions. Add all dependencies the Streamlit app uses (pandas, plotly, altair, etc.) — but replace `streamlit` with `marimo`. You should not overwrite the original file. + +3. **Map Streamlit components to marimo equivalents** using the reference tables below. Key principles: + - UI elements are **assigned to variables** and their current value is accessed via `.value`. + - Cells that reference a UI element automatically re-run when the user interacts with it — no callbacks needed. + +4. **Handle conceptual differences** in execution model, state, and caching (see below). + +5. **Run `uvx marimo check`** on the result and fix any issues. + +## Widget Mapping Reference + +### Input Widgets + +| Streamlit | marimo | Notes | +|-----------|--------|-------| +| `st.slider()` | `mo.ui.slider()` | | +| `st.select_slider()` | `mo.ui.slider(steps=[...])` | Pass discrete values via `steps` | +| `st.text_input()` | `mo.ui.text()` | | +| `st.text_area()` | `mo.ui.text_area()` | | +| `st.number_input()` | `mo.ui.number()` | | +| `st.checkbox()` | `mo.ui.checkbox()` | | +| `st.toggle()` | `mo.ui.switch()` | | +| `st.radio()` | `mo.ui.radio()` | | +| `st.selectbox()` | `mo.ui.dropdown()` | | +| `st.multiselect()` | `mo.ui.multiselect()` | | +| `st.date_input()` | `mo.ui.date()` | | +| `st.time_input()` | `mo.ui.text()` | No dedicated time widget | +| `st.file_uploader()` | `mo.ui.file()` | Use `.contents()` to read bytes | +| `st.color_picker()` | `mo.ui.text(value="#000000")` | No dedicated color picker | +| `st.button()` | `mo.ui.button()` or `mo.ui.run_button()` | Use `run_button` for triggering expensive computations | +| `st.download_button()` | `mo.download()` | Returns a download link element | +| `st.form()` + `st.form_submit_button()` | `mo.ui.form(element)` | Wraps any element so its value only updates on submit | + +### Display Elements + +| Streamlit | marimo | Notes | +|-----------|--------|-------| +| `st.write()` | `mo.md()` or last expression | | +| `st.markdown()` | `mo.md()` | Supports f-strings: `mo.md(f"Value: {x.value}")` | +| `st.latex()` | `mo.md(r"$...$")` | marimo uses KaTeX; see `references/latex.md` | +| `st.code()` | `mo.md("```python\n...\n```")` | | +| `st.dataframe()` | `df` (last expression) | DataFrames render as interactive marimo widgets natively; use `mo.ui.dataframe(df)` only for no-code transformations | +| `st.table()` | `df` (last expression) | Use `mo.ui.table(df)` if you need row selection | +| `st.metric()` | `mo.stat()` | | +| `st.json()` | `mo.json()` or `mo.tree()` | `mo.tree()` for interactive collapsible view | +| `st.image()` | `mo.image()` | | +| `st.audio()` | `mo.audio()` | | +| `st.video()` | `mo.video()` | | + +### Charts + +| Streamlit | marimo | Notes | +|-----------|--------|-------| +| `st.plotly_chart(fig)` | `fig` (last expression) | Use `mo.ui.plotly(fig)` for selections | +| `st.altair_chart(chart)` | `chart` (last expression) | Use `mo.ui.altair_chart(chart)` for selections | +| `st.pyplot(fig)` | `fig` (last expression) | Use `mo.ui.matplotlib(fig)` for interactive matplotlib | + +### Layout + +| Streamlit | marimo | Notes | +|-----------|--------|-------| +| `st.sidebar` | `mo.sidebar([...])` | Pass a list of elements | +| `st.columns()` | `mo.hstack([...])` | Use `widths=[...]` for column ratios | +| `st.tabs()` | `mo.ui.tabs({...})` | Dict of `{"Tab Name": content}` | +| `st.expander()` | `mo.accordion({...})` | Dict of `{"Title": content}` | +| `st.container()` | `mo.vstack([...])` | | +| `st.empty()` | `mo.output.replace()` | | +| `st.progress()` | `mo.status.progress_bar()` | | +| `st.spinner()` | `mo.status.spinner()` | Context manager | + +## Key Conceptual Differences + +### Execution Model + +Streamlit reruns the **entire script** top-to-bottom on every interaction. Marimo uses a **reactive cell DAG** — only cells that depend on changed variables re-execute. + +- No need for `st.rerun()` — reactivity is automatic. +- No need for `st.stop()` — structure cells so downstream cells naturally depend on upstream values. + +### State Management + +| Streamlit | marimo | +|-----------|--------| +| `st.session_state["key"]` | Regular Python variables between cells | +| Callback functions (`on_change`) | Cells referencing `widget.value` re-run automatically | +| `st.query_params` | `mo.query_params` | + +### Caching + +| Streamlit | marimo | +|-----------|--------| +| `@st.cache_data` | `@mo.cache` | Caches based on function arguments; marimo-aware | +| `@st.cache_resource` | `@mo.persistent_cache` | Persists across notebook restarts (serializes to disk) | + +`@mo.cache` is the primary caching decorator — it works like `functools.cache` but is aware of marimo's reactivity. `@mo.persistent_cache` goes further by persisting results to disk across sessions, useful for expensive computations like model training. + +### Multi-Page Apps + +Marimo offers two approaches for multi-page Streamlit apps: + +- **Single notebook with routing**: Use `mo.routes` with `mo.nav_menu` or `mo.sidebar` to build multiple "pages" (tabs/routes) inside one notebook. +- **Multiple notebooks as a gallery**: Run a folder of notebooks with `marimo run folder/` to serve them as a gallery with navigation. + +### Deploying + +marimo features molab to host marimo apps instead of the streamlit community cloud. You can generate an "open in molab" button via the `add-molab-badge` skill. + +### Custom components + +streamlit has a feature for custom components. These are not compatible with marimo. You might be able to generate an equivalent anywidget via the `marimo-anywidget` skill but discuss this with the user before working on that. diff --git a/config/claude/skills/wasm-compatibility/SKILL.md b/config/claude/skills/wasm-compatibility/SKILL.md new file mode 100644 index 0000000..f718037 --- /dev/null +++ b/config/claude/skills/wasm-compatibility/SKILL.md @@ -0,0 +1,140 @@ +--- +name: wasm-compatibility +description: Check if a marimo notebook is compatible with WebAssembly (WASM) and report any issues. +--- + +# WASM Compatibility Checker for marimo Notebooks + +Check whether a marimo notebook can run in a WebAssembly (WASM) environment — the marimo playground, community cloud, or exported WASM HTML. + +## Instructions + +### 1. Read the notebook + +Read the target notebook file. If the user doesn't specify one, ask which notebook to check. + +### 2. Extract dependencies + +Collect every package the notebook depends on from **both** sources: + +- **PEP 723 metadata** — the `# /// script` block at the top: + ```python + # /// script + # dependencies = [ + # "marimo", + # "torch>=2.0.0", + # ] + # /// + ``` +- **Import statements** — scan all cells for `import foo` and `from foo import bar`. Map import names to their PyPI distribution name using this table: + + | Import name | Distribution name | + |---|---| + | `sklearn` | `scikit-learn` | + | `skimage` | `scikit-image` | + | `cv2` | `opencv-python` | + | `PIL` | `Pillow` | + | `bs4` | `beautifulsoup4` | + | `yaml` | `pyyaml` | + | `dateutil` | `python-dateutil` | + | `attr` / `attrs` | `attrs` | + | `gi` | `PyGObject` | + | `serial` | `pyserial` | + | `usb` | `pyusb` | + | `wx` | `wxPython` | + + For most other packages, the import name matches the distribution name. + +### 3. Check each package against Pyodide + +For each dependency, determine if it can run in WASM: + +1. **Is it in the Python standard library?** Most stdlib modules work, but these do **not**: + - `multiprocessing` — browser sandbox has no process spawning + - `subprocess` — same reason + - `threading` — emulated, no real parallelism (WARN, not a hard fail) + - `sqlite3` — use `apsw` instead (available in Pyodide) + - `pdb` — not supported + - `tkinter` — no GUI toolkit in browser + - `readline` — no terminal in browser + +2. **Is it a Pyodide built-in package?** See [pyodide-packages.md](references/pyodide-packages.md) for the full list. These work out of the box. + +3. **Is it a pure-Python package?** Packages with only `.py` files (no compiled C/Rust extensions) can be installed at runtime via `micropip` and will work. To check: look for a `py3-none-any.whl` wheel on PyPI (e.g. visit `https://pypi.org/project//#files`). If the only wheels are platform-specific (e.g. `cp312-cp312-manylinux`), the package has native extensions and likely won't work. + + Common pure-Python packages that work (not in Pyodide built-ins but installable via micropip): + - `plotly`, `seaborn`, `humanize`, `pendulum`, `arrow`, `tabulate` + - `dataclasses-json`, `marshmallow`, `cattrs`, `pydantic` (built-in) + - `httpx` (built-in), `tenacity`, `backoff`, `wrapt` (built-in) + +4. **Does it have C/native extensions not built for Pyodide?** These will **not** work. Common culprits: + - `torch` / `pytorch` + - `tensorflow` + - `jax` / `jaxlib` + - `psycopg2` (suggest `psycopg` with pure-Python mode) + - `mysqlclient` (suggest `pymysql`) + - `uvloop` + - `grpcio` + - `psutil` + +### 4. Check for WASM-incompatible patterns + +Scan the notebook code for patterns that won't work in WASM: + +| Pattern | Why it fails | Suggestion | +|---|---|---| +| `subprocess.run(...)`, `os.system(...)`, `os.popen(...)` | No process spawning in browser | Remove or gate behind a non-WASM check | +| `multiprocessing.Pool(...)`, `ProcessPoolExecutor` | No process forking | Use single-threaded approach | +| `threading.Thread(...)`, `ThreadPoolExecutor` | Emulated threads, no real parallelism | WARN only — works but no speedup; use `asyncio` for I/O | +| `open("/absolute/path/...")`, hard-coded local file paths | No real filesystem; only in-memory fs | Fetch data via URL (`httpx`, `urllib`) or embed in notebook | +| `sqlite3.connect(...)` | stdlib sqlite3 unavailable | Use `apsw` or `duckdb` | +| `pdb.set_trace()`, `breakpoint()` | No debugger in WASM | Remove breakpoints | +| Reading env vars (`os.environ[...]`, `os.getenv(...)`) | Environment variables not available in browser | Use `mo.ui.text` for user input or hardcode defaults | +| `Path.home()`, `Path.cwd()` with real file expectations | Virtual filesystem only | Use URLs or embedded data | +| Large dataset loads (>100 MB) | 2 GB total memory cap | Use smaller samples or remote APIs | + +### 5. Check PEP 723 metadata + +WASM notebooks should list all dependencies in the PEP 723 `# /// script` block so they are automatically installed when the notebook starts. Check for these issues: + +- **Missing metadata:** If the notebook has no `# /// script` block, emit a WARN recommending one. Listing dependencies ensures they are auto-installed when the notebook starts in WASM — without it, users may see import errors. +- **Missing packages:** If a package is imported but not listed in the dependencies, emit a WARN suggesting it be added. +Note: version pins and lower bounds in PEP 723 metadata are fine — marimo strips version constraints when running in WASM. + +### 6. Produce the report + +Output a clear, actionable report with these sections: + +**Compatibility: PASS / FAIL / WARN** + +Use these verdicts: +- **PASS** — all packages and patterns are WASM-compatible +- **WARN** — likely compatible, but some packages could not be verified as pure-Python (list them so the user can check) +- **FAIL** — one or more packages or patterns are definitely incompatible + +**Package Report** — table with columns: Package, Status (OK / WARN / FAIL), Notes + +Example: +| Package | Status | Notes | +|---|---|---| +| marimo | OK | Available in WASM runtime | +| numpy | OK | Pyodide built-in | +| pandas | OK | Pyodide built-in | +| torch | FAIL | No WASM build — requires native C++/CUDA extensions | +| my-niche-lib | WARN | Not in Pyodide; verify it is pure-Python | + +**Code Issues** — list each problematic code pattern found, with the cell or line and a suggested fix. + +**Recommendations** — if the notebook fails, suggest concrete fixes: +- Replace incompatible packages with WASM-friendly alternatives +- Rewrite incompatible code patterns +- Suggest moving heavy computation to a hosted API and fetching results + +## Additional context + +- WASM notebooks run via [Pyodide](https://pyodide.org) in the browser +- Memory is capped at 2 GB +- Network requests work but may need CORS-compatible endpoints +- Chrome has the best WASM performance; Firefox, Edge, Safari also supported +- `micropip` can install any pure-Python wheel from PyPI at runtime +- For the full Pyodide built-in package list, see [pyodide-packages.md](references/pyodide-packages.md) diff --git a/config/claude/skills/wasm-compatibility/references/pyodide-packages.md b/config/claude/skills/wasm-compatibility/references/pyodide-packages.md new file mode 100644 index 0000000..af3ba65 --- /dev/null +++ b/config/claude/skills/wasm-compatibility/references/pyodide-packages.md @@ -0,0 +1,73 @@ +# Pyodide Built-in Packages + +These packages are pre-built for Pyodide and available in WASM environments. +Any package **not** on this list must have a pure Python wheel on PyPI to work. + +> **Note:** This list was snapshotted on 2026-02-26 from Pyodide's docs. +> For the latest list, check https://pyodide.org/en/stable/usage/packages-in-pyodide.html + +affine, aiohappyeyeballs, aiohttp, aiosignal, altair, annotated-types, anyio, +apsw, argon2-cffi, argon2-cffi-bindings, asciitree, astropy, astropy_iers_data, +asttokens, async-timeout, atomicwrites, attrs, audioop-lts, autograd, +awkward-cpp, b2d, bcrypt, beautifulsoup4, bilby.cython, biopython, bitarray, +bitstring, bleach, blosc2, bokeh, boost-histogram, Bottleneck, brotli, +cachetools, Cartopy, casadi, cbor-diag, certifi, cffi, cffi_example, cftime, +charset-normalizer, clarabel, click, cligj, clingo, cloudpickle, cmyt, cobs, +colorspacious, contourpy, coolprop, coverage, cramjam, crc32c, cryptography, +css-inline, cssselect, cvxpy-base, cycler, cysignals, cytoolz, decorator, +demes, deprecation, diskcache, distlib, distro, docutils, donfig, +ewah_bool_utils, exceptiongroup, executing, fastapi, fastcan, fastparquet, +fiona, fonttools, freesasa, frozenlist, fsspec, future, galpy, geopandas, +gmpy2, google-crc32c, gsw, h11, h3, h5py, healpy, highspy, html5lib, httpcore, +httpx, idna, igraph, imageio, imgui-bundle, iminuit, iniconfig, inspice, +ipython, jedi, Jinja2, jiter, joblib, jsonpatch, jsonpointer, jsonschema, +jsonschema_specifications, kiwisolver, lakers-python, lazy_loader, +lazy-object-proxy, libcst, lightgbm, logbook, lxml, lz4, MarkupSafe, +matplotlib, matplotlib-inline, memory-allocator, micropip, ml_dtypes, mmh3, +more-itertools, mpmath, msgpack, msgspec, msprime, multidict, munch, mypy, +narwhals, ndindex, netcdf4, networkx, newick, nh3, nlopt, nltk, numcodecs, +numpy, openai, opencv-python, optlang, orjson, packaging, pandas, parso, patsy, +pcodec, peewee, pi-heif, Pillow, pillow-heif, pkgconfig, platformdirs, pluggy, +ply, pplpy, primecountpy, prompt_toolkit, propcache, protobuf, pure-eval, py, +pyarrow, pycdfpp, pyclipper, pycparser, pycryptodome, pydantic, pydantic_core, +pyerfa, pygame-ce, Pygments, pyheif, pyiceberg, pyinstrument, pylimer-tools, +PyMuPDF, pynacl, pyodide-http, pyodide-unix-timezones, pyparsing, pyproj, +pyrodigal, pyrsistent, pysam, pyshp, pytaglib, pytest, pytest-asyncio, +pytest-benchmark, pytest_httpx, python-calamine, python-dateutil, python-flint, +python-magic, python-sat, python-solvespace, pytz, pywavelets, pyxel, pyxirr, +pyyaml, rasterio, rateslib, rebound, reboundx, referencing, regex, requests, +retrying, rich, river, RobotRaconteur, rpds-py, ruamel.yaml, rustworkx, +scikit-image, scikit-learn, scipy, screed, setuptools, shapely, simplejson, +sisl, six, smart-open, sniffio, sortedcontainers, soundfile, soupsieve, +sourmash, soxr, sparseqr, sqlalchemy, stack-data, starlette, statsmodels, +strictyaml, svgwrite, swiglpk, sympy, tblib, termcolor, texttable, +texture2ddecoder, threadpoolctl, tiktoken, tomli, tomli-w, toolz, tqdm, +traitlets, traits, tree-sitter, tree-sitter-go, tree-sitter-java, +tree-sitter-python, tskit, typing-extensions, typing-inspection, tzdata, ujson, +uncertainties, unyt, urllib3, vega-datasets, vrplib, wcwidth, webencodings, +wordcloud, wrapt, xarray, xgboost, xlrd, xxhash, xyzservices, yarl, yt, zengl, +zfpy, zstandard + +## Also available (part of Pyodide runtime or marimo WASM) + +- marimo +- duckdb +- polars +- micropip (for installing additional pure-Python packages at runtime) + +## Common third-party packages that do NOT work in WASM + +These popular packages have C/native extensions not built for Pyodide: + +| Package | Why | Alternative | +|---|---|---| +| torch / pytorch | C++/CUDA extensions | None for WASM | +| tensorflow | C++ extensions | None for WASM | +| jax / jaxlib | C++ extensions | None for WASM | +| psycopg2 | Requires libpq | `psycopg[binary]` or use `duckdb` | +| mysqlclient | Requires libmysqlclient | `pymysql` (pure Python) | +| uvloop | Requires libuv | `asyncio` (default loop) | +| grpcio | C extensions | `grpclib` (pure Python) | +| psutil | OS-level syscalls | None for WASM | +| gevent | C extensions | `asyncio` | +| celery | Requires message broker | Not applicable in browser | diff --git a/config/nvim/lazy-lock.json b/config/nvim/lazy-lock.json index f7cb79a..bb9da2c 100644 --- a/config/nvim/lazy-lock.json +++ b/config/nvim/lazy-lock.json @@ -18,6 +18,7 @@ "kanagawa.nvim": { "branch": "master", "commit": "aef7f5cec0a40dbe7f3304214850c472e2264b10" }, "lazy.nvim": { "branch": "main", "commit": "85c7ff3711b730b4030d03144f6db6375044ae82" }, "lazydev.nvim": { "branch": "main", "commit": "5231c62aa83c2f8dc8e7ba957aa77098cda1257d" }, + "lean.nvim": { "branch": "main", "commit": "fde0d5692edcc0a0cadad77067dfd49a51332236" }, "lualine.nvim": { "branch": "master", "commit": "47f91c416daef12db467145e16bed5bbfe00add8" }, "mason-lspconfig.nvim": { "branch": "main", "commit": "a676ab7282da8d651e175118bcf54483ca11e46d" }, "mason.nvim": { "branch": "main", "commit": "44d1e90e1f66e077268191e3ee9d2ac97cc18e65" }, diff --git a/ghosttyremovalplan.md b/ghosttyremovalplan.md new file mode 100644 index 0000000..e5315b8 --- /dev/null +++ b/ghosttyremovalplan.md @@ -0,0 +1,304 @@ +# anu — Terminal Host Plan (replacing generic Ghostty with an anu-aware terminal) + +> Where the *outermost* layer of anu is going, and why. Read top-to-bottom. +> Each phase is independently shippable; stop at any phase if priorities shift. +> +> **This plan does not touch tmux, the bash orchestration, the agents, or the +> swarm/mesh/box model. Those stay exactly as they are, forever.** The only +> thing being replaced is the *generic terminal emulator* anu currently runs +> inside. + +--- + +## 1. Thesis + +anu runs inside Ghostty today, but Ghostty has no idea anu exists. Its entire +relationship to anu is a **1 KB config file** (`config/ghostty/config`) — font, +colors, padding, three macOS window keys. It is a general-purpose terminal, and +anu is just text scrolling by inside it. + +That means everything anu shows you — `pd`, `pds`, `reviewd`, `swarm +dashboard`, the atlas dossiers — is either **ASCII art painted into a tmux +pane** or **HTML kicked out to a browser**, because the terminal host can't +render anything richer. The host is a dumb pixel surface. + +**The plan is to replace that generic host with a bespoke, anu-aware terminal** +— built *from* Ghostty's now-open-source core (`libghostty`) — that understands +what it is rendering. tmux keeps multiplexing underneath; the bash layer keeps +orchestrating. The terminal stops being a passive surface and becomes a +**participant in anu**. + +"Removal" here means removing the dependency on *stock, anu-unaware Ghostty as a +separate app*. It does **not** mean removing terminal emulation — that tech is +inherited wholesale from libghostty. We fold the host into anu. + +One-line reframe: + +> Keep tmux + bash exactly as-is. Replace the dumb terminal host with one that +> speaks anu. + +--- + +## 2. The layer being replaced + +``` +┌─ Terminal host ←── THIS layer. Generic Ghostty today → anu-terminal. +│ bytes → pixels, GPU render, font shaping, input, window chrome +│ +├─ tmux ←── UNCHANGED. The multiplexer. Sessions/panes/detach. +├─ bash anu fns ←── UNCHANGED. swarm / box / mesh / nv / review / trail. +└─ agents / nvim ←── UNCHANGED. The programs in the panes. +``` + +Why this is the right cut: + +- **anu's value is in the orchestrator, not the renderer.** ~9000 lines of bash + + ~14000 lines of Claude plugins are platform-agnostic logic. The terminal is + the one layer that is currently *generic and replaceable* — and replacing it + is purely additive. +- **Keeping tmux is what makes this safe.** tmux gives us + sessions/panes/detach/reattach and — critically — the **remote story** (SSH, + mesh over Tailscale, the cluster login node). None of that is at risk here, + because tmux is untouched. (Contrast: rebuilding the multiplexer natively + would amputate remote. We are explicitly *not* doing that — see §10.) + +--- + +## 3. Why now + +- **Ghostty is open source** (Zig, released late 2024) and was designed as + `libghostty` — an embeddable terminal *core* with thin app shells over it + (macOS Swift/AppKit + Metal; Linux GTK). The apps are the embedder; the core + is reusable. That is the unlock: we don't fork "the Ghostty app," we embed the + same core Mitchell ships and build **anu's chrome as the shell around it**. +- **anu already owns both ends of the wire.** The bash layer and the agents + write to the terminal; the terminal renders. When one project controls both + producer and consumer, it can define a **private protocol** between them. That + is impossible with a generic terminal and trivial with a bespoke one. +- **Caveat:** libghostty's *stable, public* embedding API is still maturing as + of early 2026. Early phases may pin a Ghostty commit and track upstream + closely. This is the recurring tax of being a host (see §9). + +--- + +## 4. The core mechanism — the anu private escape protocol + +This is the foundation everything else stands on. Build it first. + +### 4.1 The idea + +Because anu controls both the writer (bash/agents) and the reader (the +anu-terminal), anu can define **its own escape sequences** that only the +anu-terminal interprets. An agent or an anu function emits a structured +sequence on the normal output stream; the host parses it and drives **native +UI** instead of printing characters. + +This is exactly how existing private terminal protocols work: + +| Protocol | Mechanism | Used for | +|---|---|---| +| Kitty graphics | `APC G ... ST` | inline images | +| iTerm2 | `OSC 1337 ; ... ST` | inline images, badges, marks | +| Desktop notifications | `OSC 9` / `OSC 777` | notifications (Ghostty supports this) | +| Progress | `OSC 9 ; 4 ; ...` | taskbar/progress (Ghostty supports this) | +| Clipboard | `OSC 52` | set clipboard remotely | + +anu gets its own entry in that table: a private **`OSC ; ; + ST`** (or APC) vocabulary, namespaced so it never collides. + +### 4.2 It survives tmux + +Private sequences pass *through* tmux to the host with passthrough enabled: + +``` +set -g allow-passthrough on # tmux.conf +# and/or wrap in the DCS tmux passthrough envelope: +# \ePtmux;\e \e\\ +``` + +So the anu function emits the sequence, tmux forwards it untouched, and the +anu-terminal at the outer edge renders native chrome. tmux never has to know +the sequence exists. + +### 4.3 It degrades gracefully — this is the keystone property + +**Any terminal that does not understand an anu sequence simply ignores it.** +That gives us, for free: + +- **Local, in the anu-terminal:** rich native chrome. +- **Remote, over SSH in a generic terminal (phone, cluster login node, someone + else's machine):** anu still works — it falls back to plain tmux + ASCII, + exactly as today. + +The native layer is a **progressive enhancement, not a fork in the road.** We +never have to choose between "polished local cockpit" and "reachable +anywhere." We keep both, precisely because we kept tmux. + +### 4.4 Example vocabulary (illustrative) + +``` +OSC anu ; agent.status ; agent-3 ; running ; +142/-17 ST +OSC anu ; swarm.open ; star ; 4 ; ST +OSC anu ; panel ; review ; show ST +OSC anu ; mark ; src/foo.py:42 ; "touched here" ST +OSC anu ; render ; html ; ~/.anu/atlas//index.html ST +OSC anu ; notify ; agent-2 done ; clickable=focus ST +``` + +The bash layer learns one helper — `_anu_emit ` — that wraps +the payload in the OSC/DCS envelope. Single choke point. No native code smeared +across the functions; they just *announce* structured state on a side channel. + +--- + +## 5. What the anu-terminal renders natively + +Each native feature **upgrades an existing anu command** rather than inventing a +new surface. The bash command stays; it gains a richer rendering when the host +understands it, and keeps its ASCII rendering when the host doesn't. + +| Today (generic terminal) | With the anu-terminal | +|---|---| +| `pd` / `pds` — ASCII dashboard in a popup/pane | Native, always-on sidebar: swarm agents, branches, worktrees, live git-diff stats | +| `reviewd` — auto-refreshing tmux pane of text | Native review surface; click a finding → jump in nvim via `nv` | +| `swarm dashboard` / `swarm status` — painted boxes | Native swarm panel: per-agent status, mailbox, capture, diff | +| `map` / atlas dossiers — HTML opened in a browser | Inline native HTML/image surface the terminal renders (Kitty-graphics-style) | +| trail / investigate / ledger — text + browser HTML | Native decision-graph / metrics panel driven by the protocol | +| agent-done → one-way `osascript` notification | Native, **bidirectional, clickable** notification (focus the agent) + dock badge of running agents | +| `file:line` in agent output — plain text | Clickable → opens in nvim through the existing `nv` bridge | +| anu verbs — typed at the shell | Native ⌘K command palette → calls the bash layer (`swarm start`, `taa`, `review`, …) | +| matte-black theme via 1 KB config | Baked into the app; one install, batteries included | + +Note the `nv` bridge already drives nvim over RPC — the anu-terminal is its +natural GUI counterpart: the editor is driveable, and now the *host* is too. + +--- + +## 6. Architecture + +``` +┌──────────────────────────────────────────────────────────┐ +│ anu-terminal (macOS app first; GTK later) │ +│ │ +│ ┌─ native chrome ──────────────────────────────────┐ │ +│ │ swarm sidebar · review surface · command palette │ │ +│ │ inline HTML/image · native notifications │ │ +│ └───────────────▲───────────────────────────────────┘ │ +│ │ parses OSC anu;… │ +│ ┌─ libghostty (embedded terminal core) ────────────┐ │ +│ │ cell grid · GPU render · font shaping · PTY · input│ │ +│ └───────────────▲───────────────────────────────────┘ │ +└───────────────────┼────────────────────────────────────────┘ + │ PTY (unchanged) + tmux → bash anu fns → agents / nvim +``` + +- **libghostty** does all terminal emulation (we inherit it; we do not rewrite + it). +- **The anu shell** is the new code: it (a) hosts libghostty, (b) parses the + `OSC anu;…` side channel, (c) renders native panels, (d) translates native + input (clicks, ⌘K) back into anu/tmux commands. +- **Fork vs embed:** start by **embedding libghostty** if its API is ready + enough; otherwise **fork Ghostty** and add the shell inside the fork. Either + way the boundary is the same — anu owns the chrome and the protocol; libghostty + owns the cells. + +--- + +## 7. Build order + +Front-load the value; defer the GUI cost. + +**Phase 0 — Protocol, on the *current* terminal (low cost, high leverage).** +Define the `OSC anu;…` vocabulary and the `_anu_emit` helper. Teach the existing +bash functions (`swarm`, `review`, `pd`, agent lifecycle) to emit it. Enable +`allow-passthrough` in `tmux.conf`. Stock Ghostty ignores the sequences today — +that's fine. Now anu *announces* structured state, and we have a target to build +a host against. This is the no-slop choke point: one verb, one envelope, not +native logic scattered everywhere. + +**Phase 1 — Minimal anu-terminal, one panel.** Embed libghostty (or fork). Parse +the side channel. Render exactly **one** native surface end-to-end — the swarm +sidebar — driven by Phase 0's `agent.status` / `swarm.open` sequences. Prove the +loop: bash emits → tmux forwards → host renders native. Ship it. + +**Phase 2 — Grow the surfaces.** Review surface, command palette, clickable +`file:line` → nvim, native notifications + dock badge. Each one is an additive +listener on the protocol; none touches tmux or the orchestration. + +**Phase 3 — Inline rich rendering.** Native HTML/image surface for atlas +dossiers, trail/investigate/ledger graphs. Retire the browser hop for those +artifacts when local. + +**Phase 4 (optional) — Linux flavor.** GTK embedder of libghostty, same +protocol, same chrome. Aligns with the engine/flavors split in `PLAN.md`. + +At every phase: tmux unchanged, bash unchanged, remote unaffected. + +--- + +## 8. The graceful-degradation contract + +Non-negotiable, because it's what keeps the remote story alive: + +1. **Every native feature must have an ASCII fallback** that already works in a + generic terminal. The native version is a render *upgrade* of an existing + command, never a replacement that only exists in the anu-terminal. +2. **The bash layer never assumes the host is the anu-terminal.** It emits the + protocol unconditionally; rendering is the host's problem. Over SSH the + sequence is silently dropped and the user sees today's anu. +3. **Capability handshake (optional):** the host may announce itself (e.g. a + response to a query sequence) so anu *can* skip the ASCII version when it + knows native chrome is live — but the default is "emit both / emit and + ignore." + +This is the whole reason keeping tmux matters: it is the substrate that lets the +same anu run rich-local and plain-remote with no branching in the orchestration. + +--- + +## 9. Costs, risks, open questions + +- **Host maintenance tax.** Becoming the host means tracking libghostty/Ghostty + upstream — its Zig build, releases, security fixes. Pin a commit early; budget + ongoing upkeep. +- **GUI is a different skill from anu's core.** Native chrome is Swift/AppKit + + Metal (then GTK). That's the genuinely expensive, genuinely *new* work — and + it's unlike the bash/tmux work anu is made of. +- **API maturity.** libghostty's stable embedding surface is still settling; a + fork may be the pragmatic Phase-1 path even though embedding is the cleaner + end state. +- **Discipline cost.** The §8 fallback contract is permanent overhead on every + feature. Worth it; not free. +- **macOS-first.** Linux doubles the host work (Phase 4). +- **Open question:** OSC vs APC for the namespace, and the exact private code — + pick numbers that don't collide with Kitty/iTerm2/Ghostty's own. +- **Open question:** how much native chrome belongs *in* the terminal window vs. + separate native windows the app manages (a dedicated review window, a + dashboard window). +- **Open question:** does the command palette shell out to the bash fns, or talk + to a future anu daemon (the empty `kernel/`)? The protocol makes either + possible; defer the choice. + +--- + +## 10. Non-goals (explicit) + +- **Not** replacing or rebuilding tmux. No native multiplexer. tmux owns + sessions/panes/detach/reattach/persistence forever. +- **Not** changing the swarm/box/mesh/ncn model, the bash functions, the agents, + or the Claude plugins. +- **Not** sacrificing the remote story. If a feature can't degrade to plain + tmux + ASCII over SSH, it doesn't ship in the orchestration layer (it can live + as host-only polish, but the core must work headless). +- **Not** a from-scratch terminal emulator. Terminal emulation is inherited from + libghostty; we build only the anu-aware shell on top. + +--- + +## Appendix — the principle, in one sentence + +anu's orchestration already treats the editor as a driveable surface (`nv`). +This plan extends the same idea to the **host**: the terminal stops being a +place anu *runs* and becomes a surface anu *drives* — over a private protocol +that degrades to plain text the moment you leave the local machine. diff --git a/plugins/marimo/skills/marimo-authoring/SKILL.md b/plugins/marimo/skills/marimo-authoring/SKILL.md index d6435fc..ed85033 100644 --- a/plugins/marimo/skills/marimo-authoring/SKILL.md +++ b/plugins/marimo/skills/marimo-authoring/SKILL.md @@ -1,183 +1,54 @@ --- name: marimo-authoring -description: Use when authoring, editing, or debugging marimo notebooks (.py files containing @app.cell decorators or import marimo as mo). Covers reactive execution, cell anatomy, PEP 723 inline dependencies, mo.ui widgets, SQL cells via mo.sql, plotting rules, script mode, pytest integration, sharing/WASM export, and the `marimo check` lint contract, plus the Jupyter differences that trip up new users. +description: SUPPLEMENT to the official `marimo-notebook` skill (marimo-team/skills) — load both. The official skill + its references/ cover the canonical authoring rules (format, reactivity, PEP 723, mo.cache/mo.lazy, run_button+mo.stop for expensive compute, SQL, deployment, marimo check). This file holds only the gotchas the official set does NOT cover, learned the hard way: running event-loop-bound / paid SDKs from cells (async + await), keeping billable/side-effecting calls non-reactive, hide_code for clean reading views, extracting domain logic into a module, and the formatter that mangles dynamic mo.md cells. --- -# Authoring marimo notebooks +# marimo authoring — gotchas beyond the official skill -Marimo is a **reactive** Python notebook. Cells re-run automatically when their inputs change. Unlike Jupyter, there is no hidden state and no global-by-default scoping: each cell explicitly declares the names it produces. +The official **`marimo-notebook`** skill (installed via `npx skills add marimo-team/skills`) is the canonical reference — defer to it and its `references/*.md` for format, reactivity, PEP 723, `mo.cache`/`mo.persistent_cache`/`mo.lazy`, `run_button`+`mo.stop` basics, SQL, columns, deployment, pytest, and `marimo check`. Everything below is the **delta** that bit us and isn't in the official set. -Notebooks are saved as **pure Python files** (`notebook.py`), not JSON. They diff cleanly in git. +## Async cells & event-loop-bound libraries *(not in the official skill)* -## The mental model in three rules +marimo runs every cell inside a running asyncio event loop. Cells may use **top-level `await`** — mark the cell `async def _(...)` (marimo writes this when the body awaits). -1. **A cell's outputs are its return value.** If a cell defines `x` and `y` that other cells use, the cell must `return (x, y)`. -2. **A cell's inputs are its free variables.** Marimo parses the cell, finds names not defined locally, and treats them as inputs. The runtime re-runs the cell when any input's value changes. -3. **There are no globals across cells.** A name defined inside a cell is local to that cell unless it's returned. - -Together, these mean: edit a cell that produces `x`, and every downstream cell that *uses* `x` automatically re-executes. No "Run All", no out-of-order surprises. - -## Cell anatomy +This bites with any library whose *synchronous* methods internally drive an event loop or block on network I/O: called from a cell they warn (`sync method called from async context`) and can **deadlock**. Use the async API and await it: ```python -# /// script -# requires-python = ">=3.12" -# dependencies = [ -# "marimo", -# ] -# /// - -import marimo - -__generated_with = "0.x.x" -app = marimo.App(width="medium") - - -@app.cell -def _(): - import marimo as mo - return (mo,) - - -@app.cell -def _(mo): - slider = mo.ui.slider(0, 100, value=50, label="Speed") - slider - return (slider,) - - @app.cell -def _(slider): - speed = slider.value * 2 - return (speed,) - - -@app.cell -def _(mo, speed): - mo.md(f"**Speed:** {speed} mph") - return - - -if __name__ == "__main__": - app.run() +async def _(service): + caps = await service.get_server_capabilities_async() # not the sync method + return (caps,) ``` -Notice: -- The file starts with a **PEP 723 inline-script header** declaring every dependency. The notebook is self-contained: `uv run notebook.py` and `uvx marimo edit --sandbox` both work from it. -- Each cell is `def _(...)`, single underscore, decorated with `@app.cell`. Marimo manages signatures and returns; don't hand-pick fancy names. -- Function args = the cell's inputs. Exports are parenthesized tuples: `return (slider,)`. -- Put all imports in the **first cell**, including `import marimo as mo`. -- Only the **final expression** of a cell renders. `return` produces no output; indented or conditional expressions don't render. For conditional output, assign to a variable in each branch and put the bare variable last. - -## Hard rules (`marimo check` flags these) +- Prefer `await client.do_async(...)` over `client.do(...).result()` in cells. Some SDKs' async *submit* returns a future, so you then `await future.result_async()`. +- Plain in-process work (tokenizers, numpy, pure functions) stays sync — only loop-driving/network calls need the async variant. +- This is why a script that "works" misbehaves once pasted into a notebook: the surrounding event loop, not your logic. -- **One owning cell per name.** No redefining the same name in two cells. -- **No `if` guards around cell bodies, no try/except as control flow.** Reactivity handles re-execution; only catch genuinely expected exceptions. -- **Don't mutate objects across cells.** Mutation is invisible to the dependency graph; derive a new name instead (`df2 = df.drop(...)`, not `df.drop(..., inplace=True)` in another cell). -- **Underscore-prefixed names (`_tmp`) are cell-local** and invisible to other cells. Keep to ≤ 2 per cell; prefer real names. -- **Don't read a widget's `.value` in the cell that defines it.** Read it one cell downstream. -- **Avoid `mo.state()`** unless you genuinely need bidirectional widget sync; plain reactivity covers nearly everything. +## Billable / side-effecting calls must never be reactive -## mo.ui widgets +The official skill frames `run_button`+`mo.stop` as a perf tool. The sharper rule: a **paid API call, write, or training run** must be gated, or a slider drag re-fires it and spends money/mutates state on every nudge. -Widgets are **reactive values**. Reading `.value` in a cell makes that cell depend on the widget. - -Common widgets: ```python -mo.ui.slider(0, 100, value=50, label="x") -mo.ui.number(value=0, step=0.1) -mo.ui.text(value="hello", placeholder="name") -mo.ui.dropdown(options=["a", "b", "c"], value="a") -mo.ui.checkbox(value=True, label="enabled") -mo.ui.file(filetypes=[".csv"]) -mo.ui.button(label="run") -mo.ui.refresh(default_interval=1.0) # for polling +run_btn = mo.ui.run_button(label="▶ Run") # cell A +# cell B (downstream, may be async): +mo.stop(not run_btn.value, mo.md("Set controls, then click Run.")) +result = await paid_api.call_async(prompt.value) # only on click ``` -Compose with `mo.ui.array([...])` (value is a list) or `mo.ui.dictionary({"alpha": mo.ui.slider(0, 1), ...})` (value is a dict). - -## mo.md and layout - -```python -mo.md("# Heading\nSome **bold** text.") -mo.md(f"x = {x}") # f-string interpolation works -mo.callout(mo.md("..."), kind="info") -mo.hstack([a, b]); mo.vstack([a, b]) -mo.tabs({"Tab 1": content_a, "Tab 2": content_b}) -mo.accordion({"Section": content}) -``` - -## SQL cells - -```python -df = mo.sql( - f""" - SELECT a, COUNT(*) AS n FROM tbl GROUP BY a - """ -) -``` - -Assign the result to a dataframe. **No comments inside the SQL string.** Pass `engine=my_engine` for non-DuckDB engines. - -## Plots - -- **matplotlib**: build the plot, end the cell with `plt.gca()`, never `plt.show()`. -- **plotly / altair**: make the figure/chart object the cell's final expression. - -## Script mode - -```python -if mo.app_meta().mode == "script": - data = load_full_dataset() -else: - data = sample_dataset() -``` - -Swap data sources by mode; keep UI code identical and widgets always visible. `uv run notebook.py` executes the notebook top-to-bottom as a script. - -## Testing - -Functions named `test_*` defined in cells are auto-discovered: `pytest notebook.py` just works. Add `pytest` to the PEP 723 header. - -## The contract: `marimo check` - -After **every** edit to a marimo file: - -```bash -uvx marimo check --fix notebook.py -``` - -`--fix` rewrites what it can and exits 0 regardless of remaining warnings, so resolve the leftover findings by hand. To make warnings fail (exit nonzero) and gate CI, run `--strict`. It catches the marimo-specific breakage plain linters miss: cross-cell redefinitions, cycles, formatting drift. `--format=json` emits machine-readable findings (the default is `full`). In Claude Code this plugin runs it automatically via a PostToolUse hook; elsewhere (e.g. pi), run it yourself. - -## The Jupyter → marimo gotchas - -If you're translating from Jupyter, these trip people up: - -- **No `display(x)`**: just write `x` as the last expression. -- **No `%matplotlib inline`**: end the cell with `plt.gca()` (or the figure object). -- **No mutable global dicts shared across cells**: marimo will error on multi-cell definitions of the same name. Pick one owning cell. -- **`del` doesn't help**: to "reset" state, restart the kernel via the UI. -- **No cell-order surprises**: execution order follows dependencies, not visual order. If you need a side effect at startup, return a marker value and depend on it. +`run_btn.value` is `True` only for the single run right after the click. **Render the result in the *same* cell that runs it** — splitting compute into one cell and display into another makes clicking the button look like nothing happened (output lands elsewhere). End the gated cell with `mo.vstack([...])` so feedback (and errors/spinners) appear where the user clicked. -## Editing +## `hide_code` for a clean reading view -- `uvx marimo edit --sandbox notebook.py --watch`: preferred. Isolated venv built from the PEP 723 header, live-reloads as the file changes on disk. Plain fallback: `marimo edit notebook.py`. -- `marimo convert old_notebook.ipynb -o new_notebook.py`: one-shot Jupyter conversion (then run `marimo check` and audit). Without `-o`, `marimo convert` prints to stdout and writes no file. +In `marimo edit` every cell shows source *and* rendered output — clutter for prose/controls/output cells. Mark them `@app.cell(hide_code=True)`: only the output shows, with a toggle to reveal code. It persists in the file. Leave your *teaching* cells (the ones showing the real API call) visible; for a fully code-free view, `marimo run`. -## Sharing and deployment +## Keep notebooks thin: push domain logic into a sibling module -- `marimo run notebook.py`: read-only app mode, code hidden. Use for dashboards. -- `marimo export html-wasm notebook.py -o site/`: static site that runs entirely in the browser via WASM (not all packages are WASM-compatible). -- `uv run notebook.py`: plain script execution; the PEP 723 header supplies the deps. +A cell that reads a widget, calls one `mylib.do(...)`, and displays beats one that inlines a long SDK/data pipeline — clearer, testable, reusable, and it makes the boundary obvious (notebook = UI/reactivity, module = the work). marimo puts the notebook's own directory on `sys.path`, so a sibling `mylib.py` imports directly. -## Performance and large data +## Formatter gotcha: markdown-ification mangles dynamic `mo.md` -- Cells re-run when *inputs change*, not on every interaction. To gate a heavy cell behind a click, use `mo.ui.run_button()` with `mo.stop(not btn.value)`. A plain `mo.ui.button()` only changes value if given `on_click` (e.g. `value=0, on_click=lambda v: v + 1`). -- Use `mo.ui.refresh` for explicit polling cells. -- For very large data, load once in a cell with no widget dependencies; downstream cells filter/transform. -- `mo.stop(condition, output)` short-circuits a cell early, useful when a widget hasn't been touched yet. +`marimo check --fix` rewrites a cell whose *only* statement is `mo.md()` into a triple-quoted markdown cell. With a plain literal that's clean — but a single `mo.md(f"..." f"..." "...")` (multi-piece **implicit string concatenation**) gets garbled: literal `f"`/`"` fragments end up in the rendered text. Fix: make it not a bare-`mo.md` cell — assign to a variable first (`text = f"..."; mo.md(text)`), or build inside `mo.vstack([...])`. Single-literal `mo.md("...")` is fine. -## When to choose marimo over Jupyter +## Cross-cell name collisions even when never returned -- Notebooks re-run by other people (reactive guarantees reproducibility), deployed as small apps (`marimo run`), or under version control (clean diffs). -- Stay with Jupyter for rich third-party widgets that haven't ported, or throwaway scratch where reactive re-execution gets in the way. +Any top-level (non-underscore) assignment is owned by its cell, so two cells that each compute a throwaway `rows = ...`/`pieces = ...` collide (`multiple-definitions`), even though neither is returned. Make per-cell temporaries cell-local with an underscore (`_rows`), or compute once and pass downstream. (The runtime smoke `python notebook.py` catches these even when you forget `marimo check`.)