diff --git a/myst.yml b/myst.yml index 40ead0c7..159ab1e5 100644 --- a/myst.yml +++ b/myst.yml @@ -25,7 +25,11 @@ project: - https://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-scan?projshort=SPITZER # GH is rather flaky for CI right now, temporarily measure - 'https://github.com/**' - + plugins: + # note: we need to use the built plugin bundle here so make sure to build it with + # `cd plugins && npm install && npm run build` after any changes to the plugins/src/ + - plugins/dist/notebook-gallery.mjs + extends: - toc.yml site: diff --git a/plugins/.gitignore b/plugins/.gitignore new file mode 100644 index 00000000..b9470778 --- /dev/null +++ b/plugins/.gitignore @@ -0,0 +1,2 @@ +node_modules/ +dist/ diff --git a/plugins/README.md b/plugins/README.md new file mode 100644 index 00000000..b9aeec5b --- /dev/null +++ b/plugins/README.md @@ -0,0 +1,69 @@ +# IRSA MyST Plugins + +Custom [MyST](https://mystmd.org) plugins for the irsa-tutorials. + +## Building + +```bash +cd plugins +npm install +npm run build +``` + +This bundles `src/notebook-gallery.mjs` into `dist/notebook-gallery.mjs`. Must *re-run* after any changes to `src/`. + +## Registering with MyST + +Add the built bundle to `myst.yml`: + +```yaml +project: + plugins: + - plugins/dist/notebook-gallery.mjs +``` + +--- + +## Plugin: `notebook-gallery` directive + +Renders a list of notebooks as a responsive grid of clickable cards. Each card shows a title and a one-sentence description pulled from a shared metadata file. + +### Metadata file + +The directive argument is a path to a YAML or JSON file (relative to the repo root) that lists notebook metadata. Each entry must have at least: + +| Field | Type | Description | +| ------------- | ------ | ---------------------------------------------------- | +| `file` | string | Path to the notebook relative to the repo root | +| `title` | string | Display title shown in the card header | +| `description` | string | One-sentence summary shown in the card body | + +Example `notebook_metadata.yml`: + +```yaml +- file: tutorials/wise/wise_catalog_search.md + title: WISE Catalog Search + description: Search the AllWISE catalog using astroquery. +- file: tutorials/euclid/1_Euclid_intro_MER_images.md + title: Euclid MER Images + description: Access and visualise Euclid Q1 MER mosaic images. +``` + +### Directive syntax + +````markdown +```{notebook-gallery} notebook_metadata.yml +tutorials/wise/wise_catalog_search.md +tutorials/euclid/1_Euclid_intro_MER_images.md +``` +```` + +- The **argument** (on the opening fence line) is the path to the metadata file. +- The **body** lists one notebook path per line (matching the `file` field in the metadata). Lines starting with `#` are treated as comments. + +### Error handling + +| Situation | Rendered output | +| ----------------------------------- | -------------------------------------------------------- | +| Metadata file not found / unreadable | An `{error}` admonition with the bad path | +| Notebook path not in metadata | A warning card: _⚠️ Unrecognised notebook_ | diff --git a/plugins/package-lock.json b/plugins/package-lock.json new file mode 100644 index 00000000..da619d93 --- /dev/null +++ b/plugins/package-lock.json @@ -0,0 +1,515 @@ +{ + "name": "irsa-myst-plugins", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "irsa-myst-plugins", + "dependencies": { + "yaml": "^2.7.0" + }, + "devDependencies": { + "esbuild": "^0.25.0" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.25.12.tgz", + "integrity": "sha512-Hhmwd6CInZ3dwpuGTF8fJG6yoWmsToE+vYgD4nytZVxcu1ulHpUQRAB1UJ8+N1Am3Mz4+xOByoQoSZf4D+CpkA==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.25.12.tgz", + "integrity": "sha512-VJ+sKvNA/GE7Ccacc9Cha7bpS8nyzVv0jdVgwNDaR4gDMC/2TTRc33Ip8qrNYUcpkOHUT5OZ0bUcNNVZQ9RLlg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.25.12.tgz", + "integrity": "sha512-6AAmLG7zwD1Z159jCKPvAxZd4y/VTO0VkprYy+3N2FtJ8+BQWFXU+OxARIwA46c5tdD9SsKGZ/1ocqBS/gAKHg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.25.12.tgz", + "integrity": "sha512-5jbb+2hhDHx5phYR2By8GTWEzn6I9UqR11Kwf22iKbNpYrsmRB18aX/9ivc5cabcUiAT/wM+YIZ6SG9QO6a8kg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.25.12.tgz", + "integrity": "sha512-N3zl+lxHCifgIlcMUP5016ESkeQjLj/959RxxNYIthIg+CQHInujFuXeWbWMgnTo4cp5XVHqFPmpyu9J65C1Yg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.25.12.tgz", + "integrity": "sha512-HQ9ka4Kx21qHXwtlTUVbKJOAnmG1ipXhdWTmNXiPzPfWKpXqASVcWdnf2bnL73wgjNrFXAa3yYvBSd9pzfEIpA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.25.12.tgz", + "integrity": "sha512-gA0Bx759+7Jve03K1S0vkOu5Lg/85dou3EseOGUes8flVOGxbhDDh/iZaoek11Y8mtyKPGF3vP8XhnkDEAmzeg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.25.12.tgz", + "integrity": "sha512-TGbO26Yw2xsHzxtbVFGEXBFH0FRAP7gtcPE7P5yP7wGy7cXK2oO7RyOhL5NLiqTlBh47XhmIUXuGciXEqYFfBQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.25.12.tgz", + "integrity": "sha512-lPDGyC1JPDou8kGcywY0YILzWlhhnRjdof3UlcoqYmS9El818LLfJJc3PXXgZHrHCAKs/Z2SeZtDJr5MrkxtOw==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.25.12.tgz", + "integrity": "sha512-8bwX7a8FghIgrupcxb4aUmYDLp8pX06rGh5HqDT7bB+8Rdells6mHvrFHHW2JAOPZUbnjUpKTLg6ECyzvas2AQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.25.12.tgz", + "integrity": "sha512-0y9KrdVnbMM2/vG8KfU0byhUN+EFCny9+8g202gYqSSVMonbsCfLjUO+rCci7pM0WBEtz+oK/PIwHkzxkyharA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.25.12.tgz", + "integrity": "sha512-h///Lr5a9rib/v1GGqXVGzjL4TMvVTv+s1DPoxQdz7l/AYv6LDSxdIwzxkrPW438oUXiDtwM10o9PmwS/6Z0Ng==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.25.12.tgz", + "integrity": "sha512-iyRrM1Pzy9GFMDLsXn1iHUm18nhKnNMWscjmp4+hpafcZjrr2WbT//d20xaGljXDBYHqRcl8HnxbX6uaA/eGVw==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.25.12.tgz", + "integrity": "sha512-9meM/lRXxMi5PSUqEXRCtVjEZBGwB7P/D4yT8UG/mwIdze2aV4Vo6U5gD3+RsoHXKkHCfSxZKzmDssVlRj1QQA==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.25.12.tgz", + "integrity": "sha512-Zr7KR4hgKUpWAwb1f3o5ygT04MzqVrGEGXGLnj15YQDJErYu/BGg+wmFlIDOdJp0PmB0lLvxFIOXZgFRrdjR0w==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.25.12.tgz", + "integrity": "sha512-MsKncOcgTNvdtiISc/jZs/Zf8d0cl/t3gYWX8J9ubBnVOwlk65UIEEvgBORTiljloIWnBzLs4qhzPkJcitIzIg==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.25.12.tgz", + "integrity": "sha512-uqZMTLr/zR/ed4jIGnwSLkaHmPjOjJvnm6TVVitAa08SLS9Z0VM8wIRx7gWbJB5/J54YuIMInDquWyYvQLZkgw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.25.12.tgz", + "integrity": "sha512-xXwcTq4GhRM7J9A8Gv5boanHhRa/Q9KLVmcyXHCTaM4wKfIpWkdXiMog/KsnxzJ0A1+nD+zoecuzqPmCRyBGjg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.25.12.tgz", + "integrity": "sha512-Ld5pTlzPy3YwGec4OuHh1aCVCRvOXdH8DgRjfDy/oumVovmuSzWfnSJg+VtakB9Cm0gxNO9BzWkj6mtO1FMXkQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.25.12.tgz", + "integrity": "sha512-fF96T6KsBo/pkQI950FARU9apGNTSlZGsv1jZBAlcLL1MLjLNIWPBkj5NlSz8aAzYKg+eNqknrUJ24QBybeR5A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.25.12.tgz", + "integrity": "sha512-MZyXUkZHjQxUvzK7rN8DJ3SRmrVrke8ZyRusHlP+kuwqTcfWLyqMOE3sScPPyeIXN/mDJIfGXvcMqCgYKekoQw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.25.12.tgz", + "integrity": "sha512-rm0YWsqUSRrjncSXGA7Zv78Nbnw4XL6/dzr20cyrQf7ZmRcsovpcRBdhD43Nuk3y7XIoW2OxMVvwuRvk9XdASg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.25.12.tgz", + "integrity": "sha512-3wGSCDyuTHQUzt0nV7bocDy72r2lI33QL3gkDNGkod22EsYl04sMf0qLb8luNKTOmgF/eDEDP5BFNwoBKH441w==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.25.12.tgz", + "integrity": "sha512-rMmLrur64A7+DKlnSuwqUdRKyd3UE7oPJZmnljqEptesKM8wx9J8gx5u0+9Pq0fQQW8vqeKebwNXdfOyP+8Bsg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.25.12.tgz", + "integrity": "sha512-HkqnmmBoCbCwxUKKNPBixiWDGCpQGVsrQfJoVGYLPT41XWF8lHuE5N6WhVia2n4o5QK5M4tYr21827fNhi4byQ==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.25.12.tgz", + "integrity": "sha512-alJC0uCZpTFrSL0CCDjcgleBXPnCrEAhTBILpeAp7M/OFgoqtAetfBzX0xM00MUsVVPpVjlPuMbREqnZCXaTnA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/esbuild": { + "version": "0.25.12", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.12.tgz", + "integrity": "sha512-bbPBYYrtZbkt6Os6FiTLCTFxvq4tt3JKall1vRwshA3fdVztsLAatFaZobhkBC8/BrPetoa0oksYoKXoG4ryJg==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.25.12", + "@esbuild/android-arm": "0.25.12", + "@esbuild/android-arm64": "0.25.12", + "@esbuild/android-x64": "0.25.12", + "@esbuild/darwin-arm64": "0.25.12", + "@esbuild/darwin-x64": "0.25.12", + "@esbuild/freebsd-arm64": "0.25.12", + "@esbuild/freebsd-x64": "0.25.12", + "@esbuild/linux-arm": "0.25.12", + "@esbuild/linux-arm64": "0.25.12", + "@esbuild/linux-ia32": "0.25.12", + "@esbuild/linux-loong64": "0.25.12", + "@esbuild/linux-mips64el": "0.25.12", + "@esbuild/linux-ppc64": "0.25.12", + "@esbuild/linux-riscv64": "0.25.12", + "@esbuild/linux-s390x": "0.25.12", + "@esbuild/linux-x64": "0.25.12", + "@esbuild/netbsd-arm64": "0.25.12", + "@esbuild/netbsd-x64": "0.25.12", + "@esbuild/openbsd-arm64": "0.25.12", + "@esbuild/openbsd-x64": "0.25.12", + "@esbuild/openharmony-arm64": "0.25.12", + "@esbuild/sunos-x64": "0.25.12", + "@esbuild/win32-arm64": "0.25.12", + "@esbuild/win32-ia32": "0.25.12", + "@esbuild/win32-x64": "0.25.12" + } + }, + "node_modules/yaml": { + "version": "2.8.3", + "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.8.3.tgz", + "integrity": "sha512-AvbaCLOO2Otw/lW5bmh9d/WEdcDFdQp2Z2ZUH3pX9U2ihyUY0nvLv7J6TrWowklRGPYbB/IuIMfYgxaCPg5Bpg==", + "license": "ISC", + "bin": { + "yaml": "bin.mjs" + }, + "engines": { + "node": ">= 14.6" + }, + "funding": { + "url": "https://github.com/sponsors/eemeli" + } + } + } +} diff --git a/plugins/package.json b/plugins/package.json new file mode 100644 index 00000000..886c688a --- /dev/null +++ b/plugins/package.json @@ -0,0 +1,14 @@ +{ + "name": "irsa-myst-plugins", + "type": "module", + "private": true, + "scripts": { + "build": "esbuild src/notebook-gallery.mjs --bundle --platform=node --format=esm --external:yaml --outfile=dist/notebook-gallery.mjs" + }, + "dependencies": { + "yaml": "^2.7.0" + }, + "devDependencies": { + "esbuild": "^0.25.0" + } +} diff --git a/plugins/src/notebook-gallery.mjs b/plugins/src/notebook-gallery.mjs new file mode 100644 index 00000000..407a3207 --- /dev/null +++ b/plugins/src/notebook-gallery.mjs @@ -0,0 +1,121 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import process from 'node:process'; +import YAML from 'yaml'; + +// myst build always runs from the repo root +const REPO_ROOT = process.cwd(); + +// Create a path-keyed cache so multiple directives calls sharing the same +// metadata file only trigger one disk read per build. +const metadataCache = new Map(); + +function loadMetadata(metadataPath) { + if (metadataCache.has(metadataPath)) return metadataCache.get(metadataPath); + try { + const metadataText = fs.readFileSync(metadataPath, 'utf8'); + const metadataList = /\.json$/i.test(metadataPath) + ? JSON.parse(metadataText) : YAML.parse(metadataText); + // convert [{file, title, description}] to {file: {file, title, description}} for easy lookup + const metadataMap = Object.fromEntries(metadataList.map(e => [e.file, e])); + metadataCache.set(metadataPath, metadataMap); + return metadataMap; + } catch { + metadataCache.set(metadataPath, null); + return null; + } +} + + +// MyST markdown templates — kept separate from logic for readability +const clickableCardMyst = (link, title, description) => [ + '```{card}', + `:link: ${link}`, + `:header: [${title} →](${link})`, + description, + '```', +].join('\n'); + +const unrecognizedCardMyst = (filePath) => [ + '```{card}', + ':header: ⚠️ _Unrecognised notebook_', + `Could not find \`${filePath}\``, + '```', +].join('\n'); + +const metadataErrorMyst = (metadataPath) => [ + `:::{error} \`notebook-gallery\` error`, + `Could not read metadata from \`${metadataPath}\``, + ':::', +].join('\n'); + +const gridOfCardsMyst = (cards) => [ + '````{grid} 1 2 2 3', + ...cards, + '````', +].join('\n\n'); +// End of MyST markdown templates + + +/** + * MyST directive: notebook-gallery + * + * Renders a list of notebooks as gallery: responsive grid of clickable cards. + * + * Arg: + * Path to a YAML or JSON metadata file, relative to the repo root. + * Must be a list of objects; each object must have at least the following fields: + * - title {string} Display title for the card header + * - file {string} Path to the notebook relative to the repo root + * - description {string} One-sentence summary shown in the card body + * + * Body: + * One notebook file path (relative to the repo root) per line. This must + * match the `file` field in the metadata. Lines starting with # are + * treated as comments. + * + * Example: + * ```{notebook-gallery} metadata.yml + * tutorials/abc.md + * tutorials/def.md + * ``` + */ +const notebookGalleryDirective = { + name: 'notebook-gallery', + arg: { type: String, required: true }, + body: { type: String, required: true }, + + run(data, vfile, ctx) { + // Resolve the metadata file path relative to the repo root and load it + const metadataPath = path.resolve(REPO_ROOT, data.arg); + const metadataByFile = loadMetadata(metadataPath); + + if (!metadataByFile) { + return ctx.parseMyst(metadataErrorMyst(metadataPath)).children; + } + + // Links in cards must be relative to the file that contains this directive + const callerDir = path.dirname(path.resolve(vfile.path)); + + // Parse the body: one notebook path per line, skip blank lines and comments + const notebookFilePaths = data.body.split('\n') + .map(l => l.trim()) + .filter(l => l && !l.startsWith('#')); + + // Build a MyST card string for each notebook path + const cards = notebookFilePaths.map(filePath => { + const meta = metadataByFile[filePath]; + if (!meta) return unrecognizedCardMyst(filePath); + const link = path.relative(callerDir, path.resolve(REPO_ROOT, filePath)); + const title = meta.title ?? path.basename(filePath, path.extname(filePath)); // fall back to filename + const description = meta.description ?? ''; // fall back to no description + return clickableCardMyst(link, title, description); + }); + + // Wrap all cards in a grid, parse the combined MyST string, and return the + // resulting AST nodes to be inserted in place of this directive. + return ctx.parseMyst(gridOfCardsMyst(cards)).children; + }, +}; + +export default { name: 'IRSA Notebook Gallery', directives: [notebookGalleryDirective] }; diff --git a/tox.ini b/tox.ini index 0b494a75..f8a84bb8 100644 --- a/tox.ini +++ b/tox.ini @@ -93,6 +93,9 @@ commands = buildhtml: bash -c "echo 'Notebooks ignored (not tested/executed) in this job:'; cat ignore_execute | sort | uniq" + # Build the MyST plugin bundle before building the site with jupyter-book + buildhtml: bash -c "cd plugins && npm install && npm run build" + # Using srtict so we fail with trackbacks and debug mode to have a richer log # For full build we disable parallel runs to easy server load buildhtml: bash -c "jupyter-book build --execute --execute-parallel 1 --html --strict -d" diff --git a/tutorials/euclid/euclid.md b/tutorials/euclid/euclid.md index 42bc6835..63cb0d71 100644 --- a/tutorials/euclid/euclid.md +++ b/tutorials/euclid/euclid.md @@ -12,69 +12,15 @@ Data products include: - Catalogs of MER objects, photometric redshifts and classifications (PHZ), and spectroscopic redshifts and line measurements (SPE); - Merged Objects Catalog (created by IRSA) containing the MER, PHZ, and SPE catalogs in a single HATS Catalog. -````{grid} 1 2 2 3 - -```{card} -:link: 1_Euclid_intro_MER_images.md -:header: [MER Image Mosaics →](1_Euclid_intro_MER_images.md) -Retrieve both a full MER mosaic image and multi-wavelength cutouts, then subtract the background from the cutouts and extract sources. -``` - -```{card} -:link: 3_Euclid_intro_1D_spectra.md -:header: [SIR 1D Spectra →](3_Euclid_intro_1D_spectra.md) -Load a galaxy spectrum and plot it. Understand the wavelength, flux, and mask values. -``` - -```{card} -:link: 2_Euclid_intro_MER_catalog.md -:header: [MER Catalogs →](2_Euclid_intro_MER_catalog.md) - -Explore the columns in the MER final catalog, query for stars, and create a color-magnitude diagram. -``` - -```{card} -:link: 4_Euclid_intro_PHZ_catalog.md -:header: [PHZ Catalogs →](4_Euclid_intro_PHZ_catalog.md) - -Join the PHZ and MER catalogs to query galaxies with quality redshifts in a box region, create MER mosaic cutouts with catalog overlays, and plot the brightest galaxy's SIR spectrum. -``` - -```{card} -:link: 5_Euclid_intro_SPE_catalog.md -:header: [SPE Catalogs →](5_Euclid_intro_SPE_catalog.md) - -Join the SPE and MER catalogs and query for galaxies with H-alpha line detections, then plot the SIR spectrum of a galaxy with a high SNR H-alpha line measurement. -``` - -```{card} -:link: merged-objects-hats-catalog/1-euclid-q1-hats-intro.md -:header: [Merged Objects Catalog →](merged-objects-hats-catalog/1-euclid-q1-hats-intro.md) -Introduction: Understand the content and format of the Euclid Q1 Merged Objects HATS Catalog, then perform a basic query. -``` - -```{card} -:link: merged-objects-hats-catalog/4-euclid-q1-hats-magnitudes.md -:header: [Merged Objects Catalog →](merged-objects-hats-catalog/4-euclid-q1-hats-magnitudes.md) -Magnitudes: Review the types of flux measurements available, load template-fit and aperture magnitudes, and plot distributions and comparisons for different object types. -``` - -```{card} -:link: euclid-cloud-access.md -:header: [Cloud Access →](euclid-cloud-access.md) -Browse the on-cloud copy of Q1, then efficiently retrieve a MER mosaic cutout and a SIR spectrum. -``` - -```{card} -:link: euclid_clusters_tutorial.md -:header: [Galaxy Clusters →](euclid_clusters_tutorial.md) -Use Euclid Q1 data to detect and validate a galaxy cluster. -``` - -```{card} -:link: Euclid_ERO.md -:header: [ERO Star Clusters →](Euclid_ERO.md) -Create multi-wavelength ERO image cutouts of a globular cluster, extract sources, and measure photometry. Match Gaia sources with Euclid ERO catalogs, then visualize with Firefly. -``` - -```` +```{notebook-gallery} notebook_metadata.yml +tutorials/euclid/1_Euclid_intro_MER_images.md +tutorials/euclid/3_Euclid_intro_1D_spectra.md +tutorials/euclid/2_Euclid_intro_MER_catalog.md +tutorials/euclid/4_Euclid_intro_PHZ_catalog.md +tutorials/euclid/5_Euclid_intro_SPE_catalog.md +tutorials/euclid/merged-objects-hats-catalog/1-euclid-q1-hats-intro.md +tutorials/euclid/merged-objects-hats-catalog/4-euclid-q1-hats-magnitudes.md +tutorials/euclid/euclid-cloud-access.md +tutorials/euclid/euclid_clusters_tutorial.md +tutorials/euclid/Euclid_ERO.md +``` \ No newline at end of file diff --git a/tutorials/simulated-data/simulated.md b/tutorials/simulated-data/simulated.md index ccf579a3..9e8e4c3f 100644 --- a/tutorials/simulated-data/simulated.md +++ b/tutorials/simulated-data/simulated.md @@ -5,12 +5,10 @@ Because this collection is heterogeneous in coverage, structure, and intended us Access methods are tailored to the structure and scale of each product. These tutorials are designed to help users get started with accessing, visualizing, and analyzing simulated datasets hosted at IRSA. -- [Roman HLSS Number Density](roman_hlss_number_density.md) - Query the catalog and derive galaxy number density. - -- [OpenUniverse2024 Roman Coadds](openuniverse2024_roman_simulated_wideareasurvey.md) - Access OpenUniverse2024 wide-area simulated survey data. - -- [OpenUniverse2024 Visualization](OpenUniverse2024Preview_Firefly.md) - Use Firefly to get an overview of survey structure and visualize content. - -- [OpenUniverse2024 Time Domain](openuniverse2024_roman_simulated_timedomainsurvey.md) - Access and analyze the simulated time-domain OpenUniverse2024 survey. - -- [CosmoDC2 Data Access](cosmoDC2_TAP_access.md) - Access, query, and visualize the CosmoDC2 catalog. +```{notebook-gallery} notebook_metadata.yml +tutorials/simulated-data/roman_hlss_number_density.md +tutorials/simulated-data/openuniverse2024_roman_simulated_wideareasurvey.md +tutorials/simulated-data/OpenUniverse2024Preview_Firefly.md +tutorials/simulated-data/openuniverse2024_roman_simulated_timedomainsurvey.md +tutorials/simulated-data/cosmoDC2_TAP_access.md +``` diff --git a/tutorials/spherex/spherex.md b/tutorials/spherex/spherex.md index c6a9ced9..86faff75 100644 --- a/tutorials/spherex/spherex.md +++ b/tutorials/spherex/spherex.md @@ -6,10 +6,9 @@ Its science goals span cosmology, galaxy evolution, and the interstellar medium, SPHEREx data releases include weekly [Quick Release spectral image products](https://caltech-ipac.github.io/spherex-archive-documentation/spherex-data-products/) (multi-extension FITS files containing calibrated near-infrared surface brightness, variance, flags, modeled backgrounds, PSFs, and wavelength WCS) along with ancillary calibration and metadata files such as gain matrices, dark current maps, solid angle pixel maps, and detailed spectral WCS products for each detector. -- [Data Overview](spherex_intro.md) - Find Spectral Images that overlap a sky position and understand the structure of the Multi-Extension FITS files. - -- [Spectral Image Cutouts](spherex_cutouts.md) - Generate and work with spatial and spectral cutouts. - -- [PSF Models](spherex_psf.md) - Understand how SPHEREx point spread function (PSF) information is organized and accessed. - -- [Source Discovery Tool](spherex_source_discovery/spherex_source_discovery_tool_demo.md) - Discover, extract, and visualize sources from SPHEREx Spectral Images. +```{notebook-gallery} notebook_metadata.yml +tutorials/spherex/spherex_intro.md +tutorials/spherex/spherex_cutouts.md +tutorials/spherex/spherex_psf.md +tutorials/spherex/spherex_source_discovery/spherex_source_discovery_tool_demo.md +``` diff --git a/tutorials/techniques-and-tools/techniques.md b/tutorials/techniques-and-tools/techniques.md index 48714562..9c2b43fd 100644 --- a/tutorials/techniques-and-tools/techniques.md +++ b/tutorials/techniques-and-tools/techniques.md @@ -3,10 +3,9 @@ This section gathers tutorials that go beyond mission specific data access and analysis to explore broader methods, workflows, and utilities applicable across missions and datasets. This collection includes guidance on cloud-based data access, use of powerful big data tools such as lsdb, interactive visualization tools, and computational techniques like parallel image convolution. -- [Cloud Access](cloud-access-intro.md) — Access and work with data served by IRSA from AWS S3 cloud storage. - -- [HATS with LSDB](irsa-hats-with-lsdb.md) - Use LSDB for user-friendly cross matching and querying of HATS Collections. - -- [SED Visualization](SEDs_in_Firefly.md) - Explore and interactively visualize multi-wavelength SEDs using Firefly. - -- [Parallelization](Parallelize_Convolution.md) - Learn parallelization techniques to speed up operations on large astronomical images. +```{notebook-gallery} notebook_metadata.yml +tutorials/techniques-and-tools/cloud-access-intro.md +tutorials/techniques-and-tools/irsa-hats-with-lsdb.md +tutorials/techniques-and-tools/SEDs_in_Firefly.md +tutorials/techniques-and-tools/Parallelize_Convolution.md +``` diff --git a/tutorials/wise/wise.md b/tutorials/wise/wise.md index b8576a51..73594302 100644 --- a/tutorials/wise/wise.md +++ b/tutorials/wise/wise.md @@ -6,12 +6,10 @@ After exhausting its cryogen, the mission was repurposed as NEOWISE in 2013 to d WISE and NEOWISE data are released publicly through the NASA/IPAC Infrared Science Archive (IRSA), including calibrated images, source catalogs, and single-exposure source tables that together enable multi-epoch photometry, light curves, and motion studies for a wide range of astrophysical and Solar System applications. Successive NEOWISE data releases, issued with annual updates, provided users with increasingly deep coverage and time-domain information across the infrared sky. -- [AllWISE Images](sia_allwise_atlas.md) - Retrieve coadded images and make coordinate-based cutouts. - -- [AllWISE Source Catalog](wise-allwise-catalog-demo.md) - Query the Parquet copy of the AllWISE Source Catalog. - -- [NEOWISE Solar System Objects](NEOWISE_light_curve_demo.md) - Visualize and analyze light curves of Solar System objects using Firefly. - -- [NEOWISE Strategies](neowise-source-table-strategies.md) - Learn efficient strategies for accessing and handling this very large Parquet dataset. - -- [NEOWISE Light Curves](neowise-source-table-lightcurves.md) - Build multi-epoch photometric light curves for given coordinates. +```{notebook-gallery} notebook_metadata.yml +tutorials/wise/sia_allwise_atlas.md +tutorials/wise/wise-allwise-catalog-demo.md +tutorials/wise/NEOWISE_light_curve_demo.md +tutorials/wise/neowise-source-table-strategies.md +tutorials/wise/neowise-source-table-lightcurves.md +```