fix(motoko): restore missing pages, fix nav order, rewrite stale links#261
Merged
Conversation
This was referenced May 18, 2026
- Restore language-manual.md, style-guide.md, compiler-ref.md to reference/ (all three were silently dropped by the sync script which only checked directories, not top-level .md files) - Restructure fundamentals/ from a flat 53-file list to the correct grouped layout (basic-syntax/, actors/, types/, declarations/, control-flow/) matching the source directory tree; this also fixes the silent page loss caused by two files mapping to the same flat name after stripping numeric prefixes (both basic-syntax/functions.md and types/function-types.md now exist) - Update sidebar.mjs with explicit ordered groups for all Motoko sections so the navbar reflects the logical learning progression instead of alphabetical order - Add URL rewrite table to postprocess-motoko.mjs: all internetcomputer.org/docs/ links in synced pages are now rewritten to internal developer-docs paths - Remove dead docs.motoko.org redirect (was used for language-manual links; that page is now included directly) - Extend sync-motoko.sh guard to check top-level .md files, not just directories - Fix cross-reference in message-execution-properties.md to new actors/ path - Add .docs-plan/motoko-repo-sync-proposal.md: roadmap for eliminating the sync transformation entirely by adopting Starlight-native conventions in the Motoko repo
…RESOLVED-EXTERNAL warnings Add three robustness improvements to the Motoko docs sync pipeline: 1. Slug collision detector: a Python3 check runs before any files are copied and warns when two source files would produce the same basename after stripping numeric prefixes. Catches regressions if the upstream repo gains new files that collide with existing slugs. 2. icp-features uses autogenerate in sidebar.mjs so new upstream pages appear in the nav automatically without requiring sidebar.mjs edits on every sync. The postprocess script converts sidebar_position → sidebar.order for these files so Starlight orders them correctly. 3. UNRESOLVED-EXTERNAL warnings: postprocess-motoko.mjs emits a warning when an internetcomputer.org/docs URL has no entry in the rewrite table, surfaces the count in a final error line, and sync-motoko.sh captures those lines into SYNC_WARNINGS so they appear in the post-sync summary.
…irst, changelog last in Reference
…whitespace in regex
…to remark-include-file
…r at sync time, update proposal
… rewrite
- Copy orthogonal-persistence/index.md as overview.md during sync (has real
content comparing both persistence modes; was silently dropped before)
- Fix rewriteLink index handler to produce full path URL and resolve ./index.md
links to overview.md when that file exists in the same directory
- Remove opIdx deletion from postprocess (overview.md must survive postprocess)
- Add overview entry to sidebar for Orthogonal persistence section
- Fix stale sync-motoko.sh comment ("expand file-embeds" → "rewrite file-embed paths")
- Update motoko-repo-sync-proposal: fix Problem 3 inconsistency, add missing
upstream items (overview.md rename, frontmatter, sidebar_position gaps),
fix migration step numbering
…ge 7 Documents the history of the Docusaurus moc interpreter feature, which flags exist (run, no-repl, name=, include=, _include=), their distribution across synced vs excluded sections, and confirmed current behaviour in Starlight (all silently ignored by Expressive Code). Adds targeted sed commands for upstream cleanup scoped to synced sections only.
- Problem/Change 8: ~146 internal relative links break after prefix removal; upstream must update all cross-references atomically with the renames - Problem/Change 9: changelog.mdx contains a Docusaurus md reference block that fetches Changelog.md from GitHub; must be replaced upstream - Problem/Change 10: two motoko-tooling links in comments.md need external URL - Fix Problem 6: clarify :::warn (invalid) vs :::warning (valid Starlight type); correct :::info count to 21 in synced sections (not 60 full-tree) - Fix Change 6: add note not to change :::warning - Fix Change 7: add mandatory space-variant normalization step before REPL sed - Fix Change 5: complete the external URL table with 5 missing entries that are actually present in source; note 2 entries no longer in source
…x, and tooling clarifications - §4/5 (mo:core): note that motoko team's core/ sync script can be dropped once mops.one links replace ./core/<Module>.md in the upstream source; add mops.one URL pattern note - §5 (portal URLs): expand mapping table from 17 to 25 entries with specific section anchors (query-calls, update-calls, composite-queries, upgrade, storage variants, ic-interface-spec sub-file anchors for global-timer, inspect-message, heartbeat, raw_rand) - §8 (internal links): add "pure functions" content quality note — link from printing-values.md lands at function-types.md with no matching anchor - §9 (changelog): revise approach to copy Changelog.md into doc/md at release time rather than fetching at post-process time; this prevents version drift at the pinned submodule commit - §10 (motoko-tooling): note that docs.motoko.org does not exist; replace proposed URL with github.com/dfinity/motoko/releases; document why motoko-tooling is excluded and what developer-docs could add hand-written - Appendix: add Docusaurus-deprecation implications and base-core-migration sidebar decision (keep current approach: hidden, linked from index)
…n to remark-include-file - Add <motokoRoot> placeholder resolving to .sources/motoko/ so synced docs can reference Changelog.md directly without any copy step - Add md language inline inclusion: a code fence with lang=md and file= is replaced with parsed AST children (rendered prose), not a code block; uses this.parse() so the same remark plugins apply to included content - Update docs(plan): revise changelog approach — upstream stub uses file=<motokoRoot>/Changelog.md, no release automation needed since the file is always correct at the pinned submodule commit
…mentals stays explicit - Correct changelog sidebar.order from 3 to 6 (last in reference section) - Add renumbering table for reference/ section: positions 1–3 and 14–16 live in separate legacy namespaces; after the rename all six files share one directory and need sequential order 1–6 (language-manual=1, changelog=6) - Clarify that fundamentals/ cannot use autogenerate after prefix removal: subdirectory names sort alphabetically (actors, basic-syntax, ...) which does not match the desired display order; explicit sidebar.mjs list stays - Limit the "switch to autogenerate" recommendation to icp-features and reference only; update migration step 3 and "what the sync becomes" section
The Motoko sidebar section was 120 lines inline in sidebar.mjs. Moving it to its own file makes it easier to find and update when pages are added, removed, or reordered after a submodule bump. No behaviour change. fundamentals/ stays as an explicit items list (subdirectory order cannot be derived automatically after numeric prefix removal). icp-features/ and reference/ stay on autogenerate.
…erate
sidebar-motoko.mjs is now marked TRANSITIONAL: it exists only until
upstream adds metadata index.md files to each fundamentals/ subdir
(with sidebar.order) and sidebar: { hidden: true } to
base-core-migration.md. Once those land, the file can be deleted and
replaced with a single autogenerate directive.
Task 11 in the sync proposal documents the exact upstream changes
required and the corresponding developer-docs rsync/sidebar cleanup.
Prepares developer-docs for the upstream caffeinelabs/motoko reorganization: - sync-motoko.sh: replaced with simplified rsync (no postprocess step). Structure guard fails early if the old numeric-prefix directories are still present, so the weekly CI fails cleanly until the upstream PR merges. Uses --exclude='/index.md' (anchored) for fundamentals/ to keep subdir ordering stubs (index.md files with sidebar.order) while dropping the top-level section landing page. - postprocess-motoko.mjs: marked deprecated. No longer called; will be deleted in the same PR that merges the synced content. - motoko-repo-sync-proposal.md: fix --exclude='index.md' → --exclude='/index.md' for fundamentals in "What the sync becomes"; add upstream agent instructions for _category_.yml and index.md files; update migration path to reflect the sync script and postprocess deprecation being already done.
sync-motoko.sh and postprocess-motoko.mjs were replaced/deprecated in the previous commit but that would break the automated weekly sync CI permanently until the upstream caffeinelabs/motoko reorganization lands. Reverted both to the working versions from main. The proposal migration path is updated to clearly distinguish what is done in PR 261 from what belongs in a separate PR that is only merged after the upstream reorganization PR merges. The simplified sync-motoko.sh and postprocess deletion are documented there with an explicit warning not to merge early.
…sion) The previous revert accidentally used `git checkout main --` which brought back the old flat-copy version of sync-motoko.sh (from before this branch introduced the subdirectory structure for fundamentals/). The correct version is from commit 81c82b1, which preserves basic-syntax/, actors/, types/, declarations/, control-flow/ subdirs and handles the functions.md slug collision. Restored both scripts from 81c82b1.
… §2 recommendation Three corrections: - overview.md rename reason: was "to avoid rsync dropping it via --exclude='index.md'" which is wrong with the anchored --exclude='/index.md'. Correct reason: clarity — it's a content page, not an ordering stub. - _category_.yml note: the exclude flag is a transitional safety net; the correct end state is these files are deleted from upstream entirely once index.md stubs exist (unconditionally redundant for both Starlight and Docusaurus at that point). - §2 fundamentals recommendation: was "keep explicit list" as the recommendation. Updated to reflect that Change §11 enables full autogenerate via the anchored exclude, which is the recommended path. Same corrections applied to the upstream issue (caffeinelabs/motoko#6131).
The sync script already handles numeric prefix stripping via strip_num() and link rewrites via syncRenames. Mark §1 (prefix removal) and §8 (link updates) as optional for the current sync — only required if the goal is a fully transform-free rsync. Add a priority guide to the migration path.
…ore/base dirs A core maintainer confirmed mops.one should be the single source of truth for mo:core docs. Update §4 to make directory removal an explicit recommendation with three concrete steps: replace links, remove core/ and its CI generation step, consider removing base/.
…alternative Top-level section index.md files are Docusaurus nav artifacts; upstream should delete them, not leave them for us to exclude. Simplify the rsync block to reflect this. Add a symlink alternative section documenting that the copy step can be replaced with committed symlinks once all upstream changes land, with the auditability tradeoff noted.
…eletes those files
- Fix two broken anchors in stable-memory.md: #heap-memory → #heap-wasm-linear-memory, #motoko-storage-handling → #motoko-true-orthogonal-persistence (old portal slugs differed from developer-docs headings) - Fix postprocess anchor rewrite: try full URL (with anchor) before stripping so explicit anchor-remap entries in the table take effect; add the two storage anchor mappings - Fix mo-doc links in comments.md: docs.motoko.org does not exist; redirect to caffeinelabs/motoko/releases - Fix org name throughout: dfinity/motoko → caffeinelabs/motoko (repo was transferred; affects releases URL and Changelog.md refs)
marc0olo
force-pushed
the
docs/motoko-sync-fixes
branch
from
8a54d6a to
2e0a8d9
Compare
Add mo-doc to the developer-tools overview under Command-line tools, replacing the GitHub releases links in the Motoko comments page with an internal reference.
…al developer-tools page Replace the GitHub releases fallback with /developer-tools/#mo-doc in both the motoko-tooling relative-path redirect and the external URL map for the non-existent docs.motoko.org domain.
…m CDKs Motoko is ICP's native language and doesn't require a CDK. Move it out of the CDKs section into a dedicated ## Motoko section, pulling the VS Code extension (from Editor tooling) and mo-doc together under one roof. CDKs now accurately describes only language bridges (Rust, Azle, Kybra, etc.).
…on, not precede it
…o-doc after upstream §10 lands
…andling in symlink section Add `title:`/`description:` migration to §2 as a universal requirement for all 68 synced files (not just the 3 files with no frontmatter). Document that this is a hard prerequisite for the symlink approach and clarify the copy-vs-symlink tradeoff around `docs.internetcomputer.org/` link rewriting.
…and external URL regex Add specific anchored entries for #global-timer, #heartbeat, #system-api-inspect-message, and #ic-raw_rand so these links route to the correct canister-interface / management-canister sub-pages instead of falling through to the index page. Also expand extLinkRe to match docs.internetcomputer.org/ and docs.motoko.org domains alongside internetcomputer.org/docs/, so rewrites fire correctly when upstream §5/§10 changes land.
Add three concrete reasons (internal links, anchor routing, auditability) to replace the vague "preferred for auditability" note. Captures why absolute external URLs break local dev and staging even when anchors are correct.
The remark-include-file plugin resolves <motokoExamples> to .sources/motoko/doc/md/examples/ at build time. Without the submodule, all file-embed blocks throw and pages like error-handling, timers, and caller-identification render with empty content in preview and production.
…xamples Nat→Int is a safe upgrade because Nat <: Int (subtype relationship). The example was factually incorrect per the Motoko compatibility docs. Replaced with Int→Float and Nat→Text, which are genuinely incompatible. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove stale "keep in sync" comment pointing to deleted 1-home.mdx - Change "AI agents" to "developers" to match doc/md/index.md in motoko repo - Update Upstream comment to reflect the file now mirrors .sources/motoko
adamspofford-dfinity
approved these changes
May 19, 2026
This was referenced May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Comprehensive fix for the Motoko docs sync: restores missing pages, restructures the fundamentals section into subdirectories, extends the sync and build infrastructure, and documents the full upstream roadmap.
Content fixes
.mdfiles):language-manual.md(2,999 lines),style-guide.md,compiler-ref.md. All three now appear underlanguages/motoko/reference/.basic-syntax/,actors/,types/,declarations/,control-flow/subdirectories matching the upstream source layout and the intended learning progression.orthogonal-persistence/overview.md: previously dropped because the sync excluded allindex.mdfiles — the upstream source usesindex.mdfor this page. Now explicitly copied and renamed during sync.basic-syntax/functions.mdandtypes/functions.mdproduced the same flat filename; the oldmvsilently dropped the first. Both now live at distinct paths (basic-syntax/functions.mdandtypes/function-types.md).internetcomputer.org/docs/...links in synced pages are rewritten to internal developer-docs paths via a lookup table. Anchors that changed slug between the old portal and developer-docs (e.g.#heap-memory→#heap-wasm-linear-memory) are handled with explicit anchored entries; the rewrite tries the full URL with anchor before stripping, enabling precise anchor remapping. The IC interface spec is split across sub-pages on developer-docs; anchors like#global-timer,#heartbeat,#system-api-inspect-message, and#ic-raw_randnow route to the correct sub-page (canister-interfaceormanagement-canister) rather than the index. The external URL regex is also expanded to matchdocs.internetcomputer.org/anddocs.motoko.orgdomains alongside the old portal, so rewrites fire automatically when upstream §5/§10 changes land.mo:corelinks:./core/<Module>.mdrelative links (excluded from sync) are redirected tohttps://mops.one/core/docs/<Module>.mo-doclinks:motoko-tooling/relative paths and the non-existentdocs.motoko.orgdomain are now redirected to/developer-tools/#mo-doc. Thedeveloper-toolspage was restructured in this PR to give Motoko its own dedicated section (removed from CDKs) with### mo-docand the Motoko VS Code extension grouped together. The VS Code extension link is updated to the newcaffeinelabs/vscode-motokorepo.data-persistence.mdx:Nat → Intwas listed as an unsafe upgrade change, but it is safe becauseNat <: Int. Replaced withInt → FloatandNat → Text, which are genuinely incompatible per the Motoko compatibility docs.Infrastructure
sync-motoko.sh: preserves subdirectory structure instead of flattening; copies the 3 reference files from the top-leveldoc/md/; guards both top-level files and directories so new upstream content triggers a warning in the automated PR; detects slug collisions before they cause silent data loss; capturesUNRESOLVED-EXTERNALwarnings from postprocess for the PR body.postprocess-motoko.mjs: adds the external URL rewrite table with full-URL-first anchor lookup (handles slug remapping, including ic-interface-spec sub-page routing); rewrites Docusaurus file-embed paths to use the<motokoExamples>placeholder (resolved by the build plugin at build time from the pinned submodule); handles the changelogmd referenceblock; normalizes Starlight aside syntax (:::info→:::note,:::warn→:::caution, strips link URLs from aside titles). Extended to matchdocs.internetcomputer.org/anddocs.motoko.orgdomains so rewrites are ready for upstream §5/§10 changes.plugins/remark-include-file.mjs: extended with<motokoExamples>placeholder (resolves to.sources/motoko/doc/md/examples/at build time),#L<n>-L<m>line-range slicing,<motokoRoot>placeholder, and inline markdown inclusion (mdlanguage fence → rendered prose AST, not a code block). File-embed blocks in synced pages now read directly from the pinned submodule at build time — examples stay live with every submodule bump.sidebar-motoko.mjs: new file. The Motoko sidebar section is extracted fromsidebar.mjsand maintained here as a transitional file. It is explicitly enumerated because the upstream source does not yet have the Starlightsidebar.orderfrontmatter or subdirindex.mdordering stubs needed for autogenerate. The file documents exactly what upstream changes are required and what the replacement one-liner looks like.sidebar.mjs: importsmotokoSidebarfromsidebar-motoko.mjs; the reference section ordering is corrected (language-manual first, changelog last).docs/developer-tools/index.md: Motoko moved out of the CDKs section into its own## Motokosection (it is ICP's native language and does not require a CDK). The Motoko VS Code extension (moved from## Editor tooling) and### mo-docare grouped there. CDKs now accurately describes only language bridges (Rust, Azle, Kybra, etc.).preview-deployment.ymlanddeploy-ic.ymlnow initialize.sources/motokoalongside.sources/examples, fixing empty page renders for pages that use the<motokoExamples>file-embed placeholder.Planning
.docs-plan/motoko-repo-sync-proposal.md: full roadmap for eliminating all sync-time transformations. Documents 11 upstream tasks with a priority guide (7 required for correct rendering, §11 for autogenerate, §1+§8 optional quality-of-life, §7 optional cleanup). Documents known content issues outside the sync scope:dfxreferences in prose (tracked upstream in PR #6069, which must not merge before §1–§11 land) and em-dashes in two files. Documents the copy-over-symlinks decision with three concrete reasons: internal paths validate at build time and work in local dev and staging; anchor routing to developer-docs sub-pages requires knowledge upstream cannot have; submodule bumps produce reviewable diffs. The developer-docs migration path is split into two stages: what is done in this PR, and what belongs in a separateinfra/sync-motoko-v2PR that must only merge after the upstream reorganization lands. Upstream tasks tracked as caffeinelabs/motoko#6131.Sync recommendation
sync from caffeinelabs/motoko doc/md