Skip to content

improvement(library): single JSON-LD source, Markdown articles, and IndexNow pings - #9355

Merged
waleedlatif1 merged 5 commits into
stagingfrom
improvement/library-crawl-structured-data
Oct 24, 2026
Merged

waleedlatif1 merged 5 commits into
stagingfrom
improvement/library-crawl-structured-data

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Library and blog posts emit structured data only as JSON-LD: removed the duplicate microdata BlogPosting (publisher as plain text) and FAQPage from the post layout and FAQ. Markup is unchanged apart from the attributes, an empty <meta> branch, and an empty FAQ wrapper div. The speakable selectors now point at data-speakable hooks instead of the removed itemprops
  • BlogPosting gets @id (<canonical>#article) and isPartOf → the site WebSite node (#website). Authors link to their on-site profile (url plus a shared @id that the author page's Person node uses too), and their external profile stays in sameAs
  • Library articles are served as Markdown at /library/<slug>.md and at /library/<slug> for Accept: text/markdown (afterFiles rewrites, so /library/tags and the feeds are untouched). Responses carry Vary: Accept, are no-store like the HTML page, and send a canonical Link header. Drafts and unknown slugs return a Markdown 404. Post pages advertise <link rel="alternate" type="text/markdown">, and llms.txt notes the .md convention (matching docs)
  • Blog and library post pages advertise their RSS feed. buildPostMetadata now takes the section at every call site
  • IndexNow: a public key file under apps/sim/public, and scripts/ping-indexnow.ts, which submits the library and comparison URLs whose source changed between the previous and new main head (renamed and deleted posts included, drafts never; every comparison when Sim's own profile changes). A new indexnow job runs it after trigger-promote confirms the prod cutover. The job uses continue-on-error with a 5-minute timeout, nothing depends on it, and it skips when nothing changed

Type of Change

  • Improvement

Testing

  • scripts/ping-indexnow.test.ts (6 cases against a real git repo). Mutation-checked: removing the draft guard, rename handling, sim-profile fan-out, or profile-id lookup each turns a case red
  • Dry run of URL collection on the last main release: 5 library URLs, no request sent
  • Extended test-landing-seo-e2e.ts (CI's seo HTTP group): .md and Accept negotiation, Vary, no-store, canonical Link, alternates, /library/tags still HTML, Markdown 404, no microdata, @id/isPartOf/author @id cross-page match, and speakable selectors resolve. The live site fails these checks before this change (verified with curl)
  • Existing tests: content-routing.test.tsx, customers/page.test.tsx, lib/content/* (145 passing), dev-deployment-workflow, desktop-prune-workflow
  • bun run lint, check:comment-hygiene, source-conventions, file-names, site-urls, library-content, client-boundary, boundaries, unused-exports, utils, test-patterns. type-check and check:audits are left to CI

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 23, 2026 11:39pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 16 files

Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.

Turn on auto-fix | Re-trigger cubic

Comment thread .github/workflows/ci.yml Outdated
Comment thread scripts/ping-indexnow.ts Outdated
Comment thread scripts/ping-indexnow.ts Outdated
Comment thread apps/sim/lib/content/seo.ts
Comment thread scripts/ping-indexnow.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@greptile-apps

greptile-apps Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The latest changes appear safe to merge; no new blocking issue was established.

Summary

Adds Markdown library articles, consolidates structured data into JSON-LD, and submits changed library and comparison URLs to IndexNow after production cutover.

  • Post pages use linked structured data for article and author details.
  • Library articles are available as Markdown beside their HTML pages.
  • Production releases send IndexNow the library and comparison URLs they changed.
  • Docs builds fetch history from the commit being built.

Reviews (5) · Last reviewed commit: "fix(library): noindex direct Markdown ar..." · Reviewed by Greptile

Comment thread scripts/ping-indexnow.ts Outdated
Comment thread apps/sim/app/library.md/[slug]/route.ts
… renamed pages, and give customer stories speakable hooks
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed

Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.

Turn on auto-fix | Re-trigger cubic

Comment thread apps/sim/app/library.md/[slug]/route.ts
Comment thread scripts/ping-indexnow.ts Outdated
Comment thread .github/workflows/ci.yml Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 19 files

Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

Turn on auto-fix | Re-trigger cubic

Comment thread .github/workflows/ci.yml Outdated
…loyments and diff IndexNow from the last submitted revision
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 21 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 0753608 into staging Oct 24, 2026
69 checks passed

This branch was successfully deployed

1 active deployment
Preview — c22d4145 Deployed Oct 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant