refactor(core): rename per-datasource org.yaml/ORGANIZATION.md → datasource.yaml/datasource.md by vishalkalbi27 · Pull Request #146 · AgamiAI/agami-core

vishalkalbi27 · 2026-07-25T11:27:59Z

Why

F15 introduced a company-wide organization.yaml (the shared deployment record). That made the two files living inside each profile — named after "org" — ambiguous. This renames them to say what they are: one datasource.

<profile>/org.yaml → <profile>/datasource.yaml
<profile>/ORGANIZATION.md → <profile>/datasource.md

Stacked on top of #143 (F15); base is feat/ace-067-org-context so this PR is only the rename. Once #143 merges, GitHub retargets this to main.

The core rule — "organization" meant three things; two rename, one stays

Concept	Action
Per-datasource model (the file + its `organization:` field / `Organization.datasource` attr)	rename → `datasource.yaml`, field `datasource`
Per-datasource narrative memory (`ORGANIZATION.md`, DB `kind='organization'`)	rename → `datasource.md`, `kind='datasource'`
Company / tenancy (`org_id`, `organization.yaml`, `OrgRecord`, class name `Organization`, the `DatasourceSchemaResult.organization` MCP output field, the explorer "Organization" tab)	unchanged

Two deliberate KEEPs worth calling out:

_schema_payload emits "organization": org.description right next to "datasource": profile — renaming it would collide + break the MCP output contract.
The root company narrative is organization.md (pairs with organization.yaml), not datasource.md — fixed in the third commit after the blanket pass wrongly collapsed both ORGANIZATION.md levels.

What changed

src (~16 files): path literals, the datasource: field + .datasource attr, local vars, the kind='datasource' DB label + its consumers, a 001_serving.sql comment.
Vendored plugins/agami/lib/: regenerated via dev.py sync-lib (not hand-edited).
Skills / shared docs / scripts / HTML, tests (~24 files), docs + CHANGELOG, dev/reset-yamls.sh.
Sample model git mv'd (history preserved).
No back-compat / on-disk migration — pre-existing profiles re-run agami-connect or rename the two files by hand (documented in CHANGELOG). Version bumped to 0.5.0.

Out of scope (intentional)

Pydantic class name Organization; the MCP output key organization; all company/tenancy naming (org_id, organization.yaml, OrgRecord); the explorer "Organization" tab; USER_MEMORY.md.

Testing

uv run dev.py check green (ruff + 1557 tests + gitleaks + lib-drift), run hermetically (AGAMI_ARTIFACTS_DIR=$(mktemp -d)).
End-to-end: fresh agami-connect onboarding of a Salesforce datasource — writes datasource.yaml/datasource.md + root organization.yaml/organization.md, validates clean, review/curate + example-seeding all work, and agami-query reads the renamed model.

🤖 Generated with Claude Code

…asource.yaml/datasource.md The company-wide organization.yaml (F15) made the per-datasource files' "org" naming ambiguous. Rename them to say what they are — one datasource: - <profile>/org.yaml -> <profile>/datasource.yaml - <profile>/ORGANIZATION.md -> <profile>/datasource.md - model root field organization: -> datasource: (pydantic Organization.datasource) - served memory row kind='organization' -> kind='datasource' The company record (organization.yaml, org_id, tenancy), the MCP output field DatasourceSchemaResult.organization, the pydantic class name Organization, and the model-explorer "Organization" tab are intentionally unchanged. The vendored plugins/agami/lib/ slice is regenerated via `dev.py sync-lib`. No on-disk migration: existing profiles must re-run agami-connect or rename the two files by hand (documented in CHANGELOG). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Bump plugin/marketplace/package version to 0.5.0 so the directory-marketplace install lands in a fresh cache dir, and retitle the CHANGELOG section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

ORGANIZATION.md existed at two levels — the artifacts-dir ROOT (the company narrative, beside organization.yaml) and per-profile. The rename correctly made the per-profile file datasource.md, but the blanket pass also renamed the ROOT company narrative, which must instead be organization.md to pair with organization.yaml and stay distinct from the per-datasource datasource.md. Fixes NARRATIVE_FILENAME (org_record.py, the single source of truth) plus the comments/docs/skill that named the company narrative — agami-connect now writes and detects the company narrative at <artifacts_dir>/organization.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This PR completes the “org → datasource” rename for per-profile semantic-model artifacts to avoid ambiguity with the new deployment-wide organization.yaml/organization.md introduced in F15, while explicitly keeping company/tenancy naming and MCP output contract keys unchanged.

Changes:

Renames per-profile model roots and narrative docs (org.yaml → datasource.yaml, ORGANIZATION.md → datasource.md) and updates the model’s root field (organization → datasource).
Renames the per-datasource DB “memory” kind from organization to datasource and updates all readers/writers accordingly.
Updates tests, plugin skill/docs/scripts/templates, and bumps versions to 0.5.0 + CHANGELOG entry.

Reviewed changes

Copilot reviewed 90 out of 91 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
tests/test_units.py	Update fixtures for datasource.yaml
tests/test_table_scope_gate.py	Rename Organization ctor field
tests/test_sensitive_gate.py	Rename Organization ctor field
tests/test_semantic_model_validator_v2.py	Rename Organization ctor field
tests/test_semantic_model_runtime.py	Rename Organization ctor field
tests/test_semantic_model_models.py	Rename Organization ctor field
tests/test_semantic_model_loader.py	Update disk round-trip expectations
tests/test_semantic_model_introspect.py	Assert datasource.yaml exists
tests/test_semantic_model_curate.py	Update docstrings + YAML paths
tests/test_semantic_model_cli.py	Update CLI test fixtures/comments
tests/test_second_order_metrics.py	Rename Organization ctor field
tests/test_render_model_explorer.py	Use datasource.md + datasource.yaml
tests/test_prepare_deploy.py	Stage datasource.yaml in bundles
tests/test_organization_context_e2e.py	Update profile write_tree + paths
tests/test_org_record.py	Update legacy lift + profile scan fixtures
tests/test_org_record_deploy.py	Update deploy fixtures + memory kind
tests/test_org_identity_e2e.py	Update ensure-org-id + legacy fixtures
tests/test_org_draft.py	Update narrative filename + expectations
tests/test_org_cache.py	Update resolver comments/fixtures
tests/test_multi_area_membership.py	Rename Organization ctor field
tests/test_model_store_roundtrip.py	Update memory API + YAML fixtures
tests/test_model_snapshots.py	Snapshot expects datasource.yaml
tests/test_model_deploy.py	Deploy reads datasource.md + datasource.yaml
tests/test_metric_binding_and_prune.py	Rename Organization ctor field
tests/test_metadata_sources.py	Update fixtures + ctor field
tests/test_mcp_harness.py	Update comments + YAML fixtures
tests/test_guard_context.py	Rename Organization ctor field
tests/test_get_datasource_schema_sizing.py	Update YAML root key + filename
tests/test_entity_anchor_and_area_desc.py	Update YAML root key + filename
tests/test_derived_metrics.py	Rename Organization ctor field
tests/test_coverage_underenriched.py	Rename Organization ctor field
tests/test_coverage_gate_self_evident.py	Rename Organization ctor field
tests/test_compose_org_context.py	Rename Organization ctor field
tests/test_column_scope_gate.py	Rename Organization ctor field
tests/test_column_scope_adversarial.py	Rename Organization ctor field
tests/test_cloud_neutral_config.py	Update model_validate payload key
tests/test_aggregation_enforcement.py	Rename Organization ctor field
tests/test_admin_model.py	Update seeded ORG payload + memory API
tests/test_ace051_fail_closed.py	Update fixtures + YAML root key
tests/test_ace047_schema_index.py	Update fixtures + YAML root key
tests/test_ace046_incremental_curation.py	Update fixtures + YAML root key
plugins/agami/skills/agami-serve/SKILL.md	Skill docs: datasource.yaml path
plugins/agami/skills/agami-save-correction/SKILL.md	Skill docs: datasource.md routing
plugins/agami/skills/agami-reconcile/SKILL.md	Skill docs: datasource.yaml precheck
plugins/agami/skills/agami-query/SKILL.md	Skill docs: datasource.yaml + md
plugins/agami/skills/agami-model/SKILL.md	Skill docs: datasource.md editor
plugins/agami/skills/agami-deploy/SKILL.md	Skill docs: datasource.yaml precheck
plugins/agami/skills/agami-connect/SKILL.md	Skill docs: new filenames + flow
plugins/agami/shared/user-memory-format.md	Docs: datasource.md vs USER_MEMORY
plugins/agami/shared/organization-context-format.md	Docs: datasource.md + organization.md
plugins/agami/shared/model-explorer-template.html	UI copy: datasource.md references
plugins/agami/shared/file-layout.md	Layout docs: datasource.yaml/md
plugins/agami/shared/examples-validation-template.html	UI hint: datasource.md
plugins/agami/shared/credentials-format.md	Docs: datasource.yaml mention
plugins/agami/scripts/render_model_explorer.py	Read datasource.md for manifest
plugins/agami/scripts/prepare_deploy.py	Stage/detect datasource.yaml
plugins/agami/scripts/parse_model_feedback.py	Update comment: datasource.md
plugins/agami/samples/store/model/datasource.yaml	Sample root key datasource
plugins/agami/samples/store/model/datasource.md	New sample narrative doc
plugins/agami/lib/semantic_model/init.py	Vendored docs: datasource.yaml
plugins/agami/lib/execute_sql.py	Vendored guard model path
plugins/agami/lib/agami_paths.py	Vendored docs: datasource.yaml
plugins/agami/.claude-plugin/plugin.json	Version bump to 0.5.0
packages/agami-core/src/ui.py	Docstring: datasource.md naming
packages/agami-core/src/tools.py	Domain context read/write + filenames
packages/agami-core/src/semantic_model/snapshot.py	Snapshot requires datasource.yaml
packages/agami-core/src/semantic_model/org_record.py	Root narrative is organization.md
packages/agami-core/src/semantic_model/org_draft.py	Compose context headings + docs
packages/agami-core/src/semantic_model/models.py	Schema field rename to datasource
packages/agami-core/src/semantic_model/loader.py	Loader reads datasource.yaml/key
packages/agami-core/src/semantic_model/introspect.py	Introspect writes datasource name
packages/agami-core/src/semantic_model/curate.py	Writes to datasource.yaml + output key
packages/agami-core/src/semantic_model/cli.py	CLI paths + ensure-org-id target
packages/agami-core/src/semantic_model/build.py	write_tree writes datasource.yaml
packages/agami-core/src/semantic_model/init.py	Docs: datasource.yaml layout
packages/agami-core/src/model_store.py	Memory kind datasource + API rename
packages/agami-core/src/model_deploy.py	Deploy reads datasource.md + writes memory
packages/agami-core/src/migrations/core/001_serving.sql	Memory kind comment updated
packages/agami-core/src/mcp_http.py	Log source string datasource.yaml
packages/agami-core/src/execute_sql.py	Guard model path datasource.yaml
packages/agami-core/src/agami_paths.py	Paths doc: datasource.yaml
packages/agami-core/src/admin.py	Admin UI uses org.datasource + memory key
packages/agami-core/pyproject.toml	Version bump to 0.5.0
docs/usage.md	Usage docs: datasource.yaml
docs/troubleshooting.md	Troubleshooting docs: datasource.md
docs/privacy.md	Privacy docs: datasource.yaml/md
docs/mcp-server.md	MCP docs: datasource.md
docs/format-spec.md	Format spec: datasource.yaml/md
dev/reset-yamls.sh	Reset script paths: datasource.*
CHANGELOG.md	Add 0.5.0 rename notes
.claude-plugin/marketplace.json	Marketplace version bump

Comments suppressed due to low confidence (3)

plugins/agami/skills/agami-save-correction/SKILL.md:180

Grammar: "NOT an datasource.md" should be "NOT a datasource.md".
plugins/agami/skills/agami-save-correction/SKILL.md:321
Grammar: "invent an datasource.md" should be "invent a datasource.md".
tests/test_org_identity_e2e.py:100
This fixture is now writing datasource.yaml but still uses the legacy organization: key. Use datasource: to keep the example consistent with the renamed field.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

    """Two-level org context (F15 / ACE-069). Renders the shared COMPANY block ONCE from the ``OrgRecord``
-    (name/description + the root ``ORGANIZATION.md`` narrative + display conventions + company glossary),
+    (name/description + the root ``datasource.md`` narrative + display conventions + company glossary),
    then each datasource's model-derived ontology under its own heading. Single-source passes one


…est-fixture field Follow-up nits from the rename's blanket pass: - Grammar: "an datasource.{md,yaml}" -> "a datasource.{md,yaml}" (article was correct only for the old "an ORGANIZATION.md"). 6 spots across model_deploy.py, models.py, ui.py, agami-save-correction/SKILL.md. - org_draft.py: the shared COMPANY block docstring named the root narrative datasource.md; it is organization.md (datasource.md is the per-profile narrative). - Test fixtures writing datasource.yaml still used the legacy `organization:` field key -> `datasource:` (test_org_cache, test_org_identity_e2e, test_org_record). No behavior change. Includes the two instances Copilot's sampling missed (model_deploy.py grammar, test_org_cache.py fixtures). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

ashwin-agami · 2026-07-25T12:39:55Z

@@ -633,11 +633,11 @@ def defined_table(self, name: str) -> Optional[Table]:

 class Organization(_Base):


This is confusing. The pydantic class is still called Organization?

ashwin-agami · 2026-07-25T12:41:15Z

+    """The Domain-context page — the deployed datasource.md rendered as (safe) markdown."""
    tree = _model_tree_html(org, datasource, datasources, active_view="context")
-    org_md = memory.get("organization")
+    org_md = memory.get("datasource")


Shouldn't the variable names also change accordingly? What about the real Organization.md that exists above, how is that read?

ashwin-agami · 2026-07-25T12:42:01Z

@@ -106,15 +106,15 @@ def _build_auth_provider() -> AuthProvider:

 def _build_org_resolver() -> SingleTenantOrgResolver:


This is confusing as well. A lot of this assumes that this is a datasource. But all functions are still called org

ashwin-agami · 2026-07-25T12:42:24Z

@@ -21,7 +21,7 @@
 def _default_org() -> str:


Same comment here. How are we managing the orgs?

ashwin-agami · 2026-07-25T12:44:13Z

    const newExamples = [];
    let orgEdit = null;
-    let orgRaw = false;   // ORGANIZATION.md editor mode: false = rich text, true = raw Markdown
+    let orgRaw = false;   // datasource.md editor mode: false = rich text, true = raw Markdown


Same comment here.

ashwin-agami · 2026-07-25T12:45:36Z

@@ -29,7 +29,7 @@

 def _org(name: str, area: str, account_means: str) -> Organization:
    return Organization(


Same issue here. It really returning a datasource, right?

ashwin-agami · 2026-07-25T12:47:20Z

 Two levels: a **company** description written **once** at the deployment root and shared by every datasource, and a **per-database** narrative for the datasource being connected now. The skill never decides yes/skip for the user; "don't ask clarifying questions" does NOT cancel this — it's required state-gathering. Ask **A** then **B**.

-**A. Company context — ask ONCE, on the first onboard into this deployment.** Decide "already authored" by content, not by file existence: treat it as done only if the root `<artifacts_dir>/datasource.md` exists **or** the root `<artifacts_dir>/organization.yaml` already has a non-empty `name`/`description`. If neither, this is the first onboard → ask:
+**A. Company context — ask ONCE, on the first onboard into this deployment.** Decide "already authored" by content, not by file existence: treat it as done only if the root `<artifacts_dir>/organization.md` exists **or** the root `<artifacts_dir>/organization.yaml` already has a non-empty `name`/`description`. If neither, this is the first onboard → ask:


This seems incorrect right. If I have multiple orgs, how would this work?

…atasource Finishes the datasource rename so the per-datasource *model* symbols match the files/fields, per PR review feedback (the half-state read "org means datasource here but not there"). Concept 1 only — deployment/tenant/company identity is left alone (it is legitimately "org"). - class `Organization` -> `Datasource` - `load_organization`/`write_organization`/`starter_organization_md`/`draft_organization_md` -> `*_datasource` (the `*_record` company functions are untouched) - local/JS vars: `org_md`/`org_doc`/`org_path`/`orgp`/`orgRaw`/`orgEdit` -> datasource/ds forms - model-explorer tab `id:'organization'`/`renderOrganizationTab` -> `datasource`/`renderDatasourceTab`, and the `organization_md` manifest + `organization-md` back-channel key -> `datasource_md`/`datasource-md` (render script + template moved in lockstep so the tab still renders + narrative edits still parse) - fix stale org_id-source comments: the id resolves from organization.yaml (F15), not datasource.yaml KEEP (unchanged): OrgRecord, org_id, resolved_org_id, SingleTenantOrgResolver, load/write_organization_record, load_org_record, organization.yaml/.md. Verified: ruff + 1557 tests + gitleaks + lib-drift green (hermetic); explorer re-renders with the Datasource tab (0 old tokens); real model loads as Datasource; feedback round-trips datasource-md; validator clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 98 out of 99 changed files in this pull request and generated 7 comments.

+from semantic_model.models import Datasource  # noqa: E402

 _MODEL_ORG = {
    "organization": "acme",


-model_store.write_organization(_s, "MARKETING", Organization.model_validate(_MODEL_ORG))  # → picker
+model_store.write_datasource(_s, "SALES_DATA", Datasource.model_validate(_MODEL_ORG))
+model_store.write_datasource(_s, "MARKETING", Datasource.model_validate(_MODEL_ORG))  # → picker
 model_store.write_memory(_s, "SALES_DATA", organization=_ORG_MD)


- dev/render_previews.py: the preview script (dev-only, outside tests + lint) still used the pre-rename shapes -> two crash-on-run bugs: * `_MODEL_DS` used the old top-level key `organization:` -> `Datasource.model_validate` failed (extra="forbid" + missing required `datasource`). Fixed the key. * `write_memory(..., organization=...)` -> the param is `datasource_doc`. Fixed the kwarg. (verified by actually running the script — it renders all previews now.) - Grammar: "an Datasource" -> "a Datasource" in 5 docstrings/comments (the Organization->Datasource sed left the wrong article). No product-path impact (render_previews is a manual UI-preview dev tool, not imported by the server or any flow). Gate green: ruff + 1557 tests + gitleaks + lib-drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

vishalkalbi27 and others added 3 commits July 25, 2026 11:32

release: 0.5.0 — datasource file rename

ab34595

Bump plugin/marketplace/package version to 0.5.0 so the directory-marketplace install lands in a fresh cache dir, and retitle the CHANGELOG section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

vishalkalbi27 requested review from ashwin-agami and sandeep-agami as code owners July 25, 2026 11:28

ashwin-agami requested a review from Copilot July 25, 2026 11:31

Copilot started reviewing on behalf of ashwin-agami July 25, 2026 11:31 View session

Copilot AI reviewed Jul 25, 2026

View reviewed changes

ashwin-agami reviewed Jul 25, 2026

View reviewed changes

ashwin-agami requested a review from Copilot July 25, 2026 13:52

Copilot started reviewing on behalf of ashwin-agami July 25, 2026 13:53 View session

Copilot AI reviewed Jul 25, 2026

View reviewed changes

Base automatically changed from feat/ace-067-org-context to main July 26, 2026 06:40

ashwin-agami merged commit 8c2ecf5 into main Jul 26, 2026
6 checks passed

ashwin-agami deleted the refactor/rename-org-to-datasource branch July 26, 2026 06:40

github-actions Bot locked and limited conversation to collaborators Jul 26, 2026

		@@ -633,11 +633,11 @@ def defined_table(self, name: str) -> Optional[Table]:

		class Organization(_Base):

		@@ -106,15 +106,15 @@ def _build_auth_provider() -> AuthProvider:

		def _build_org_resolver() -> SingleTenantOrgResolver:

		@@ -29,7 +29,7 @@

		def _org(name: str, area: str, account_means: str) -> Organization:
		return Organization(

Uh oh!

Conversation

vishalkalbi27 commented Jul 25, 2026

Why

The core rule — "organization" meant three things; two rename, one stays

What changed

Out of scope (intentional)

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ashwin-agami Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

ashwin-agami Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

ashwin-agami Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

ashwin-agami Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

ashwin-agami Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

ashwin-agami Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

ashwin-agami Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants