feat(core): F15 organization context — shared company record + per-datasource ontology (ACE-067–070)#143
Merged
Merged
Conversation
…tasource ontology (ACE-067–070)
A deployment can connect several databases under ONE company. Today org == profile ==
datasource are fused 1:1:1, so company-wide facts (org_id, name, description, fiscal year,
glossary) get duplicated into every per-profile org.yaml and drift. F15 gives the deployment
one shared company record, written once, while each datasource keeps its own vocabulary — so
"Account" resolves to a customer in the CRM and a GL account in the ERP, and a federated
question carries the same company context once plus both vocabularies. Stays single-tenant (N=1).
- ACE-067: OrgRecord/DisplayConventions types; new org_record.py (ensure/load_org_record over
<artifacts_dir>/organization.yaml); relocate org_id into the record (mint-once, immutable,
idempotent legacy lift) and delete F14's adopt-sibling scan; resolved_org_id reads the record.
- ACE-068: 013_organization.sql (the one table keyed on org_id alone); write/load_organization_record
wired once-per-run into model_deploy. The write is an FK-SAFE UPSERT (ON CONFLICT DO UPDATE), not
the clear-then-insert used elsewhere, and the table is a COLUMN SUPERSET of agami-hosted's tenant
`organization` (org_name + created_at present, doc DEFAULT '{}') so one physical table serves both
and a redeploy can't violate the hosted org_membership/license foreign keys.
- ACE-069: two-level compose_org_context (company block once + per-datasource ontology), wired into
both the local CLI (org-context) and the served path (get_datasource_schema); byte-identical
graceful degradation when no record exists; content-routing rule documented.
- ACE-070: agami-connect onboarding UX — author company context once, attach subsequent databases
under the org, and route a different company to a new folder.
Scope: agami-core only. Backward-compatible — pre-F15 deployments load and compose unchanged. No
data egress (organization.yaml is local file I/O; the privacy gate stays green). 1552 tests pass,
including the FK-safety crux (a re-upsert survives an FK-referencing membership row under
PRAGMA foreign_keys=ON) and hosted coexistence.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
vishalkalbi27
requested review from
ashwin-agami and
sandeep-agami
as code owners
July 23, 2026 13:58
There was a problem hiding this comment.
Pull request overview
This PR implements F15 “organization context” by introducing a deployment-level company record (shared across datasources) alongside per-datasource ontologies, wiring the two-level context into both the CLI (org-context) and the served schema path (get_datasource_schema), and persisting the company record into a new organization table.
Changes:
- Add a deployment-level
OrgRecord(<artifacts_dir>/organization.yaml+ rootORGANIZATION.md) and relocateorg_idresolution to prefer the record (with legacy fallbacks). - Add migration
013_organization.sqland deploy-time upsert logic to write a single FK-safeorganizationrow keyed byorg_idonly. - Implement two-level context composition (
compose_org_context) and integrate it into both CLI output and the served schema context, with tests covering degradation/compatibility.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_schema.py | Asserts the new organization table exists with the expected PK/columns. |
| tests/test_organization_context_e2e.py | E2E coverage for two-level org context across CLI + served schema path. |
| tests/test_org_record.py | Validates record mint/lift semantics and resolver precedence. |
| tests/test_org_record_deploy.py | Validates DB upsert semantics (FK-safe) + hosted coexistence + shape. |
| tests/test_compose_org_context.py | Unit tests for two-level composition + byte-identical degradation. |
| plugins/agami/skills/agami-connect/SKILL.md | Updates onboarding UX to route “same vs different company” and author company context once. |
| plugins/agami/shared/organization-context-format.md | Documents the two-level routing and composition behavior. |
| packages/agami-core/src/tools.py | Reads + composes two-level domain context for get_datasource_schema. |
| packages/agami-core/src/semantic_model/org_record.py | New module implementing load/ensure/write for organization.yaml. |
| packages/agami-core/src/semantic_model/org_draft.py | Adds compose_org_context and helpers for company + datasource blocks. |
| packages/agami-core/src/semantic_model/models.py | Introduces OrgRecord and DisplayConventions model types. |
| packages/agami-core/src/semantic_model/cli.py | Updates org-context command to render two-level composition. |
| packages/agami-core/src/semantic_model/build.py | Updates org-id ensure flow to use the deployment record (and supports dry-run). |
| packages/agami-core/src/model_store.py | Adds DB write/load for the organization record with FK-safe UPSERT. |
| packages/agami-core/src/model_deploy.py | Deploys the org record + company narrative once per run (deployment-level). |
| packages/agami-core/src/migrations/core/013_organization.sql | Adds organization table keyed on org_id alone with hosted-compatible columns/defaults. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+866
to
+870
| org_md_raw, user_md_raw = _domain_memory(profile) | ||
| domain_context = _OD.compose_context(org_md_raw, org) | ||
| # Two-level (F15 / ACE-069): the shared COMPANY block from the deployment record + this datasource's | ||
| # source-specific narrative + derived summary. No record ⇒ compose_org_context degrades to the exact | ||
| # pre-F15 single-level output, so a deployment without a record is unaffected. | ||
| record, company_md = _company_context(_current_org_id()) |
| `Yes` → write to the **per-profile** `<artifacts_dir>/<profile>/ORGANIZATION.md` under `# About this database` (source-specific narrative only). `Skip` → leave it absent; Phase 2f writes a short per-database starter. | ||
|
|
||
| `Yes — I'll type it now (Other field)` → write their words to `<artifacts_dir>/<profile>/ORGANIZATION.md` under `# About this database`. **Their narrative ONLY** — do NOT append model facts (subject areas, metrics, glossary). Those are *derived from the model at read time* (the query path assembles them via `cli org-context`), so they never get baked into the editable prose file where a human could clobber them. `Skip — I'll auto-fill it from my data (Recommended)` → leave ORGANIZATION.md absent for now; Phase 2f writes a short human-narrative starter. `chmod 600` whatever you write. See [`shared/organization-context-format.md`](../../shared/organization-context-format.md). | ||
| `chmod 600` whatever you write. See [`shared/organization-context-format.md`](../../shared/organization-context-format.md) for the content-routing rule (company-wide → root; source-specific → per-profile; per-column units → the structured model; personal → `USER_MEMORY.md`). |
| | Level | Where | Holds | Written | | ||
| |---|---|---|---| | ||
| | **Company** (the deployment) | `<artifacts_dir>/organization.yaml` (the `OrgRecord`) + `<artifacts_dir>/ORGANIZATION.md` (company narrative) | company name/description, `fiscal_year_start_month`, `display_conventions` (currency/rounding/week_start), the company-wide `glossary` — and the company narrative prose | once, at first onboarding; edited via `/agami-model` | | ||
| | **Datasource** (each profile) | `<artifacts_dir>/<profile>/org.yaml` + `<profile>/ORGANIZATION.md` | that source's ontology (`key_terminology`, subject areas, …) and a **source-specific** narrative only | per profile | |
Collaborator
|
@ashwin-agami - please review as well. |
|
|
||
| Free-form Markdown describing **what this database is about** in the human's own words: the company / product, what the data represents, who the users are. One file per profile at `<artifacts_dir>/<profile>/ORGANIZATION.md` — context is database-specific. | ||
|
|
||
| ## Two levels: the company record vs the per-datasource narrative (F15) |
Collaborator
There was a problem hiding this comment.
Internal project related details don't need to go into the skill.
| - **Re-introspect `<profile>`** — refresh the structure from the live DB (new/changed tables, columns, FKs) while preserving descriptions, entities, metrics, caveats, and sign-offs (the `reintrospect` path). | ||
| - **Open model explorer** — browse + curate the existing model and review/sign off the trust layer (`/agami-model`). | ||
| - **Onboard another database** — set up a **different** database (a different connection) under a **new** profile, leaving `<profile>` untouched. On this choice, **start a fresh onboarding for a new profile**: jump to the profile-naming step (Phase 0a's naming question) → have the user name the new profile (must differ from `<profile>` and any existing `[section]` in `<artifacts_dir>/local/credentials`) and pick its DB type → write that profile's `credentials.example` → run the full flow for it. Never reuse or overwrite the current profile's credentials or model. | ||
| - **Onboard another database** — set up a **different** database (a different connection), leaving `<profile>` untouched. On this choice, **first ask whether it's the same company or a different one** (F15 — see 1.1a), then start a fresh onboarding: jump to the profile-naming step (Phase 0a's naming question) → have the user name the new profile (must differ from `<profile>` and any existing `[section]` in `<artifacts_dir>/local/credentials`) and pick its DB type → write that profile's `credentials.example` → run the full flow for it. Never reuse or overwrite the current profile's credentials or model. |
Collaborator
There was a problem hiding this comment.
Remove F15 and similar references from the PR
… + review fixes Follow-up to the F15 organization-context work, addressing testing feedback and PR review: - Populate the root record: onboarding now writes the company name + description via a new `sm set-org` CLI command (set_org_fields), and the seeded connect todo list includes a mandatory organization-context step so the company question is actually asked (it was being skipped because it wasn't in the plan). - Auto-maintained `datasources` list on OrgRecord: rebuilt from the on-disk profile dirs on every onboard (build.write_tree) and deploy (model_deploy), carried into the Postgres organization row's doc. Never hand-edited, so it can't drift. - Keep per-profile ORGANIZATION.md as the datasource narrative (restored the per-database question every onboard); the root organization.yaml is purely additive. Review fixes (PR #143): - Served context path opens ONE DB connection instead of two: _domain_memory + _company_context collapsed into a single _context_sources() that reads per-datasource memory, the org record, and the company narrative on one Store (hot tool path). - chmod 644, not 600, on ORGANIZATION.md narrative files — non-secret model files that must stay readable in deploy bundles / for teammates (600 causes the deploy container crash-loop). Secrets (local/, credentials, .config) keep 600. - Fully-rooted <artifacts_dir>/<profile>/ORGANIZATION.md paths in the shared doc. - Scrubbed remaining internal project references from the plugin skills/shared docs (instructions only). Verified: full gate green (ruff + tests + gitleaks), 99% patch coverage. Second-datasource scenario verified — same org_id, name/description preserved, datasources list grows to include both. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
A deployment can connect several databases under one company. Today
org == profile == datasourceare fused 1:1:1, so company-wide facts (org_id, name, description, fiscal year, glossary) get duplicated into every per-profileorg.yamland drift. F15 gives the deployment one shared company record, written once, while each datasource keeps its own vocabulary — soAccountresolves to a customer in the CRM and a GL account in the ERP, and a federated question carries the same company context once plus both vocabularies. Stays single-tenant (N=1); a different company is a different folder.Companion to F14 (portable
org_id, #141): F14 gave identity, F15 gives the content + relocates the id into the record it should have lived in.Specs
OrgRecord/DisplayConventionstypes; neworg_record.py(ensure/load_org_recordover<artifacts_dir>/organization.yaml); relocateorg_idinto the record (mint-once, immutable, idempotent legacy lift) and delete F14's adopt-sibling scan;resolved_org_idreads the record.013_organization.sql— the one table keyed onorg_idalone;write/load_organization_recordwired once-per-run intomodel_deploy.compose_org_context(company block once + per-datasource ontology), wired into both the local CLI (org-context) and the served path (get_datasource_schema); byte-identical graceful degradation when no record exists; content-routing rule documented.agami-connectonboarding UX — author company context once, attach subsequent databases under the org, route a different company to a new folder.Key design decision — shared
organizationtable with agami-hostedThe Postgres table is deliberately named
organizationand is a column superset of agami-hosted's tenantorganizationtable (org_name+created_atpresent,doc DEFAULT '{}'), so one physical table serves both repos. Two consequences, both load-bearing:ON CONFLICT DO UPDATE), not the clear-then-insert used elsewhere inmodel_store.py— because in the hosted stackorg_membership.org_idandlicense.org_idforeign-key this row, so aDELETEon redeploy would crash underPRAGMA foreign_keys=ON.INSERT (org_id, org_name, created_at)keeps working untouched (thedocDEFAULT fills the gap;COALESCEpreserves hosted-owned columns on upsert).Scope is agami-core only — no agami-hosted changes are required for correctness now.
Backward compatibility & safety
organization.yamlis local file I/O; the privacy gate stays green.organization.yamlis written with default perms (notchmod 600) — it's a deploy-read model file, and mode-600 model files trip the known deploy container crash-loop.Verification
uv run dev.py check: ruff + tests + gitleaks, all green), incl. the 8 F14 regression tests.write_organization_recordsurvives an FK-referencingorg_membershiprow underPRAGMA foreign_keys=ON(aDELETE-based write would fail there), plus a hosted-coexistence test.🤖 Generated with Claude Code