feat(cli): embed plugin registry#8
Draft
QMalcolm wants to merge 121 commits into
Draft
Conversation
… ontology spec that refs the core spec so that both may evolve independently.
Implements OSI → Honeydew and Honeydew → OSI conversion with full round-trip fidelity. Relationship names are stored natively on the relation, and relations are always placed on the many side. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix empty-string expression bypassing None guard - Restore calculated_attribute routing via HONEYDEW type hint - Preserve bool datatype through OSI round-trip - Store string metric ai_context in osi metadata for recovery - Warn on duplicate metric names instead of silently overwriting - Restore connection_expr from HONEYDEW custom_extension on OSI→Honeydew - Warn on malformed JSON in _read_osi_metadata instead of silent drop - Remove filter limitation from README (not applicable) - Update Honeydew docs link to honeydew.ai/docs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ssion guards - Restore Honeydew-specific fields (display_name, hidden, format_string, timegrain, owner, folder) in OSI→Honeydew direction by reading them back from the HONEYDEW custom_extension on both entity and attribute objects - Add path traversal guard in main() using os.path.normpath + startswith check - Guard against whitespace-only expressions (not expr or not expr.strip()) - Warn when field expression is a non-dict value instead of silently dropping it - Simplify elif not from_cols: to else: in _osi_relation_to_honeydew - Strengthen test_ai_context_string_preserved to assert the string value is recoverable in description - Add 5 new tests: display_name/format round-trip, calc attr Honeydew fields, entity owner, whitespace expression skipped, non-dict expression warns Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rewrite entire test file from classes to module-level pytest functions; parametrize repeated cases (is_simple_identifier, parse_osi_source, field datatypes, entity honeydew fields, dataset/calc attr fields, empty/whitespace expressions, path traversal guard, check_safe_path) - Extract _check_safe_path into a named helper in the converter (was inlined in main()) so path traversal logic is independently testable - Add parametrized test_check_safe_path covering ../evil.yml and ../../etc/passwd rejection and legitimate nested paths - Add test_empty_or_whitespace_metric_expression_skipped to cover the OSI→Honeydew whitespace guard on metrics (was previously untested) - Parametrize entity/attr/calc Honeydew field round-trip tests (owner, display_name, hidden, folder each verified independently) - Update _write_workspace helper to pass through entity-level fields (owner, display_name, hidden, folder) so round-trip tests use the standard _honeydew_roundtrip() helper instead of manual workspace setup - Promote _HD_ATTR_KEYS tuple to module-level constant (was defined inside the field loop on every iteration) - Drop from __future__ import annotations — requires Python 3.12+ - Add pyproject.toml with requires-python = ">=3.12" - Add Python 3.12+ requirement section to README Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove requirements.txt (superseded by pyproject.toml) - Extract _fields_to_honeydew() from _dataset_to_files(): field classification is now a named, independently testable function; four direct unit tests added - Warn when a relationship has neither from_columns nor connection_expr so callers discover the incomplete join before it reaches Honeydew - Round-trip the vendors list: non-HONEYDEW vendors are stored in the workspace osi metadata on OSI→Honeydew and merged back on the return trip (HONEYDEW always appears first) - Add main() CLI smoke tests via subprocess: osi-to-honeydew writes the expected workspace.yml, honeydew-to-osi writes a parseable OSI YAML, and path traversal in an entity name is rejected with exit code 1 - Update README setup instructions to use pip install . / pip install -e . Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Collapse 13 individual OSI→Honeydew tests, 7 Honeydew→OSI tests, 10 OSI round-trip tests, and 13 Honeydew round-trip tests into four @pytest.mark.parametrize blocks. Every assertion now compares the entire output dict rather than cherry-picked fields. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Docstring: relationship name is mapped directly to Honeydew's relation name field (not osi metadata); ai_context is mapped natively to description/labels/AI metadata, not treated as having no equivalent - README mapping table: rename rows to use Honeydew's canonical terms (Source Attribute, Calculated Attribute) and add missing ai_context row - README limitations: rewrite the confusing "One dataset per entity" bullet to clarify that OSI dataset = one table/query, Honeydew supports multiple dataset files per entity but the converter generates exactly one Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rror changes to the core semantic model spec.
…(#206) Mirrors orionbelt-semantic-layer#201 (canonical source) into the ossie converter: - Metric round-trip: OSItoOBML resolves metric references by BOTH the OSI dataset/field name and the physical code (source-table code and field expression, quoted or bare), so metrics emitted against physical codes (e.g. SUM(fact_orders.amount)) round-trip instead of dropping when a data object's code differs from its display name. Resolved refs are bracket-quoted so display names with spaces stay intact through the dataset.column parsers. - Dimension collision: _extract_dimensions qualifies a field name that occurs in more than one dataset (Orders.date / Invoices.date) with its data object and warns, instead of silently overwriting the earlier dimension. - Validation robustness: validate_osi guards its semantic loops against malformed structures (datasets/fields that are not lists of dicts) and returns schema errors instead of raising AttributeError. Adds tests/test_osi_converter_roundtrip_robustness.py. 154 converter tests pass; ruff clean; mypy clean with the dev extra.
* feat(cli): scaffold go module with cobra command tree
The OSI converters currently require manual environment setup and
per-converter invocation. This lays the foundation for a unified CLI
(osi) that will discover and invoke converters as plugins via a
stdin/stdout JSON protocol.
Creates cli/ at the repo root as a self-contained Go module
(github.com/open-semantic-interchange/osi/cli). All commands are
stubbed — no logic is wired in this commit.
Design decisions:
- Go chosen for static binary distribution; no runtime dependency
for end users (brew/apt installable)
- internal/osidir owns ~/.osi/plugins/ initialization and respects
$OSI_PLUGIN_DIR for override; uses os.UserHomeDir() rather than
$HOME for Windows portability
- PersistentPreRunE on the root command ensures dir init runs before
every subcommand; commented caveat that Cobra does not chain this
automatically if a subcommand defines its own
- MarkFlagsMutuallyExclusive("from", "to") handles the both-set case
on osi convert; the neither-set case is validated manually in RunE
since Cobra only guards against both being provided
Build pipeline (Makefile, .goreleaser.yaml) and CI follow in
separate commits.
* chore(cli): add makefile and goreleaser build pipeline
Enables local builds and cross-platform release artifacts for the
OSI CLI.
Makefile provides standard targets for day-to-day development:
build, test, lint, release-dry-run, and clean. All targets are
designed to run from within cli/.
GoReleaser config targets linux/amd64, linux/arm64, darwin/amd64,
darwin/arm64, and windows/amd64. windows/arm64 is excluded — no
widely available CI runner and negligible current demand.
CGO_ENABLED=0 is set for fully static binaries, enabling
cross-compilation from any host without a C toolchain. Version,
commit, and date are injected at build time via ldflags from the
vars declared in main.go.
* ci(cli): add github actions workflow for build and test
Runs go build, go vet, and go test on every push and pull request
that touches cli/ or the workflow file itself.
The paths filter prevents CLI changes from triggering unrelated
workflows in this polyglot repo and vice versa.
Go version is derived from go-version-file: cli/go.mod so the
workflow automatically picks up any future toolchain bumps without
a separate workflow edit. defaults.run.working-directory avoids
repeating cd cli/ on every step.
Cross-platform build testing is not included — CGO_ENABLED=0
means the linux/amd64 build is representative of all targets.
GoReleaser snapshot builds are deferred to a future release workflow.
* test(cli): add unit tests for internal/osidir
Covers the only logic in F1 that warrants testing: $OSI_PLUGIN_DIR
env var override, default path construction via os.UserHomeDir(),
directory creation, and idempotent re-invocation of EnsurePluginDir.
t.Setenv is used throughout so env var mutations are automatically
restored after each test. t.TempDir is used for filesystem tests so
no cleanup is needed and tests are safe to run in parallel.
* chore(cli): rename osi to ossie throughout cli scaffold
Project branding has changed from OSI to OSSIE. Updates all
user-facing and internal references in the CLI:
- Binary name: osi → ossie
- Go module path: .../osi/cli → .../ossie/cli
- Default plugin directory: ~/.osi → ~/.ossie
- Environment variable: OSI_PLUGIN_DIR → OSSIE_PLUGIN_DIR
- GoReleaser project name and archive ids
- All command descriptions and flag help text
- Output directory default: ./osi-output → ./ossie-output
* chore(cli): rename internal osidir package to ossiedir
Completes the OSI → OSSIE rename by updating the internal package
directory, package declaration, and import reference in cmd/root.go.
* fix(ossiedir): rename defaultOSIDir constant to defaultOssieDir
The osi → ossie rename missed this unexported constant. The value
(.ossie) was already correct; only the identifier name was stale.
* chore(cli): fix .gitignore to ignore cli/ossie build artifact
The osi → ossie rename missed the gitignore entry, so the local
build artifact cli/ossie would no longer be ignored by git.
* chore(ossiedir): rename osidir.go and osidir_test.go to ossiedir
File names were inconsistent with the package directory name (ossiedir/).
Pure rename — no code changes.
* Swap references to Open Semantic Interchange to Apache
Ossie has moved from the Open Semantic Interchange to incubation with
the Apache Software Foundation. As such, the references in this PR
needed to be updated accordingly.
* chore(cli): align go.mod version with .tool-versions pin
cli/go.mod declared `go 1.22` while cli/.tool-versions pins
golang 1.26.2, giving contributors two conflicting sources of
truth for which Go version this module targets. Flagged in
review by khush-bhatia on PR #151.
There's no existing compatibility requirement forcing a lower
minimum, so align go.mod to the same version .tool-versions
already pins rather than introduce a toolchain directive or
lower the asdf pin.
* refactor(cli): let cobra enforce --from/--to via MarkFlagsOneRequired
convert.go hand-rolled a check for the "neither --from nor --to
set" case alongside MarkFlagsMutuallyExclusive, which already
covers the "both set" case. Suggested by khush-bhatia on PR #151
that cobra's MarkFlagsOneRequired covers this directly.
Combining MarkFlagsMutuallyExclusive with MarkFlagsOneRequired on
the same flag set gives "exactly one of" semantics entirely
through cobra's flag annotations, so the manual check and its
now-unneeded fmt.Errorf branch in runConvert are removed.
* Apply suggestions from code review
Co-authored-by: Khushboo Bhatia <khushboo.kanjani@gmail.com>
* chore(cli): stop tracking .tool-versions, gitignore for local use
cli/go.mod and cli/.tool-versions each pinned the Go version
independently — even after b834d75 aligned their values, nothing
forced the two to move together, so they could drift apart again
on the next bump.
Removed cli/.tool-versions from the repo so go.mod is the only
in-repo source of truth. Verified this doesn't break tooling:
CI already resolves its Go version from go.mod via actions/setup-go's
`go-version-file: cli/go.mod` (.github/workflows/cli-ci.yml), and
asdf's golang plugin (~/.asdf/plugins/golang/bin/parse-legacy-file)
reads the version straight out of go.mod's `go` directive when no
.tool-versions is present.
Rather than deleting the file with no path back, gitignored
cli/.tool-versions so contributors whose version manager still
wants its own pin file can keep one locally without risk of it
getting committed and re-introducing a second tracked source of
truth. Treats the toolchain-manager pin as personal environment
config, not project config — the same rationale as gitignoring
editor-specific config instead of committing it.
Caveats:
- Relies on asdf's legacy-version-file lookup, which is off by
default and must be enabled per-user via `legacy_version_file =
yes` in ~/.asdfrc (or ASDF_LEGACY_VERSION_FILE=yes) for `asdf
install`/`asdf current` to auto-detect the version from go.mod.
Not adding a repo-level workaround since asdf has no per-project
way to set this.
- No automated enforcement that a contributor's local pin matches
go.mod; a stale local .tool-versions can still silently drift, as
demonstrated on this machine when the global asdf fallback (golang
1.20.1) was too old to parse a 3-component go directive. Accepted
as the tradeoff for not owning tooling-manager config in the repo.
---------
Co-authored-by: Khushboo Bhatia <khushboo.kanjani@gmail.com>
…efile (#225) Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
* Checkin snapshot files for consistent testing * Add header to ambr files * Keep syrupy snapshots command
…icks) (#224) Bidirectional, offline converter between Apache Ossie semantic models and Databricks Unity Catalog Metric Views (YAML v1.1), filling the DATABRICKS spoke already listed in converters/README.md. Packaged like the sibling spokes: pyproject.toml (apache-ossie-databricks), an ossie_databricks package under src/, ASF license headers, and a tests/ suite (example-based + Hypothesis property-based round-trip; 74 tests). PyYAML is the only runtime dependency. Co-authored-by: jackstein21 <82542300+jackstein21@users.noreply.github.com>
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
* Switch to UV * Switch to UV * Switch to UV * Restored ASF header
* Added FS working group to working_groups.md added the Financial Services Common Semantics Working Group to working+groups.md. Included Google Calendar and Slack links. * Update working_groups.md added Apache Ossie (incubating) Calendar link to Financial Services Common Semantics Working Group.
* Docs UpdateS: Addings Slack links and updating slack channel names for accuracy * Removing Sync API and Composability entry
* Add WisdomAI domain export converter Adds converters/wisdom, a one-way converter from a WisdomAI domain export JSON (format 1.0) to an Ossie semantic model YAML, following the layout of the dbt converter. Mapping: domain -> semantic model; domain knowledge and system instructions -> model-level ai_context; tables/columns/formulas -> datasets/fields; relationship graph -> relationships (cardinality folded into edge direction, compound AND-joins flattened to composite keys); per-table measures -> model-level metrics. Expressions are emitted verbatim under the dialect mapped from the table's connection (snowflake, databricks), falling back to ANSI_SQL with a warning for unsupported dialects. Information loss is reported through the same ConverterResult/ConverterIssue pattern as ossie-dbt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add WISDOM to well-known custom-extension vendors Adds WISDOM to the OSIVendor enum, the vendor examples in the spec documents and JSON schema, and the supported-vendors table in the converter guide. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add Ossie -> wisdom domain export direction Adds osi-to-wisdom, the reverse of the wisdom converter: an Ossie document becomes a wisdom domain export (format 1.0) consumable by wisdom's importDomain API. The mapping inverts wisdom-to-osi so round-trips are stable in both directions: model ai_context splits back into system instructions and knowledge items, fields split into columns and formulas, relationship direction is read as many-to-one with ai_context notes restoring one-to-one/many-to-many and composite keys becoming compound AND join conditions, and metrics attach to the table their expression references. IDs are deterministic (derived from names) and connections are per-dialect placeholders remapped at import time. Verified: Ossie -> wisdom -> Ossie round-trips of two real domain exports are byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Adapt wisdom converter to BIGQUERY dialect and UV conventions BigQuery connections now map to the BIGQUERY dialect added to the spec instead of falling back to ANSI_SQL with a warning, with backtick identifier quoting; both directions round-trip it. The pyproject follows the repo's UV layout (dependency-groups, uv sources for the in-repo apache-ossie). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address review feedback: issue-reason fallback and FIPS-safe MD5 Use .get() with the enum value as fallback when printing converter warnings so a ConverterIssueType without a reason entry cannot crash the CLI, and pass usedforsecurity=False to hashlib.md5 so deterministic ID generation works on FIPS-enabled builds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(cli): add plugin discovery infrastructure
Downstream segments (convert, plugin list, plugin install) all need
a way to find and parse installed plugins. This adds the shared
foundation: a typed Plugin struct and a Discover() function that
scans ~/.ossie/plugins/ for valid plugin.yaml files.
Design decisions:
- internal/plugin accepts pluginsDir as a parameter rather than
calling ossiedir.PluginDir() internally — keeps the package
testable without env var manipulation
- plugin.yaml supports both ossie_* (preferred) and osi_* (deprecated
fallback) key families to avoid breaking plugins written before the
OSI → OSSIE rename; ossie_* wins when both are present
- Malformed plugin dirs emit a warning to an io.Writer and are
skipped; only an unreadable plugins directory is a hard error,
matching the behavior specified in the architecture doc
- Non-directory entries in the plugins dir are silently skipped
(no warning); non-existent dir returns nil, nil
Caveats:
- Symlinked plugin directories are not yet handled — os.ReadDir
reports symlinks with ModeSymlink, not IsDir(); a TODO comment
marks the location for a future follow-up
- cmd/plugin/list.go wiring is deferred to P1, which also requires
the embedded registry before the full table output can be rendered
* test(cli): replace equalStringSlice helper with slices.Equal
go.mod declares go 1.22 so slices.Equal (stdlib since 1.21) is
available. Removes the hand-rolled utility in favour of the standard
library equivalent.
* fix(cli): use errors.Is for ErrNotExist checks in discover
os.IsNotExist does not handle wrapped errors correctly. Replace both
occurrences with errors.Is(err, os.ErrNotExist), the modern idiom
since Go 1.13.
* test(cli): use t.TempDir for nonexistent dir test fixture
Replace hardcoded /this/path/does/not/exist with a path derived from
t.TempDir(). The parent is guaranteed to exist by the test harness
but the child subdir does not, making the fixture deterministically
nonexistent without relying on filesystem assumptions.
* test(cli): add test coverage for plugin setup field
Adds two cases: setup path is correctly parsed when present, and
setup is empty string when the field is absent from plugin.yaml.
The setup path is consumed by the install segment (P2) when running
the plugin's setup script.
* docs(cli): document intentional lenient YAML unmarshaling in loadPlugin
Unknown fields in plugin.yaml are silently ignored by yaml.Unmarshal.
Adding a comment to make clear this is a deliberate forward-
compatibility choice, not an oversight.
* refactor(plugin): rename osi → ossie in ConvertConfig, yaml fields, and invoke args
ConvertConfig.ToOSI/FromOSI and the plugin.yaml fields to_osi/from_osi
were left using the old brand name. This sweeps them to ossie
consistently with the rest of the CLI rename.
to_ossie/from_ossie are now the canonical yaml keys. to_osi/from_osi
are kept as deprecated fallbacks in rawPlugin, using the same
ossie-wins merge pattern already established for
ossie_plugin_spec/osi_plugin_spec. The invoke-arg convention
(to-ossie, from-ossie) is updated in all canonical test fixtures; the
two backward-compat tests (TestDiscover_ossieKeyPreferred,
TestDiscover_osiKeyFallback) intentionally retain to_osi/from_osi
since they exercise the deprecated-key path.
* refactor(plugin): drop osi_* backward compat from rawPlugin
The preemptive osi_* fallback fields (osi_plugin_spec, osi_spec_version,
to_osi, from_osi) were added in anticipation of backward compatibility,
but no plugins exist yet using that format — the rename is happening
before any external consumers. Carrying dead compatibility code with no
real users adds noise and keeps osi in the codebase.
Drops all deprecated fields and merge logic from rawPlugin, simplifying
validate() and toPlugin() back to direct field access.
TestDiscover_ossieKeyPreferred and TestDiscover_osiKeyFallback (which
existed solely to exercise the now-removed paths) are replaced with
TestDiscover_unknownFieldsIgnored, which covers the lenient-YAML
behavior (future spec fields are silently tolerated) that discover.go
explicitly documents.
* feat(plugin): implement basic ossie plugin list command
Wires plugin list to the discovery infrastructure added in F2. Reads
from the resolved plugin directory, prints a NAME/PLATFORM/SPEC table
via tabwriter, and falls back to a "no plugins installed" message when
the directory is empty or missing.
This is a minimal first pass — the full P1 spec calls for
cross-referencing against the embedded plugin registry (installed vs.
available, latest version, update indicator), which depends on registry
embedding (F4) not yet implemented.
* chore(cli): add make install target for local development
go install . names the binary after the module path's last segment
(cli) rather than the intended ossie. make install uses go build -o
directly into $GOPATH/bin with the correct binary name.
* refactor(plugin): flatten platform.name/vendor into top-level name/platform
PR review on the plugin manifest schema concluded that plugins lack
their own identity: platform.name is documented as "matched against
--from/--to values" — i.e. it identifies the target platform, not the
plugin package itself. `ossie plugin list` papers over this by
displaying the install directory's basename as a stand-in "name",
which isn't stable across reinstalls/renames and gives a future
registry nothing durable to key on.
Considered adding a second, distinct field for the plugin's own
identity alongside platform.name/vendor, but with no existing
manifests to preserve, that just trades one problem for a schema with
two similarly-named concepts to document and keep straight. Settled on
reusing the existing required matching key as the plugin's identity
instead: top-level `name` (e.g. `dbt` vs a hypothetical `dbt-community`)
now does double duty, so official/community disambiguation happens by
naming convention at author time rather than needing runtime
conflict-resolution logic in discovery for two plugins that would
otherwise both claim `name: dbt`. `platform.vendor` collapses into a
plain top-level `platform` string, since nothing parses it — it's
freeform display text ("dbt Labs", "PowerBI") shown in `plugin list`,
not a second identifier.
`ossie plugin list` now shows the manifest's real name instead of the
directory basename.
Deferred: if two installed plugins do declare the same `name`,
discovery still has no conflict-resolution behavior. Not addressed
here — pre-existing gap, orthogonal to this schema change.
…#244) The project was renamed to Apache Ossie. Replace remaining occurrences of the former name Open Semantic Interchange in package metadata, docstrings, and converter docs with Ossie. The historical formerly known as references in README, CONTRIBUTING, and docs are intentionally kept. Also adds the missing final newline to several pyproject.toml files to satisfy .editorconfig (insert_final_newline = true).
…r (#113) * Add datatype field to Field and Metric; reframe is_time as role marker Introduces a top-level `datatype` on Field and Metric with a closed logical enum: string, integer, number, boolean, date, time, timestamp, timestamp_tz, other. Addresses issue #84. `datatype` and `dimension.is_time` are independent and orthogonal: - `datatype` declares the field's logical data type (casting/serialization). - `is_time` is a temporal-role marker (time-series analysis, temporal filtering). A field with `is_time: true` may carry any `datatype` (e.g. integer for a year grain, string for a month name, date for a calendar date). When `is_time` is unset, it defaults to `true` for temporal datatypes (`date`, `time`, `timestamp`, `timestamp_tz`) and `false` otherwise. Explicit `is_time` always wins, so authors can set `is_time: false` on an audit `created_at` to keep it off the time axis. Taxonomy and type/role split were chosen after benchmarking 14 peer semantic layers and 5 portable type standards. Notable precedent: Snowflake Semantic Views' YAML authoring form has a `time_dimensions:` collection whose entries can carry any `data_type` (the published example annotates `order_year` with `data_type: NUMBER`); LookML's `dimension_group` accepts `date`, `datetime`, `timestamp`, `epoch`, and `yyyymmdd`. Snowflake converter updated: `_classify_field` honors explicit `is_time` first, then falls back to the temporal-datatype default. 9 new tests cover the datatype paths and the mixed-metadata cases (`d_year` with `datatype: integer` and `is_time: true`, audit timestamp opt-out, etc.). tpcds_semantic_model.yaml demonstrates three coexistence patterns: datatype-only, datatype + is_time, and is_time-only. Also added .gitignore for __pycache__ directories. * Align DataType enum with ontology value types * Add DataType to Python bindings * Map Ossie DataType to Snowflake fields * Map Ossie DataType in GoodData conversions * Map Ossie DataType in Polaris conversions * Map Ossie DataType in Salesforce conversions * Consolidate datatype test coverage * Fix GoodData datatype edge cases * Honor effective time roles in dbt export * Warn on lossy Salesforce DateTime export * Define datatype enum in YAML spec * Remove trailing whitespace from spec example * Add syncing todo
* feat(cli): add plugin invocation protocol
C1 (osi convert base) needs a subprocess layer to pipe requests to
plugins and receive responses. This adds the Request/Response/Issue
envelope types and an Invoke() function that owns the full lifecycle:
marshal request to stdin, capture stdout, forward stderr, enforce
context timeout, and decode the JSON response.
Design decisions:
- Invoke takes pluginDir and invoke []string directly rather than
*Plugin — decouples the invocation layer from the discovery types
and makes the function independently testable
- cmd.Stdout assigned as *bytes.Buffer rather than using cmd.Output()
— cmd.Output() is incompatible with a pre-assigned cmd.Stderr writer
and would silently discard the caller's stderr destination
- ctx.Err() checked after cmd.Run() for timeout detection — exec
kills the process with SIGKILL and returns "signal: killed", not
context.DeadlineExceeded, so the context must be checked explicitly
- Issue.Path uses omitempty — an absent key is semantically distinct
from an empty string; plugins that have no path omit the field
- A non-empty Issues slice is NOT a Go error — the caller (C1) is
responsible for inspecting severities and determining exit code
Tests use the TestMain subprocess self-invocation pattern: the test
binary re-invokes itself with GO_TEST_PLUGIN=1, enters the fake
plugin branch in TestMain, and exits without running any tests.
This avoids compiling a separate helper binary and works portably.
Caveats:
- File collection, output writing, issue rendering, and wiring into
cmd/convert.go are deferred to C1
* fix(cli): normalise nil Request.Files to empty map before marshaling
A Request with nil Files marshals as {"files":null} rather than
{"files":{}}. Plugins that iterate over the files object may crash
on a null value. Guard added at the top of Invoke(); test added to
verify the wire format is always an object.
* fix(cli): guard against empty invoke slice in Invoke
An empty invoke slice would panic with an index out of range on
invoke[0]. While F2 validation prevents this in normal usage, an
explicit check with a descriptive error is safer and makes the
failure mode obvious if the invariant is ever violated.
* Flatten concept declarations in extends example snippet * Flatten concept declarations in schema and remaining doc snippets * Flatten concept declarations in flights example
Adds cli/internal/registry/ with the plugin registry embedded into the binary via //go:embed. The registry is read at runtime with no network call required. Initial registry contains placeholder 0.1.0 entries for the five known platforms (dbt, gooddata, polaris, salesforce, snowflake). Checksums are marked sha256:placeholder pending real plugin packaging. Entries will be updated with real tags and checksums when converters are packaged as distributable OSSIE plugins. Wiring into cmd/plugin/list.go and cmd/plugin/install.go is deferred to P1 and P2 respectively.
- Add comment on EntryType constants noting they are consumed by P2 - Clarify why knownPlatforms is package-level (slices cannot be const) - Guard TestRegistry_Platforms_sorted against vacuous pass on empty slice
Replaces the basic installed-only listing with a full P1 implementation that cross-references installed plugins against the embedded registry. Output now shows NAME/STATUS/INSTALLED/LATEST for all registry-known platforms, with status values of installed, update available, or not installed. Community plugins (installed but absent from the registry) are printed in a separate section below. The previous no-plugins-installed short-circuit is replaced by a no-plugins-available message that fires only when the registry is empty and nothing is installed. Matching installed plugins to registry entries is done by Platform.Name, not by directory name, since the registry is keyed by platform.
QMalcolm
force-pushed
the
qmalcolm--feat-registry-embedding
branch
from
July 24, 2026 16:57
0f5d030 to
22a4f3c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The big part of this PR is defining the protocol for the "plugin registry", i.e. how first party plugins are tracked. In addition we've added stubs for known first party plugins. Though these plugins are now "visible" in
ossie plugin list, they are not actually installable yet as we haven't yetScreen.Recording.2026-06-24.at.14.02.40.mov
Migrated from apache/ossie#156 following the project's move to the Apache Software Foundation.