Skip to content

architecture: control plane - #69

Closed
croadfeldt wants to merge 3 commits into
mainfrom
pr/da4-control-plane
Closed

architecture: control plane#69
croadfeldt wants to merge 3 commits into
mainfrom
pr/da4-control-plane

Conversation

@croadfeldt

@croadfeldt croadfeldt commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

UC map — Realizes UC-3 (control plane).

Implements UDLM contracts: dcm-project/udlm#18 dcm-project/udlm#20 — core wire contracts + provider/policy the control plane speaks.
Depends on the UDLM substrate landing first (Wave C); wave index: dcm-project/udlm#35.

Decision: Control-plane components, self-health, internal auth, session revocation, API versioning.

Part of the staged croadfeldt/dcmdcm-project/dcm upstreaming: single-concern PRs, ≤3k lines, logical boundaries. (2430 lines, 5 files.)

Draft — pending team re-engagement.

@croadfeldt
croadfeldt force-pushed the pr/da4-control-plane branch from 35b7937 to 260572e Compare June 17, 2026 16:44

## 2. Breaking Change Definition

A change is **breaking** if it requires any existing client to modify its code or configuration to continue working correctly. The following changes are always breaking:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is well known fact, do we need to have it here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will trim — drop the textbook semver preamble and keep only the DCM-specific versioning rules.

DCM maintains version-compatible with registered OIS versions during the support lifecycle. A DCM instance running OIS v2 must continue to dispatch to providers registered on OIS v1 until the version is sunset.

When the OIS version is incremented:
1. DCM announces the new OIS version via the event `governance.ois_version_released`

@machacekondra machacekondra Jun 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OIS version update is done via event and the rest of the communication is done via REST calls? Why?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new OIS version is a broadcast to all registered providers — one→many, asynchronous — so it's an event (governance.ois_version_released); a REST call would mean DCM polling each provider individually. Per-request dispatch stays REST because it's point-to-point and needs the provider's synchronous realized result. Announcement = fan-out (event); dispatch = request/response (REST). Will state this in the doc.

Comment on lines +35 to +36
Provider API (OIS):https://{dcm-instance}/provider/api/v1/
Flow GUI API: https://{dcm-instance}/flow/api/v1/

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a reason those are separate endpoints?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're separate because Consumer / Admin / Provider (OIS) / Flow are distinct trust zones with their own auth + RBAC, and they version independently — the Consumer API can be at v2 while OIS is still v1. Separating the surfaces lets each evolve and be secured on its own. Will document the rationale.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up on my earlier note: that sentence is now in §1.1 — the four surfaces are separate because they're distinct trust zones (Consumer, Admin, Provider/OIS, and Flow each carry their own auth + RBAC, and each versions independently — Consumer at v2 while OIS is still v1). So it's stated in the doc now, not just here.

When a new major version is released, the previous version enters a **deprecation period**. The deprecation timeline is profile-governed — production deployments require longer support windows than development environments:

```yaml
api_version_support_lifecycle:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain what this means? why there is dev? Why dev needs any deprecation timeline?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dev surfaces don't need a deprecation window — I'll drop the timeline for dev.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — dev now carries deprecation_notice_period: P0D (no guarantee) and the dev deprecation window is dropped. A dev surface can change without a timeline.

### 2.4 Event Routing Model

```
Event published: { type: "request.initiated", payload: {...}, entity_uuid: X }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So policy engine is using event-driven architecture, while the communication with providers is synchronous REST driven? What is a reason behind this?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deliberate: policy evaluation is one→many (a request event fans out to all matching policies, plus other subscribers like cost/drift) → event bus; provider dispatch is one→one and needs a synchronous realized result to advance the pipeline → REST. Fan-out/decoupled vs request/response. Will document.

policies:
- step: 1
handle: "org/policies/cost-check"
condition: "request.initiated AND resource_type=Compute.VirtualMachine AND tenant.profile=prod"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is:

  • request.initiated
  • resource_type=
  • tenant.profile

Where it's taken from? Who evaluate this object and where those vars are taken? How does the system knows it's a string or a variable?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's policy match-condition grammar. I'll document the fields, how variables resolve, and how string-vs-reference is distinguished.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — the match-condition grammar is now documented: the grammar production, how variables resolve, and how a string literal is distinguished from a reference. So request.initiated, resource_type=, and tenant.profile are no longer opaque.

Comment on lines +247 to +270
cost_estimation_request:
catalog_item_uuid: <uuid>
assembled_fields:
cpu_count: 4
memory_gb: 8
storage_gb: 100
tenant_uuid: <uuid>
requested_duration: P30D # optional; lifecycle estimate

cost_estimation_response:
estimated_cost:
per_hour: 0.32
per_month: 230.40
lifecycle_estimate: 691.20 # if requested_duration provided
currency: USD
confidence: high # high: current Cost Analysis data
# medium: data > PT1H old
# low: static fallback
breakdown:
- component: compute
per_hour: 0.28
- component: ip_allocation
per_hour: 0.04
cost_data_timestamp: <ISO 8601>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We really need to document every attribute of this "object". It's API endpoint etc... same for all other such random yamls in the doc. We need to know what the fields are. Like component: ip_allocation what are the component types, where are they defined. What a breakdown means, nothing is explained here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — the inline YAML examples need a field reference. I'll add per-field docs (this is the recurring "document every field" theme across #69/#20 — handling it as one field-reference pass).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — every attribute now has a field reference under that block (component, breakdown, confidence, cost_data_source, and the rest). This was the recurring "document every field" theme across #69/#20, so I did it as one pass — if you spot another inline YAML that still reads as a black box, point me at it and it gets the same treatment.

on_fail: halt
- step: 3
handle: "org/policies/security-review"
condition: "always"

@machacekondra machacekondra Jun 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is always? Variable, string, reserved word? Can that be documented? If we would implement the engine processing this conditions, we don't know how and what are all the options.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

always is a reserved match-condition keyword. I'll document the reserved words + the condition grammar so it's unambiguous.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — always is now documented as a reserved match-condition keyword (unconditionally true), alongside the other reserved tokens (AND, OR, IN, true, false), so an implementer knows it's not a variable or string.

croadfeldt referenced this pull request in croadfeldt/dcm Jun 27, 2026
…y foundational principles (#15)

* taxonomy: drop Dynamic Plugin dep; RHDH = one non-required GUI target (dcm-project/dcm #64, pkliczewski)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* taxonomy: consolidate dcm-project/dcm #64 review (pkliczewski) — apply foundational principles

- Lifecycle events consumer-neutral: Request Orchestrator routes to ALL subscribers (Policy
  Engine + cost mgmt + drift), not one engine (principle: consumer-neutral mechanisms; #64 L68).
- Add **Placement Policy** (8th typed policy) → answers "what about placement policy?" (L40),
  references ADR-019.
- Trim the UI block to the Unified Shell concept + cross-ref the GUI specs; drop enumerated
  feature lists + GUI-001–010 detail (principle: right altitude — taxonomy = concept, spec =
  detail; #64 L183).
- RTO/RPO + Dual-Write = standard DR/cutover concepts adopted by reference, keep only DCM-specific
  targets (don't-redefine-solved-standard gate, T5; #64 L192).
- Liveness/readiness = standard k8s probe convention by reference, keep DCM-specific deps (T5; L218).

Earlier commit removed the Dynamic-Plugin dependency (rejected upstream; L140). Design-ish items
(#69 field docs, dual-write rationale depth) carried to the #163 re-sync.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* generalize homelab name in DISCUSSION-TOPICS (roadfeldt-observability -> a homelab observability stack)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
croadfeldt referenced this pull request in croadfeldt/dcm Jun 28, 2026
…s) (#20)

api-versioning.md:
- defer the well-known SemVer baseline (cite semver.org); keep only the
  DCM-specific application
- frame the breaking-change list as DCM's explicit authoritative checklist,
  not a generic tutorial; call out the DCM-specific entries
- dev profile gets no deprecation guarantee (was P60D/P90D) — dev is for
  iteration, versions can break without a window
- §7.2: add the event-vs-REST rationale for OIS version propagation
  (version/capability change = one-to-many fan-out = event; dispatch =
  point-to-point = sync REST)

components.md:
- §2.4: document why policy evaluation is event-driven (fan-out to many
  policies) but provider dispatch is synchronous REST (point-to-point,
  result needed to advance)
- §2.5: add the match-condition grammar — expression syntax, the `always`
  reserved keyword, variable-path-by-reference vs literal, unresolved-path
  semantics
- §3.4: add a field reference for the cost estimation/attribution objects

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
croadfeldt added a commit that referenced this pull request Jun 28, 2026
…(sync)

Sync from upstream croadfeldt/dcm. GateKeeper policy type -> Gating Policy
(action gatekeep -> gate, op gatekeeping -> gating, enum gatekeeper ->
gating). OPA Gatekeeper / Gatekeeper ConstraintTemplate references preserved.
Files: architecture/control-plane/api-versioning.md architecture/control-plane/components.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@croadfeldt

Copy link
Copy Markdown
Collaborator Author

Pushed the changes for this PR (synced from the upstream work).

api-versioning.md

  • the SemVer baseline is now deferred (cited, not restated); §1.3 keeps only DCM's specific application, and §2 reframes the breaking-change list as DCM's explicit authoritative checklist rather than a generic tutorial (calling out the DCM-specific entries).
  • dev profile gets no deprecation guarantee — dev is for iteration, versions can break without a window.
  • §7.2 adds the rationale for announcing OIS version changes as events vs REST: a version/capability change is one-to-many fan-out (event); a dispatch is point-to-point needing a result (sync REST).

components.md

  • §2.4 documents why policy evaluation is event-driven (a single lifecycle event fans out to any number of independent policies) while provider dispatch is synchronous REST (point-to-point, result needed to advance the request).
  • §2.5 adds the match-condition grammar: expression syntax, the always reserved keyword (+ other reserved tokens), variable-path-by-reference vs literal, and unresolved-path → false.
  • §3.4 adds a field reference for the cost estimation/attribution objects (the "document every attribute" ask).

Separately, the GateKeeper → Gating Policy rename landed across the spec (the policy type collided with the OPA/Kubernetes Gatekeeper operator) — you'll see it in this and the other slices; the genuine OPA Gatekeeper references are preserved.

croadfeldt and others added 2 commits July 8, 2026 08:31
Control-plane components, self-health, internal auth, session revocation, API versioning.

Signed-off-by: Chris Roadfeldt <chris@roadfeldt.com>
Signed-off-by: croadfeldt <chris@roadfeldt.com>
…(sync)

Sync from upstream croadfeldt/dcm. GateKeeper policy type -> Gating Policy
(action gatekeep -> gate, op gatekeeping -> gating, enum gatekeeper ->
gating). OPA Gatekeeper / Gatekeeper ConstraintTemplate references preserved.
Files: architecture/control-plane/api-versioning.md architecture/control-plane/components.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: croadfeldt <chris@roadfeldt.com>
croadfeldt referenced this pull request in croadfeldt/dcm Jul 12, 2026
api-versioning.md:
- defer the well-known SemVer baseline (cite semver.org); keep only the
  DCM-specific application
- frame the breaking-change list as DCM's explicit authoritative checklist,
  not a generic tutorial; call out the DCM-specific entries
- dev profile gets no deprecation guarantee (was P60D/P90D) — dev is for
  iteration, versions can break without a window
- §7.2: add the event-vs-REST rationale for OIS version propagation
  (version/capability change = one-to-many fan-out = event; dispatch =
  point-to-point = sync REST)

components.md:
- §2.4: document why policy evaluation is event-driven (fan-out to many
  policies) but provider dispatch is synchronous REST (point-to-point,
  result needed to advance)
- §2.5: add the match-condition grammar — expression syntax, the `always`
  reserved keyword, variable-path-by-reference vs literal, unresolved-path
  semantics
- §3.4: add a field reference for the cost estimation/attribution objects

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
croadfeldt referenced this pull request in croadfeldt/dcm Jul 12, 2026
review: resolve open review comments (dcm #64/#66/#69 doc edits)
@croadfeldt

Copy link
Copy Markdown
Collaborator Author

Thanks @machacekondra — captured, not closed. Superseded by the ordered, scoped set (#109#117), tracked in #108; your feedback carries forward as CF-9 (control-plane doc rigor — document every field, state the event-vs-REST rationale). See #108; please re-review on the new set.

@croadfeldt croadfeldt closed this Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants