Host conformance: where the novasama host differs from TrUAPI

[decrypto21]

Spans: paritytech/truapi, paritytech/triangle-js-sdks (@novasamatech/host-api + host-container)

Summary:
the wire shapes - envelope, wire-id table, and most payload/error types - match between the generated @parity/truapi reference and the hand-written novasama host. What remains is the error channel, and it has two open threads: (D1) a terminal typed frame must be emitted on every framework fault - both the reference and novasama sides currently have silent paths, and closing it also waits on the WASM runtime landing; and (D4) the published 0.4.0/0.4.1/0.1.0 landed a tiered CallError on the wire that flat 0.3.x clients silently mis-map. Direction: the wire converges on that tiered shape deliberately via #259 (negotiated, on the new core); the interim flat-restore is not being pursued, and until #259 lands the fleet is held safe only by the ^0.3.2 pin. Transport, rate-limiting, and coin_payment are out of scope (below). This is not a "bump a version" cleanup.

Background

TrUAPI is the one wire language a product speaks to a host: fixed methods, fixed request/response/error shapes, a fixed numeric wire id each, all SCALE-encoded (positional binary - field and variant order is the contract). It is defined by the Rust crate rust/crates/truapi (the source of truth), from which the @parity/* code is generated.

Two things implement the host contract, and either can diverge:

• novasama @novasamatech/host-api + host-container - hand-written TypeScript. Its codecs are hand-maintained and can drift from the crate.
• the generated reference runtime - today the TypeScript @parity/truapi-host; the Rust truapi-server → @parity/truapi-host-wasm is the successor but lives on an unmerged branch (PR-104).

Every response frame:

┌───────────┬───────────┬────────────┬────────────────┬───────────┐
│ requestId │ action id │ version    │ Result tag     │ payload   │
│  (str)    │ (1 byte)  │ V1 = 0x00  │ Ok=00 / Err=01 │ per type  │
└───────────┴───────────┴────────────┴────────────────┴───────────┘

What conforms

• Envelope + wire-id table - [requestId str][action id u8][version V1=0x00][Result Ok=0x00/Err=0x01][payload], verified end-to-end through encodeWireMessage and the generated indexedTaggedUnion({V1:[0,...]}). The table has 55 non-coin_payment method-actions plus 9 coin_payment (ids ≈136-163).
• Payload/error types cross-checked are byte-identical across the crate, @parity/truapi, and current host-api: e.g. HostSignPayloadData (15 fields), HostGetUserIdResponse { primary_username }, Statement/SignedStatement with proof enums, chain types, genesis_hash as variable Vec<u8>.
• Error channel: there are 22 concrete per-method domain error enums (plus a shared GenericError struct). By design, the generic CallError tier (Domain / Denied / Unsupported / MalformedFrame / HostFailure) is stripped from the wire by codegen and unwrapped to its inner domain type D - so the only typed error on the wire is the per-method domain enum. D4 is where a published build broke this invariant.

Coverage honesty: full per-method input-vector coverage across all 55 is the CI's job (T3), not yet fully executed.

Open divergences (remaining)

D1 · A terminal typed frame on every framework fault

The contract: a fault the host doesn't implement, denies, throws on, or can't decode must still answer a terminal Result::Err frame (folded into the per-method domain enum's catch-all, …Err::Unknown { reason }) - never a silent drop that hangs the caller. Both implementations have silent fault paths to close:

• reference / core side - truapi-server must encode error frames as the flat domain enum on every fault path, and drop the empty end-of-stream interrupt.
• novasama side - host-container must answer a typed terminal frame on handler throw / reject, unsupported version tag, and undecodable payload (today: no frame - the caller hangs).

Close: land + publish the WASM reference runtime (T1) and assert a terminal typed frame on faults in the conformance CI (T3).

D4 · Error-frame encoding must stay flat (not the tiered CallError)

@parity/truapi 0.4.0 / 0.4.1 / @parity/truapi-host 0.1.0 shipped a richer, tiered CallError on the wire - reversing the invariant above (0.4.1 added the Firefox iframe fix #283 on top of the same tiered encoder). Clients on the flat 0.3.x decoder read the new variants positionally and silently mis-map them:

                 published 0.4.0/0.1.0        flat 0.3.x decoder reads
PartialPayment   tiered CallError frame   →   InsufficientFunds   ❌
Denied{reason}   tiered CallError frame   →   InvalidSource       ❌

The fleet is safe only because product-sdk-host pins @parity/truapi: ^0.3.2 (caret caps below 0.4.x); the misread goes live on the first bump past 0.3.x, or when host-api-test-sdk#40 adopts 0.1.0.

Direction (updated): the tiered shape is adopted deliberately via #259 (handshake-versioned, on the new core) — the interim flat-restore PR was closed in favour of that committed migration. A flat-restore was written and proven (byte-identical across engines) and remains available locally as a reference/fallback for the #259 work. ⚠️ Until #259 lands, 0.4.0/0.4.1 stay published with the tier and the fleet is safe only by the ^0.3.2 pin — any bump past ^0.3.x before #259 re-exposes the misread.

Fixes in progress — draft PRs (each gated)

Divergence	Status	Gate / direction
D4 — reference/core error shape	→ #259 (tiered, committed). Interim flat-restore PR closed.	Held safe by the ^0.3.2 pin until #259 lands; #259 is the negotiated migration that moves the wire (and every host) to the tier.
D1 (novasama) — host silent on faults (no terminal frame)	Draft PR <NOVASAMA-PR>	review; shape-neutral (answers a frame at all on faults), so unaffected by the shape direction.

T3 (conformance CI: terminal frame on every fault) and T1 (WASM reference runtime) still stand for the D1/host-side work.

Known & out of scope

• Transport establishment - by design, not a task. Client and host establish the byte channel differently (MessagePort vs raw window.postMessage). Same SCALE bytes, different carrier. Out of scope for wire conformance.
• Rate-limiting - integrator-owned on both sides, not a divergence. host-container ships createRateLimiter as a standalone opt-in util; TrUAPI standardizes none.
• coin_payment - a shared future milestone, not a divergence. Both sides unimplemented → nothing to close today.

Tasks

#	Task	Closes	Owner (repo)
T1	Land + publish the WASM reference runtime (truapi-server → @parity/truapi-host-wasm): emits terminal frames on framework faults and gives the CI a stable runtime reference. Currently on the PR-104 branch and unpublished.	D1	truapi
T3	Conformance CI in triangle-js-sdks: per distinct type decode → re-encode + direct dual-encode + failure-path frames, vs the crate codecs. Assert a terminal typed frame on faults, and assert the flat error shape (regression cover for D4). Codec half runs today; runtime half waits on T1.	D1 + D4	triangle-js-sdks

[filvecchiato]

Thanks for looking into this! regarding D2: it was solved here but I think I did not push it to main. I will raise a PR now to fix it.

[filvecchiato]

On further checking, on main it also follows the same order, and checking the ordering in triangle-js-sdk seems to be matching, am I looking in the wrong place?

[decrypto21]

Updated the ticket to close the already shipped ones