Skip to content

Swith to Rust core#70

Open
pgherveou wants to merge 28 commits into
mainfrom
codex/bulletin-preimage-in-core-dotli
Open

Swith to Rust core#70
pgherveou wants to merge 28 commits into
mainfrom
codex/bulletin-preimage-in-core-dotli

Conversation

@pgherveou

@pgherveou pgherveou commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Port the dot.li web host to the shared TrUAPI Rust core

This supersedes #54 and keeps its Rust-runtime port while adding the Bulletin preimage path now owned by the core. dot.li moves off the @novasamatech/host-api stack and onto the shared TrUAPI Rust core. Protocol logic previously maintained in TypeScript now lives once in truapi-server (Rust -> WASM); the web host becomes a byte transport plus typed platform callbacks.

Architecture

A product iframe speaks TrUAPI over a MessageChannel. The host forwards SCALE frames to a Web Worker running truapi-server WASM without decoding the protocol. The core executes protocol logic and calls typed truapi-platform callbacks for browser capabilities.

Product iframe --MessageChannel--> host bridge (byte pipe) --> Web Worker (truapi-server WASM)
                                                                          | typed callbacks
                                                                          v
                                                            dot.li platform capabilities
  • Core owns: wire framing, dispatch, subscriptions, permission gating, auth/SSO state, session interpretation, signing orchestration, Bulletin preimage submission, and the chainHead runtime.
  • Host owns: storage, navigation, confirmation UI, physical chain providers and RPC transport, theme, preimage content lookup, and product identity/pairing configuration.

The same core is embedded by native hosts over UniFFI, keeping protocol behavior and wire formats aligned across platforms.

Changed surface

  • Removes packages/auth, the old Nova container/statement-store adapters, allowance-signer bridge, wallet queue, and related @novasamatech/* dependencies.
  • Adds generated typed host callbacks backed by @parity/truapi-host and @parity/truapi.
  • Rewrites the bridge around the worker-backed Rust core.
  • Keeps chain ownership in dot.li while moving JSON-RPC shaping, chainHead state, transaction construction, signing, dry-run, and Bulletin submission into Rust.
  • Keeps Bulletin available to the core's internal gateway provider without advertising it through the sandboxed dApp chain gate.
  • Keeps preimage content lookup in the host; returned bytes are verified by the core.
  • Preserves in-flight Rust callback requests when granting non-device permissions; browser-policy device grants still reload the product iframe.

Permission grants are core-owned state. The dot.li topbar admin screen projects them into browser-specific iframe permissions and reload behavior rather than maintaining a second protocol policy store.

Legacy product compatibility

Existing applications such as host-playground still use the Nova host wrapper. Nova exposes synthetic PAPI chain-head follow IDs such as follow_0, while the Rust core intentionally requires the exact wire request ID from the follow subscription start frame. Passing the synthetic ID through makes otherwise valid legacy header, body, storage, call, unpin, continue, and stop-operation requests fail core validation.

The legacy window bridge now tracks active follow wire IDs per genesis hash and rewrites only those follow-bound legacy frames. Modern MessagePort products are unchanged, and strict subscription-ID validation remains in the shared Rust core. Keeping this compatibility at the deprecated boundary also avoids a new TrUAPI package release solely for a host-specific migration shim.

Packages

This branch depends on the published @parity/truapi@0.4.0 and @parity/truapi-host@0.1.0. bun run link:truapi swaps them for symlinks to a sibling TrUAPI checkout (and unlink:truapi restores the published packages), so the parent truapi repo's make e2e-dotli runs against this branch directly while a direct dotli checkout installs and tests the published packages.

Scope

The primary PR contains only the Nova-stack removal, the typed Rust-runtime host architecture, the Bulletin preimage integration, and the permission-lifecycle behavior required to keep Rust callback requests alive. Everything else is consolidated in the stacked follow-up #77: preimage lookup retries, protocol-broker hardening, login-failure UX copy, permission-prompt rate limiting, allowance-key at-rest encryption, the per-frame wire debug tap, debug-panel fixes, resolver dependency hygiene, and chainSend flush error surfacing.

Deferred architecture work: SharedWorker topology, eliminating the per-CID *.app.dot.li sandbox iframe, and moving content fetching into Rust behind the PreimageHost boundary.

Left before merge

Related

Verification

Primary branch:

  • bun install --frozen-lockfile
  • bun run format:check
  • bunx --bun turbo run lint --force
  • bunx --bun turbo run typecheck --force
  • bunx --bun turbo run test --force
  • bunx --bun turbo run build:prod --force
  • bun audit

Compatibility patch:

  • Dotli format, lint, typecheck, unit tests, and production build pass locally.
  • Parent TrUAPI make test passes.
  • Two signer-backed make e2e-dotli runs each completed 43 methods successfully, including every Chain/* legacy compatibility method. The sole failure in both runs was Preimage/lookup_subscribe: the external signer-bot authority request timed out at its 45-second boundary. A subsequent authority request in each run succeeded, isolating the failure from the chain bridge patch.

@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Bundle Size Report

Chunks over 500 KB:

File Raw Brotli Gzip
host/assets/paseo.smol-DboPaEh1.json 1.84 MB 941.7 KB 1019.4 KB
host/assets/paseo-people-next.smol.json 3.36 MB 1.68 MB 1.82 MB
host/assets/previewnet.smol.json 1.88 MB 181.4 KB 353.0 KB
host/assets/smoldot.js 2.98 MB 2.21 MB 2.22 MB
host/assets/smoldot_worker.js 2.95 MB 2.21 MB 2.21 MB
host/assets/wasm/web/truapi_server_bg.wasm 1.95 MB 614.4 KB 808.7 KB
Total 15.87 MB (+5.08 MB) 8.10 MB (+2.87 MB) (-49%) 8.73 MB (+3.07 MB)
All files
File Raw Brotli Gzip
host/.well-known/apple-app-site-association 738 B 738 B 738 B
host/.well-known/assetlinks.json 1.3 KB 317 B 391 B
host/assets/bridge.js 54.9 KB (+49.5 KB) 15.6 KB (+13.8 KB) 17.8 KB (+15.8 KB)
host/assets/browser.js 22.9 KB (-10 B) 7.6 KB (-9 B) 8.6 KB (-8 B)
host/assets/client.js 100.1 KB (+6.9 KB) 29.4 KB (+2.0 KB) 32.4 KB (+2.2 KB)
host/assets/dist.js 39.0 KB (+14.2 KB) 12.9 KB (+5.1 KB) 14.5 KB (+5.9 KB)
host/assets/dotli-debug-bus.js 646 B (+151 B) 646 B (+151 B) 646 B (+151 B)
host/assets/get-sync-provider.js 2.8 KB 1.1 KB 1.2 KB
host/assets/hex.js 152 B 152 B 152 B
host/assets/index.js 133.7 KB (+27.1 KB) 36.5 KB (+7.4 KB) 42.8 KB (+8.9 KB)
host/assets/index.css 44.8 KB (-3.1 KB) 7.1 KB (-382 B) 7.9 KB (-423 B)
host/assets/manifest.js 22.5 KB (-133 B) 7.2 KB (-65 B) 7.9 KB (-59 B)
host/assets/panel.js 72.5 KB (-12.3 KB) 19.7 KB (-3.3 KB) 22.3 KB (-4.0 KB)
host/assets/paseo.smol-DboPaEh1.json 1.84 MB 941.7 KB 1019.4 KB
host/assets/paseo-people-next.smol.json 3.36 MB 1.68 MB 1.82 MB
host/assets/paseo.smol.json 25.4 KB 4.9 KB 5.6 KB
host/assets/previewnet.smol.json 1.88 MB 181.4 KB 353.0 KB
host/assets/resolve.js 128 B (-24 B) 128 B (-24 B) 128 B (-24 B)
host/assets/rpc-resolve.js 2.4 KB (-71 B) 1.0 KB (-14 B) 1.1 KB (-33 B)
host/assets/shared-mode.js 1.8 KB (-82 B) 747 B (-46 B) 851 B (-46 B)
host/assets/smoldot.js 2.98 MB 2.21 MB 2.22 MB
host/assets/smoldot_worker.js 2.95 MB 2.21 MB 2.21 MB
host/assets/src.js 1.8 KB (-119 B) 846 B (-57 B) 945 B (-56 B)
host/assets/styles.css 15.1 KB 3.2 KB 3.8 KB
host/assets/wasm/web/README.md 10.9 KB 10.9 KB 10.9 KB
host/assets/wasm/web/package.json 371 B 371 B 371 B
host/assets/wasm/web/truapi_server.d.ts 6.9 KB 6.9 KB 6.9 KB
host/assets/wasm/web/truapi_server.js 35.6 KB 6.2 KB 7.2 KB
host/assets/wasm/web/truapi_server_bg.wasm 1.95 MB 614.4 KB 808.7 KB
host/assets/wasm/web/truapi_server_bg.wasm.d.ts 2.5 KB 2.5 KB 2.5 KB
host/assets/web.js 13.2 KB 3.5 KB 3.9 KB
host/assets/worker-runtime.js 6.3 KB 1.6 KB 1.8 KB
host/assets/worker-runtime.js 106 B 106 B 106 B
host/assets/ws.js 23.1 KB (-2.8 KB) 7.5 KB (-856 B) 8.2 KB (-941 B)
host/dotli.png 11.5 KB 11.5 KB 11.5 KB
host/favicon.svg 1.8 KB 1.8 KB 1.8 KB
host/host-sw.js 2.7 KB (-110 B) 1.1 KB (-7 B) 1.2 KB (-20 B)
host/icon-192.png 12.5 KB 12.5 KB 12.5 KB
host/icon-512.png 42.8 KB 42.8 KB 42.8 KB
host/index.html 19.9 KB (-320 B) 4.4 KB (-68 B) 5.4 KB (-69 B)
host/manifest.webmanifest 441 B 441 B 441 B
host/workbox.js 14.8 KB 4.6 KB 5.1 KB
sandbox/app-sw.js 9.6 KB 3.1 KB (-19 B) 3.5 KB
sandbox/assets/bitswap-bridge.js 840 B 840 B 840 B
sandbox/assets/fetch.js 3.4 KB 1.2 KB 1.4 KB (-1 B)
sandbox/assets/index.css 44.8 KB (-3.1 KB) 7.1 KB (-382 B) 7.9 KB (-423 B)
sandbox/assets/index.js 118.0 KB 33.7 KB (-3 B) 39.6 KB (-2 B)
sandbox/favicon.svg 1.8 KB 1.8 KB 1.8 KB
sandbox/index.html 1.7 KB 580 B (-3 B) 787 B (-1 B)
Total 15.87 MB (+5.08 MB) 8.10 MB (+2.87 MB) (-49%) 8.73 MB (+3.07 MB)

Commit: 3af13bf

@socket-security

socket-security Bot commented Jul 13, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​parity/​truapi-host@​0.1.0781009591100
Added@​parity/​truapi@​0.4.18110010096100

View full report

@github-actions

Copy link
Copy Markdown
Contributor

⚡ Performance Report

⚠️ No baseline found on main. This PR's results are recorded but cannot be compared.
Merge to main to establish a baseline.

Replace the Nova host-container, auth, and signing path with the worker-backed Rust core. Keep dotli responsible only for browser policy, persistence, UI, and physical chain transports.
Notification grants do not alter iframe Permissions Policy. Reloading disposed the in-flight Rust request before its response reached the product.
@pgherveou
pgherveou force-pushed the codex/bulletin-preimage-in-core-dotli branch from ff8115e to 95b810c Compare July 15, 2026 09:03
The pending-message flush drops send failures silently, matching the
pre-port behavior. Surfacing them as JSON-RPC errors moves to a
follow-up PR with test coverage.
@leonardocustodio

Copy link
Copy Markdown
Member

Seems unable to connect? https://host-playground.dotli.dev/?chainBackend=smoldot-direct
Is Smoldot using WSS endpoints with valid SSL certificates?

Nova exposes synthetic follow ids to legacy products, while the Rust core requires the subscription wire request id. Translate only at the legacy window boundary so existing product-sdk apps keep working without weakening core validation.
Comment thread packages/ui/src/host-callbacks/OpenUrl.ts Outdated
Comment thread packages/ui/src/topbar.ts Outdated
Comment thread packages/ui/src/bridge.ts
Comment thread packages/ui/src/host-callbacks/UserConfirmation.ts
Comment thread packages/ui/src/legacy-host-bridge.ts Outdated
Comment thread packages/ui/src/host-callbacks/Preimage.ts
Comment thread packages/ui/src/host-callbacks/Preimage.ts
All OpenUrl requests are normalized and scheme-validated by the Rust core before they reach this browser callback, so duplicating that policy in TypeScript would create a second allowlist that can drift.

Add the noopener window feature to every routing branch instead. This prevents an allowed destination from retaining window.opener access to the dotli host while preserving the existing dot-domain, localhost, and external URL routing behavior.

Cover all three branches with a regression test that verifies each new tab is opened with noopener.
Changing a permission dispatched a synchronous permission-changed event whose listener started rendering the open popover, then the dropdown success path started a second render directly. Both asynchronous renders appended their rows after the same clear, doubling every entry.

Remove only the redundant success-path render and keep the catch-path render for recovery. The permission event remains the single successful refresh trigger for both device and non-device permissions.

Add a regression test that changes a dropdown and asserts the popover still contains exactly one row per permission.
The topbar login promise previously listened only for response frames. If the core provider closed during pairing, no response could arrive and the pending promise retained its message subscription indefinitely.

Subscribe to provider close events and reject with the transport error. Centralize settlement and cleanup so message responses, malformed frames, send failures, and provider closure all unsubscribe both listeners exactly once.

The cleanup also handles the provider contract where an already-closed provider invokes the close callback during subscription. Add a regression test that closes a pending provider and verifies rejection plus listener cleanup.
The confirmation adapter already rendered and resolved the newer review variants, but its suite exercised only legacy payload signing and omitted transaction creation and the dedicated preimage flow.

Add product-account payload and transaction assertions so account formatting and transaction fields are locked down. Add both allow and cancel cases for preimage submission to verify that the dedicated modal maps user intent to the boolean host response.

This intentionally changes tests only: the existing production behavior satisfies the review request once these untested branches are covered.
The legacy window transport accepted inbound frames based only on Window identity and sent outbound frames with a wildcard target. The iframe keeps its real origin because its sandbox includes allow-same-origin; credentialless mode does not make that origin opaque.

Require the resolved product origin for inbound provider frames, use it as the outbound postMessage target, and enforce it on the initial modern-versus-legacy probe before replaying the first frame. This closes the wildcard path without changing the legacy Nova wire compatibility layer.

Add coverage for the exact outbound target and for rejecting both wrong-origin and wrong-source inbound messages.
Preimage lookup trusted non-empty bytes returned by both the IPFS gateway and Bitswap and cached them under the requested key. A faulty or hostile backend could therefore poison the process-wide cache and serve mismatched bytes to later subscribers.

Recompute the requested Blake2b-256 content hash with the existing content verification utility before either cache write. Verification failures follow the existing backend-error stream path and never populate the cache.

Cover successful verified caching and corrupt responses from both backend modes. The corruption cases subscribe twice to prove that rejected bytes are fetched again rather than retained.
@pgherveou
pgherveou deployed to dotli.dev July 17, 2026 19:59 — with GitHub Actions Active
@leonardocustodio

leonardocustodio commented Jul 17, 2026

Copy link
Copy Markdown
Member

Tested everything manually on: https://host-playground.dotli.dev

  1. Get product account ✅

  2. Get legacy account ✅

  3. Get product account alias 🟥
    Seems to be stuck on "pending running", idk if it is the product or not, but should this time out? At least on this application, it locks the UI, and we cannot do anything until refreshing it.

  4. Product account signer ✅

  5. Account connection status ✅

  6. Sign raw message ✅

  7. Sign raw with DotNS Identity 🟥
    Also got stuck on "pending running"

  8. Create Transaction with Product Account ✅

  9. Sign & Submit Batch 🟥
    I did receive the signature on the mobile, signed it, but the dialog did not disappear.

  10. Well-Known Chain ✅

  11. String Write & Read ✅

  12. Bytes Write & Read ✅

  13. JSON Write & Read ✅

  14. Storage Clear ✅

  15. Storage Factory ✅

  16. Feature Check ✅

  17. Permission: Camera 🟥
    All permissions that would need to refresh the iframe seem to get stuck in a blank screen afterwards

  18. Permission: Microphone 🟥

  19. Permission: Location 🟥

  20. Permission: Bluetooth 🟥

  21. Permission: Notifications ✅

  22. Permission: NFC 🟥

  23. Permission: Clipboard 🟥

  24. Permission: Open URL ✅

  25. Permission: Biometrics

  26. Permission: Remote ✅

  27. Permission: WebRTC ✅

  28. Permission: Chain Submit ✅

  29. Permission: Preimage Submit ✅

  30. Permission: Statement Submit ✅

  31. Create Proof 🟥

  32. Create Proof Authorized ✅

  33. Submit Statement 🟥

  34. Subscribe Statements ✅

  35. Subscribe Statements (matchAny) ✅

  36. Submit Preimage 🟥
    Also got stuck on running

  37. Lookup Preimage ✅

  38. Preimage Factory ✅

  39. Upload File to Bulletin & Fetch by CID ✅

  40. Navigate In-App ✅

  41. Navigate to Polkadot URL ✅

  42. Navigate to HTTP URL ✅

  43. Chain Spec: Genesis ✅

  44. Chain Spec: Chain Name ✅

  45. Chain Spec: Properties ✅

  46. Transaction Broadcast ✅

  47. Transaction Stop ✅

  48. Query Balance ✅

  49. Contract: Query Stored Value ✅

  50. Contract: Store Value 🟥

  51. Contract: Query Data Length ✅

  52. Contract: Query Balance ✅

  53. Contract: Deposit ✅

  54. Contract: Withdraw ✅

  55. Contract: Query Total Deposits ✅

  56. Subscribe Theme ✅

  57. Derive Entropy ✅

  58. Request Login ✅

  59. Get User Identity ✅

  60. Subscribe Balance ✅

  61. Statement Store allowance ✅

  62. Bulletin allowance 🟥
    Click on " Allow, " and nothing seems to happen

  63. SmartContract allowance ✅

  64. All resource allowances 🟥
    Same here, nothing seems to happen when clicking on allow

  65. Got the following error when I loaded the page:

index-CE7TavAh.js:17 [signer:host] failed to get product account 
Error: Host rejected product account request: RequestCredentialsErr::NotConnected: RequestCredentials: not connected at...

Is this a product issue? May be requesting the product-account before checking if it is even connected?

@leonardocustodio

Copy link
Copy Markdown
Member

Let me know if some of those will be addressed in subsequent PR's instead of this one

@pgherveou

Copy link
Copy Markdown
Collaborator Author

thanks for the report @leonardocustodio that should be fixed in this PR
will look into it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants