Skip to content

CHANGELOG.md

Latest commit

 

History

History
1254 lines (1076 loc) · 93.3 KB

CHANGELOG.md

File metadata and controls

1254 lines (1076 loc) · 93.3 KB

Changelog

All notable changes to this project will be documented in this file.

Unreleased

Breaking

  • CLI upgrade required: the new InterfaceV3 onchain account format adds flex_algo_node_segments (RFC-18). CLI versions prior to this release cannot deserialize device accounts written by the new program. Operators must upgrade the CLI before or alongside the program upgrade.

Changes

  • CLI
    • Extend doublezero resource verify to check MulticastPublisherBlock against multicast publisher users' dz_ip allocations; legacy dz_ips that fall outside the block's range are ignored so pre-existing users allocated before this extension existed do not produce false discrepancies
    • Fix doublezero resource verify to report missing TunnelIds resource extensions for all devices, including those without any users; previously the discrepancy was suppressed when a device had no users, hiding unallocated extensions
  • Client
    • Filter devices by type-specific capacity during auto-selection so clients are not provisioned onto devices that have reached their unicast, multicast publisher, or multicast subscriber limits
  • Collector
    • fallback to any probe if anchor probes aren't available
  • Smartcontract
    • Fix multicast group allowlist add/remove for AccessPasses created with allow_multiple_ip=true; the processors were rejecting requests with a real client IP because the stored IP is always 0.0.0.0 for these passes (#3551)
    • SDK now auto-detects the correct AccessPass PDA (static or dynamic) for allowlist operations based on whether an allow_multiple_ip pass exists
  • Sentinel
    • Make the multicast publisher worker's --client-filter flag repeatable so multiple validator client names can be matched in one run (OR semantics), matching the admin CLI behavior

v0.18.0 - 2026-04-17

Breaking

Changes

  • Device Health Oracle
    • Add interface_counters activation criterion to device-health-oracle to verify devices have recent interface counter data in ClickHouse before activation
    • Add controller_success activation criterion to device-health-oracle to verify devices have consistent controller call coverage over a configurable burn-in period by querying ClickHouse
  • Telemetry
    • Add GET /device-link/agent-versions endpoint to data-api and agent-versions subcommand to data-cli, exposing per-device telemetry agent version and commit from onchain DeviceLatencySamplesHeader
  • Smartcontract
    • Allow SubscribeMulticastGroup for users in Pending status so that CreateSubscribeUser can be followed by additional subscribe calls before the activator runs (#3521)
    • Add optional owner field to UpdateMulticastGroup instruction, allowing foundation members to reassign ownership of a multicast group (#3527)
    • Rename SubscribeMulticastGroup instruction variant to UpdateMulticastGroupRoles and rename associated processor functions, args struct, and SDK command to use "roles" terminology, clarifying they manage publisher/subscriber roles rather than just subscriptions
  • Geolocation
    • Add optional result destination to GeolocationUser so LocationOffsets can be sent to an alternate endpoint instead of the target IP; supports both IP and domain destinations (e.g., 185.199.108.1:9000 or results.example.com:9000); includes SetResultDestination onchain instruction, CLI user set-result-destination command, and Go SDK deserialization (backwards-compatible with existing accounts)
  • CLI
    • Add --owner flag to multicast group update, accepting a pubkey or me (#3527)
    • Polish terminal output of connect and disconnect: fix emoji semantics, normalize message phrasing across IBRL and multicast code paths, resolve tenant to human-readable code on connect (errors if tenant not found), and fix progress bar not clearing before output in disconnect (#3529)
  • Client
    • Reduce default probing interval to 5m from 30s since DZDs don't generally move.
  • Dependencies
    • Bump vulnerable packages across Rust, Go, and Python to address Dependabot security alerts (14 packages fixed)
  • DevContainer
    • Add optional DZ_WORKTREES_DIR mount exposing a host worktrees directory at /workspaces/worktrees inside the container; useful when docker exec-ing into the persistent dev container to work on git worktrees outside the repo. Defaults to /tmp/worktrees (empty, harmless) when unset

v0.17.0 - 2026-04-10

Breaking

Changes

  • Activator
    • Fix duplicate tunnel underlay pairs after restart by registering device.public_ip as in-use for legacy users with unset tunnel_endpoint during allocation reload
  • Client
    • Rank devices and tunnel endpoints by minimum observed latency (min_latency_ns) instead of average when selecting a connection target, preferring paths with the best achievable round-trip time
  • Tools
    • Add IsRetryableFunc field to RetryOptions for configurable retry criteria in the Solana JSON-RPC client; add "rate limited" string match and RPC code -32429 to the default implementation
  • Telemetry
    • Add optional TLS support to state-ingest server via --tls-cert-file and --tls-key-file flags; when set, the server listens on both HTTP (:8080) and HTTPS (:8443) simultaneously
    • Remove --additional-child-probes CLI flag from telemetry-agent; child geoprobe discovery now relies entirely on the onchain Geolocation program
    • Add BGP status submitter: on each tick, reads BGP socket state from the device namespace, maps each activated user to their tunnel peer IP, and submits SetUserBGPStatus onchain; supports a configurable down grace period and periodic keepalive refresh; enabled via --bgp-status-enable with --bgp-status-interval, --bgp-status-refresh-interval, and --bgp-status-down-grace-period flags
    • Bound CachingFetcher RPC calls with an explicit 30s timeout; context.WithoutCancel drops the parent deadline as well as cancellation, so without this a hung Solana RPC would block all singleflight waiters indefinitely
  • Monitor
    • Add ClickHouse as a telemetry backend for the global monitor alongside existing InfluxDB
  • E2E tests
    • Add TestE2E_GeoprobeIcmpTargets verifying end-to-end ICMP outbound offset delivery via onchain outbound-icmp targets
    • Refactor geoprobe E2E tests to use testcontainers entrypoints and onchain target discovery
    • Add TestE2E_UserBGPStatus verifying that the telemetry BGP status submitter correctly reports onchain status transitions as clients connect and establish BGP sessions
  • Monitor
    • Add ClickHouse as a telemetry backend for the global monitor alongside existing InfluxDB
  • SDK
    • Deserialize agent_version and agent_commit from device latency samples in Go, TypeScript, and Python SDKs
    • Add BGPStatus type (Unknown/Up/Down) and SetUserBGPStatus executor instruction to the Go serviceability SDK
  • Sentinel
    • Improve find-validator-multicast-publishers and create-validator-multicast-publishers with multi-value --client filter, --ip filter, nearest-device selection, dynamic capacity re-evaluation, and a fix for multicast publisher owner being set to the sentinel payer instead of the validator's owner
  • Smartcontract
    • Add agent_version ([u8; 16]) and agent_commit ([u8; 8]) fields to DeviceLatencySamplesHeader, carved from the existing reserved region; accept both fields in the InitializeDeviceLatencySamples instruction via incremental deserialization (fully backward compatible)
    • Implement SetUserBGPStatus processor: validates metrics publisher authorization, updates bgp_status, last_bgp_reported_at, and last_bgp_up_at fields on the user account
    • Add human-readable error messages for serviceability program errors in the Go SDK, including program log extraction for enhanced debugging
    • user get no longer fails when no Access Pass exists; it prints a warning to stderr and continues, showing an empty access pass field
    • Replace manual account validation assertions with validate_program_account! macro across serviceability processor files, adding consistent data_is_empty checks and fixing a missing is_writable validation in ResumeLink (#3436)
    • Extend validate_program_account! migration to remaining user and multicastgroup allowlist processors (set_bgp_status, delete, closeaccount, publisher/subscriber add/remove)
    • Add OutboundIcmp target type (= 2) to the geolocation onchain program, enabling ICMP-based probing as an alternative to TWAMP for outbound geolocation targets
    • Allow pending users with subs to be deleted
    • Add doublezero link topology {create,delete,clear,backfill,list} subcommands for managing flex-algo topologies; topology clear auto-discovers tagged links when --links is omitted
    • Add TopologyInfo onchain account for IS-IS flex-algo link classification: auto-assigned TE admin-group bit (1–62), derived flex-algo number (128 + bit), and constraint type (include-any/include-all); capped at 62 topologies via AdminGroupBits resource extension
    • Add link_topologies: Vec<Pubkey> (capped at 8) and link_flags: u32 (bit 0 = unicast-drained) to the Link account
    • Add include_topologies to the Tenant account for topology-filtered routing opt-in
    • Enforce UNICAST-DEFAULT topology existence as a precondition for link activation
    • Extend link get and link list to display topology assignments and drain status; add --link-topology <name> filter to link list and --link-topology (comma-separated topology names) / --unicast-drained flags to link update; use default as the value to clear all topology assignments
    • Extend tenant get and tenant list to display included topologies; add --include-topologies (comma-separated topology names) flag to tenant update; use default to clear
  • Onchain programs
    • Add tunnel_endpoint field to the UpdateUser instruction (UserUpdateArgs), allowing the activator to overwrite a user's tunnel endpoint onchain; field is optional and backward compatible via incremental deserialization
  • Telemetry
    • Device telemetry agent now posts agent_version and agent_commit in the DeviceLatencySamplesHeader when initializing new sample accounts, enabling version attribution of onchain telemetry data
    • Add optional TLS support to state-ingest server via --tls-cert-file and --tls-key-file flags; when set, the server listens on both HTTP (:8080) and HTTPS (:8443) simultaneously
    • Remove --additional-child-probes CLI flag from telemetry-agent; child geoprobe discovery now relies entirely on the onchain Geolocation program
    • Add BGP status submitter: on each tick, reads BGP socket state from the device namespace, maps each activated user to their tunnel peer IP, and submits SetUserBGPStatus onchain; supports a configurable down grace period and periodic keepalive refresh; enabled via --bgp-status-enable with --bgp-status-interval, --bgp-status-refresh-interval, and --bgp-status-down-grace-period flags
  • Tools
    • Add IsRetryableFunc field to RetryOptions for configurable retry criteria in the Solana JSON-RPC client; add "rate limited" string match and RPC code -32429 to the default implementation
    • Add doublezero-admin migrate flex-algo [--dry-run] command to backfill link topology assignments and VPNv4 loopback flex-algo node segments across all existing devices and links
  • Geolocation
    • Standardize CLI flag naming: probe mutation commands use --probe (was --code) accepting pubkey or code; rename --signing-keypair → --signing-pubkey and --target-pk → --target-signing-pubkey; add --json-compact to get commands
    • geoprobe-target can now store LocationOffset messages in ClickHouse
    • Add ICMP pinger to geoprobe-agent for measuring outbound ICMP targets with interleaved batch send/receive, integrated into the existing measurement cycle alongside TWAMP
    • Remove --additional-parent, --additional-targets, --additional-icmp-targets, and --allowed-pubkeys CLI flags from geoprobe-agent; all configuration now comes from onchain state via parent and target discovery
    • Add MinCache for tracking minimum-RTT measurements with best/backup promotion over a rolling TTL window, used by both geoprobe-target and geoprobe-target-sender to suppress redundant output and surface only new-best events

v0.16.0 - 2026-04-03

Breaking

Changes

  • Smartcontract
    • Require that the access pass provided to SubscribeMulticastGroup belongs to the payer; foundation allowlist members may use any access pass.
    • Add Index account for onchain key uniqueness enforcement and O(1) key-to-pubkey lookup, with standalone CreateIndex/DeleteIndex instructions for migration backfill
    • Set minimum client version to 0.10.0
    • Enforce 9000-byte MTU on links and non-CYOA/non-DIA device interfaces; CYOA/DIA interfaces must be 1500. Onchain validation now returns InvalidMtu (error 46) for non-conforming values.
    • Add OutboundIcmp target type (= 2) to the geolocation onchain program, enabling ICMP-based probing as an alternative to TWAMP for outbound geolocation targets
  • CLI
    • Allow incremental multicast group addition without disconnecting
    • Reset SIGPIPE to SIG_DFL at the start of main() in all 3 CLI binaries (doublezero, doublezero-geolocation, doublezero-admin) so the process exits silently like standard CLI tools
    • Support --type outbound-icmp in geolocation user add-target, remove-target, and get commands
    • Add sentinel admin commands to find and create multicast publishers for IBRL validators
    • handle non-user owned disconnects gracefully
    • Add user's multicast pub/sub groups if applicable to status
  • Sentinel
    • Add multicast publisher worker with Solana RPC-based validator discovery
    • Add e2e tests for multicast publisher worker with validator-metadata-service mock
  • SDK
    • Add Go SDK for shred subscription program with read-only account deserialization (epoch state, seat assignments, pricing, settlement, validator client rewards), PDA derivation helpers, RPC fetchers, compatibility tests, and a fetch example CLI
    • Add GeoLocationTargetTypeOutboundIcmp to Go geolocation SDK with deserialization and round-trip test support
  • Device Health Oracle
    • Update link.health and device.health to ready-for-service and ready-for-users when they are not already in that state
  • Tools
    • Add twamp-debug diagnostic tool for testing kernel timestamping support on switches; sends real TWAMP probes to verify which SO_TIMESTAMPING modes (RX/TX software/hardware/sched) actually deliver timestamps, and reports RTT statistics comparing userspace vs kernel timestamp sources
  • E2E Tests
    • Switch backward compatibility test to install versioned CLI binaries from GitHub releases instead of Cloudsmith apt repos; version enumeration now uses the GitHub API directly from Go rather than querying apt-cache inside the container
  • Client
    • Add doublezero_connection_info Prometheus metric exposing connection metadata (user_type, network, current_device, metro, tunnel_name, tunnel_src, tunnel_dst) (#3201)
    • Add doublezero_connection_rtt_nanoseconds and doublezero_connection_loss_percentage Prometheus metrics reporting RTT and packet loss to the current connected device

v0.15.0 - 2026-03-27

  • Client
    • fix(client): fix latency field overflow by changing i32 to i64 (#3382)
    • fix(client): add user feedback to latency, add flag to limit icmp probe concurrency (#3385)

Breaking

Changes

  • Funder
    • Top up contributor owner keys alongside device metrics publishers, multicast group owners, and the internet latency collector
  • Smartcontract
    • Fix multicast publisher/subscriber device counter divergence: multicast_publishers_count never decremented and multicast_subscribers_count over-decremented on user disconnect because the decrement logic checked !publishers.is_empty(), which is always false at delete time. Add a durable tunnel_flags field to the User struct with a CreatedAsPublisher bit, set at activation, and use it in the delete and closeaccount instructions.
    • Allow foundation allowlist members and the sentinel to create multicast users with a custom owner via a new owner field on CreateSubscribeUser, enabling user creation on behalf of another identity's access pass
  • CLI
    • Add --owner flag to doublezero user create-subscribe for specifying a custom user owner (foundation/sentinel only)

v0.14.0 - 2026-03-24

Breaking

Changes

  • Controller
    • Log an error when duplicate tunnel-id assignments are detected on the same device during state cache update, instead of silently overwriting
  • Onchain Programs
    • Serviceability: update device interface IPs when tunnel_net is changed via UpdateLink, matching the existing ActivateLink behavior (#3365)
    • Serviceability: AcceptLink supports combined accept+activate via use_onchain_allocation flag, gated on OnChainAllocation feature flag (#3369)
    • Serviceability: add feed_authority to RemoveMulticastGroupSubAllowlist auth check, matching AddMulticastGroupSubAllowlist
  • Client
    • Get client IP from the daemon in the disconnect command, matching the connect command's behavior, to avoid IP mismatches behind NAT
  • Onchain Programs
    • Add target_update_count field to GeoProbe account, incremented on AddTarget and RemoveTarget; uses BorshDeserializeIncremental so existing accounts default to 0 (non-breaking)
  • SDK
    • Add TargetUpdateCount field to Go GeoProbe struct with backward-compatible deserialization
  • Telemetry
    • Skip expensive GetGeolocationUsers RPC scan in geoprobe-agent when the probe's target_update_count is unchanged, with a forced full refresh every ~5 minutes as safety net
    • Add Prometheus metrics to geoprobe-agent: build info, error counters by type, discovery/measurement cycle durations, offset send/receive/reject counters, and discovered target/parent gauges; exposed via optional --metrics-enable flag

v0.13.0 - 2026-03-20

Breaking

Changes

  • Activator
    • Reserve device allocations (loopback IPs and segment routing IDs) for devices in Drained, DeviceProvisioning, and LinkProvisioning states at startup, preventing collisions with new device allocations
    • Reserve user allocations (tunnel_net, tunnel_id, dz_ip, publisher IPs, tunnel endpoints) for users in Updating and OutOfCredits states at startup, preventing collisions with new user allocations
    • Fix duplicate tunnel_net/tunnel_id allocation by reserving addresses for links in HardDrained, SoftDrained, and Provisioning states during startup initialization
  • Onchain Programs
    • Allow foundation to remove targets from GeolocationUser accounts via the RemoveTarget instruction, unblocking foundation-initiated user deletion when targets still exist
    • Serviceability: fix SubscribeMulticastGroup deriving the AccessPass PDA from payer_account.key instead of user.owner, which caused user delete to fail with "Invalid AccessPass PDA" when a foundation allowlist key signed and the user had active multicast subscriptions
    • feat(smartcontract): add tunnel_net/tunnel_id reallocation to UpdateLink (#3326)
  • Client
    • Fix v2/status returning empty current_device and metro for multicast subscribers by adding a clientIP + UserType fallback in status enrichment when DzIp and tunnel_dst matching both fail
    • Set tunnel interface administratively down before deleting during teardown, so external applications with sockets bound to the tunnel's overlay IP receive errors before the interface is removed
  • CLI
    • Include feed authority in global-config authority get output
    • Add geolocation user subcommands to manage GeolocationUser accounts and targets: create, delete, get, list, add-target, remove-target, and update-payment-status
  • Monitor
    • Fix slack user reporting
  • SDK
    • Add GeolocationUser types, Borsh deserialization, PDA derivation, and read-only client methods (GetGeolocationUserByCode, GetGeolocationUsers) to the Go geolocation SDK
  • Telemetry
    • Add onchain target discovery to the geoProbe agent: polls GeolocationUser accounts at 60s intervals, filters for activated+paid users, dynamically updates outbound probe targets and inbound signed TWAMP authorized keys
    • Fix geoProbe parent discovery incorrectly adding parent authority keys to the signed TWAMP reflector allowlist; parent DZDs use the unsigned reflector
  • Controller
    • Retry transient Solana RPC failures when fetching onchain serviceability accounts so controller polls are more resilient to short-lived provider resets

v0.12.0 - 2026-03-16

Breaking

Changes

  • Onchain Programs
    • Add GeolocationUser account state to the geolocation program with owner, billing config (flat-per-epoch), payment/user status, and a target list supporting outbound (IP + port) and inbound (pubkey) geolocation targets
    • Add GeolocationUser CRUD instruction processors (create, update, delete) with owner authorization and target-empty guard on delete
    • Add GeolocationUser target management (AddTarget, RemoveTarget) with duplicate/not-found guards, public IP validation, MAX_TARGETS limit, and GeoProbe reference count tracking
    • Add foundation-only UpdatePaymentStatus instruction to set payment status and last deduction epoch on GeolocationUser billing config
  • Client
    • Demote passive-mode liveness session-down log messages from Info to Debug to reduce log noise when no dataplane action is taken
  • Telemetry
    • Add Version (uint8) and TargetIP ([4]byte) fields to LocationOffset wire format (v1, 174 bytes), with version validation on unmarshal to enable safe future format evolution
  • Tools
    • Update TWAMP signed packet parser byte offsets and OffsetInfo struct for LocationOffset v1 layout
  • Onchain Programs
    • Serviceability: allow reservation authority to create, update, and close access passes, with ownership restriction preventing modification of passes created by other authorities
    • Serviceability: fix multicast user creation failing when access-pass has a tenant_allowlist; tenant enforcement only applies to unicast connections since multicast users are not tenant-scoped
    • Serviceability: rename reservation_authority_pk to feed_authority_pk in GlobalState and rename RESERVATION permission flag to FEED_AUTHORITY
    • Serviceability: remove ReserveConnection, CloseReservation, CreateReservedSubscribeUser, DeleteReservedSubscribeUser instructions and Reservation account type
  • E2E Tests
    • Fix TestE2E_UserLimits not asserting command failure: the ; echo EXIT_CODE=$? pattern caused the shell to always exit 0 regardless of the doublezero exit code, making err always nil; replace with require.Error assertions so the test fails if a limit-exceeded connect unexpectedly succeeds
    • Fix intermittent flake in TestE2E_MultiClientIBRL_RouteLiveness: add explicit route convergence checks after each unblock before starting the next block/unblock cycle, ensuring the liveness subsystem and BGP have fully settled
    • Add geoprobe E2E test (TestE2E_GeoprobeDiscovery) that exercises the full geolocation flow: deploy geolocation program, create probe onchain, start geoprobe-agent container, and verify the telemetry-agent discovers and measures the probe via TWAMP
    • Add geoprobe Docker image, geolocation program build/deploy support, and manager geolocation CLI configuration to the E2E devnet infrastructure
    • Extend geoprobe E2E test with outbound offset forwarding (agent → target with signature chain verification) and inbound signed TWAMP probing (target-sender → agent with DZD offset embedding); build geoprobe-target and geoprobe-target-sender binaries into the geoprobe Docker image
  • Telemetry
    • Add onchain parent DZD discovery to geoprobe-agent: periodically queries the Geolocation program for this probe's parent devices and resolves their metrics publisher keys from Serviceability, replacing the need for static --parent-dzd CLI flags. Static parents from CLI are merged with onchain parents, with onchain taking precedence for duplicate keys.
    • Optimize inbound probe-measured RTT accuracy: pre-sign both TWAMP probes before network I/O so probe 1 fires immediately after reply 0 with no signing delay, measure Tx-to-Rx interval (reply 0 Tx → probe 1 Rx) instead of Rx-to-Rx to exclude processing overhead on both sides, use kernel SO_TIMESTAMPNS receive timestamps on the reflector, and add a 15ms busy-poll window on the sender to avoid scheduler wakeup latency
    • Optimize outbound probe RTT accuracy: send a staggered warmup probe on a separate socket 2ms before the measurement probe to wake the reflector's thread, then take the min RTT of both

v0.11.0 - 2026-03-12

Breaking

Changes

  • Onchain Programs
    • Serviceability: split per-device multicast user tracking into separate subscriber and publisher counters (multicast_subscribers_count/max_multicast_subscribers and multicast_publishers_count/max_multicast_publishers); publisher and subscriber limits are now enforced independently
  • Controller
    • Downgrade transient ClickHouse write errors from ERROR to WARN, escalating to ERROR only after 3 consecutive flush failures to reduce alert noise (#3220)
  • Telemetry
    • Detect and replace unresponsive RIPE Atlas source probes that stop returning ping results, with a 24-hour TTL on the unresponsive probe blacklist so probes are retried after expiry
    • Compare source probe IDs (not just location codes) during measurement reconciliation so that probe replacements trigger measurement recreation
    • Fix race condition in internet-latency-collector where export and management goroutines independently loaded/saved the same state file, causing newly created RIPE Atlas measurement metadata to be overwritten and measurements to be stuck in a create-destroy-create loop (#3195)
    • Add wheresitup job backlog observability: pending_jobs Prometheus gauge, in_progress_count/pending_jobs in export summary logs, and API response duration histogram (#3203)
    • Change geoprobe-agent and geoprobe-target default TWAMP reflector port from 862 to 8925 to avoid DZD ACL blocks, use per-probe TWAMP port instead of hardcoded constant, and update --additional-child-probes/--additional-targets format to host or host:offset_port:twamp_port (two-field host:port rejected as ambiguous)
  • Activator
    • Fix tunnel ID leak in user activation: eagerly-allocated tunnel_id, tunnel_net, and dz_ip were not rolled back when the activation transaction failed, causing ghost IDs to accumulate and eventually exhaust the controller's tunnel slot range
    • Suppress noisy program log output from race conditions caused by dual event processing (websocket + snapshot poll). The SDK's new execute_transaction_quiet returns a SimulationError with program logs; the activator verifies suspected races by re-fetching user state before deciding whether to print logs (#3197)
  • CLI
    • Add doublezero-geolocation CLI for managing geolocation program entities: GeoProbe CRUD (create, get, list, update, delete), parent device management (add/remove), program config initialization, and geolocation-specific config get/set
    • Add --multicast-publishers-count and --multicast-subscribers-count flags to device update for foundation-gated count correction; rename --max-multicast-users to --max-multicast-subscribers and add --max-multicast-publishers
    • Add doublezero-admin device migrate-multicast-counts [--dry-run] to correct stale multicast_subscribers_count/multicast_publishers_count on existing deployments where all multicast users were previously counted as subscribers; supports dry-run preview and continues past per-device failures
    • Add doublezero-admin device migrate-unicast-counts [--dry-run] to correct stale unicast_users_count on all devices; same behaviour as the multicast counts command
  • SDK
    • Add read-only Go SDK for doublezero-geolocation program with state deserialization, PDA derivation, and RPC client for querying geoprobe configuration
    • Add GetGeoProbeKeys to geolocation SDK for lightweight account key fetching using DataSlice to minimize RPC bandwidth
    • Add Rust SDK instruction builders for GeolocationUser management: CRUD (create, update, delete), target management (add, remove), payment status updates, and read queries (get by code/pubkey, list all)
  • Telemetry
    • Add onchain GeoProbe discovery to the telemetry agent: periodically queries the Geolocation program for child probes parented to the local device, replacing the need for --additional-child-probes CLI flag
    • Embed LocationOffsets from parent DZDs in signed TWAMP replies so inbound probes carry geolocation context, and make signed TWAMP replies more like LocationOffsets to couple with a new double-probe system for inbound probing.
  • Client
    • Fix mainnet-beta Cloudsmith package containing the testnet binary by giving the mainnet-beta Rust build a separate CARGO_TARGET_DIR to prevent GoReleaser artifact collision
    • Increase default onchain fetch timeout from 20s to 60s to improve resilience on high-latency RPC paths; add -reconciler-fetch-timeout flag to allow operators to override
    • Add prometheus metrics for onchain RPC fetches: fetch duration histogram, result counter (success/error with stale cache/error with no cache), and stale cache age gauge
    • Increase default route liveness probe interval (TxMin/RxMin) from 300ms to 1s and raise MaxTxCeil from 1s to 3s to preserve backoff headroom
    • Throttle O(n) per-service scheduler queue length metric from every event to once per 10s to fix excessive CPU usage on nodes with many liveness sessions
  • Smartcontract
    • Serviceability: add foundation-only unicast_users_count, multicast_subscribers_count, and multicast_publishers_count fields to UpdateDevice instruction for direct count correction, with corresponding --unicast-users-count, --multicast-subscribers-count, and --multicast-publishers-count CLI flags
    • Serviceability: fix validate_account_code forcing lowercase on all entity types — restrict lowercase normalization to device and link codes only, preserving original case for locations, exchanges, contributors, and other entities
    • feat(smartcontract): atomic onchain allocation for CreateDevice (#3216)
    • Serviceability: RequestBanUser instruction supports atomic deallocate when OnchainAllocation feature is enabled
    • Serviceability: DeviceInterfaceUpdate instruction supports onchain allocation of node_segment_idx
    • Serviceability: fix validate_account_code forcing lowercase on all entity types — restrict lowercase normalization to device and link codes only, preserving original case for locations, exchanges, contributors, and other entities
    • feat(smartcontract): atomic onchain allocation for CreateDevice (#3216)
    • Serviceability: CreateDeviceInterface instruction supports atomic create+allocate+activate when OnchainAllocation feature is enabled
    • Serviceability: DeleteDeviceInterface instruction supports atomic deallocate+remove when OnchainAllocation feature is enabled
    • Serviceability: redesign reservations from per-IP accounts to per-(device, owner) reservation blocks with a configurable seat count, and integrate with user creation so creating a user can consume a reserved seat (#3224)
  • CLI
    • Add access-pass user-balances command to show per-payer SOL balance, required amount (rent + gas reserve), and missing amount, with filters (--user-payer, --min-balance, --max-balance, --min-missing, --max-missing), sorting, and --top N
    • Add access-pass fund command to top up underfunded user payers, with --dry-run, --force (skip confirmation), --min-balance, and a pre-transfer sender balance check; required balance floor includes a gas-fee reserve (50 × 5,000 lamports) and the wallet rent-exempt minimum to prevent on-chain transfer failures
    • Fix access-pass fund and access-pass user-balances slot counting to track remaining slots per (payer, client_ip) pair, preventing a connected user on IP_B from consuming open slots for IP_A; also fix required balance formula to always add wallet_rent_min on top of needs_rent rather than taking the max
    • Add --user-payer filter to user list command
    • Serviceability: onchain activation - atomic close for DeleteDevice (#3188)
    • Fix access-pass user-balances and access-pass fund underestimating the required wallet balance: the wallet's own rent-exempt minimum was used as a floor rather than being added, causing missing: 0 to be reported even when provisioning would fail with insufficient funds (#3213)
  • Onchain Programs
    • Serviceability: DeleteUser instruction supports atomic deallocate+closeaccount when OnchainAllocation feature is enabled
    • Serviceability: CreateLink instruction supports atomic create+allocate+activate when OnchainAllocation feature is enabled
    • Serviceability: DeleteLink instruction supports atomic deallocate+closeaccount when OnchainAllocation feature is enabled
    • Serviceability: CreateMulticastGroup instruction supports atomic create+allocate+activate when OnchainAllocation feature is enabled
    • Serviceability: DeleteMulticastGroup instruction supports atomic deallocate+closeaccount when OnchainAllocation feature is enabled
    • Serviceability: CreateSubscribeUser instruction supports atomic create+allocate+activate when OnchainAllocation feature is enabled
    • Serviceability: MulitcastGroupUpdate instruction supports atomic ip allocation when OnchainAllocation feature is enabled
    • Serviceability: UpdateUser instruction supports atomic allocate+deallocate when OnchainAllocation feature is enabled

v0.10.0 - 2026-03-04

Breaking

Changes

  • CLI
    • doublezero resource verify command will now suggest creating resources or create them with --fix
    • Print an explicit message when the tenant is resolved implicitly from the configuration file (Using tenant '...' from configuration file.) or from the Access Pass allowlist (Using tenant '...' from Access Pass.)
    • All get commands now display output as a formatted table and support a --json flag for machine-readable output
    • Remove solana-multicast-publisher and solana-multicast-subscriber from access-pass set type options (multicast roles use the tenant field on a prepaid pass instead); make --solana-validator optional for solana-rpc type
    • get commands now expose all onchain account fields: user get adds tunnel_id, tunnel_endpoint, and validator_pubkey; device get adds reserved_seats; link get adds tunnel_id; multicastgroup get adds tenant, publisher_count, and subscriber_count; tenant get adds payment_status, billing, administrators, and token_account; exchange get adds device1_pk and device2_pk
  • SDK
    • Fix multicast group deserialization in smartcontract/sdk/go to correctly read publisher and subscriber counts and align status enum with onchain definition
  • Smartcontract
    • Serviceability: add Reservation account and ReserveConnection/CloseReservation instructions for pre-reserving connection seats on devices, with reserved_seats factored into capacity checks on both reservation and user creation
    • Allow sentinel authority to add/remove multicast publisher and subscriber allowlist entries
  • SDK
    • Add Rust SDK for geolocation program with GeoClient, GeoProbe CRUD operations (create, get, list, update, delete, add/remove parent device), and ProgramConfig management (init, get, update)
    • Add geo_program_id to NetworkConfig for geolocation program discovery
  • Telemetry
    • Add geoprobe-target-sender CLI tool for sending signed TWAMP probes to a GeoProbe and verifying signed replies (RFC16 inbound probing flow)
    • Add Signed TWAMP reflector to geoprobe-agent with configurable listen port and allowed-pubkeys allowlist
    • Fix global monitor crash when IBRL and multicast users share the same client IP but are on different devices, by preferring non-multicast users in client IP lookups to match status device selection
  • Client
    • Add onchain reconciler to daemon — automatically provisions/removes tunnels by polling onchain User state, replacing CLI-driven provisioning and the doublezerod.json state file (RFC-17)
    • Add doublezero enable / doublezero disable CLI commands to toggle the reconciler at runtime
  • E2E tests
    • Publish TestQA_AllDevices_UnicastConnectivity results to ClickHouse (qa_alldevices_results and qa_alldevices_metadata tables) in addition to InfluxDB; configured via CLICKHOUSE_ADDR env var, skipped gracefully when not set
  • Onchain Programs
    • Serviceability: CreateUser instruction supports atomic create+allocate+activate when OnchainAllocation feature is enabled

v0.9.0 - 2026-02-27

Breaking

  • CLI: --bandwidth is now a required argument for doublezero device interface create and doublezero device interface update; callers that previously omitted it (relying on the default of 0) must now explicitly pass a value with a unit (e.g. --bandwidth 10Gbps)

Changes

  • Tools
    • Add signed TWAMP sender and reflector with Ed25519 authentication and per-pubkey rate-limited signature verification
  • Activator
    • Adds a stateless mode for when onchain allocation is enabled. This prevents activator/onchain from becoming out-of-sync.
    • --onchain-allocation cli flag replaced with feature flag from GlobalState onchain
  • SDK
    • Add retry with exponential backoff (3 retries, 500ms–5s) to all read-only RPC calls in DZClient, improving resilience to transient RPC timeouts
  • CLI
    • Fix doublezero status showing "Current Device" and "Metro" as N/A for multicast subscribers when the tunnel destination is a user_tunnel_endpoint loopback interface IP rather than the device's public_ip
    • Remove redundant connect ibrl unit tests that were duplicates of hybrid-device equivalents
    • doublezero global-config feature-flags commands added
    • Fix multicast subscriber tunnel source resolution for NAT environments — resolve local interface IP instead of using public IP
    • Added multicast filters to access-pass list, enabling filtering by publisher/subscriber role and identifying access passes not authorized for a specific multicast group.
    • Device interface --bandwidth and --cir flags now accept Kbps, Mbps, or Gbps units; interface list displays those values as human-readable strings
    • Add duplicate IP check to prevent a user from assigning the same IP more than once
  • Client
    • Fix BGP OnClose deleting routes from all peers instead of only the closing peer, preventing multicast teardown from nuking unicast routes
    • Skip route deletion on OnClose for NoInstall peers (multicast) since they never install kernel routes
    • Reject BGP martian addresses (CGNAT, multicast, reserved, benchmarking, etc.) as client IP during connect
  • Controller
    • detect duplicate (UnderlaySrcIP, UnderlayDstIP) pairs for tunnels and only render the first to the device config and write a log error for the second
    • allow MTU to be configurable
  • Onchain Programs
    • Serviceability: skip field validation for users in Deleting status to prevent accounts from getting stuck during cleanup when validation rules change
    • Serviceability: require foundation_allowlist privileges to update node_segment_idx on a device interface
    • Serviceability: add feature flags support
    • Serviceability: expand is_global to reject all BGP martian address ranges (CGNAT 100.64/10, IETF 192.0.0/24, benchmarking 198.18/15, multicast 224/4, reserved 240/4, 0/8)
    • Serviceability: allow update and deletion of interfaces even when sibling interfaces have invalid CYOA configuration
    • Geolocation: add doublezero-geolocation program scaffolding and GeoProbe account type and related instructions as per rfcs/rfc16-geolocation-verification.md
  • SDK
    • SetFeatureFlagCommand added to manage on-chain feature flags for conditional behavior rollouts
  • Dependencies
    • Upgrade Solana SDK workspace dependencies from 2.2.7 to 2.3.x (solana-sdk, solana-client, solana-program-test, and others)
  • Internet telemetry
    • Reduce RIPE Atlas sampling interval from 6 minutes to 10 minutes to work around service limit of 100_000 samples per day
  • E2E
    • e2e: add multi-tenant access control negative tests (#3081)

v0.8.11 - 2026-02-27

Breaking

  • N/A

Changes

  • Client
    • Fix BGP OnClose deleting routes from all peers instead of only the closing peer, preventing multicast teardown from nuking unicast routes

v0.8.10 – 2026-02-19

Breaking

  • N/A

Changes

  • Activator
    • removes accesspass monitor task (that expires access passes)
  • Monitor
    • Add Prometheus metrics for multicast publisher block utilization (doublezero_multicast_publisher_block_total_ips, doublezero_multicast_publisher_block_allocated_ips, doublezero_multicast_publisher_block_utilization_percent) — enables Grafana alerting on IP pool exhaustion thresholds
    • Include tenant name in the Slack notification table when new users are activated
  • SDK (Go)
    • Add GetMulticastPublisherBlockResourceExtension() method to serviceability client for fetching the global multicast publisher IP allocation bitmap
    • Fix LinkDesiredStatus discriminants (hard-drained=6, soft-drained=7)
  • Onchain Programs
    • Upgrade programs from system_instruction to solana_system_interface
    • Refactor user creation to validate all limits (max_users, max_multicast_users, max_unicast_users) before incrementing counters — improves efficiency by avoiding wasted work on validation failures and follows fail-fast best practice
    • Serviceability: UnlinkDeviceInterface now only allows Activated or Pending interfaces; when an associated link account is provided for an Activated interface, the link must be in Deleting status
    • Links and devices can no longer be deleted from Activated status — must be drained first; deletion is rejected with InvalidStatus
    • Contributors, locations, multicast groups, and users can now be deleted from any operational status (not just Activated); only Deleting/Updating states are blocked
    • SDK: UnlinkDeviceInterfaceCommand automatically discovers and passes associated link accounts
    • Serviceability: allow contributors to update prefixes when for IBRL when no users are allocated
  • CLI
    • doublezero status now shows a Tenant column (between User Type and Current Device) with the tenant code associated with the user; empty when no tenant is assigned
  • Client
    • Fix tunnel overlay address scope to prevent kernel from selecting link-local /31 as source for routed traffic
    • Serviceability: close orphaned blocks when dz_prefixes shrink
    • Serviceability: require ip-net for CYOA interfaces
  • E2E / QA Tests
    • Fix QA unicast test flake caused by RPC 429 rate limiting during concurrent user deletion — treat transient RPC errors as non-fatal in the deletion polling loop
    • Backward compatibility test: use --status instead of --desired-status for drain commands; fix version ranges (link drain compatible since v0.7.2, device drain since v0.8.1)
    • Add daily devnet QA test for device provisioning lifecycle (RFC12) — deletes/recreates device and links, restarts daemons with new pubkey via Ansible
    • Remove devnet/testnet environment filter from TestQA_MulticastPublisherMultipleGroups — test now runs against all environments

v0.8.9 – 2026-02-16

Breaking

  • N/A

Changes

  • Activator
    • Fail to start if any global config network blocks (device_tunnel_block, user_tunnel_block, multicastgroup_block, multicast_publisher_block) are unset (0.0.0.0/0)
    • Fix multicast publisher dz_ip leak in offchain deallocation — IPs from the global publisher pool were never freed on user deletion because the check required non-empty publishers, which the smartcontract already clears before allowing deletion
  • Client
    • Fix heartbeat sender not restarting after disconnect due to poisoned done channel
  • Onchain Programs
    • bugfix(serviceability): contributors can now update their interfaces, CYOA interfaces saved on create, physical interfaces remain after unlink (#2993)
  • Device controller
    • Reject users with BGP martian DZ IPs (RFC 1918, loopback, multicast, link-local, documentation nets, shared address space, reserved) to prevent invalid addresses from being advertised via BGP or permitted in device ACLs
  • Claude
    • chore: update CLAUDE.md (#2999)
  • e2e
    • e2e: Add network contributor e2e flow tests (#2997)

v0.8.8 – 2026-02-13

Breaking

  • None for this release

Changes

  • Activator
    • Assign multicast publisher IPs from global pool in serviceability GlobalConfig instead of per-device blocks
  • Client
    • Add multicast publisher heartbeat sender — sends periodic UDP packets to each multicast group to keep PIM (S,G) mroute state alive on devices
    • Fix panic in heartbeat sender when concurrent teardown requests race on close
  • E2E tests
    • Add daily devnet QA test for device provisioning lifecycle (RFC12) — deletes/recreates device and links, restarts daemons with new pubkey via Ansible
  • Serviceability: prevent creating or activating links on interfaces with CYOA or DIA assignments, and prevent setting CYOA/DIA on interfaces that are already linked
  • CLI: add early validation in link wan-create and link dzx-create to reject interfaces with CYOA or DIA assignments

v0.8.7 – 2026-02-10

Breaking

  • None for this release

Changes

  • SDK
    • Added Tenant to all sdks
  • E2E tests
    • Added multi-tenancy deletion test coverage
  • Telemetry
    • Add doublezero-geoprobe-agent, intermediary probe server for RFC16
    • Adds support for per-tenant metro routing policy
    • Add --geoprobe-pubkey flag to doublezero-geoprobe-agent for device identity
    • LocationOffset struct now includes SenderPubkey to distinguish individual devices that share the same signing authority
  • Cli
    • Automatic detection of the authorized tenant is added.
    • The delete tenant command allows cascading deletion of users and access passes.
  • Telemetry
    • extend device telemetry agent to measure RTT to child geoProbes via TWAMP, generate signed LocationOffset structures, and deliver them via UDP as per rfcs/rfc16-geolocation-verification.md
    • geoprobe-target: example target listener for geolocation verification with TWAMP reflector, UDP offset receiver, signature chain verification, distance calculation logging, and DoS protections (5-reference depth limit and per-source-IP rate limiting) (#2901)
  • Onchain programs
    • feat(serviceability): add TenantBillingConfig and epoch tracking to UpdatePaymentStatus (#2922)
    • feat(smartcontract): add payment_status, token_account fields and UpdatePaymentStatus instruction (#2880)
    • fix(smartcontract): correctly ser/deser ops_manager_pk (#2887)
    • Serviceability: add metro_routing and route_liveness boolean fields to Tenant for routing configuration
    • Serviceability: add Tenant account type with immutable code-based PDA derivation, VRF ID, administrator management, and reference counting for safe deletion
    • Serviceability: add TenantAddAdministrator and TenantRemoveAdministrator instructions for foundation-managed administrator lists
    • Serviceability: extend UserUpdate instruction to support tenant_pk field updates with automatic reference count management on old and new tenants (backward compatible with old format)
    • Serviceability: extend UserCloseAccount instruction to decrement tenant reference count when closing user with assigned tenant
    • Serviceability: add reference count validation in DeleteMulticastGroup to prevent deletion when active publishers or subscribers exist
    • Serviceability: fix multicast group closeaccount to use InvalidStatus error and remove redundant publisher/subscriber count check
    • Serviceability: add tenant_allowlist field to AccessPass to restrict which tenants can use specific access passes (backward compatible with existing accounts)
    • Serviceability: bypass validation for link delete (#2934)
    • Serviceability: add per-device unicast and multicast user limits with separate counters and configurable max values (RFC-14)
    • Fix link device & link updates
  • SDK
    • Add metro_routing and route_liveness fields to CreateTenantCommand and UpdateTenantCommand
    • Add CreateTenant, UpdateTenant (vrf_id only), DeleteTenant, GetTenant, and ListTenant commands with support for code or pubkey lookup
    • Add AddAdministratorTenant and RemoveAdministratorTenant commands for tenant administrator management
    • UpdateUserCommand extended with tenant_pk field and automatic tenant account resolution for reference counting
    • SetAccessPassCommand extended with tenant field to specify allowed tenant for access pass
    • TypeScript SDK updated with tenantAllowlist field in AccessPass interface and deserialization
  • CLI
    • Fix multicastgroup update command to properly parse human-readable bandwidth values (e.g., "1Gbps", "100Mbps") in --max-bandwidth flag
    • Add --metro-route and --route-aliveness flags to tenant create and update commands
    • Add tenant subcommands (create, update, delete, get, list, add-administrator, remove-administrator) to doublezero and doublezero-admin CLIs
    • Support simultaneous publisher and subscriber multicast via --publish and --subscribe flags
    • Add --max-unicast-users and --max-multicast-users flags to device update command
    • Add filtering options and desired_status & metrics_publisher_pk field to device and link list commands
    • Added activation check for existing users before subscribing to new groups (#2782)
    • access-pass set: add --tenant argument to specify tenant code for access pass restriction (converts to tenant PDA onchain)
    • tenant list: improve output formatting with table support and JSON serialization options (--json, --json-compact)
    • default tenant support added to config
  • SDK
    • Add read-only Go SDK (revdist) for the revenue distribution Solana program, with typed deserialization of all onchain accounts and Rust-generated fixture tests for cross-language compatibility
    • Add revdist-cli tool for inspecting onchain revenue distribution state
    • Add Python and TypeScript SDKs for serviceability, telemetry, and revdist programs with typed deserialization, RPC clients, PDA derivation, enum string types, and cross-language fixture tests
    • Add shared borsh-incremental library (Go, Python, TypeScript) for cursor-based Borsh deserialization with backward-compatible trailing field defaults
    • Add npm and PyPI publish workflows for serviceability and telemetry SDKs
    • DeleteUserCommand updated to wait for activator to process multicast user unsubscribe before deleting the user
  • Device controller
    • Record successful GetConfig gRPC calls to ClickHouse for device telemetry tracking
    • Multi-tenancy vrf support added
    • Skip isis and pim config for CYOA/DIA tagged interfaces
  • Onchain programs
    • Enforce that CloseAccessPass only closes AccessPass accounts when connection_count == 0, preventing closure while active connections are present.
  • Monitor
    • Add sol-balance watcher to track SOL balances for configured accounts and export Prometheus metrics for alerting
  • Client
    • Support simultaneous publisher and subscriber multicast in the daemon
  • Telemetry
    • Add consecutive-loss-based sender eviction to the telemetry collector so broken TWAMP senders are recreated quickly instead of persisting until TTL expiry (--max-consecutive-sender-losses, default 30)
  • E2E tests
    • e2e: add multi-tenancy VRF isolation test (#2891)
    • Add backward compatibility test that validates older CLI versions against the current onchain program by cloning live state from testnet and mainnet-beta
    • QA multicast tests: add diagnostic dumps on failure (status, routes, latency, multicast reports, onchain user/device state), cleanup stale test groups at test start, and fix disconnect blocking on stuck daemon status

v0.8.6 – 2026-02-04

Breaking

  • None for this release

Changes

  • CLI
    • Remove log noise on resolve route
    • doublezero resource verify command added to verify onchain resources
    • Enhance delete multicast group command to cascade into deleting AP entry (#2754)
  • Onchain programs
    • Removed device and user allowlist functionality, updating the global state, initialization flow, tests, and processors accordingly, and cleaning up unused account checks.
    • Serviceability: require DeactivateMulticastGroup to only close multicast group accounts when both publisher_count and subscriber_count are zero, preventing deletion of groups that still have active publishers or subscribers.
    • Deprecated the user suspend status, as it is no longer used.
    • Serviceability: enforce that CloseAccountUser instructions verify the target user has no multicast publishers or subscribers (both publishers and subscribers are empty) before closing, and add regression coverage for this behavior.
    • Enhance access pass functionality with new Solana-specific types
    • fix default desired status
  • Telemetry
    • Fix goroutine leak in TWAMP sender — cleanUpReceived goroutines now exit on Close() instead of living until process shutdown
  • Client
    • Cache network interface index/name lookups in liveness UDP service to fix high CPU usage caused by per-packet RTM_GETLINK netlink dumps
    • Add observability to BGP handleUpdate: log withdrawal/NLRI counts per batch and track processing duration via doublezero_bgp_handle_update_duration_seconds histogram
  • E2E tests
    • The QA alldevices test now skips devices that are not calling the controller
    • e2e: Expand RFC11 end-to-end testing (#2801)
    • e2e(RFC11): add dz prefix rollover allocation test (#2820)

v0.8.5 – 2026-02-02

Breaking

  • None for this release

Changes

  • Smartcontract
    • fix(smartcontract): reserve first IP of DzPrefixBlock for device (#2753)
  • Client
    • Fix race in bgp status handling on peer deletion

v0.8.4 – 2026-01-28

Breaking

  • None for this release

Changes

  • Telemetry
    • Force IPv4-only connections for gNMI tunnel client and fix TLS credential handling
  • Client
    • Support simultaneous unicast and multicast tunnels in doublezerod
    • Support publishing and subscribing to multiple multicast groups simultaneously
  • CLI
    • Support publishing and subscribing a user to multiple multicast groups via --group flag
    • Remove single tunnel constraint
  • SDK
    • Go SDK can now perform batch writes to device.health and link.health as per rfc12
  • Activator
    • fix(activator): add on-chain allocation support for users (#2744)
    • On-chain allocation enabled
  • Smartcontract
    • feat(smartcontract): add use_onchain_deallocation flag to MulticastGroup (#2748)
  • CLI
    • Remove restriction for a single tunnel per user; now a user can have a unicast and multicast tunnel concurrently (but can only be a publisher or a subscriber) (2728)

v0.8.3 – 2026-01-22

  • Data
    • Add indexer that syncs serviceability and telemetry data to ClickHouse and Neo4J

Breaking

  • None for this release

Changes

  • CLI
    • Remove log noise on resolve route
    • Add global-config qa-allowlist commands to manage QA identity allowlist to bypass status and max_users checks in QA
    • Add "-skip-capacity-check" flag to bypass status and max_users checks in QA to test devices that are still being provisioned
    • Remove "unknown" status from doublezero status command and implement "failed" and "unreachable" statuses
  • Client
    • Enable route liveness passive-mode by default
    • Add make install make target. To build and deploy from source, users can now run cd client && make build && make install to install the doublezero and doublezerod binaries and the doublezerod systemd unit.
  • Onchain programs
    • Serviceability: remove validation check for interface delete (#2707)
    • Serviceability: interface-cyoa only on physical interfaces, don't require interfaces to be tagged, add same validation logic to update interface (#2700)
    • Enforce Activated status check before suspending contributor, exchange, location, and multicastgroup accounts
    • Removed device and user allowlist functionality, updating the global state, initialization flow, tests, and processors accordingly, and cleaning up unused account checks.
    • Serviceability: require DeactivateMulticastGroup to only close multicast group accounts when both publisher_count and subscriber_count are zero, preventing deletion of groups that still have active publishers or subscribers.
    • Deprecated the user suspend status, as it is no longer used.
    • Serviceability: enforce that CloseAccountUser instructions verify the target user has no multicast publishers or subscribers (both publishers and subscribers are empty) before closing, and add regression coverage for this behavior.
    • Removed device and user allowlist functionality, updating the global state, initialization flow, tests, and processors accordingly, and cleaning up unused account checks.
    • SetGlobalConfig, ActivateDevice, UpdateDevice and CloseAccountDevice instructions updated to manage resource accounts.
    • Add option for Contributor B to reject a link created by Contributor A just as Contributor A can cancel its own created link
  • Telemetry
    • Add gNMI tunnel client for state collection
  • Activator
    • fix(activator): ip_to_index fn honors ip range #2658
  • E2E tests
    • Add influxdb, prometheus, and device-health-oracle containers
    • Add interface lifecycle tests (#2700)
    • Only fail QA alldevices test run if device status is "Activated" and max users > 0
  • SDK
    • Commands for setting global config, activating devices, updating devices, and closing device accounts now manage resource accounts.
    • Serviceability: return error when GetProgramAccounts returns empty result instead of silently returning empty data
  • Smartcontract
    • feat(smartcontract): RFC 11 activation for User entity
    • feat(smartcontract): RFC 11 add on-chain resource allocation for Link
  • Device Health Oracle
    • Add new device-health-oracle component. See rfcs/rfc12-network-provisioning.md for details.
    • Calculate burn-in timestamp based from slot numbers (current minus 200_000 slots for provisioning, current minus 5_000 slots for maintenance)
  • CI
    • Add separate apt repo for doublezero-testnet

v0.8.2 – 2025-01-13

Breaking

  • None for this release

Changes

  • Client
    • Always delegate RouteAdd regardless of noUninstall flag
  • Telemetry
    • Include solana vote pubkey in global monitor metrics
    • Run telemetry agent on pending and drained links

v0.8.1 – 2025-01-12

Breaking

Changes

  • Onchain programs
    • Serviceability: enforce that ActivateLink and CloseAccountLink instructions verify the provided side A/Z device accounts match the link's stored side_a_pk and side_z_pk before proceeding.
  • CLI
    • Update contributor, device, exchange, link, location, and multicast group commands to ignore case when matching codes
    • ActivateMulticastGroup now supports on-chain IP allocation from ResourceExtension bitmap (RFC 11).
    • IP address lookup responses that do not contain a valid IPv4 address (such as upstream timeout messages) are now treated as retryable errors instead of being parsed as IPs.
    • doublezero resource commands added for managing ResourceExtension accounts.
    • Added health_oracle to the smart contract global configuration to manage and authorize health-related operations.
    • Added --ip-net support to create to match the existing behavior in update.
    • Use DZ IP for user lookup during status command instead of client IP
  • Onchain programs
    • Fix CreateMulticastGroup to use incremented globalstate.account_index for PDA derivation instead of client-provided index, to ensure the contract is the authoritative source for account indices
    • Add on-chain validation to reject CloseAccountDevice when device has active references (reference_count > 0)
    • Allow contributor owner to update ops manager key
    • Add new arguments on create interface cli command
    • Serviceability: enforce that resume instructions for locations, exchanges, contributors, devices, links, and users only succeed when the account status is Suspended, returning InvalidStatus otherwise, and add tests to cover the new behavior.
    • RequestBanUser: only allow requests when user.status is Activated or Suspended; otherwise return InvalidStatus
    • Serviceability: require device interfaces to be in Pending status before they can be rejected, and add tests to cover the new status check
    • Add ResourceExtension to track IP/ID allocations. Foundation instructions added to create/allocate/deallocate.
    • ResourceExtension optimization using first_free_index for searching bitmaps
    • Added the INSTRUCTION_GUIDELINES document defining the standard for instruction creation.
    • Enforce best practices for instruction implementation across onchain programs
    • Add missing system program account owner checks in multiple instructions
    • Refactor codebase for improved maintainability and future development
    • Introduced health management for Devices and Links, adding explicit health states, authorized health updates, and related state, processor, and test enhancements.
    • Require that BanUser can only be executed when the target user's status is PendingBan, enforcing the expected user ban workflow (request-ban -> ban).
    • Introduce desired status to Link and Devices
    • Introduced health management for Devices and Links, adding explicit health states, authorized health updates, and related state, processor, and test enhancements.
    • Restrict DeleteDeviceInterface to interfaces in Activated or Unlinked status; attempting to delete interfaces in other statuses now fails with InvalidStatus.
    • Updated validation to allow public IP prefixes for CYOA/DIA, removing the restriction imposed by type-based checks.
    • Transit devices can now be provisioned without a public IP, aligning the requirements with their actual networking model and avoiding unnecessary configuration constraints.
    • Enforce that ActivateDeviceInterface only activates interfaces in Pending or Unlinked status, returning InvalidStatus for all other interface states
    • Introduce desired status to Link and Devices
  • Internet Latency Telemetry
    • Fixed a bug that prevented unresponsive ripeatlas probes from being replaced
    • Fixed a bug that caused ripeatlas samples to be dropped when they were delayed to the next collection cycle
  • Link & device Latency Telemetry
    • Telemetry data can now be received while entities are in provisioning and draining states.
  • Device controller
    • Add histogram metric for GetConfig request duration
    • Add gRPC middleware for prometheus metrics
    • Add device status label to controller_grpc_getconfig_requests_total metric
    • Add logic to shutdown user BGP, IBGP sessions, MSDP neighbors, and ISIS when device.status is drained
  • Device agents
    • Increase default controller request timeout in config agent
    • Initial state collect in telemetry agent
  • Client
    • Route liveness treats peers that advertise passive mode as selectively passive; does not manage their routes directly.
    • Route liveness runs in passive mode for IBRL with allocated IP, if global passive mode is enabled.
    • Advertise peer client version with route liveness control packets.
    • Add doublezero_bgp_routes_installed gauge metric for number of installed BGP routes
    • Add route liveness gauges for in-memory maps
    • Route liveness sets set of routes configured as excluded to AdminDown.
    • Add histogram metric for BGP session establishment duration
    • For IBRL with allocated IP mode, resolve tunnel source IP from routing table via resolve-route API endpoint instead of using client IP to support clients behind NAT
    • Configure MTU down to 1476 on client tunnel in case path MTU discovery is not working
    • Increase route liveness max backoff duration
  • Global monitor
    • Initial implementation
  • Release
    • Publish a Docker image for core components.
  • Telemetry
    • Refactor flow enricher
    • Add metrics to flow enricher
    • Add serviceability data fetching to flow enricher
    • Add flow-ingest service
    • Add annotation of flow records with serviceability data
    • Add pcap input and json ouput to flow enricher
    • Initial state-ingest service with client SDK
    • Collect BGP socket state from devices
  • CI
    • Cancel existing e2e test runs on the push of new commits
  • RFCs
    • RFC - Network Provisioning
    • RFC-11: Onchain Activation (#2302)
  • Monitor
    • Add link status to device-telemetry metrics to enable Grafana alerts to filter out links that are not in activated status
    • Add validation for 2Z oracle swapRate to ensure it is an unsigned integer, with warning logs and metrics for malformed values
  • E2E tests
    • Add GetLatency call to qaagent
    • The QA alldevices test now considers device location and connects hosts to nearby devices
    • QA agent and tests now support doublezero connect ibrl's --allocate-addr flag
    • The QA alldevices test now publishes success/failure metrics to InfluxDB in support of rfc12
  • Onchain programs
    • Fix CreateMulticastGroup to use incremented globalstate.account_index for PDA derivation instead of client-provided index, to ensure the contract is the authoritative source for account indices
    • ReactivateMulticastGroup now enforces that the multicast group status must be Suspended before reactivation, returning InvalidStatus otherwise; negative-path regression tests were added.

v0.8.0 – 2025-12-02

Breaking

  • None for this release

Changes

  • RFCs
    • RFC-10: Version Compatibility Windows
  • CLI
    • IP address lookups via ifconfig.me are retried up to 3 times to minimize transient network errors.
    • Added global --no-version-warning flag to the doublezero client and now emit version warnings to STDERR instead of STDOUT to improve scriptability and logging.
    • Add the ability to update a Device’s location, managing the reference counters accordingly.
    • Added support in the link update command to set a link’s status to soft_drained or hard_drained.
    • Added support for specifying device_type at creation, updating it via device update, and displaying it in list/detail outputs.
    • Add support for updating contributor.ops_manager_key.
    • Add migrate command to upgrade legacy user accounts from index-based PDAs to the new IP + connection-type scheme.
    • Enhance access-pass list with client-IP and user-payer filters
    • Support added to load keypair from stdin
  • Client
    • Add route liveness fault-injection simulation tests.
    • Updated the interface list command to display all interfaces when no device is specified.
  • Funder
    • Fund multicast group owners
  • Onchain programs
    • Serviceability Program: Updated the device update command to allow modifying a device’s location.
    • Added new soft-drained and hard-drained link status values to serviceability to support traffic offloading as defined in RFC9.
    • Fix ProgramConfig resize during global state initialization.
    • Standardized the device_type enum to Edge, Transit, and Hybrid, added validation rules, and defaulted existing devices to Hybrid for backward compatibility.
    • Add contributor.ops_manager_key for authorizing incident management operations.
    • User Account: Replace global-index PDA generation with deterministic IP + connection-type seeds, eliminating user-creation race conditions.
    • Enable on-chain storage of InterfaceV2, allowing devices to register updated interface metadata
    • Serviceability: validate that a device's public IP doesn't clash with its dz_prefixes
  • QA
    • Traceroute when packet loss is detected
  • Tools
    • Add solana-tpu-quic-ping tool for testing Solana TPU-QUIC connections with stats emitted periodically
  • Device controller
    • Handle new link.status values (soft-drained and hard-drained) as per RFC9
  • Monitor
    • Export links data to InfluxDB
  • Activator
    • Uses asynchronous coroutines instead of blocking operations and threads.

v0.7.1 – 2025-11-18

Breaking

  • None for this release

Changes

  • RFCs
    • RFC9 Link Draining
  • Client
    • Switch to 64 byte latency probes instead of 32 bytes
    • Route liveness admin-down signalling and ignore stale remote-down messages
    • Added an on-chain minimum supported CLI version to allow multiple CLI versions to operate simultaneously.
  • Smart contract
    • Delay V2 Interface Activation Until All Clients Support V2 Reading
  • Device controller
    • Now accepts the config agent's version in the grpc GetConfig call and includes it as a label in the controller_grpc_getconfig_requests_total metric
  • Device Agent
    • Now sends its version to the controller in the grpc GetConfig call

v0.7.0 – 2025-11-14

Breaking

  • Smart contract
    • Introduces CYOA and DIA as new possible interface types

Changes

  • Onchain programs
  • CLI
    • Added support for specifying the interface type during interface creation and modification, introducing CYOA and DIA as new possible interface types.
    • Improve error message when connecting to a device that is at capacity or has max_users=0. Users now receive "Device is not accepting more users (at capacity or max_users=0)" instead of the confusing "Device not found" error when explicitly specifying an ineligible device.
    • Add link latency command to display latency statistics from the telemetry program. Supports filtering by percentile (p50, p90, p95, p99, mean, min, max, stddev, all), querying by link code or all links, and filtering by epoch. Resolves: #1942
    • Added --contributor | -c filter to device list, interface list, and link list commands. (#1274)
    • Validate AccessPass before client connection (#1356)
  • Client
    • Add initial route liveness probing, initially disabled for rollout
    • Add route liveness prometheus metrics

v0.6.11 – 2025-11-13

Breaking

  • None for this release

Changes

  • Note that the changes from this release have been bundled into 0.7.0

v0.6.10 – 2025-11-05

Breaking

  • None for this release

Changes

  • CI
    • Add automated compatibility tests in CI to validate all actual testnet and mainnet state against the current codebase, ensuring backward compatibility across protocol versions.
    • Add --delay-override-ms option to doublezero link update
    • Add ability to configure excluded routes
  • Device controller
    • Remove the deprecated -enable-interfaces-and-peers flag
    • Use link.delay_override to set isis metric when in valid range. This provides a simple workflow for contributors to temporarily change a link's delay value without overwriting the existing value.
  • Onchain programs
    • serviceability: add delay_override_ns field to link

v0.6.9 – 2025-10-24

Breaking

  • None for this release

Changes

  • Onchain programs
    • serviceability: add auto-assignment and validation for exchange.bgp_community
    • serviceability: prevent device interface name duplication
    • Update serviceability and telemetry program instruction args to use the BorshDeserializeIncremental derive macro incremental, backward-compatible, deserialization of structs.
    • Add explicit signer checks for payer accounts across various processors to improve security and ensure correct transaction authorization.
  • CLI
    • Removed --bgp-community option from doublezero exchange create since these values are now assigned automatically
    • Add --next-bgp-community option to doublezero global-config set so authorized users can control which bgp_community will be assigned next
  • Tools
    • TWAMP: Verify that the sequence number and timestamp of the received packet matches those of the sent packet
    • Uping: Add minimal ICMP echo library for user-space liveness probing over doublezero interfaces, even when certain routes are not in the the kernel routing table.
  • Device controller
    • Deprecate the -enable-interfaces-and-peers flag. The controller now always renders interfaces and peers
    • Intra-exchange routing policy, which uses the onchain exchange.bgp_community value to route traffic between users in the local exchange over the internet
  • Monitor
    • Add metrics that detect when duplicate or out-of-range exchange.bgp_community values exist in serviceability

v0.6.8 – 2025-10-17

Breaking

  • Multicast group change: Regeneration of all multicast group allowlists required, as allowlists are now stored within each Access Pass instead of at the multicast group level.

Changes

  • Onchain programs
    • Serviceability: enforce that ActivateLink and CloseAccountLink instructions verify the provided side A/Z device accounts match the link's stored side_a_pk and side_z_pk before proceeding.
  • CLI
    • Added a wait in the disconnect command to ensure the account is fully closed before returning, preventing failures during rapid disconnect/reconnect sequences.
    • Display multicast group memberships (publisher/subscriber) in AccessPass listings to improve visibility.
    • Allow AccessPass creation without 'client_ip'
    • Add 'allow_multiple_ip' argument to support AccessPass connections from multiple IPs
    • Include validator pubkey in export output
    • Rename exchange.loc_id to exchange.bgp_community
    • status command now shows connected and lowest latency DZD
  • Activator
    • Reduce logging noise when processing snapshot events
    • Wrap main select handler in loop to avoid shutdown on branch error
  • Onchain programs
    • Remove user-level allowlist management from CLI and admin interfaces; manage multicast group allowlists through AccessPass.
    • Add Validate trait for core types (AccessPass, Contributor, Interface, etc.) and enforce runtime checks before account operations.
    • Fix: resize AccessPass account before serialization to prevent errors; standardized use of resize_account_if_needed across processors.
    • Enable AccessPass with 'client_ip=0.0.0.0' to dynamically learn the user’s IP on first connection
    • Enable AccessPass to support connections from multiple IPs (allowlist compatibility)
    • Rename exchange.loc_id to exchange.bgp_community, and change it from u32 to u16
  • Internet telemetry
    • Add circuit label to metrics; create a new metric for missing circuit samples
    • Create a new metric that tracks how long it takes collector tasks to run
    • Submit partitions of samples in parallel
    • Include circuit label on submitter error metric
  • Monitor
    • Reduce logging noise in 2z oracle watcher
    • Include response body on 2z oracle errors
    • Collect contributors and exchanges into InfluxDB
  • Device controller
    • When a device is missing required loopback interfaces, provide detailed errors to agent instead of " not found". Also, log these conditions as warnings instead of errors, and don't emit "unknown pubkey requested" error metrics for these conditions
    • Add device info as labels to controller_grpc_getconfig_requests_total metric
  • Device agents
    • Submit device-link telemetry partitions in parallel
  • CLI
    • Allow AccessPass creation without 'client_ip'
    • Add 'allow_multiple_ip' argument to support AccessPass connections from multiple IPs
    • Rename exchange.loc_id to exchange.bgp_community
  • Onchain programs
    • Enable AccessPass with 'client_ip=0.0.0.0' to dynamically learn the user’s IP on first connection
    • Enable AccessPass to support connections from multiple IPs (allowlist compatibility)
    • Rename exchange.loc_id to exchange.bgp_community, and change it from u32 to u16
  • Telemetry data API
    • Filter by contributor and link type
  • SDK/Go
    • String serialization for exchanges status
    • Exclude empty tags from influx serialization

v0.6.6 – 2025-09-26

Breaking

  • None for this release

Changes

  • Monitor
    • Update 2Z oracle to emit response code metrics on errors too
  • Activator
    • A mitigation was added to handle situations where blockchain updates are missed by the Activator due to timeouts on the websocket. This mitigation processes pending accounts on a 1-minute timer interval.
  • CLI
    • Connect command updated to provide better user experience with regard to activator websocket timeouts (see above).

v0.6.5 – 2025-09-25

Breaking

  • None for this release

Changes

  • CLI
    • Connect now waits for doublezerod to get all latencies
    • Latency command sorts unreachable to bottom
  • Device controller
    • Update device template to set default BGP timers and admin distance
    • Update device template so all "default interface TunnelXXX" commands for user tunnels come before any other user tunnel config
  • Activator
    • Fix access pass check status accounts list
  • Onchain programs
    • Implemented strict validation to ensure that only AccessPass accounts owned by the program and of the correct type can be closed.
    • Fix Access Pass set Instruction.
    • Switched to using account_close helper for closing accounts instead of resizing and serializing.
    • Make interface name comparison case insensitive
  • Onchain monitor
    • Check for unlinked interfaces in a link
    • Emit user events
    • Add watcher for 2Z/SOL swap oracle

v0.6.4 – 2025-09-10

Breaking

  • None for this release

Changes

  • Onchain programs
    • Fix bug preventing re-opening of AccessPass after closure
    • sc/svc: guard against empty account data
  • Device controller
    • Support server dual listening on TLS and non-TLS ports
  • Device and Internet Latency Telemetry
    • Create one ripeatlas measurement per exchange instead of per exchange pair to avoid concurrent measurement limit

v0.6.3 – 2025-09-08

Breaking

  • None for this release

Changes

  • Onchain programs
    • Expand DoubleZeroError with granular variants (invalid IPs, ASN, MTU, VLAN, etc.) and derive PartialEq for easier testing.
    • Rename Config account type to GlobalConfig for clarity and consistency.
    • Fix bug in user update that caused DZ IP to be 0.0.0.0
    • Add more descriptive error logging
    • Telemetry program: embed serviceability program ID via build feature instead of env variable
  • Activator
    • Support for interface IP reclamation
    • Devices are now initialized with max_users = 0 by default.
    • Devices with max_users = 0 cannot accept user connections until updated.
  • Onchain monitor
    • Emit metric for telemetry account not found in device and internet telemetry watchers
    • Emit metric with serviceability program onchain version
    • Delete telemetry counter metrics if circuit was deleted
  • Telemetry
    • Fix dashboard API to handle partitioned query with no samples
    • Add summary view with committed RTT and jitter, compared to measured values
  • Device agents
    • Remove log of keypair path on telemetry agent start up
    • Drop device telemetry samples if submission attempts exhausted and buffer is at capacity
  • Device controller
    • Each environment can now have a different device BGP Autonomous System Number (ASN) per environment. (This is the remote ASN from the client's perspective.)
    • Add flag for enabling pprof for runtime profiling
  • E2E tests
    • Updated unit tests and e2e tests to validate the new initialization and activation flow.
  • Contributor Operations
    • Contributors must explicitly run device update to set a valid max_users and activate a Device.

v0.6.2 – 2025-09-02

Breaking

  • None for this release

Changes

  • Onchain programs
    • Fix: Serviceability now correctly enforces device.max_users
    • Fix: Restored the validator_pubkey field from AccessPass. This field had been removed in the previous version but is required by Sentinel.
    • Fix: Skip client version check in status command to prevent version errors during automated state checks.
    • New instructions were added to support device interface create/update/delete that prevents a race condition that could cause some updates to be lost when changes were made in quick succession.
    • Add deserialization vector with capacity + 1 to avoid memory allocation failed, out of memory error
  • CLI
    • Added filtering options to access-pass list and user list CLI commands.
    • New filters include access pass type (prepaid or solana-validator) and Solana identity public key.
    • Updated command arguments and logic, with tests adjusted to cover new options.
    • Contributors: Interface creation no longer takes an "interface type (physical/loopback)" argument. The type is now inferred from the interface name.
  • Device controller
    • Use serviceability onchain delay for link metrics

v0.6.0 – 2025-08-28

Breaking

  • Onchain programs
    • Implement access pass management commands and global state authority updates
    • Update access pass PDA function to include payer parameter

Changes

  • Onchain programs
    • Introducing new link instruction processor acceptance criteria
    • Add support for custom deserializers and add for pubkey fields
    • Move serialization and network_v4 to program-common
    • Refactor account type assertions in processors and state modules in serviceability program
    • Add validator identity to SolanaValidator type AccessPass.
  • User client
    • Add access pass management commands to CLI
    • Restructuring device and global config CLI commands for better authority and interface management
    • Enhance the handling and display of access pass epoch information in the CLI
    • Configure CLI network settings with shorthand network code. Usage: doublezero config set --env <testnet|mainnet-beta>
    • Configure doublezerod network settings with shorthand network code. Usage doublezerod --env <testnet|mainnet-beta>
    • Add associated AccessPass to user commands.
  • Activator
    • Introduce new user monitoring thread in activator for access pass functionality
    • Remove validator verification via gossip. This functionality is migrated to AccessPass.
  • Device controller
    • Implement user tunnel ACLs in device agent configuration
    • Add "mpls icmp ttl-exceeded tunneling" config statement so intermediate hops in the doublezero network respond to traceroutes.
    • Set protocol timers for ibgp and isis to improve to speed up network re-convergence
    • Add TLS support to gRPC server
  • Onchain monitor
    • Initial implementation and component release
    • Monitor onchain device telemetry metrics
    • Monitor onchain internet latency metrics
  • E2E tests
    • Simplify fixtures with loop rollups
    • Add user ban workflow test
    • Deflake user reconnect race and device interface assigned IP race
    • Add single device stress test
    • Adjust user validation commands to use the new AccessPass column.
  • CLI
    • Refactor: Updated SetAccessPassCliCommand (doublezero access-pass set) to use --epochs instead of --last_access_epoch, with sensible default values.
    • AccessPass now requires passing the validator identity for the SolanaValidator type.
  • Device Agents
    • Periodically recreate telemetry agent sender instances in case of interface reconfiguration.
  • Telemetry
    • Optimize onchain data dashboard API responses with field filtering
    • Optimize onchain data data CLI execution with parallel queries
    • Dashboard API support for circuit set partitioning using query parameters

v0.5.3 – 2025-08-19

  • CLI & UX Improvements
    • Improve sorting of device, exchange, link, location, and user displays
    • New installation package for the admin CLI for contributors based on controller/doublezero-admin
    • Do not allow users to connect to a device with zero available tunnel slots remaining
    • Improve handling of interface names for doublezero device interface commands
  • Serviceability Model Improvements
    • funder: configure recipients as flag
    • sdk/rs: add record program handling
    • config: use ledger RPC LB endpoint
    • Validate account codes and replace whitespace
    • config: add ability to override DZ ledger RPC url; update URLs
    • Remove old CloseAccount instruction from both the smart contract and SDK client code
  • Network Controller Improvements
    • Increase user tunnel slots per device from 64 to 128
    • Add flag controlling whether interfaces and peers are rendered to assist with testnet migration
  • Device and Internet Latency Telemetry
    • Internet latency samples in data CLI and dashboard API
    • internet-latency-collector, telemetry data api/cli: collect internet latency between exchanges, not locations
    • internet-latency-collector: add ripeatlas credit metric
  • End-to-End Tooling
    • New doublezero QA agent improves quality by thoroughly testing the software stack end-to-end in each doublezero environment (devnet, testnet, mainnet-beta) after each release.

v0.5.0 – 2025-08-11

  • CLI & UX Improvements
    • doublezero connect now waits for the user account to be visible onchain.
    • doublezero device interface commands. Interface names get normalized.
    • General improved consistency
    • Easy switching between devnet and testnet using the --env flag
  • Device Latency Telemetry
    • Data CLI and API use epoch from ledger
    • Backpressure support to avoid continual buffer growth in error conditions
    • Link pubkey used for circuit uniqueness
  • Internet Latency Telemetry
    • Adds the environement (devnet/testnet/mainnet-beta) to the ripeatlas measurement description.
    • Now funded by the funder
  • Serviceability Model Improvements
    • Device extended to add DZD metadata (including Interfaces)
    • DZX Link types added (clearly distinguished from WAN links)
    • Removed foundation allowlist check, streamlining link workflow
    • Validate that link.account_type has type AccountType::Link
  • Network Controller Improvements
    • doublezero-controller now manages more of the DZD configuration, including:
      • DNS servers
      • NTP servers
      • DZ WAN interfaces
      • Necessary loopback interfaces
      • BGP neighbor configuration
      • MSDP configuration
    • doublezero-activator now assigns IP addresses for use by the controller to give to DZD wan interfaces as well as loopbacks.

v0.4.0 – 2025-08-04

This release adds contributor ownership, reference counting, and improved CLI outputs for devices and links. It introduces internet latency telemetry, with support for collection, Prometheus metrics, and writing samples to the ledger. Device telemetry now uses ledger epochs for network-wide consistency.

  • Serviceability Model Improvements
    • Contributor creation includes an owner field; device/link registration enforces contributor consistency
    • Contributor field shown in CLI list and get commands for devices and links
    • reference_count added to contributors, devices, locations, and exchanges
    • New fields added to Device and Link, including an interfaces array for Device
    • Go SDK updated to support new DZD metadata account layouts
  • CLI & UX Improvements
    • Provisioning (connect, decommission) UX improved: clearer feedback, better spinners, and more accurate status messages
    • doublezero latency output includes device code alongside pubkey
    • doublezero device and doublezero link commands updated to show new metadata fields
    • Added doublezero device interface subcommands for managing interfaces
    • keygen command now supports --outfile (-o) flag to generate keys directly to a file
  • Device Latency Telemetry
    • Agent now uses ledger epoch instead of wallclock-based epoching
    • Account layout updated to move epoch after discriminator for efficient filtering
  • Internet Latency Telemetry
    • Internet latency collectors write samples to the ledger using epoch-based partitioning
    • Telemetry program supports ingesting external control-plane latency samples
    • Prometheus metrics expose collector operation, failure rates, and credit balances
    • Go SDK support for initializing and submitting latency samples
  • End-to-End Tooling
    • Multicast monitor utility added for provisioning validation
    • Multi-client e2e tests cover IBRL with and without IP allocation

v0.3.0 - 2025-07-28

This release introduces network contributor registration, device interface management, and the initial telemetry system for link latency. Prometheus metrics were added to the activator and client for observability. Provisioning flows now enforce contributor presence and IP uniqueness per user.

  • Contributor Support
    • Added CLI support for contributor management via doublezero contributor
    • Used to register network contributors in the DoubleZero system
  • Device Interface Management
    • Added device interface CRUD commands for managing interfaces on a device
    • Interface metadata will be used by the controller to generate device configuration
  • Link Telemetry System
    • Introduced TWAMP-based telemetry agent and onchain program for measuring RTT and packet loss between devices
    • Lays the foundation for performance-based rewards for bandwidth contributors
  • Prometheus Metrics
    • Activator and client now export Prometheus metrics (build info, BGP session status)
  • Provisioning & Decommissioning
    • Enforced one tunnel per user per IP address
    • Contributor field now required when creating devices and links
  • Activator
    • Improved metrics and error handling
    • Added graceful shutdown and signal handling
  • Client
    • Added -json output flag for status and latency commands