v26.2.7

usulnet v26.2.7 — 2026-02-23

Fixed

• Issue #16 — Container file editing crash: WriteContainerFile passed entire base64-encoded content as a single shell argument via echo '…' | base64 -d > file, hitting shell argument length limits on files larger than ~100 KB. Replaced with Docker's CopyToContainer tar-based API (no shell limits). Falls back to chunked 48 KB shell writes for remote agent-proxied hosts. Also added a 10 MB request body limit in the handler.

• Host metrics cleanup error: DeleteOldMetrics query referenced non-existent column created_at — changed to recorded_at.

• Container stats cleanup error: DeleteOldStats query referenced non-existent column collected_at — changed to recorded_at.

• System backup "unsupported type": Scheduled automatic database backup job used BackupTypeSystem which had no handler. Added createSystemBackup that finds the database container and backs up its volumes.

• /quotas endpoint 500: Docker API calls for container stats had no timeout, causing the page to hang/crash when Docker was slow. Added 8-second overall timeout and 2-second per-container stats timeout.

• Host File Browser user switch: su - {user} -c failed silently for system users with /usr/sbin/nologin shell (www-data, nginx, nobody). Changed to su -s /bin/sh to force a usable shell.

• File Browser showing only symlinks (host + non-BusyBox containers): Both parseHostLS and parseContainerLS required len(fields) < 8 to accept an ls -la line, but with the ISO time format (--time-style=+%Y-%m-%dT%H:%M:%S) regular entries have only 7 fields — so every directory, file, and device node was silently dropped. Only symlinks (9+ fields due to -> target) passed the filter, causing the host and Debian/GNU-based container file browsers to show only a handful of symlink entries. Changed threshold to len(fields) < 7. BusyBox containers were unaffected because BusyBox ls doesn't support --time-style and falls back to the traditional 9-field format.

• File Browser symlinks to directories: Clicking a symlink pointing to a directory (e.g. /bin → /usr/bin) tried to open it as a file instead of navigating into it. Root cause: ls -la mode prefix is l not d for symlinks, so is_dir was always false. Fixed by adding symlink type resolution — host browser uses stat -L -c '%F %n' (GNU coreutils), container browser uses POSIX test -d (works in BusyBox/Alpine). Also replaced stat -L -c '%F' with test -d in ReadContainerFile for BusyBox compatibility, and directory errors now return 400 instead of 500.

• File browser freezes browser tab when opening files: Clicking any file in the host or container file browser froze the entire browser tab for 16-68+ seconds. Root cause: monaco.editor.create() (3.7 MB editor.main.js) blocks the main thread during initialization, and was called for every file open — even just to view. Monaco is also fundamentally incompatible with the file browser's modal context (flex layout + backdrop-blur + x-show transition) — automaticLayout: true causes a ResizeObserver layout-thrashing loop, and automaticLayout: false still freezes because Monaco's DOM creation (~3000+ nodes) triggers a reflow cascade against the flex+blur compositing layers. Fixed by removing Monaco Editor entirely from both file browsers. File viewing now uses a native <pre><code> block (instant rendering, zero JS). File editing in the container file browser uses a styled <textarea> with monospace font, dark theme, and Ctrl+S/Cmd+S save shortcut. The full Monaco editing experience remains available on the dedicated /editor page, which uses a static full-page layout without modal/flex conflicts.

• File editing broken in container file browser: Clicking "Edit" on a file opened a read-only editor due to a race condition: editFile() called viewFile() which reset editingFile=false, then initMonaco(false) created a read-only editor, and the .then() callback that set editingFile=true ran too late. Refactored viewFile() to accept an edit parameter and editFile() to call viewFile(file, true), ensuring the edit state is set correctly from the start.

• Alpine.js sidebar errors on every page load: The sidebar section template used try { ... } catch(e) {} in Alpine.js x-init expressions, but Alpine 3.x doesn't support try/catch in inline expressions (throws Uncaught SyntaxError). Also used the x-collapse directive without the Collapse plugin installed. Moved try/catch logic into a global _sidebarSectionOpen() helper function and replaced x-collapse with the built-in x-transition.

• DNS Service Discovery startup failure: Creating the containers.local zone for DNS service discovery failed with NOT NULL constraint violation on "forwarders". The Forwarders field was Go's nil slice, which PostgreSQL receives as NULL. Initialized to empty slice []string{}.

• Agent Node Metrics Showing 0s and Docker Version "dev" (critical — agent nodes appear empty): The gateway's handleHeartbeat() discarded the QuickStats payload (CPU%, memory, container counts) from every agent heartbeat, and handleInventory() never persisted SystemInfo (Docker version, CPUs, memory) to the hosts table. Agent nodes always showed 0 running containers, 0 images, 0 CPUs, 0 memory, and Docker version "dev" in the UI. Fixed by storing heartbeat metrics to the host_metrics table and inventory system info to the hosts table. ListSummaries() now enriches agent host data from the latest DB metrics instead of only from the local client pool.

• Terminal on Agent Node Opens Master's Shell Instead (critical — security issue): The host terminal WebSocket handler (WSHostExec) used nsenter via PID 1 to enter the host namespace, but never checked whether the requested host was local or remote. Navigating to /nodes/{agent-id} and opening a terminal would silently give a shell on the master node, not the agent. Added endpoint type validation in both the page handler and WebSocket handler to reject non-local hosts with a clear error message. Terminal and Files buttons are now hidden in the UI for remote/agent nodes.

• Log Streaming "Not Available for Remote Hosts": WSContainerLogs and WSContainerStats performed a type assertion to *docker.Client, which fails for AgentProxyClient (agent hosts connected via NATS). Any attempt to view logs or stats for a container on an agent node returned an error. Replaced with a dual-path approach: direct streaming for local hosts (existing behavior), and buffered polling fallback for agent hosts — logs poll via ContainerLogsSince every 5 seconds, stats poll via ContainerStatsOnce every 3 seconds.

• Stack Deployment on Agent Node Appears Successful but Never Installs (critical — silent failure): The stack service's execCompose() ran docker compose locally with a DOCKER_HOST env var pointing to the remote host's endpoint URL. Agent hosts have no TCP endpoint (they connect via NATS), so the compose command silently failed while the UI showed the stack as deployed. Added handleStackDeploy and handleStackRemove command handlers to the agent executor, added GatewayCommandSender interface to the stack service, and the Deploy() method now detects agent hosts and routes deployments through the NATS gateway. The agent writes compose files to /data/stacks/{name}/ and runs docker compose up -d locally. Gateway sender wired at application startup.

• Security Scanning Fails for Agent Node Containers: securityAdapter.Scan() called ContainerInspectRaw() which returns ErrNotSupportedRemote on AgentProxyClient. The agent executor already supported CmdSecurityScan (collecting inspect data and sending it to the master for analysis), but it was never triggered from the web UI path. Added isAgentHost() detection and scanViaGateway() routing to the security adapter. Agent scans now: agent collects container inspect data → sends via NATS → master runs all security analyzers + Trivy on the received data → persists results. Both single-container and scan-all operations work for agent nodes.

Changed

• guacd updated to 1.6.0: Apache Guacamole daemon updated from 1.5.5 to 1.6.0 in all Docker Compose files, documentation, and public website.

Added

• WireGuard VPN: Native WireGuard VPN management with auto-generated Curve25519 keys, multi-interface support, peer management with QR config generation, transfer statistics, and persistent keepalive. Features include:

  • Create and manage multiple WireGuard interfaces per host
  • Add peers with auto-generated key pairs and preshared keys
  • Client configuration generation for easy setup
  • Transfer statistics tracking (rx/tx) per interface and peer
  • Post-up/post-down script support for routing
  • Database: migration 053_wireguard_vpn
  • Backend: internal/services/wireguard/, internal/repository/postgres/wireguard.go

• Container Image Builder: Build Docker images from Dockerfiles in the UI with multi-stage build support, build arguments, platform targeting, and reusable Dockerfile templates.

• Automated Rollback: Automatic stack rollback on deploy failure or health check failure, with configurable rollback policies, retry limits, cooldown periods, and full execution history.

• Firewall Manager: Visual iptables/nftables management directly from the web UI — no CLI commands needed. Create, edit, delete, apply, and sync firewall rules across hosts. Features include:

  • Chain Support: INPUT, OUTPUT, FORWARD, and DOCKER-USER chains.
  • Protocol & Action Coverage: TCP, UDP, ICMP, and all protocols with ACCEPT, DROP, REJECT, and LOG actions.
  • Full Audit Log: Every rule change is recorded with user, action, timestamp, and rule details for compliance and troubleshooting.
  • Agent-Based Backend Detection: Automatically detects whether the host uses iptables or nftables and applies rules through the appropriate backend.
  • Rule Application & Sync: Apply individual rules or sync the entire ruleset to the host firewall in one click.
  • Database: Migration 048_firewall.
  • Backend: internal/models/firewall.go, internal/repository/postgres/firewall.go, internal/services/firewall/service.go, internal/agent/executor/firewall.go, internal/web/handler_firewall.go, 5+ templ templates under templates/pages/firewall/.

• SSL Observatory (Business): TLS/SSL certificate scanner with SSL Labs-style grading for monitoring certificate health across infrastructure. Features include:

  • Certificate Scanning: Scans targets using Go crypto/tls, analyzing protocol versions (TLS 1.0–1.3), cipher suites, certificate chains, OCSP stapling, HSTS headers, and Certificate Transparency logs.
  • Grade Scale: A+ to F letter grades with 0–100 numeric scoring, matching SSL Labs methodology.
  • Dashboard: Grade distribution chart and expiring certificate alerts for at-a-glance infrastructure TLS health.
  • Detailed Reports: Per-target scan results with protocol breakdown, cipher suite analysis, chain validation, and actionable remediation guidance.
  • Database: Migration 049_ssl_observatory.
  • Backend: internal/models/ssl_observatory.go, internal/repository/postgres/ssl_observatory.go, internal/services/sslobservatory/service.go, internal/web/handler_ssl_observatory.go, templ templates under templates/pages/sslobservatory/.

• Backup Verification: Automated backup integrity verification ensuring backups are actually restorable — not just present. Features include:

  • Three Verification Methods: Extract (unpack and validate archive contents), Container (mount backup in a temporary container and verify accessibility), and Database (restore to a temporary instance and run integrity queries).
  • Integrity Checks: Validates checksums, file readability, container accessibility, and data integrity across all verification methods.
  • Schedulable Runs: Verification jobs configurable via cron expressions for automated, recurring backup validation.
  • Verification History: Full run history with status, method used, duration, and detailed error reports for failed verifications.
  • Database: Migration 050_backup_verification.
  • Backend: internal/models/backup_verification.go, internal/repository/postgres/backup_verification.go, internal/services/backupverify/service.go, internal/web/handler_backup_verify.go, templ templates under templates/pages/backupverify/.

• Nginx as Default Reverse Proxy (Caddy and NPM Removed): The reverse proxy system has been simplified to nginx-only. The Caddy backend (caddy/client.go, caddy/builder.go, caddy/types.go, caddy_backend.go) and NPM integration (integrations/npm/, adapter_proxy_npm.go, models/npm.go, npm_repo.go) have been completely removed — ~6,000 lines of dead code eliminated. The Caddy adapter has been renamed to a generic proxyServiceAdapter. Nginx is now always enabled when the encryption key is available. Config simplified: removed caddy.* and npm.* configuration sections.

• DNS-01 Challenge Support for Wildcard Certificates: New RequestCertificateDNS01 method in the ACME client enables Let's Encrypt wildcard certificates (*.example.com) via DNS-01 challenges. New dns_cloudflare.go implements Cloudflare DNS TXT record CRUD via the REST API for automated _acme-challenge record management. RequestCertificate in the nginx backend automatically selects DNS-01 when any domain is a wildcard or a DNS provider is configured, falling back to HTTP-01 otherwise.

• Docker Exec Mode for Nginx Backend: When nginx runs in a separate Docker container (standard deployment), the nginx client now uses the Docker API to execute nginx -t and nginx -s reload inside the container instead of local shell commands. New DockerExecer interface, DockerExecOpts/DockerExecResult types, and ContainerName config option. The SetDockerExecer() method wires the Docker client into the nginx backend at startup.

• Host Service Gateway Command Sender: host.Service now exposes SendCommand() for routing arbitrary protocol commands to remote agent hosts via the NATS gateway. Used by the security adapter and stack service for agent operations.

• Embedded DNS Server: Full authoritative DNS server built into usulnet, powered by miekg/dns (the Go DNS library behind CoreDNS). Runs in-process as part of the usulnet binary — no external DNS software required. Features include:

  • Zone Management: Create and manage DNS zones (primary, secondary, forward) with full SOA configuration (primary NS, admin email, refresh, retry, expire, minimum TTL). Zone serial auto-increments on every record change.
  • 10 Record Types: A, AAAA, CNAME, MX, TXT, NS, SRV, PTR, CAA, SOA. Per-record TTL, priority/weight/port fields for SRV/MX, and enable/disable toggle.
  • TSIG Keys: Transaction Signature keys for secure zone transfers. Secrets encrypted at rest with AES-256-GCM via the existing encryption infrastructure.
  • Upstream Forwarding: Non-authoritative queries forwarded to configurable upstream resolvers (default: Cloudflare 1.1.1.3 + 1.0.0.3 malware-blocking DNS). UDP and TCP protocol detection for forwarding.
  • PostgreSQL-Backed: All zone, record, and TSIG key data stored in PostgreSQL (source of truth) via 2 new migrations (046_dns.up.sql). Full dataset synced to the in-memory server on every change via SyncBackend interface.
  • Live Statistics: Real-time query counters (total, success, failed), zones loaded, and server uptime. Lock-free atomics for stats. Health check via self-query.
  • Audit Logging: All zone, record, and TSIG key changes logged with user ID, action, resource type, resource name, and timestamp. Browsable from the DNS audit log page.
  • Web UI: 8 new templ templates — zone list with stats cards, zone create/edit forms, zone detail with inline record management, record edit form, DNS server settings/status page, and audit log table. Navigation tabs for Zones, Settings, and Audit Log.
  • Configuration: dns.enabled (default: true), dns.listen_addr (default: :53), dns.forwarders (default: Cloudflare 1.1.1.3/1.0.0.3). Environment variables: USULNET_DNS_ENABLED, USULNET_DNS_LISTEN_ADDR.
  • Backend: internal/services/dns/ (service layer with interfaces, backend abstraction, Sync pattern), internal/services/dns/embedded/ (miekg/dns implementation), internal/repository/postgres/dns_repo.go (4 repositories), internal/models/dns.go (zone/record/TSIG/audit models), internal/web/handler_dns.go (14 handler methods), 8 templ templates, 14 routes under /dns.

• DNS Service Discovery: Automatic container-to-DNS registration — running Docker containers are automatically registered as DNS records in the embedded DNS server, eliminating manual record management. Features include:

  • A Records: Every container is registered as <container-name>.containers.local (A record), resolved by the embedded DNS server in real time.
  • SRV Records: Containers with exposed ports automatically get SRV records (_<port>._tcp.<container-name>.containers.local), enabling clients to discover services by name and port.
  • Real-Time via Docker Event Stream: Registration and deregistration happen instantly through Docker event stream callbacks — container start triggers DNS registration, container stop/die triggers removal. No polling delay.
  • Reconciliation: Periodic full-state reconciliation catches any events missed during transient Docker API disconnects, ensuring DNS always reflects the actual running container set.
  • Configurable: Domain suffix (default: containers.local), per-record TTL, and SRV record creation are all configurable. SRV generation can be disabled independently of A record registration.
  • Environment Variables: USULNET_DNS_SD_ENABLED (default: true when DNS server is enabled), USULNET_DNS_SD_DOMAIN (default: containers.local).

• Crontab Manager: Web-based cron job scheduling and management — create, edit, enable/disable, and execute cron jobs directly from the usulnet UI. Features include:

  • Three Command Types: Shell commands (with configurable working directory), Docker exec (target container), and HTTP webhooks (GET/POST/PUT/DELETE with configurable URL).
  • Cron Scheduling: Standard 5-field cron expressions powered by robfig/cron/v3. Jobs are registered in an in-process scheduler and fire automatically.
  • Execution History: Every run is recorded with status (success/failed/running), stdout/stderr output, exit code, duration in milliseconds, and start/finish timestamps. Browsable from the job detail page with expandable output.
  • Run Now: Execute any cron job immediately from the UI, independent of its schedule.
  • Auto-Cleanup: Execution records older than 30 days are automatically pruned by a daily background worker.
  • Database: Migration 047 creates crontab_entries and crontab_executions tables with indexes on host_id, enabled, entry_id, and started_at.
  • Backend: internal/services/crontab/ (service layer with robfig/cron scheduler), internal/repository/postgres/crontab.go (2 repositories), internal/models/crontab.go (entry/execution/stats models), internal/web/handler_crontab.go (10 handler methods), 5 templ templates, 10 routes under /crontab.

• Interactive Network Topology Graph: The topology page (/topology) has been upgraded from a static card-based layout to an interactive D3.js force-directed graph visualization. Features include:

  • Force-Directed Layout: Networks and containers are rendered as interactive nodes with physics-based positioning via D3.js v7 force simulation. Networks shown as rounded rectangles, containers as circles.
  • Drag & Drop: Nodes can be dragged to rearrange the layout. Pinned nodes stay in place until released.
  • Zoom & Pan: Full zoom/pan support with mouse wheel and drag. Reset button restores the default view.
  • Hover Highlighting: Hovering a node highlights its connections and dims unrelated nodes for clear topology inspection.
  • Click Details Panel: Clicking a node shows a details sidebar with type, driver, subnet, state, and connection count.
  • Color-Coded: Networks colored by driver (bridge=blue, overlay=green, host=yellow, macvlan=purple). Containers colored by state (running=green, stopped=red).
  • Fullscreen Mode: Expand the graph to fullscreen for large topologies.
  • Toggle Labels: Show/hide node labels for cleaner visualization.
  • JSON API: New /topology/api endpoint returns graph data (nodes + links) as JSON for the D3.js client.
  • Collapsible Detail View: The original network card list is preserved below the graph in a collapsible <details> section.
  • D3.js v7 added as self-hosted vendor library (web/static/vendor/js/d3-7.9.0.min.js).

Changed

• Sidebar search: Added compact search input below the logo that filters navigation items in real-time. Escape key clears the filter. Collapsible sections auto-expand when searching.

• Image Builder: to Images page as a "Build" action button alongside Pull Image and Prune.

• DNS: Added DNS Server entry to the Integrations section in the sidebar.

• Proxy Labels Renamed: Docker labels LabelCaddyDomain, LabelCaddyPort, etc. renamed to LabelProxyDomain, LabelProxyPort for backend-agnostic naming.

• DNS Provider Map Updated: SupportedDNSProviders values changed from Caddy Go module paths to descriptive strings (e.g., "Cloudflare DNS API", "AWS Route 53").

• Proxy Setup Page Simplified: The proxy setup page no longer shows an NPM connection form. It now displays nginx backend status (healthy/unhealthy, config directory, certificate directory, ACME email).

• Documentation Updated: Installation guide, architecture documentation, README, and public website updated to reflect nginx-only proxy, embedded DNS server, master/agent terminology, and current feature set. README now lists 54 migrations, 54 services, 45 handlers. Public website (usulnet.com) and docs (docs.usulnet.com) updated with DNS Server feature card, comparison table row, configuration section, sidebar links, and feature documentation.