Skip to content

feat: box.agent heartbeat plugin with its own push token (v0.13.0) - #29

Merged
yakari007 merged 3 commits into
masterfrom
feat/box-agent
Oct 10, 2026
Merged

yakari007 merged 3 commits into
masterfrom
feat/box-agent

Conversation

@yakari007

Copy link
Copy Markdown
Contributor

Summary

  • New box.agent ingester: LLM boxes running vLLM POST a ~1 s JSON heartbeat to POST /ingest/v1/{check_id}; objects host, vllm, gpu:<idx>.
  • Own monitoring-only push token (mpush_…, shown once, rotatable via /v1/checks/{id}/rotate-token), separate from the token box.agent uses with llmocean.api. No NATS.
  • XID 79/48/64 or uncorrectable ECC is CRITICAL at once; booting/downloading/restarting/stopped never raise incidents; silence is CRITICAL after 5 missed beats.
  • Response codes tracked: server counter ingest_http_responses_total{code}, plus the box's own counts (monitor_responses) stored as metrics.
  • Ingest rate limit raised 1/s -> 2/s (burst 20) for all push checks.
  • Weight 2 per box is a placeholder pending PlusClouds billing confirmation.

Provisioning flow

llmocean.api creates the device + box.agent check (interval_seconds: 1) with the tenant API key, receives push_token once, and injects -monitor-url / -monitor-token into box.agent. On deprovision it disables/deletes the check. The tenant's min_check_interval_seconds must be 1.

Test plan

  • plugins/boxagent unit tests + 10 s fuzz run (Go 1.25 in an isolated module)
  • make check on Go 1.27: the changes outside plugins/boxagent (ingest, inventory, config, migration, TestBoxAgent) were NOT compiled or run locally (no Go 1.27 toolchain or test DB available)
  • Alert route to the llmocean.api webhook configured after merge

🤖 Generated with Claude Code

yakari007 and others added 3 commits October 10, 2026 16:40
Add the box.agent ingester: LLM boxes running vLLM POST a ~1 s JSON
heartbeat to the push endpoint with a monitoring-only token (not the
llmocean.api token). Objects: host, vllm, gpu:<idx>. Hard GPU faults
(XID 79/48/64, uncorrectable ECC) are CRITICAL at once; booting,
downloading, restarting and stopped are exempt; silence is CRITICAL
after 5 missed beats. The box's own count of server answers by class
is stored as metrics, and the push endpoint counts answers by status
code (ingest_http_responses_total). Ingest rate limit 1/s -> 2/s.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@yakari007
yakari007 merged commit 4fedf68 into master Oct 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant