Updates for runZero 5.1

[hdm]

Updates for runZero 5.1: a bigger Starlark toolkit and self-describing integration scripts

Important

This branch requires runZero 5.0 or newer. The scripts here load builtins
(get_json, network_interface, to_custom_attributes, kwargs, the
runzero.* protocol modules, ...) and declare embedded CONFIG blocks that
only the 5.0 Explorer and console understand. Running them on an earlier
release will fail. Scripts that set minVersion enforce this automatically.

TL;DR

Every integration in this repo has been refactored onto the new 5.0 custom-integration
API surface. The result is dramatically less boilerplate, scripts that describe their
own credential form, new non-HTTP data sources, and explicit control over how imported
assets merge.

• Far less boilerplate: auth headers, retries, pagination, IP/MAC handling, and
  custom-attribute flattening are now builtins instead of copy-pasted helpers.
• Self-describing scripts: a CONFIG block at the top of each script renders the
  credential form, validates input, and encrypts secrets — no platform change needed
  to add a new integration.
• New data sources: pull from SSH/SMB/WinRM/WMI and SQL databases, not just REST APIs.
• Match control: a matchBehavior flag lets each script tell the cruncher how
  aggressively to merge the assets it imports.

---

Before vs. after

Authentication — before, scripts base64-encoded basic auth and hand-built bearer headers:

# before
import_b64 = base64.encode(user + ":" + password)
headers = {"Authorization": "Basic " + import_b64}

# now
load("http", "basic", "bearer", "oauth2_token")
headers = {"Authorization": basic(user, password)}        # or bearer(token)
token = oauth2_token(token_url=..., client_id=..., client_secret=...)  # client_credentials in one call

Fetching JSON with retries/pagination — before, every script re-implemented status
checks, JSON decoding, and backoff:

# before
resp = http_get(url, headers=headers)
if resp.status_code != 200:
    return []
data = json_decode(resp.body)

# now: retries 408/425/429/5xx with backoff, honors Retry-After, returns (data, err)
load("http", "get_json")
data, err = get_json(url, headers=headers, params={"limit": 100})
if err:
    print("fetch failed:", err)
    return []

Reading credentials — before, raw kwargs.get(...) with manual casting; now typed,
validating accessors:

# now
load("kwargs", "require", "get_int", "get_bool", "get_list")
require(kwargs, "client_id", "client_secret")
page_size = get_int(kwargs, "page_size", default=100)
regions   = get_list(kwargs, "regions", default=[])   # CSV string or list -> list

Building network interfaces — before, scripts looped to classify v4/v6 and normalize
MACs; now one helper does it (mixed list, addr:port/%zone stripping, dedupe, caps):

# now
load("net", "network_interface")
nic = network_interface(mac="AA:BB:CC:DD:EE:FF", ips=["10.0.0.5", "fe80::1%eth0"])

---

New capabilities

Self-describing scripts (embedded CONFIG). Each script now declares a top-level
CONFIG = {...} literal with its id/name/type, version, optional minVersion,
and credential params. The platform renders the form from it, applies defaults, coerces
and validates types (required/min/max/pattern/enum), and routes type: "secret"
fields through encrypted storage. Shared OPTIONS_HTTP/OPTIONS_TLS includes give every
integration consistent connection/TLS controls without copy-paste. This replaces the old
config.json metadata files, which have been removed.

Reach beyond HTTP. New modules open raw connections to sources without a REST API:

• socket (tcp/udp/tls), runzero.ssh, runzero.smb, runzero.winrm, runzero.wmi
• runzero.sql for Postgres/MySQL/SQL Server (parameterized query/exec)

Connections opened by a script are tracked and closed automatically when the run ends.

A real standard library. re (RE2 regex), xml, csv, jsonstream (streaming large
arrays/NDJSON), jwt (encode/decode/verify), runzero.progress (progress bar + log lines
in the UI), plus crypto (hmac_*, AWS SigV4 sign_v4, random_bytes/random_hex) and
hex/base32 encodings. URL helpers (url_parse/url_join/multipart) and
http.head/http.put round out the HTTP verbs.

Richer assets and merge control.

• runzero.types gains Service/ServiceProtocolData and a to_custom_attributes helper
  that flattens arbitrary values into the string -> string shape (nested dicts, lists,
  length caps) the platform expects.
• matchBehavior on ImportAsset accepts a flag string so each script chooses how the
  cruncher merges — e.g. "no-mac-break no-ip-break no-name-break" when it has a stable
  foreign id, or "no-id-match no-id-break" for ephemeral ids.

---

What changed in this repo

• Migrated every script to the new toolkit: to_custom_attributes(...),
  network_interface(...), get_json/post_json, basic/bearer/oauth2_token, and the
  typed kwargs accessors, removing hand-rolled flattening, header building, and pagination.
• Added an embedded CONFIG block to every script and removed the old config.json
  metadata files. Credential field names were normalized away from the generic
  access_key/access_secret pair to descriptive keys (api_token, client_id,
  username/password, etc.) that match each API.
• Renamed every script to drop the custom-integration- / custom_integration-
  filename prefix. Each directory now contains a <name>.star (e.g. tailscale.star,
  cisco-ise.star, boilerplate.star).
• New integration: Kubernetes — pulls Nodes (and optional LoadBalancer/NodePort
  Services) from the Kubernetes API using a ServiceAccount bearer token.
• Documentation: new helpers reference (docs/starlark-helpers.md), updated AGENTS.md
  and PR template for the CONFIG convention and <name>.star naming, a new
  "Asset IDs and match behavior" README section, and refreshed cross-references in
  docs/integrations.json and individual READMEs.

---

Match behavior reference

ImportAsset accepts an optional matchBehavior string. The default matches and breaks on
all four dimensions (id, MAC, IP, name), which is correct when the integration owns a strong
id. When the id is weak or absent, use the knobs below to tell the cruncher which dimensions
are unreliable for matching (finding the right existing asset to merge into) and which
are unreliable for breaking (refusing a merge that one dimension would otherwise block).

Flag	Effect
no-id-match	Do not use the foreign id to find candidate assets to merge with.
no-id-break	Allow a merge even when the foreign id differs from the existing asset.
no-mac-match	Do not use MAC addresses to find merge candidates.
no-mac-break	Allow a merge even when MAC addresses conflict.
no-ip-match	Do not use IP addresses to find merge candidates.
no-ip-break	Allow a merge even when IP addresses conflict.
no-name-match	Do not use hostnames to find merge candidates.
no-name-break	Allow a merge even when hostnames conflict.

Combine flags with spaces. Recommended presets:

• Strong, stable foreign id (most cloud / EDR / MDM APIs): leave matchBehavior unset.
  The default uses every signal.
• Strong id, but the source also reports churny MAC/IP/hostnames (ephemeral cloud
  workloads, VPN clients): matchBehavior="no-mac-break no-ip-break no-name-break". Keeps
  id-based merging authoritative, but stops drift in the other dimensions from blocking a
  legitimate merge.
• No stable id at all (the source only emits per-run / ephemeral ids):
  matchBehavior="no-id-match no-id-break". Falls back to MAC / IP / name matching. Pair it
  with id=new_uuid() or a hash of stable attributes so the row still has a unique key but
  the cruncher ignores it for correlation.
• Two-stage enrichment where one integration owns "identity" and another only contributes
  attributes: use no-id-match no-id-break on the enrichment-only integration so it always
  merges into the primary asset by MAC/IP/name rather than creating a parallel record.

Rule of thumb: if the upstream id is not both stable and unique, relax id matching. If
MAC / IP / hostname are known to be unreliable for this source, relax the corresponding
-break flags so a conflict there doesn't fragment one real asset into many.

---

Safety notes for authors

• Secret params are never printed/logged (script output is redacted) and are stored encrypted
  at rest.
• SQL DSNs are restricted to network-only access; jwt rejects the none algorithm;
  xml.parse is XXE-safe.
• Long-running builtins (HTTP/socket/SSH/SQL) honor the task's wall-clock deadline and are
  cleaned up on cancel.

Docs

Authoring guidance and a runnable example that exercises the new helpers live in this repo:
the helpers reference (docs/starlark-helpers.md), AGENTS.md, and
boilerplate/boilerplate.star.

[hdm]

placeholder of 5.1 until we pick the revision for this (to enable version compat checks)