diff --git a/CHANGELOG.md b/CHANGELOG.md index 416153dc..79031d70 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **Inbound webhook receivers** (#155): external systems (CI/CD, uptime monitors, arbitrary tools) can now push events into LogTide through per-receiver tokenized endpoints (`POST /api/v1/receivers/:id/:token`, `lr_`-prefixed tokens stored as SHA-256 hashes, timing-safe compare; the URL is the credential since GitHub/Uptime Robot cannot send custom headers). Three adapters normalize payloads into log entries: GitHub (workflow_run and deployment_status events with conclusion-to-level mapping; ping and unsupported events marked skipped), Uptime (shape-detects Uptime Robot alertType and Better Stack incident payloads), and Generic JSON (optional dot-path field mapping with levelMap and defaults, validated by a shared Zod schema; full payload always preserved in metadata). The endpoint answers 202 and processing is asynchronous via a new `receiver-events` queue job (queue abstraction, both backends), which runs the adapter, validates against `logSchema` and ingests through `ingestionService.ingestLogs`, so PII masking (fail-closed), usage quotas, Sigma detection, pipelines, metering and live tail all apply automatically; the outcome (processed/skipped/failed, normalized output, error) is recorded on a `receiver_events` row pruned to the last 100 per receiver. Management: project-scoped CRUD + recent-events API under `/api/v1/projects/:projectId/receivers` (session auth, audit-logged as `receiver.created`/`receiver.deleted`), a new `receivers.max` capability limit enforced with the canonical withLimitLock pattern (4-case + race tests), and a "Webhook Receivers" section in project settings (create dialog with adapter picker and generic field-mapping inputs, one-time URL reveal with copy, enable/disable switch, recent-events viewer with raw/normalized JSON). Migration 052 (`receivers`, `receiver_events`); public docs in `docs/receivers.md` - **Soft-delete for projects with a 30-day grace window**: deleting a project now moves it to a recoverable "trash" state (`deleted_at`) instead of removing it immediately. Its logs, traces and metrics stay queryable and the project stays viewable read-only during the window (the project detail layout loads with `includeDeleted`, shows a "deleted (read-only)" banner with the permanent-deletion date, and hides the Settings tab); a "Restore" action brings it back. A daily purge worker (3 AM) hard-deletes projects past the grace window, calling `reservoir.purgeProject()` to clear logs/spans/metrics from every storage engine before deleting the row. Name and slug uniqueness moved to partial unique indexes (active projects only) so a deleted project's name/slug can be reused; restoring into a name or slug a new active project has since taken is rejected with a 409 instead of a raw constraint error. Soft-deleted projects are excluded from listings, data-availability widgets and API-key verification (ingestion is refused immediately, with the key-verification cache invalidated on delete). New `POST /api/v1/projects/:id/restore` and `GET /api/v1/projects?includeDeleted=true`. Migrations 050 (soft-delete column + partial indexes) and 051 (drop `ON DELETE CASCADE` from logs/spans/metrics so an out-of-band project delete cannot bulk-wipe data; the purge worker clears reservoir data explicitly first) ## [1.0.3] - 2026-06-26 diff --git a/docs/receivers.md b/docs/receivers.md new file mode 100644 index 00000000..dacf0efe --- /dev/null +++ b/docs/receivers.md @@ -0,0 +1,122 @@ +# Webhook Receivers + +Webhook receivers let external systems (CI/CD pipelines, uptime monitors, arbitrary tools) push events into LogTide. Each receiver exposes a unique tokenized URL; incoming payloads are normalized into standard log entries by an adapter and stored through the regular ingestion pipeline, so PII masking, usage quotas, Sigma detection and parsing rules all apply automatically. + +Receivers are project-scoped and managed from **Project Settings > Webhook Receivers**, where you can create receivers, copy their URL, enable or disable them, and inspect the most recent received events with their normalized output. + +## Endpoint contract + +``` +POST /api/v1/receivers/{receiverId}/{token} +Content-Type: application/json +``` + +The URL itself is the credential: `receiverId` identifies the receiver and `token` authenticates the request. No headers are required, which makes the endpoint compatible with systems that cannot send custom headers (GitHub webhooks, Uptime Robot). + +The body must be a single JSON object of at most 256 KB. + +### Responses + +| Status | Meaning | +| --- | --- | +| `202 Accepted` | Event stored and queued for processing. Body: `{ "eventId": "..." }` | +| `400 Bad Request` | Body is not a JSON object (arrays, scalars and null are rejected) | +| `401 Unauthorized` | Token does not match the receiver | +| `403 Forbidden` | Receiver is disabled | +| `404 Not Found` | Unknown receiver id | +| `413 Payload Too Large` | Serialized payload exceeds 256 KB | + +Processing is asynchronous: a `202` means the payload was accepted, not that it produced a log entry. The outcome (`processed`, `skipped` or `failed`, with the normalized output and any error) is visible in the receiver's recent-events view, which keeps the last 100 events per receiver. + +## Tokens + +Receiver tokens use the `lr_` prefix (distinct from `lp_` project API keys) and are shown exactly once, at creation time, embedded in the full webhook URL. Only a SHA-256 hash is stored. Treat the URL as a secret: anyone who has it can write logs into your project. To rotate a token, delete the receiver and create a new one, then update the external system with the new URL. + +A receiver token cannot read data or call any other API endpoint; a leaked URL limits the blast radius to log ingestion into one project. + +## Adapters + +The adapter chosen at creation time decides how raw payloads become log entries. + +### GitHub + +Point a GitHub repository webhook (JSON content type) at the receiver URL. Supported events: + +- **workflow_run** (only `action: completed`): `success` maps to `info`, `cancelled`/`skipped`/`neutral` to `warn`, `failure`/`timed_out`/`startup_failure`/`action_required` to `error`. The message looks like `Workflow CI completed: failure`. +- **deployment_status**: `success` maps to `info`, `failure`/`error` to `error`, anything else to `info`. The message looks like `Deployment to production: failure`. + +The `service` field is the repository `full_name` (e.g. `acme/app`). Metadata includes the event type, workflow name, conclusion, run id and URL, branch and actor. Ping events and unsupported event types are marked `skipped`, not failed. + +Example: a failed CI run + +```json +{ + "action": "completed", + "workflow_run": { "name": "CI", "conclusion": "failure", "html_url": "...", "head_branch": "main" }, + "repository": { "full_name": "acme/app" }, + "sender": { "login": "octocat" } +} +``` + +becomes an `error` log for service `acme/app` with message `Workflow CI completed: failure`. + +### Uptime + +Detects two payload shapes automatically: + +- **Uptime Robot** (`alertType` field): `1` (down) maps to `error`, `2` (up) to `info`, `3` (SSL expiry) to `warn`. The monitor friendly name becomes the service. +- **Better Stack** (`data.attributes` incident shape): status `Started` maps to `error`, `Resolved` and `Acknowledged` to `info`. The monitor name becomes the service and the incident cause is part of the message. + +Unrecognized shapes are marked `skipped`. + +### Generic JSON + +Accepts any JSON object. Without configuration it produces one `info` log with message `Received event`, the receiver name as service, and the full payload preserved under `metadata.payload`. + +An optional **field mapping** extracts log fields from the payload using dot-paths: + +```json +{ + "message": "error.message", + "level": "severity", + "service": "source.app", + "timestamp": "ts", + "levelMap": { "crit": "critical", "sev1": "error" }, + "defaults": { "level": "warn", "service": "external-system" } +} +``` + +| Key | Meaning | +| --- | --- | +| `message` | Dot-path to the log message. Missing or non-string values fall back to `Received event`. | +| `level` | Dot-path to the level value. Values are lowercased and matched against LogTide levels (`debug`, `info`, `warn`, `error`, `critical`); the synonyms `warning`, `fatal`, `err` and `crit` are also understood. | +| `service` | Dot-path to the service name (truncated to 100 chars). Falls back to `defaults.service`, then the receiver name. | +| `timestamp` | Dot-path to an ISO string or epoch value. Unparseable values fall back to the arrival time. | +| `levelMap` | Case-insensitive map applied to the extracted level value before the builtin matching (e.g. map `sev1` to `error`). | +| `defaults` | Fallback `level` and `service` used when the mapped values are missing or invalid. | + +The create dialog exposes the four path fields; `levelMap` and `defaults` can be set through the API (`PATCH /api/v1/projects/{projectId}/receivers/{id}` with a `fieldMapping` object). + +## Pipeline guarantees + +Received events are ingested through the same path as `POST /api/v1/ingest`: + +- **PII masking is fail-closed**: entries whose masking fails are rejected, and the event is marked `failed` with the rejection reason. +- **Usage quotas** (`ingestion.max_bytes_monthly`, `ingestion.max_events_monthly`, `storage.max_bytes`) apply; over-quota events fail with the quota error recorded. +- Sigma detection, exception parsing, log pipelines, metering and live tail all see receiver events like any other ingested log. + +The number of receivers per organization can be capped with the `receivers.max` capability (unlimited by default in OSS). + +## Management API + +All management endpoints require session authentication and project access. + +| Method and path | Purpose | +| --- | --- | +| `GET /api/v1/projects/{projectId}/receivers` | List receivers (never returns tokens) | +| `POST /api/v1/projects/{projectId}/receivers` | Create; returns `token` and `ingestPath` once | +| `PATCH /api/v1/projects/{projectId}/receivers/{id}` | Rename, enable/disable, update `fieldMapping` | +| `DELETE /api/v1/projects/{projectId}/receivers/{id}` | Delete (invalidates the URL immediately) | +| `GET /api/v1/projects/{projectId}/receivers/{id}/events?limit=50` | Recent events with raw payload, normalized output and status | + +Receiver creation and deletion are recorded in the audit log (`receiver.created`, `receiver.deleted`). diff --git a/packages/backend/migrations/052_receivers.sql b/packages/backend/migrations/052_receivers.sql new file mode 100644 index 00000000..812568fc --- /dev/null +++ b/packages/backend/migrations/052_receivers.sql @@ -0,0 +1,30 @@ +-- Inbound webhook receivers (#155): external systems POST events that get +-- normalized into log entries by per-receiver adapters. +CREATE TABLE IF NOT EXISTS receivers ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + project_id UUID NOT NULL REFERENCES projects(id) ON DELETE CASCADE, + name TEXT NOT NULL, + adapter_type TEXT NOT NULL CHECK (adapter_type IN ('github', 'uptime', 'generic')), + token_hash TEXT NOT NULL UNIQUE, + field_mapping JSONB, + enabled BOOLEAN NOT NULL DEFAULT true, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + last_received_at TIMESTAMPTZ +); + +CREATE INDEX IF NOT EXISTS idx_receivers_project ON receivers(project_id); + +-- Recent raw/normalized events per receiver, capped at 100 rows per receiver +-- by the worker (pruneEvents). Powers the "recent events" UI. +CREATE TABLE IF NOT EXISTS receiver_events ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + receiver_id UUID NOT NULL REFERENCES receivers(id) ON DELETE CASCADE, + status TEXT NOT NULL CHECK (status IN ('pending', 'processed', 'skipped', 'failed')), + raw_payload JSONB NOT NULL, + normalized JSONB, + error TEXT, + received_at TIMESTAMPTZ NOT NULL DEFAULT now() +); + +CREATE INDEX IF NOT EXISTS idx_receiver_events_receiver + ON receiver_events(receiver_id, received_at DESC); diff --git a/packages/backend/src/capabilities/registry.ts b/packages/backend/src/capabilities/registry.ts index f2445b65..1d5ca1ca 100644 --- a/packages/backend/src/capabilities/registry.ts +++ b/packages/backend/src/capabilities/registry.ts @@ -94,6 +94,11 @@ export const CAPABILITIES = { defaultLimit: null, description: 'Maximum custom dashboards per organization', }, + 'receivers.max': { + kind: 'limit', + defaultLimit: null, + description: 'Maximum inbound webhook receivers per organization', + }, // Consumption quotas (OSS-permissive: null = unlimited). signal maps to #212 metering types. 'ingestion.max_bytes_monthly': { diff --git a/packages/backend/src/database/tenant-tables.ts b/packages/backend/src/database/tenant-tables.ts index ff155d88..d91d303f 100644 --- a/packages/backend/src/database/tenant-tables.ts +++ b/packages/backend/src/database/tenant-tables.ts @@ -14,6 +14,7 @@ export const TENANT_TABLES = new Set([ 'pii_masking_rules', 'organization_pii_salts', 'audit_log', 'metrics_hourly_stats', 'metrics_daily_stats', 'metrics', 'metric_exemplars', 'custom_dashboards', 'log_pipelines', 'digest_configs', 'digest_recipients', 'projects', + 'receivers', ]); /** @@ -23,7 +24,7 @@ export const CHILD_TABLES = new Set([ 'monitor_status', 'status_incident_updates', 'incident_alerts', 'incident_comments', 'incident_history', 'stack_frames', 'alert_rule_channels', 'sigma_rule_channels', 'monitor_channels', - 'incident_channels', 'error_group_channels', + 'incident_channels', 'error_group_channels', 'receiver_events', ]); /** Intentionally global tables (no tenant scope). */ diff --git a/packages/backend/src/database/types.ts b/packages/backend/src/database/types.ts index 43d4a90e..72c5e03a 100644 --- a/packages/backend/src/database/types.ts +++ b/packages/backend/src/database/types.ts @@ -924,6 +924,40 @@ export interface WebhookDeliveryAttemptsTable { created_at: Generated; } +// ============================================================================ +// INBOUND WEBHOOK RECEIVERS (#155) +// ============================================================================ + +export interface ReceiversTable { + id: Generated; + project_id: string; + name: string; + adapter_type: string; // 'github' | 'uptime' | 'generic' + token_hash: string; + field_mapping: ColumnType< + Record | null, + Record | null, + Record | null + >; + enabled: Generated; + created_at: Generated; + last_received_at: ColumnType; +} + +export interface ReceiverEventsTable { + id: Generated; + receiver_id: string; + status: string; // 'pending' | 'processed' | 'skipped' | 'failed' + raw_payload: ColumnType< + Record, + Record, + Record + >; + normalized: ColumnType; + error: string | null; + received_at: Generated; +} + // ============================================================================ // PII MASKING TABLES // ============================================================================ @@ -1209,4 +1243,7 @@ export interface Database { // Outbound webhook delivery (#218) webhook_deliveries: WebhookDeliveriesTable; webhook_delivery_attempts: WebhookDeliveryAttemptsTable; + // Inbound webhook receivers (#155) + receivers: ReceiversTable; + receiver_events: ReceiverEventsTable; } diff --git a/packages/backend/src/modules/audit-log/actions.ts b/packages/backend/src/modules/audit-log/actions.ts index 135684cb..c6dca7fc 100644 --- a/packages/backend/src/modules/audit-log/actions.ts +++ b/packages/backend/src/modules/audit-log/actions.ts @@ -23,6 +23,9 @@ export const AUDIT_ACTIONS = { // api keys 'apikey.created': 'config_change', 'apikey.revoked': 'config_change', + // inbound webhook receivers (#155) + 'receiver.created': 'config_change', + 'receiver.deleted': 'config_change', // users and membership 'user.registered': 'user_management', 'user.created': 'user_management', diff --git a/packages/backend/src/modules/auth/plugin.ts b/packages/backend/src/modules/auth/plugin.ts index f4241af7..dc056447 100644 --- a/packages/backend/src/modules/auth/plugin.ts +++ b/packages/backend/src/modules/auth/plugin.ts @@ -87,7 +87,10 @@ const authPlugin: FastifyPluginAsync = async (fastify) => { request.url.startsWith('/api/v1/invitations') || request.url.startsWith('/api/v1/status') || request.url.startsWith('/api/v1/status-incidents') || - request.url.startsWith('/api/v1/maintenances') + request.url.startsWith('/api/v1/maintenances') || + // Inbound webhook receivers (#155): the URL token is the credential, + // verified by the route itself. + request.url.startsWith('/api/v1/receivers') ) { return; } diff --git a/packages/backend/src/modules/receivers/adapters/generic.ts b/packages/backend/src/modules/receivers/adapters/generic.ts new file mode 100644 index 00000000..09079726 --- /dev/null +++ b/packages/backend/src/modules/receivers/adapters/generic.ts @@ -0,0 +1,84 @@ +import type { LogLevel } from '@logtide/shared'; +import { LOG_LEVELS } from '@logtide/shared'; +import type { ReceiverAdapter } from './types.js'; + +/** Resolve a dot-path ("a.b.c") into a nested object. */ +export function getPath(obj: unknown, path: string): unknown { + let current: unknown = obj; + for (const key of path.split('.')) { + if (current === null || typeof current !== 'object') return undefined; + current = (current as Record)[key]; + } + return current; +} + +const LEVEL_SYNONYMS: Record = { + warning: 'warn', + fatal: 'critical', + err: 'error', + crit: 'critical', +}; + +function coerceLevel( + value: unknown, + levelMap: Record | undefined, + fallback: LogLevel +): LogLevel { + if (typeof value !== 'string' && typeof value !== 'number') return fallback; + const raw = String(value).toLowerCase(); + if (levelMap) { + for (const [key, mapped] of Object.entries(levelMap)) { + if (key.toLowerCase() === raw) return mapped; + } + } + if ((LOG_LEVELS as readonly string[]).includes(raw)) return raw as LogLevel; + if (raw in LEVEL_SYNONYMS) return LEVEL_SYNONYMS[raw]; + return fallback; +} + +export const genericAdapter: ReceiverAdapter = (payload, receiver) => { + const mapping = receiver.fieldMapping ?? {}; + const defaults = mapping.defaults ?? {}; + + const messageRaw = mapping.message ? getPath(payload, mapping.message) : undefined; + const message = + typeof messageRaw === 'string' && messageRaw.length > 0 ? messageRaw : 'Received event'; + + const serviceRaw = mapping.service ? getPath(payload, mapping.service) : undefined; + const service = ( + typeof serviceRaw === 'string' && serviceRaw.length > 0 + ? serviceRaw + : (defaults.service ?? receiver.name) + ).slice(0, 100); + + const level = coerceLevel( + mapping.level ? getPath(payload, mapping.level) : undefined, + mapping.levelMap, + defaults.level ?? 'info' + ); + + let time = new Date().toISOString(); + if (mapping.timestamp) { + const ts = getPath(payload, mapping.timestamp); + if (typeof ts === 'string' || typeof ts === 'number') { + const parsed = new Date(ts); + if (!Number.isNaN(parsed.getTime())) time = parsed.toISOString(); + } + } + + return { + kind: 'logs', + logs: [ + { + time, + service, + level, + message, + metadata: { + receiver: { id: receiver.id, name: receiver.name, adapter: 'generic' }, + payload, + }, + }, + ], + }; +}; diff --git a/packages/backend/src/modules/receivers/adapters/github.ts b/packages/backend/src/modules/receivers/adapters/github.ts new file mode 100644 index 00000000..80a38a45 --- /dev/null +++ b/packages/backend/src/modules/receivers/adapters/github.ts @@ -0,0 +1,111 @@ +import type { LogInput, LogLevel } from '@logtide/shared'; +import type { ReceiverAdapter } from './types.js'; + +function conclusionLevel(conclusion: string | null): LogLevel { + switch (conclusion) { + case 'success': + return 'info'; + case 'cancelled': + case 'skipped': + case 'neutral': + return 'warn'; + case 'failure': + case 'timed_out': + case 'startup_failure': + case 'action_required': + return 'error'; + default: + return 'info'; + } +} + +function isoOrNow(value: unknown): string { + if (typeof value === 'string') { + const parsed = new Date(value); + if (!Number.isNaN(parsed.getTime())) return parsed.toISOString(); + } + return new Date().toISOString(); +} + +function asRecord(value: unknown): Record | undefined { + return value !== null && typeof value === 'object' ? (value as Record) : undefined; +} + +/** + * GitHub webhook adapter (#155). Detects events by payload shape rather than + * the X-GitHub-Event header so replayed/stored payloads normalize identically. + * Supported: workflow_run (completed), deployment_status. Ping and everything + * else is skipped. + */ +export const githubAdapter: ReceiverAdapter = (payload, receiver) => { + if ('zen' in payload && 'hook_id' in payload) { + return { kind: 'skipped', reason: 'github ping event' }; + } + + const repoName = asRecord(payload.repository)?.full_name; + const service = (typeof repoName === 'string' ? repoName : 'github').slice(0, 100); + const senderLogin = asRecord(payload.sender)?.login; + const actor = typeof senderLogin === 'string' ? senderLogin : undefined; + const action = typeof payload.action === 'string' ? payload.action : undefined; + + const run = asRecord(payload.workflow_run); + if (run) { + if (action !== 'completed') { + return { kind: 'skipped', reason: `workflow_run action "${action ?? 'unknown'}" ignored` }; + } + const conclusion = typeof run.conclusion === 'string' ? run.conclusion : null; + const workflowName = typeof run.name === 'string' ? run.name : 'workflow'; + const log: LogInput = { + time: isoOrNow(run.updated_at), + service, + level: conclusionLevel(conclusion), + message: `Workflow ${workflowName} completed: ${conclusion ?? 'unknown'}`, + metadata: { + receiver: { id: receiver.id, name: receiver.name, adapter: 'github' }, + event: 'workflow_run', + action, + repository: service, + workflow: workflowName, + conclusion, + run_id: run.id, + run_url: run.html_url, + branch: run.head_branch, + actor, + }, + }; + return { kind: 'logs', logs: [log] }; + } + + const status = asRecord(payload.deployment_status); + if (status) { + const deployment = asRecord(payload.deployment); + const state = typeof status.state === 'string' ? status.state : 'unknown'; + const environment = + typeof status.environment === 'string' + ? status.environment + : typeof deployment?.environment === 'string' + ? deployment.environment + : 'unknown'; + const level: LogLevel = + state === 'success' ? 'info' : state === 'failure' || state === 'error' ? 'error' : 'info'; + const log: LogInput = { + time: isoOrNow(status.updated_at), + service, + level, + message: `Deployment to ${environment}: ${state}`, + metadata: { + receiver: { id: receiver.id, name: receiver.name, adapter: 'github' }, + event: 'deployment_status', + action, + repository: service, + environment, + state, + deployment_url: status.target_url, + actor, + }, + }; + return { kind: 'logs', logs: [log] }; + } + + return { kind: 'skipped', reason: 'unsupported github event' }; +}; diff --git a/packages/backend/src/modules/receivers/adapters/index.ts b/packages/backend/src/modules/receivers/adapters/index.ts new file mode 100644 index 00000000..e946d364 --- /dev/null +++ b/packages/backend/src/modules/receivers/adapters/index.ts @@ -0,0 +1,19 @@ +import type { ReceiverAdapterType } from '@logtide/shared'; +import type { ReceiverAdapter } from './types.js'; +import { genericAdapter } from './generic.js'; +import { githubAdapter } from './github.js'; +import { uptimeAdapter } from './uptime.js'; + +const ADAPTERS: Record = { + generic: genericAdapter, + github: githubAdapter, + uptime: uptimeAdapter, +}; + +export function getAdapter(type: ReceiverAdapterType): ReceiverAdapter { + const adapter = ADAPTERS[type]; + if (!adapter) throw new Error(`No adapter registered for type "${type}"`); + return adapter; +} + +export type { AdapterReceiverInfo, AdapterResult, ReceiverAdapter } from './types.js'; diff --git a/packages/backend/src/modules/receivers/adapters/types.ts b/packages/backend/src/modules/receivers/adapters/types.ts new file mode 100644 index 00000000..67c5dd04 --- /dev/null +++ b/packages/backend/src/modules/receivers/adapters/types.ts @@ -0,0 +1,22 @@ +import type { LogInput, ReceiverAdapterType, ReceiverFieldMapping } from '@logtide/shared'; + +export interface AdapterReceiverInfo { + id: string; + name: string; + adapterType: ReceiverAdapterType; + fieldMapping: ReceiverFieldMapping | null; +} + +export type AdapterResult = + | { kind: 'logs'; logs: LogInput[] } + | { kind: 'skipped'; reason: string }; + +/** + * A receiver adapter is a pure function turning a raw external payload into + * zero or more log entries. New adapters plug in via adapters/index.ts and + * need no changes to the receiver core. + */ +export type ReceiverAdapter = ( + payload: Record, + receiver: AdapterReceiverInfo +) => AdapterResult; diff --git a/packages/backend/src/modules/receivers/adapters/uptime.ts b/packages/backend/src/modules/receivers/adapters/uptime.ts new file mode 100644 index 00000000..ddfdd258 --- /dev/null +++ b/packages/backend/src/modules/receivers/adapters/uptime.ts @@ -0,0 +1,94 @@ +import type { LogInput, LogLevel } from '@logtide/shared'; +import type { ReceiverAdapter } from './types.js'; + +function asRecord(value: unknown): Record | undefined { + return value !== null && typeof value === 'object' ? (value as Record) : undefined; +} + +function str(value: unknown): string | undefined { + return typeof value === 'string' && value.length > 0 ? value : undefined; +} + +/** + * Uptime monitoring adapter (#155). Shape-detects two providers: + * - Uptime Robot: flat payload with alertType (1=down, 2=up, 3=ssl expiry) + * - Better Stack: { data: { type: 'incident', attributes: {...} } } + */ +export const uptimeAdapter: ReceiverAdapter = (payload, receiver) => { + // Uptime Robot + if ('alertType' in payload || 'monitorFriendlyName' in payload) { + const alertType = Number(payload.alertType); + const monitor = str(payload.monitorFriendlyName) ?? str(payload.monitorURL) ?? 'monitor'; + const details = str(payload.alertDetails); + let level: LogLevel; + let message: string; + if (alertType === 1) { + level = 'error'; + message = `Monitor ${monitor} is DOWN${details ? `: ${details}` : ''}`; + } else if (alertType === 2) { + level = 'info'; + message = `Monitor ${monitor} is UP${details ? `: ${details}` : ''}`; + } else if (alertType === 3) { + level = 'warn'; + message = `Monitor ${monitor} SSL certificate expiry${details ? `: ${details}` : ''}`; + } else { + return { kind: 'skipped', reason: `unknown uptime robot alertType "${String(payload.alertType)}"` }; + } + const log: LogInput = { + time: new Date().toISOString(), + service: monitor.slice(0, 100), + level, + message, + metadata: { + receiver: { id: receiver.id, name: receiver.name, adapter: 'uptime' }, + provider: 'uptimerobot', + monitor_id: payload.monitorID, + monitor_url: payload.monitorURL, + alert_type: str(payload.alertTypeFriendlyName) ?? alertType, + details, + }, + }; + return { kind: 'logs', logs: [log] }; + } + + // Better Stack + const attributes = asRecord(asRecord(payload.data)?.attributes); + if (attributes) { + const status = (str(attributes.status) ?? '').toLowerCase(); + const monitor = str(attributes.name) ?? str(attributes.url) ?? 'monitor'; + const cause = str(attributes.cause) ?? 'unknown cause'; + let level: LogLevel; + let message: string; + if (status === 'started') { + level = 'error'; + message = `Incident started: ${cause}`; + } else if (status === 'resolved') { + level = 'info'; + message = `Incident resolved: ${cause}`; + } else if (status === 'acknowledged') { + level = 'info'; + message = `Incident acknowledged: ${cause}`; + } else { + return { kind: 'skipped', reason: `unknown better stack incident status "${status}"` }; + } + const log: LogInput = { + time: new Date().toISOString(), + service: monitor.slice(0, 100), + level, + message, + metadata: { + receiver: { id: receiver.id, name: receiver.name, adapter: 'uptime' }, + provider: 'betterstack', + incident_id: asRecord(payload.data)?.id, + status, + cause, + monitor_url: attributes.url, + started_at: attributes.started_at, + resolved_at: attributes.resolved_at, + }, + }; + return { kind: 'logs', logs: [log] }; + } + + return { kind: 'skipped', reason: 'unrecognized uptime payload shape' }; +}; diff --git a/packages/backend/src/modules/receivers/ingest-routes.ts b/packages/backend/src/modules/receivers/ingest-routes.ts new file mode 100644 index 00000000..550e09f4 --- /dev/null +++ b/packages/backend/src/modules/receivers/ingest-routes.ts @@ -0,0 +1,57 @@ +import type { FastifyInstance } from 'fastify'; +import { z } from 'zod'; +import { receiversService } from './service.js'; +import { createQueue } from '../../queue/connection.js'; +import { RECEIVER_EVENTS_QUEUE, type ReceiverEventJobData } from '../../queue/jobs/receiver-event.js'; + +const paramsSchema = z.object({ + receiverId: z.string().uuid(), + token: z.string().min(1).max(200), +}); + +/** Serialized raw payloads above this size are rejected (protects receiver_events). */ +const MAX_PAYLOAD_BYTES = 256 * 1024; + +const receiverEventsQueue = createQueue(RECEIVER_EVENTS_QUEUE); + +/** + * Public inbound webhook endpoint (#155): POST /api/v1/receivers/:receiverId/:token + * Token authentication happens here (the URL is the credential; GitHub and + * Uptime Robot cannot send custom headers). The auth plugin skips this prefix. + */ +export async function receiversIngestRoutes(fastify: FastifyInstance) { + fastify.post('/:receiverId/:token', async (request, reply) => { + const params = paramsSchema.safeParse(request.params); + if (!params.success) { + return reply.status(404).send({ error: 'Receiver not found' }); + } + const { receiverId, token } = params.data; + + const receiver = await receiversService.getReceiverForIngest(receiverId); + if (!receiver) { + return reply.status(404).send({ error: 'Receiver not found' }); + } + if (!receiversService.tokenMatches(token, receiver.tokenHash)) { + return reply.status(401).send({ error: 'Invalid receiver token' }); + } + if (!receiver.enabled) { + return reply.status(403).send({ error: 'Receiver is disabled' }); + } + + const payload = request.body; + if (payload === null || typeof payload !== 'object' || Array.isArray(payload)) { + return reply.status(400).send({ error: 'Payload must be a JSON object' }); + } + if (Buffer.byteLength(JSON.stringify(payload), 'utf8') > MAX_PAYLOAD_BYTES) { + return reply.status(413).send({ error: 'Payload too large (max 256 KB)' }); + } + + const eventId = await receiversService.recordEvent( + receiverId, + payload as Record + ); + await receiverEventsQueue.add('process', { eventId }, { maxAttempts: 1 }); + + return reply.status(202).send({ eventId }); + }); +} diff --git a/packages/backend/src/modules/receivers/routes.ts b/packages/backend/src/modules/receivers/routes.ts new file mode 100644 index 00000000..a23c7bf7 --- /dev/null +++ b/packages/backend/src/modules/receivers/routes.ts @@ -0,0 +1,202 @@ +import type { FastifyInstance } from 'fastify'; +import { z } from 'zod'; +import { receiverAdapterTypeSchema, receiverFieldMappingSchema } from '@logtide/shared'; +import { context } from '@logtide/shared/context'; +import { receiversService } from './service.js'; +import { authenticate } from '../auth/middleware.js'; +import { projectsService } from '../projects/service.js'; +import { auditLogService } from '../audit-log/index.js'; +import { assertWithinLimit, withLimitLock } from '../../capabilities/index.js'; +import { CapabilityError } from '../../capabilities/errors.js'; + +const createReceiverSchema = z + .object({ + name: z.string().min(1, 'Name is required').max(100, 'Name too long'), + adapterType: receiverAdapterTypeSchema, + fieldMapping: receiverFieldMappingSchema.optional().nullable(), + }) + .refine((v) => v.adapterType === 'generic' || v.fieldMapping == null, { + message: 'fieldMapping is only supported by the generic adapter', + path: ['fieldMapping'], + }); + +const updateReceiverSchema = z + .object({ + name: z.string().min(1).max(100).optional(), + enabled: z.boolean().optional(), + fieldMapping: receiverFieldMappingSchema.optional().nullable(), + }) + .refine((v) => v.name !== undefined || v.enabled !== undefined || v.fieldMapping !== undefined, { + message: 'At least one field must be provided', + }); + +const projectIdSchema = z.object({ + projectId: z.string().uuid('Invalid project ID format'), +}); + +const receiverIdSchema = z.object({ + projectId: z.string().uuid('Invalid project ID format'), + id: z.string().uuid('Invalid receiver ID format'), +}); + +const eventsQuerySchema = z.object({ + limit: z.coerce.number().int().min(1).max(100).default(50), +}); + +export async function receiversRoutes(fastify: FastifyInstance) { + // All routes require authentication + fastify.addHook('onRequest', authenticate); + + // List receivers for a project + fastify.get('/:projectId/receivers', async (request: any, reply) => { + try { + const { projectId } = projectIdSchema.parse(request.params); + + const project = await projectsService.getProjectById(projectId, request.user.id); + if (!project || project.deletedAt) { + return reply.status(404).send({ error: 'Project not found or access denied' }); + } + + const receivers = await receiversService.listReceivers(projectId); + return reply.send({ receivers }); + } catch (error) { + if (error instanceof z.ZodError) { + return reply.status(400).send({ error: 'Invalid project ID format' }); + } + throw error; + } + }); + + // Create a receiver + fastify.post('/:projectId/receivers', async (request: any, reply) => { + try { + const { projectId } = projectIdSchema.parse(request.params); + const body = createReceiverSchema.parse(request.body); + + const project = await projectsService.getProjectById(projectId, request.user.id); + if (!project || project.deletedAt) { + return reply.status(404).send({ error: 'Project not found or access denied' }); + } + + const result = await withLimitLock(project.organizationId, 'receivers.max', async () => { + await context.runAsSystem('receivers:create-limit-check', async () => { + await context.with({ organizationId: project.organizationId }, async () => { + const count = await receiversService.countReceiversForOrg(project.organizationId); + await assertWithinLimit('receivers.max', count); + }); + }); + + return receiversService.createReceiver({ + projectId, + name: body.name, + adapterType: body.adapterType, + fieldMapping: body.fieldMapping ?? null, + }); + }); + + await auditLogService.record({ + action: 'receiver.created', + target: { type: 'receiver', id: result.id }, + organizationId: project.organizationId, + metadata: { name: body.name, adapterType: body.adapterType, projectId }, + }); + + return reply.status(201).send({ + id: result.id, + token: result.token, + ingestPath: `/api/v1/receivers/${result.id}/${result.token}`, + message: 'Receiver created. Save the URL securely - the token will not be shown again.', + }); + } catch (error) { + if (error instanceof CapabilityError) { + throw error; + } + if (error instanceof z.ZodError) { + return reply.status(400).send({ error: 'Validation error', details: error.errors }); + } + throw error; + } + }); + + // Update a receiver (rename, enable/disable, edit field mapping) + fastify.patch('/:projectId/receivers/:id', async (request: any, reply) => { + try { + const { projectId, id } = receiverIdSchema.parse(request.params); + const body = updateReceiverSchema.parse(request.body); + + const project = await projectsService.getProjectById(projectId, request.user.id); + if (!project || project.deletedAt) { + return reply.status(404).send({ error: 'Project not found or access denied' }); + } + + const updated = await receiversService.updateReceiver(id, projectId, body); + if (!updated) { + return reply.status(404).send({ error: 'Receiver not found' }); + } + return reply.send({ success: true }); + } catch (error) { + if (error instanceof z.ZodError) { + return reply.status(400).send({ error: 'Validation error', details: error.errors }); + } + throw error; + } + }); + + // Delete a receiver + fastify.delete('/:projectId/receivers/:id', async (request: any, reply) => { + try { + const { projectId, id } = receiverIdSchema.parse(request.params); + + const project = await projectsService.getProjectById(projectId, request.user.id); + if (!project || project.deletedAt) { + return reply.status(404).send({ error: 'Project not found or access denied' }); + } + + const deleted = await receiversService.deleteReceiver(id, projectId); + if (!deleted) { + return reply.status(404).send({ error: 'Receiver not found' }); + } + + await auditLogService.record({ + action: 'receiver.deleted', + target: { type: 'receiver', id }, + organizationId: project.organizationId, + metadata: { projectId }, + }); + + return reply.status(204).send(); + } catch (error) { + if (error instanceof z.ZodError) { + return reply.status(400).send({ error: 'Invalid ID format' }); + } + throw error; + } + }); + + // Recent events for a receiver (debug view) + fastify.get('/:projectId/receivers/:id/events', async (request: any, reply) => { + try { + const { projectId, id } = receiverIdSchema.parse(request.params); + const { limit } = eventsQuerySchema.parse(request.query ?? {}); + + const project = await projectsService.getProjectById(projectId, request.user.id); + if (!project || project.deletedAt) { + return reply.status(404).send({ error: 'Project not found or access denied' }); + } + + // Confirm the receiver belongs to this project before listing its events. + const receivers = await receiversService.listReceivers(projectId); + if (!receivers.some((r) => r.id === id)) { + return reply.status(404).send({ error: 'Receiver not found' }); + } + + const events = await receiversService.listEvents(id, limit); + return reply.send({ events }); + } catch (error) { + if (error instanceof z.ZodError) { + return reply.status(400).send({ error: 'Invalid ID format' }); + } + throw error; + } + }); +} diff --git a/packages/backend/src/modules/receivers/service.ts b/packages/backend/src/modules/receivers/service.ts new file mode 100644 index 00000000..e2653751 --- /dev/null +++ b/packages/backend/src/modules/receivers/service.ts @@ -0,0 +1,296 @@ +import crypto from 'crypto'; +import type { LogInput, ReceiverAdapterType, ReceiverFieldMapping } from '@logtide/shared'; +import { db } from '../../database/connection.js'; + +export interface Receiver { + id: string; + projectId: string; + name: string; + adapterType: ReceiverAdapterType; + fieldMapping: ReceiverFieldMapping | null; + enabled: boolean; + createdAt: Date; + lastReceivedAt: Date | null; +} + +export interface IngestReceiver { + id: string; + projectId: string; + organizationId: string; + name: string; + adapterType: ReceiverAdapterType; + fieldMapping: ReceiverFieldMapping | null; + enabled: boolean; + tokenHash: string; +} + +export type ReceiverEventStatus = 'pending' | 'processed' | 'skipped' | 'failed'; + +export interface ReceiverEvent { + id: string; + receiverId: string; + status: ReceiverEventStatus; + rawPayload: Record; + normalized: unknown | null; + error: string | null; + receivedAt: Date; +} + +export interface EventWithReceiver extends IngestReceiver { + eventId: string; + status: ReceiverEventStatus; + rawPayload: Record; +} + +export interface CreateReceiverInput { + projectId: string; + name: string; + adapterType: ReceiverAdapterType; + fieldMapping?: ReceiverFieldMapping | null; +} + +export class ReceiversService { + static readonly MAX_EVENTS_PER_RECEIVER = 100; + + private hashToken(token: string): string { + return crypto.createHash('sha256').update(token).digest('hex'); + } + + generateToken(): string { + return `lr_${crypto.randomBytes(32).toString('hex')}`; + } + + /** Timing-safe comparison of a presented token against the stored hash. */ + tokenMatches(token: string, tokenHash: string): boolean { + const presented = Buffer.from(this.hashToken(token), 'hex'); + const stored = Buffer.from(tokenHash, 'hex'); + return presented.length === stored.length && crypto.timingSafeEqual(presented, stored); + } + + async createReceiver(input: CreateReceiverInput): Promise<{ id: string; token: string }> { + const token = this.generateToken(); + const result = await db + .insertInto('receivers') + .values({ + project_id: input.projectId, + name: input.name, + adapter_type: input.adapterType, + token_hash: this.hashToken(token), + field_mapping: input.fieldMapping ?? null, + }) + .returning(['id']) + .executeTakeFirstOrThrow(); + return { id: result.id, token }; + } + + async listReceivers(projectId: string): Promise { + const rows = await db + .selectFrom('receivers') + .select([ + 'id', + 'project_id', + 'name', + 'adapter_type', + 'field_mapping', + 'enabled', + 'created_at', + 'last_received_at', + ]) + .where('project_id', '=', projectId) + .orderBy('created_at', 'desc') + .execute(); + return rows.map((r) => ({ + id: r.id, + projectId: r.project_id, + name: r.name, + adapterType: r.adapter_type as ReceiverAdapterType, + fieldMapping: (r.field_mapping as ReceiverFieldMapping | null) ?? null, + enabled: r.enabled, + createdAt: new Date(r.created_at), + lastReceivedAt: r.last_received_at ? new Date(r.last_received_at) : null, + })); + } + + async updateReceiver( + id: string, + projectId: string, + patch: { name?: string; enabled?: boolean; fieldMapping?: ReceiverFieldMapping | null } + ): Promise { + const values: Record = {}; + if (patch.name !== undefined) values.name = patch.name; + if (patch.enabled !== undefined) values.enabled = patch.enabled; + if (patch.fieldMapping !== undefined) values.field_mapping = patch.fieldMapping; + if (Object.keys(values).length === 0) return false; + const result = await db + .updateTable('receivers') + .set(values) + .where('id', '=', id) + .where('project_id', '=', projectId) + .executeTakeFirst(); + return Number(result.numUpdatedRows || 0) > 0; + } + + async deleteReceiver(id: string, projectId: string): Promise { + const result = await db + .deleteFrom('receivers') + .where('id', '=', id) + .where('project_id', '=', projectId) + .executeTakeFirst(); + return Number(result.numDeletedRows || 0) > 0; + } + + /** Count receivers across all projects of an organization (capability limit input). */ + async countReceiversForOrg(organizationId: string): Promise { + const row = await db + .selectFrom('receivers') + .innerJoin('projects', 'projects.id', 'receivers.project_id') + .select((eb) => eb.fn.countAll().as('count')) + .where('projects.organization_id', '=', organizationId) + .executeTakeFirst(); + return Number(row?.count ?? 0); + } + + /** Load a receiver for the public ingest endpoint (org derived via projects join). */ + async getReceiverForIngest(receiverId: string): Promise { + const row = await db + .selectFrom('receivers') + .innerJoin('projects', 'projects.id', 'receivers.project_id') + .select([ + 'receivers.id', + 'receivers.project_id', + 'receivers.name', + 'receivers.adapter_type', + 'receivers.field_mapping', + 'receivers.enabled', + 'receivers.token_hash', + 'projects.organization_id', + ]) + .where('receivers.id', '=', receiverId) + .where('projects.deleted_at', 'is', null) + .executeTakeFirst(); + if (!row) return null; + return { + id: row.id, + projectId: row.project_id, + organizationId: row.organization_id, + name: row.name, + adapterType: row.adapter_type as ReceiverAdapterType, + fieldMapping: (row.field_mapping as ReceiverFieldMapping | null) ?? null, + enabled: row.enabled, + tokenHash: row.token_hash, + }; + } + + async recordEvent(receiverId: string, rawPayload: Record): Promise { + const result = await db + .insertInto('receiver_events') + .values({ receiver_id: receiverId, status: 'pending', raw_payload: rawPayload }) + .returning(['id']) + .executeTakeFirstOrThrow(); + return result.id; + } + + async getEventWithReceiver(eventId: string): Promise { + const row = await db + .selectFrom('receiver_events') + .innerJoin('receivers', 'receivers.id', 'receiver_events.receiver_id') + .innerJoin('projects', 'projects.id', 'receivers.project_id') + .select([ + 'receiver_events.id as event_id', + 'receiver_events.status', + 'receiver_events.raw_payload', + 'receivers.id', + 'receivers.project_id', + 'receivers.name', + 'receivers.adapter_type', + 'receivers.field_mapping', + 'receivers.enabled', + 'receivers.token_hash', + 'projects.organization_id', + ]) + .where('receiver_events.id', '=', eventId) + .where('projects.deleted_at', 'is', null) + .executeTakeFirst(); + if (!row) return null; + return { + eventId: row.event_id, + status: row.status as ReceiverEventStatus, + rawPayload: row.raw_payload as Record, + id: row.id, + projectId: row.project_id, + organizationId: row.organization_id, + name: row.name, + adapterType: row.adapter_type as ReceiverAdapterType, + fieldMapping: (row.field_mapping as ReceiverFieldMapping | null) ?? null, + enabled: row.enabled, + tokenHash: row.token_hash, + }; + } + + async completeEvent( + eventId: string, + outcome: { + status: Exclude; + normalized?: LogInput[] | null; + error?: string | null; + } + ): Promise { + await db + .updateTable('receiver_events') + .set({ + status: outcome.status, + // JSON.stringify because the pg driver would serialize a JS array root + // as a Postgres array literal, which is invalid for JSONB. + normalized: outcome.normalized != null ? JSON.stringify(outcome.normalized) : null, + error: outcome.error ?? null, + }) + .where('id', '=', eventId) + .execute(); + } + + async listEvents(receiverId: string, limit = 50): Promise { + const rows = await db + .selectFrom('receiver_events') + .select(['id', 'receiver_id', 'status', 'raw_payload', 'normalized', 'error', 'received_at']) + .where('receiver_id', '=', receiverId) + .orderBy('received_at', 'desc') + .limit(limit) + .execute(); + return rows.map((r) => ({ + id: r.id, + receiverId: r.receiver_id, + status: r.status as ReceiverEventStatus, + rawPayload: r.raw_payload as Record, + normalized: r.normalized ?? null, + error: r.error, + receivedAt: new Date(r.received_at), + })); + } + + async touchLastReceived(receiverId: string): Promise { + // tenant-scope-ok: receiverId comes from a token-authenticated ingest request or a receiver row already loaded with its project join + await db + .updateTable('receivers') + .set({ last_received_at: new Date() }) + .where('id', '=', receiverId) + .execute(); + } + + /** Keep only the newest `keep` events for a receiver. */ + async pruneEvents(receiverId: string, keep = ReceiversService.MAX_EVENTS_PER_RECEIVER): Promise { + await db + .deleteFrom('receiver_events') + .where('receiver_id', '=', receiverId) + .where('id', 'not in', (eb) => + eb + .selectFrom('receiver_events') + .select('id') + .where('receiver_id', '=', receiverId) + .orderBy('received_at', 'desc') + .limit(keep) + ) + .execute(); + } +} + +export const receiversService = new ReceiversService(); diff --git a/packages/backend/src/queue/jobs/receiver-event.ts b/packages/backend/src/queue/jobs/receiver-event.ts new file mode 100644 index 00000000..b3cdf720 --- /dev/null +++ b/packages/backend/src/queue/jobs/receiver-event.ts @@ -0,0 +1,116 @@ +/** + * Inbound receiver event processing (#155). The public endpoint stores the raw + * payload and enqueues this job; here the adapter normalizes it and the result + * goes through the normal ingestion pipeline (PII masking, quotas, Sigma). + * Failures are recorded on the event row for the UI; no automatic retry. + */ +import { logSchema, type LogInput } from '@logtide/shared'; +import { context } from '@logtide/shared/context'; +import type { IJob } from '../abstractions/types.js'; +import { receiversService, ReceiversService } from '../../modules/receivers/service.js'; +import { getAdapter, type AdapterResult } from '../../modules/receivers/adapters/index.js'; +import { ingestionService } from '../../modules/ingestion/service.js'; + +export const RECEIVER_EVENTS_QUEUE = 'receiver-events'; + +export interface ReceiverEventJobData { + eventId: string; +} + +export async function processReceiverEvent(job: IJob): Promise { + const event = await receiversService.getEventWithReceiver(job.data.eventId); + if (!event) { + console.warn(`[ReceiverEvent] event ${job.data.eventId} not found, skipping`); + return; + } + if (event.status !== 'pending') { + return; // idempotent: already terminal + } + + let result: AdapterResult; + try { + const adapter = getAdapter(event.adapterType); + result = adapter(event.rawPayload, { + id: event.id, + name: event.name, + adapterType: event.adapterType, + fieldMapping: event.fieldMapping, + }); + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + await receiversService.completeEvent(event.eventId, { + status: 'failed', + error: `adapter error: ${message}`, + }); + return; + } + + if (result.kind === 'skipped') { + await receiversService.completeEvent(event.eventId, { + status: 'skipped', + error: result.reason, + }); + await finalize(event.id); + return; + } + + const valid: LogInput[] = []; + const invalid: string[] = []; + for (const log of result.logs) { + const parsed = logSchema.safeParse(log); + if (parsed.success) valid.push(parsed.data); + else invalid.push(parsed.error.errors.map((e) => e.message).join('; ')); + } + + if (valid.length === 0) { + await receiversService.completeEvent(event.eventId, { + status: 'failed', + normalized: result.logs, + error: `adapter produced no valid logs: ${invalid.join(' | ')}`, + }); + await finalize(event.id); + return; + } + + try { + const ingestResult = await context.runAsSystem('receiver-event', () => + context.with( + { organizationId: event.organizationId, projectId: event.projectId }, + () => ingestionService.ingestLogs(valid, event.projectId) + ) + ); + + if (ingestResult.received === 0) { + const reasons = ingestResult.rejected.map((r) => r.reason).join('; '); + await receiversService.completeEvent(event.eventId, { + status: 'failed', + normalized: valid, + error: `all logs rejected by ingestion: ${reasons}`, + }); + } else { + const partial = + ingestResult.rejected.length > 0 + ? `${ingestResult.rejected.length} of ${result.logs.length} logs rejected` + : null; + await receiversService.completeEvent(event.eventId, { + status: 'processed', + normalized: valid, + error: partial, + }); + } + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + await receiversService.completeEvent(event.eventId, { + status: 'failed', + normalized: valid, + error: `ingestion failed: ${message}`, + }); + } + + await finalize(event.id); +} + +async function finalize(receiverId: string): Promise { + await receiversService.touchLastReceived(receiverId); + await receiversService.pruneEvents(receiverId, ReceiversService.MAX_EVENTS_PER_RECEIVER); +} diff --git a/packages/backend/src/server.ts b/packages/backend/src/server.ts index a20173fb..1733622a 100644 --- a/packages/backend/src/server.ts +++ b/packages/backend/src/server.ts @@ -17,6 +17,8 @@ import { organizationsRoutes } from './modules/organizations/routes.js'; import { invitationsRoutes } from './modules/invitations/routes.js'; import { notificationsRoutes } from './modules/notifications/routes.js'; import { apiKeysRoutes } from './modules/api-keys/routes.js'; +import { receiversRoutes } from './modules/receivers/routes.js'; +import { receiversIngestRoutes } from './modules/receivers/ingest-routes.js'; import dashboardRoutes from './modules/dashboard/routes.js'; import { sigmaRoutes } from './modules/sigma/routes.js'; import { siemRoutes } from './modules/siem/routes.js'; @@ -189,6 +191,8 @@ export async function build(opts = {}) { await fastify.register(registerSiemSseRoutes); await fastify.register(exceptionsRoutes); await fastify.register(apiKeysRoutes, { prefix: '/api/v1/projects' }); + await fastify.register(receiversRoutes, { prefix: '/api/v1/projects' }); + await fastify.register(receiversIngestRoutes, { prefix: '/api/v1/receivers' }); await fastify.register(dashboardRoutes); await fastify.register(adminRoutes, { prefix: '/api/v1/admin' }); await fastify.register(settingsRoutes, { prefix: '/api/v1/admin/settings' }); diff --git a/packages/backend/src/tests/database/tenant-tables.test.ts b/packages/backend/src/tests/database/tenant-tables.test.ts index 1fd5f6b3..65805348 100644 --- a/packages/backend/src/tests/database/tenant-tables.test.ts +++ b/packages/backend/src/tests/database/tenant-tables.test.ts @@ -18,7 +18,7 @@ const ALL_TABLES = [ 'organization_default_channels', 'pii_masking_rules', 'organization_pii_salts', 'audit_log', 'metrics_hourly_stats', 'metrics_daily_stats', 'metrics', 'metric_exemplars', 'custom_dashboards', 'log_pipelines', 'digest_configs', - 'digest_recipients', + 'digest_recipients', 'receivers', 'receiver_events', ]; describe('tenant-tables manifest', () => { diff --git a/packages/backend/src/tests/modules/capabilities/receivers-limit.test.ts b/packages/backend/src/tests/modules/capabilities/receivers-limit.test.ts new file mode 100644 index 00000000..1b1ba306 --- /dev/null +++ b/packages/backend/src/tests/modules/capabilities/receivers-limit.test.ts @@ -0,0 +1,193 @@ +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import Fastify, { FastifyInstance } from 'fastify'; +import crypto from 'crypto'; +import { db } from '../../../database/index.js'; +import { receiversRoutes } from '../../../modules/receivers/routes.js'; +import { contextPlugin } from '../../../context/index.js'; +import { capabilities } from '../../../capabilities/index.js'; +import { createTestContext } from '../../helpers/factories.js'; + +async function createTestSession(userId: string) { + const token = crypto.randomBytes(32).toString('hex'); + await db + .insertInto('sessions') + .values({ user_id: userId, token, expires_at: new Date(Date.now() + 24 * 60 * 60 * 1000) }) + .execute(); + return token; +} + +async function insertReceiver(projectId: string, n = 1) { + const tokenHash = crypto + .createHash('sha256') + .update(`lr_test_${crypto.randomBytes(16).toString('hex')}`) + .digest('hex'); + const [receiver] = await db + .insertInto('receivers') + .values({ project_id: projectId, name: `Receiver ${n}`, adapter_type: 'generic', token_hash: tokenHash }) + .returningAll() + .execute(); + return receiver; +} + +describe('receivers.max enforcement', () => { + let app: FastifyInstance; + let orgId: string; + let userId: string; + let projectId: string; + let token: string; + + beforeAll(async () => { + app = Fastify(); + await app.register(contextPlugin); + await app.register(receiversRoutes, { prefix: '/api/v1/projects' }); + await app.ready(); + }); + + afterAll(async () => { + await app.close(); + }); + + beforeEach(async () => { + await db.deleteFrom('organization_entitlements').execute(); + await db.deleteFrom('receiver_events').execute(); + await db.deleteFrom('receivers').execute(); + await db.deleteFrom('api_keys').execute(); + await db.deleteFrom('sessions').execute(); + await db.deleteFrom('organization_members').execute(); + await db.deleteFrom('projects').execute(); + await db.deleteFrom('organizations').execute(); + await db.deleteFrom('users').execute(); + + const ctx = await createTestContext(); + orgId = ctx.organization.id; + userId = ctx.user.id; + projectId = ctx.project.id; + token = await createTestSession(userId); + capabilities.invalidate(orgId); + }); + + // Case 1: org-wide blocking across projects + it('blocks receiver creation when the org-wide limit is reached (receiver on a different project)', async () => { + await db + .insertInto('organization_entitlements') + .values({ organization_id: orgId, capability: 'receivers.max', enabled: null, limit_value: 1 }) + .execute(); + capabilities.invalidate(orgId); + + const project2 = await db + .insertInto('projects') + .values({ + name: 'Project Two', + slug: `proj2-${Date.now()}`, + organization_id: orgId, + user_id: userId, + }) + .returningAll() + .executeTakeFirstOrThrow(); + + await insertReceiver(projectId, 1); + + const res = await app.inject({ + method: 'POST', + url: `/api/v1/projects/${project2.id}/receivers`, + headers: { Authorization: `Bearer ${token}` }, + payload: { name: 'New Receiver', adapterType: 'generic' }, + }); + + expect(res.statusCode).toBe(403); + const body = JSON.parse(res.body); + expect(body.statusCode).toBe(403); + expect(body.code).toBe('capability.receivers.max.limit_reached'); + }); + + // Case 2: under limit => passes + it('allows receiver creation when under the limit', async () => { + await db + .insertInto('organization_entitlements') + .values({ organization_id: orgId, capability: 'receivers.max', enabled: null, limit_value: 2 }) + .execute(); + capabilities.invalidate(orgId); + + await insertReceiver(projectId, 1); + + const res = await app.inject({ + method: 'POST', + url: `/api/v1/projects/${projectId}/receivers`, + headers: { Authorization: `Bearer ${token}` }, + payload: { name: 'New Receiver', adapterType: 'generic' }, + }); + + expect(res.statusCode).toBe(201); + }); + + // Case 3: unlimited default (no entitlement row) => passes + it('allows receiver creation when no limit is configured (unlimited default)', async () => { + const res = await app.inject({ + method: 'POST', + url: `/api/v1/projects/${projectId}/receivers`, + headers: { Authorization: `Bearer ${token}` }, + payload: { name: 'New Receiver', adapterType: 'generic' }, + }); + + expect(res.statusCode).toBe(201); + }); + + // Case 5: concurrent creates must not race past a finite limit + it('serializes concurrent creates so the limit is never exceeded (race)', async () => { + await db + .insertInto('organization_entitlements') + .values({ organization_id: orgId, capability: 'receivers.max', enabled: null, limit_value: 2 }) + .execute(); + capabilities.invalidate(orgId); + + const attempts = 6; + const results = await Promise.all( + Array.from({ length: attempts }, (_, i) => + app.inject({ + method: 'POST', + url: `/api/v1/projects/${projectId}/receivers`, + headers: { Authorization: `Bearer ${token}` }, + payload: { name: `Race Receiver ${i}`, adapterType: 'generic' }, + }) + ) + ); + + const created = results.filter((r) => r.statusCode === 201).length; + const blocked = results.filter((r) => r.statusCode === 403).length; + + expect(created).toBe(2); + expect(blocked).toBe(attempts - 2); + + // Hard invariant: the org never ends up over its configured limit. + const total = await db + .selectFrom('receivers') + .innerJoin('projects', 'projects.id', 'receivers.project_id') + .select((eb) => eb.fn.countAll().as('c')) + .where('projects.organization_id', '=', orgId) + .executeTakeFirstOrThrow(); + expect(Number(total.c)).toBe(2); + }); + + // Case 4: org isolation => org A at limit does not block org B + it('does not block org B when org A is at the limit', async () => { + await db + .insertInto('organization_entitlements') + .values({ organization_id: orgId, capability: 'receivers.max', enabled: null, limit_value: 1 }) + .execute(); + capabilities.invalidate(orgId); + await insertReceiver(projectId, 1); + + const ctxB = await createTestContext(); + const tokenB = await createTestSession(ctxB.user.id); + capabilities.invalidate(ctxB.organization.id); + + const res = await app.inject({ + method: 'POST', + url: `/api/v1/projects/${ctxB.project.id}/receivers`, + headers: { Authorization: `Bearer ${tokenB}` }, + payload: { name: 'New Receiver', adapterType: 'generic' }, + }); + + expect(res.statusCode).toBe(201); + }); +}); diff --git a/packages/backend/src/tests/modules/receivers/generic-adapter.test.ts b/packages/backend/src/tests/modules/receivers/generic-adapter.test.ts new file mode 100644 index 00000000..d89a8081 --- /dev/null +++ b/packages/backend/src/tests/modules/receivers/generic-adapter.test.ts @@ -0,0 +1,84 @@ +import { describe, it, expect } from 'vitest'; +import { genericAdapter, getPath } from '../../../modules/receivers/adapters/generic.js'; +import type { AdapterReceiverInfo } from '../../../modules/receivers/adapters/types.js'; + +function receiver(fieldMapping: AdapterReceiverInfo['fieldMapping'] = null): AdapterReceiverInfo { + return { id: 'r-1', name: 'my receiver', adapterType: 'generic', fieldMapping }; +} + +describe('getPath', () => { + it('resolves nested dot paths', () => { + expect(getPath({ a: { b: { c: 42 } } }, 'a.b.c')).toBe(42); + }); + it('returns undefined for missing segments and non-objects', () => { + expect(getPath({ a: 1 }, 'a.b')).toBeUndefined(); + expect(getPath(null, 'a')).toBeUndefined(); + }); +}); + +describe('genericAdapter', () => { + it('maps fields via dot paths', () => { + const result = genericAdapter( + { sev: 'ERROR', txt: 'disk full', app: 'billing', ts: '2026-07-01T10:00:00Z' }, + receiver({ message: 'txt', level: 'sev', service: 'app', timestamp: 'ts' }) + ); + expect(result.kind).toBe('logs'); + if (result.kind !== 'logs') return; + expect(result.logs).toHaveLength(1); + const log = result.logs[0]; + expect(log.message).toBe('disk full'); + expect(log.level).toBe('error'); + expect(log.service).toBe('billing'); + expect(log.time).toBe('2026-07-01T10:00:00.000Z'); + expect((log.metadata as any).payload.txt).toBe('disk full'); + }); + + it('applies levelMap before builtin coercion', () => { + const result = genericAdapter( + { sev: 'CRIT', txt: 'x' }, + receiver({ message: 'txt', level: 'sev', levelMap: { crit: 'critical' } }) + ); + if (result.kind !== 'logs') throw new Error('expected logs'); + expect(result.logs[0].level).toBe('critical'); + }); + + it('coerces common synonyms (warning, fatal)', () => { + const r1 = genericAdapter({ l: 'WARNING' }, receiver({ level: 'l' })); + const r2 = genericAdapter({ l: 'fatal' }, receiver({ level: 'l' })); + if (r1.kind !== 'logs' || r2.kind !== 'logs') throw new Error('expected logs'); + expect(r1.logs[0].level).toBe('warn'); + expect(r2.logs[0].level).toBe('critical'); + }); + + it('falls back to defaults and receiver name without mapping', () => { + const result = genericAdapter({ anything: true }, receiver(null)); + if (result.kind !== 'logs') throw new Error('expected logs'); + const log = result.logs[0]; + expect(log.message).toBe('Received event'); + expect(log.level).toBe('info'); + expect(log.service).toBe('my receiver'); + }); + + it('uses configured defaults over builtin fallbacks', () => { + const result = genericAdapter( + {}, + receiver({ defaults: { level: 'warn', service: 'ext' } }) + ); + if (result.kind !== 'logs') throw new Error('expected logs'); + expect(result.logs[0].level).toBe('warn'); + expect(result.logs[0].service).toBe('ext'); + }); + + it('ignores unparseable timestamps', () => { + const result = genericAdapter({ ts: 'not-a-date' }, receiver({ timestamp: 'ts' })); + if (result.kind !== 'logs') throw new Error('expected logs'); + // falls back to "now": still a valid ISO string + expect(new Date(result.logs[0].time as string).getTime()).not.toBeNaN(); + }); + + it('truncates service to 100 chars', () => { + const result = genericAdapter({ s: 'x'.repeat(200) }, receiver({ service: 's' })); + if (result.kind !== 'logs') throw new Error('expected logs'); + expect((result.logs[0].service as string).length).toBe(100); + }); +}); diff --git a/packages/backend/src/tests/modules/receivers/github-adapter.test.ts b/packages/backend/src/tests/modules/receivers/github-adapter.test.ts new file mode 100644 index 00000000..c34ce8c8 --- /dev/null +++ b/packages/backend/src/tests/modules/receivers/github-adapter.test.ts @@ -0,0 +1,100 @@ +import { describe, it, expect } from 'vitest'; +import { githubAdapter } from '../../../modules/receivers/adapters/github.js'; +import type { AdapterReceiverInfo } from '../../../modules/receivers/adapters/types.js'; + +const receiver: AdapterReceiverInfo = { + id: 'r-1', + name: 'gh', + adapterType: 'github', + fieldMapping: null, +}; + +describe('githubAdapter', () => { + it('skips ping events', () => { + const result = githubAdapter({ zen: 'Keep it simple.', hook_id: 1 }, receiver); + expect(result.kind).toBe('skipped'); + }); + + it('maps a failed workflow_run to an error log', () => { + const result = githubAdapter( + { + action: 'completed', + workflow_run: { + id: 42, + name: 'CI', + conclusion: 'failure', + html_url: 'https://github.com/acme/app/actions/runs/42', + head_branch: 'main', + updated_at: '2026-07-01T10:00:00Z', + }, + repository: { full_name: 'acme/app' }, + sender: { login: 'octocat' }, + }, + receiver + ); + if (result.kind !== 'logs') throw new Error('expected logs'); + const log = result.logs[0]; + expect(log.level).toBe('error'); + expect(log.service).toBe('acme/app'); + expect(log.message).toBe('Workflow CI completed: failure'); + const meta = log.metadata as any; + expect(meta.event).toBe('workflow_run'); + expect(meta.run_id).toBe(42); + expect(meta.branch).toBe('main'); + expect(meta.actor).toBe('octocat'); + }); + + it('maps a successful workflow_run to info and cancelled to warn', () => { + const base = { + action: 'completed', + repository: { full_name: 'acme/app' }, + }; + const ok = githubAdapter( + { ...base, workflow_run: { name: 'CI', conclusion: 'success', updated_at: '2026-07-01T10:00:00Z' } }, + receiver + ); + const cancelled = githubAdapter( + { ...base, workflow_run: { name: 'CI', conclusion: 'cancelled', updated_at: '2026-07-01T10:00:00Z' } }, + receiver + ); + if (ok.kind !== 'logs' || cancelled.kind !== 'logs') throw new Error('expected logs'); + expect(ok.logs[0].level).toBe('info'); + expect(cancelled.logs[0].level).toBe('warn'); + }); + + it('skips non-completed workflow_run actions', () => { + const result = githubAdapter( + { action: 'requested', workflow_run: { name: 'CI' }, repository: { full_name: 'acme/app' } }, + receiver + ); + expect(result.kind).toBe('skipped'); + }); + + it('maps deployment_status events', () => { + const result = githubAdapter( + { + action: 'created', + deployment_status: { + state: 'failure', + environment: 'production', + updated_at: '2026-07-01T10:00:00Z', + }, + deployment: { environment: 'production' }, + repository: { full_name: 'acme/app' }, + sender: { login: 'octocat' }, + }, + receiver + ); + if (result.kind !== 'logs') throw new Error('expected logs'); + expect(result.logs[0].level).toBe('error'); + expect(result.logs[0].message).toBe('Deployment to production: failure'); + }); + + it('skips unsupported events', () => { + const result = githubAdapter( + { action: 'opened', issue: { number: 1 }, repository: { full_name: 'acme/app' } }, + receiver + ); + expect(result.kind).toBe('skipped'); + }); +}); diff --git a/packages/backend/src/tests/modules/receivers/ingest.test.ts b/packages/backend/src/tests/modules/receivers/ingest.test.ts new file mode 100644 index 00000000..9af26870 --- /dev/null +++ b/packages/backend/src/tests/modules/receivers/ingest.test.ts @@ -0,0 +1,150 @@ +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import Fastify, { FastifyInstance } from 'fastify'; +import { db } from '../../../database/index.js'; +import { contextPlugin } from '../../../context/index.js'; +import { receiversIngestRoutes } from '../../../modules/receivers/ingest-routes.js'; +import { receiversService } from '../../../modules/receivers/service.js'; +import { processReceiverEvent } from '../../../queue/jobs/receiver-event.js'; +import type { IJob } from '../../../queue/abstractions/types.js'; +import type { ReceiverEventJobData } from '../../../queue/jobs/receiver-event.js'; +import { createTestContext } from '../../helpers/factories.js'; + +function fakeJob(eventId: string): IJob { + return { id: 'test-job', name: 'process', data: { eventId } } as IJob; +} + +describe('receiver ingest endpoint', () => { + let app: FastifyInstance; + let projectId: string; + let receiverId: string; + let token: string; + + beforeAll(async () => { + app = Fastify(); + await app.register(contextPlugin); + await app.register(receiversIngestRoutes, { prefix: '/api/v1/receivers' }); + await app.ready(); + }); + + afterAll(async () => { + await app.close(); + }); + + beforeEach(async () => { + await db.deleteFrom('receiver_events').execute(); + await db.deleteFrom('receivers').execute(); + await db.deleteFrom('api_keys').execute(); + await db.deleteFrom('organization_members').execute(); + await db.deleteFrom('projects').execute(); + await db.deleteFrom('organizations').execute(); + await db.deleteFrom('users').execute(); + const ctx = await createTestContext(); + projectId = ctx.project.id; + const created = await receiversService.createReceiver({ + projectId, + name: 'generic in', + adapterType: 'generic', + }); + receiverId = created.id; + token = created.token; + }); + + it('accepts a valid payload with 202 and records a pending event', async () => { + const res = await app.inject({ + method: 'POST', + url: `/api/v1/receivers/${receiverId}/${token}`, + payload: { msg: 'hello' }, + }); + expect(res.statusCode).toBe(202); + const { eventId } = JSON.parse(res.body); + const event = await receiversService.getEventWithReceiver(eventId); + expect(event).not.toBeNull(); + expect(event!.status).toBe('pending'); + }); + + it('rejects wrong tokens with 401 and unknown receivers with 404', async () => { + const bad = await app.inject({ + method: 'POST', + url: `/api/v1/receivers/${receiverId}/lr_${'0'.repeat(64)}`, + payload: { a: 1 }, + }); + expect(bad.statusCode).toBe(401); + + const missing = await app.inject({ + method: 'POST', + url: `/api/v1/receivers/00000000-0000-4000-8000-000000000000/${token}`, + payload: { a: 1 }, + }); + expect(missing.statusCode).toBe(404); + }); + + it('rejects disabled receivers with 403', async () => { + await receiversService.updateReceiver(receiverId, projectId, { enabled: false }); + const res = await app.inject({ + method: 'POST', + url: `/api/v1/receivers/${receiverId}/${token}`, + payload: { a: 1 }, + }); + expect(res.statusCode).toBe(403); + }); + + it('rejects non-object payloads with 400', async () => { + const res = await app.inject({ + method: 'POST', + url: `/api/v1/receivers/${receiverId}/${token}`, + payload: JSON.stringify([1, 2, 3]), + headers: { 'content-type': 'application/json' }, + }); + expect(res.statusCode).toBe(400); + }); + + it('processes an event end to end: logs are ingested, event marked processed', async () => { + const res = await app.inject({ + method: 'POST', + url: `/api/v1/receivers/${receiverId}/${token}`, + payload: { msg: 'from external' }, + }); + const { eventId } = JSON.parse(res.body); + + await processReceiverEvent(fakeJob(eventId)); + + const event = await receiversService.getEventWithReceiver(eventId); + expect(event!.status).toBe('processed'); + + const events = await receiversService.listEvents(receiverId, 10); + expect(events[0].normalized).not.toBeNull(); + + const receivers = await receiversService.listReceivers(projectId); + expect(receivers[0].lastReceivedAt).not.toBeNull(); + }); + + it('marks github ping events as skipped', async () => { + const gh = await receiversService.createReceiver({ + projectId, + name: 'gh', + adapterType: 'github', + }); + const res = await app.inject({ + method: 'POST', + url: `/api/v1/receivers/${gh.id}/${gh.token}`, + payload: { zen: 'Design for failure.', hook_id: 1 }, + }); + const { eventId } = JSON.parse(res.body); + await processReceiverEvent(fakeJob(eventId)); + const event = await receiversService.getEventWithReceiver(eventId); + expect(event!.status).toBe('skipped'); + }); + + it('is idempotent: reprocessing a non-pending event is a no-op', async () => { + const res = await app.inject({ + method: 'POST', + url: `/api/v1/receivers/${receiverId}/${token}`, + payload: { msg: 'x' }, + }); + const { eventId } = JSON.parse(res.body); + await processReceiverEvent(fakeJob(eventId)); + await processReceiverEvent(fakeJob(eventId)); // must not throw or double-ingest + const event = await receiversService.getEventWithReceiver(eventId); + expect(event!.status).toBe('processed'); + }); +}); diff --git a/packages/backend/src/tests/modules/receivers/routes.test.ts b/packages/backend/src/tests/modules/receivers/routes.test.ts new file mode 100644 index 00000000..bd98ee4a --- /dev/null +++ b/packages/backend/src/tests/modules/receivers/routes.test.ts @@ -0,0 +1,135 @@ +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import Fastify, { FastifyInstance } from 'fastify'; +import crypto from 'crypto'; +import { db } from '../../../database/index.js'; +import { receiversRoutes } from '../../../modules/receivers/routes.js'; +import { contextPlugin } from '../../../context/index.js'; +import { createTestContext } from '../../helpers/factories.js'; +import { receiversService } from '../../../modules/receivers/service.js'; + +async function createTestSession(userId: string) { + const token = crypto.randomBytes(32).toString('hex'); + await db + .insertInto('sessions') + .values({ user_id: userId, token, expires_at: new Date(Date.now() + 24 * 60 * 60 * 1000) }) + .execute(); + return token; +} + +describe('receivers management routes', () => { + let app: FastifyInstance; + let projectId: string; + let userId: string; + let token: string; + + beforeAll(async () => { + app = Fastify(); + await app.register(contextPlugin); + await app.register(receiversRoutes, { prefix: '/api/v1/projects' }); + await app.ready(); + }); + + afterAll(async () => { + await app.close(); + }); + + beforeEach(async () => { + await db.deleteFrom('receiver_events').execute(); + await db.deleteFrom('receivers').execute(); + await db.deleteFrom('sessions').execute(); + await db.deleteFrom('api_keys').execute(); + await db.deleteFrom('organization_members').execute(); + await db.deleteFrom('projects').execute(); + await db.deleteFrom('organizations').execute(); + await db.deleteFrom('users').execute(); + const ctx = await createTestContext(); + projectId = ctx.project.id; + userId = ctx.user.id; + token = await createTestSession(userId); + }); + + it('creates a receiver and returns token and ingest path once', async () => { + const res = await app.inject({ + method: 'POST', + url: `/api/v1/projects/${projectId}/receivers`, + headers: { Authorization: `Bearer ${token}` }, + payload: { name: 'ci', adapterType: 'github' }, + }); + expect(res.statusCode).toBe(201); + const body = JSON.parse(res.body); + expect(body.token).toMatch(/^lr_[0-9a-f]{64}$/); + expect(body.ingestPath).toBe(`/api/v1/receivers/${body.id}/${body.token}`); + }); + + it('rejects fieldMapping on non-generic adapters and validates mapping shape', async () => { + const bad = await app.inject({ + method: 'POST', + url: `/api/v1/projects/${projectId}/receivers`, + headers: { Authorization: `Bearer ${token}` }, + payload: { name: 'x', adapterType: 'github', fieldMapping: { message: 'a' } }, + }); + expect(bad.statusCode).toBe(400); + + const badMapping = await app.inject({ + method: 'POST', + url: `/api/v1/projects/${projectId}/receivers`, + headers: { Authorization: `Bearer ${token}` }, + payload: { name: 'x', adapterType: 'generic', fieldMapping: { nope: 'a' } }, + }); + expect(badMapping.statusCode).toBe(400); + }); + + it('lists receivers without exposing token hashes', async () => { + await receiversService.createReceiver({ projectId, name: 'a', adapterType: 'generic' }); + const res = await app.inject({ + method: 'GET', + url: `/api/v1/projects/${projectId}/receivers`, + headers: { Authorization: `Bearer ${token}` }, + }); + expect(res.statusCode).toBe(200); + const body = JSON.parse(res.body); + expect(body.receivers).toHaveLength(1); + expect(res.body).not.toContain('token'); + }); + + it('updates and deletes receivers', async () => { + const { id } = await receiversService.createReceiver({ projectId, name: 'a', adapterType: 'generic' }); + + const patch = await app.inject({ + method: 'PATCH', + url: `/api/v1/projects/${projectId}/receivers/${id}`, + headers: { Authorization: `Bearer ${token}` }, + payload: { enabled: false }, + }); + expect(patch.statusCode).toBe(200); + + const del = await app.inject({ + method: 'DELETE', + url: `/api/v1/projects/${projectId}/receivers/${id}`, + headers: { Authorization: `Bearer ${token}` }, + }); + expect(del.statusCode).toBe(204); + }); + + it('lists recent events', async () => { + const { id } = await receiversService.createReceiver({ projectId, name: 'a', adapterType: 'generic' }); + await receiversService.recordEvent(id, { hello: 1 }); + const res = await app.inject({ + method: 'GET', + url: `/api/v1/projects/${projectId}/receivers/${id}/events`, + headers: { Authorization: `Bearer ${token}` }, + }); + expect(res.statusCode).toBe(200); + expect(JSON.parse(res.body).events).toHaveLength(1); + }); + + it('denies access to projects of other users (404)', async () => { + const other = await createTestContext(); + const res = await app.inject({ + method: 'GET', + url: `/api/v1/projects/${other.project.id}/receivers`, + headers: { Authorization: `Bearer ${token}` }, + }); + expect(res.statusCode).toBe(404); + }); +}); diff --git a/packages/backend/src/tests/modules/receivers/service.test.ts b/packages/backend/src/tests/modules/receivers/service.test.ts new file mode 100644 index 00000000..f0060659 --- /dev/null +++ b/packages/backend/src/tests/modules/receivers/service.test.ts @@ -0,0 +1,120 @@ +import { describe, it, expect, beforeEach } from 'vitest'; +import { db } from '../../../database/index.js'; +import { receiversService } from '../../../modules/receivers/service.js'; +import { createTestContext } from '../../helpers/factories.js'; + +describe('ReceiversService', () => { + let projectId: string; + let orgId: string; + + beforeEach(async () => { + await db.deleteFrom('receiver_events').execute(); + await db.deleteFrom('receivers').execute(); + await db.deleteFrom('api_keys').execute(); + await db.deleteFrom('organization_members').execute(); + await db.deleteFrom('projects').execute(); + await db.deleteFrom('organizations').execute(); + await db.deleteFrom('users').execute(); + const ctx = await createTestContext(); + projectId = ctx.project.id; + orgId = ctx.organization.id; + }); + + it('creates a receiver and returns the plaintext token once', async () => { + const { id, token } = await receiversService.createReceiver({ + projectId, + name: 'ci events', + adapterType: 'github', + }); + expect(token).toMatch(/^lr_[0-9a-f]{64}$/); + + const list = await receiversService.listReceivers(projectId); + expect(list).toHaveLength(1); + expect(list[0].id).toBe(id); + expect(list[0].adapterType).toBe('github'); + expect(list[0].enabled).toBe(true); + // plaintext token never stored + expect(JSON.stringify(list[0])).not.toContain(token); + }); + + it('verifies tokens with tokenMatches and rejects wrong tokens', async () => { + const { id, token } = await receiversService.createReceiver({ + projectId, + name: 'x', + adapterType: 'generic', + }); + const row = await receiversService.getReceiverForIngest(id); + expect(row).not.toBeNull(); + expect(row!.organizationId).toBe(orgId); + expect(receiversService.tokenMatches(token, row!.tokenHash)).toBe(true); + expect(receiversService.tokenMatches('lr_wrong', row!.tokenHash)).toBe(false); + }); + + it('updates and deletes only within the owning project', async () => { + const { id } = await receiversService.createReceiver({ + projectId, + name: 'x', + adapterType: 'generic', + }); + const other = await createTestContext(); + + expect(await receiversService.updateReceiver(id, other.project.id, { enabled: false })).toBe(false); + expect(await receiversService.updateReceiver(id, projectId, { enabled: false, name: 'y' })).toBe(true); + const list = await receiversService.listReceivers(projectId); + expect(list[0].enabled).toBe(false); + expect(list[0].name).toBe('y'); + + expect(await receiversService.deleteReceiver(id, other.project.id)).toBe(false); + expect(await receiversService.deleteReceiver(id, projectId)).toBe(true); + expect(await receiversService.listReceivers(projectId)).toHaveLength(0); + }); + + it('counts receivers across the org', async () => { + await receiversService.createReceiver({ projectId, name: 'a', adapterType: 'generic' }); + await receiversService.createReceiver({ projectId, name: 'b', adapterType: 'github' }); + expect(await receiversService.countReceiversForOrg(orgId)).toBe(2); + }); + + it('records, completes, lists and prunes events', async () => { + const { id } = await receiversService.createReceiver({ + projectId, + name: 'x', + adapterType: 'generic', + }); + + const eventId = await receiversService.recordEvent(id, { hello: 'world' }); + let joined = await receiversService.getEventWithReceiver(eventId); + expect(joined).not.toBeNull(); + expect(joined!.status).toBe('pending'); + expect(joined!.projectId).toBe(projectId); + + await receiversService.completeEvent(eventId, { + status: 'processed', + normalized: [{ time: new Date().toISOString(), service: 's', level: 'info', message: 'm' }], + }); + joined = await receiversService.getEventWithReceiver(eventId); + expect(joined!.status).toBe('processed'); + + const events = await receiversService.listEvents(id); + expect(events).toHaveLength(1); + expect((events[0].rawPayload as any).hello).toBe('world'); + + // prune: insert 5 more, keep 3 + for (let i = 0; i < 5; i++) { + await receiversService.recordEvent(id, { n: i }); + } + await receiversService.pruneEvents(id, 3); + expect(await receiversService.listEvents(id)).toHaveLength(3); + }); + + it('touches last_received_at', async () => { + const { id } = await receiversService.createReceiver({ + projectId, + name: 'x', + adapterType: 'generic', + }); + await receiversService.touchLastReceived(id); + const list = await receiversService.listReceivers(projectId); + expect(list[0].lastReceivedAt).not.toBeNull(); + }); +}); diff --git a/packages/backend/src/tests/modules/receivers/uptime-adapter.test.ts b/packages/backend/src/tests/modules/receivers/uptime-adapter.test.ts new file mode 100644 index 00000000..75c8de16 --- /dev/null +++ b/packages/backend/src/tests/modules/receivers/uptime-adapter.test.ts @@ -0,0 +1,93 @@ +import { describe, it, expect } from 'vitest'; +import { uptimeAdapter } from '../../../modules/receivers/adapters/uptime.js'; +import type { AdapterReceiverInfo } from '../../../modules/receivers/adapters/types.js'; + +const receiver: AdapterReceiverInfo = { + id: 'r-1', + name: 'uptime', + adapterType: 'uptime', + fieldMapping: null, +}; + +describe('uptimeAdapter - Uptime Robot', () => { + it('maps a down alert (alertType 1) to error', () => { + const result = uptimeAdapter( + { + monitorID: '777', + monitorFriendlyName: 'API prod', + monitorURL: 'https://api.example.com', + alertType: '1', + alertTypeFriendlyName: 'Down', + alertDetails: 'Connection Timeout', + }, + receiver + ); + if (result.kind !== 'logs') throw new Error('expected logs'); + const log = result.logs[0]; + expect(log.level).toBe('error'); + expect(log.service).toBe('API prod'); + expect(log.message).toBe('Monitor API prod is DOWN: Connection Timeout'); + expect((log.metadata as any).monitor_url).toBe('https://api.example.com'); + }); + + it('maps an up alert (alertType 2) to info and ssl expiry (3) to warn', () => { + const up = uptimeAdapter( + { monitorFriendlyName: 'API prod', alertType: 2, alertTypeFriendlyName: 'Up' }, + receiver + ); + const ssl = uptimeAdapter( + { monitorFriendlyName: 'API prod', alertType: 3, alertTypeFriendlyName: 'SSL expiry' }, + receiver + ); + if (up.kind !== 'logs' || ssl.kind !== 'logs') throw new Error('expected logs'); + expect(up.logs[0].level).toBe('info'); + expect(up.logs[0].message).toBe('Monitor API prod is UP'); + expect(ssl.logs[0].level).toBe('warn'); + }); +}); + +describe('uptimeAdapter - Better Stack', () => { + it('maps a started incident to error', () => { + const result = uptimeAdapter( + { + data: { + id: 'inc-1', + type: 'incident', + attributes: { + name: 'API prod', + cause: 'Status 500', + status: 'Started', + started_at: '2026-07-01T10:00:00Z', + url: 'https://api.example.com', + }, + }, + }, + receiver + ); + if (result.kind !== 'logs') throw new Error('expected logs'); + expect(result.logs[0].level).toBe('error'); + expect(result.logs[0].service).toBe('API prod'); + expect(result.logs[0].message).toBe('Incident started: Status 500'); + }); + + it('maps a resolved incident to info', () => { + const result = uptimeAdapter( + { + data: { + type: 'incident', + attributes: { name: 'API prod', cause: 'Status 500', status: 'Resolved' }, + }, + }, + receiver + ); + if (result.kind !== 'logs') throw new Error('expected logs'); + expect(result.logs[0].level).toBe('info'); + expect(result.logs[0].message).toBe('Incident resolved: Status 500'); + }); +}); + +describe('uptimeAdapter - unknown shapes', () => { + it('skips unrecognized payloads', () => { + expect(uptimeAdapter({ hello: 'world' }, receiver).kind).toBe('skipped'); + }); +}); diff --git a/packages/backend/src/worker.ts b/packages/backend/src/worker.ts index 9daf3dee..e7efc061 100644 --- a/packages/backend/src/worker.ts +++ b/packages/backend/src/worker.ts @@ -17,6 +17,7 @@ import { processDigestGeneration } from './queue/jobs/digest-generation.js'; import type { DigestJobPayload } from './modules/digests/scheduler.js'; import { processWebhookDelivery } from './queue/jobs/webhook-delivery.js'; import type { WebhookDeliveryJobData } from './modules/webhooks/types.js'; +import { processReceiverEvent, type ReceiverEventJobData } from './queue/jobs/receiver-event.js'; import { alertsService } from './modules/alerts/index.js'; import { monitorService } from './modules/monitoring/index.js'; import { maintenanceService } from './modules/maintenances/service.js'; @@ -103,6 +104,11 @@ const webhookDeliveryWorker = createWorker('webhook-deli await processWebhookDelivery(job); }); +// Create worker for inbound receiver events (#155) +const receiverEventsWorker = createWorker('receiver-events', async (job) => { + await processReceiverEvent(job); +}); + // Start workers (required for graphile-worker backend, no-op for BullMQ) console.log(`[Worker] Using queue backend: ${getQueueBackend()}`); await startQueueWorkers(); @@ -340,6 +346,17 @@ webhookDeliveryWorker.on('failed', (job, err) => { } }); +receiverEventsWorker.on('failed', (job, err) => { + console.error(`Receiver event job ${job?.id} failed:`, err); + if (isInternalLoggingEnabled()) { + hub.captureLog('error', `Receiver event job failed: ${err.message}`, { + error: { name: err.name, message: err.message, stack: err.stack }, + jobId: job?.id, + eventId: job?.data?.eventId, + }); + } +}); + // Lock to prevent overlapping alert checks (race condition protection) let isCheckingAlerts = false; @@ -896,6 +913,7 @@ async function gracefulShutdown(signal: string) { await pipelineWorker.close(); await digestWorker.close(); await webhookDeliveryWorker.close(); + await receiverEventsWorker.close(); console.log('[Worker] Workers closed'); // Close queue system (Redis/PostgreSQL connections) diff --git a/packages/frontend/src/lib/api/receivers.ts b/packages/frontend/src/lib/api/receivers.ts new file mode 100644 index 00000000..45a2cd51 --- /dev/null +++ b/packages/frontend/src/lib/api/receivers.ts @@ -0,0 +1,121 @@ +import { getApiBaseUrl } from '$lib/config'; +import { getAuthToken } from '$lib/utils/auth'; + +export type ReceiverAdapterType = 'github' | 'uptime' | 'generic'; +export type ReceiverEventStatus = 'pending' | 'processed' | 'skipped' | 'failed'; + +export interface ReceiverFieldMapping { + message?: string; + level?: string; + service?: string; + timestamp?: string; + levelMap?: Record; + defaults?: { level?: string; service?: string }; +} + +export interface Receiver { + id: string; + projectId: string; + name: string; + adapterType: ReceiverAdapterType; + fieldMapping: ReceiverFieldMapping | null; + enabled: boolean; + createdAt: string; + lastReceivedAt: string | null; +} + +export interface ReceiverEvent { + id: string; + receiverId: string; + status: ReceiverEventStatus; + rawPayload: Record; + normalized: unknown | null; + error: string | null; + receivedAt: string; +} + +export interface CreateReceiverInput { + name: string; + adapterType: ReceiverAdapterType; + fieldMapping?: ReceiverFieldMapping | null; +} + +export interface CreateReceiverResponse { + id: string; + token: string; + ingestPath: string; + message: string; +} + +export class ReceiversAPI { + constructor(private getToken: () => string | null) {} + + private async request(path: string, options: RequestInit = {}): Promise { + const token = this.getToken(); + if (!token) { + throw new Error('Not authenticated'); + } + + return fetch(`${getApiBaseUrl()}${path}`, { + ...options, + headers: { + ...(options.body ? { 'Content-Type': 'application/json' } : {}), + Authorization: `Bearer ${token}`, + ...options.headers, + }, + }); + } + + async list(projectId: string): Promise<{ receivers: Receiver[] }> { + const response = await this.request(`/projects/${projectId}/receivers`); + if (!response.ok) { + throw new Error('Failed to fetch receivers'); + } + return response.json(); + } + + async create(projectId: string, input: CreateReceiverInput): Promise { + const response = await this.request(`/projects/${projectId}/receivers`, { + method: 'POST', + body: JSON.stringify(input), + }); + if (!response.ok) { + const error = await response.json(); + throw new Error(error.error || 'Failed to create receiver'); + } + return response.json(); + } + + async update( + projectId: string, + id: string, + patch: { name?: string; enabled?: boolean; fieldMapping?: ReceiverFieldMapping | null } + ): Promise { + const response = await this.request(`/projects/${projectId}/receivers/${id}`, { + method: 'PATCH', + body: JSON.stringify(patch), + }); + if (!response.ok) { + throw new Error('Failed to update receiver'); + } + } + + async delete(projectId: string, id: string): Promise { + const response = await this.request(`/projects/${projectId}/receivers/${id}`, { + method: 'DELETE', + }); + if (!response.ok && response.status !== 204) { + throw new Error('Failed to delete receiver'); + } + } + + async listEvents(projectId: string, id: string, limit = 50): Promise<{ events: ReceiverEvent[] }> { + const response = await this.request(`/projects/${projectId}/receivers/${id}/events?limit=${limit}`); + if (!response.ok) { + throw new Error('Failed to fetch receiver events'); + } + return response.json(); + } +} + +export const receiversAPI = new ReceiversAPI(getAuthToken); diff --git a/packages/frontend/src/lib/components/receivers/CreateReceiverDialog.svelte b/packages/frontend/src/lib/components/receivers/CreateReceiverDialog.svelte new file mode 100644 index 00000000..7ef1f7ef --- /dev/null +++ b/packages/frontend/src/lib/components/receivers/CreateReceiverDialog.svelte @@ -0,0 +1,273 @@ + + + + + {#if !created} + + Create Webhook Receiver + + Receive events from an external system and store them as log entries in this project. + + + +
+
+ + +

+ Used as the default service name for generic events. +

+
+ +
+ +
+ {#each ADAPTER_OPTIONS as option (option.value)} + + {/each} +
+
+ + {#if adapterType === 'generic'} +
+ +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+

+ Unmapped fields fall back to sensible defaults; the full payload is always kept in metadata. +

+
+ {/if} + + {#if error} +
+ {error} +
+ {/if} + + + + + +
+ {:else} + + Receiver Created + Point the external system at this URL to start receiving events. + + +
+ + Important: Save this URL now + + The URL contains the receiver token and is the only credential the external system needs. + This is the only time it will be shown. If you lose it, delete the receiver and create a new one. + + + +
+ +
+
+
+ {ingestUrl} +
+
+ +
+
+ +
+

Usage Example:

+
curl -X POST {ingestUrl} \
+  -H "Content-Type: application/json" \
+  -d '{`{"message": "deploy finished"}`}'
+
+
+ + + + + {/if} +
+
diff --git a/packages/frontend/src/lib/components/receivers/ReceiverEventsDialog.svelte b/packages/frontend/src/lib/components/receivers/ReceiverEventsDialog.svelte new file mode 100644 index 00000000..2311efe0 --- /dev/null +++ b/packages/frontend/src/lib/components/receivers/ReceiverEventsDialog.svelte @@ -0,0 +1,128 @@ + + + + + + Recent Events{receiver ? ` - ${receiver.name}` : ''} + + The last events received by this webhook, with their normalized output. Only the most recent + 100 events are kept. + + + + {#if loading} +
+ + Loading events... +
+ {:else if events.length === 0} +
+

No events received yet

+

Send a test payload to the webhook URL to see it here.

+
+ {:else} +
+ {#each events as event (event.id)} +
+ + {#if expandedId === event.id} +
+
+

Raw Payload

+
{JSON.stringify(event.rawPayload, null, 2)}
+
+ {#if event.normalized} +
+

Normalized Logs

+
{JSON.stringify(event.normalized, null, 2)}
+
+ {/if} + {#if event.error} +
+

Error

+

{event.error}

+
+ {/if} +
+ {/if} +
+ {/each} +
+ {/if} +
+
diff --git a/packages/frontend/src/lib/components/receivers/index.ts b/packages/frontend/src/lib/components/receivers/index.ts new file mode 100644 index 00000000..278b4f5a --- /dev/null +++ b/packages/frontend/src/lib/components/receivers/index.ts @@ -0,0 +1,2 @@ +export { default as CreateReceiverDialog } from './CreateReceiverDialog.svelte'; +export { default as ReceiverEventsDialog } from './ReceiverEventsDialog.svelte'; diff --git a/packages/frontend/src/routes/dashboard/projects/[id]/settings/+page.svelte b/packages/frontend/src/routes/dashboard/projects/[id]/settings/+page.svelte index 65650590..0f8db77a 100644 --- a/packages/frontend/src/routes/dashboard/projects/[id]/settings/+page.svelte +++ b/packages/frontend/src/routes/dashboard/projects/[id]/settings/+page.svelte @@ -18,7 +18,10 @@ import Spinner from '$lib/components/Spinner.svelte'; import * as AlertDialog from '$lib/components/ui/alert-dialog'; import CreateApiKeyDialog from '$lib/components/CreateApiKeyDialog.svelte'; - import { Trash2, Plus } from '@lucide/svelte/icons'; + import { CreateReceiverDialog, ReceiverEventsDialog } from '$lib/components/receivers'; + import Switch from '$lib/components/ui/switch/switch.svelte'; + import { receiversAPI, type Receiver, type ReceiverAdapterType, type ReceiverFieldMapping } from '$lib/api/receivers'; + import { Trash2, Plus, ScrollText } from '@lucide/svelte/icons'; let project = $state(null); let loading = $state(false); @@ -36,6 +39,14 @@ let apiKeyToDelete = $state(null); let lastLoadedApiKeysKey = $state(null); + let receivers = $state([]); + let loadingReceivers = $state(false); + let showCreateReceiverDialog = $state(false); + let receiverToDelete = $state(null); + let eventsReceiver = $state(null); + let showEventsDialog = $state(false); + let lastLoadedReceiversKey = $state(null); + const projectId = $derived(page.params.id); async function loadProject() { @@ -92,6 +103,18 @@ loadApiKeys(); }); + $effect(() => { + if (!browser || !projectId) { + receivers = []; + lastLoadedReceiversKey = null; + return; + } + + if (projectId === lastLoadedReceiversKey) return; + + loadReceivers(); + }); + async function handleSave() { if (!$currentOrganization || !projectId) return; @@ -169,6 +192,73 @@ } } + async function loadReceivers() { + if (!projectId) return; + + loadingReceivers = true; + + try { + const response = await receiversAPI.list(projectId); + receivers = response.receivers; + lastLoadedReceiversKey = projectId; + } catch (e) { + toastStore.error(e instanceof Error ? e.message : 'Failed to load receivers'); + } finally { + loadingReceivers = false; + } + } + + async function handleCreateReceiver(data: { + name: string; + adapterType: ReceiverAdapterType; + fieldMapping?: ReceiverFieldMapping | null; + }) { + if (!projectId) throw new Error('No project ID'); + + const response = await receiversAPI.create(projectId, data); + toastStore.success('Receiver created successfully'); + await loadReceivers(); + return response; + } + + async function handleToggleReceiver(receiver: Receiver, enabled: boolean) { + if (!projectId) return; + + try { + await receiversAPI.update(projectId, receiver.id, { enabled }); + toastStore.success(enabled ? 'Receiver enabled' : 'Receiver disabled'); + await loadReceivers(); + } catch (e) { + toastStore.error(e instanceof Error ? e.message : 'Failed to update receiver'); + await loadReceivers(); + } + } + + async function handleDeleteReceiver() { + if (!projectId || !receiverToDelete) return; + + try { + await receiversAPI.delete(projectId, receiverToDelete.id); + toastStore.success('Receiver deleted successfully'); + await loadReceivers(); + } catch (e) { + toastStore.error(e instanceof Error ? e.message : 'Failed to delete receiver'); + } finally { + receiverToDelete = null; + } + } + + function openReceiverEvents(receiver: Receiver) { + eventsReceiver = receiver; + showEventsDialog = true; + } + + function adapterLabel(type: ReceiverAdapterType): string { + if (type === 'github') return 'GitHub'; + if (type === 'uptime') return 'Uptime'; + return 'Generic JSON'; + } + function formatDate(date: string | null): string { if (!date) return 'Never'; return new Date(date).toLocaleString('en-US'); @@ -360,6 +450,91 @@ + + +
+
+ Webhook Receivers + + Receive events from external systems (CI/CD, uptime monitors) as log entries + +
+ +
+
+ + {#if loadingReceivers} +
+ + Loading receivers... +
+ {:else if receivers.length === 0} +
+

No receivers yet

+

Create a receiver to ingest events from external tools

+
+ {:else} +
+ + + + + + + + + + + + {#each receivers as receiver (receiver.id)} + + + + + + + + {/each} + +
NameSourceEnabledLast ReceivedActions
{receiver.name} + + {adapterLabel(receiver.adapterType)} + + + handleToggleReceiver(receiver, checked)} + /> + + {formatDate(receiver.lastReceivedAt)} + + + +
+
+ {/if} +
+
+ Danger Zone @@ -384,6 +559,30 @@ + + + + + !open && (receiverToDelete = null)} +> + + + Delete Receiver + + Are you sure you want to delete the receiver {receiverToDelete?.name}? + Its webhook URL will stop working immediately and cannot be restored. This action cannot + be undone. + + + + Cancel + Delete Receiver + + + + !open && (apiKeyToDelete = null)} diff --git a/packages/shared/src/constants/index.ts b/packages/shared/src/constants/index.ts index 69883500..f6e26b46 100644 --- a/packages/shared/src/constants/index.ts +++ b/packages/shared/src/constants/index.ts @@ -7,3 +7,4 @@ export * from './siem-constants.js'; export * from './exception-constants.js'; export * from './mitre-constants.js'; export * from './monitoring-constants.js'; +export * from './receiver-constants.js'; diff --git a/packages/shared/src/constants/receiver-constants.ts b/packages/shared/src/constants/receiver-constants.ts new file mode 100644 index 00000000..2fa66b6e --- /dev/null +++ b/packages/shared/src/constants/receiver-constants.ts @@ -0,0 +1,2 @@ +export const RECEIVER_ADAPTER_TYPES = ['github', 'uptime', 'generic'] as const; +export type ReceiverAdapterType = (typeof RECEIVER_ADAPTER_TYPES)[number]; diff --git a/packages/shared/src/schemas/index.ts b/packages/shared/src/schemas/index.ts index 9edd8f51..56d849ab 100644 --- a/packages/shared/src/schemas/index.ts +++ b/packages/shared/src/schemas/index.ts @@ -71,3 +71,4 @@ export type AlertRuleInput = z.infer; export * from './metadata-filter.js'; export * from './webhook-events.js'; +export * from './receiver.js'; diff --git a/packages/shared/src/schemas/receiver.test.ts b/packages/shared/src/schemas/receiver.test.ts new file mode 100644 index 00000000..aeeaad07 --- /dev/null +++ b/packages/shared/src/schemas/receiver.test.ts @@ -0,0 +1,33 @@ +import { describe, it, expect } from 'vitest'; +import { receiverFieldMappingSchema, RECEIVER_ADAPTER_TYPES } from '../index.js'; + +describe('receiverFieldMappingSchema', () => { + it('accepts a full valid mapping', () => { + const result = receiverFieldMappingSchema.safeParse({ + message: 'msg.text', + level: 'severity', + service: 'source.app', + timestamp: 'ts', + levelMap: { crit: 'critical', warning: 'warn' }, + defaults: { level: 'info', service: 'external' }, + }); + expect(result.success).toBe(true); + }); + + it('accepts an empty mapping', () => { + expect(receiverFieldMappingSchema.safeParse({}).success).toBe(true); + }); + + it('rejects unknown keys', () => { + expect(receiverFieldMappingSchema.safeParse({ nope: 'x' }).success).toBe(false); + }); + + it('rejects invalid levels in levelMap and defaults', () => { + expect(receiverFieldMappingSchema.safeParse({ levelMap: { a: 'verbose' } }).success).toBe(false); + expect(receiverFieldMappingSchema.safeParse({ defaults: { level: 'verbose' } }).success).toBe(false); + }); + + it('exposes the adapter type list', () => { + expect(RECEIVER_ADAPTER_TYPES).toEqual(['github', 'uptime', 'generic']); + }); +}); diff --git a/packages/shared/src/schemas/receiver.ts b/packages/shared/src/schemas/receiver.ts new file mode 100644 index 00000000..4702cecc --- /dev/null +++ b/packages/shared/src/schemas/receiver.ts @@ -0,0 +1,29 @@ +import { z } from 'zod'; +import { LOG_LEVELS, RECEIVER_ADAPTER_TYPES } from '../constants/index.js'; + +export const receiverAdapterTypeSchema = z.enum(RECEIVER_ADAPTER_TYPES); + +const dotPath = z.string().min(1).max(200); + +/** + * Field mapping for the 'generic' receiver adapter (#155). + * Values are dot-paths into the raw payload (e.g. "error.message"). + */ +export const receiverFieldMappingSchema = z + .object({ + message: dotPath.optional(), + level: dotPath.optional(), + service: dotPath.optional(), + timestamp: dotPath.optional(), + levelMap: z.record(z.enum(LOG_LEVELS)).optional(), + defaults: z + .object({ + level: z.enum(LOG_LEVELS).optional(), + service: z.string().min(1).max(100).optional(), + }) + .strict() + .optional(), + }) + .strict(); + +export type ReceiverFieldMapping = z.infer;