Commit 5db62b8
authored
feat(observability): extend audit log, PostHog, and storage metering coverage (#5269)
* feat(observability): extend audit log, PostHog, and storage metering coverage
Instruments previously-uncaptured resources and actions across the three
observability layers (audit log, PostHog product analytics, usage metering):
- Exfiltration audit: file downloads (workspace/public-share/v1), table,
workflow, and workspace exports.
- Credential access audited at the token-issuance boundary (success-only).
- Full revenue trail: invoice paid/failed, overage billed, disputes, credit
fulfillment, subscription lifecycle, plan/seat changes.
- Session/login lifecycle audit via Better Auth hooks (login, blocked sign-in,
logout, session revoke, account delete).
- v1/admin programmatic surface and copilot tool handlers instrumented.
- Dead audit/PostHog constants wired (lock/unlock, table, custom tool, etc.).
- Storage metering extended to KB documents and copilot files.
- PostHog person identify + workspace/organization group hygiene.
Audit package hardened to null the actor FK for system actors (admin-api) and
adds an awaitable recordAuditNow for pre-delete hooks. All instrumentation is
fire-and-forget and never blocks or breaks the primary operation.
* fix(observability): audit file/credential egress only on success
Address Cursor Bugbot review:
- GET OAuth token: emit CREDENTIAL_ACCESSED/credential_used after
refreshTokenIfNeeded succeeds (matches POST), not before.
- File export: audit on each actual success exit via a shared helper —
including the non-markdown serve redirect (previously unaudited) and
after the zip is generated (previously before asset fetch).
* fix(audit): record null actor for anonymous public-share downloads
Address Greptile P1: setting actorId to the file owner made every anonymous
external download read as a self-download, undermining the exfiltration trail.
recordAudit now accepts a null actor; the public content/inline routes record
actorId=null with the owner in metadata.sharedByUserId and rely on ip/user-agent
for the forensic trail. The misleading owner-attributed file_downloaded PostHog
event is dropped on these anonymous paths.
* fix(observability): audit admin exports after zip; tidy comments
- Admin workflow/workspace ZIP exports now audit only after the archive is
built (via a local helper), so a zip/build failure no longer logs an export.
- Remove redundant 'success-only placement' inline comments and tighten the
remaining design-rationale notes (export helper now TSDoc).
* fix(billing): complete the chargeback + overage financial trail
Address Cursor review:
- handleDisputeClosed now records CHARGE_DISPUTE_CLOSED for every closed
dispute (won/lost/warning_closed), unblocking only on favorable outcomes.
dispute.status in the metadata distinguishes the outcome, so lost
chargebacks are no longer missing from the trail.
- Threshold overage now emits OVERAGE_BILLED + overage_billed even when credits
fully cover the overage (settledVia: 'credits' vs 'stripe'), so credit-settled
overages are audited instead of silently returning null.
* fix(storage): decrement copilot quota before deleting metadata
Address Greptile P1: deleting the metadata row before the decrement meant a
decrement failure left the quota permanently inflated with no record to retry
from. Decrement first; only remove the metadata row once it succeeds.
* fix(observability): atomic copilot storage release; signup-blocked action
- Copilot file delete now releases storage via a single transaction
(releaseDeletedFileStorage): the soft-delete is the idempotency claim and the
decrement shares the transaction, so neither a partial failure (inflated
counter) nor a retry (double-decrement) can desync the quota. Resolves the
conflicting Greptile/Cursor ordering findings.
- Policy-blocked sign-ups now record USER_SIGNUP_BLOCKED instead of
USER_SIGNIN_BLOCKED, so account-lifecycle events aren't mislabeled.
* fix(storage): meter copilot ingest centrally for path symmetry
Address Cursor review: only uploadCopilotFile incremented storage, but copilot
files also enter via the generic upload route and presigned uploads — all of
which persist metadata through insertFileMetadata. Move the increment into
insertFileMetadata (scoped to context='copilot', on genuine insert/restore) so
every ingest path is symmetric with the delete-time decrement, and drop the now
redundant per-path increment in uploadCopilotFile. Other contexts are metered by
their own managers and remain unaffected.
* fix(audit): skip WORKFLOW_EXPORTED when admin export is empty
Address Cursor review: when every requested workflow fails to load, the admin
export still returned 200 but recorded a successful export of zero workflows.
Guard auditExport so an empty result records nothing.
* fix(storage): settle copilot accounting before deleting the blob
Address Cursor review: the blob was removed before releaseDeletedFileStorage, so
a release failure left the counter inflated and the metadata active with the blob
gone. Now the atomic soft-delete + decrement runs first and the blob is deleted
only if it succeeds, so a failure leaves the file fully intact and retryable.
* fix(storage): decrement KB document storage atomically with deletion
Address Cursor review: hardDeleteDocuments deleted the rows then decremented
best-effort, so a decrement failure left billed storage inflated with no row to
reconcile. Resolve each owner's subscription up front, then decrement inside the
same transaction that deletes the embeddings/documents (decrementStorageUsageInTx,
also now shared by releaseDeletedFileStorage), so the counter and the content
commit or roll back together. Connector docs remain excluded.
* fix(audit): don't treat email verification as a login
Address Cursor review: /verify-email could emit USER_LOGIN when verification
mints or refreshes a session, mislabeling (or double-counting) a non-sign-in as
a login. Restrict isLoginPath to genuine sign-in entrypoints.
* fix(audit): null actor FK when the user lookup throws
Address Greptile P1: the catch branch left the original actorId, so a system
actor like 'admin-api' (or a since-deleted user) would FK-violate the insert and
lose the row when the existence lookup errored. Mirror the not-found branch —
null the FK with a readable label — so the audit row always persists.
* revert(audit): drop session/account-lifecycle auth instrumentation
Remove the Better Auth login/logout/session-revoke/account-delete/blocked-signin
audit hooks from auth.ts — they touch sensitive auth paths and are noisy. Also
removes the now-unused taxonomy (USER_LOGIN/_FAILED, USER_SIGNIN_BLOCKED,
USER_SIGNUP_BLOCKED, USER_LOGOUT, SESSION_REVOKED, ACCOUNT_DELETED,
ACCOUNT_EMAIL_CHANGED actions; SESSION/USER resource types) and the awaitable
recordAuditNow helper that only the pre-delete hook used. auth.ts is back to the
staging baseline.
* fix(storage): harden copilot+KB delete accounting against read errors and concurrency
Final-audit follow-ups:
- deleteCopilotFile: a failed metadata *read* (vs a genuine missing row) now
blocks the blob delete too, so a transient read error can't leave an active
row un-decremented with the blob gone.
- hardDeleteDocuments: drive the per-user decrement from the delete's
returning() (the rows this tx actually removed), so two concurrent deletes of
the same ids can't both decrement.
* chore(observability): drop two unused definitions
Final-audit cleanup: remove the orphaned knowledge_base_searched PostHog event
(never wired; KB search analytics already flow through the OpenTelemetry channel)
and the redundant AuditAction.SUBSCRIPTION_UPDATED (subscription/plan changes are
audited via ORG_PLAN_CONVERTED). Every remaining new action/event has a real
emit site.
* fix(knowledge): key hard-delete result off rows actually deleted
Address Cursor review: hardDeleteDocuments returned existingIds.length (the
requested count) and cleaned storage for the full pre-tx set, even though the
decrement was driven by the rows the transaction actually deleted. Under a
concurrent delete that claimed some ids first, that overstated the result and
re-touched storage for rows this call didn't delete. Drive the storage cleanup,
log, and return value from deletedDocs (the returning() rows) so all four are
consistent.
* fix(storage): gate copilot quota on all ingest paths; tidy inline comments
- Copilot uploads via the generic /api/files/upload route and presigned URLs now
run checkStorageQuota before writing (matching uploadCopilotFile), so no copilot
ingest path can grow usage past the plan limit. The central increment stays in
insertFileMetadata.
- Trim/remove redundant inline comments across the diff; keep only concise notes
on non-obvious decisions (left pre-existing comments untouched).
* fix(analytics): omit empty workspace_id on workspace-less file downloads
Address Greptile P1: the generic key-based /api/files/download route and the
no-workspace markdown-export path emitted file_downloaded with workspace_id:''
creating a phantom '' bucket in PostHog. Make workspace_id optional on the event
and omit it when there is no workspace (workspace-scoped routes still pass it).
* fix(analytics): omit empty workspace_id/workflow_id on copilot_chat_sent
Final-sweep nit: copilot_chat_sent sent '' for workspace_id/workflow_id in the
agent (workspace-less) branch, same phantom-bucket issue as file_downloaded.
Make both properties optional and omit them when absent.
* fix(analytics): clear stale org PostHog group on personal workspaces
Address Cursor review: switching from a team workspace to a personal one left
the previous organization group set, so later events kept rolling up under it.
When the active workspace has no organizationId, resetGroups() to drop the stale
org group, then re-apply the workspace group.
* fix: address review — export audit timing, copilot delete signal, group reset
- Table export (Cursor MED): audit fires before streaming begins, not after
controller.close(), so a mid-stream failure still records the partial export.
- deleteCopilotFile (Greptile P1): throw when storage accounting can't be settled
instead of silently returning, so callers can detect the file was not deleted.
- workspace-scope-sync (Cursor MED): only resetGroups() once workspace metadata is
loaded (activeWorkspace present), so a team workspace doesn't transiently lose
its org group while organizationId is still null during load.
* refactor(observability): final line-justification cleanup
Address final audit flags:
- Table import: move the 'columns added' audit OUT of the import transaction
into a post-commit auditTableColumnsAdded() helper called by the three
tx-owning callers, so a mid-import row-batch rollback no longer logs a false
'added N columns' (matches the PR's success-only discipline).
- Omit empty-string analytics dimensions on workflow_lock_toggled (workspace_id)
and organization_created (name), consistent with file_downloaded/copilot_chat_sent.
- Restore an unrelated capacity-check comment removed incidentally.
- Move copilot_chat_sent emit above the traceparent comment so the comment sits
with the code it documents.
* fix(table): attribute import column audit to the importing user
Address Cursor review: addImportColumns (async createColumns import path) now
threads the importing userId into auditTableColumnsAdded instead of falling back
to table.createdBy, so column additions are attributed to the actual member who
ran the import rather than the table creator.
* feat(observability): drop copilot from storage accounting
Per design decision: copilot files are working/conversational artifacts, so
gating their materialization on storage quota would fail an agent operation
mid-flow when a user is over limit, and metering them would inflate usage enough
to block KB/workspace uploads indirectly. Remove copilot quota gates + copilot
ingest metering entirely (revert copilot-file-manager, metadata, and the upload
route's copilot branch to baseline) and drop the now-unused releaseDeletedFileStorage.
KB document metering + quota enforcement (the deliberate, persistent storage path)
is unchanged.
* fix(analytics): switch workspace + org PostHog groups together
Address Cursor review: gate the group-sync effect on workspace metadata being
loaded (activeWorkspace present) so the workspace and organization groups always
update atomically. Acting during the load window paired the new workspace group
with the previous workspace's org group; until metadata loads, events stay
consistently attributed to the previous workspace.
* fix(table): audit async export at authorization, not job completion
Address Cursor review: async exports only emitted TABLE_EXPORTED when the
background job reached 'ready', so an authorized export whose job later failed or
was abandoned left no audit trail — inconsistent with the sync export route,
which audits before streaming. Move the audit + analytics to the async route's
authorization point (after the job is claimed/dispatched) and remove it from the
runner. Drop the now-unused userId from TableExportPayload.
* fix(billing): never let payment_failed instrumentation skip user blocking
Address Greptile P1: the payment_failed audit hoisted an unguarded
isSubscriptionOrgScoped DB read (for entity_type) directly before the
attempt-count user-blocking block. A transient failure of that read would throw
out of the handler and skip blocking. Wrap the whole audit/analytics block in
try/catch (best-effort), and let the blocking compute its own isSubscriptionOrgScoped
as it did originally — instrumentation can no longer abort payment processing.
* chore(observability): trim verbose inline comments to concise notes
* fix(billing): wrap new dispute/enterprise/subscription instrumentation in Stripe webhook idempotency
recordAudit/captureServerEvent calls added for charge disputes, enterprise
subscription provisioning, and free->paid subscription creation ran
unconditionally with no idempotency guard, unlike their sibling handlers
in the same files. Stripe redelivers webhooks at-least-once, so a retry
would double-record the audit row and PostHog event even though the
underlying DB writes were already idempotent.
* fix(observability): tag org-scoped audit metadata with organizationId, guard concurrent table delete
The org-scoped self-service audit-log endpoint matches org-level rows via
metadata.organizationId (or resourceType=organization). Six recordAudit
call sites (subscription create/cancel, admin credit issuance, threshold
overage billing, charge disputes, credit purchase, invoice payment
succeeded/failed) tagged org-scoped events with a differently-named key
(referenceId/entityId/targetOrgId), making them invisible to org admins
querying their own audit trail despite being stored in the DB. Add the
missing organizationId key everywhere the pattern was missed.
Also guard deleteTable's archive UPDATE with isNull(archivedAt) so a
concurrent duplicate delete request is a no-op instead of re-archiving
and re-firing a duplicate TABLE_DELETED audit row.
Adds test coverage for the ORG_MEMBER_ADDED audit/analytics emission in
acceptInvitation, which previously had none.
* fix(test): queue resolveBillingActorId's owner lookup in payment-failure email test
handleInvoicePaymentFailed's new payment_failed audit instrumentation
resolves the billing actor via an extra db.select before
sendPaymentFailureEmails runs. The test's fixed select-response queue
didn't account for it, so the org-admin lookup consumed the wrong
queued row and the assertion saw zero email sends. Production behavior
is unaffected — each query is independent; this was a mock-queue
ordering issue only.
* fix(billing): don't let a post-commit usage-limit sync failure suppress the seat audit
Cursor Bugbot flagged: reconcileOrganizationSeats committed the seat
change and Stripe outbox enqueue in a transaction, then called
syncSubscriptionUsageLimits outside it before recording the audit/
PostHog events. A thrown sync left a genuinely-changed seat count with
no ORG_SEAT_PROVISIONED/DEPROVISIONED trail. Wrap the sync in its own
try/catch (log + continue) so the events always fire for a committed
change, matching the fire-and-forget instrumentation pattern used
elsewhere in this PR.
* fix(billing): guard the new subscription-created instrumentation block
Greptile P1: the org-scope check I added for the audit-metadata fix
(isSubscriptionOrgScoped) was a raw DB call with no error guard, unlike
resolveSubscriptionActorId next to it. Since it ran inside the
idempotency lambda with an unconditional rethrow, a transient DB error
would abort and retry the whole webhook after the free -> paid usage
reset had already committed. Wrap the actor/org-scope resolution +
audit + analytics block in try/catch, matching the same guarded
instrumentation pattern already used in handleInvoicePaymentFailed.1 parent fea3391 commit 5db62b8
61 files changed
Lines changed: 2412 additions & 477 deletions
File tree
- apps
- realtime/src/handlers
- sim
- app
- api
- auth/oauth/token
- billing/switch-plan
- environment
- files
- download
- export/[id]
- public/[token]
- content
- inline
- organizations
- [id]/members/[memberId]
- table/[tableId]
- export-async
- export
- users/me
- api-keys
- [id]
- subscription/[id]/transfer
- v1
- admin
- credits
- organizations
- [id]
- members
- [memberId]
- transfer-ownership
- workflows/export
- workspaces/[id]
- export
- members
- [memberId]
- files/[fileId]
- workflows/[id]
- deploy
- workspaces
- [id]/files
- [fileId]/download
- download
- members/[id]
- workspace/[workspaceId]/providers
- lib
- auth
- billing
- organizations
- storage
- webhooks
- copilot
- chat
- tools
- handlers/management
- server/table
- invitations
- knowledge/documents
- posthog
- table
- workflows
- orchestration
- schedules
- packages
- audit/src
- testing/src/mocks
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
1 | 2 | | |
2 | 3 | | |
3 | 4 | | |
| |||
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
21 | 25 | | |
22 | 26 | | |
23 | 27 | | |
| |||
177 | 181 | | |
178 | 182 | | |
179 | 183 | | |
| 184 | + | |
| 185 | + | |
180 | 186 | | |
181 | 187 | | |
182 | 188 | | |
| |||
196 | 202 | | |
197 | 203 | | |
198 | 204 | | |
| 205 | + | |
| 206 | + | |
199 | 207 | | |
200 | 208 | | |
201 | 209 | | |
| |||
231 | 239 | | |
232 | 240 | | |
233 | 241 | | |
234 | | - | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
235 | 247 | | |
236 | 248 | | |
237 | 249 | | |
| |||
294 | 306 | | |
295 | 307 | | |
296 | 308 | | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
297 | 322 | | |
298 | 323 | | |
299 | 324 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
1 | 2 | | |
2 | 3 | | |
3 | 4 | | |
| |||
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
| 15 | + | |
14 | 16 | | |
15 | 17 | | |
16 | 18 | | |
| |||
96 | 98 | | |
97 | 99 | | |
98 | 100 | | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
99 | 119 | | |
100 | 120 | | |
101 | 121 | | |
| |||
120 | 140 | | |
121 | 141 | | |
122 | 142 | | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
123 | 172 | | |
124 | 173 | | |
125 | 174 | | |
| 175 | + | |
126 | 176 | | |
127 | 177 | | |
128 | 178 | | |
| |||
137 | 187 | | |
138 | 188 | | |
139 | 189 | | |
| 190 | + | |
140 | 191 | | |
141 | 192 | | |
142 | 193 | | |
| |||
165 | 216 | | |
166 | 217 | | |
167 | 218 | | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
168 | 222 | | |
169 | 223 | | |
170 | 224 | | |
171 | 225 | | |
172 | 226 | | |
173 | 227 | | |
174 | 228 | | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
175 | 255 | | |
176 | 256 | | |
177 | 257 | | |
| |||
247 | 327 | | |
248 | 328 | | |
249 | 329 | | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
250 | 333 | | |
251 | 334 | | |
252 | 335 | | |
253 | 336 | | |
254 | 337 | | |
255 | 338 | | |
256 | 339 | | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
257 | 366 | | |
258 | 367 | | |
259 | 368 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
1 | 2 | | |
2 | 3 | | |
3 | 4 | | |
| |||
174 | 175 | | |
175 | 176 | | |
176 | 177 | | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
177 | 201 | | |
178 | 202 | | |
179 | 203 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
| |||
89 | 90 | | |
90 | 91 | | |
91 | 92 | | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
92 | 98 | | |
93 | 99 | | |
94 | 100 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
1 | 2 | | |
2 | 3 | | |
3 | 4 | | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| 8 | + | |
7 | 9 | | |
8 | 10 | | |
9 | 11 | | |
| |||
80 | 82 | | |
81 | 83 | | |
82 | 84 | | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
83 | 101 | | |
84 | 102 | | |
85 | 103 | | |
86 | | - | |
| 104 | + | |
87 | 105 | | |
88 | 106 | | |
89 | 107 | | |
| |||
0 commit comments