Skip to content

Commit eac6cc0

Browse files
waleedlatif1claude
andcommitted
docs(data-drains): clarify retention pairing is user-coupled, not enforced
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent ccd1695 commit eac6cc0

1 file changed

Lines changed: 19 additions & 1 deletion

File tree

apps/docs/content/docs/en/enterprise/data-drains.mdx

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ import { FAQ } from '@/components/ui/faq'
77

88
Data Drains let organization owners and admins on Enterprise plans continuously export Sim data to a destination they control — a customer-owned S3 bucket or an HTTPS webhook. A drain runs on a schedule, picks up only new rows since its last successful run, and writes them as NDJSON to the destination. Viewing drain configuration and run history is restricted to owners and admins as well, since destinations expose internal bucket names and webhook URLs.
99

10-
Drains pair naturally with [Data Retention](/enterprise/data-retention): drain into long-term storage first, then let retention safely delete from Sim.
10+
Drains are independent of [Data Retention](/enterprise/data-retention) but designed to compose with it — see [Pairing with Data Retention](#pairing-with-data-retention) below.
1111

1212
---
1313

@@ -124,6 +124,24 @@ The **last 10 runs** for each drain are visible by expanding its row in the sett
124124

125125
---
126126

127+
## Pairing with Data Retention
128+
129+
Drains and [Data Retention](/enterprise/data-retention) are independent modules. Sim does **not** gate retention on drain progress — if a drain is failing, retention will still purge data on its own schedule. This matches the model used by Datadog Archives and AWS CloudWatch + S3 Export: keep the two configurations orthogonal and let the customer pair them deliberately.
130+
131+
To safely use both together, set the drain cadence shorter than the retention period for the same data category:
132+
133+
| Drain source | Pairs with retention setting |
134+
|---|---|
135+
| Workflow logs, Job logs | **Log retention** |
136+
| Copilot chats, Copilot runs | **Task cleanup** |
137+
| Audit logs | *(no retention setting today — audit logs are kept indefinitely)* |
138+
139+
For example, with **Log retention** set to 30 days, set the workflow-logs drain to **Hourly** or **Daily** so every row is exported well before retention purges it from Sim. Monitor recent drain runs in the settings page; if a drain has been failing for longer than your retention window, you may lose rows that retention purges before they are exported.
140+
141+
After data lands in your bucket or webhook system, archive lifecycle (transitions to Glacier, expiration, GDPR right-to-erasure propagation) is governed by your own infrastructure — Sim has no further visibility into that data once delivery succeeds.
142+
143+
---
144+
127145
<FAQ items={[
128146
{
129147
question: "Who can configure data drains?",

0 commit comments

Comments
 (0)