Skip to content

Commit b26c3f6

Browse files
sebastian-ederersebastian-ederer
committed
docs: Add docs for retention policies
1 parent ba15fb7 commit b26c3f6

2 files changed

Lines changed: 204 additions & 0 deletions

File tree

‎docs/data-annotations/retention-policies.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Retention Policies
2+
3+
A retention policy automatically drops old chunks from a hypertable or continuous aggregate on a scheduled basis. This keeps storage consumption bounded without requiring manual intervention and is the standard approach for managing time-series data lifecycle in TimescaleDB.
4+
5+
Each hypertable or continuous aggregate supports at most one retention policy.
6+
7+
[See also: add_retention_policy](https://docs.tigerdata.com/api/latest/data_retention/add_retention_policy/)
8+
9+
## Drop Modes
10+
11+
Two mutually exclusive drop modes are available:
12+
13+
- **`DropAfter`**: Drops chunks whose data falls outside a time window relative to the current time. This is the standard mode.
14+
- **`DropCreatedBefore`**: Drops chunks created before a specified interval ago, regardless of the data they contain.
15+
16+
Exactly one of `DropAfter` or `DropCreatedBefore` must be specified. Providing both or neither raises an exception.
17+
18+
## Basic Example
19+
20+
Here is a complete example of configuring a retention policy on an `ApplicationLog` hypertable using `DropAfter`.
21+
22+
```csharp
23+
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.Hypertable;
24+
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.RetentionPolicy;
25+
using Microsoft.EntityFrameworkCore;
26+
27+
[Hypertable(nameof(Time), ChunkTimeInterval = "1 day")]
28+
[PrimaryKey(nameof(Id), nameof(Time))]
29+
[RetentionPolicy("30 days")]
30+
public class ApplicationLog
31+
{
32+
public Guid Id { get; set; }
33+
public DateTime Time { get; set; }
34+
public string ServiceName { get; set; } = string.Empty;
35+
public string Level { get; set; } = string.Empty;
36+
public string Message { get; set; } = string.Empty;
37+
}
38+
```
39+
40+
## Using `DropCreatedBefore`
41+
42+
Pass `null` as the first argument and provide `dropCreatedBefore` as a named argument:
43+
44+
```csharp
45+
[Hypertable(nameof(Time), ChunkTimeInterval = "1 day")]
46+
[PrimaryKey(nameof(Id), nameof(Time))]
47+
[RetentionPolicy(dropCreatedBefore: "30 days")]
48+
public class ApiRequestLog
49+
{
50+
public Guid Id { get; set; }
51+
public DateTime Time { get; set; }
52+
public string Path { get; set; } = string.Empty;
53+
public int StatusCode { get; set; }
54+
}
55+
```
56+
57+
> :warning: **Note:** Due to a known bug in TimescaleDB ([#9446](https://github.com/timescale/timescaledb/issues/9446)), `alter_job` fails when used with `DropCreatedBefore` policies. The library works around this by skipping the `alter_job` call for `DropCreatedBefore` policies. As a result, job scheduling parameters (`ScheduleInterval`, `MaxRuntime`, `MaxRetries`, `RetryPeriod`) are accepted by the API but have no effect at the database level when `DropCreatedBefore` is used.
58+
59+
## Complete Example
60+
61+
```csharp
62+
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.Hypertable;
63+
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.RetentionPolicy;
64+
using Microsoft.EntityFrameworkCore;
65+
66+
[Hypertable(nameof(Time), ChunkTimeInterval = "1 day")]
67+
[PrimaryKey(nameof(Id), nameof(Time))]
68+
[RetentionPolicy("30 days",
69+
InitialStart = "2025-10-01T03:00:00Z",
70+
ScheduleInterval = "1 day",
71+
MaxRuntime = "30 minutes",
72+
MaxRetries = 3,
73+
RetryPeriod = "5 minutes")]
74+
public class ApplicationLog
75+
{
76+
public Guid Id { get; set; }
77+
public DateTime Time { get; set; }
78+
public string ServiceName { get; set; } = string.Empty;
79+
public string Level { get; set; } = string.Empty;
80+
public string Message { get; set; } = string.Empty;
81+
}
82+
```
83+
84+
## Supported Parameters
85+
86+
| Parameter | Description | Type | Database Type | Default Value |
87+
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------- | ------------- | ------------------------------------------- |
88+
| `DropAfter` | The interval after which chunks are dropped. Mutually exclusive with `DropCreatedBefore`. Can be passed as the first positional argument. | `string?` | `INTERVAL` | — |
89+
| `DropCreatedBefore` | The interval before which chunks created are dropped. Based on chunk creation time. Only supports `INTERVAL`. Not available for integer-based time columns. Mutually exclusive with `DropAfter`. | `string?` | `INTERVAL` | — |
90+
| `InitialStart` | The first time the policy job is scheduled to run, specified as a UTC date-time string in ISO 8601 format. If `null`, the first run is based on the `ScheduleInterval`. | `string?` | `TIMESTAMPTZ` | `null` |
91+
| `ScheduleInterval` | The interval at which the retention policy job runs. | `string?` | `INTERVAL` | `'1 day'` |
92+
| `MaxRuntime` | The maximum amount of time the job is allowed to run before being stopped. If `null`, there is no time limit. | `string?` | `INTERVAL` | `'00:00:00'` |
93+
| `MaxRetries` | The number of times the job is retried if it fails. | `int` | `INTEGER` | `-1` |
94+
| `RetryPeriod` | The amount of time the scheduler waits between retries of a failed job. | `string?` | `INTERVAL` | Equal to the `scheduleInterval` by default. |

‎docs/fluent-api/retention-policies.md‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# Retention Policies
2+
3+
A retention policy automatically drops old chunks from a hypertable or continuous aggregate on a scheduled basis. This keeps storage consumption bounded without requiring manual intervention and is the standard approach for managing time-series data lifecycle in TimescaleDB.
4+
5+
Each hypertable or continuous aggregate supports at most one retention policy.
6+
7+
[See also: add_retention_policy](https://docs.tigerdata.com/api/latest/data_retention/add_retention_policy/)
8+
9+
## Drop Modes
10+
11+
Two mutually exclusive drop modes are available:
12+
13+
- **`dropAfter`**: Drops chunks whose data falls outside a time window relative to the current time. This is the standard mode.
14+
- **`dropCreatedBefore`**: Drops chunks created before a specified interval ago, regardless of the data they contain.
15+
16+
Exactly one of `dropAfter` or `dropCreatedBefore` must be specified. Providing both or neither raises an exception.
17+
18+
## Basic Example
19+
20+
```csharp
21+
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.RetentionPolicy;
22+
using Microsoft.EntityFrameworkCore;
23+
using Microsoft.EntityFrameworkCore.Metadata.Builders;
24+
25+
public class ApplicationLogConfiguration : IEntityTypeConfiguration<ApplicationLog>
26+
{
27+
public void Configure(EntityTypeBuilder<ApplicationLog> builder)
28+
{
29+
builder.HasKey(x => new { x.Id, x.Time });
30+
31+
builder.IsHypertable(x => x.Time)
32+
.WithChunkTimeInterval("1 day");
33+
34+
// Drop chunks older than 30 days, running the job daily
35+
builder.WithRetentionPolicy(
36+
dropAfter: "30 days",
37+
scheduleInterval: "1 day");
38+
}
39+
}
40+
```
41+
42+
## Using `dropCreatedBefore`
43+
44+
```csharp
45+
public class ApiRequestLogConfiguration : IEntityTypeConfiguration<ApiRequestLog>
46+
{
47+
public void Configure(EntityTypeBuilder<ApiRequestLog> builder)
48+
{
49+
builder.HasKey(x => new { x.Id, x.Time });
50+
51+
builder.IsHypertable(x => x.Time)
52+
.WithChunkTimeInterval("1 day");
53+
54+
// Drop chunks created more than 30 days ago
55+
builder.WithRetentionPolicy(
56+
dropCreatedBefore: "30 days",
57+
scheduleInterval: "1 day");
58+
}
59+
}
60+
```
61+
62+
> :warning: **Note:** Due to a known bug in TimescaleDB ([#9446](https://github.com/timescale/timescaledb/issues/9446)), `alter_job` fails when used with `drop_created_before` policies. The library works around this by skipping the `alter_job` call for `drop_created_before` policies. As a result, job scheduling parameters (`scheduleInterval`, `maxRuntime`, `maxRetries`, `retryPeriod`) are accepted by the API but have no effect at the database level when `dropCreatedBefore` is used.
63+
64+
## Complete Example
65+
66+
```csharp
67+
using CmdScale.EntityFrameworkCore.TimescaleDB.Configuration.RetentionPolicy;
68+
using Microsoft.EntityFrameworkCore;
69+
using Microsoft.EntityFrameworkCore.Metadata.Builders;
70+
71+
public class ApplicationLogConfiguration : IEntityTypeConfiguration<ApplicationLog>
72+
{
73+
public void Configure(EntityTypeBuilder<ApplicationLog> builder)
74+
{
75+
builder.HasKey(x => new { x.Id, x.Time });
76+
77+
builder.IsHypertable(x => x.Time)
78+
.WithChunkTimeInterval("1 day");
79+
80+
builder.WithRetentionPolicy(
81+
dropAfter: "30 days",
82+
initialStart: new DateTime(2025, 10, 1, 3, 0, 0, DateTimeKind.Utc),
83+
scheduleInterval: "1 day",
84+
maxRuntime: "30 minutes",
85+
maxRetries: 3,
86+
retryPeriod: "5 minutes");
87+
}
88+
}
89+
90+
public class ApplicationLog
91+
{
92+
public Guid Id { get; set; }
93+
public DateTime Time { get; set; }
94+
public string ServiceName { get; set; } = string.Empty;
95+
public string Level { get; set; } = string.Empty;
96+
public string Message { get; set; } = string.Empty;
97+
}
98+
```
99+
100+
## Supported Parameters
101+
102+
| Parameter | Description | Type | Database Type | Default Value |
103+
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------- | ------------- | ------------------------------------------- |
104+
| `dropAfter` | The interval after which chunks are dropped. Mutually exclusive with `dropCreatedBefore`. | `string?` | `INTERVAL` | — |
105+
| `dropCreatedBefore` | The interval before which chunks created are dropped. Based on chunk creation time. Only supports `INTERVAL`. Not available for integer-based time columns. Mutually exclusive with `dropAfter`. | `string?` | `INTERVAL` | — |
106+
| `initialStart` | The first time the policy job is scheduled to run, as a UTC `DateTime`. If `null`, the first run is based on the `scheduleInterval`. | `DateTime?` | `TIMESTAMPTZ` | `null` |
107+
| `scheduleInterval` | The interval at which the retention policy job runs. | `string?` | `INTERVAL` | `'1 day'` |
108+
| `maxRuntime` | The maximum amount of time the job is allowed to run before being stopped. If `null`, there is no time limit. | `string?` | `INTERVAL` | `'00:00:00'` |
109+
| `maxRetries` | The number of times the job is retried if it fails. | `int?` | `INTEGER` | `-1` |
110+
| `retryPeriod` | The amount of time the scheduler waits between retries of a failed job. | `string?` | `INTERVAL` | Equal to the `scheduleInterval` by default. |

0 commit comments

Comments
 (0)