You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: master/_sources/configuration/prune.rst.txt
+9Lines changed: 9 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -50,6 +50,7 @@ Example Configuration:
50
50
regex: "^manual_.*"
51
51
52
52
.. DANGER::
53
+
53
54
You might have **existing snapshots** of filesystems affected by pruning which you want to keep, i.e. not be destroyed by zrepl.
54
55
Make sure to actually add the necessary ``regex`` keep rules on both sides, like with ``manual`` in the example above.
55
56
@@ -71,6 +72,14 @@ It only makes sense to specify this rule for the ``keep_sender``.
71
72
The reason is that, by definition, all snapshots on the receiver have already been replicated to there from the sender.
72
73
To determine whether a sender-side snapshot has already been replicated, zrepl uses the :ref:`replication cursor bookmark <replication-cursor-and-last-received-hold>` which corresponds to the most recent successfully replicated snapshot.
73
74
75
+
.. ATTENTION::
76
+
77
+
If you use ``not_replicated`` in your pruning rules, make sure to :ref:`monitor <monitoring>` replication.
78
+
If your replication gets stuck then ``not_replicated`` causes snapshots to pile uip on the sender.
79
+
ZFS and especially the ``zfs`` management command are known to degrade in performance with a lot of snapshots.
80
+
Such degradation impacts zrepl, any other scripts, and human ability to manage your zpool.
Copy file name to clipboardExpand all lines: master/_sources/configuration/snapshotting.rst.txt
+37-17Lines changed: 37 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,8 +25,9 @@ The following snapshotting types are supported:
25
25
26
26
The ``periodic`` and ``cron`` snapshotting types share some common options and behavior:
27
27
28
-
* **Naming:** The snapshot names are composed of a user-defined ``prefix`` followed by a UTC date formatted like ``20060102_150405_000``.
29
-
We use UTC because it will avoid name conflicts when switching time zones or between summer and winter time.
28
+
* **Naming:** The snapshot names are composed of a user-defined ``prefix`` followed by a UTC date formatted like ``20060102_150405_000`` by default.
29
+
We either use UTC or timestamp with timezone information because it will avoid name conflicts when switching time zones or between summer and winter time.
30
+
Note that if timestamp with time zone information is used, the "+" of the timezone (i.e. +02:00) is replaced by "_" (i.e. _02:00) to conform to allowed characters in ZFS snapshots.
30
31
* **Hooks:** You can configure hooks to run before or after zrepl takes the snapshots. See :ref:`below <job-snapshotting-hooks>` for details.
31
32
* **Push replication:** After creating all snapshots, the snapshotter will wake up the replication part of the job, if it's a ``push`` job.
32
33
Note that snapshotting is decoupled from replication, i.e., if it is down or takes too long, snapshots will still be taken.
@@ -65,6 +66,9 @@ The ``periodic`` and ``cron`` snapshotting types share some common options and b
65
66
# Timestamp format that is used as snapshot suffix.
66
67
# Can be any of "dense" (default), "human", "iso-8601", "unix-seconds" or a custom Go time format (see https://go.dev/src/time/format.go)
67
68
timestamp_format: dense
69
+
# Specifies in which time zone the snapshot suffix is generated, optional, defaults to UTC, used only if time zone information is part of timestamp_format.
70
+
# Can be "UTC" (default), "Local" (time zone information from the OS) or any of the IANA Time Zone names (see https://nodatime.org/TimeZones)
71
+
timestamp_location: UTC
68
72
hooks: ...
69
73
pruning: ...
70
74
@@ -97,27 +101,16 @@ The snapshotter uses the ``prefix`` to identify which snapshots it created.
97
101
# Timestamp format that is used as snapshot suffix.
98
102
# Can be any of "dense" (default), "human", "iso-8601", "unix-seconds" or a custom Go time format (see https://go.dev/src/time/format.go)
99
103
timestamp_format: dense
104
+
# Specifies in which time zone the snapshot suffix is generated, optional, defaults to UTC, used only if time zone information is part of timestamp_format.
105
+
# Can be "UTC" (default), "Local" (time zone information from the OS) or any of the IANA Time Zone names (see https://nodatime.org/TimeZones)
106
+
timestamp_location: UTC
100
107
pruning: ...
101
108
102
109
In ``cron`` mode, the snapshotter takes snaphots at fixed points in time.
103
110
See https://en.wikipedia.org/wiki/Cron for details on the syntax.
104
111
zrepl uses the ``the github.com/robfig/cron/v3`` Go package for parsing.
105
112
An optional field for "seconds" is supported to take snapshots at sub-minute frequencies.
106
113
107
-
.. _job-snapshotting-timestamp_format:
108
-
109
-
Timestamp Format
110
-
~~~~~~~~~~~~~~~~
111
-
112
-
The ``cron`` and ``periodic`` snapshotter support configuring a custom timestamp format that is used as suffix for the snapshot name.
113
-
It can be used by setting ``timestamp_format`` to any of the following values:
114
-
115
-
* ``dense`` (default) looks like ``20060102_150405_000``
116
-
* ``human`` looks like ``2006-01-02_15:04:05``
117
-
* ``iso-8601`` looks like ``2006-01-02T15:04:05.000Z``
118
-
* ``unix-seconds`` looks like ``1136214245``
119
-
* Any custom Go time format accepted by `time.Time#Format <https://go.dev/src/time/format.go>`_.
120
-
121
114
122
115
``manual`` Snapshotting
123
116
-----------------------
@@ -137,6 +130,33 @@ See :ref:`this quickstart guide <quickstart-backup-to-external-disk>` for an exa
137
130
138
131
To trigger replication after taking snapshots, use the ``zrepl signal wakeup JOB`` command.
139
132
133
+
.. _job-snapshotting-timestamp_format:
134
+
135
+
Timestamp Format
136
+
----------------
137
+
138
+
The ``cron`` and ``periodic`` snapshotter support configuring a custom timestamp format that is used as suffix for the snapshot name.
139
+
140
+
By default, the current time is converted to UTC and then formatted using the ``dense`` format.
141
+
142
+
Both time zone and format can be customized by setting ``timestamp_format`` and/or ``timestamp_location``.
143
+
144
+
``timestamp_location`` is fed verbatim into `time.LoadLocation <https://pkg.go.dev/time#LoadLocation>`_.
145
+
146
+
``timestamp_format`` supports the following values (case-insensitive).
147
+
148
+
* ``dense`` => Go format string ``20060102_150405_000``
149
+
* ``human`` => Go format string ``2006-01-02_15:04:05``
150
+
* ``iso-8601`` => Go format string ``2006-01-02T15:04:05.000Z07:00``
151
+
* ``unix-seconds`` => Unix seconds since the epoch, formatted as a decimal string (`time.Time.Unix <https://pkg.go.dev/time#Time.Unix>`_)
152
+
* Any custom Go time format accepted by `time.Time#Format <https://go.dev/src/time/format.go>`_.
153
+
154
+
The ``dense``, ``human``, and ``unix-seconds`` formats require the ``timestamp_location`` to be ``UTC`` because they do not contain timezone information.
155
+
156
+
The ``+`` character is `not allowed in ZFS snapshot names <https://github.com/openzfs/zfs/blob/1d3ba0bf01020f5459b1c28db3979129088924c0/module/zcommon/zfs_namecheck.c#L334-L351>`_.
157
+
Format strings are disallowed to contain the ``+`` character (it's not a character that is interpreted by ``time.Time.Format``, so there's no reason to use it).
158
+
If a format _produces_ a ``+``, that ``+`` is replaced by an `_`` character.
159
+
140
160
.. _job-snapshotting-hooks:
141
161
142
162
Pre- and Post-Snapshot Hooks
@@ -173,7 +193,7 @@ Most hook types take additional parameters, please refer to the respective subse
0 commit comments