Skip to content

Commit 2eed768

Browse files
author
Carlos Gonzalez Betancort
committed
MB-34705 Add adoc files for tools that miss them
Some of the tools in the repo don't have a matching adoc file, instead the documentation is in a separate repo, to ensure the documentation is up to date, adoc files have been added to this repo and updated to match current functionality. Change-Id: Id97569faf7fc1bc4bd31d947d794a041355e1c60 Reviewed-on: http://review.couchbase.org/111151 Tested-by: Build Bot <build@couchbase.com> Reviewed-by: Patrick Varley <patrick@couchbase.com>
1 parent 7ebfd8b commit 2eed768

7 files changed

Lines changed: 1630 additions & 0 deletions

File tree

‎docs/modules/cli/pages/cbtools/cbbackup.adoc‎

Lines changed: 326 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,326 @@
1+
= cbbackup(1)
2+
ifndef::doctype-manpage[:doctitle: cbbackup]
3+
4+
ifdef::doctype-manpage[]
5+
== NAME
6+
7+
cbbackup -
8+
endif::[]
9+
Backups an entire running cluster, a bucket or a single node
10+
11+
== SYNOPSIS
12+
[verse]
13+
_cbbackup_ [--username <user>] [--password <password>] [--ssl]
14+
[--no-ssl-verify] [--cacert <path>] [--bucket-source <name>]
15+
[--single-node] [--id <vbid>] [--key <regexp>]
16+
[--mode <full|diff|accu>] [--dry-run] [--verbose] [--silent]
17+
[--extra <options>] [--help] source backup_dir
18+
19+
== DESCRIPTION
20+
21+
This backup tools creates a copy of the `source` data onto disk in the
22+
specified `backup_dir`, if the directory does not exist it will be created.
23+
The directory should only be used by `cbbackup` and `cbrestore` as any
24+
alterations to the data by any other process can corrupt the backup data.
25+
26+
The tool can be found in the following directories:
27+
[cols="1,5"]
28+
|===
29+
| Operating system | Location
30+
31+
| Linux
32+
| `/opt/couchbase/bin/cbbackup`
33+
34+
| Windows
35+
| `C:\Program Files\Couchbase\Server\bin\cbbackup`
36+
37+
| Mac OS X
38+
| `/Applications/Couchbase Server.app/Contents/Resources/couchbase-core/bin/cbbackup`
39+
|===
40+
41+
42+
The data is stored in a unique format that will then allow
43+
a fast restore using `cbrestore`. This tool will not only backup the documents
44+
but it can backup metadata for full text indexes and aliases, design documents
45+
and global secondary indexes, thus on a restore there is no need to redefine them.
46+
Although the metadata for this other services is also backed up and restore the
47+
indexes will still have to be rebuilt after a restore.
48+
49+
The first backup made is always a full backup regardless of the mode given.
50+
The backup directory will looks as follows after a succesfull first backup:
51+
52+
└── 2019-06-25T124727Z
53+
└── 2019-06-25T124727Z-full
54+
├── bucket-beer-sample
55+
│   ├── design.json
56+
│   ├── fts_index.json
57+
│   ├── index.json
58+
│   └── node-127.0.0.1%3A8091
59+
│   ├── data-0000.cbb
60+
│   ├── failover.json
61+
│   ├── meta.json
62+
│   ├── seqno.json
63+
│   └── snapshot_markers.json
64+
└── fts_alias.json
65+
66+
The backup directory will have as a name the timestamp of the first backup.
67+
Inside it there will be one or more backup folders each named with a timestamp
68+
of when they occurred and suffixed with the mode in this case
69+
`2019-06-25T124727Z-full`. Inside each backup folder we have a folder for each
70+
bucket that was backed up, in this case only the "beer-sample" was backed up.
71+
At this level we also have a JSON file that contains the full text aliases.
72+
Inside each bucket folder we can find the design documents, full text indexes
73+
and global secondary indexes definitions. There is also a folder per node in
74+
the cluster, which contains the documents for that bucket and cluster. In this
75+
case there is only one node in the cluster.
76+
77+
== Options
78+
79+
-u,--username <user>::
80+
Specifies the username of the user executing the command. If you do not have
81+
a user account with permission to execute the command then it will fail with
82+
an unauthorized error.
83+
84+
-p,--password <password>::
85+
Specifies the password of the user executing the command. If you do not have
86+
a user account with permission to execute the command then it will fail with
87+
an unauthorized error.
88+
89+
-s,--ssl::
90+
(Deprecated) Specifies that the connection should use SSL verification. If
91+
this flag is used then SSL will be used but the cluster certificate will not
92+
be verified by the Certificate Authority. This flag is deprecated and not
93+
recommended. If you wish to use SSL encryption it is recommended that you
94+
specify the cluster host name using either _couchbases://_ or _https://_.
95+
Each of these connection schemes will ensure that the connection is
96+
encrypted with SSL. You may then use either --no-ssl-verify or --cacert in
97+
order to customize how your SSL connection is set up.
98+
99+
--no-ssl-verify::
100+
Specifies that SSL verification should be used but that verifying that the
101+
cluster certificate is valid should be skipped. Use of this flag is not
102+
recommended for production environments because it does not protect the user
103+
from a man-in-the-middle attack.
104+
105+
--cacert <path>::
106+
Specifies that the SSL connection should use the cacert provided when
107+
connecting to the cluster. This argument takes the path the certificate
108+
file as its value. This is the most secure way to connect to your cluster.
109+
110+
-b,--bucket-source <bucket>::
111+
Single named bucket from source cluster to transfer. If the backup directory
112+
only contains a single bucket, then that bucket is automatically used.
113+
114+
--single-node::
115+
Transfer data only from the source data node.
116+
117+
-i,--id <vbid>::
118+
Transfer only items that match a vBucket ID.
119+
120+
-k,--key <regexp>::
121+
Transfer only items with keys that match the given regular expression.
122+
123+
-m,--mode <full|diff|accu>::
124+
Select tha backup mode, the available options are: full, diff, accu.
125+
If an option is not provided it defaults to `diff`. The `full` mode will
126+
perform a full backup. The `diff` mode will perform a differential
127+
incremental backup which backups only the changes since the last full or
128+
incremental backup. The `accu` mode performs a cumulative incremental backup,
129+
which backs up all changes since the last full backup.
130+
131+
-n,--dry-run::
132+
When specified the tool will not transfer data but only validate
133+
parameters, files, connectivity and configuration.
134+
135+
-v,--verbose::
136+
Verbose logging; more -v's provide more verbosity. Max is -vvv
137+
138+
--silent::
139+
Reduces the logging verbosity to only include errors.
140+
141+
-x,--extra <options>::
142+
Provide extra, uncommon configuration parameters. Comma-separated
143+
key=val pairs
144+
145+
== EXTRAS
146+
147+
The following are extra, specialized command options with the `cbbackup -x` parameter.
148+
149+
.cbbackup -x options
150+
[cols="100,223"]
151+
|===
152+
| -x options | Description
153+
154+
| `backoff_cap=10`
155+
| Maximum backoff time during the rebalance period.
156+
157+
| `batch_max_bytes=400000`
158+
| Transfer this # of bytes per batch.
159+
160+
| `batch_max_size=1000`
161+
| Transfer this # of documents per batch.
162+
163+
| `cbb_max_mb=100000`
164+
| Split backup file on destination cluster if it exceeds the MB.
165+
166+
| `conflict_resolve=1`
167+
| By default, disable conflict resolution.
168+
169+
This option doesn't work in Couchbase Server versions 4.0 and 4.1 but will be
170+
re-implemented in version 4.1.1 and in subsequent versions.
171+
172+
| `data_only=0`
173+
| For value 1, transfer only data from a backup file or cluster.
174+
175+
| `design_doc_only=0`
176+
| For value 1, transfer only design documents from a backup file or cluster.
177+
Default: 0.
178+
179+
Back up only design documents which include view and secondary index
180+
definitions from a cluster or bucket with the option `design_doc_only=1`.
181+
Restore only design documents with `cbrestore -x design_doc_only=1`.
182+
183+
| `max_retry=10`
184+
| Max number of sequential retries if the transfer fails.
185+
186+
| `mcd_compatible=1`
187+
| For value 0, display extended fields for stdout output.
188+
189+
| `nmv_retry=1`
190+
| 0 or 1, where 1 retries transfer after a NOT_MY_VBUCKET message.
191+
Default: 1.
192+
193+
| `recv_min_bytes=4096`
194+
| Amount of bytes for every TCP/IP batch transferred.
195+
196+
| `rehash=0`
197+
| For value 1, rehash the partition id's of each item.
198+
This is required when transferring data between clusters with different number
199+
of partitions, such as when transferring data from an Mac OS X server to a
200+
non-Mac OS X cluster.
201+
202+
| `report=5`
203+
| Number batches transferred before updating progress bar in console.
204+
205+
| `report_full=2000`
206+
| Number batches transferred before emitting progress information in console.
207+
208+
| `seqno=0`
209+
| By default, start seqno from beginning.
210+
211+
| `try_xwm=1`
212+
| Transfer documents with metadata.
213+
Default: 1.
214+
Value of 0 is only used when transferring from 1.8.x to 1.8.x.
215+
216+
| `uncompress=0`
217+
| For value 1, restore data in uncompressed mode.
218+
219+
This option is unsupported.
220+
To create backups with compression, use `cbbackupmgr`, which is available for
221+
Couchbase Server Enterprise Edition only.
222+
See xref:backup-restore:enterprise-backup-restore.adoc[Backup].
223+
|===
224+
225+
== EXAMPLES
226+
227+
*Basic operations*
228+
229+
The most basic operation is to backup an entire cluster, this can be done using
230+
the following command:
231+
232+
$ cbbackup http://10.112.193.101:8091 ~/backups -u Administrator -p password
233+
234+
This command will create the following directory structure:
235+
236+
└── backups
237+
└── 2019-06-25T124727Z
238+
└── 2019-06-25T124727Z-full
239+
├── bucket-beer-sample
240+
│   ├── design.json
241+
│   ├── fts_index.json
242+
│   ├── index.json
243+
│   ├── node-10.112.193.101%3A8091
244+
│   ├── node-10.112.193.102%3A8091
245+
│   ├── node-10.112.193.103%3A8091
246+
├── bucket-gamesim-sample
247+
│   ├── design.json
248+
│   ├── node-10.112.193.101%3A8091
249+
│   ├── node-10.112.193.102%3A8091
250+
│   └── node-10.112.193.103%3A8091
251+
└── fts_alias.json
252+
253+
Inside our requested backup directory `cbbackup` has created another folder
254+
named after the time when the command was run. Inside there is only one backup,
255+
`2019-06-25T124727Z-full` we can see by the prefix that is a full backup.
256+
This is expected as there where no other backups in the `~/backups` directory
257+
Inside out backup we can find the full text aliases in the `fts_alias.json` as
258+
well as another two folders. one for each bucket in the cluster. This cluster
259+
had buckets beer-sample and gamesim-sample. Inside the folder for each bucket
260+
we find the design documents, full text index adn general secondary index
261+
definitions. We can see that the gamesim-bucket does not have either a
262+
`fts_index.json` or an `index.json` this s because this bucket had no
263+
indexes. Inside the bucket folders we have a folder for each node in the
264+
cluster, this folder contains the documents stored in each node for the
265+
corresponding bucket.
266+
267+
After some time data in the buckets will change and a new backup should be
268+
made, but instead of backing up all the data again `cbbackup` can make
269+
incremental backups that only contain the data that has changed since the last
270+
backup. The command below would create a new differential incremental backup.
271+
272+
$ cbbackup http://10.112.193.101:8091 ~/backups -u Administrator -p password \
273+
-m diff
274+
275+
After this the backup directory would look as follows (only showing beer-sample for clarity):
276+
277+
2019-06-25T141453Z
278+
├── 2019-06-25T141453Z-full
279+
│   └── bucket-beer-sample
280+
│   ├── design.json
281+
│   ├── node-10.112.193.101%3A8091
282+
│   ├── node-10.112.193.102%3A8091
283+
│   └── node-10.112.193.103%3A8091
284+
└── 2019-06-25T141553Z-diff
285+
└── bucket-beer-sample
286+
├── design.json
287+
├── node-10.112.193.101%3A8091
288+
├── node-10.112.193.102%3A8091
289+
└── node-10.112.193.103%3A8091
290+
291+
*Filtering*
292+
293+
`cbbackup` allows the user control of what data is backed up. To backup only a
294+
the bucket `beer-sample` use the following command:
295+
296+
$ cbbackup http://10.112.193.101:8091 ~/backups -u Administrator -p password \
297+
-b beer-sample
298+
299+
This command will backup only the data relevant to bucket `beer-sample` from
300+
evey node in the cluster. The user can also want only data from one node in the
301+
cluster this can be done as follows.
302+
303+
$ cbbackup http://10.112.193.101:8091 ~/backups -u Administrator -p password \
304+
--single-node
305+
306+
The command above will only backup the data stored in node
307+
`10.112.193.101:8091`. To backup only the data of one bucket in one node the
308+
options can be combined as follows:
309+
310+
$ cbbackup http://10.112.193.101:8091 ~/backups -u Administrator -p password \
311+
--single-node -b beer-sample
312+
313+
The data that is backed up can also be filtered y key using the `-k` flag. To
314+
only backup the documents which keys start with "KEY:" we can use the command:
315+
316+
$ cbbackup http://10.112.193.101:8091 ~/backups -u Administrator -p password \
317+
-k '^KEY:.*'
318+
319+
The `-k` option will take any regular expression.
320+
321+
The user can also chose to only backup certain services for example to only
322+
backup the documents of the data service we can pass the `data_only=1`
323+
extra option as follows:
324+
325+
$ cbbackup http://10.112.193.101:8091 ~/backups -u Administrator -p password \
326+
-x data_only=1

0 commit comments

Comments
 (0)