patrick/zfs_autobackup
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update

Edwin Eefting
2022-01-07 13:29:42 +01:00
parent 9c9a59d932
commit 44e18e8985
1 changed files with 75 additions and 94 deletions
Download Patch File Download Diff File Expand all files Collapse all files

141
Home.md

@ -28,7 +28,7 @@ On older servers you might have to use easy_install
## Example
In this example we're going to backup a machine called `server1` to a machine called `backup`.
In this example we're going to backup a machine called `pve01` to a machine called `backup`.
### Setup SSH login
@ -50,27 +50,23 @@ Enter same passphrase again:
root@backup:~#
```
#### Copy SSH key to `server1`
#### Copy SSH key to `pve01`
Now you need to copy the public part of the key to `server1`
Now you need to copy the public part of the key to `pve01`
The `ssh-copy-id` command is a handy tool to automate this. It will just ask for your password.
```console
root@backup:~# ssh-copy-id root@server1.server.com
root@backup:~# ssh-copy-id root@pve01
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@server1.server.com'"
and check to make sure that only the key(s) you wanted were added.
root@backup:~#
```
This allows the backup-server to login to `server1` as root without password.
This allows the backup-server to login to `pve01` as root without password.
### Select filesystems to backup
@ -79,16 +75,16 @@ Its important to choose a unique and consistent backup name. In this case we nam
On the source zfs system set the ```autobackup:offsite1``` zfs property to true:
```console
[root@server1 ~]# zfs set autobackup:offsite1=true rpool
[root@server1 ~]# zfs get -t filesystem,volume autobackup:offsite1
[root@pve01 ~]# zfs set autobackup:offsite1=true rpool
[root@pve01 ~]# zfs get -t filesystem,volume autobackup:offsite1
NAME PROPERTY VALUE SOURCE
rpool autobackup:offsite1 true local
rpool/ROOT autobackup:offsite1 true inherited from rpool
rpool/ROOT/server1-1 autobackup:offsite1 true inherited from rpool
rpool/ROOT/pve-1 autobackup:offsite1 true inherited from rpool
rpool/data autobackup:offsite1 true inherited from rpool
rpool/data/vm-100-disk-0 autobackup:offsite1 true inherited from rpool
rpool/swap autobackup:offsite1 true inherited from rpool
...
rpool/data/vm-101-disk-0 autobackup:offsite1 true inherited from rpool
rpool/tmp autobackup:offsite1 true inherited from rpool
```
ZFS properties are ```inherited``` by child datasets. Since we've set the property on the highest dataset, we're essentially backupping the whole pool.
@ -96,16 +92,16 @@ ZFS properties are ```inherited``` by child datasets. Since we've set the proper
Because we don't want to backup everything, we can exclude certain filesystem by setting the property to false:
```console
[root@server1 ~]# zfs set autobackup:offsite1=false rpool/swap
[root@server1 ~]# zfs get -t filesystem,volume autobackup:offsite1
[root@pve01 ~]# zfs set autobackup:offsite1=false rpool/tmp
[root@pve01 ~]# zfs get -t filesystem,volume autobackup:offsite1
NAME PROPERTY VALUE SOURCE
rpool autobackup:offsite1 true local
rpool/ROOT autobackup:offsite1 true inherited from rpool
rpool/ROOT/server1-1 autobackup:offsite1 true inherited from rpool
rpool/ROOT/pve-1 autobackup:offsite1 true inherited from rpool
rpool/data autobackup:offsite1 true inherited from rpool
rpool/data/vm-100-disk-0 autobackup:offsite1 true inherited from rpool
rpool/swap autobackup:offsite1 false local
...
rpool/data/vm-101-disk-0 autobackup:offsite1 true inherited from rpool
rpool/tmp autobackup:offsite1 false local
```
The autobackup-property can have 3 values:
@ -113,53 +109,56 @@ The autobackup-property can have 3 values:
* ```false```: Dont backup the dataset and all its children. (used to exclude certain datasets)
* ```child```: Only backup the children off the dataset, not the dataset itself.
Only use the zfs-command to set these properties, not the zpool command.
(Only use the zfs-command to set these properties, not the zpool command.)
### Running zfs-autobackup
Run the script on the backup server and pull the data from the server specified by --ssh-source.
```console
[root@backup ~]# zfs-autobackup --ssh-source server1.server.com offsite1 backup/server1 --progress --verbose
[root@backup ~]# zfs-autobackup -v --ssh-source pve01 offsite1 data/backup/pve01
zfs-autobackup v3.1.1 - (c)2021 E.H.Eefting (edwin@datux.nl)
#### Settings summary
[Source] Datasets on: server1.server.com
Selecting dataset property : autobackup:offsite1
Snapshot format : offsite1-%Y%m%d%H%M%S
Hold name : zfs_autobackup:offsite1
#### Source settings
[Source] Datasets on: pve01
[Source] Keep the last 10 snapshots.
[Source] Keep every 1 day, delete after 1 week.
[Source] Keep every 1 week, delete after 1 month.
[Source] Keep every 1 month, delete after 1 year.
[Source] Send all datasets that have 'autobackup:offsite1=true' or 'autobackup:offsite1=child'
#### Selecting
[Source] rpool: Selected
[Source] rpool/ROOT: Selected
[Source] rpool/ROOT/pve-1: Selected
[Source] rpool/data: Selected
[Source] rpool/data/vm-100-disk-0: Selected
[Source] rpool/data/vm-101-disk-0: Selected
[Source] rpool/tmp: Excluded
#### Snapshotting
[Source] Creating snapshots offsite1-20220107131107 in pool rpool
#### Target settings
[Target] Datasets are local
[Target] Keep the last 10 snapshots.
[Target] Keep every 1 day, delete after 1 week.
[Target] Keep every 1 week, delete after 1 month.
[Target] Keep every 1 month, delete after 1 year.
[Target] Receive datasets under: backup/server1
[Target] Receive datasets under: data/backup/pve01
#### Selecting
[Source] rpool: Selected (direct selection)
[Source] rpool/ROOT: Selected (inherited selection)
[Source] rpool/ROOT/server1-1: Selected (inherited selection)
[Source] rpool/data: Selected (inherited selection)
[Source] rpool/data/vm-100-disk-0: Selected (inherited selection)
[Source] rpool/swap: Ignored (disabled)
#### Synchronising
[Target] data/backup/pve01/rpool@offsite1-20220107131107: receiving full
[Target] data/backup/pve01/rpool/ROOT@offsite1-20220107131107: receiving full
[Target] data/backup/pve01/rpool/ROOT/pve-1@offsite1-20220107131107: receiving full
[Target] data/backup/pve01/rpool/data@offsite1-20220107131107: receiving full
[Target] data/backup/pve01/rpool/data/vm-100-disk-0@offsite1-20220107131107: receiving full
[Target] data/backup/pve01/rpool/data/vm-101-disk-0@offsite1-20220107131107: receiving full
#### Snapshotting
[Source] rpool: No changes since offsite1-20200218175435
[Source] rpool/ROOT: No changes since offsite1-20200218175435
[Source] rpool/data: No changes since offsite1-20200218175435
[Source] Creating snapshot offsite1-20200218180123
#### Sending and thinning
[Target] backup/server1/rpool/ROOT/server1-1@offsite1-20200218175435: receiving full
[Target] backup/server1/rpool/ROOT/server1-1@offsite1-20200218175547: receiving incremental
[Target] backup/server1/rpool/ROOT/server1-1@offsite1-20200218175706: receiving incremental
[Target] backup/server1/rpool/ROOT/server1-1@offsite1-20200218180049: receiving incremental
[Target] backup/server1/rpool/ROOT/server1-1@offsite1-20200218180123: receiving incremental
[Target] backup/server1/rpool/data@offsite1-20200218175435: receiving full
[Target] backup/server1/rpool/data/vm-100-disk-0@offsite1-20200218175435: receiving full
...
#### All operations completed successfully
```
Note that this is called a "pull" backup: The backup server pulls the backup from the server. This is usually the preferred way.
@ -180,45 +179,27 @@ Or just create a script and run it manually when you need it.
Dont forget to monitor the results of your backups, look at [Monitoring](Monitoring) for more info.
### Splitting up snapshot and backup job
You might want to make snapshots during the week, and only transfer data during the weekends.
In this case you would run this every day:
> zfs-autobackup -v --ssh-source pve01 offsite1 data/backup/pve01 --no-send
And this in the weekends:
> zfs-autobackup -v --ssh-source pve01 offsite1 data/backup/pve01
## Use as snapshot tool
You can use zfs-autobackup to only make snapshots.
You can use zfs-autobackup as a standalone snapshot tool.
Just dont specify the target-path:
```console
root@ws1:~# zfs-autobackup test --verbose
zfs-autobackup v3.0 - Copyright 2020 E.H.Eefting (edwin@datux.nl)
#### Source settings
[Source] Datasets are local
[Source] Keep the last 10 snapshots.
[Source] Keep every 1 day, delete after 1 week.
[Source] Keep every 1 week, delete after 1 month.
[Source] Keep every 1 month, delete after 1 year.
[Source] Selects all datasets that have property 'autobackup:test=true' (or childs of datasets that have 'autobackup:test=child')
> zfs-autobackup -v --ssh-source pve01 offsite1
#### Selecting
[Source] test_source1/fs1: Selected (direct selection)
[Source] test_source1/fs1/sub: Selected (inherited selection)
[Source] test_source2/fs2: Ignored (only childs)
[Source] test_source2/fs2/sub: Selected (inherited selection)
#### Snapshotting
[Source] Creating snapshots test-20200710125958 in pool test_source1
[Source] Creating snapshots test-20200710125958 in pool test_source2
#### Thinning source
[Source] test_source1/fs1@test-20200710125948: Destroying
[Source] test_source1/fs1/sub@test-20200710125948: Destroying
[Source] test_source2/fs2/sub@test-20200710125948: Destroying
#### All operations completed successfully
(No target_path specified, only operated as snapshot tool.)
```
This also allows you to make several snapshots during the day, but only backup the data at night when the server is not busy.
**Note**: In this mode it doesnt take a specified target-schedule into account when thinning, it only knows a snapshot is the common snapshot by looking at the holds. So make sure your source-schedule keeps the snapshots you still want to transfer at a later point.
Only use this if you dont want to make any backup at all or if a target isn't reachable during the snapshotting phase.
### Specifying ssh port or options