From 44e18e89859127d917177f71bac700d0e7ad5e00 Mon Sep 17 00:00:00 2001 From: Edwin Eefting Date: Fri, 7 Jan 2022 13:29:42 +0100 Subject: [PATCH] update --- Home.md | 169 +++++++++++++++++++++++++------------------------------- 1 file changed, 75 insertions(+), 94 deletions(-) diff --git a/Home.md b/Home.md index 742309b..c70dca2 100644 --- a/Home.md +++ b/Home.md @@ -28,7 +28,7 @@ On older servers you might have to use easy_install ## Example -In this example we're going to backup a machine called `server1` to a machine called `backup`. +In this example we're going to backup a machine called `pve01` to a machine called `backup`. ### Setup SSH login @@ -50,27 +50,23 @@ Enter same passphrase again: root@backup:~# ``` -#### Copy SSH key to `server1` +#### Copy SSH key to `pve01` -Now you need to copy the public part of the key to `server1` +Now you need to copy the public part of the key to `pve01` The `ssh-copy-id` command is a handy tool to automate this. It will just ask for your password. ```console -root@backup:~# ssh-copy-id root@server1.server.com +root@backup:~# ssh-copy-id root@pve01 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub" /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed -/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys Password: Number of key(s) added: 1 -Now try logging into the machine, with: "ssh 'root@server1.server.com'" -and check to make sure that only the key(s) you wanted were added. - root@backup:~# ``` -This allows the backup-server to login to `server1` as root without password. +This allows the backup-server to login to `pve01` as root without password. ### Select filesystems to backup @@ -79,16 +75,16 @@ Its important to choose a unique and consistent backup name. In this case we nam On the source zfs system set the ```autobackup:offsite1``` zfs property to true: ```console -[root@server1 ~]# zfs set autobackup:offsite1=true rpool -[root@server1 ~]# zfs get -t filesystem,volume autobackup:offsite1 -NAME PROPERTY VALUE SOURCE -rpool autobackup:offsite1 true local -rpool/ROOT autobackup:offsite1 true inherited from rpool -rpool/ROOT/server1-1 autobackup:offsite1 true inherited from rpool -rpool/data autobackup:offsite1 true inherited from rpool -rpool/data/vm-100-disk-0 autobackup:offsite1 true inherited from rpool -rpool/swap autobackup:offsite1 true inherited from rpool -... +[root@pve01 ~]# zfs set autobackup:offsite1=true rpool +[root@pve01 ~]# zfs get -t filesystem,volume autobackup:offsite1 +NAME PROPERTY VALUE SOURCE +rpool autobackup:offsite1 true local +rpool/ROOT autobackup:offsite1 true inherited from rpool +rpool/ROOT/pve-1 autobackup:offsite1 true inherited from rpool +rpool/data autobackup:offsite1 true inherited from rpool +rpool/data/vm-100-disk-0 autobackup:offsite1 true inherited from rpool +rpool/data/vm-101-disk-0 autobackup:offsite1 true inherited from rpool +rpool/tmp autobackup:offsite1 true inherited from rpool ``` ZFS properties are ```inherited``` by child datasets. Since we've set the property on the highest dataset, we're essentially backupping the whole pool. @@ -96,16 +92,16 @@ ZFS properties are ```inherited``` by child datasets. Since we've set the proper Because we don't want to backup everything, we can exclude certain filesystem by setting the property to false: ```console -[root@server1 ~]# zfs set autobackup:offsite1=false rpool/swap -[root@server1 ~]# zfs get -t filesystem,volume autobackup:offsite1 -NAME PROPERTY VALUE SOURCE -rpool autobackup:offsite1 true local -rpool/ROOT autobackup:offsite1 true inherited from rpool -rpool/ROOT/server1-1 autobackup:offsite1 true inherited from rpool -rpool/data autobackup:offsite1 true inherited from rpool -rpool/data/vm-100-disk-0 autobackup:offsite1 true inherited from rpool -rpool/swap autobackup:offsite1 false local -... +[root@pve01 ~]# zfs set autobackup:offsite1=false rpool/tmp +[root@pve01 ~]# zfs get -t filesystem,volume autobackup:offsite1 +NAME PROPERTY VALUE SOURCE +rpool autobackup:offsite1 true local +rpool/ROOT autobackup:offsite1 true inherited from rpool +rpool/ROOT/pve-1 autobackup:offsite1 true inherited from rpool +rpool/data autobackup:offsite1 true inherited from rpool +rpool/data/vm-100-disk-0 autobackup:offsite1 true inherited from rpool +rpool/data/vm-101-disk-0 autobackup:offsite1 true inherited from rpool +rpool/tmp autobackup:offsite1 false local ``` The autobackup-property can have 3 values: @@ -113,53 +109,56 @@ The autobackup-property can have 3 values: * ```false```: Dont backup the dataset and all its children. (used to exclude certain datasets) * ```child```: Only backup the children off the dataset, not the dataset itself. -Only use the zfs-command to set these properties, not the zpool command. +(Only use the zfs-command to set these properties, not the zpool command.) ### Running zfs-autobackup Run the script on the backup server and pull the data from the server specified by --ssh-source. ```console -[root@backup ~]# zfs-autobackup --ssh-source server1.server.com offsite1 backup/server1 --progress --verbose - - #### Settings summary - [Source] Datasets on: server1.server.com +[root@backup ~]# zfs-autobackup -v --ssh-source pve01 offsite1 data/backup/pve01 + zfs-autobackup v3.1.1 - (c)2021 E.H.Eefting (edwin@datux.nl) + + Selecting dataset property : autobackup:offsite1 + Snapshot format : offsite1-%Y%m%d%H%M%S + Hold name : zfs_autobackup:offsite1 + + #### Source settings + [Source] Datasets on: pve01 [Source] Keep the last 10 snapshots. [Source] Keep every 1 day, delete after 1 week. [Source] Keep every 1 week, delete after 1 month. [Source] Keep every 1 month, delete after 1 year. - [Source] Send all datasets that have 'autobackup:offsite1=true' or 'autobackup:offsite1=child' - + + #### Selecting + [Source] rpool: Selected + [Source] rpool/ROOT: Selected + [Source] rpool/ROOT/pve-1: Selected + [Source] rpool/data: Selected + [Source] rpool/data/vm-100-disk-0: Selected + [Source] rpool/data/vm-101-disk-0: Selected + [Source] rpool/tmp: Excluded + + #### Snapshotting + [Source] Creating snapshots offsite1-20220107131107 in pool rpool + + #### Target settings [Target] Datasets are local [Target] Keep the last 10 snapshots. [Target] Keep every 1 day, delete after 1 week. [Target] Keep every 1 week, delete after 1 month. [Target] Keep every 1 month, delete after 1 year. - [Target] Receive datasets under: backup/server1 - - #### Selecting - [Source] rpool: Selected (direct selection) - [Source] rpool/ROOT: Selected (inherited selection) - [Source] rpool/ROOT/server1-1: Selected (inherited selection) - [Source] rpool/data: Selected (inherited selection) - [Source] rpool/data/vm-100-disk-0: Selected (inherited selection) - [Source] rpool/swap: Ignored (disabled) - - #### Snapshotting - [Source] rpool: No changes since offsite1-20200218175435 - [Source] rpool/ROOT: No changes since offsite1-20200218175435 - [Source] rpool/data: No changes since offsite1-20200218175435 - [Source] Creating snapshot offsite1-20200218180123 - - #### Sending and thinning - [Target] backup/server1/rpool/ROOT/server1-1@offsite1-20200218175435: receiving full - [Target] backup/server1/rpool/ROOT/server1-1@offsite1-20200218175547: receiving incremental - [Target] backup/server1/rpool/ROOT/server1-1@offsite1-20200218175706: receiving incremental - [Target] backup/server1/rpool/ROOT/server1-1@offsite1-20200218180049: receiving incremental - [Target] backup/server1/rpool/ROOT/server1-1@offsite1-20200218180123: receiving incremental - [Target] backup/server1/rpool/data@offsite1-20200218175435: receiving full - [Target] backup/server1/rpool/data/vm-100-disk-0@offsite1-20200218175435: receiving full - ... + [Target] Receive datasets under: data/backup/pve01 + + #### Synchronising + [Target] data/backup/pve01/rpool@offsite1-20220107131107: receiving full + [Target] data/backup/pve01/rpool/ROOT@offsite1-20220107131107: receiving full + [Target] data/backup/pve01/rpool/ROOT/pve-1@offsite1-20220107131107: receiving full + [Target] data/backup/pve01/rpool/data@offsite1-20220107131107: receiving full + [Target] data/backup/pve01/rpool/data/vm-100-disk-0@offsite1-20220107131107: receiving full + [Target] data/backup/pve01/rpool/data/vm-101-disk-0@offsite1-20220107131107: receiving full + + #### All operations completed successfully ``` Note that this is called a "pull" backup: The backup server pulls the backup from the server. This is usually the preferred way. @@ -180,45 +179,27 @@ Or just create a script and run it manually when you need it. Dont forget to monitor the results of your backups, look at [Monitoring](Monitoring) for more info. +### Splitting up snapshot and backup job + +You might want to make snapshots during the week, and only transfer data during the weekends. + +In this case you would run this every day: + +> zfs-autobackup -v --ssh-source pve01 offsite1 data/backup/pve01 --no-send + +And this in the weekends: + +> zfs-autobackup -v --ssh-source pve01 offsite1 data/backup/pve01 + ## Use as snapshot tool -You can use zfs-autobackup to only make snapshots. +You can use zfs-autobackup as a standalone snapshot tool. Just dont specify the target-path: -```console -root@ws1:~# zfs-autobackup test --verbose - zfs-autobackup v3.0 - Copyright 2020 E.H.Eefting (edwin@datux.nl) - #### Source settings - [Source] Datasets are local - [Source] Keep the last 10 snapshots. - [Source] Keep every 1 day, delete after 1 week. - [Source] Keep every 1 week, delete after 1 month. - [Source] Keep every 1 month, delete after 1 year. - [Source] Selects all datasets that have property 'autobackup:test=true' (or childs of datasets that have 'autobackup:test=child') +> zfs-autobackup -v --ssh-source pve01 offsite1 - #### Selecting - [Source] test_source1/fs1: Selected (direct selection) - [Source] test_source1/fs1/sub: Selected (inherited selection) - [Source] test_source2/fs2: Ignored (only childs) - [Source] test_source2/fs2/sub: Selected (inherited selection) - - #### Snapshotting - [Source] Creating snapshots test-20200710125958 in pool test_source1 - [Source] Creating snapshots test-20200710125958 in pool test_source2 - - #### Thinning source - [Source] test_source1/fs1@test-20200710125948: Destroying - [Source] test_source1/fs1/sub@test-20200710125948: Destroying - [Source] test_source2/fs2/sub@test-20200710125948: Destroying - - #### All operations completed successfully - (No target_path specified, only operated as snapshot tool.) -``` - -This also allows you to make several snapshots during the day, but only backup the data at night when the server is not busy. - -**Note**: In this mode it doesnt take a specified target-schedule into account when thinning, it only knows a snapshot is the common snapshot by looking at the holds. So make sure your source-schedule keeps the snapshots you still want to transfer at a later point. +Only use this if you dont want to make any backup at all or if a target isn't reachable during the snapshotting phase. ### Specifying ssh port or options