patrick/zfs_autobackup
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update

Edwin Eefting
2022-01-07 09:36:33 +01:00
parent b96392a2c1
commit 529f48ea80
2 changed files with 94 additions and 135 deletions
Download Patch File Download Diff File Expand all files Collapse all files

225
Home.md

@ -1,6 +1,7 @@
# Getting started
Look at the [README.md](../blob/master/README.md) for the introduction.
***Wiki is currently being build***
## Installation
@ -255,139 +256,6 @@ Look in man ssh_config for many more options.
* Set the ```readonly``` property of the target filesystem to ```on```. This prevents changes on the target side. (Normally, if there are changes the next backup will fail and will require a zfs rollback.) Note that readonly means you cant change the CONTENTS of the dataset directly. Its still possible to receive new datasets and manipulate properties etc.
* Use ```--clear-refreservation``` to save space on your backup server.
* Use ```--clear-mountpoint``` to prevent the target server from mounting the backupped filesystem in the wrong place during a reboot.
## More information
* [Performance tips (recommended)](Performance%20tips)
* [Common problems and errors](Common%20problems)
* [Thinning out obsolete snapshots](Thinner)
* [Handling ZFS encryption](Encryption)
* [Transfer buffering, compression and rate limiting.](Transfer%20buffering,%20compression%20and%20rate%20limiting)
* [Custom Pre- and post-snapshot commands](Pre%20and%20post%20snapshot%20commands)
## Usage
```console
usage: zfs-autobackup [-h] [--ssh-config CONFIG-FILE] [--ssh-source USER@HOST]
[--ssh-target USER@HOST] [--keep-source SCHEDULE]
[--keep-target SCHEDULE] [--pre-snapshot-cmd COMMAND]
[--post-snapshot-cmd COMMAND] [--other-snapshots]
[--no-snapshot] [--no-send] [--no-thinning] [--no-holds]
[--min-change BYTES] [--allow-empty] [--ignore-replicated]
[--strip-path N] [--clear-refreservation]
[--clear-mountpoint] [--filter-properties PROPERTY,...]
[--set-properties PROPERTY=VALUE,...] [--rollback]
[--destroy-incompatible] [--destroy-missing SCHEDULE]
[--ignore-transfer-errors] [--decrypt] [--encrypt]
[--zfs-compressed] [--test] [--verbose] [--debug]
[--debug-output] [--progress] [--send-pipe COMMAND]
[--recv-pipe COMMAND] [--compress TYPE] [--rate DATARATE]
[--buffer SIZE]
backup-name [target-path]
zfs-autobackup v3.1 - (c)2021 E.H.Eefting (edwin@datux.nl)
positional arguments:
backup-name Name of the backup (you should set the zfs property
"autobackup:backup-name" to true on filesystems you
want to backup
target-path Target ZFS filesystem (optional: if not specified,
zfs-autobackup will only operate as snapshot-tool on
source)
optional arguments:
-h, --help show this help message and exit
--ssh-config CONFIG-FILE
Custom ssh client config
--ssh-source USER@HOST
Source host to get backup from.
--ssh-target USER@HOST
Target host to push backup to.
--keep-source SCHEDULE
Thinning schedule for old source snapshots. Default:
10,1d1w,1w1m,1m1y
--keep-target SCHEDULE
Thinning schedule for old target snapshots. Default:
10,1d1w,1w1m,1m1y
--pre-snapshot-cmd COMMAND
Run COMMAND before snapshotting (can be used multiple
times.
--post-snapshot-cmd COMMAND
Run COMMAND after snapshotting (can be used multiple
times.
--other-snapshots Send over other snapshots as well, not just the ones
created by this tool.
--no-snapshot Don't create new snapshots (useful for finishing
uncompleted backups, or cleanups)
--no-send Don't send snapshots (useful for cleanups, or if you
want a serperate send-cronjob)
--no-thinning Do not destroy any snapshots.
--no-holds Don't hold snapshots. (Faster. Allows you to destroy
common snapshot.)
--min-change BYTES Number of bytes written after which we consider a
dataset changed (default 1)
--allow-empty If nothing has changed, still create empty snapshots.
(same as --min-change=0)
--ignore-replicated Ignore datasets that seem to be replicated some other
way. (No changes since lastest snapshot. Useful for
proxmox HA replication)
--strip-path N Number of directories to strip from target path (use 1
when cloning zones between 2 SmartOS machines)
--clear-refreservation
Filter "refreservation" property. (recommended, safes
space. same as --filter-properties refreservation)
--clear-mountpoint Set property canmount=noauto for new datasets.
(recommended, prevents mount conflicts. same as --set-
properties canmount=noauto)
--filter-properties PROPERTY,...
List of properties to "filter" when receiving
filesystems. (you can still restore them with zfs
inherit -S)
--set-properties PROPERTY=VALUE,...
List of propererties to override when receiving
filesystems. (you can still restore them with zfs
inherit -S)
--rollback Rollback changes to the latest target snapshot before
starting. (normally you can prevent changes by setting
the readonly property on the target_path to on)
--destroy-incompatible
Destroy incompatible snapshots on target. Use with
care! (implies --rollback)
--destroy-missing SCHEDULE
Destroy datasets on target that are missing on the
source. Specify the time since the last snapshot, e.g:
--destroy-missing 30d
--ignore-transfer-errors
Ignore transfer errors (still checks if received
filesystem exists. useful for acltype errors)
--decrypt Decrypt data before sending it over.
--encrypt Encrypt data after receiving it.
--zfs-compressed Transfer blocks that already have zfs-compression as-
is.
--test dont change anything, just show what would be done
(still does all read-only operations)
--verbose verbose output
--debug Show zfs commands that are executed, stops after an
exception.
--debug-output Show zfs commands and their output/exit codes. (noisy)
--progress show zfs progress output. Enabled automaticly on ttys.
(use --no-progress to disable)
--send-pipe COMMAND pipe zfs send output through COMMAND (can be used
multiple times)
--recv-pipe COMMAND pipe zfs recv input through COMMAND (can be used
multiple times)
--compress TYPE Use compression during transfer, defaults to zstd-adapt
if TYPE is not specified. (gzip, pigz-fast, pigz-slow,
zstd-fast, zstd-slow, zstd-adapt, xz, lzo, lz4)
--rate DATARATE Limit data transfer rate (e.g. 128K. requires
mbuffer.)
--buffer SIZE Add zfs send and recv buffers to smooth out IO bursts.
(e.g. 128M. requires mbuffer)
Full manual at: https://github.com/psy0rz/zfs_autobackup
```
## Restore example
@ -399,6 +267,95 @@ root@fs1:/home/psy# zfs send fs1/zones/backup/zfsbackups/smartos01.server.com/z
After that you can rename the disk image from the temporary location to the location of a new SmartOS machine you've created.
## More information
* [Performance tips (recommended)](Performance%20tips)
* [Common problems and errors](Common%20problems)
* [Thinning out obsolete snapshots](Thinner)
* [Handling ZFS encryption](Encryption)
* [Transfer buffering, compression and rate limiting.](Transfer%20buffering,%20compression%20and%20rate%20limiting)
* [Custom Pre- and post-snapshot commands](Pre%20and%20post%20snapshot%20commands)
## Usage
```console
usage: zfs-autobackup [-h] [--ssh-config CONFIG-FILE] [--ssh-source USER@HOST] [--ssh-target USER@HOST] [--keep-source SCHEDULE] [--keep-target SCHEDULE] [--pre-snapshot-cmd COMMAND]
[--post-snapshot-cmd COMMAND] [--other-snapshots] [--no-snapshot] [--no-send] [--no-thinning] [--no-holds] [--min-change BYTES] [--allow-empty] [--exclude-unchanged] [--exclude-received]
[--strip-path N] [--clear-refreservation] [--clear-mountpoint] [--filter-properties PROPERTY,...] [--set-properties PROPERTY=VALUE,...] [--rollback] [--destroy-incompatible]
[--destroy-missing SCHEDULE] [--ignore-transfer-errors] [--decrypt] [--encrypt] [--zfs-compressed] [--test] [--verbose] [--debug] [--debug-output] [--progress] [--send-pipe COMMAND]
[--recv-pipe COMMAND] [--compress [TYPE]] [--rate DATARATE] [--buffer SIZE] [--snapshot-format FORMAT] [--property-format FORMAT] [--hold-format FORMAT] [--version]
[BACKUP-NAME] [TARGET-PATH]
zfs-autobackup v3.1.1 - (c)2021 E.H.Eefting (edwin@datux.nl)
positional arguments:
BACKUP-NAME Name of the backup (you should set the zfs property "autobackup:backup-name" to true on filesystems you want to backup
TARGET-PATH Target ZFS filesystem (optional: if not specified, zfs-autobackup will only operate as snapshot-tool on source)
optional arguments:
-h, --help show this help message and exit
--ssh-config CONFIG-FILE
Custom ssh client config
--ssh-source USER@HOST
Source host to get backup from.
--ssh-target USER@HOST
Target host to push backup to.
--keep-source SCHEDULE
Thinning schedule for old source snapshots. Default: 10,1d1w,1w1m,1m1y
--keep-target SCHEDULE
Thinning schedule for old target snapshots. Default: 10,1d1w,1w1m,1m1y
--pre-snapshot-cmd COMMAND
Run COMMAND before snapshotting (can be used multiple times.
--post-snapshot-cmd COMMAND
Run COMMAND after snapshotting (can be used multiple times.
--other-snapshots Send over other snapshots as well, not just the ones created by this tool.
--no-snapshot Don't create new snapshots (useful for finishing uncompleted backups, or cleanups)
--no-send Don't send snapshots (useful for cleanups, or if you want a serperate send-cronjob)
--no-thinning Do not destroy any snapshots.
--no-holds Don't hold snapshots. (Faster. Allows you to destroy common snapshot.)
--min-change BYTES Number of bytes written after which we consider a dataset changed (default 1)
--allow-empty If nothing has changed, still create empty snapshots. (same as --min-change=0)
--exclude-unchanged Exclude datasets that have no changes since any last snapshot. (Useful in combination with proxmox HA replication)
--exclude-received Exclude datasets that have the origin of their autobackup: property as "received". This can avoid recursive replication between two backup partners.
--strip-path N Number of directories to strip from target path (use 1 when cloning zones between 2 SmartOS machines)
--clear-refreservation
Filter "refreservation" property. (recommended, safes space. same as --filter-properties refreservation)
--clear-mountpoint Set property canmount=noauto for new datasets. (recommended, prevents mount conflicts. same as --set-properties canmount=noauto)
--filter-properties PROPERTY,...
List of properties to "filter" when receiving filesystems. (you can still restore them with zfs inherit -S)
--set-properties PROPERTY=VALUE,...
List of propererties to override when receiving filesystems. (you can still restore them with zfs inherit -S)
--rollback Rollback changes to the latest target snapshot before starting. (normally you can prevent changes by setting the readonly property on the target_path to on)
--destroy-incompatible
Destroy incompatible snapshots on target. Use with care! (implies --rollback)
--destroy-missing SCHEDULE
Destroy datasets on target that are missing on the source. Specify the time since the last snapshot, e.g: --destroy-missing 30d
--ignore-transfer-errors
Ignore transfer errors (still checks if received filesystem exists. useful for acltype errors)
--decrypt Decrypt data before sending it over.
--encrypt Encrypt data after receiving it.
--zfs-compressed Transfer blocks that already have zfs-compression as-is.
--test, --dry-run, -n
Dry run, dont change anything, just show what would be done (still does all read-only operations)
--verbose, -v verbose output
--debug, -d Show zfs commands that are executed, stops after an exception.
--debug-output Show zfs commands and their output/exit codes. (noisy)
--progress show zfs progress output. Enabled automaticly on ttys. (use --no-progress to disable)
--send-pipe COMMAND pipe zfs send output through COMMAND (can be used multiple times)
--recv-pipe COMMAND pipe zfs recv input through COMMAND (can be used multiple times)
--compress [TYPE] Use compression during transfer, defaults to zstd-fast if TYPE is not specified. (gzip, pigz-fast, pigz-slow, zstd-fast, zstd-slow, zstd-adapt, xz, lzo, lz4)
--rate DATARATE Limit data transfer rate (e.g. 128K. requires mbuffer.)
--buffer SIZE Add zfs send and recv buffers to smooth out IO bursts. (e.g. 128M. requires mbuffer)
--snapshot-format FORMAT
Snapshot naming format. Default: {}-%Y%m%d%H%M%S
--property-format FORMAT
Select property naming format. Default: autobackup:{}
--hold-format FORMAT Hold naming format. Default: zfs_autobackup:{}
--version Show version.
Full manual at: https://github.com/psy0rz/zfs_autobackup
```
## Monitoring with Zabbix-jobs
You can monitor backups by using my zabbix-jobs script. (<https://github.com/psy0rz/stuff/tree/master/zabbix-jobs>)

4
_Sidebar.md

@ -1,5 +1,7 @@
zfs-autobackup:
* [Getting started]()
* [Introduction](../blob/master/README.md)
* [Getting started](Home)
* [Performance tips (recommended)](Performance%20tips)
* [Common problems and errors](Common%20problems)
* [Thinning out obsolete snapshots](Thinner)