The most current version is 3.10.8.0
Ubuntu Linux (Intel 64bit)
- The implication is that Virtual Backup can literally turn any network or cloud storage allowing only basic file operations into a sophisticated deduplication server. Currently 5 storage backends are supported, including local or NFS-mounted disk, SFTP server, Amazon S3 storage, Wasabi, Microsoft Azure, and Backblaze.
- NFS. No Mount Options FTP. Read-only Download on the Mac App Store 'Pro Setting' features require In-App Purchase + Helper Required Mounting into /Volumes does not require the Pro Settings In-App Purchase, but Helper is Required System Requirements: macOS 10.12+ AutoMounter has some nifty features to support schools through to enterprise.
The most current version is 3.10.8.0 Ubuntu Linux (Intel 64bit) Required: Quickstart Guide: here SDFS Binaries: Redhat Linux (Intel 64bit) Required: Quickstart Guide: here SDFS Binaries: Window.
Redhat Linux (Intel 64bit)
Windows (Intel 64bit)
Backup Exec Supported Downloads
Virtual NAS Appliance (5.2)
This module implements data backup for NethServer using different engines.
Available engines:
- duplicity: execute a full backup once a week, an incremental snapshot all other days of the week. Compression is enabled by default, encryption is not currently supported.It supports only storage backend which can be mounted on a local directory. Used also for the single backup.
- restic: always execute incremental backups using file deduplication. Encryption is always enabled, compression is not implemented.It supports local and remote backends.
- rsync: Time Machine-style backup using rsync. Very fast and reliable, the destination contains just regular files which can be easily accessed.
The nethserver-backup-data
package requires nethserver-backup-config
.
Configuration¶
Backups can be scheduled in different hours to multiple storage backends.
Global properties¶
This configuration is applied to all backups.It uses the key backup-data
inside configuration
database.
Properties:
IncludeLogs
: if enabled, add/var/log
directory to backup, can beenabled
ordisabled
. Default isdisabled
.
Local properties¶
This configuration is applied only to the selected backups.Every backup record is saved inside the backups
database. Each record can have 3 different types:
duplicity
restic
Backblaze Nfs Underground 2
rsync
The key of the record is referred as the backup name
.
Common properties:
status
: enable or disable the automatic backup, can beenabled
ordisabled
. Default isenabled
. Regardless of this property, the backup is always executed if started manuallyBackupTime
: time of the scheduled backup. Must be in the cron-style syntax: Es.157***
. Runs on 7:15.VFSType
: set the backup medium, can beusb
,cifs
,nfs
orwebdav
.SMBShare
: contains the Samba share nameSMBHost
: host name of the SMB serverSMBLogin
: login user for the SMB serverSMBPassword
: password for the SMB serverUSBLabel
: contains the filesystem labelNFSHost
: host name of the NFS serverNFShare
: contains the NFS share nameNotify
: if set toalways
, always send a notification with backup status; if set toerror
, send a notification only on error; if set tonever
, never send a notificationNotifyTo
: send the notification to given mail address, default isroot@localhost
WebDAVUrl
: contains the WebDAV URL addressWebDAVLogin
: login user for the WebDAV serverWebDAVPassword
: password for the WebDAV serverCleanupOlderThan
: time to retain backups, accept duplicity syntax (eg. 7D, 1M). Default is: never (keep all backups)
Supported VFSType:
cifs
: save the backup on a remote SMB server. Authentication is mandatory.nfs
: save the backup on a remote NFS server. No authentication supported.usb
: save the backup on a USB device. The device must have a writable filesystem with a custom label.When the backup is started, the system will search for an USB device with the filesystem label saved inUSBLabel
.webdav
: save the backup on a WebDAV server. When using a secure connection make sure the target WebDAV server has a valid SSL certificate, otherwise the system will fail mounting the filesystem.
Backward compatibility¶
To retain the backward compatibility with the old “single backup” feature, a backup named backup-data
has the following special features:
- can have a
NotifyFrom
prop to specify the sender address of notification mail - the backup can be modified from the old Server Manger and the status is reported inside the dashboard
- can be selectively restored using nethserver-restore-data package
Backup¶
The main command is /sbin/e-smith/backup-data-b<name>
which starts the backup process. The backup is composed of three parts:
- pre-backup-data event: prepare the system (eg. dump of mysql tables)
- /etc/e-smith/events/actions/backup-data-<program> action: execute the backupThis actions must implement full/incremental logic and should also take care to mount and umount the destination
- post-backup-data: cleanup. Actions in this event can also implement retention policies
Logs and wrapper¶
Everything is logged to standard output and standard error.
If the backup is executed using backup-data-wrapper
,a new log will be created inside /var/log/backup
.
After backup execution, the wrapper will also callall executable scripts inside /etc/backup-data.hooks/
directory.
Each script is invoked with the following parameters:
- backup name
- log file
- backup exit code
Default hooks¶
backup-dashboard-status
: save the status of the backup/var/spool/backup/status-<backup_name>
backup-notify
: send backup notifications mails
Adding a backup¶
Create a backup record with all required options. Example: create a restic backup named
t1
using sFTP backend:Enable the configuration:
Start a backup¶
Start the backup, by passing the name of the backup to backup-data
command. Example:
Disk usage¶
Each backup script creates statistics about disk utilization on the backup destination.Statistics are available only for: cifs, nfs and usb.
Data are saved inside /var/spool/backup/disk_usage-<backup_name>
.
Indexing¶
In the pre-backup-data event the disk analyzer (Duc) make an indexing of filesystem, useful to create the graphical tree.
The name of the actions is /etc/e-smith/events/actions/nethserver-restore-data-duc-index
and it compose the JSON file to createthe navigable graphic tree.
The indexing feature is limited to the backup named backup-data
.
Customization¶
Global¶
Add custom include/exclude inside following files:
/etc/backup-data.d/custom.include
/etc/backup-data.d/custom.exclude
This configuration is applied to all backups.
Local¶
Each backup can override the global list of include/exclude by creating two special files:
/etc/backup-data/<name>.include
/etc/backup-data/<name>.exclude
Where name
is the name of the backup.
Retention policy¶
All backups can be deleted after a certain amount of time. Cleanup process takes place in post-backup-data event.See CleanupOlderThan
property.
Restore¶
Restore from command line¶
The main command is /sbin/e-smith/restore-data-b<name>
which starts the restore process:
- pre-restore-data event: used to prepare the system (Eg. mysql stop)
- restore-data-<program> action: search for a backup in the configuration database and restore it
- post-restore-data event: used to inform programs about new available data (eg. mysql restart)
To restore all data into the original location, use:
To restore a file or directory, use:
List backup contents¶
The list of file inside each backup can be obtained using:
Duplicity¶
The default program used for backup is duplicity using the globbing file list feature. Encryption is disabled and duplicity cache is stored in /var/lib/nethserver/backup/duplicity/directory
.We plan to support all duplicity features in the near future.
See http://duplicity.nongnu.org/ for more information.
Extra options¶
Properties valid only for duplicity engine, see “Single backup” section for an explanation of each property:
Type
: can befull
orincremental
. Iffull
, a full backup will be executed every time.Ifincremental
, a full backup will be executed once a week atFullDay
, all other backups will be incrementalFullDay
: number of day of the week when a full backup will be executed. Can be a number from 0 (Sunday) to 6 (Saturday). Defaults is0
.VolSize
: size of chunks in MB, if supported byProgram
. Default is 250
Storage backends¶
Supported VFSType
:
usb
cifs
nfs
webdav
Listing backup sets¶
To list current backup sets:
- Mount the backup directory
- Query duplicity status
- Umount the backup directory
Just execute:
Restic¶
Implement backup engine using restic (https://restic.net/), it can be used as duplicity replacement for standardbackup or as multiple backup.
In restic, cleanup operations are composed by two commands: forget, to remove a snapshot, and prune, to actually remove the datathat was referenced by the deleted snapshot.The prune operation is quite slow and should be executed at least once a week.
Extra options¶
Prune
: execute the pruning on the specified time. Valid values are:always
: run the prune every time at the end of backup- a number between
0
and6
: run the prune on the selected week day (0 is Sunday, 1 is Monday)
Storage backends¶
Supported VFSType
:
usb
cifs
nfs
webdav
s3
: Amazon S3 (or compatible server like Minio)sftp
: FTP over SSHb2
: BackBlaze B2rest
: Restic REST server
sftp¶
SFTP
Connection to remote host uses a specific public key. A password is needed only once to copy the public key to the remote host.SSH client configuration is added to /etc/ssh/sshd_config
.
Properties:
SftpHost
: SSH host name or IP addressSftpUser
: SSH userSftpPort
: SSH portSftpDirectory
: destination directory, must be writable by SSH user
Example:
The temporary file containing the password will be deleted at the end of nethserver-backup-data-save
event.
s3¶
Amazon S3 (https://aws.amazon.com/s3/) compatible (like https://www.minio.io/).
Properties
S3AccessKey
: user access keyS3Bucket
: bucket (directory) nameS3Host
: S3 host, use s3.amazonaws.com for AmazonS3SecretKey
: secret access key
Example:
How to setup Amazon S3 access keys: https://restic.readthedocs.io/en/stable/080_examples.html
b2¶
BackBlaze B2 (https://www.backblaze.com/b2/cloud-storage.html)
Properties:
B2AccountId
: B2 account nameB2AccountKey
: B2 account secret keyB2Bucket
: B2 bucket (directory)
Example:
Backblaze Nfs
Rest¶
Restic REST server (https://github.com/restic/rest-server)
Properties:
RestDirectory
: destination directoryRestHost
: REST server host name or IP addressRestPort
: REST server port (default for server is 8000)RestProtocol
: REST protocol, can behttp
orhttps
RestUser
: user for authentication (optional)RestPassword
: password for authentication (optional)
Example:
Backblaze Nfs Most Wanted
Database example:
REST server¶
To manually install the REST server, download it from https://github.com/restic/rest-server/releases and save itunder /usr/local/bin/rest-server
, example Linux 64bit:
Then configure it for NethServer:
rsync¶
Implement Time machine-style backup engine using rsync_tmbackup.sh
(https://github.com/laurent22/rsync-time-backup),based on rsync (https://rsync.samba.org/). It can be used as duplicity replacement for standardbackup or as multiple backup.
Retention policy¶
Backup sets are automatically deleted when the disk is getting full.
More info on expiration strategy: https://github.com/laurent22/rsync-time-backup#backup-expiration-logic
Storage backends¶
Supported VFSType
:
usb
cifs
nfs
webdav
sftp
: FTP over SSH
sftp¶
SFTP
Connection to remote host uses a specific public key. A password is needed only once to copy the public key to the remote host.SSH client configuration is added to /etc/ssh/sshd_config
.
Properties:
SftpHost
: SSH host name or IP addressSftpUser
: SSH userSftpPort
: SSH portSftpDirectory
: destination directory, must be writable by SSH user
Example:
The temporary file containing the password will be deleted at the end of nethserver-backup-data-save
event.
Database example: