Note

You can download this article as a PDF

CitrixHypervisor (XenServer) plugin

Important

Remember to read the Best Practices chapter common for all of our hypervisor plugins.

Features Summary

  • Snapshot-based online backup of any guest VM

  • VSS-based guest snapshots for quiescing VSS-based applications

  • Full, Incremental and Differential block level image backup

  • Ability to restore complete virtual machine image

  • Ability to restore VM archive (.xva) to an alternate directory

  • Support for CitrixHypervisor and XCP-ng

Note

This plugin was introduced with Bacula Enterprise version 10.0.

Incremental and Differential backup support was introduced with version 12.4.1.

Single item restore for XenServer VMs was introduced with Bacula Enterprise version 12.8.8.

Guest VM Backup Strategies

Installing Bacula Client on each Guest

With this first strategy, you do not use the Bacula Enterprise XenServer Plugin, but instead install a Bacula Enterprise File Daemon on every virtual machine as if they were normal physical clients. In order to optimize the I/O usage on your XenServer hypervisor, you will use Bacula’s Schedules, Priorities, and Maximum Concurrent Jobs to spread your backup jobs over your backup window. Since all VMs could use the same storage on the XenServer hypervisor, running all your backup jobs at the same time could create a bottleneck on the disk/network subsystem.

Installing a Bacula Enterprise File Daemon on each virtual machine permits you to manage your virtual servers like physical servers and also to use all Bacula Enterprise’s features such as:

  • Quick restores of individual files.

  • Checksum of individual files for Virus and Spyware detection.

  • Verify Jobs.

  • File/Directory exclusion (such as swap or temporary files).

  • File level compression.

  • Accurate backups.

  • etc.

Image Backup With XenServer Plugin

With the image backup level strategy, the Bacula Enterprise XenServer Plugin will save the Client disks at the raw level, in the XenServer context.

For this to work, you don’t need a Bacula File Daemon on each guest VM. Bacula’s XenServer plugin will contact your XenServer hypervisor to read and save the contents of your virtual machine disks using snapshots and XAPI. In this case Plugin can perform full range of block level image backup including Incremental and Differential ones.

Bacula doesn’t need to walk through the Client filesystem to open/read/close/stat files, so it consumes less resources on your XenServer infrastructure than a file level backup on each guest machine would. On the other hand, Bacula will also read and save useless data such as swap files or Internet temporary files. The XenServer Plugin will save not only the disk images of the:term:guest VM, but also the guest VM configurations which allows for very easy:term:guest VM restores.

Backup and Restore Operations

Backup

The backup operation of a single guest VM takes the following steps:

  • Find and delete any old backup snapshots list in Full level backup.

  • Create a new guest VM Bacula snapshot and prepare it for backup.

  • Export VM Guest metadata configuration for future restore.

  • For any Incremental or Differential backups compute block changed list for every virtual disk in snapshot.

  • Export all raw images data or data based on changed block list if required.

  • Execute the XenServer vm-export command and save the data to a Bacula storage daemon.

  • Delete a backup snapshot data and maintaining a snapshot metadata only.

Backup snapshot set.

Backup snapshot set.

Backups can be performed for a guest VM in any power state (running or halted). For proper execution of Incremental or Differential backups it is required by XenServer to maintain snapshot metadata which stores changed block information used to compute changed block list during backup. You can find it and display on VDI objects list. Every single backup for single VM will create single metadata VDI snapshot for every VDI attached to guest VM.

Metadata snapshots take minimal space and cannot be used directly for restore. They save a changed block bitmaps and no real data blocks. Every CBT-enabled disk has an additional CBT-metadata disk which is named as <vdi_uuid>.cbtlog, on the same SR.

  • Size of a CBT-metadata disk on LVM based SRs is 4MB

  • Size of a CBT-metadata disk on file based SRs is proportional to the size of the VDI, and can grow up to a size of 4MB (for a 2TB VDI)

Blocks of 64 kB within the VDI are tracked and changes to these blocks recorded in the log layer.

Metadata snapshots free space.

Metadata snapshots free space.

Any guest VM snapshot with a name-label which matches the following template: BaculaSnapshot_<UUID>_JobID_<NR> or VM Guest VDI snapshot <VDI-name-label>:BaculaSnapshot__JobID_<NR> will be treated as an old backup snapshot for this VM Guest and automatically deleted during backup (VDI snapshots during Full backups). You should avoid creating a such snapshots manually.

Any other guest VM snapshots will be unaffected. The XenServer Plugin will inform you about every guest VM backup start and finish including information about old stalled backup snapshots and backup snapshot activities. For example:

JobId 1936: xenapi: Start Backup vm: vmtest1 (8024379c-c753-872a-5c25-6c815ee617b4)
JobId 1936: xenctx: Cleaning old snapshots ...
JobId 1936: xenctx: Snapshot created: 5af34331-c6f2-e11b-c2c5-f1482c779eda
JobId 1936: xenapi: Finish backup of vmtest1-disk0:8066b4e4-9b42-4ed7-b908-3494c9bd9094
...
VM snapshot during backup.

VM snapshot during backup.

The backup will create a following backup files during backup:

  • a single file for VM configuration metadata saved in the form of: /@xen/<name-label>/<vmuuid>.conf.

  • a single file for VM disks configuration saved in the form of: /@xen/<name-label>/<vmuuid>.vmdisks.

  • a single file for every VM Guest VDI saved in the form of: /@xen/<name-label>/<vmuuid>/<vdi-name:vdi-uuid>.vdi.

Multiple files will be created during backup if multiple VM Guest found to backup. You can use this information to locate the proper VM Guest archive during restore.

+---------------------------------------------------------------------------------------------------+
| filename                                                                                          |
+---------------------------------------------------------------------------------------------------+
| /@xen/vmtest1/8024379c-c753-872a-5c25-6c815ee617b4.conf                                           |
| /@xen/vmtest1/8024379c-c753-872a-5c25-6c815ee617b4.vmdisks                                        |
| /@xen/vmtest1/8024379c-c753-872a-5c25-6c815ee617b4/disk0:8066b4e4-9b42-4ed7-b908-3494c9bd9094.vdi |
+---------------------------------------------------------------------------------------------------+

XenServer Backup preparation

Before you start configuring your Backup Jobs you need to configure your XenServer to allows proper network operations with Bacula Enterprise. This part of the prerequisites includes:

  1. enable network access to the XenServer API from the backup server - HTTP/HTTPS - ports tcp:80 and tcp:443

  2. enable network access to the XenServer NBD service from backup server - NBD/NBD-SSL - port tcp:10809

  3. enable the XAPI NBD server on required network

To enable network access to the XenServer API you should enable and verify the firewall rules:

# iptables -L|grep http
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:https

To enable the XAPI NBD Server you should first check available virtual networks:

# xe network-list
uuid ( RO)                : 774e87dd-a096-827a-5a30-6ef9123cfd7b
          name-label ( RW): Pool-wide network associated with eth0
    name-description ( RW):
              bridge ( RO): xenbr0

uuid ( RO)                : 9fa3ecb2-6493-2849-e30b-eb772d1dbc1c
          name-label ( RW): Host internal management network
    name-description ( RW): Network on which guests will be assigned a private link-local IP
              bridge ( RO): xenapi

Then you should enable NBD by setting up a nbd-purpose on selected networks. You can enable the NBD service in FORCEDTLS or NOTLS mode. You cannot have a mix of normal NBD (FORCEDTLS) and insecure NBD (NOTLS) networks. To switch the purpose of all networks, you must first disable normal NBD connections on all networks before enabling either normal or insecure NBD connections on any network.

Note

The XenServer vendor recommend to use TLS with NBD connections. When NBD connections with TLS are enabled, any NBD clients that attempt to connect to the XenServer must use TLSv1.2.

The Bacula Enterprise XenServer Plugin will work in either of the available NBD modes. To enable NBD connections with or without TLS, use the purpose parameter of the network. Set this parameter to include the value nbd for FORCEDTLS mode and the value insecure_nbd for NOTLS mode. Ensure that you wait for the setting to propagate before attempting to use this network for NBD connections. The time it takes for the setting to propagate depends on your network and is at least 10 seconds.

Below you can find an example of how to enable the NBD service in FORCEDTLS mode for selected <network-uuid>.

# xe network-param-add param-name=purpose param-key=nbd uuid=<network-uuid>

Some examples below.

 # Remove Insecure
    [19:14 po-xcp-ngserver home]# xe network-param-remove param-name=purpose param-key=insecure_nbd uuid=b17e189c-5b13-feb6-c5cf-e509153ce3f0
    [19:15 po-xcp-ngserver home]# xe network-param-remove param-name=purpose param-key=insecure_nbd uuid=53265e56-8ca0-db10-2c78-e80c89e2e4dd

# Enable Secure
    [19:15 po-xcp-ngserver home]# xe network-param-add param-name=purpose param-key=nbd uuid=e80f1afe-ce32-af6e-a8d2-3047350d44ef
    [19:15 po-xcp-ngserver home]# xe network-param-add param-name=purpose param-key=nbd uuid=b17e189c-5b13-feb6-c5cf-e509153ce3f0
    [19:15 po-xcp-ngserver home]# xe network-param-add param-name=purpose param-key=nbd uuid=53265e56-8ca0-db10-2c78-e80c89e2e4dd

 # Restart Xapi
    systemctl restart xapi

For NOTLS mode you should replace param-key=nbd with param-key=insecure_nbd. You can check the network configuration as follows:

# xe network-param-list uuid=774e87dd-a096-827a-5a30-6ef9123cfd7b
uuid ( RO)                    : 774e87dd-a096-827a-5a30-6ef9123cfd7b
              name-label ( RW): Pool-wide network associated with eth0
        name-description ( RW):
               VIF-uuids (SRO): e61b768a-6460-af03-abf5-a2361b5b2f36;
               PIF-uuids (SRO): cbeab33d-b1c9-6fcb-bf03-d434c8623628
                     MTU ( RW): 1500
                  bridge ( RO): xenbr0
                 managed ( RO): true
            other-config (MRW):
                   blobs ( RO):
                    tags (SRW):
    default-locking-mode ( RW): unlocked
                 purpose (SRW): insecure_nbd

If you want to use the NBD service in FORCEDTLS mode you should setup the Bacula Enterprise plugin using the secure nbd configuration parameters to use a TLS certificate. Check the certfile, keyfile, cacertfile and tlshostname parameters for more information.

To use secure, TLS-encrypted and authenticated transport, note that you need to use a certificate and related key trusted by the Xen or XCP-ng server. If you are running a XCP-ng environment or a recent XenServer version, this will be mandatory, as any access to the APIs are protected by default with TLS and a self-signed certificate. More information:

If you plan to override your server certificates and use it with this plugin, below you will find an example of how to generate and install a self-signed one, but note that in most cases, these steps would involve a properly managed Certificate Authority and specific procedures:

# Generate a Certificate with its key
backupteam@example.org:~/certs# openssl req -new -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -out MyCertificate.crt -keyout MyKey.key
Generating a RSA private key
................................................................................................++++
..............................................++++
writing new private key to 'MyKey.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:
Email Address []:

# Copy Certificates to XCP-NG Server
scp MyKey.key  root@My.Sample.Xen.Server:/tmp
scp MyCertificate.crt  root@My.Sample.Xen.Server:/tmp

# Generate a Pem Key on XCP-NG server
openssl req -new -x509 -key /etc/xensource/xapi-ssl.pem -subj '/CN=XCP-ng hypervisor/' -out xcp-ng.csr

# Install Certificates on XCP-NG Server
xe host-server-certificate-install certificate=/tmp/MyCertificate.crt private-key=/tmp/MyKey.key certificate-chain=/home/xcp-ng.csr

# Now use that certificate and key with the plugin parameters 'certfile' and 'keyfile'

Restore

The XenServer Plugin provides three main targets for restore operations:

  • Restore to XenServer system as new VM Guest

  • Restore to local directory as a number of archive files

  • Restore individual files

Restore to XenServer

To use this restore method you have to set a “where=/” Bacula restore parameter. The guest VM archive will be sent to the XenServer hypervisor and always restored as a new guest VM. You can change the Storage Repository where your VM Guest will be restored.

To list available Storage Repositories you can use a listing mode, see listing. If you set an improper (eg: non-existent) Storage Repository for restore, then the restore process will fail. The Restore process requires a block level patching of saved disks which has to be performed on local filesystem (default directory: /$WorkingDirectory/xenapi/$JobID/).

Note

This is a XenServer limitation as XAPI does not support block level incremental restore but full image restore only.

Restore To Local Directory

To use this restore method you have to set a where=/some/path Bacula restore parameter. The path has to be a directory location on the server where the XenServer plugin is installed. If the path doesn’t exist, it will be created by the XenServer Plugin.

Installation

You have to install and configure the Bacula File Daemon on your Backup machine which has access to XenServer API and XAPI-NBD services.

Note

It is not recommended to install it on Hypervisor dom0 as some XenServer services will not work properly:

“NDB connections do not work when an NBD client is in dom0 on the same host as the NBD server.”

(source: https://support.citrix.com/article/CTX230619/how-to-troubleshoot-changed-block-tracking-in-xenserver)

As all backup interactions are network based, any Bacula Enterprise File Daemon with access to the necessary XenServer endpoints can be used to run the plugin.

Important

Since Bacula Enterprise 16.0.5, the XenServer Plugin is available for Debian 11 (bullseye).

Configuration

The Plugin Directory directive of the File Daemon resource in /opt/bacula/etc/bacula-fd.conf should point where the xenserver-fd.so plugin is installed. The standard Bacula plugin directory is: /opt/bacula/plugins

FileDaemon {
  Name = bacula-fd
  Plugin Directory = /opt/bacula/plugins
...
}

Installation of the Plugin

An example for a Debian based Linux distributions would be a file

/etc/apt/sources.list.d/bacula.list with the following content:

# Bacula Enterprise
deb https://www.baculasystems.com/dl/@cust@/debs/bin/@ver@/bullseye-64/ bullseye main
deb https://www.baculasystems.com/dl/@cust@/debs/xenserver/@ver@/bullseye-64/ bullseye xenserver

After that, a run of apt-get update is needed. Then, the Plugin can be installed using apt-get install bacula-enterprise-xenserver-plugin

Plugin Configuration

The plugin is configured using Plugin Parameters defined in a FileSet’s “Include” section of the Bacula Enterprise Director’s configuration.

Generic Plugin Parameters

The following XenServer plugin parameters effect any type of Job (Backup, Estimation, or Restore).

url=<address>

specifies the XenServer API url address used for operations. This parameter is optional. If omitted the parameter will be assembled from server=… and port=… parameters defined below. You have to define the one of url or server parameters to connect to the XenServer API.

server=<address>

specifies the XenServer API address used for operations. This parameter is optional if url=… parameter above is defined. This is the address used in xe command as -s <address> parameter during restore in legacy mode.

port=<number>

specifies the XenServer API port used for operations. The value of the parameter have to be in range 1..65536. Invalid value will abort the job. This parameter is optional. If omitted the default 80/http access will be used. This is the value used in xe command as -p <number> parameter during restore in legacy mode.

user=<string>

specifies the user name used to access the XenServer API. This parameter is required. This is the value used in xe command as -u <string> parameter during restore in legacy mode.

password=<string>

specifies the password used to access the XenServer API. This parameter is optional if passfile=… is provided else it is required. This is the value used in xe command as -pw <string> parameter during restore in legacy mode. It is advised to use the passfile= option for more security.

passfile=<string>

specifies a file local to the File Daemon that contains the password for the user name. This parameter is optional if password=… is provided else it is required. This is the value used in xe command as -pwf <string> parameter during restore in legacy mode.

abort_on_error[=<0 or 1>]

specifies whether or not the plugin should abort it’s execution if a fatal error happens during Backup, Estimation or Restore. This parameter is optional. The default value is 0.

ignore_ssl[=<0 or 1>]

specifies whether or not the plugin should ignore SSL certificate checking when connecting to XenServer API. This parameter is optional. The default value is 0. With NBD-SSL configuration this parameter is not needed.

certfile=<string>

Enable NBD-SSL communication during backup and use the specified file as the client certificate for TLS authentication to the server. This parameter is optional.

cacertfile=<string>

Enable NBD-SSL communication during backup and use the specified file as the CA certificate for TLS authentication to the server. This parameter is optional.

keyfile=<string>

Enable NBD-SSL communication during backup and use the specified file as the private key for the client cerificate. This parameter is optional.

tlshostname=<string>

Enable NBD-SSL communication during backup and use the specified hostname for the TLS context. If not specified, the hostname used to connect to the server will be used. This parameter is optional even if NBD-SSL connection is desired and normally it is not needed, but if you get any ‘tls hostname not matching’ kind of error, set a hostname here that is accepted in your TLS certificate CN values, as the behavior of the underlying NBD tools can vary among different environments.

Important

certfile, cacertfile, keyfile and tlshostname are available since Bacula Enterprise 18.0.4.

Estimation and Backup Plugin Parameters

vm=<name-label>

specifies a guest VM name to backup. All guest VMs with a name-label provided will be selected for backup. Multiple vm=... parameters may be provided. If a guest VM with <name-label> can not be found, then a single job error will be generated and the backup will proceed to the next VM unless abort_on_error is set which will cause the backup job to be aborted. This parameter is optional.

uuid=<uuid>

specifies a guest VM UUID to backup. Multiple uuid=... parameters may be provided. If a guest VM with <uuid> can not be found, then a single job error will be generated and the backup will proceed to the next VM unless abort_on_error is set which will cause the backup job to be aborted. This parameter is optional.

include=<name-label-regex>

specifies list of a guest VM names to backup using regular expression syntax. All guest VMs which match the name-label-regex provided will be selected for backup. Multiple include=... parameters may be provided. If no guest VMs match the <name-label-regex> provided, the backup will proceed to the next VM parameters or finish successfully without backing up any VMs. The abort_on_error parameter will not abort the job when no guest VMs are found using a <name-label-regex>. This parameter is optional.

exclude=<name-label-regex>

specifies list of a guest VMs names which will be excluded from backup using regular expression syntax. All guest VMs which match the name-label-regex provided and were selected for backup using include=... parameters will be excluded. This parameter does not affect any guest VMs selected to backup with vm=... or uuid=... parameters. Multiple exclude=... parameters may be provided. This parameter is optional.

quiesce[=<0 or 1>]

specifies if the guest VM snapshot should be created using a quiesce method or not. The quiesce method is supported by XenServer for Windows OS with Guest-Tools installed only. This is a limitation of the XenServer itself. If the guest VM snapshot with quiesce cannot be created, the whole backup job will be aborted. In this case you should repeat a backup without the quiesce parameter.

If none of the parameters vm=..., uuid=..., include and exclude are specified, all available guest VMs hosted on the XenServer hypervisor will be backed up.

Plugin Restore Parameters

During restore, the XenServer Plugin will use the same parameters which were set for the backup job and saved. Some of them may be changed during the restore process if required. You can change all the parameters described in chapter Generic Plugin Parameters during restore.

  • storage_res: <storage> specifies a XenServer Storage Repository where restored guest VMs will be saved. If not set, then a guest VM will be saved to a XenServer Storage Repository configured as the default. This parameter is optional.

  • preserve: <yes or no> (this option is deprecated and works for legacy (*.xva) restores only) specifies if a restore job should preserve as much guest VM configuration parameters as possible. The default is to create a new VM on restore. A restore job with this preserve option set to ’yes’ could fail if the restore might create duplicate objects on the XenServer hypervisor. This parameter is optional.

FileSet Examples

In the example below, all guest VMs will be backed up.

FileSet {
  Name = FS_XenAll
  Include {
    Plugin = "xenserver: url=http://10.10.10.10/ user=root password=root"
  }
}

In the example below, a single guest VM with a name-label of “VM1” will be backed up.

FileSet {
  Name = FS_Xen_VM1
  Include {
    Plugin = "xenserver: url=http://10.10.10.10/ user=root password=root vm=VM1"
  }
}

The same example as above, but using uuid instead:

FileSet {
  Name = FS_Xen_VM1
  Include {
    Plugin = "xenserver: url=(...) uuid=fe1ccf3b-1865-3942-c928-d98138397ff1"
  }
}

where: url=(...) is a short for: url=http://10.10.10.10/ user=root password=root above and below.

In the example below, all guest VMs which have ’Prod’ in the name will be backed up.

FileSet {
  Name = FS_Xen_ProdAll
  Include {
    Plugin = "xenserver: url=(...) include=Prod"
  }
}

In the example below, all guest VMs except VMs whose name-label begins with “Test” will be backed up.

FileSet {
  Name = FS_Xen_AllbutTest
  Include {
    Plugin = "xenserver: url=(...) include=.* exclude=^Test"
 }
}

In the example below, we connect through NBD-SSL mode to backup a XenExampleVM host:

Fileset {
   Name = "Xen-Example-Secure-VM"
   EnableVss = no
   IgnoreFilesetChanges = no
   Include {
      Options {
         Signature = Sha256
      }
      Plugin = "xenserver: server=\"my-xcp-server.example.org\" user=\"root\" password=\"SamplePass\" abort_on_error include=XenExampleVM certfile=\"/root/certs/MyCertificate.crt\""
   }
}

Restore

Restore to a XenServer Hypervisor

To restore a VM or VMs to a XenServer hypervisor, you should execute the restore command and specify the “where” parameter as in this example:

* restore where=/
     ...

Then set any other required restore plugin parameters for your restore.

In the following restore session example, the “Preserve vm config on restore” plugin restore option is set to “yes”:

* restore where=/
...
Run Restore job
JobName:         RestoreFiles
Bootstrap:       /opt/bacula/working/srv-xen-01-dir.restore.2.bsr
Where:           /
Replace:         Always
FileSet:         Full Set
Backup Client:   srv-xen-01-fd
Restore Client:  srv-xen-01-fd
Storage:         File1
When:            2018-01-05 12:47:16
Catalog:         MyCatalog
Priority:        10
Plugin Options:  *None*
OK to run? (yes/mod/no): mod
Parameters to modify:
     1: Level
     2: Storage
     3: Job
     4: FileSet
     5: Restore Client
     6: When
     7: Priority
     8: Bootstrap
     9: Where
    10: File Relocation
    11: Replace
    12: JobId
    13: Plugin Options
Select parameter to modify (1-13): 13
Automatically selected : xenserver: uuid=fe1ccf3b-1865-3942-c928-d98138397ff1
Plugin Restore Options
server:              *None*               (*None*)
port:                *None*               (*None*)
user:                *None*               (*None*)
password:            *None*               (*None*)
passfile:            *None*               (*None*)
storage_res:         *None*               (*Default location*)
preserve:            *None*               (*No*)
Use above plugin configuration? (yes/mod/no): mod
You have the following choices:
     1: server (Restore server name)
     2: port (Restore server port number)
     3: user (Restore user name)
     4: password (Restore user password)
     5: passfile (Restore user password file)
     6: storage_res (Storage Resource location for restore)
     7: preserve (Preserve vm config on restore)
Select parameter to modify (1-7): 7
Please enter a value for preserve: yes
Plugin Restore Options
server:              *None*               (*None*)
port:                *None*               (*None*)
user:                *None*               (*None*)
password:            *None*               (*None*)
passfile:            *None*               (*None*)
storage_res:         *None*               (*Default location*)
preserve:            yes                  (*No*)
Use above plugin configuration? (yes/mod/no):

The restore job log will inform you about what guest VM is restored and what new guest VM was created.

JobId 131: Start Restore Job RestoreFiles.2017-12-28_14.42.25_15
JobId 131: Using Device "FileChgr1-Dev2" to read.
JobId 131: xenserver: VM restore: vm1/10908c8a-f932-6f91-9cac-3034e3acf45b
JobId 131: Forward spacing Volume "Vol-0002" to addr=1758441248
JobId 131: Elapsed time=00:04:51, Transfer rate=3.158 M Bytes/second
JobId 131: xenserver: VM UUID created: 45c49e07-ff20-ab55-e622-05ff2fbb0c1f

The new VM Guest created during restore will get the same name-label as the original VM. All VDIs connected to the restored VM will be marked with -restored suffix.

Restore to Local Directory

* restore where=/tmp/bacula/restores

Please check the following example for the test “VM local restore”:

JobId 112: Start Restore Job RestoreFiles.2017-12-28_11.30.19_34
JobId 112: Using Device "FileChgr1-Dev2" to read.
JobId 112: xenserver: VM local restore
JobId 112: Forward spacing Volume "Vol-0001" to addr=5190619786
JobId 112: Elapsed time=00:00:30, Transfer rate=30.64 M Bytes/second

The restore job log will inform you that a restore will go to a local directory.

Other

Resource listing

The Bacula Enterprise XenServer Plugin supports the new Plugin Listing feature of Bacula Enterprise 8.x or newer. This mode allows a Plugin to display some useful information about available XenServer resources such as:

The new feature uses the special .ls command with a new plugin=<plugin> parameter. The command requires the following parameters to be set:

  • client=<client> A Bacula Client name where the XenServer Plugin is installed.

  • plugin=<plugin> A XenServer Plugin name - xenserver: with optional plugin parameters described at Sec. Generic Plugin Parameters

  • path=<path> An object path to display

The supported values for the path=<path> parameter are:

  • / - Display object types available to list

  • vm - Display a list of guest VM name-labels

  • uuid - Display a list of guest VM UUIDs and name-label pointers

  • storage_res - Display a list of Storage Repositories

To display available object types, run the following command example:

*.ls client=srv-xen-01-fd plugin="xenserver: url=http://10.10.10.10/ user=root \
    password=root" path=/
Connecting to Client srv-xen-01-fd at 127.0.0.1:9102
drwxr-x---   1 root     root               0 2018-01-02 09:36:32  vm
drwxr-x---   1 root     root               0 2018-01-02 09:36:32  storage_res
drwxr-x---   1 root     root               0 2018-01-02 09:36:32  uuid
2000 OK estimate files=3 bytes=0

To display the list of all available guest VMs, run the following command example:

*.ls client=srv-xen-01-fd plugin="xenserver: url=http://10.10.10.10/ user=root \
    password=root" path=VM
Connecting to Client srv-xen-01-fd at 127.0.0.1:9102
-rw-r-----   1 root     root      8589934592 2017-12-29 17:12:48  Another-Copy of vm1
-rw-r-----   1 root     root     13958643712 2017-12-29 17:12:48  vm2
-rw-r-----   1 root     root      8589934592 2017-12-29 17:12:48  vm1
-rw-r-----   1 root     root     10737418240 2017-12-29 17:12:48  CentOS 7
-rw-r-----   1 root     root      8589934592 2017-12-29 17:12:48  Copy of vm1 a label with spaces
-rw-r-----   1 root     root     10737418240 2017-12-29 17:12:48  Copy of CentOS 7
-rw-r-----   1 root     root     19327352832 2017-12-29 17:12:48  vm1-orig
2000 OK estimate files=7 bytes=80,530,636,800

To display a list of all available guest VM UUIDs, run the following command example:

*.ls client=srv-xen-01-fd plugin="xenserver: url=http://10.10.10.10/ user=root \
    password=root" path=uuid
Connecting to Client srv-xen-01-fd at 127.0.0.1:9102
...
   8589934592 2018-01-02 09:39:06  4f5c9e10-a3c4-fc29-c967-4981f22d3f86 -> Another-Copy of vm1
  13958643712 2018-01-02 09:39:06  50705972-0a88-5aa7-6721-f70b866ed0b6 -> vm2
   8589934592 2018-01-02 09:39:06  10908c8a-f932-6f91-9cac-3034e3acf45b -> vm1
  10737418240 2018-01-02 09:39:06  fe1ccf3b-1865-3942-c928-d98138397ff1 -> CentOS 7
   8589934592 2018-01-02 09:39:06  c8efc2ca-ca1a-ebdf-5409-5dd8c158e3eb -> Copy of vm1 a label with spaces
  10737418240 2018-01-02 09:39:06  6e84929a-1c52-4c79-c67c-8455f76d3e7c -> Copy of CentOS 7
  19327352832 2018-01-02 09:39:07  03fad8c9-d88b-ea7e-98da-2f3bcd20d0c4 -> vm1-orig
2000 OK estimate files=7 bytes=80,530,636,800

The VM and UUID lists display an estimated size of the guest VM. To display a XenServer Storage Repositories, run the following command example:

*.ls client=srv-xen-01-fd plugin="xenserver: url=http://10.10.10.10/ user=root \
    password=root" path=storage_res
Connecting to Client srv-xen-01-fd at 127.0.0.1:9102
brw-r-----   1 root     root    586081632256 2018-01-02 09:39:22  ISO
brw-r-----   1 root     root    586081419264 2018-01-02 09:39:22  Local storage
brw-r-----   1 root     root               0 2018-01-02 09:39:22  Removable storage
brw-r-----   1 root     root      1073741312 2018-01-02 09:39:22  DVD drives
brw-r-----   1 root     root    586081632256 2018-01-02 09:39:22  Exported Storage
brw-r-----   1 root     root              -1 2018-01-02 09:39:22  XenServer Tools
2000 OK estimate files=6 bytes=0

Single Item Restore

Note

You can download this article as a PDF

While it is technically possible to backup multiple VMs in one Bacula hypervisor plugin backup job (VMware, Hyper-V, RHV, Proxmox, etc), this is not necessarily the best way to perform VM backups. It is strongly recommended that one backup Job is created for each VM being backed up for the following reasons:

  • By default, if one of your VMs fails to backup in a “multi-VM” backup job, the main Bacula job will terminate “Backup OK – with warnings.” The JobStatus for jobs that terminate “Backup OK” and “Backup OK – with warnings” are not differentiated in the catalog. They are both ‘T’, so this means that you will have to carefully monitor your backup job logs in case some VM backups fail and pay attention to the JobErrors field in the job summaries.

  • To address this issue, there is a plugin option called “abort_on_error” in each of the Bacula hypervisor plugins, which causes Bacula to immediately fail the job as soon as an error is detected while backing up a VM. However, if you use this option, and the backup of VM number 11 in a list of 50 VMs fails, then the whole job will be failed, and VMs 12-50 will not be backed up during that job’s run.

  • A 1:1 configuration (one VM backed up per job) means that the “abort_on_error” option will make more sense to enable in each job so you will immediately know when a VM fails to backup since the Bacula job will terminate with a “Backup failed” message and ‘f’ in the catalog for the job.

  • With a 1:1 VM/Job configuration, re-running a specific VM backup job is simple to do after the cause of the failure is investigated and fixed.

  • In the example about the 50 VMs, without a 1:1 configuration, there is no way to re-run a backup of just the one VM that failed to backup.

  • Additionally, with a 1:1 VM/Job configuration, job metrics will have more meaning because each VM will be one job, and you will know to expect a specific number of jobs each night with each job representing one VM.

  • With a multi-VM per job configuration, each VM will be backed up “serially”, one at a time, disk by disk, VM by VM. A 1:1 configuration will allow several VM backups to be run concurrently which will reduce the overall time to perform the VM backups. Of course, you will need to pay close attention to SD and ESXi storage and networking resources, and adjust the number of concurrent jobs accordingly.

  • For some hypervisors (VMware, Proxmox, etc) Bacula provides automation scripts (eg: scan_datacenter.pl for VMware). These scripts are designed so that they will create 1:1 VM/Job configurations. If you plan to make use of these automation scripts, it is a good idea to already be thinking this way, and having your hypervisor plugin backup configurations in a 1:1 configuration from the beginning.

Limitations

  • Snapshot with quiesce backups are only supported for Windows OSes with the XenServer Guest-Tools installed. This is a XenServer limitation and not a limitation of the Bacula Enterprise XenServer Plugin.

  • You cannot run two concurrent backups of the single VM Guest if the later one is a Full backup.

  • You have to provide enough free space at /$workingDirectory/xenapi/ which allows to raw disk images to restore and perform a block level incremental/differential patching. This is a XenServer limitation and not the Bacula Enterprise XenServer Plugin as API requires full virtual disk image uploading only - no partial block level patching. This limitation could be removed in the future as soon as XenServer API will provide a sufficient functionality.

  • The XenServer Plugin requires an NBD connection, which is not compatible with Red Hat systems, consequently impacting RHEL derivative platforms such as Alma or Rocky Linux.

  • The restart command has limitations with plugins, as it initiates the Job from scratch rather than continuing it. Bacula determines whether a Job is restarted or continued, but using the restart command will result in a new Job.