New Features in Bacula Enterprise

This chapter presents the new features that have been added to the two recent Enterprise version of Bacula. These features are only available with a subscription from Bacula Systems.

Important

Bacula offers top-notch technical assistance for the two most recent versions currently accessible.

Note

Click here to find information about new features introduced in earlier versions of Bacula Enterprise.

Bacula Enterprise 18.0

BGuardian Poisoning Detection & Configuration Assessment

This plugin serves as a valuable assistant for system administrators, enabling them to establish a more robust and secure environment. It encompasses several essential features:

  • Detection of backup poisoning: Identifies any anomalies in the data processing volume, which may indicate ransomware activities.

  • Assessment of secure configuration: Provides recommendations for configuration modifications to ensure compliance with secure guidelines and best practices.

  • Detection of failure patterns: Identifies potential issues associated with running services.

  • Generation of friendly reports: Offers detailed logs of analysis activities during runtime.

  • Generation of persistent alerts: Summarizes information and generates Bacula Events for each alert created or resolved, ensuring continuous updates with every analysis execution.

More information can be found in the BGuardian documentation: BGuardianFeatures:.

BConsole Security Enhancements

The BConsole restricted console access now offers enhanced restrictions to minimize the data shown when using the restore command and different list commands.

BWeb Security Center Dashboard

The latest addition to BWeb is a new Dashboard, the Security Center Dashboard, which allows users to conveniently manage all the security features of Bacula. It is fully integrated with the BGuardian Plugin, and other Bacula security features.

Error Files Tracking

The FileEvent catalog table is currently filled with errors that occur during a Job, such as when I/O error hinders the successful backup of a file. The list fileevents jobid=xx bconsole command can be used to view the errors associated with a specific JobId.

Enable/Disable Pruning per Job or JobId

If the Job resource directives PruneFiles and/or PruneJobs are set to never in the Director configuration file, all job instances associated with this particular Job will be designated to never be pruned from the Catalog

This setting can also be applied for individual Job instances by using the bconsole update jobid=x prunefiles=never prunejobs=never command.

When setting a Job or specific JobId(s) to never be pruned, all form of pruning will be disabled. Bacula will not be able to prune or delete these Job catalog records or recycle the volumes used by these Jobs.

Automatic workload configuration (scan_plugin)

The scan_plugin program and the associated BWeb dashboard (Automation Center) have been developed to dynamically adjust the Bacula configuration depending on external systems such as VMware, HyperV, PostgreSQL, MySQL, and more. The primary objective is to set up the configuration once and then forget about it. As an illustration, when new virtual machines (VMs) are created, they will automatically be configured for backup, and VMs that have been removed will have their corresponding backup Job disabled.

Quobyte Integration

The support for Quobyte aims to streamline and enhance the efficiency of backing up and restoring data on your Quobyte NAS.

By utilizing the Quobyte File Daemon snapshot backend, Bacula Enterprise will identify and retrieve information from the Quobyte server, including tenants, volumes, and create snapshots for the ongoing Job. Accessing files necessitates the use of the Quobyte network share (fuse.quobyte).

More information can be found in the Quobyte integration: Quobyte.

OpenStack VM Plugin (Cinder Driver)

The Bacula Enterprise Openstack VM Plugin is fully integrated with the OpenStack Cinder framework to providing comprehensive block-level image backup for instances volumes. This integration supports Full, Incremental, and Differential backup.

More information can be found in the OpenStack Plugin documentation: main Openstack VM page.

VMWare/vSphere: nvram/vxdd File Support, FIPs

With the vSphere Plugin, the VMWare NVRAM and VMX files are automatically included as part of the backup process. They are downloaded from the datastore and the directory where the given Virtual Machine stores its disks and other files.

During the restore process, the NVRAM and VMX files are automatically restored to the host where the FD and the vSphere Plugin are running. Additionally, the NVRAM file is uploaded to the destination folder of the restored Virtual Machine.

The NVRAM file contains BIOS information and can be useful in certain restore cases. The VMX file contains configuration information of Virtual Machines, allowing users to access this information if needed.

More information can be found in the vSpherePlugin.

Kubernetes CSI Snapshot Support

The Kubernetes Plugin now includes built-in support for CSI Snapshot of persistent volumes. The Kubernetes Plugin has the ability to automatically detect whether a persistent volume is compatible with CSI Snapshot capabilities, and the backup will be based on the persistent volume snapshot. Additionally, users have the option to enforce the use of CSI Volume Cloning or a Standard mode backup. These backup modes can be configured either using pod annotations or the new backup_mode plugin option.

More information can be found in the CSI Volume Features Support.

Hyper-V WinAPI Plugin

The Hyper-V Winapi Plugin uses the Virtual Disk Service (VDS) technology to backup and restore virtual machines. It supports Incremental and Differential backups, making it the ideal choice for intricate Hyper-V servers such as Failover Clusters with multiple nodes, especially when local disk space is limited. Depending on possible relocations of the VMs on the Cluster, migrating the backed-up VMs across specific nodes may be necessary.

More information can be found in the Hyper-V Winapi plugin.

Amazon Elastic Compute Cloud Plugin (EC2 VMs)

Bacula Enterprise Amazon EC2 Plugin is built over the Amazon Web Services APIs to perform various operations, such as retrieving data from the Amazon Cloud and restoring it when needed. This plugin enables the backup and restore of EC2 instances.

The plugin runs a Java Daemon which uses the official Amazon AWS SDK version 2.x built by Amazon.

More information can be found in the Architecture.

Nutanix AHV Protection Domain Selection

It is possible to select the Nutanix virtual machines that belongs to a protection domain.

More information can be found in the nutanix_ahv.

Improved Storage Group Policies

The Storage Group has two new policies available:

  • LastBackupedTo : choose the oldest storage used for the current job.

  • FreeSpaceLeastUsedStore: choose the storage with more free space and the least used.

Microsoft Exchange EWS Single Email Level Plugin

Bacula Enterprise Exchange EWS Plugin provides backup and restore operations at item level of elements managed by a Microsoft Exchange platform. This includes getting, downloading, organizing and restoring individual emails, attachments, calendar events, tasks or contacts. It is important to note that the functionality is all about traditional on-premise Exchange instances.

For additional information, visit ExchangeEWSMainPage.

For information about backing up and restoring emails or any other service of Microsoft 365, refer to M365Plugin.

M365 Plugin Enhancements

  • M365 Activity Support:

    The Microsoft 365 Activity module is now supported by the M365 Plugin.

  • M365 PST Export

    The Microsoft 365 Plugin can export emails and calendars directly into the PST format.

Oracle RAC support on AIX

The Oracle RMAN Plugin has been validated on Oracle RAC and on AIX 7 platform.

Bacula Enterprise 16.0

Bacula Enterprise 16.0.2

Advanced Job Queue Control

The Job RunScript feature has been enhanced to control the start of a Job inside the Run Queue. When a Job is starting, the Director controls that resources are available for the Job to start properly, if these resources are not available, the Job will stay in the queue, waiting to acquire them.

It is now possible to execute a script to control any kind of external and custom resources and decide when a Job should start. For example, a script might control the load average of a server before to start a Job to find the best execution time.

More information can be found in the Director documentation.

NDMP NetApp Cluster Aware Backup CAB Extension

CAB (Cluster Aware Backup) is an NDMP protocol extension. This extension enables the NDMP server to establish a data connection on a node that owns a volume.

More information can be found in the NDMP plugin documentation PluginNDMP

Storage Group

New policies have been added:

LastBackupedTo - This policy ensures that job is backed up on the storage where the same job (with same level i.e. Full or Incremental)

has been backed up to the longest time ago. The goal is to spread the jobs to improve redundancy.

FreeSpaceLeastUsed - This policy ensures that a job is backed up to the storage with the most free space and least running jobs. Within the candidates storages, the least

used one will be selected. Candidate storages are determined by the StorageGroupPolicyThreshold directive. If MaxFreeSpace is the largest amount of free space for all storages in the group, a storage will be a candidate if its free space is above MaxFreeSpace-StorageGroupPolicyThreshold. For example:

with StorageGroupPolicyThreshold=100MB

and storages free space being:
Storage1 = 500GB free
Storage2 = 200GB free
Storage3 = 400GB free
storage4 = 500GB free

In this case MaxFreeSpace=500GB.

Storage 1, 4 and 3 are candidates.

If 5 jobs are running on Storage1, 2 on Storage4, and 3 on Storage3 then Storage4 will be the selected storage.

Storage Groups can be used as follows (as part of Job and Pool configuration):

Job {
    ...
    Storage = File1, File2, File3
    ...
}
Pool {
    ...
    Storage = File4, File5, File6
    StorageGroupPolicy = FreeSpaceLeastUsed
    StorageGroupPolicyThreshold = 200 MB
    ...
}

Bacula Enterprise 16.0.0

Azure VM Plugin

A new Azure VM Hypervisor plugin bacula-enterprise-azure-vm-plugin with backup and restore support for Azure VM Hypervisor was added.

Full and Incremental block level image backup, based on online snapshot of any guest VM is supported.

More information can be found in the Azure VM Hypervisor plugin documentation Features Summary

S3 Object Plugin

A new S3 Object plugin bacula-enterprise-s3-plugin with backup and restore support for S3 Drive was added. The plugin provides the ability to download, catalog and store the data from S3 in any other kind of storage supported by Bacula Enterprise directly, without using any other intermediary service.

More information can be found in the S3 plugin documentation blb:Plugin:S3

Google Workspace Plugin

A new Google Workspace plugin bacula-enterprise-google-workspace-plugin with backup and restore support for both Google Drive and Google Mail services was added.

More information can be found in the Google Workspace plugin documentation Overview

Nutanix AHV Plugin

A new Nutanix AHV Hypervisor plugin bacula-enterprise-nutanix-ahv-plugin with backup and restore support for Nutanix AHV Hypervisor was added.

More information can be found in the Nutanix AHV Hypervisor plugin documentation nutanix_ahv

New Global Endpoint Deduplication Storage System

A new Dedup engine comes with a new storage format for the data on disk. The new format keeps the data of a backup grouped together. It significantly increases both the speed of the backup and restore operations. The new dedup vacuum command integrates a defragmentation procedure that compacts the scattered data in order to clear large and contiguous areas for the new data whilst reducing fragmentation.

More information can be found in the Global Endpoint Deduplication documentation blb:GED

Security

Storage Daemon Encryption

The Bacula Storage Daemon can now encrypt the data at the volume level to enhance security of data at rest. The volumes cannot be read by a system that doesn’t have the correct encryption keys.

More information can be found in the Security chapter of the documentation Storage Daemon Data Volume Encryption.

Malware Detection

Bacula allows you to configure your jobs to detect known Malware. The detection can be done at the end of the Backup job and/or with a Verify job. The Malware database can be downloaded from different providers, the default is set to abuse.ch. If a Backup job detects a malware in the backup content, an error is reported and the Job status is adapted.

 20-Sep 12:26 zog8-dir JobId 9: Start Backup JobId 9, Job=backup.2022-09-20_12.26.30_13
 ...
 20-Sep 12:26 zog8-dir JobId 9: [DI0002] Checking file metadata for Malwares
 20-Sep 12:26 zog8-dir JobId 9: Error: [DE0007] Found Malware(s) on JobIds 9
    Build OS:               x86_64-pc-linux-gnu archlinux
    JobId:                  9
    Job:                    backup.2022-09-20_12.26.30_13
    Backup Level:           Full
    ...
    Last Volume Bytes:      659,912,644 (659.9 MB)
    Non-fatal FD errors:    1
    SD Errors:              0
    FD termination status:  OK
    SD termination status:  OK
    Termination:            Backup OK -- with warnings


The list of the Malware detected in a given Job can be displayed with the ``list files type=malware``
command.

*list files type=malware jobid=1
+-------+-----------------------------+---------------+----------+
| jobid | filename                    | description   | source   |
+-------+-----------------------------+---------------+----------+
|     1 | /tmp/regress/build/po/fr.po | Malware found | abuse.ch |
+-------+-----------------------------+---------------+----------+

See the Malware Detection section of this manual for more information.

Volume Protection Enhancements

For File-based volumes Bacula can set the file permission to Read Only when the Volume is marked as Full or Used by the Storage Daemon. This will prevent Volumes to lost data by being ovewritten. The Access Time file attribute is also updated with the MinimumVolumeProtectionTime.

Some hardware vendors such as NetApp with the SnapLock feature, or EMC Datadomain will mark the volume file internally as immutable, which guarantee that no one can modify the file. In this situation, when a volume is Full and has the immutable flag is set, it cannot be relabeled and reused until the expiration period elapses. This helps to protect volumes from being reused too early, according to the protection period set with MinimumVolumeProtectionTime.

Device {
   Name = FileStorage
   Archive Device = /nfs/Bacula
   Media Type = NetAppFile
   Label Media = yes
   ...
   Set Volume Read Only = yes
   MinimumVolumeProtectionTime = 6 months
}

On a standard file system, the file will be marked as read only, but the file can be changed back at any time.

The file permissions are updated back when the volume is being relabeled.

There are three new directives available to set on a per-device basis to control the the Volume Protection behavior:

SetVolumeReadOnly = <yes/No>

Determines if Bacula should set the permissions to Read Only attribute when marking the volume as Used/Full.

MinimumVolumeProtectionTime = <time>

Specifies how much time has to elapse before Bacula is able to change the permissions.

In some cases, for example when the status of the Volume is changed by the Director via the update volume command, the Storage Daemon will not be able to change the permission on the Volume. Some Volumes may have the Full/Used status without the proper protection.

The command update volumeprotect is designed to determine the list of the volumes that are not protected and connect the Storage Daemon to update the permissions. It can be executed in an Admin job once a day.

*update
Update choice:
     1: Volume parameters
     2: Pool from resource
     3: Slots from autochanger
     4: Long term statistics
     5: Snapshot parameters
     6: Volume protection attributes on Storage Daemon
Choose catalog item to update (1-6): 6
Found 1 volumes with status Used/Full that must be protected
Connected to Storage "File2" at zog8:8103 with TLS
3000 Marking volume "Vol-0009" as read-only.

or via update volumeprotect

*update volumeprotect
Found 1 volumes with status Used/Full that must be protected
Connected to Storage "File2" at zog8:8103 with TLS
3000 Marking volume "Vol-0009" as read-only.

The command can be scheduled in an Admin job

Job {
   Name = adm-update-protected
   Type = Admin
   Runscript {
      Console = "update volumeprotect"
      RunsOnClient = no
      RunsWhen = Before
   }
   JobDefs = DefaultJob
}
Antivirus Plugin Enhancements

The Antivirus Plugin can now record events into the catalog for each problem that is detected. The list fileevents command can be used to list them.

*list fileevents jobid=3
Using Catalog "MyCatalog"
+-------+-------------------+----------+----------+------+-------------------------------+
| jobid | path              | filename | severity | type | description                   |
+-------+-------------------+----------+----------+------+-------------------------------+
|     3 | /home/johndoe/mp/ | eicar    |      100 | a    | stream: Eicar-Signature FOUND |
+-------+-------------------+----------+----------+------+-------------------------------+

The severity represents the level of threat, 0 behing harmless, 100 and above harmful.

The type indicates which kind of event is listed, here a meaning “antivirus”.

BWeb Management Console Enhancements

M365 Dashboard

A new dashboard for Microsoft 365 Plugin has been added to BWeb Management Console.

M365 BWeb Management Console Dashboard :name: fig:m365-dashboard
Search Function

BWeb Management Console has a new Search function that can be used to get a quick access to Jobs, Clients, Media and Menu entries.

Search function in Bweb Management Console 16

Misc

PostgreSQL Plugin fast_backup Option

The new fast_backup plugin parameter has been added to the PostgreSQL plugin in PITR mode. It can be set to true or false (default to false) and is used in the pg_backup_start() command. From the PostgreSQL documentation:

Online backups are always started at the beginning of a checkpoint. By default, pg_backup_start will wait for the next regularly scheduled checkpoint to complete, which may take a long time. This is usually preferable as it minimizes the impact on the running system. If you want to start the backup as soon as possible, pass true as the second parameter to pg_backup_start and it will request an immediate checkpoint, which will finish as fast as possible using as much I/O as possible.

More information can be found in the PostgreSQL documentation at PostgreSQL Plugin.

New Storage Daemon Disk Volume Format

BEE version 16.0 uses a new volume version named BB03. The new format adds the support for Volume Encryption, and the previous 32bits CRC32 checksum was replaced by the faster 64bits XXH64.

Volumes written with the BB03 format can only be read by Bacula Enterprise version 16 or later. Old BB02 volumes can still be restored, and Volumes may start with BB02 blocks, and continue with BB03 blocks.

It is not possible to use the Volume Encryption=yes directive on a volume that was labeled using the BB02 format. In that case, the volume will be automatically marked as Used.

Progress Status for Copy/Migration Jobs

The status director command can now report the progress of Copy and Migration Jobs.

Inventory Plugin

The new Inventory plugin bacula-enterprise-inventory-plugin will help the system administrator and the management console to determine what services are running on a given Client. The inventory can be used to automate backup job configuration.

Scheduled Job List

The status director bconsole command has been updated to limit the number of scheduled jobs listed by default to 50.

The keyword limit can be used to choose how many lines will be printed.

* status director limit=5
127.0.0.1-dir Version: 14.1.5 (26 October 2022) x86_64-pc-linux-gnu archlinux
Daemon started 31-Oct-22 17:27, conf reloaded 31-Oct-2022 17:27:55
  Jobs: run=2, running=0 max=4 mode=1,2010
  Crypto: fips=no crypto=OpenSSL 1.0.2u  20 Dec 2019
  Heap: heap=675,840 smbytes=774,259 max_bytes=2,058,994 bufs=2,470 max_bufs=4,024
  Res: njobs=24 nclients=1 nstores=3 npools=1 ncats=1 nfsets=25 nscheds=3
  Plugin: ldap totp

Scheduled Jobs (5/1440):
Level          Type     Pri  Scheduled          Job Name           Volume
===================================================================================
Full           Backup    10  31-Oct-22 17:29    NightlySave0        TestVolume001
Full           Backup    10  31-Oct-22 17:30    NightlySave1        TestVolume001
Full           Backup    10  31-Oct-22 17:31    NightlySave2        TestVolume001
Full           Backup    10  31-Oct-22 17:32    NightlySave3        TestVolume001
Full           Backup    10  31-Oct-22 17:33    NightlySave4        TestVolume001

5 scheduled Jobs over 1440 are displayed. Use the limit parameter to display more Jobs.
====

Running Jobs:
Console connected using TLS at 31-Oct-22 17:27
No Jobs running.
====

Terminated Jobs:
 JobId  Level     Files      Bytes   Status   Finished        Name
====================================================================
     1  Full          35    5.070 M  OK       31-Oct-22 15:50 NightlySave
     2  Full          35    5.070 M  OK       31-Oct-22 17:27 NightlySave
     3  Full          35    5.070 M  OK       31-Oct-22 17:28 NightlySave

====

The APIv2 json output support has been added for the .status dir scheduled command.

SIEM Console Integration

The Security information and event management (SIEM) Wazuh software (based on OSSEC) can be configured to analyse Bacula logs and events.

See the OSSEC - Wazuh - SIEM Integration section of this manual for more information.

Catalog Enhancements

  • FileSet Content Overview

Now, the Catalog stores an overview of the FileSet definition in the Content field. If the FileSet handles files and directories, the Content field will be set to files. If any plugins are used, each plugin will be inserted into the Content field.

*sql
SELECT Content FROM FileSet;
+------------------------------+
| content                      |
+------------------------------+
| mysql,postgresql             |
+------------------------------+

  • New Job Attributes

New SQL attributes have been added to the Job table such as isVirtualFull, Encrypted, LastReadStorageId, WriteStorageId, Rate, CompressRatio, StatusInfo, and so on.

  • New Media Attributes

New SQL attributes have been added to the Media table such as UseProtect, Protected, VolEncrypted

Director’s PID File timestamp

The Director PID file timestamp is now updated after a successful reload command. External tools can use this information to perform certain actions such as to clear the cache.

NDMP Incremental Enhancements

The max_level NDMP plugin command parameter can be used to specify the value of dump LEVEL that can be done on a NAS system with the NDMP protocol.

By default, the maximum value of LEVEL is 9, but some systems such as NetApp can have up to 32 levels.

The Incremental Forever feature using the BASE_DATE NDMP feature can be enabled with the use_base_date NDMP plugin command parameter. It enables an infinite number of incremental backups.

More information can be found in the NDMP plugin documentation NDMP Plugin.