Release Notes for Bacula Enterprise 18.0

Catalog format in version 18.0 and greater:
Important notes:
Known Issues and Workarounds:
Release 18.0.5 / 14 October 2024
Release 18.0.4 / 06 September 2024
Release 18.0.3 / 27 March 2024
Detailed Changes since Beta 18.0.2
Beta 18.0.2 / 05 March 2024
Beta 18.0.1 / 06 February 2024

Both the Director (Dir) and all Storage Daemons (SDs) must be upgraded to version 18.0 as they are not compatible with earlier versions.

Catalog format in version 18.0 and greater:

The major release 18.0 of Bacula uses a new catalog format. We provide a set of scripts that convert 6.6.x and earlier catalog versions to the new 18.0 format (1027). See the Migration to New Major Version of Bacula for detailed instructions on the migration process.

The database upgrade from 1026 to 1027 is fast and simple for most users.

The Bacula Enterprise RPM spec files have been modified to detect the database change and to perform the upgrade for you (from 8.10, 10.0, 10.2, etc.). We strongly recommend that you save a copy of your existing database before upgrading using RPMs. If you want to have more control over the catalog upgrade step when upgrading using RPMs, you may want to contact the Bacula Systems support team to get the update_bacula_tables script from the updated package and run it before the RPM installation step, or you can stop the database server before the upgrade.

The Bacula Enterprise Debian and Ubuntu packages have been designed to upgrade the catalog with the dbconfig database management tool. The update_bacula_tables script from the Director package can be used to upgrade the catalog manually.

If you upgrade the BWeb Management Suite from 8.x and you use a MySQL catalog, you must first update the BWeb SQL tables with the script upgrade-8.8_10.2_mysql.sql.

If you are starting from scratch, you will not need to run the update_bacula_tables script since the create_bacula_tables script creates the tables in the new format automatically.

Important notes:

If you upgrade to from a new 16.0.0-5 installation to 16.0.6-8 with a MySQL catalog, the following SQL command must be executed in order to use the JobHisto table and features:

ALTER TABLE JobHisto
  ADD COLUMN RealStartTime DATETIME,
  ADD COLUMN isVirtualFull -  TINYINT - default 0,
  ADD COLUMN CompressRatio -  float -   default 0,
  ADD COLUMN Rate -  -  -   float -   default 0,
  ADD COLUMN LastReadStorageId Integer - default 0,
  ADD COLUMN LastReadDevice - blob,
  ADD COLUMN WriteStorageId - integer - default 0,
  ADD COLUMN WriteDevice -    blob,
  ADD COLUMN StatusInfo -  - blob,
  ADD COLUMN Encrypted -  -  int -  - default 0;

For Restricted Console users, we have changed slightly the DirectoryACL=*all* and UserIdACL=*all*, if the directives are not defined in a restricted console, the restore code will allow the file listing.

Old File Daemon versions are still compatible and can be used to protect older systems such as Redhat 5, SLES 10-11.2.

Known Issues and Workarounds:

#1723 The hardlinks=yes FileSet option is not compatible with VirtualFull jobs.

This incompatibility might lead to problems performing a restore. The error message displayed during a restore job is:

Error: create_file.c:327 Could not hard link /path: ERR=No such file or directory

To restore the file correctly, the first occurrence of the hardlink file in the backup must be selected manually. The procedure to distinguish hardlinks in the catalog is available from the Bacula Systems Support team.

#4957 Copy/Migration/VirtualFull jobs based on Incomplete jobs - a potential issue.

Under some circumstances a Copy/Migration/VirtualFull job may not be able to copy the data from a Job that was interrupted and restarted using the Incomplete Job feature. In versions 10.2.3 and above, the Copy/Migration/VirtualFull job is aborted with the following error message:

Fatal error: catreq.c:691 SHA1 digest not same FileIndex=344 as attributes FI=343

or

Fatal error: catreq.c:691 MD5 digest not same FileIndex=344 as attributes FI=343

It is recommended to re-execute the Job that was interrupted and restarted to be able to copy/migrate the data. We do NOT recommend including Incomplete Jobs into a Progressive Virtual Full chain of jobs.

The automatic rescheduling of Incomplete Jobs should be disabled to avoid restarted Incomplete jobs finishing successfully and being selected to be copied, migrated, or consolidated in a Virtual Full. To prevent Bacula from rescheduling Incomplete Jobs, use the “RescheduleIncompleteJobs = no” Job directive.

Please contact the Bacula Systems Support team for more information on any of the above issues.

Release 18.0.5 / 14 October 2024 

18.0.5 is a minor bug fix release.

BWeb:

bweb: Add few more plugins to the list

bweb: Add korean support

bweb: Add support for Xenserver Plugin in the Automation Center

bweb: Disable VSS for hyperv and mssql filesets generated with the scan_plugin

bweb: Display only plugins compatible with the scan plugin in the Automation Center

bweb: Fix #11117 About device status icon in Storage Overview

bweb: Fix #11128 About displaying empty pie charts when we have no data

bweb: Fix #11159 about inconsistent displayed plugin logo during backup

bweb: Fix #11168 About incorrect behavior when nb_grouping_separator is not set

bweb: Fix device status with running jobs

bweb: Fix graph display with no volume

bweb: Fix pie/stackpercent charts with lot of elements

bweb: Add HyperV, AzureVM, AmazonEC2 and Nutanix to the Single Item Restore shortcut

bweb: Add AuthenticationMethods directives

Dedup 2 Plugin:

dedup2: Fix bug in bextract when reporting rehydration errors

Documentation:

docs: Add list fileevents jobid=<xx> to Console commands

docs: Add reference about list fileevents jobid=<xx> to Jobs monitoring article

docs: Add the MySQL plugin option backup_software to address some special cases

docs: Add verbose parameter to the proxmox documentation

docs: Added more details about object lock / immutability in the Cloud S3/Amazon documentation

docs: Adding Hyper-V WinAPI link to main page + creating notes with recommendations

docs: Adding troubleshooting answer: Impact of HBA-attached disks on BMR restores

docs: Changed paragraph about Archive Device

docs: Corrected BlobEndpoint description to contain both Azure and Amazon mentions

docs: Fix default value for Device/SyncOnClose directive

docs: Fixing vSphere table

docs: MySQL limitation discarded

docs: Publish RNs 18.0.4

docs: Reformat directives from Console resource (excluded ConsoleFont)

docs: Reformatting Client Client directives

docs: Removing untrue information about Cinder Driver supporting Incremental and Differential backup

docs: Restart command updated with a limitation regarding plugins

docs: Security Plugin reviewed and restructured

docs: Small changes in the recycling algorithm section

docs: Update documentation on lastbackedupto

docs: Update upgrade chapter to 18.0

Hyper-V Plugin:

hyperv-winapi: Fix #11136 GetRestoreObject VM retrieval

hyperv-winapi: Fix #11163 add “vm” as parse_command parameter

hyperv-winapi: Fix #11110 Remove unused restore params

Kubernetes Plugin:

k8s: Add new level(In pvc annotations) in selection of backup mode

k8s: Add support pvcdata list in estimation jobs

k8s: Fix show pvdatada message when you don’t use it

Microsoft 365 Plugin:

m365: Backup sharedWithMe Onedrive objects just once

m365: Do not let java process running after a failed restore just because not receiving the jobend meta protocol phase

m365: Fix runtime error with missing com.azure.core.implementation.util.HttpUtils from azure-core package

m365: Fixes on OneDrive downloads for SystemLibraries

m365: Increase DeviceCode Auth timeout. Fix estimate. Improve restore drive speed

m365: Rename too long path on restore (Drive). Checkin files that were checked-out (sharepoint)

m365: Replace checked-in files only if restore replace variable is always or ifolder

m365: Small fix to avoid warning message

MySQL Plugin:

mssql: Fix #11121 About restoring datatabase on a different server/instance

MSSQL Plugin:

mysql: Fix #11141 Use connection parameters prior to the dump_opt option

Openstack VM Plugin:

openstack-vm: Add the possibility to backup multiple VMs in one command

openstack-vm: Now handles wildcards in -b argument

Proxmox Plugin:

proxmox: Fix #11133 add verbose flag to display more information during a backup

CitrixHypervisor (Xenserver) Plugin:

xenserver: Document secure mode with new parameters

xenserver: Fix #11157 About canceling a job after a nbd-mount or a snapshot creation issue when abort_on_error is set

xenserver: Fix .query interface for scan_plugin

Miscellaneous:

Add use_import_export configuration option to mtx-changer

Fix #11107 About incorrect list metadata help command

Fix #11126 Rework the LastBackedUpTo Storage Group Policy

Fix #11126 Use Pool’s StoragePolicy when using the Storage resource defined in the Pool

Fix #11165 generating a BSR file when the disk is full

Fix #9305/2 At restore time, show wrong Backup Client when modifying client

Use case insensitive search to assign storage group policies

Fix #11178 About authentication issue with 9.0 FileDaemon

Fix segfault on ARM64 with json tools

rpms: Add spec file for big-fs on sles

rpms: Fix #11125 About incorrect permission on callhome plugin file

rpms: Fix dedup2 compilation

rpms: Fix libcurl in sles

scan_plugin: Accept xenserver malformed output

scan_plugin: Fix JSON help message information

scan_plugin: escape @ to _at_ instead of 0x40

sir: Fix #11150 about segfault during initialization

sir: Fix check procedure when the console has pending messages

update bacula-resource-auto-creator

update mtx-changer-python

Bugs fixed/closed since last release: 11107 11110 11117 11121 11125 11126 11128 11133 11136 11141 11150 11157 11159 11163 11165 11168 8 9305

Release 18.0.4 / 06 September 2024 

18.0.4 is a minor bug fix release.

Aligned Volumes:

aligned: Add support for Immutable and ReadOnly to Aligned volumes

Cloud Plugin:

cloud: display error returned by the driver

cloud: Fix #10836 clarify cloud status display

cloud: Add .query interface to manage cloud functions

cloud: Implement Tier Support

Kubernetes Plugin:

k8s: Add dockerfile to create an image to compile k8s plugin in any debian/ubuntu distribution

k8s: Add ingress component in backup list and add recomendation when to try to restore pods that create new data if pvc is empty

k8s: Add more options to debug

k8s: Avoid pvc data when pvc is in Pending status. k8s: Fix redoing backup when pvcdata is 0 bytes and the backup mode is standard

k8s: Avoid pvcs backup when the pvc status is Terminating

k8s: Define backup_mode parameter

k8s: Fix #0010901 - Problem when restore service clusterIPs

k8s: Fix #0011005: ModuleNotFoundError: No module named ‘baculak8s.plugins.k8sbackend.ingress’

k8s: Fix restore problem where pod require the pvc data when it starts

k8s: Add k8s dependency versions in requirements

k8s: Add Ingress integration backup/restore

BWeb:

bweb: Add check for server version in bweb_testconfig.pl

bweb: Add empty message in the Automation center page

bweb: Add function to get Job JSON description with JobDefs

bweb: Add information during initial configuration

bweb: Add level input to Automation Wizard

bweb: Add new StorageClass directive values

bweb: Add to initial wizard setting up pruning options

bweb: Display Job and Admin job for automation job list. Add JobDefs information such as Schedule

bweb: Display mode option for QEMU plugin

bweb: Fix #10976 set in fileset include/exclude fields unix root directory path

bweb: Fix #11054 Display Bytes, ReadBytes, Average and Current Speed in MiB-GiB/s

bweb: Fix #11059/2 incorrect timestamp in object version browser

bweb: Fix #11068 about incorrect QR code display

bweb: Fix #11092 About ClearIO function using loaded slots

bweb: Fix SSH key generation

bweb: Fix StorageGroupPolicy list in Job/JobDefs page

bweb: Fix amazon-rds doc link

bweb: Fix displayed Virtual machine dashboard and database dashboard

bweb: Fix documentation links

bweb: Fix duplicate entries in the Automation Center job list

bweb: Fix incorrect label in Object dashboard for the Backup Count graph

bweb: Fix issue with proxmox edition page

bweb: Fix job Automation center job list

bweb: Fix missing plugin_option in Automation Center

bweb: Generate unique automation names

bweb: Implement Plugin queries for MySQL

bweb: Improve Automation Center scan options

bweb: Improve QEMU/Proxmox automation part

bweb: Improve displaying long email address in M365 restore email browser

bweb: Instruct to use Level=Full in proxmox plugin

bweb: Pass autocommit when deleting an automation job

bweb: Reduce the size of the plugin section names

bweb: Report .query errors in the Automation Center

bweb: Fix #10254 in storage wizard disable AllowCompression directive for aligned type storage

bweb: Fix #10807 add support for multiple automations per client and plugin

bweb: Fix #10897 update bsys_report.pl to version 3.52

bweb: Fix #10944 in restore wizard add message about jobs that have file records purged

bweb: Fix #10956 display job level for copy jobs

bweb: Fix #10926 and #10927 add support for colon character in configuration resource names

bweb: Rework Autochanger feature

bweb: Select automatically the Storage Daemon in the overview page

Dedup 2 Plugin:

dedup2: All repair modes can use -r to replace the old index with the new one.

dedup2: Fix #11028 add File.FileId to btools parse_dedup_badref output

dedup2: Fix #11029 add a suffix to db name generated by btools parse_dedup_badref

dedup2: Deny vacuum on a read-only dedupengine

dedup2: Fix #10974 “unavailable” extent wrongly converted into “scrub”

dedup2: In-place rollback and repair of the index

dedup2: New -I option to bls for developers only

dedup2: add jobid and stime information in the extent

dedup2: bls or bextract display dedup2 open error message

dedup2: bug fixes and enhancements for rebuilding index fromcontainers and vaccum rebase

dedup2: bug fixes for managing the vacuum_overloaded variable in the storage daemon

dedup2: enforce read-only mode in init_dedup_engine2 for storage and index

dedup2: fix bug about moving/moved extents

dedup2: fix extra messages for bad index record in bdde-check

dedup2: fix in-place rollback is always running

dedup2: fix race condition responsible for overloaded extent

dedup2: fix the vacuum that can leave the DDE corrupted when unexpectedly interrupted

dedup2: improve error message for chunk not found in the index

dedup2: open the index in read-only mode

dedup2: reduce container footprint in cache via POSIX_FADV_DONTNEED

dedup2: rollback must keep rdy4sync extent as used

dedup2: switch back to bzero when initializing tcindex_stat in bdde-fsck

dedup: Use O_CLOEXEC by default in open calls

Documentation:

docs: Add an example to create a stub job to call RMAN

docs: Add documentation for JobTimestamp variable

docs: Added comment to flag %I that it applies to runscript

docs: Added example to advanced restore chapter with using custom plugin options

docs: Added link to Percona Tools installation to Binary Mode - Percona XtraBackup and Mariabackup articles

docs: Added paragraph in best practices about long-running restores

docs: Adding BGuardian mention to Security Features articles

docs: Adding a note about AppStream repository in all SIR installation articles

docs: Adding dedup console command

docs: Adding limitation about Catalog running to BGuardian

docs: Adding note about reading general best practices for hypervisors

docs: Adding speed type into Recognized Primitive Data Types

docs: Adjusting location of the cloud directives

docs: new_netwrok - definition modification - new_network_device_name - definition modification

docs: Changing sentence about Proxmox plugin compatibility with Proxmox VE

docs: Dedup2 index rebuild documentation

docs: Deleting Autochanger resource from Director

docs: Document Job::Encrypted field

docs: Document how to identify and remove orphaned java backend processes

docs: Fix LastBackedUp to keyword in New Features section

docs: How-to guides for simple BWeb operations

docs: Improve PITR Configuration subarticles

docs: Improve documentation in Restoring Complete Cluster Using PITR

docs: Improvements to the definition of the namespace and pvcdata options

docs: Meta page about mssql plugins

docs: Moving Fileset examples from Operations to Configuration

docs: Removed recommendation to use snapshot with databases

docs: Removed version number from the title of new features article

docs: Renaming file recovery.conf to postgresql.recovery.conf

docs: 16.0.14 release notes to published

docs: SAP plugin article changed option list to table

docs: Sentence about not supporting version higher than 8.0 with MySQL

docs: Sharepoints comments about restore and Query command better examples

docs: Title change to include Mariabackup

docs: Update accurate flags for accurate jobs and add NextPool directorive to the job directives

docs: amazon-ec2: New restore parameters and comments. New permissions. Hypervisor to cloud options

docs: azure-vm: Documentation update for local restore possibility

docs: changed stretch to bullseye since this plugin in only supported in Debian 11 right now

docs: hyperv-winapi: Separate wmi and winapi documentation and add new winapi documentation

docs: hyperv-winapi: Update documentation on boot order after MT10887

Hyper-V WMI Plugin:

hyperv-winapi: Fix #10887 Better handling of gen 2 VM boot order

hyperv-winapi: bump script version

hyperv-winapi: fix restore after file format changes

hyperv-winapi: move restore vm renaming at creation stage

hyperv-winapi: new file formats for SIR

sir: HYPERV WINAPI Single Item Restore implementation

Inventory Plugin:

inventory: Use bfopen instead of fopen

KVM Plugin:

kvm: Fix #10709 About backup issue with a mounted CDROM

Microsoft 365 Plugin:

m365: Adapt .query output for sharepoint to better integrate it with scanplugin when we have sub-sites

m365: Add id and other information to .query command. Adapt backup to interpret site id, even with ;

m365: Add list of allowed values to Enum and normalize int values for plugin-options .query

m365: Add new queries for BCenter (config-file list, plugin-options and service-entity-map)

m365: Add uuid in query commands potentially used by scan_plugin

m365: Added sharepoint new restore param. Added missing drive parameters to bweb

m365: Adjust plugin-options query to dictionary mode

m365: Do not fail with access denied error when uploading drive items in restore

m365: Do not show error in .ls or query commands outputs while listing sites

m365: Do not stop listing sites wity .ls or .query if we have trouble with sub-sites

m365: Downgrade building plugins to keep compatibility with old Maven and RHEL8

m365: Enable/disable noscript parameter on Drive site restores. Other internal improvements

m365: Fix Sharepoint race condition with associated drive items #10827

m365: Fix contacts not generating plugin object when all users are included

m365: Fix filter(s) by name fields when value is a number. Handle 500 response for not found message rules. Added .query decode|url method

m365: Fix invalid field ref for Sharepoint template

m365: Fix issue with Sharepoint and different domains in the organization

m365: Fix not all Sharepoint sites being listed when we have more than 200 #10915

m365: Fix owner restore protection user is removed. Included Java PID

m365: Fix restore to M365 from Sites that contain a ‘/’ in their name. #10961

m365: Format different new .query methods to accomodate BCenter needs

m365: Ignore Files tab at restore time in teams module #11010

m365: Improve date selection on meta table. Fix intervention message and date

m365: Increase Jar version after .query changes

m365: Libraries upgrade

m365: Remove BWeb explicit word on registration

m365: Restore improvements on error conditions. Added more debug. Include SitePages in Sharepoint backup.

m365: Retry 400 request when listing drive units. Do not repeat those requests if possible. Control extra case on restore

m365: Return full objects for user, group, site, team in .query with parameter=json|xxxx

m365: Sharepoint improvements about very specific situations on restore

m365: Sharepoint improvements while restoring full site with system lists included

m365: Show message in .query login, when no user is specified

m365: Support query=team in addition to query=teams in order to properly support scan_plugin

m365: Update PnP framework

m365: Version internal alignment with Bacula

m365-pst: Align m365 version dependency

m365-pst: Fix compilation error by not finding mvn.wrapper

m365-pst: Fix dependency problem

m365-pst: Fix reference to the main project

MySQL Plugin:

mssql: Fix incorrect fopen() call

mssql: Fix warning reported by cpp-check

mysql: Add support for MySQL Community 8.2 and 8.4

mysql-prepare: Use bfopen instead of fopen

NDMP Plugin:

ndmp: Fix #10977 About SMTAPE restore issue with incorrect FILESYSTEM variable

ndmp: Fix #11047 restore issue when using CAB extension

ndmp: Use bfopen instead of fopen

Openstack VM Plugin:

openstack: Add second check for cinder-backup in volume list in case it is containered

openstack: Fix an issue that would crash the restore when non-bacula backups were present on the system

openstack: Fix user detection in install script when tripel is not defined

openstack: Fixing openstack tmpdir naming inconsistencies

Big FS Plugin:

big-fs: Add master job id recognition when stating in CLI mode

big-fs: Add test job during with install test directive

big-fs: Additional check in install script

big-fs: Change type of sbtctx value from bool to int

big-fs: Each run of big-fs has its own lockfile

big-fs: Keeping track of master jobid across backint/fd

big-fs: Remove problematic check of empty include clause

big-fs: Replace openstack occurence in install script

big-fs: Solves an issue that made the program crash when getting a create file call

big-fs: Solves slowness issues

big-fs: Use additional lock return code to fix an issue when old pid file is not clear correctly

big-fs: jobfile create and check for fifo waiting

big-fs: Add another return code for create lockfile in case program calls itself

Amazon EC2 Plugin:

amazon-ec2: Adapt block headers to reference a disk without headers

amazon-ec2: Fix log4.dat problem in some platforms

amazon-ec2: Keep the same block order in .bmp than in the .bimg disk file

Amazon RDS Plugin:

amazon-rds: Backup cycle completed. Recycling and removing process implemented

amazon-rds: Cluster AZ zone control. Catalog path simplification. Cluster backup fixes

amazon-rds: Clusters management. Code refactoring. Query-list instances and clusters

amazon-rds: DAO layer implemented for cluster, instance and snapshots

amazon-rds: Export initial actions. Basic backup cycle

amazon-rds: Exports management and s3 data download

amazon-rds: Fix parquet_socket parameter name

amazon-rds: Format all source files to standarize indentation

amazon-rds: Logging code to try to catch unhandled exceptions. Fixed MetaPlugin restore cycle when errors happen

amazon-rds: Logging improvements

amazon-rds: MySQL connection code test and dependencies

amazon-rds: MySQL restore integrated. Code comments and updated Amazon SDK

amazon-rds: Parquet restore mode adjustments. Working now

amazon-rds: Pitr recovery logic

amazon-rds: Project initialization

amazon-rds: Remove some redundant code lines, include FAILED state check for export task

amazon-rds: Restore instance and cluster from snapshot

amazon-rds: Restore parquet logic

amazon-rds: Restore tables even if they are empty. MySQL fixes

amazon-rds: Amazon RDS Backup Plugin implementation

Azure Plugin:

azure-vm: Add a fallback is default snapshot name is too big

azure-vm: Add single header for the size of whole disk at full backup time

azure-vm: Adds diskSize header to bmp files

azure-vm: Better regexp compilation + correctly handles diskSize headers in bmp files

azure-vm: Change configuration from list to stack

azure-vm: Change permission during backup on stat pkt

azure-vm: Depencies update

azure-vm: Fixing sending protocol for bmp files

azure-vm: Get mode for stat pkt now handles ‘E’mpty job cancellation on abort on error

azure-vm: Handling bitmap pkt at restore time

azure-vm: Implement file index and coherent disk naming

azure-vm: Implement local restore mode

azure-vm: Local restore implementation

azure-vm: Now sending configuration as restored object

azure-vm: Plugin now uses offset flag for disk backup

azure-vm: Restore for single header file from full backup updated

azure-vm: Sending of bmp/abmp file to bacula at backup time

azure-vm: Using exposed flags instead of locally defines var in backend

Nutanix-AHV Plugin:

nutanix-ahv: Dependency update

nutanix-ahv: Fix issue with permission and UID GID at backup time

nutanix-ahv: Plugin now has the possibility to use offset flag

nutanix-ahv: Using exposed flag instead of locally defines var in backend

QEMU Plugin:

qemu: Add UUID to the .query vm output

qemu: Add scan_plugin support for qemu proxmox (mode=pve)

qemu: Fix .query output with vm/vmid

qemu: Fix file headers

qemu: Remove UUID from VM listing

qemu: Use O_CLOEXEC flag

Quobyte Plugin:

quobyte: Add message for the OneFS=no and update the message when Enabling Snapshots

quobyte: Fix #10935 subvolume selection

quobyte: Fix issue with volume that never had snapshots

quobyte: Set oneFS=no automatically when using the plugin

quobyte: Wait for snapshot creation

Sharepoint Plugin:

sharepoint: Fix incorrect error message

vSphere Plugin:

vsphere: Adjust restore messages for new_network parameters

vsphere: Fix #10939 About keeping the snapshot generated during backup

vsphere: Fix bad openssl variable name. Add datastores when listing hostsystems

vsphere: Fix duplicated server= entry for query parameter=server. Renamed api one to server_api

vsphere: Fix error while setting network at restore time

vsphere: Fix instant recovery disk paths. Avoid searching for targets in NFS and other commands (to speed them up). JDK 21 compatibility

vsphere: Recover FIPS related configurations in vsphere-ctl

vsphere: Search network by moref, in addition to by name at restore time. #10983

Windows BMR Plugin:

winbmr: Fix cpp-check warning

CitrixHypervisor (Xenserver) Plugin:

xenserver: Use bfopen instead of fopen

xenserver: Add few O_CLOEXEC to open calls

xenserver: Enhance parameter scanning for new options

xenserver: Fix log message about secure mode putting it in certfile param instead of cacertfile param

xenserver: Fix sending pluginobject in incremental jobs

xenserver: Handle secure connection parameters (certfile, keyfile…). Invoke modprobe nbd. Send PluginObject always

xenserver: Log secure/insecure mode

Miscellaneous:

win32: Add support for GSASL

win32: Comment out our nanosleep()

win32: Disable inherit in bfopen()

win32: Limit the scope of the open files

win32: Use pthread_cond_wait instead of mingw nanosleep on Windows platform

meta-plugin: Added offset flag handling. Updated libraries

meta-plugin: Exposing delta/offset flag via static getter method

meta-plugin: Fix delta-offset flags when they are set alone

metaplugin: Update meta-plugin dependencies and JUnit to have it fine for JDK 21. Update lombok everywhere aswell

metaplugin: Include bacula jobid in the backend calls as BACULA_JOBID env variable

msad: Fix #10118 About missing error message when using incorrect HBINDPASS attribute

msad: Report a warning if HBINDPASS is too short

rpms: Add amazon-rds spec file

rpms: Add bacula python tools for mtx and autochangers

rpms: Add bweb server.stream-response-body configuration automatically if needed

rpms: Add python docopt dependency for rhel8 and rhel7

rpms: Fix amazon-rds spec file

rpms: update spec file to include GSASL

bfuse: If vmbackend cannot create a disk it exit 1 with an error messgage instead of segfault

bfuse: Replace deprecated ENOATTR with ENODATA

bfuse: Retrun false earlier in azure vm disk recrod when parsing phase goes bad

bfuse: Special case if the header match diskSize for azurevm

bfuse: Working parsing implementation still need to remove gdb comments

bfuse: amazon-ec2 Single Item Restore implementation

bfuse: azure-vm and winapi give correct moref for vmanme to be compatible with beweb

scan-plugin: Do not override elements created by a different configuration file having another uuid

scan_plugin: Do not generate job names with :

scan_plugin: Use vmid as uuid

security-fd: Use bfopen instead of fopen

security-fd: Fix #11072 Print a message when starting the security checks in job log

Add SOCK_CLOEXEC to bnet_server sockets

Add .jobs [enabled|disabled] option

Add extended version with -v -? on all daemons

Add list of installed SD drivers to .status storage header

Add mtx-changer-python.py from https://github.com/waa/mtx-changer-python

Add bacula-resource-auto-creator from https://github.com/waa/bacula-resource-auto-creator

Add new .jlist metadata target=folders|tags parameter

BSOCK improve POLL to detect and report errors

Check for backquote in check_for_invalid_chars() function

cdp: Use bfopen instead of fopen

Fix #10262 segfault when permission cannot be changed on a read-only FS

Fix #10793 extent query strorage filename to return access and attributes

Fix #10919 About extra job message sent during startup

Fix #10947 time output without century for locale that use multi bytes utf8

Fix #10985 Report the FD/SD Encryption in the Job record and the job output

Fix #11002 About crash while purging client jobs

Fix #11009 About issue when migrating community catalog 9.4

Fix #11048 About LastBackedUpTo StorageGroup policy not correctly set

Fix #11058 About fsync error reported for tape driver

Fix #11062 Improve BLOCKED message when the tape device was disabled by TapeAlert

Fix Scheduled Jobs counting in status director command

Fix qa-core-features#44 About incorrect PruneJobs=yes/PruneFiles=yes handling

Fix segfault in ua_prune introduced with avanced Pruning checking

Fix segfault when using bregex on large chunks of data

Fix the FileEvent list is not reset between files

Handle new PostgreSQL public schema permissions from grant_bacula_privileges

Refactor code around Autoprune/PruneFiles/PruneJobs

Use bclose_from() in daemon.c

Use bfopen instead of fopen in various tools

Use the base SHA for the bacula source tree in the build information

acsls: Improvements after code review for open files management

add baselist.clear() to remove all the items

mtx-changer-python: Changed log level in two places, and added a throw-away first sg_logs test based on physical tape drive tests.

Add SCRAM-SHA-256 authentication algorigthm

Bugs fixed/closed since last release: 10118 10254 10262 10709 10793 10807 10827 10836 10887 10897 10901 10915 10919 10926 10935 10939 10944 10947 10956 10961 10974 10976 10977 10983 10985 11002 11005 11009 11010 11028 11029 11047 11048 11054 11058 11059 11062 11068 11072 11092 44

Release 18.0.3 / 27 March 2024 

18.0.3 is a major release.

Security & Monitoring:

Introduction of BGuardian for Poisoning Detection and Configuration Assessment.

Enhanced Key Management for Storage Level Encryption.

Security Improvements in BConsole.

Development of a Cybersecurity Dashboard (Work in Progress).

Error Files Tracking feature.

Pruning Enable/Disable option per Job.

Storage, NAS & FileSystems:

Support for NetApp Cluster Aware Backup (NDMP CAB).

NDMP Incremental Forever feature.

Automatic workload configuration via scan plugin.

HPE StoreOnce Catalyst Plugin now supports Immutability.

Integration with Quobyte (Work in Progress).

Cloud, Containers & Hypervisors:

New plugins for OpenStack Cinder and Azure Virtual Machine with Incremental Backup support.

Enhanced VMWare/vSphere support for nvram/vxdd files and FIPs compliance.

Kubernetes CSI Snapshot support.

Introduction of Hyper-V WinAPI and Amazon EC2 Plugins.

Core Enhancements:

Advanced Job Queue Control.

Improved Storage Group Policies.

Remote Daemon Configuration for FD & SD (Work in Progress).

Native Multitenancy support.

Applications & Databases:

New Microsoft Exchange EWS Single Email Level Plugin.

Oracle RAC support on AIX.

Usability Improvements:

General Enhancements to BWeb.

New automation wizard via scan plugin.

Miscellaneous:

Added ControlJobId in Catalog job record for better migration/copy job tracking.

Introduced bVarSnapshot for snapshot enablement by plugins.

Detailed Changes since Beta 18.0.2 

Amazon EC2 Plugin:

amazon-ec2: AMI SSM Resolver. Command line restore without references completed

amazon-ec2: Allow to choose restore root disk from file disk name or disk index. Handle correctly root disk + key. Better log

amazon-ec2: Fix disk attach name when exists. Find existing instance when restoring only disks

amazon-ec2: Fixes and improvements on the experience with bec2 cli

amazon-ec2: Fixes on image based restore and instance-only restore from bec2

amazon-ec2: Fixes on no-referenced restores. Enriched logging and code simplifications

amazon-ec2: Add –parameters call for h2cloud and other small fixes

amazon-ec2: Fixes for cross-hypervisor restore. Ovf matching with instance type. Subnet and other extra parameters

amazon-ec2: Log vCpus and RAM detected from ovf

amazon-ec2: Add Management of root volume at restore time. Restore code refactor

amazon-ec2: Map network and name from Ovf. Generalize Ovf code to support future different mechanisms

amazon-ec2: Remove duplicated log

amazon-ec2: Select root volume for instance attach (without creation)

amazon-ec2: Support for restore volume over an existing instance

amazon-ec2: New parameters added for restoring and h2cloud use cases

Azure Plugin:

azure-vm: Changed accurate plugin parameter to make it mandatory for plugin

BWeb:

bweb: Add StorageGroupPolicyThreshold and DisableQueryCommand directives support

bweb: Add backup_mode option to kubernetes plugin configuration

bweb: Add hyper-v winapi plugin support

bweb: Add message to plugin options in automation wizard

bweb: Add need_accurate information to plugin json description

bweb: Add quobyte plugin support

bweb: Add s3 force_path_style

bweb: Add timeout option support to runscript subresource

bweb: Add to automation functions accurate mode in jobs which plugins require it

bweb: Add version option support to winbmr plugin settings

bweb: Fix #10801 about use_uuid in scan_plugin

bweb: Fix #10812 About sorting description field with scan_plugin

bweb: Fix losing job options setting in automation wizard

bweb: Fix title in window with hyper-v wmi plugin setting

bweb: keep the name instead of the uuid for the fileset name in scan_plugin

Cloud Plugin:

cloud: Fix 10788 s3 driver clean uses alist instead of ilist

cloud: generic driver clean uses alist instead of ilist

cloud: Fix 10788 cloud volume deletion crash

Dedup 2 Plugin:

dedup2: Fix #10790 SD stuck in a copy job because of a MISS chunk

dedup2: Fix copy/mig false error “size in reference doesn’t match size of chunk”

Documentation:

docs: Changed version from 16.0.7 to 16.0.12 in BGuradian Scope

docs: Publishing Amazon EC2

Hyper-V Winapi Plugin:

hyperv-winapi: order recovery actions

Kubernetes Plugin:

k8s: Add backup_mode parameter, add resiliance when snapshot backup is not compatible and clone backup fails and minor fixes

k8s: Fix #10669: When a node have not any pvc

k8s: Add check snapshot api-resource available

k8s: Fix when ‘volumesnapshotclasses’ resource is not available in cluster

Openstack VM Plugin:

openstack-vm: Additional check of configuration/compatibility for os-backint

openstack-vm: Automatic snapshot deletion

openstack-vm: Better error handling via return codes

openstack-vm: Better verbose output and check for query -q

openstack-vm: Changed argument parsing for other main python procedure

openstack-vm: Concurrent job option passed as string

openstack-vm: Fix double job issue at backup time

openstack-vm: Fixed restore mode with openstack strange debug message

openstack-vm: Fixes install script default dir conf comment

openstack-vm: Now procedure fails gracefully when snapshot fails

openstack-vm: Number of max concurrent job set at runtime and removed from os-backint-conf

openstack-vm: Option -b to -B new option -b backup from instance name

openstack-vm: Remove mention of cluster in default conf file

openstack-vm: Adds explicit number of parallel job parameters

Microsoft 365 Plugin:

m365-pst: Conversion for emails, attachments and contacts. Code organization. Management of external lib

m365-pst: Handle calendars and contacts. Fix folder management

Nutanix-AHV Plugin:

nutanix-ahv: Add restore object send capabilities + sending configuration as said restore object during backup

nutanix-ahv: Add fileIndex compatibilities

Quobyte Plugin:

quobyte: Add job message when enabling the snapshot from the plugin

quobyte: Force snapshot when quobyte plugin is used

quobyte: Get quobyte plugin helper working with the scan_plugin tool

vSphere Plugin:

vsphere: Add StorageProfile.View permission #10786. Use manual versioning

vsphere: Add a backtrace procedure after a bad signal

vsphere: Avoid to print ‘vm config not found’ errors in query that can provoke problems with scan_datacenter

vsphere: Do not fail if overridevm is used, but the target VM is no more in server

vsphere: Find vmx and nvram even if vm was renamed or moved

vsphere: re-fix query showing errors by system.out and causing trouble with scandatacenter #10781

Miscellaneous:

Add information about the number of protected jobs when using autopruning

Add information messages when using update jobid=x prunejobs/prunefiles command

Fix DisableQueryCommand check in bsdjson and bacula-sd

Only normal backup jobs can become “Incomplete”, MAC must return errors

Revert “Fix #8354 About confusing log message during the ‘dump’ mysql backup”

Revert “Fix #9882 About tapealert script issue on rhel8”

create-config: Add restart service when configuration works

Bugs fixed/closed since last release: 10781 10786 10790 10801 10812 8354 9882

Beta 18.0.2 / 05 March 2024 

18.0.2 is a minor bug fix beta release.

Amazon EC2 Plugin:

amazon-ec2: Fix keep_full_snapshot behavior and chain Inc(s) with Diff(s) backups correctly

BGuardian:

bguardian: Change description of the connection method to the catalog. Adjust examples with bacula user

bguardian: Improve connection and do it by default using unix sockets

BWeb:

bweb: Add include missing chart library to BWeb spec files

bweb: Fix #10775 allow in config forms use capitalized time units

bweb: Fix #10776 pool filter on media page

bweb: Fix #9693 not working disable bacula configuration menu option

bweb: New M365 Activity module integration

Dedup 2 Plugin:

dedup2: Fix #10486 corrupted chunk with huge size crash the SD

dedup2: updates to clean up code and fix remaining endian issues for bdde-fsck and bdde-check

dedup2: Added check to show bug saving rollback flag when hdbohashmem = true

dedup2: Two DDE utilities for regress testing and debugging, bdde-tool and bdde-fsck

dedup2: fix where vaccum exits early during inventory and removes data from containers

dedup2: small changes to improve DDE2 rebuild and rollback functions on startup

Hyper-V Plugin:

hyperv: Fix #10698 Free space debug report missing a field

hyperv: Fix #10699 Make sure restore options are applied

hyperv: rename Script version to Version.

Hyper-V Winapi Plugin:

hyperv-winapi: hyperv-recovery script

hyperv-winapi: attach recoverable actions to vm notes

hyperv-winapi: disable Recover-action to avoid parallel jobs collision

hyperv-winapi: Clean snapshot and reference points on cancel

hyperv-winapi: Fix #10701 new disks handling

hyperv-winapi: Make sure Restore is cleaned up

hyperv-winapi: attach recoverable actions to vm notes

hyperv-winapi: disable Recover-action to avoid parallel jobs collision

hyperv-winapi: fixing migration in cluster mode

hyperv-winapi: get rid of vm_path. Use vhd_path all the way

hyperv-winapi: handle sparse-like restore

hyperv-winapi: Added hyperv-recovery script

hyperv-winapi: use cluster_mode instead of localhost_only

Hyper-V WMI Plugin:

hyperv-wmi: Fix #10698 move restore initialisation from bEventStartRestoreJob to bEventRestoreCommand

Microsoft 365 Plugin:

m365: Add onedrive file exclude example

m365: PST export extension documentation

Nutanix-AHV Plugin:

nutanix-ahv: Accurate flag check for plugin at bacula level

nutanix-ahv: Add protection domain parameter to c part

Openstack VM Plugin:

openstack-vm: Added default volume name in restore to be complient

openstack-vm: Added verbose output for query process check

openstack-vm: Added a timeout on fifo open/close

openstack-vm: Better handling of options errors + additional user info

openstack-vm: Changed signal handler

openstack-vm: Changed parser for interactive delete

openstack-vm: Checked access rights from openstack user to admin openrc

openstack-vm: Removed default user in cinder error message

openstack-vm: Updated instructions if cinder-backup not running

Quobyte Plugin:

quobyte: Fix subvolumes generation for bsnapshot

quobyte: Improve quobyte-fd plugin and bsnapshot subvolumes support

vSphere Plugin:

vsphere: Save vm profile when restoring to correct missing- name from previous possible problems #10758

vsphere: add resiliency to profileallvm management with unexpected vm moref or name

Windows BMR Plugin:

winbmr: don’t mount recovery partition for the estimate command

Miscellaneous:

registration wizard: Fix #10766 correct backslashes

show minimal backtrace if gdb is not installed

Added get_os_version() call and display it to status client/storage/director header

Enabled Device/SyncOnClose by default

Improved gdb btraceback to get the variables inside the faulty thread

Fix #10752 storage option on the restore command line not working

Fix org#2706 Volume encryption force RSA:2048 instead of gpg default

Fix org#2707 Check the NODUMP flag only on files and directories

Improve messages for job protection. issues/27

Report immutable/readonly error messages to the job log

add bVarEstimate to let the FD knows this is an estimate and not a real backup

Bugs fixed/closed since last release:: 10486 10698 10699 10701 10752 10758 10766 10775 10776 9693

Beta 18.0.1 / 06 February 2024 

18.0.1 is a beta release.

Hyper-V Plugin:

Hyper-v: Fix #10530 fix ConvertVMSnapshotToRefpoint

Hyper-v: make sure the full restore relies on self-generated bmp file, not vhdx.exe

Hyper-v: Add a test that validates a whole drive integrity over full backup-restore

Hyper-v: Add a test that validates disk integrity over full backup-restore (used for linux01)

Hyper-v: Add PORTABLE data flag

Hyper-v: Change full backup size to hd logical size. Fix pwd null.

Hyper-v: Update pattern and integrity tests

Hyper-v: debug vhdx,exe output on restore. Return io_status

Amazon EC2 Plugin:

amazon-ec2: Add new snapshot-instance-volume timeout parameters

amazon-ec2: Introducing new parameters for timeout control on different objects. Libraries upgrade

BWeb:

bweb: Fix #10652 NDMP volume not required in connection part of the automation wizard

bweb: Fix #10658 display media view for multiple volumes at once

bweb: Fix #10680 update links to online plugin documentation

bweb: Fix #10684 About incorrect –vm parameter with scan_datacenter

Cloud Plugin:

cloud: Fix #10291 Assume that driver ls can return error 1 when the path doesnt exist. Handle it in generic driver

cloud: Fix #10291 clean_cloud_volume uses “ls”, so we handle its return code the same way

cloud: Fix #10685 TruncateCache at endofjob was not processed due to wrong transfer status verification

cloud: proof guard truncation

cloud: Fix #10591 missing n in returned lists in AWS

cloud: Fix #10604 upload() might not exit with correct error code in AWS

Dedup 2 Plugin:

dedup2: Fix #10714 vacuum delete all entries of the index

dedup2: Vacuum continues despite an error (or cancel) while reading volumes

dedup2: check rwlock for deadlock error

dedup2: display orphan ref per volume

dedup2: don’t block backups during vacuum merge

dedup2: fix vacuum crashes hashes=0x0 connection is not fully registered

dedup2: improve vacuum merge, better detect empty extents

dedup2: index rebuild can reset extent status to USED

dedup2: lock index for flush_entries() during merge

dedup2: vacuum improvements: messages, accounting and cancel handling

Documentation:

docs: Automated Disk Backup navigation fixed 2

docs: Moving Ingestion and Network Restore to Backup Strategies

Microsoft 365 Plugin:

m365: Activity & Export PST WIP

m365: Activity reports backup feature. Improved message when failing to register app. PST project initiation

m365: Add activity parameters

m365: Exclude files with regex options

m365: Reference activity params so they can be interpreted

Openstack VM Plugin:

openstack-vm: Add additional safeguard when looking for drivers in installation

openstack-vm: Add check for admin-rc file to avoid unknown crash in configure

openstack-vm: Adjusted install script

openstack-vm: Query -q for cinder install now has better error handling

Nutanix-AHV Plugin:

nutanix-ahv: Add PROTECTION DOMAIN keyword for specific vm backups

nutanix-ahv: Add rest element to query portection domains

nutanix-ahv: Adds query operation and client call to rest API for protection domain

nutanix-ahv: Correct some INTERGER fields to LONG

Security Plugin:

security: Fix incorrect warning about Archive directories

security: Fix warning in lynis output

security: Remove incomplete checks for mysql and postgresql

vSphere Plugin:

vsphere: Add more resiliency to query operations which should solve some scan_datacenter issues #10673

vsphere: Add template marking priviledges. Do not fail the restore if template marking fails

vsphere: Fix vsphere-ctl update command when vm is not found

vsphere: Ignore empty entries

vsphere: Improvements on setting target vms

vsphere: Re-apply v18 features from code without problems to find VMs

Miscellaneous:

Detect unsolvable volume cycle in split_bsr_loop()

Fix #10631 (1) remove unauthorized Jmsg() in BSOCK::recv()

Fix org#2704 about old FD compatibility

Fix org#2705 about issue with accurate checking of new file signature attributes

Registration_wizard: Adding BCenter Tab

Fix FD using 100% CPU after SD crash

lin_tape: Patch for end of medium detection

Add Storage:Director::DisableQueryCommand: and list new storage query cmds

rpms: Fix single item restore issue with rhel9

Bugs fixed/closed since last release:: 10291 10530 10572 10591 10604 10631 10652 10658 10673 10680 10684 10685 10714