Verify File System Integrity

The term “File System Integrity” can be interpreted in two ways: First, the integrity of the file system at the technical level, to ensure that file metadata like permissions, or the file contents, do not change unexpectedly. In case of a disk failing, or faults at the file system driver level, comparing what is stored on disk against what was last backed up may indicate problems before other checks trigger – for example, a file system check is usually only run at system startup, and S.M.A.R.T. 2-monitoring is well known to not trigger in cases of failing hard disks.

Bacula implements a verification of the actual file system contents against what was last backed up as a DiskToCatalog verification. Again, not the contents of files is compared but only metadata stored in the Catalog; this keeps the “cost” of those verifications manageable as the backed up data does not need to be read from the storage media, but using checksums file contents changes are likely to be noticed. These verification Jobs are discussed in more detail in section Disk to Catalog Verification.

The other interpretation of “File System Integrity” aims at a higher level of integrity. The goal here is to detect changes to files that should not change during normal operations. For many unix / linux systems, the contents of the /etc/ and /sbin/-directories are typical candidates: These are system configuration files and programs, which, except for very special cases, will only be intentionally modified by an administrator. As such, unexpected modifications to these files may indicate security issues like break-ins and unathorized attempts to change the system’s behaviour or to install malicious software.

Bacula can keep track of those files by collecting their metadata in its catalog without actually saving them, and later compare the system’s files against the cataloged information, which is done by a pair of jobs called InitCatalog and Catalog, respectively. This sort of verification is explained in InitCatalog/Catalog Verification.

When comparing Bacula Enterprise’s verify features against those of other backup packages, a user may miss the ability to compare the file system’s data against what is stored on backup media. We believe this feature is, in practice, not needed: A byte-per-byte comparison will only rarely be more reliable than the checksum-based verification Bacula implements, and the ability to separate the comparison into two distinct steps – Catalog vs. File System and vs. Volume data – allows more flexibility and better utilization of the available resources.

Linking together Bacula’s DiskToCatalog and VolumeToCatalog jobs results in a “near-end-to-end” comparison, where discrepancies between the source file system and the data on the backup media will be reported. If this linkage is desired, it is possible to schedule the needed jobs in a way that they are executed directly after each other.