bscan
The bscan program is designed to recreate database (Catalog) records from
backup data stored on one or more Volumes. This process is typically necessary
in the event of a disaster or when one or more Volumes have been pruned or purged
from your Catalog, resulting in the loss of their records, or for Volumes that have been archived.
Important
In the event of a total disaster of the Catalog database, bscan should be used solely to recover jobs that ran after the point-in-time-recovery of the Catalog database. It means that you should first restore the latest backup of the Catalog database, followed by using bscan to recover the jobids that ran after that.
Note
If you perform a scan on Volumes that have been previously purged, you will still be able to restore data from those Volumes.
However, unless you modify the Job and File retention times for the Jobs added through scanning, any subsequent backup Job with the same name will lead to the records being pruned.
The default configuration for recovered Volumes designates their status as Archive. The timestamp for the last write operation on the Volume is recorded as the time of recovery, and the Volume retention period is established at one month. Consequently, the data within the Catalog will become invalid one month after the completion of the bscan recovery process.
Below is the general usage of the bscan command, along with the available options:
Usage: bscan [ options ] <bacula-archive>
-b bootstrap specify a bootstrap file
-c <file> specify configuration file
-d <nn> set debug level to <nn>
-dt print timestamp in debug output
-m update media info in database
-D <driver name> specify the driver database name (default NULL)
-n <name> specify the database name (default bacula)
-u <user> specify database user name (default bacula)
-P <password> specify database password (default none)
-h <host> specify database host (default NULL)
-t <port> specify database port (default 0)
-p proceed inspite of I/O errors
-r list records
-s synchronize or store in database
-S show scan progress periodically
-v verbose
-V <Volumes> specify Volume names (separated by |)
-w <dir> specify working directory (default from conf file)
-? print this message
Note
When using MySQL or PostgreSQL, specifying a working directory is unnecessary, as bscan
automatically identifies the database locations. However, if your database is secured,
you may need to provide the database name (-n option), along with the user name
(-u option) and/or the password (-p option).
Important
For bscan to function properly, it requires at least a minimal valid database present.
If your database exists but some records have been pruned, you are ready to proceed.
However, if your database has been lost or destroyed, you must first ensure that
the SQL server (MySQL or PostgreSQL) is operational. Next, you must create the Bacula database
(typically named bacula), and set up Bacula tables using the scripts
in the /opt/bacula/scripts/ directory. This is explained in the
Installation chapter.
Finally, before scanning into an empty database, you must start and stop the
Director with the appropriate bacula-dir.conf file so that it can create
the Client and Storage records that are not stored on the Volumes.
Without these records, the scanning process cannot link the Job records
to the proper Client.
The main use cases for bscan are outlined below.
Using bscan to Compare Volume to Existing Catalog
To compare the contents of a Volume to an existing Catalog without changing the Catalog, it is essential to refrain from using the -m or the -s options in order to avoid modifying the Catalog. It is important to note that not all records in the Catalog are found on the Volumes, which is why we generally advise against using bscan except for testing purposes. The most effective method to compare the contents of a Volume with those in the Catalog is to use one of the Verify Job options.
bscan -c bacula-sd.conf -v -V Vol001\|Vol002 /dev/nst0
This command will provide insight into the expected outcomes without making any changes to your Catalog. Be aware that you may need to adjust the path to the Storage Daemon’s configuration file, the Volume name, and the name of your Bacula device (tape, disk, dedup, etc.). This command requires reading the entire Volume, which can be time-consuming if it contains a large amount of data; therefore, you may wish to promptly use the command provided below.
Note
If you are writing to a disk file, ensure that you substitute the device name with the path to the directory containing the Volumes, as this must align with the Archive Device specified in the configuration file.
Using bscan to Synchronize Catalog with Volumes
With some care, bscan can also be used to synchronize your existing Catalog with a Volume.
Although there have been no reported instances of bscan causing damage to a Catalog,
it is advisable to backup the existing Bacula database prior to executing bscan,
as it modifies the Catalog. See Compacting Your PostgreSQL Database or
Compacting Your MySQL Database for details of making
a copy of your database, or Backing Up Your Bacula Database to use the make_catalog_backup.pl
program provided by Bacula.
For example, if all records associated with one or both of these Volumes are removed from
the database, you can utilize the bscan tool to recreate the Catalog
entries for those Volumes. Following that, you can employ the restore command in the
Console to retrieve the desired data.
bscan -c bacula-sd.conf -v -V Vol001\|Vol002 /dev/nst0
This command will provide insight into the expected outcomes without making any changes to your Catalog. Be aware that you may need to adjust the path to the Storage Daemon’s configuration file, the Volume name, and the name of your Bacula device (tape, disk, dedup, etc.). This command requires reading the entire Volume, which can be time-consuming if it contains a large amount of data; therefore, you may wish to promptly use the command provided below.
Note
If you are writing to a disk file, ensure that you substitute the device name with the path to the directory containing the Volumes, as this must align with the Archive Device specified in the configuration file.
Then, to actually write or store the records in the Catalog, add the -s option as follows:
bscan -s -m -c bacula-sd.conf -v -V Vol001\|Vol002 /dev/nst0
When writing to the database, if bscan finds existing records, it will generally either
update them if discrepancies are detected or leave them unchanged. Therefore, if the Volumes being scanned
are already partially or fully in the Catalog, the existing data will remain unaffected.
Any missing data will simply be added accordingly.
If you have multiple Volumes, it is recommended to scan them using:
bscan -s -m -c bacula-sd.conf -v -V Vol001\|Vol002\|Vol003 /dev/nst0
Due to the command line length limitation of 511 bytes imposed by bscan, if you
possess an excessive number of Volumes, it will be necessary to create a bootstrap file manually.
See the Bootstrap for bscan article.
For instance, the .bsr file for the above example might appear as follows:
Volume=Vol001
Volume=Vol002
Volume=Vol003
Note
bscan does not support supplying Volume names on the command line and at the same time in a bootstrap file. Use only one or the other.
It is advisable to always specify the tapes in the order in which they were written.
Failing to do so may result in incomplete or improper restore of any Jobs that span multiple Volumes.
However, bscan can handle scanning tapes that are not sequential. Any incomplete
records at the end of the tape will simply be ignored in that case.
While this may be acceptable if you are merely repairing an existing Catalog, it could lead to an incorrect database state if you are creating a new Catalog from scratch.
If all necessary Volumes are not included in a single bscan command, the tool will be
unable to accurately restore records that span across two Volumes. Therefore, it is far
more effective to specify two or three Volumes within a single bscan command
(or in a .bsr file) rather than executing bscan multiple times, each time with
a single Volume.
Using bscan to Recreate Catalog from Volume
This is the mode for which bscan is most useful. You can either bscan into a freshly
created Catalog, or directly into your existing Catalog, provided you have backed up the current Bacula database as previously outlined.
It is generally advisable to start with a freshly created Catalog that contains no data.
This can be achieved using the create_bacula_database, make_bacula_tables, and grant_bacula_privileges scripts.
Additionally, bscan has the capability to recreate and list records,
including Restore Objects and Plugin Objects.
Assuming you start with a single Volume named TestVolume1, you execute a command such as:
./bscan -V TestVolume1 -v -s -m -c bacula-sd.conf /dev/nst0
In the case of multiple Volumes, simply add them to the initial Volume, separating each with a vertical bar.
It may be necessary to escape the vertical bar with a forward slash to avoid shell interpretation,
e.g. TestVolume1\|TestVolume2. The -v option was added for verbose output,
although it can be omitted if desired. The -s option instructs bscan to store information in the database.
The physical device name, /dev/nst0 (which corresponds to the ArchiveDevice
value in the bacula-sd.conf) is specified following all the options.
For example, after performing a Full backup of a directory followed by two Incremental backups, having recovered the Bacula Catalog as described previously, and using the backup Job boostrap file, the following command was executed:
./bscan -b bootstrap.bsr -v -s -c bacula-sd.conf /dev/nst0
which produced the following output:
bscan: bscan.c:182 Using Database: bacula, User: bacula
bscan: bscan.c:673 Created Pool record for Pool: Default
bscan: bscan.c:271 Pool type "Backup" is OK.
bscan: bscan.c:632 Created Media record for Volume: TestVolume1
bscan: bscan.c:298 Media type "DDS-4" is OK.
bscan: bscan.c:307 VOL_LABEL: OK for Volume: TestVolume1
bscan: bscan.c:693 Created Client record for Client: Rufus
bscan: bscan.c:769 Created new JobId=1 record for original JobId=2
bscan: bscan.c:717 Created FileSet record "Kerns Files"
bscan: bscan.c:819 Updated Job termination record for new JobId=1
bscan: bscan.c:905 Created JobMedia record JobId 1, MediaId 1
bscan: Got EOF on device /dev/nst0
bscan: bscan.c:693 Created Client record for Client: Rufus
bscan: bscan.c:769 Created new JobId=2 record for original JobId=3
bscan: bscan.c:708 Fileset "Kerns Files" already exists.
bscan: bscan.c:819 Updated Job termination record for new JobId=2
bscan: bscan.c:905 Created JobMedia record JobId 2, MediaId 1
bscan: Got EOF on device /dev/nst0
bscan: bscan.c:693 Created Client record for Client: Rufus
bscan: bscan.c:769 Created new JobId=3 record for original JobId=4
bscan: bscan.c:708 Fileset "Kerns Files" already exists.
bscan: bscan.c:819 Updated Job termination record for new JobId=3
bscan: bscan.c:905 Created JobMedia record JobId 3, MediaId 1
bscan: Got EOF on device /dev/nst0
bscan: bscan.c:652 Updated Media record at end of Volume: TestVolume1
bscan: bscan.c:428 End of Volume. VolFiles=3 VolBlocks=57 VolBytes=10,027,437
It is important to highlight that bscan prints a line of output when each major record is created.
However, due to the volume of output, it does not print a line for each file record unless
the -v option is specified two or more times in the command line.
Note
The restore process using bscan is not identical to the original creation
of the Catalog data. This is because certain data such as Client records and other
non-essential data such as Volume reads, Volume mounts, etc. is not stored on the Volume,
and thus is not restored by bscan. The results of bscanning are, however, perfectly
valid, and will permit restoration of any or all the files in the Catalog using the
standard Bacula console commands.
If you are starting with an empty Catalog and
expecting bscan to reconstruct it, this may not occur.
It is essential to ensure that your bacula-dir.conf file is the same as what it previously
was, meaning it must contain all the appropriate Client resources so that they will be
recreated in your new database before running bscan. Typically, when the Director
starts, it will recreate any missing Client records in the Catalog.
Additionally, even if the Volumes (Media records) are successfully restored in the database,
in the case of tape volumes, their inchanger status and slot assignments will not be correctly
configured. Consequently, you will need to repair that by using the update slots command.
Important
It is crucial to note that instead of using bscanning, one should always prioritize recovering the previous Catalog backup.
In the case of a Job record, the newly assigned JobId will typically differ from the original JobId.
For example, for the first JobId above, the new JobId is 1, but the original JobId is 2.
This is is not a cause for concern as it is a standard characteristic of database operations.
bscan will ensure that everything remains organized.
Although bscan indicates that a Client record for Client: Rufus was created three times,
it was actually only created the first time. This is expected.
You will also notice that it reads an end of file after each Job (Got EOF on device…).
The final line will present the total statistics for the bscan.
If you had included a second -v option to the command line, Bacula would have been even more verbose, revealing nearly all the details of each Job record it encountered.
Now, if you start Bacula and enter a list jobs command to the console program, you will get:
+-------+----------+------------------+------+-----+----------+----------+---------+
| JobId | Name | StartTime | Type | Lvl | JobFiles | JobBytes | JobStat |
+-------+----------+------------------+------+-----+----------+----------+---------+
| 1 | kernsave | 2002-10-07 14:59 | B | F | 84 | 4180207 | T |
| 2 | kernsave | 2002-10-07 15:00 | B | I | 15 | 2170314 | T |
| 3 | kernsave | 2002-10-07 15:01 | B | I | 33 | 3662184 | T |
+-------+----------+------------------+------+-----+----------+----------+---------+
which aligns with its previous state prior to recovery and restore process with bscan.
All Jobs and Files found on the tape have been successfully restored, including the majority
of the Media records.
The Volume (Media) records restored will be marked as Achive, with a retention period of
one month, so that they cannot be rewritten without the intervention of an operator.
It is important to acknowledge that bscan cannot restore a database to its exact prior condition,
as much of the less critical information within the database is not preserved on the Bacula Volume.
However, the reconstruction is sufficiently comprehensive, allowing for the execution of
restore command to yield valid outcomes.
An interesting aspect of restoring a Catalog backup using bscan is that the backup
occurs while Bacula is running and writing to a tape. At the point the Catalog backup
is created, the tape that Bacula is writing to may contain, for instance, 10 files.
Subsequently, after the Catalog backup is completed, there will be 11 files on the
Volume Bacula is writing to.
his discrepancy highlights the difference between the contents of the backed-up
Catalog and the actual files present on the tape. If, after restoring a Catalog,
an attempt is made to write on the same Volume that was used to backup the Catalog,
Bacula will detect the difference in the number of files registered in the Catalog
compared to what is on the volume, and it will fix the Catalog automatically.
Disaster Recovery
bscan can be useful in a disaster recovery scenarios, particularly following the failure of
a hard disk. If there is no valid Catalog database backup, it is possible to use bscan to
recover the Catalog records from the Volumes. In cases where a valid bootstrap file is
unavailable for the recovery, or if a Volume has been pruned/purged/recycled but not overwritten,
bscan can assist in reconstructing your database.
This reconstructed database can subsequently be used to restore your data as usual using Bacula.
For instance, if you have created backups in the Volumes “Vol001” and “Vol002”, and all
associated records have been removed from the Catalog, you can use bscan to restore
those records:
bscan -c bacula-sd.conf -v -V Vol001\|Vol002 /dev/nst0
If all necessary Volumes are not included in a single bscan command, the tool will be
unable to accurately restore records that span across multiple Volumes. It is more
effective to specify multiple volumes in a single bscan command (or in a .bsr file)
rather than executing bscan multiple times.
After bscan
When using bscan to enter the contents of the Volume into an existing Catalog,
employing the options -m update media info in database or -s synchronize or store in database
will result in the Volume status being set as Archive, with the LastWritten time
updated to the current time at which bscan accessed the Volume.
If neither of these two options is used, the Volume status or the LastWritten values are not changed.
To permit Bacula to reuse this Volume and to prune jobs and files contained within it after
the retention period has elapsed, it is necessary to manually adjust the volume status to
either Full or Used.
It is possible to modify the volume status (volstatus) by using the bconsole update command.
update volstatus=Used volume=Vol-0001
Go back to the Tape/Volume Management chapter.
Go back to the main Bacula Enterprise Management page.