Note

You can download this article as a PDF

Exchange VSS Plugin

See also

• Exchange Single Item Restore

Introduction to Bacula

This chapter is mostly intended for readers without any familiarity with Bacula. We expect that many Windows administrators might decide to research how to back up their Exchange, as the required procedures have changed significantly over time.

Note that the functionality described here is all about traditional Exchange instances, not about Microsoft 365 Software-as-a-Service email functionality. For information about backing up and restoring that, refer to Microsoft 365 Plugin.

Bacula Enterprise provides commercial, enterprise-level support options and additional features compared to the community-supported open source Bacula software and can be one solution considered by many Exchange administrators. In particular, on top of the “normal” open source advantages provided by the community version, it provides a plugin to back up and recover Microsoft Exchange.

High Level Overview

Bacula, like most other enterprise-ready backup software, consists of several parts which work together through a network:

Director (Dir)

is the component that controls all operations.

Storage daemon (SD)

is the component that manages the final targets for backed up data.

File daemon (FD)

is the agent that is run on all systems backed up.

Console

refers to one of the various user interface programs. Terminal, graphical, and web interface programs are available.

There are more Bacula components, but we don’t need to discuss them here. For now, it’s sufficient to understand that Bacula File Daemons (FD s) are deployed to all your clients to back up and require minimal configuration and maintenance, Bacula Storage Daemons (SD s) require a bit more configuration, but normally few configuration changes over time, and all the important configuration exists in one single location, the Bacula Director program (Dir)’s configuration.

Configuration

All Bacula configuration is traditionally done in simple text files, using a text editor. This may seem a bit inconvenient to many Windows administrators used to graphical user interfaces, but it allows configuration under almost all circumstances, even from a handheld computer and through a low-bandwidth network connection.

A graphical, web-based user interface to operate and configure Bacula Enterprise is available for Bacula Systems customers: BWeb Management Suite. This Web GUI allows to configure and execute backing up of Exchange data, as well as most other specific application data.

Platforms

While FD**s are available for a wide range of operating systems including Windows, the server components of **Bacula Enterprise require a Linux or Solaris operating system. This may cause some slight culture shock in Windows-only organizations, but experience shows that for systems dedicated to a single purpose, which can essentially be viewed as a black box, this often poses no serious problems. (Actually, even seemingly Windows-only organizations often run a number of different operating systems, often even Linux, on network infrastructure and storage appliances.)

Bacula Systems still recommends that a Bacula Enterprise administrator should have some fundamental understanding of the operating system chosen as the platform, especially if disaster recovery is one of the reasons to create backups.

Exchange: An Overview

Microsoft Exchange is Microsoft’s server application to provide email, groupware, and unified communication services. Echange runs on Microsoft Windows Servers and consists of a number of services. The actual list of features is quite flexible, as not all organizations will need all the features Exchange can offer.

Backup and restore of Exchange 2003 is not supported by the VSS plugins, so is not discussed in this white paper.

Exchange integrates tightly into Microsoft’s Active Directory (AD) technology and can not be run outside of an AD domain. Both AD and Exchange can be run in clusters, providing better performance and higher reliability than single instance installations. All instances making up an Exchange cluster can be managed centrally.

Both the configuration and the data of an Exchange installation can be spread over a number of computers, but will usually be kept consistent between the involved Exchange instances.

However, several distinct Exchange sites may be closely linked to each other, even sharing some configuration information. For purposes of backup and recovery, these need to be handled independently while an Exchange cluster itself could be considered only one backup data source.

Microsoft provides tools and procedures to allow deployment and management of Exchange installations by non-trained administrators, but to get the best out of Exchange, a trained, dedicated administrator is required. Also, to design, implement, test, and maintain backup and recovery procedures an experienced Exchange administrator is helpful. However, Bacula Systems believes that, with the knowl edge presented in this White Paper, robust backup and recovery scenarios can be implemented even without that level of in-depth knowledge.

As for any application of a certain complexity, the set of data needed to re-create the full functionality of the application requires restoring both the configuration and the actual (user-) data. As Exchange keeps most of its data in a dedicated database, backing up and restoring that data poses the typical challenges of any database backup:

• Consistency of disk files is not guaranteed while the database engine is running normally, because many changes may still be in the computer RAM not yet written to disk;

• ensuring consistency of disk files requires database-specific tools and knowledge;

• and feeding restored data into the database requires database-specific procedures and knowledge.

The Bacula Enterprise VSS plugin handles most of those tasks automatically, only requiring minimal Exchange administrator intervention.

Regarding the configuration of an Exchange installation, backing up the configuration is a bit complicated, because the configuration information is stored in the Windows’ Registry, and can be managed and distributed by AD. Accordingly, to allow the full recovery of an Exchange installation, one has to ensure availability or recovery of the base system’s configuration including the Registry. The Bacula Enterprise VSS Plugin can handle that, as well as the AD information. In an environment with several AD servers, this requires Disaster Recovery (DR) procedures to ensure recovery of at least one AD server.

Naturally, the programs making up Exchange are also required to be running when restoring Exchange data. Ensuring this is in many cases best done by preparing for Bare-Metal Recovery of server machines hosting Exchange.

Microsoft Windows Disaster Recovery is described in more detail in the Bacula Systems White Paper “Windows Bare Metal Recovery”, so we will not go into much detail here.

The Bacula Enterprise VSS Plugin

Volume Shadow Copy Service (VSS) is the name for Microsoft’s snapshot technology, which not only creates snapshots of file systems, but also interfaces to applications to make on-disk data consistent (freezing, quiescing) to allow proper backups of application data.

In the simplest case, a vss-aware application’s data can simply be backed up from disk, reading from the snapshot, not the original file system. This functionality is part of Bacula for a number of years.

However, in order to do online restores (i. e. restores while Windows is running rather than a Bare Metal Recovery) some applications require additional data processing, so using vss alone is not sufficient to back up and restore their data. In particular, understanding of database log files may be required, or, on restore, files that are always locked when Windows is running need to be replaced during a subsequent system boot.

The Bacula Enterprise VSS plugin handles the additional data processing needed to be able to do such online restores of many applications, among them Microsoft Exchange.

Please note, in this White Paper, we discuss only the Bacula Enterprise version 6.x VSS plugin and not the older Exchange plugin (exchange-fd.dll) that was previously supported by the Bacula project. To avoid all possible interference with the VSS plugin, we recommend that the older exchange-fd.dll should not exist in the plugin directory defined in the FD ’s configuration file. If it does, Exchange backups and restores could possibly fail.

The VSS plugin needs to be explicitly included in a FileSet used for backups (the plugin takes care of files that would be backed up, making sure no unnecessary or even harmful files get stored).

In addition, the VSS plugin will also create and store some data that needs to be restored or is needed to control the restore process prior to the actual data files being restored. This data is not stored on the backup media, but in Bacula’s catalog. This allows restoring data in a different order than it is written, but it requires that all the Bacula components: Bacula Director program, Bacula Storage Daemon and Bacula File Daemon must be the exact same versions to ensure that all components know how to handle that additional job data. It also means that you can not restore Exchange data with only the volumes available, after losing the catalog. 1

As backing up through the vss plugin uses the functionality provided in Microsoft application specific writers, some limitations exist when using this approach to backup. Essentially, Bacula can not do things the Microsoft VSS application writers do not provide.

Important Points

For Exchange, some important points are:

• In Exchange 2005, the Recovery Storage Groups (RSGs) exist, so in principle, the Exchange Administrator can enable restore to the RSG, and whe Bacula does a full restore it will go into the RSG rather than restore to the active Exchange database. Once the RSG is restored, the Windows Exchange Administrator can do individual mailbox restores. We have not yet tested this.

• In Exchange 2010 Microsoft eliminated the Recovery Storage Group code that was in previous versions of ex. The Recovery Storage Group allowed the Exchange Administrator to automatically direct a restore to the Recovery Storage Group. Thus other techniques such as use of the Recovery Database are necessary for individual mail box recovery.

• Exchange 2010 permits a single Recovery Database (RDB), which works somewhat similarly to the Recovery Storage Groups in prior versions. The difference is to restore to a Recovery Database, the user must explicitly specify the restore location. This feature is not implemented in the initial version 6.0 release of the VSS plugin but available as of Bacula Enterprise version 6.0.4.

• The Exchange VSS plugin is supplied in a separate Bacula Enterprise VSS plugin installer that must be executed after installing the File daemon.

• The Exchange VSS plugin described in this white paper may not work with the old exchange-fd.dll plugin installed in the PluginDirectory. Please make sure it is not present. How to check which plugins are loaded is described in the general VSS Plugin White paper provided by Bacula Systems.

• We have not fully tested the Differential backup feature, and recommend that you not use it, since Microsoft’s concept of Differential backups is different from Bacula’s. Doing a Differential backup after several Incremental backups, could possibly result in lost log files and thusunrecoverable databases.

• We recommend that you use Exchange 2010 SP2 since it corrects many of the problems in previous Exchange versions.

• When you do a restore, you must always do a full restore (item 5 on the restore prompt) that includes the Full backup as well as all the Incremental backups. If you attempt to select Jobs to restore by individual JobIds, the restore will fail. This is because in order for the Microsoft VSS writer to work correctly, it needs all the information about all the backup jobs that were made.

• If you unmount (dismount in Microsoft terminology) a database, its time and date stamp will be updated, and thus it and its log files will be backed up on the next Bacula backup. The same is true if you reboot your system. Backing up active Exchange data files without the VSS plugin and not using Accurate mode will cause incomplete data to be stored.

• Bacula’s Accurate mode must be enabled in your Job resource. If it is not, you will get a warning message, but much worse, the files being backed up will not be the set of files that needs to be backed up (files may be missed or erroneously be backed up). Bacula needs Accurate mode enabled in order to optimize the backup.

• Do not use Bacula’s Accurate mode to check MD5 (accurate flag ‘5’) or SHA1 (accurate flag ‘1’) signatures as they will produce a large number of error messages during backup operations. In fact, the default criteria for Accurate Mode, “mcs”, are sufficient. Using a Signature directive with an MD5 or SHA1 is OK.

• We do not support and do not guarantee that the VSS plugin can properly backup and restore Exchange in a clustered environment. Databases that are part of A Database Availability Group, however, can usually be backed up and restored correctly.

• Due to the tight coupling between Exchange and Active Directory, we do not support backup of Exchange and restore to a machine that has a different Active Directory environment. It should work if Active Directory on both machines is identical, i. e. the machines are members of the same AD domain.

Fortunately, these are usually not serious restrictions, as Exchange by default provides settings to make partial restores unnecessary, so actually restoring Exchange data would only be needed in case of a disaster recovery, never during regular operations. This also implies that Exchange should always be run in a clustered environment (Database Availability Group or DAG), as Microsoft recommends.

Plugin Options

• index Creates additional internal parameters needed for Single Item Restore

• cinclude=<glob> Specifies the names of mailbox databases to backup. It is possible to specify the cinclude parameter multiple times on the plugin command line.

• cexclude=<glob> Specifies the names of mailbox databases to exclude from the backup. It is possible to specify the cexclude parameter multiple times on the plugin command line.

Backup Scenarios

Due to Microsoft Exchange’s feature to not delete mailbox items, but keep them hidden from regular users when deleted, recovery operations to get individual items back should happen rarely, if at all. 2 Thus, backing up Exchange is mostly useful as part of a dr plan. Accordingly, Bacula Systems’s procedures focus on complete backup of Exchange and full restores to the original location.

Recovery operations are similarly focused on (Exchange-specific) DR procedures. Single object restores are discussed but should not be the main goal of deploying a backup and recovery tool for Exchange.

For the purposes of this White Paper, DR does not imply a full Bare Metal Restore of the underlying Operating System and all applications and data on it, but refers to the data only. In other words, we are concerned about Application-level Disaster Recovery. This has some consequences on the presented scenarios:

• We assume the version of remains the same, i. e. the backed up data was written by an exact identical version of Exchange as the one it is restored to,

• the operating version is the same for the backup source and backup target,

• and the target locations for the Exchange databases remain unchanged or are part of the same Exchange deployment.

To clarify the outlined backup and recovery scenario, the following table may be helpful:

Backup and Recovery Scenario

Situation	Next step
Complete Mailbox Server lost	Set up Windows and Exchange with Mailbox Server role, or do Bare-Metal Recovery
All Mailbox Databases lost	Restore complete sequence of all backups to original location
One Mailbox Database lost	Restore complete sequence of this individual Mailbox Database to original location
One Mailbox Database corrupt	Restore complete sequence of this individual Mailbox Database to an alternate location and proceed as necessary
Single Mailbox or single Mailbox Item required	Restore to Recovery Database and pick out desired Items

Minimal Exchange

In our examples, we will initially use a single server running in an AD domain. The Exchange instance uses only one Mailbox storage database which is located on a disk drive with the drive letter D:, but the main ex installation is located on drive C:. Accordingly, to create a useful backup of the Exchange data, the disk drives lettered C: and D: need to be included in the backup 3. The resulting FileSet is shown below.

File Set {
  Name = Exchange-CD
  Include {
    Options {
      Verify = pnugsi1
      Signature = SHA1
    }
    File = c:/backmeup
    File = d:/backmeup
    Plugin = "vss:/@EXCHANGE/"
  }
}

Note the dummy files included on each drive where data is located.

To allow the VSS plugin of Bacula Enterprise to work correctly, backup jobs must be done in accurate mode. Accordingly, the job definition we use in our examples includes that option; the job definition we use is shown below.

Job {
  Name = Exchange
  Type = Backup
  Level = Incremental
  Accurate = Yes
  File Set = Exchange-CD
  Client = wsb-exch10-fd
  Storage = File
  Messages = Standard
  Pool = Tier1
  Priority = 10
  Write Bootstrap = "/var/lib/bacula/%n.bsr"
}

Backing Up

To create a backup of a data set, one needs to specify a FileSet to use the Exchange functionality of Bacula Enterprise’s VSS plugin as outlined in chapter The Bacula Enterprise VSS Plugin. One essential consideration is that at least one file from each drive that is used by Exchange has to be included explicitly.

On the Exchange side, databases that are to be backed up with bsee’s VSS plugin must be set to not do circular logging. This has to be done for each database and can be achieved with the Database Properties panel in Exchange Management Console (see figure Mailbox Database Maintenance Properties need to Allow Overwriting, the last line of the checkboxes, below the arrow) or through ps.

Backup Levels

Backups with Bacula can be of three levels: Full, Differential, and Incremental. A Full backup is simple to understand; it just backs up everything included in its FileSet then allows Exchange to remove the log files that are no longer needed. A Differential level backup backs up everything changed since the latest Full level backup, but does not trigger log file removal. Incremental backup takes everything since the last backup job run and allow Exchange to remove any log files that are no longer needed.

Transaction Log Truncation

A transaction log as written by many database engines stores all database transactions done so they can be replayed in a recovery situation. To ensure that disk space used by those logs can be reclaimed, those logs should be truncated from time to time. When using the VSS Exchange plugin, log truncation is automatically done by the Windows Exchange writer when a Full or an Incremental backup succeeds.

Single Database Backup

A single database backup is possible, by excluding other databases from the file set and including the wanted database. Code below presents a fileset structure where 4 databases are present on volume G 4. MDB03 will be backuped while MDB01, MDB02 and MDB04 will be excluded.

File = "G:/backmeup.txt" ## volume G: will be part of the snapshot

Plugin = "vss:/@EXCHANGE/
   cexclude=*/*/*/MDB01/*   ## to cexclude MDB01/Log or MDB01/File
   cexclude=*/*/*/MDB01      ## to cexclude MDB01 folder
   cexclude=*/*/*/*/MDB01/* ## extra level to cexclude MDB01/Log or MDB01/File when in a Replica
   cexclude=*/*/*/*/MDB01   ## extra level to cexclude MDB01 folder when it's a Replica

## same for DB02
   cexclude=*/*/*/MDB02/* cexclude=*/*/*/MDB02 cexclude=*/*/*/*/MDB02/* cexclude=*/*/*/*/MDB02
## same for DB04
   cexclude=*/*/*/MDB04/* cexclude=*/*/*/MDB04 cexclude=*/*/*/*/MDB04/* cexclude=*/*/*/*/MDB04

## cinclude DB03 with the same logic
   cinclude=*/*/*/MDB03/* cinclude=*/*/*/MDB03 cinclude=*/*/*/*/MDB03/* cinclude=*/*/*/*/MDB03

However, a VSS snapshot is always global to a single volume. In this example all databases present on volume G: will get their transaction logs truncated, although some are not backuped. To avoid this, consider separating the databases on different volumes or backup all databases present on the volume in 1 single job.

Differential Backups

We have not sufficiently tested Differential backups with Exchange, and suspect that due to a difference in concept of Differential backups between Exchange and Bacula, using Differential backups may result in lost log files. Thus we recommend that you do not use Differential backups with the Exchange plugin.

Recovery

The Mailbox Database needs to be unmounted and marked to allow overwriting to be restored. An example on how to unmount a Mailbox Database is shown below. In Exchange Administrative Center, navigate to the database to restore, select it, click on ... and then Dismount:

Preparing an Exchange Database for Restore with the Exchange Administrative Center

Then edit the Database Mailbox Maintenance properties by clicking on the pencil icon and checking the box This Database can be Overwritten by a Restore:

Mailbox Database Maintenance Properties need to Allow Overwriting

Alternatively, these steps can also be done with PowerShell commands. Please refer to the documentation available with the Microsoft Exchange Management Shell.

If the database to be restored exists, for example when testing, or if parts of its files still exist (which might be the case after a file system fault) it is best to remove the remaining parts before restoring.

To do this to just safely rename the whole database directory with Window’s shell, cmd. The path can be found through Exchange Administrative Center, from the Database Properties panel, as shown in figure Database Path in Exchange Administrative Center, or with the PowerShell-based Exchange Management Shell. A way to do this, and the result, is shown in figure Database Path in Exchange Management Shell.

Database Path in Exchange Administrative Center

The full database path and name cannot be edited but can be selected and copied.

Database Path in Exchange Management Shell

The path name is shown completely – remember that you need the whole directory, not only the .edb file.

The fault that can be observed when restoring with database files remaining is caused by out-of-sequence log files that are found by the Exchange-specific database recovery and is reported in the Job Report with lines like these:

Exchange VSS Writer failed restoring a backup with error code -515
when performing an integrity-check of the log files to be used for
database recovery after restore for 'Mailbox Database 1568811476'.

If you encounter those problems the safest way to proceed is to redo the whole restore process, this time removing the remaining database files before the actual restore job starts.

Restore procedure with Bat

In BAT press the restore Button (see figure Starting the Restore Procedure).

Starting the Restore Procedure

Select the corresponding job for your exchange server and choose the “@EXCHANGE” folder presented in the directory overview (see figure Restore Selection)

Restore Selection

It is important to remove the directive in the “Where” line as shown in figure Restore Where, otherwise the database may be restored to an alternate location, which is not supported without additional preparation. This is explained in chapter Restoring to an Alternative Location.

Restore Where

After the actual restore (which could also be initiated from a Bacula console) is finished the Exchange machine may need to be rebooted, during which some of the actual database files will be moved to their final locations. If the reboot is needed, Bacula will print a message to that effect in the Job report. After the reboot, or after the restore is done, the administrator will need to mount the restored database(s), which is done similar to dismounting, and the server will be functional again.

For reasons of safety, Bacula Systems recommends to verify that the setting to allow the database to be overwritten is turned off when mounting it after the server’s restart.

Restoring a Single Database

By default, if you have multiple Exchange databases, they will all be restored when following the procedures given above. If you wish to restore only a single database, it is possible, but during the restore file selection procedure instead of marking the “@EXCHANGE” folder presented in the directory overview, you must descend several levels – usually four – into the “@EXCHANGE” directory, until you find the names of all the databases that you have. At that point you may mark only the database or databases that you want restored. An example of this, where only one database is selected for restore is given below.

  $ pwd
  cwd is: /@EXCHANGE/Microsoft Exchange Writer/Microsoft Exchange Server/Microsoft Information Store/WSB-EXCH10/
  $ ls
  Mailbox A/
  Mailbox Database 1568811476/
  mark "Mailbox Database 1568811476"
  16 files marked.
  $

Database Availability Groups

Database Availability Groups (DAG) is one of the main new features of Exchange 10. The DAG provides automatic, database-level recovery from a database, server, or network outage. In order to use the full potential of the DAG groups, two or more Exchange Servers are required. DAG is based on continuous replication, enabling the Exchange environment to offer high availability and site resilience. In a DAG the recovery mechanism is done automatically. Hence if a mailbox in the Exchange cluster has a failure another Exchange Server (within the cluster) will automatically fix the database failure. In a scenario of a Database Availability Group, the database is replicated to all involved Servers. In this situation, the Backup of only one instance is necessary because all databases are kept synchronized.

Other Ways to Cluster Exchange

There are several other ways to cluster environments. In order to give a brief but sophisticated overview it will be focused on mainly two scenarios in order to cover a simple and a more advanced solution. The first solution is the easiest one. To cluster an Exchange Server a shared storage is used and a Heartbeat in order to monitor the two nodes. If one of the Exchange Servers now realizes a fail-over situation, the running node takes over all connections and work in order to still provide availability. In this scenario no separation between the database and server is implied.

The second scenario involves in total four Exchange Servers where three actives nodes are productive and the fourth is set in a passive mode in order to cover the fail-over case. In the background redundant, shared mass storage is used to store logs and database files. With this environment an effective usage of the hardware is possible as well as a high level of security in terms of availability and consistency of the database is provided. It has to be mentioned that, if this environment is used, the database files have to be saved from just one Exchange server, since data is stored on a central, shared storage device. The other Exchange server instances will automatically pick up database changes.

Restoring to an Alternative Location

Note

This procedure is available for Exchange 2010 and later, not with older versions.

In order to do restores of single mailboxes or single mailbox items, it is necessary that the administrator restores to a secondary database and then moves the interesting items to their final destination.

For this purpose, Microsoft provides special Recovery Databases which are used to (temporarily) store the recovered objects before they are moved to their final location. These Recovery Databases are managed quite differently than regular Mailbox Databases, as they are only intended as a temporary storage during recovery operations. See http://technet.microsoft.com/en-us/library/dd876954.aspx for details.

Creating and operating on Recovery Databases is done exclusively through Exchange Management Shell; Bacula Systems provides scripts making these tasks easier.

The high-level overview of the steps required to restore individual items looks like this:

1. If necessary, create a Recovery Database.

2. Unmount this database and enable restores to it (this is the same procedure as described in Recovery)

3. Restore data (usually restoring an individual database, as in section Restoring a Single Database). Use a where= setting to direct the restored data to the Recovery Database.

4. Move the items of interest to their final location using Microsoft Exchange Management Shell.

5. After checking the success of the restore, remove the Recovery Database.

All those steps can be done through Powershell commands with Microsoft Exchange Management Shell, and some of them can only be done this way.

Most of those steps have to be done on a Windows machine and need to work with the ex server data will be restored to. For stability reasons, Bacula Systems recommends that you run the actual restore to the ex server that physically hosts the recovery mailbox, and in this case it’s most convenient to work on that server itself.

A typical restore session done the way Bacula Systems recommends accordingly looks like this:

1. Log on to a ex mailbox database server with credentials allowing to manage Microsoft Exchange.

2. Open the Microsoft Exchange Management Shell. In the start menu, this will look similar to figure Microsoft Exchange Management Shell in the Start Menu.

3. A command windows opens which will look similar to the one in figure Microsoft Exchange Management Shell Startup Window. You should know the Powershell essentials if you’re an Administrator; in-depth information can be found on Microsoft’s web site, for example at http://technet.microsoft.com/de-de/library/bb123778

4. Use appropriate commands to set up a Recovery Database and enable restores to it. Bacula Systems provides an example script which is also attached to this white paper in its pdf version. This script should, if stored in C:\Users\Administrator\Documents, be sourced with the``Powershell`` dot command, i. e. ."\C:\Users\Administrator\Documents\file.ps1". An example can be seen in figure Calling the Preparation Script, and sample output is shown in figure Preparation Script Output. In Script to Prepare Recovery Database we provide the listing of this script. The commands that are essential are New-MailboxDatabase and Set-MailboxDatabase.

5. Do the actual restore with Bacula Enterprise

   1. Set up the restore job as usual, using any convenient console. Make sure you restore a full sequence of backups beginning with a full level backup, and all jobs based on this one up to the point in time required.

   2. Select the files to restore – normally, a specific database will be selected for restore. See above in ch:singledbrestore for details.

   3. Make sure no file name mangling is done by adding a prefix or suffix.

   4. Set the restore location to the Recovery Database name. Do not use the path but the plain name!

   5. Let the job execute.

   An example restore session using bconsole is shown in figure Restore Job Relocating a Single Mailbox Database.

6. Move the relevant items from the Recovery Database to their final location. The Recovery Database has to be mounted for that purpose. The following steps can be used:

   1. Execute Mount-Database Bacula EnterpriseRecovery (substitute the Recovery Database name if necessary) in .

   2. In , you can use the following command to get a listing of all the mailboxes that exist in the Recovery Database (again, substitute the Recovery Database Name if necessary):

      Get-MailboxStatistics -Database Bacula EnterpriseRecovery | Format-List DisplayName
      

   3. The following Powershell command in restores a complete mailbox from the Recovery Database to the regular user’s database (which must exist) into a dedicated folder:

      Restore-Mailbox -Identity "DisplayName" -RecoveryDatabase Bacula EnterpriseRecovery -TargetFolder "Recovered 2010-07-10" -RecoveryMailbox "DisplayName"
      

      The DisplayName must be the displayed name of an existing mailbox – the source, i. e. the mailbox inside the Recovery Database is identified by the RecoveryMailbox parameter.

   4. The user can now arrange (or delete) recovered items as needed. It should be noted that items are restored without any permissions set, so that the user may have to give herself permissions to manage the restored folders.

      An example of the process and results of this recovery procedure is given in figures Restoring a Mailbox with Microsoft Exchange Management Shell and Restored Mailbox and Folder Permissions.

7. To clean up after all data is moved to its final location, unmount the Recovery Database, remove it from ex, and delete the data files on disk. The exs commands Dismount-Database, Remove-MailboxDatabase and Remove-Item -Recurse are suitable to achieve that.

Microsoft Exchange Management Shell in the Start Menu

Microsoft Exchange Management Shell Startup Window

Calling the Preparation Script

Preparation Script Output

Restore Job Relocating a Single Mailbox Database

   *restore client=wsb-exch10-fd before="2012-07-09 23:59:00" where="Bacula EnterpriseRecovery"
    restoreclient=wsb-exch10-fd

   First you select one or more JobIds that contain files
   to be restored. You will be presented several methods
   of specifying the JobIds. Then you will be allowed to
   select which files from those JobIds are to be restored.

   To select the JobIds, you have the following choices:
   ...
        5: Select the most recent backup for a client
   ...
       13: Cancel
   Select item:  (1-13): 5
   The defined FileSet resources are:
        1: Exchange-CD
        2: Exchange-SS
        3: WindowsUserC
   Select FileSet resource (1-3): 1
   +-------+-------+----------+-------------+---------------------+------------+
   | jobid | level | jobfiles | jobbytes    | starttime           | volumename |
   +-------+-------+----------+-------------+---------------------+------------+
   |   362 | F     |      258 | 397,610,415 | 2012-07-09 00:26:41 | F-0007     |
   |   363 | I     |       23 |   6,294,004 | 2012-07-09 00:32:58 | F-0007     |
   +-------+-------+----------+-------------+---------------------+------------+
   You have selected the following JobIds: 362,363

   Building directory tree for JobId(s) 362,363 ...  +++++++++++++++++++++++++++
   +++++++++++++++
   257 files inserted into the tree.

   You are now entering file selection mode where you add (mark) and
   remove (unmark) files to be restored. No files are initially added, unless
   you used the "all" keyword on the command line.
   Enter "done" to leave this mode.

   cwd is: /
   $ cd @EXCHANGE
   cwd is: /@EXCHANGE/
   $ cd "Microsoft Exchange Writer/"
   cwd is: /@EXCHANGE/Microsoft Exchange Writer/
   $ cd "Microsoft Exchange Server/"
   cwd is: /@EXCHANGE/Microsoft Exchange Writer/Microsoft Exchange Server/
   $ cd "Microsoft Information Store/"
   cwd is: /@EXCHANGE/Microsoft Exchange Writer/Microsoft Exchange Server/Microsoft
    Information Store/
   $ cd WSB-EXCH10/
   cwd is: /@EXCHANGE/Microsoft Exchange Writer/Microsoft Exchange Server/Microsoft
    Information Store/WSB-EXCH10/

   $ ls
   Mailbox A/
   Mailbox Database 1568811476/
   $ mark "Mailbox Database 1568811476"
   131 files marked.
   $ done

   Bootstrap records written to /var/lib/bacula/s-1-dir.restore.10.bsr

   The job will require the following
      Volume(s)                 Storage(s)                SD Device(s)
   ===========================================================================

       F-0007                    File                      FileStorage             

   Volumes marked with "*" are online.

   133 files selected to be restored.

   Run Restore job
   JobName:         RestoreFiles
   Bootstrap:       /var/lib/bacula/s-1-dir.restore.10.bsr
   Where:           Bacula EnterpriseRecovery
   Replace:         always
   FileSet:         Test Set
   Backup Client:   wsb-exch10-fd
   Restore Client:  wsb-exch10-fd
   Storage:         File
   When:            2012-07-09 00:38:03
   Catalog:         MyCatalog
   Priority:        10
   Plugin Options:  *None*
   OK to run? (yes/mod/no): yes
   Job queued. JobId=364
   *

Restoring a Mailbox with Microsoft Exchange Management Shell

Restored Mailbox and Folder Permissions

Permission level “Owner” allows the user to manage and delete restored folders.

Troubleshooting

As with any complex application, problems during backup or recovery may happen with Microsoft Exchange. In this chapter, we try to guide you through resolving some problems.

Backup

The most common problem during backup is that not all required file systems are included in the snapshot set; in these cases, the VSS writer responsible for handling ex will refuse the snapshot’s creation. The solution is to explicitly include the required drive by adding a dummy file entry to the file set used.

If the backup itself works, but no files or not all required files are being backed up, this is often caused by some of the required infrastructure not being functional. For example, in a DAG configuration, if the Microsoft Exchange Replication service is not running because of a cluster environment issue, backups may be incomplete.

Restore

In most cases, when Bacula Enterprise reports a restore as successful, but actual Mailbox access shows no restore took place, the cause of the problem can be found in Windows’ Event log. In most cases, detailed error messages will also be included in the Job report. For those Events that we encountered, we have a listing prepared below, explaining how to proceed.

Windows Events

This is a list of Events that you may find in the Windows Application Event log after a restore operation, which is reported as successful by Bacula Enterprise. When looking for problem causes, always start with the earliest Event related to the current restore attempt (this is most easily done by filtering the Event log by date and time, using the start and end dates of the restore job). We provide an example in figures Windows’ Event Log Filtered, Showing Cause for Failed Restore and Job Report is OK, but actual Restore Failed. In figure Exchange Message after Database Mount Failure the corresponding message from ex when trying to mount the database is shown, and figure Application Log Messages after Database Mount Failure shows an overview of the corresponding Windows Event Log messages. Note that, in most cases, there will be many failures reported in Windows’ Event Log, but normally the first Error is the one you are interested in. With the examples we provide it should be easy to correlate the different reports and drill down to the underlying problem.

Windows’ Event Log Filtering

Windows’ Event Log Filtered, Showing Cause for Failed Restore

Job Report is OK, but actual Restore Failed

...
20-Mar 00:31 wsb-exch10-fd JobId 20: Exchange VSS Writer successfully
 restored the backup set to database ...
20-Mar 00:31 wsb-exch10-fd JobId 20: Exchange VSS Writer will perform
 database recovery on database ...
...
20-Mar 00:31 wsb-exch10-fd JobId 20: Exchange VSS Writer failed restoring
 a backup with error code -528 when processing post-restore tasks ...
20-Mar 00:31 wsb-exch10-fd JobId 20: Exchange VSS Writer failed with error
 code -528 when processing the post-restore event.

If any databases were restored, they are likely in a dirty-shutdown state.
20-Mar 00:31 wsb-exch10-fd JobId 20: A VSS writer has rejected an event with
 error 0x00000000, The operation completed successfully.
. Changes that the writer made to the writer components while handling the
 event will not be available to the requester.
Check the event log for related events from the application hosting the VSS writer.
...
20-Mar 00:31 s-1-dir JobId 20: Bacula s-1-dir 6.0.0.5 (06Mar12):
  Build OS:               x86_64-unknown-linux-gnu suse 12.1
  JobId:                  20
  Job:                    RestoreFiles.2012-03-20_00.30.23_01
  Restore Client:         wsb-exch10-fd
  Start time:             20-Mar-2012 00:30:25
  End time:               20-Mar-2012 00:31:58
  Files Expected:         17
  Files Restored:         17
  Bytes Restored:         150,022,642
  Rate:                   1613.1 KB/s
  FD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Restore OK

Note that the actual Job report contains more detailed information!

Exchange Message after Database Mount Failure

Application Log Messages after Database Mount Failure

Problem Resolution by Windows Event

Source	Event ID	Task Category	Resolution
MSExchangeRepl	3154	Service	Mount the database through Exchange Management Console
MSExchangeIS	9663	Exchange VSS Writer	Dismount the Database prior to restoring
MSExchangeIS	9751	Exchange VSS Writer	Try the following: Move the *.env-file out of the Mailbox Database directory. Run eseutil /r <Three-char logfile base name> in the Mailbox Database directory. Re-run the restore as usual. The database should now be restored correctly.
ESE	440	Logging/Recovery	Remove the existing database log files from the data directory prior to restoring
ESE	518	Logging/Recovery	see above
ESE	454	Logging/Recovery	see above

Script to Prepare Recovery Database

# $Id: preparebacularestore-E10.ps1 1387 2014-01-26 13:57:57Z arno $
# (C) 2012 Bacula Systems SA
#
# This script is an EXAMPLE of how to prepare Exchange 2010 for an alternate location
# restore of data backed up with the vss plugin's @EXCHANGE module
#
# Use at your own risk, make sure to audit before using!
#
# Configuration is done right at the top.
#
# NULL means detect server automatically. Fails if there's more than one
# To hard-code the server to use, try
#  $server = "<nodename>"  with a hard-coded nodename or
#  $server = hostname to use the local hostname as reported by the hostname command
$server = $null


if ($server -eq $null) {
  $mbs = @(Get-ExchangeServer | where {$_.IsMailboxServer} | ForEach-Object {$_.Name})
#  $mbs += @(Get-ExchangeServer | where {$_.IsMailboxServer} | ForEach-Object {$_.Name})
  if ($mbs.Count -gt 1) {
        "More than one Mailbox host. Define explicitly, please!"
        Exit
  } else {
        $server = $mbs.Get(0)
  }
}

$me = hostname
if ([string]::Compare($me, $server, $true)) {
  "The Mailbox server we would use is not the local machine... we do not support that!"
  Exit
}
"Using server $server"

$mdbs = @(Get-MailboxDatabase | where { $_.Revovery -or $_.AllowFileRestore})

if ($mdbs.Count -gt 0) {
  "The following Mailbox Databases are Recovery Databases or set to allow Recovery. This may cause problems, please fix and retry!"
  $mdbs | Format-Table -AutoSize Server,Identity,AllowFileRestore,Recovery
  Exit
}

$tgtdrive = (Get-WMIObject Win32_logicaldisk | Sort-Object -Property FreeSpace -Descending).Get(0)

"Using drive $($tgtdrive.DeviceId) for the RDB"
"There are about $("{0:N0}" -f ($tgtdrive.FreeSpace / 1024 / 1024 /1024) ) GBytes free on it"


do {
  $tgtdir = $tgtdrive.DeviceId + "\" + [System.IO.Path]::GetRandomFileName()
} while (Test-Path $tgtdir)

"$tgtdir is our working directory."

New-Item -Path $tgtdir -Type "directory" | Out-Null

$rdbn = "Bacula EnterpriseRecovery"
$suff = $null

while (($rdb = Get-MailboxDatabase -Identity ($rdbn + $suff) -WarningAction SilentlyContinue -ErrorAction SilentlyContinue) -and ($rdb.Count -gt 0)) {
  "MailboxDatabase " + $rdbn + $suff + "exists... trying next one"
  $suff++
}

$rdbn += $suff

New-MailboxDatabase -Recovery -Name $rdbn -Server $server -EdbFilePath ($tgtdir + "\RecoveryDB.edb") -LogFolderPath $tgtdir | Out-Null

Set-MailboxDatabase -Identity $rdbn -AllowFileRestore $true -WarningAction SilentlyContinue -ErrorAction SilentlyContinue | Out-Null

"You can Restore with where=" + $rdbn + " now"

Limitations

• The restart command has limitations with plugins, as it initiates the Job from scratch rather than continuing it. Bacula determines whether a Job is restarted or continued, but using the restart command will result in a new Job.

1

Regularly saving the catalog database dumps is a practice strongly recommended by anyway, so this should not be an issue in any production system.

2

The Retention policy is defined in Exchange Management Console and can be applied per Database or per Mailbox. Read more about it at technet.microsoft.com/en-us/library/dd297955.aspx

3

Starting with version 12.5, specifying the volumes is not mandatory anymore

4

Starting with version 12.5, specifying the volumes is not mandatory anymore