Scenario 3. Dedup Legacy and Dedup2 Running on the Same Host on Two Different Storage Daemons

Goal of the scenario: Migrating data from your current dedup1 engine to a new dedup2 engine that use the same Storage Daemon host. The new Dedup2 engine uses a different Storage Daemon port, for example, SDPort 9104.

Note

In this scenario, there are two Storage Daemons running: one using 9103 SDPort, the other using 9104 SDPort.

It is required to have both dedup engines running at the same time to migrate the data.

In this case, it is not required to modify the current dedup1 configuration to use the new format, unless you wish to use the new configuration format to have a standard configuration for all the dedup engines in your environment.

As the new dedup2 engine is to be configured in a new bacula-sd.conf file, there is no need to modify the current bacula-sd.conf file used by Dedup1.

Thus, you should keep the current Dedup1 configuration in the bacula-sd.conf file (if the SDPort = 9103 is not explicitly configured, it is used by default by the Storage Daemon).

Example of the configuration of your dedup1 engine, running on port 9103:

# From bacula-sd.conf
Storage {
 Name = Dedup1-sd
 Working Directory = /opt/bacula/working
 Pid Directory = /opt/bacula/working
 Plugin Directory = /opt/bacula/plugins
 Dedup Directory = /mnt/bacula/dedup1/containers
 Dedup Index Directory = /mnt/SSD/dedup1/index    # Recommended to be on fast local SSD storage
 Maximum Container Size = 2TB # Maximum 511 containers can be created, please adapt to your need
}

Then, create a new bacula-sd-dedup2.conf file for the new dedup2 engine, using port 9104:

 Storage {
  Name = Dedup2-sd
  SDPort = 9104
  Working Directory = /opt/bacula/working
  Pid Directory = /opt/bacula/working
  Plugin Directory = /opt/bacula/plugins
  Maximum Concurrent Jobs = 20
 }

Director {
  Name = bacula-dir
  Password = "password-for-the-director-to-access-the-storage"
}

 Dedupengine {
   Name = Dedupengine_name_with_dedup2
   Dedup Directory = /mnt/bacula/dedup2/containers
   Dedup Index Directory = /mnt/SSD/dedup2/index
   Driver = Dedup2
   Maximum Container Size = 2TB
 }

 Autochanger {
   Name = FileDedup2-Chgr1
   Device = FileDedup2-Dev0, FileDedup2-Dev1
   Changer Command = /dev/null
   Changer Device = /dev/null
 }

 Device {
   Name = FileDedup2-dev0
   Drive Index = 0
   ...
   Device Type = Dedup
   DedupEngine = Dedupengine_name_with_dedup2
 }

 Device {
   Name = FileDedup2-dev1
   Drive Index = 1
   ...
   Device Type = Dedup
   DedupEngine = Dedupengine_name_with_dedup2
 }

Messages {
  Name = Default
  Director = bacula-dir = all
  Append = "/opt/bacula/log/bacula-sd.log" = All, !Skipped
}

To start the new dedup2 Storage Daemon, you can use this command: sudo -u bacula /opt/bacula/bin/bacula-sd -dt -c /opt/bacula/etc/bacula-sd-dedup2.conf, but it is recommended to create a systemd service for the new Dedup2 Storage Daemon, for example, the bacula-sd-dedup2.service file:

# Description:
#    Used to start the bacula Dedup2 storage daemon service (bacula-sd-dedup2)
#    enable : systemctl enable bacula-sd-dedup2
#    start : systemctl start bacula-sd-dedup2
#
#

# from http://www.freedesktop.org/software/systemd/man/systemd.unit.html
[Unit]
Description=Bacula Enterprise Storage Daemon service
Requires=network.target
After=network.target
RequiresMountsFor=/opt/bacula/working /opt/bacula/etc /opt/bacula/bin
/mnt/bacula/dedup2/containers /mnt/SSD/dedup2/index /mnt/bacula/dedup2/volumes

# from http://www.freedesktop.org/software/systemd/man/systemd.service.html
[Service]
Type=forking
User=bacula
Group=disk
Environment="LD_LIBRARY_PATH=/opt/bacula/lib"
ExecStart=/opt/bacula/bin/bacula-sd -dt -c /opt/bacula/etc/bacula-sd-dedup2.conf
StandardError=syslog
TimeoutStopSec=3min
LimitMEMLOCK=infinity
LimitNOFILE=10000
LimitNPROC=10000
OOMScoreAdjust=-1000

[Install]
WantedBy=multi-user.target

Then, you can enable and start the new dedup2 engine Storage Daemon using systemd commands:

$ sudo systemctl enable bacula-sd-dedup2.service
$ sudo systemctl start bacula-sd-dedup2.service

After you have the new Dedup2 engine correctly configured and running, you may use Migration Jobs to migrate jobs from the dedup1 storage to the new dedup2 storage. Once all jobs have been migrated, the current dedup1 engine can be decommissioned.

More details about Migration Jobs: Migration and Copy.