Scenario 2. Dedup Legacy and Dedup2 Running on the Same Storage Daemon

Goal of the scenario: Migrating data from your current dedup1 engine to a new dedup2 engine that use the same Storage Daemon host. The new dedup2 engine uses the same Storage Daemon port, SDPort 9103.

Note

In this scenario, there is only one Storage Daemon running.

It is required to have both dedup engines running at the same time to migrate the data.

If you decide to have both dedup engines in the same host and use the same Storage Daemon port, it is required to modify the current dedup1 engine configuration to use the new DedupEngine resource.

The new dedup2 configuration format uses the new DedupEngine resource, and it is not compatible with the old dedup1 configuration format. It means that it is not possible to add the dedup2 engine using the new Dedupengine Resource in the current Storage Daemon configuration if there is already a Dedup1 engine configured using the old configuration format.

For example, it is not possible to have both, the old and new formats, configured in the same Storage Daemon configuration file:

# From bacula-sd.conf
Storage {
 Name = Dedup1-sd
 Working Directory = /opt/bacula/working
 Pid Directory = /opt/bacula/working
 Plugin Directory = /opt/bacula/plugins
 Dedup Directory = /mnt/bacula/dedup1/containers
 Dedup Index Directory = /mnt/SSD/dedup1/index
 Maximum Container Size = 2TB
}

and

Dedupengine {
  Name = Dedupengine_name_with_dedup2
  Dedup Directory = /mnt/bacula/dedup2/containers
  Dedup Index Directory = /mnt/SSD/dedup2/index
  Driver = Dedup2
  Maximum Container Size = 2TB
}

However, it is possible to have one dedup legacy engine, and one dedup2 engine in the same Storage Daemon by modifying the current dedup1 configuration to the new format as documented in the New Dedupengine Resource section in the Global Endpoint Deduplication 2 article.

To modify your current legacy Dedup engine to the new configuration format:

  1. Stop the Storage Daemon

  2. Modify the current dedup1 engine settings by creating a DedupEngine resource with Driver = Legacy, and the same index and containers directories of your current dedup engine:

Dedupengine {
  Name = Dedupengine_legacy
  Dedup Directory = /mnt/bacula/dedup1/containers
  Dedup Index Directory = /mnt/SSD/dedup1/index
  Driver = Legacy
  Maximum Container Size = 2TB
}

  1. Add the Dedupengine = Dedupengine_legacy directive to all the dedup1 devices:

Autochanger {
  Name = FileDedup1-Chgr1
  Device = FileDedup1-Dev0, FileDedup1-Dev1
  Changer Command = /dev/null
  Changer Device = /dev/null
}

Device {
  Name = FileDedup1-dev0
  Drive Index = 0
  ...
  Device Type = Dedup
  DedupEngine = Dedupengine_legacy <--- @aga, should we highlight this?
}

Device {
  Name = FileDedup1-dev1
  Drive Index = 1
  ...
  Device Type = Dedup
  DedupEngine = Dedupengine_legacy <--- @aga, should we highlight this?
}

  1. Remove or comment the index and containers directories from the Storage resource configuration:

# From bacula-sd.conf
Storage {
 Name = Dedup1-sd
 Working Directory = /opt/bacula/working
 Pid Directory = /opt/bacula/working
 Plugin Directory = /opt/bacula/plugins
 # Dedup Directory = /mnt/bacula/dedup1/containers <--- you may remove this line
 # Dedup Index Directory = /mnt/SSD/dedup1/index <--- you may remove this line
 Maximum Container Size = 2TB
}

  1. Add the new dedup2 engine resource and autochanger/devices to the same bacula-sd.conf file:

Dedupengine {
  Name = Dedupengine_name_with_dedup2
  Dedup Directory = /mnt/bacula/dedup2/containers
  Dedup Index Directory = /mnt/SSD/dedup2/index
  Driver = Dedup2
  Maximum Container Size = 2TB
}

Autochanger {
  Name = FileDedup2-Chgr1
  Device = FileDedup2-Dev0, FileDedup2-Dev1
  Changer Command = /dev/null
  Changer Device = /dev/null
}

Device {
  Name = FileDedup2-dev0
  Drive Index = 0
  ...
  Device Type = Dedup
  DedupEngine = Dedupengine_name_with_dedup2
}

Device {
  Name = FileDedup2-dev1
  Drive Index = 1
  ...
  Device Type = Dedup
  DedupEngine = Dedupengine_name_with_dedup2
}

  1. Start the Storage Daemon.

After you have the new Dedup2 engine correctly configured and running, you may use Migration Jobs to migrate jobs from the dedup1 storage to the new dedup2 storage. Once all jobs have been migrated, the current dedup1 engine can be decommissioned.

More details about Migration Jobs: Migration and Copy.