Delta Backup
Some modules of the Google Workspace Plugin implement a specific mechanism to handle incremental or differential backups in order to optimize them and offer good performance. In general, the plugin will store information about the current state of the running backup. It will use that checkpoint information in the next backup, so Google APIs return only the new elements or modified elements since then.
The Google Workspace Drive API provides a ‘Changes’ function to track changes of some objects in an efficient way. Bacula Enterprise Google Workspace Plugin uses this function for its Delta mechanism in order to speed up Incremental/Differential processes.
For Google Email API there is a ‘History’ function where each email stores a sequential number which is tied to the date when it was created. Bacula Enterprise Google Workspace Plugin uses this function for its Delta mechanism in order to speed up Incremental/Differential processes.
Please note that using this function is not a mandatory requirement, therefore Incremental or Differential backups will function also with the services that may not support it by navigating through the information and comparing the date with the last valid backup of the current chain.
Delta function has some important characteristics:
In Google Drive, the function can only be used for full entities (full user or full shared drive). This means that selecting specific paths to backup will not trigger the Delta function.
In Google Email it is possible to select specific labels and still get the benefits of the delta function.
Google Drive delta tokens can expire at some point, or even become invalid due to internal Google issues. If this situation happens, the plugin will try to start a new Delta cycle
Any situation where the Delta function cannot be used will trigger a regular Full/Inc/Diff where every element is listed and selected or discarded according to the item dates.
The Delta backup cycle is described below:
Full backup: All entity elements are backed up. The current token (token_1) is received from the API (or the most recent historyId for Email module). This token is stored locally by the FD.
Incremental 1 backup: token_1 is used to retrieve changes since token_1’s generation so every change is backed up. A new token is generated and stored locally by the FD.
Incremental 2 backup: token_2 is used to retrieve changes since token_2’s generation so every change is backed up. A new token is generated and stored locally by the FD.
And so on…
Tokens are stored in an file placed in a path defined by the path parameter of the plugin. The name is: jobname.deltaLink
The file stores tokens required for every execution and it is renewed (emptied) during every Full backup execution.
This file is also backed up in the backup itself, so it can be restored manually, before an Incremental/Differential execution in case it was lost and in case you don’t want to run a Full backup again.
Here we can see an example of the contents of the file, with 3 executions and one user entity involved. The structure is tree-based, so it is easy to understand what would be generated in the case of backing up other entities:
{
"jobName": "GW_DEMO_JOB",
"deltaServices": {
"DRIVE": {
"entities": {
"JorgeShared1": {
"id": "JorgeShared1",
"name": "jorgeshared1",
"containers": {
"jorgeshared1": {
"id": "jorgeshared1",
"description": "jorgeshared1",
"deltaEntries": [
{
"job": "pluginTest.2022-05-12_10.54.30_15",
"date": "May 12, 2022, 10:54:30 AM",
"delta": "5677"
},
{
"job": "pluginTest.2022-05-12_11.00.59_18",
"date": "May 12, 2022, 11:00:59 AM",
"delta": "5677"
},
{
"job": "pluginTest.2022-05-12_11.03.35_21",
"date": "May 12, 2022, 11:03:35 AM",
"delta": "5683"
}
]
}
}
},
"kara@baculasystems.com": {
"id": "kara@baculasystems.com",
"name": "kara@baculasystems.com",
"containers": {
"my drive": {
"id": "my drive",
"description": "my drive",
"deltaEntries": [
{
"job": "pluginTest.2022-05-12_10.54.30_15",
"date": "May 12, 2022, 10:54:30 AM",
"delta": "490"
},
{
"job": "pluginTest.2022-05-12_11.00.59_18",
"date": "May 12, 2022, 11:00:59 AM",
"delta": "490"
},
{
"job": "pluginTest.2022-05-12_11.03.35_21",
"date": "May 12, 2022, 11:03:35 AM",
"delta": "493"
}
]
}
}
}
}
}
}
}