Google Drive

Bacula Enterprise Google Workspace Plugin can protect My Drive units associated to users from a workspace, My Drive units of free accounts, as well as Shared Drive units.

It is possible to utilize advanced selection methods to decide exactly what is backed up, as well as control precisely which items to restore and their destinations.

The detailed list of the information protected with this service is:

  • My Drive of users

    • Folders

    • Native Google services files (gdocs, gslides, gpresentation.. Export and download)

    • All other files (regular download)

    • File Versions

    • Trash bin

  • Shared drives

    • Folders

    • Native Google services files (gdocs, gslides, gpresentation.. Export and download)

    • All other files (regular download)

    • File Versions

    • Trash bin

  • Shared permissions (direct access, share links, expiration times..)

  • SharedWithMe User files

  • Files comments

Files will keep their names in the catalog and will be included in a path like this:

  • /@gw/customerId/entitykind/entityname/drives/unitname/path/to/file/name-file.extension

    (where entitykind can be users or shared_drives)

Version History

Google Drive can be configured to retain the history for files/items.

Google Drive hash check

Google Drive service stores a hash for every file hosted, using MD5 algorithm. Bacula Enterprise Google Workspace Plugin calculates this hash and compares it to ones stored in the cloud at backup time, and also at restore time in order to ensure data integrity. Debug mode shows information about these hashes. Please note that this is true only for non native Google files

Google Drive duplicated files

Google Drive stores its information in a different way compared to traditional filesystems. Instead of a tree structure, everything in google drive are pairs of keys and values (data maps). This makes some internal differences regarding the data structure and, for example, it is possible to have the same folder with the same name inside the exact same path. Similarly it is possible to have the same file with the same name several times in the same path.

Bacula Enterprise Google Drive plugin will combine the data inside folders with the same name. This is:

  • From Google Drive:
    mypath/
    DirA/

    f1 f2

    DirA/

    f3 f4

  • To Bacula Catalog:
    mypath/DirA/

    f1 f2 f3 f4

For files with the same name, bconsole will only show one and will restore the last one when using common options to find the most recent backup or a specific job id where the same file is present more than once. However, for example when using BWeb it is possible to see the different versions of each file and get the desired one using the restore Wizard.

Google Apps files

Files that come from Google online services like Google Docs, Google Spreadsheets or Google Slides are exported transparently during the backup process to open formats.

These kind of files do not expose versioning through Google APIs, so drive_version_history will not take any effect on them. The specific list of files affected by this behavior is as follows:

  • Google Docs to odt

  • Google Draw to jpeg

  • Google Photo to jpeg

  • Google Sheets to ods

  • Google Slides to odp

  • Google Scripts to json

Google Photos and Google Sites

Historically, Google Photos and Google Drive have been very close modules. However today they work with separated APIs. Bacula Workspace Plugin is not supporting Google Photos, even if it is planned to support it in the future as a different module of this plugin.

Google Sites do not support export functions so the plugin cannot protect them. Associated files are simply ignored.

Google Drive shares

Bacula Enterprise Google Drive Plugin is able to backup and restore shared elements. These kind of elements require a special treatment, as they are composed of two parts:

  • In the source account, shared elements include special information about the permissions of the share (who and how the share must work)

  • In the destination account, shared elements appear within an special category called ‘SharedWithMe’.

Shared permissions

Bacula Enterprise Google Workspace Plugin will query for the permissions of an item if this item has been shared directly. This means the plugin will not backup inherited permissions. In order to have inherited permissions in a backup, the top element where the original shared permissions were set needs to be included in the backup and in the restore. As an example, if a directory is shared, but we restore only specific files contained in it, those files will not be shared as they were originally. It is necessary to restore the whole directory in order to replicate the original inherited permissions as they were at the time of backup.

The method to store shared permissions is to include them as ‘metadata’ of every file. This implies that permissions can only be restored directly to the Google Workspace service. A File Daemon restore to a local filesystem will only restore files, and shared permissions will not be restored.

Shared permissions can include links. Links are pre-generated URLs that can include expiration dates and other configuration parameters such as scopes, types, or affected identities. Shared permissions restores have special characteristics that must be considered and they are described below:

  • Permission will be generated exactly as it was, but it will be a new permission object. This is similar to the situation with files. A restored file has the same contents as the original, but can include slightly different metadata because creation process was different.

    • If the permission had a static link, a link will be generated, but the associated URL will be different from the original.

Shared permissions are not restored by default. You need to enable the option ‘drive_restore_shared_permissions’ during the restore session.

Shared with me

SharedWtihMe elements of each target account, if included in the FileSet, are backed up in a predefined directory called SharedWithMe inside the top folder of every selected account. For example, for a given account youraccout@yourdomain.com in a workspace called customer_id:

:caption: **SharedWithMe**

``/@gw/customer_id/users/youraccout@yourdomain.com/drive/my drive/sharedWithMe/``

At restore time, sharedWithMe elements are treated as any other regular file. However, it is important to note that sharedWithMe files, as we are in the receiver account, have no sharing permissions.

The plugin has a special parameter at restore time allowing it to skip sharedWithMe elements even if they are selected. This feature is intended to facilitate full restores where source and destination accounts are included. Please, note that a restore of a source account with share elements will present those elements to any receiver account if you enable the option to restore share permissions, as we have discussed in the upper section.

Please, go to the Configuration section of this document to see how to set up the sharedWithMe skip option.

Backup parameters

The list below shows the specific backup parameters that can be set up in order to control the behavior of the drive module.

In order to select the Google Drive module, the common service parameter must be equals or be containing the value drive.

Entities that can include one drive units are: users, groups or sites.

Option

Required

Default

Values

Example

Description

drive_shared_units

No

Valid names of existing shared drives on the selected workspace separated by ‘,’

imagesShared, unitMyCompany

Will backup only selected drive units belonging to the specified entity (user, group or site)

drive_shared_units_exclude

No

Valid names of existing shared drives on the selected workspace separated by ‘,’

webassets

Will backup all drives except the excluded ones in thelist(s), belonging to the specified entity (user, group or site)

drive_shared_units_regex_include

No

Valid regex

*.pages

Backup matching drive units (based in the drive unit name)

drive_shared_units_regex_exclude

No

Valid regex

^site.*

Exclude matching drive units (based in the drive unit name)

drive_files

No

Strings representing existing folders for the given users or shared units separated by ‘,’

Customers, Partners

Backup only specified folders belonging to the selected users

drive_files_exclude

No

Strings representing existing folders for the given users or shared unitsseparated by ‘,’

Personal

Exclude selected folders belonging to the selected users

drive_files_regex_include

No

Valid regex

.*Company

Backup matching drive folders. Please, only provide list parameters (files + files_exclude) or regex ones. But do not try to combine them.

drive_files_regex_exclude

No

Valid regex

.*Plan

Exclude matching drive folders from the selection. Please, only provide list parameters (files + files_exclude) or regex ones. But do not try to combine them. If this is the only parameter found for selection, all elements will be included and this list will be excluded.

drive_include_trash

No

No

0, no, No, false, FALSE, false, off ; 1, yes, Yes, TRUE, true, on

Yes

Include trashed elements from the user or shared drive selected to backup

drive_include_comments

No

No

0, no, No, false, FALSE, false, off ; 1, yes, Yes, TRUE, true, on

Yes

Include comments of every file. Please, notice that performance is lower when this option is enabled as extra requests are needed for every single file to backup

drive_shared_with_me

No

Yes

0, no, No, false, FALSE, false, off ; 1, yes, Yes, TRUE, true, on

No

Include SharedWithMe elements of every target entity in the backup process

drive_version_history

No

No

0, no, No, false, FALSE, false, off ; 1, yes, Yes, TRUE, true, on

Yes

Include Google Drive former versions of every file into the backup process

Restore

The list below shows the subset of restore parameters that can be used to control the behavior of Google Drive module restore operations:

  • destination_user, drive_destination_shared_unit, destination_path, send_report, allow_duplicates

  • drive_skip_sharedwitme, drive_skip_versions, drive_skip_comments, drive_restore_share_permissions,

  • debug, foreign_container_generation

Use cases

The following restore scenarios are supported:

  • Restore files, directories, or file versions to original drive or to a different drive

    • Restore parameters implied: destination_user, drive_destination_shared_unit

  • Restore file(s)/dir(s) or file version(s) to original path or to a different path

    • Restore parameters implied: destination_path

  • Restore file(s)/dir(s) or file version(s) to local file system (general restore where parameter must be set to a path)

  • It is possible to make general restore selections, but avoid restoring versions

    • Restore parameters implied: drive_skip_versions

  • It is possible to restore sharing permissions of implied files

    • Restore parameters implied: drive_restore_share_permissions

  • It is possible to make general restore selections, but specify if backed up shared elements must be considered

    • Restore parameters implied: drive_skip_sharedwitme

  • It is possible to make general restore selections, but specify if backed up file comments must be considered

    • Restore parameters implied: drive_skip_comments

  • It is possible to control whether or not duplicate elements are allowed (based on file id):

    • Restore parameters implied: allow_duplicates

Particularities:

  • If no destination user and no destination shared unit are provided, the destination user or unit will be looked for inside the backed up path, so the destination entity will be the same as the original one

  • If no destination_path is provided, the destination path will be the same as the original one

    • If a destination entity was provided, but no destination_path was provided and the selected file did not belong to the destination entity:

      • A new folder will automatically be created inside the target entity

      • For each ‘foreign’ entity, a new folder will be created

      • Inside each ‘foreign’ entity folder, the original path structure will be preserved when restoring the files

        • *Unless the parameter foreign_container_generation is disabled

For more details about the behavior of each restore parameter, please check the general section of restore parameters.

Fileset examples

Please note that Google Drive Plugin works with two kind of entities:

  • User Drives

  • Shared Drives

By default, if not specifying anything on any parameters, the plugin backups everything. Therefore, in order to only backup users, we need to exclude all shared drives ; in order to backup only shared drives, we need to exclude all users. Below examples should show this more clearly.

Full Google Drive of only one user:

:caption: **Fileset Example**

FileSet {
   Name = fs-gw-drive-adelev
   Include {
      Options { signature = MD5 }
      Plugin = "gw: service=drive credentials_file=/opt/bacula/etc/bacula-gw-plugin-credentials.json customer_id=G39add31l1 admin_user_email=super@baculasystmes.com
       user=adelev@baculasystems.com drive_shared_units_regex_exclude=\".*\""
   }
}

Folders of one user and include sharedWithMe elements:

:caption: **Fileset Example**

FileSet {
   Name = fs-gw-drive-adelev-shared
   Include {
      Options { signature = MD5 }
      Plugin = "gw: service=drive credentials_file=/opt/bacula/etc/bacula-gw-plugin-credentials.json customer_id=G39add31l1 admin_user_email=super@baculasystmes.com
   user=adelev@baculasystems.com drive_shared_units_regex_exclude=\".*\" drive_files=\"dir1,dir2\" drive_shared_with_me=yes"
   }
}

Backup of some specific shared drives:

:caption: **Fileset Example**

FileSet {
   Name = fs-gw-drive-live-2-drives
   Include {
      Options { signature = MD5 }
      Plugin = "gw: service=drive credentials_file=/opt/bacula/etc/bacula-gw-plugin-credentials.json customer_id=G39add31l1 admin_user_email=super@baculasystmes.com
   drive_shared_units=myunit1,myunit2 user_regex_exclude=\".*\""
   }
}

Exclude directories of two users:

:caption: **Fileset Example**

FileSet {
   Name = fs-gw-drive-adjon-users-notemp
   Include {
      Options { signature = MD5 }
      Plugin = "gw: service=drive credentials_file=/opt/bacula/etc/bacula-gw-plugin-credentials.json customer_id=G39add31l1 admin_user_email=super@baculasystmes.com
      user=\"adelev@baculasystems.com,jonis@baculasystems.com\" drive_shared_units_regex_exclude=\".*\" drive_files_exclude=temp"
   }
}