Sharepoint
Sharepoint is a special module in the Bacula Enterprise Microsoft 365 Plugin. While all the other modules rely only on the M365 Graph API to perform all of their operations, this module relyes on the PnP PowerShell project, that simplifies many automation operations over Sharepoint Online APIs. More information about this project can be found here: https://github.com/pnp/powershell
Pnp Framework Usage Background
The current recommendation from Microsoft to handle any Azure Cloud data is to use the MS Graph API, and that is exactly the strategy that the M365 Plugin is following. However, the Graph Sharepoint API presents several limitations which do not allow the plugin to perform a full backup/restore operation at the time of the writing this document:
It is not possible to create Sites
It is not possible to get or upload web pages
It is not possible to create some types of columns for Sharepoint lists
It is not possible to create or update items including many types of allowed Sharepoint fields
That makes a full process of backup/restore not possible today using just the MS Graph API. On the other hand, Microsoft has native Sharepoint Online APIs. These APIs are expected to be entirely replaced in the future with MS Graph APIs, but today they are supporting more advanced functions for Sharepoint Online than MS Graph does.
Sharepoint Online APIs are fully featured and significantly complex. They may be used for many different purposes. In order to simplify many automation operations over Sharepoint Online using these APIs, as well as others over other M365 services, the PnP PowerShell project was created, and the Bacula Enterprise Microsoft 365 Plugin relies on it for doing its export/import procedures involving the parts not supported by Graph APIs.
Sharepoint Protected Information
The information protected by this module is detailed below, distinguishing what is protected using each method:
Sites Metadata (MS Graph)
Sites Sharing permissions (MS Graph)
Site Templates (PnP Powershell) which includes:
Site metadata
Lists metadata
ListItems metadata
WebPages metadata
DocumentLibraries metadata
Lists (MS Graph Object)
ListItems (MS Graph Object)
ListItem versions
Document Libraries Files (MS Graph Drive Items)
Files versions
The catalog stores the information with the following structure:
/@m365/tenantName/sites/siteName/siteName.site → Graph Object
siteName.site.template.xml → PnP template
drives/ → document libraries containing files
documents
sitePages
…
lists/ → Lists containing metadata
documents
sitePages
…
Version History
OneDrive and SharePoint can be configured to retain the history for files/items.
Sharepoint Backup General Guidelines
As detailed in the previous section, Sharepoint backups are composed by several elements. The most relevant element is the Sharepoint Site Template generated by the PnP framework. That xml file contains all the information needed to re-generate a complete Sharepoint site from scratch, including the different lists and their elements, but without the files included in the document libraries. Those files are backed up individually with their original name inside the ‘drive’ folder when the parameter ‘sharepoint_include_drive_items’ is activated.
For large sites and also when a large number of sites will be protected, it us a good practice to separate the backup of the sites in individual jobs, as well as separate each of them in two parts: - The first part is a Sharepoint site backup with ‘sharepoint_include_drive_items’ set to ‘No’ - The other part will be Drive module backup (setting the same ‘site’ parameter, with the same name).
The mentioned separation will have the site template in the Sharepoint backup, while document libraries files will be kept in the drive module backup.
Sharepoint backup can protect site images. It is important to note the images are usually stored inside the ‘Site Assets’ list, which is a system list. In order to backup system lists it is necessary to activate the ‘include_system’ backup parameter.
Sharepoint backup can also protect site pages. Howerver, similarly to site images, these ones are store in another system list, called ‘Site Pages’. In order to include its contents, ‘include_system’ backup parameter is required.
In the previous section, other files corresponding to lists and listitems, outside the sharepoint site template, are mentioned. The purpose of these files is to allow potential indinvidual operations of a single list or a single listitem. However, this information is already included in the site template, when it comes to full sharepoint site restore operations.
Sharepoint supports Full, Incremental and Differential backup levels. However, please note that Incremental and Differential levels will not download lists metadata (list items mentioned in the previous paragraph). Those elements are only included in Full backups, as they do not present modification dates to be properly handled.
Backup Parameters
The list below shows the specific backup parameters that can be set up in order to control the behavior of the sharepoint module.
In order to select the sharepoint module, the common service parameter must be equals or be containing the value sharepoint.
Option |
Required |
Default |
Values |
Example |
Description |
|---|---|---|---|---|---|
lists |
No |
Valid names of existing lists from the selected sharepoint site separated by ‘,’ |
siteassets, documents |
Backup only selected lists of the included sharepoint sites |
|
lists_exclude |
No |
Valid names of existing lists from the selected sharepoint site separated by ‘,’ |
sitepages |
Backup all lists of the included sharepoint sites except the ones listed in this parameter |
|
lists_regex_include |
No |
Valid regex |
*.pages |
Backup matching lists |
|
lists_regex_exclude |
No |
Valid regex |
^site.* |
Exclude matching lists |
|
sharepoint_subsites |
No |
No |
0, no, No, false, FALSE, false, off ; 1, yes, Yes, TRUE, true, on |
Yes |
Include subsites in selected sites for sharepoint backup |
sharepoint_hidden_lists |
No |
No |
0, no, No, false, FALSE, false, off ; 1, yes, Yes, TRUE, true, on |
Yes |
Include hidden sharepoint lists of selected sites for sharepoint backup |
sharepoint_system_include |
No |
No |
0, no, No, false, FALSE, false, off ; 1, yes, Yes, TRUE, true, on |
Yes |
Include system documentLibraries (sitepages, siteassets, etc). |
sharepoint_version_history |
No |
No |
0, no, No, false, FALSE, false, off ; 1, yes, Yes, TRUE, true, on |
Yes |
Include item former versions of every list item into the backup process |
sharepoint_include_drive_items |
No |
Yes |
0, no, No, false, FALSE, false, off ; 1, yes, Yes, TRUE, true, on |
No |
Every list item of sharepoint can be (and usually it is) associated to a OneDrive file. Using this option we can control if we want to backup drive information for every linked element or not. If we disable it we could run OneDrive backup in a separate job (service=drive site=siteName), or even simply use both services normally, so they get protected with the same job (service=sharepoint,drive). If you suffer performance difficulties, we recommend to try one of these two strategies, both implying deactivating this sharepoint_include_drive_items parameter |
Restore
The list below shows the subset of restore parameters that can be used to control the behavior of sharepoint module restore operations:
send_report, local_path_json_objects, skip_versions, restore_share_permissions, debug -> Common restore parameters
sharepoint_newsite_name, sharepoint_newsite_owner -> To be used for full site restore operations
destination_site, destination_path, sharepoint_list_name -> To be used for list item restore operations
sharepoint_local_template_path -> To be used for full site restore operations where we need to apply a local filesystem template, previously restored or manually built
Use Cases
For restoring over Microsoft 365 purposes, this plugin can: - Restore an entire Sharepoint site to a new one - Restore regular files over existing document libraries - Restore listItems over an existing list (source list must be the same list kind than the destination list) - Restore a list over an existing site - Restore any element locally to the File Daemon
Bellow more details on each use case:
Restore an entire site or a particular sub-site using a new site name (combines PnP and Graph actions)
Restore parameters implied: sharepoint_newsite_name,sharepoint_newsite_owner
*Note: The template provisioning operation can show some errors in the joblog because of the reasons explained in the general limitations section. Even if you see those messages, please check the restore result on Sharepoint Online as it will usually be correct as the joblog itself will suggest. If you still experience problems, please check the troubleshooting section.
*Note: This restore method will restore locally Graph objects like listItems or any other not applicable in the path pointed by: local_path_json_objects
Restore files from document libraries to their original location or to a different document library (Graph based)
Refer to section Restore, as document libraries are managed as drive objects
Restore listItems to their original list or to a list with the same structure in the same or different site (Graph based)
listItems can only contain simple fields (text, numbers..)
Restore parameters implied: sharepoint_list_name
Restore a list to its original site or to a different site (Graph based)
lists can only contain simple columns (text, numbers..)
Restore parameters implied: destination_site
Restore any information to a local filesystem (general restore where parameter must be set to a path)
Please note that the following scenarios are not currently supported:
To simultaneously restore a site and its subsites:
It is necessary to specifically select the parent site and unmark its subsites in order to restore the parent site
It is necessary to select only one subsite for each restore job in order to restore them
Restoring a site template over an existing site is not supported. It is required to always assign sharepoint_newsite_name, and use with a non-existing name
For more details about the behavior of each parameter, please check the general section of restore parameters.
Fileset Examples
Backup sharepoint site:
FileSet {
Name = fs-m365-site-company-events
Include {
Options { signature = MD5 }
Plugin = "m365: service=sharepoint tenant=57uia43-d107-17a2-a2g2-aa53c10tdahc objectid=56ddf1h9-
eb5d-42nf-bac7-7b019fd284g5 site=\"Company Events\" "
}
}
A site can also be set up by its id. However, ids in Sharepoint can include several elements separated by ‘,’. As that character is already used to separate different elements (different sites for the site parameter), the plugin will expect the separation inside the site Id to be specified with ‘;’. Example:
FileSet {
Name = fs-m365-site-company-events
Include {
Options { signature = MD5 }
Plugin = "m365: service=sharepoint tenant=57uia43-d107-17a2-a2g2-aa53c10tdahc objectid=56ddf1h9-
eb5d-42nf-bac7-7b019fd284g5 site=\"mysite.sharepoint.com;55068291-a2cb-144y-a401-03928474881141;acj948fb-9417-4bgf4-b31a-0erc6bd39ed6\" "
}
}
Backup sharepoint site including hidden lists:
FileSet {
Name = fs-m365-site-company-events-hidden
Include {
Options { signature = MD5 }
Plugin = "m365: service=sharepoint tenant=57uia43-d107-17a2-a2g2-aa53c10tdahc objectid=56ddf1h9-
eb5d-42nf-bac7-7b019fd284g5 site=\"Company Events\" system_include=yes sharepoint_hidden_lists=yes"
}
}
Backup sharepoint site and its sub-sites:
FileSet {
Name = fs-m365-site-company-events-sub
Include {
Options { signature = MD5 }
Plugin = "m365: service=sharepoint tenant=57uia43-d107-17a2-a2g2-aa53c10tdahc objectid=56ddf1h9-
eb5d-42nf-bac7-7b019fd284g5 site=\"Company Developments\" sharepoint_subsites=yes "
}
}
Backup a particular sharepoint sub-site:
FileSet {
Name = fs-m365-site-company-devs
Include {
Options { signature = MD5 }
Plugin = "m365: service=sharepoint tenant=57uia43-d107-17a2-a2g2-aa53c10tdahc objectid=56ddf1h9-
eb5d-42nf-bac7-7b019fd284g5 site=\"Company Developments/Doc Site\" "
}
}
Backup specific sharepoint site lists:
FileSet {
Name = fs-m365-site-company-events-docs-travel
Include {
Options { signature = MD5 }
Plugin = "m365: service=sharepoint tenant=57uia43-d107-17a2-a2g2-aa53c10tdahc objectid=56ddf1h9-
eb5d-42nf-bac7-7b019fd284g5 site=\"Company Events\" lists=\"documents, travel lists\" "
}
}