Cloud Plugin: Best Practices

Set the MaximumFileSize to a value smaller than or equal to the MaximumPartSize as the MaximumFileSize defines the granularity of the restore chunk. Having the MaximumFileSize directive defined this way will allow Bacula to only download the required part file(s) to restore specific files and/or directories in the case of partial restores, thus reducing download costs.
Have a reasonable value configured for MaximumPartSize. Smaller part sizes will reduce restore costs, but may require a small additional overhead to handle multiple parts. Also, some cloud providers limit the size of a single object to be uploaded. A part file is considered a single object and it will use a single upload operation. Thus you must set a value lower or equal than this limit for the MaximumPartSize, otherwise the upload of part files will fail. Please confirm with your cloud provider the maximum object size upload, if any.
Regularly run cloud upload and cloud truncate bconsole commands. They can be scheduled in an Admin Job. Even when the Cloud resource is configured to automatically upload volume part files and/or truncate them from the local cache, connection issues can prevent some part files from being successfully uploaded and a local cache truncation may not occur for part files which have not yet been uploaded. It is recommended to retry the volume part file upload and/or local cache truncation in the case of failures by manually running the cloud upload and cloud truncate bconsole commands. You can also use a Bacula Admin Job that will retry the part file upload automatically on a regular basis.

If the restore process takes a long time, it is recommended to temporarily deactivate the cloud upload (if TruncateCache = yes) and/or the cloud truncate commands to prevent essential parts needed for the restore from being truncated from the local cache.

Please find an example for such an Admin Job that can be used to periodically trigger the cloud upload and the cloud truncate commands:
```
Job {
  Name = CloudUpload-adminjob
  Type = Admin
  Client = bacula-fd  # any client can be used
  Schedule = DailyCloudUpload
  RunScript {
    RunsOnClient = No
    RunsWhen = Always
    Console = "cloud upload storage=<cloud_storage_name> allpools"
    Console = "cloud truncate storage=<cloud_storage_name> allpools"
  }
  Storage = <cloud_storage_name>
  Messages = Default
  Pool = Fake-pool
  Fileset = Fake-fileset
}
```
After a restore, the downloaded part files are not truncated from the local cache even if Truncate Cache is configured in the Cloud resource (to truncate the local cache AfterUpload or AtEndOfJob). If you have your environment configured to truncate the local cache after the part is successfully uploaded or at the end of a job, we recommend to either manually truncate the local cache after the restore successfully finishes or to have it truncated by an Admin Job that periodically truncates the local cache (the Admin Job as recommended in the previous point).
It is considered a best practice to set MaximumConcurentJobs = 1 in all of the Cloud Device resources (DeviceType = Cloud) that are defined on the SD. This will guarantee that only one job is writing to the device at one time, so data belonging to different jobs will not be interleaved in the part files. When dealing with cloud backups, this helps to minimize the overall download costs by downloading only the required part files from specific backup jobs during a restore. Of course, you need to consider defining a larger number of Cloud Devices, appropriate to the maximum number of jobs you are expecting to run concurrently.
Retention locked objects (terminology varies among providers) can be used, but to avoid failures when Volumes are recycled, it is strongly recommended to have Bacula’s retention times set in a way that they expire only after the object is no longer in retention. This applies particularly to VolumeRetention, but JobRetention can indirectly affect Volume recycling too. If Bacula attempts to recycle a Volume that is still in locked state, the corresponding job will fail, and the offending Volume will be marked with status Error.
As you may already be aware, storing data in the cloud will incur additional costs. Data transfer must also be taken into account. Although uploading data is generally free or incurs minimal costs, downloading is usually not complimentary, resulting in charges that can make cloud restores quite costly. Additionally, it is important to note that when data is written to a volume using Bacula, some data will be sent in the reverse direction for purposes such as data verification and other critical functions. You may have the opportunity to lower your storage expenses by activating one of the data compression methods offered by Bacula.