Performance

The performance of this plugin is highly dependent on many external factors:

  • Latency and bandwidth to AWS

  • Network infrastructure

  • FD Host hardware

  • FD Load

  • Ratio number of elements/size

  • And many more.

In summary, it is not possible to establish an exact reference about how much time a backup will need to complete.

As a reference and regarding the number of instances and their size:

  • Many small instance volumes to protect: More volumes per time period, but smaller speed (MB/s).

  • Big instance volumes to protect: Fewer volumes per time period, but greater speed (MB/s).

Consider also that any instance backup needs some time to make snapshot cleanup previously and after the backup, as well as the time Amazon EC2 needs to make the snapshots, list elements or calculate differences between two snapshots.

It is recommended to benchmark your own environment in base to your requirements and needs.

The automatic parallelization mechanism (using concurrent_threads=x) should work well for most scenarios, however, fine-tune is possible if we define one job per instance. We could control how many jobs to run in parallel, and then decrease the concurrent_threads value, if needed, in order to avoid throttling from EBS service.

There are many possible strategies to use this plugin, so it is recommended to study what suits your needs best before deploying the jobs in your entire environment, so you can get the best possible results:

  • You can have a job per instance

  • You can have multiple instances per job or even all your instances in the same job (not recommended)

  • You can split your workload through a schedule, or try to run all your jobs together

  • You can run jobs in parallel or take advantage of concurrent_threads and so, run less jobs in parallel

  • You can specifically select the instances you want to backup or backup them all

  • You can specifically select the volumes you want to backup or backup them all

  • You can include boot volumes in your backup or exclude them.

  • You can run your File Daemon on premise or in the cloud

  • You can use default internet connection to AWS or use a dedicated AWS connection (https://aws.amazon.com/directconnect/)

  • You can use default throttling limits, or ask AWS for more (https://docs.aws.amazon.com/AWSEC2/latest/APIReference/throttling.html#throttling-increase)

  • You can run your Storage Daemon and File Daemon in the same host, so you skip one network hop in the process (recommended).

See also

Go back to:

Go back to the Best Practices article.

Go back to the Amazon EC2 plugin main page.

Go back to the main Dedicated Backup Solutions page.