Backup-CLI
An alternative way to perform parallel backups with the Big FS Plugin is to directly pipe the content
of a find command into the fs-backint executable.
Below is an example of such a command:
find /etc | /opt/bacula/bin/fs-backint -f backup -N 5 -F 20
The parameters for the fs-backint executable are the following
-N <nb_workers>This parameter indicates the maximum number of workers permitted to operate concurrently. The specified value must exceed 1; otherwise, the default value will be 1.-F <nb_files>This parameter denotes the maximum number of files that a worker can handle simultaneously. If a value is provided, it must be greater than 20; otherwise, the default value is 20.-S <since>This parameter is used for incremental backups. It will back up all files that have been created or modified since the specified date. The parameter can be either a timestamp or a file whosemtimewill be used instead.-p <path>This parameter specifies the path to the configuration file. If this argument is not provided, the default path/opt/bacula/big-fs/fs-backint.cfgwill be used.-f <operation>This parameter indicates the operation to be performed by backint. Currently, this plugin only supports backup.-P <max_files>This parameter specifies the maximum number of files processed by a worker before it is terminated.-T <max_time>This parameter indicates the maximum processing time before a worker is terminated.
Below is an example of a full execution:
find /home/user/tmp/test | bin/fs-backint -p /tmp/backint.cfg -f backup
Profile: /tmp/backint.cfg
Input File: *None*
Output File: *None*
Client node: host-fd
#SAVED big-fs-7 /home/user/tmp/test
#SAVED big-fs-7 /home/user/tmp/test/E
#SAVED big-fs-7 /home/user/tmp/test/E/aEa6998aaaE
#SAVED big-fs-7 /home/user/tmp/test/E/aEa1980aaaE
#SAVED big-fs-7 /home/user/tmp/test/E/aEa9390aaaE
#SAVED big-fs-7 /home/user/tmp/test/E/aEa1096aaaE
#SAVED big-fs-7 /home/user/tmp/test/E/aEa1590aaaE
#SAVED big-fs-7 /home/user/tmp/test/E/aEa8116aaaE
#SAVED big-fs-7 /home/user/tmp/test/E/aEa2578aaaE
#SAVED big-fs-7 /home/user/tmp/test/E/aEa4736aaaE
#SAVED big-fs-7 /home/user/tmp/test/E/aEa3540aaaE
#SAVED big-fs-7 /home/user/tmp/test/E/aEa7882aaaE
...
#SAVED big-fs-7 /home/user/tmp/test/W/aWa4728aaaW
#SAVED big-fs-8 /home/user/tmp/test/W/aWa4286aaaW
#SAVED big-fs-7 /home/user/tmp/test/W/aWa568aaaW
#SAVED big-fs-8 /home/user/tmp/test/W/aWa360aaaW
#SAVED big-fs-7 /home/user/tmp/test/W/aWa3480aaaW
with the relevant job log:
22-dec. 13:58 host-dir JobId 7: Start Backup JobId 7, Job=BigFsTest.2023-12-22_13.58.20_03
22-dec. 13:58 host-dir JobId 7: Connected to Storage "File" at host:8103 with TLS
22-dec. 13:58 host-dir JobId 7: Using Device "FileStorage" to write.
22-dec. 13:58 host-dir JobId 7: Connected to Client "host-fd" at host:8102 with TLS
22-dec. 13:58 host-fd JobId 7: Connected to Storage at host:8103 with TLS
22-dec. 13:58 host-sd JobId 7: Volume "TestVolume001" previously written, moving to end of data.
22-dec. 13:58 host-sd JobId 7: Ready to append to end of Volume "TestVolume001" size=4,226,973
22-dec. 13:58 host-fd JobId 7: Include section of fs-backint is empty
22-dec. 13:58 host-fd JobId 7: Connecting to backint process...
22-dec. 13:58 host-fd JobId 7: fs-backint process connected.
22-dec. 13:58 host-dir JobId 8: Start Backup JobId 8, Job=BigFsTest.2023-12-22_13.58.23_05
22-dec. 13:58 host-dir JobId 8: Connected to Storage "File" at host:8103 with TLS
22-dec. 13:58 host-dir JobId 8: Using Device "FileStorage" to write.
22-dec. 13:58 host-dir JobId 8: Connected to Client "host-fd" at host:8102 with TLS
22-dec. 13:58 host-fd JobId 8: Connected to Storage at host:8103 with TLS
22-dec. 13:58 host-fd JobId 8: Include section of fs-backint is empty
22-dec. 13:58 host-fd JobId 8: Connecting to backint process...
22-dec. 13:58 host-fd JobId 8: fs-backint process connected.
22-dec. 13:58 host-dir JobId 9: Start Backup JobId 9, Job=BigFsTest.2023-12-22_13.58.27_07
22-dec. 13:58 host-dir JobId 9: Connected to Storage "File" at host:8103 with TLS
22-dec. 13:58 host-dir JobId 9: Using Device "FileStorage" to write.
22-dec. 13:58 host-dir JobId 9: Connected to Client "host-fd" at host:8102 with TLS
22-dec. 13:58 host-fd JobId 9: Connected to Storage at host:8103 with TLS
22-dec. 13:58 host-fd JobId 9: Include section of fs-backint is empty
22-dec. 13:58 host-fd JobId 9: Connecting to backint process...
22-dec. 13:58 host-fd JobId 9: fs-backint process connected.
22-dec. 13:58 host-dir JobId 10: Start Backup JobId 10, Job=BigFsTest.2023-12-22_13.58.32_09
22-dec. 13:58 host-dir JobId 10: Connected to Storage "File" at host:8103 with TLS
22-dec. 13:58 host-dir JobId 10: Using Device "FileStorage" to write.
22-dec. 13:58 host-dir JobId 10: Connected to Client "host-fd" at host:8102 with TLS
22-dec. 13:58 host-fd JobId 10: Connected to Storage at host:8103 with TLS
22-dec. 13:58 host-fd JobId 10: Include section of fs-backint is empty
22-dec. 13:58 host-fd JobId 10: Connecting to backint process...
22-dec. 13:58 host-fd JobId 10: fs-backint process connected.
22-dec. 13:58 host-sd JobId 9: Elapsed time=00:00:12, Transfer rate=2.939 K Bytes/second
22-dec. 13:58 host-sd JobId 7: Elapsed time=00:00:19, Transfer rate=114.7 K Bytes/second
22-dec. 13:58 host-sd JobId 10: Elapsed time=00:00:07, Transfer rate=2.870 K Bytes/second
22-dec. 13:58 host-sd JobId 9: Sending spooled attrs to the Director. Despooling 55,405 bytes ...
22-dec. 13:58 host-sd JobId 8: Elapsed time=00:00:16, Transfer rate=79.06 K Bytes/second
22-dec. 13:58 host-sd JobId 7: Sending spooled attrs to the Director. Despooling 3,394,732 bytes ...
22-dec. 13:58 host-sd JobId 10: Sending spooled attrs to the Director. Despooling 31,849 bytes ...
22-dec. 13:58 host-sd JobId 8: Sending spooled attrs to the Director. Despooling 1,970,026 bytes ...
22-dec. 13:58 host-dir JobId 9: Bacula host-dir 18.0.0 (20Nov23):
Build OS: x86_64-pc-linux-gnu archlinux
JobId: 9
Job: BigFsTest.2023-12-22_13.58.27_07
Backup Level: Incremental, since=1970-01-01 01:00:01
Client: "host-fd" 18.0.0 (20Nov23) x86_64-pc-linux-gnu,archlinux,
FileSet: "big-fs" 2023-12-22 13:48:22
Pool: "Default" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "File" (From Job resource)
Scheduled time: 22-dec-2023 13:58:27
Start time: 22-dec-2023 13:58:30
End time: 22-dec-2023 13:58:42
Elapsed time: 12 secs
Priority: 10
FD Files Written: 205
SD Files Written: 205
FD Bytes Written: 1,898 (1.898 KB)
SD Bytes Written: 35,271 (35.27 KB)
Rate: 0.2 KB/s
Software Compression: None
Comm Line Compression: None
Snapshot/VSS: no
Encryption: no
Accurate: no
Volume name(s): TestVolume001
Volume Session Id: 3
Volume Session Time: 1703249766
Last Volume Bytes: 8,453,022 (8.453 MB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
[...]
22-dec. 13:58 host-dir JobId 7: Bacula host-dir 18.0.0 (20Nov23):
Build OS: x86_64-pc-linux-gnu archlinux
JobId: 7
Job: BigFsTest.2023-12-22_13.58.20_03
Backup Level: Full
Client: "host-fd" 18.0.0 (20Nov23) x86_64-pc-linux-gnu,archlinux,
FileSet: "big-fs" 2023-12-22 13:48:22
Pool: "Default" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "File" (From Job resource)
Scheduled time: 22-dec-2023 13:58:20
Start time: 22-dec-2023 13:58:23
End time: 22-dec-2023 13:58:42
Elapsed time: 19 secs
Priority: 10
FD Files Written: 12,528
SD Files Written: 12,528
FD Bytes Written: 133,671 (133.6 KB)
SD Bytes Written: 2,179,861 (2.179 MB)
Rate: 7.0 KB/s
Software Compression: None
Comm Line Compression: None
Snapshot/VSS: no
Encryption: no
Accurate: no
Volume name(s): TestVolume001
Volume Session Id: 1
Volume Session Time: 1703249766
Last Volume Bytes: 8,453,022 (8.453 MB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
22-dec. 13:58 host-dir JobId 8: Bacula host-dir 18.0.0 (20Nov23):
Build OS: x86_64-pc-linux-gnu archlinux
JobId: 8
Job: BigFsTest.2023-12-22_13.58.23_05
Backup Level: Incremental, since=1970-01-01 01:00:01
Client: "host-fd" 18.0.0 (20Nov23) x86_64-pc-linux-gnu,archlinux,
FileSet: "big-fs" 2023-12-22 13:48:22
Pool: "Default" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "File" (From Job resource)
Scheduled time: 22-dec-2023 13:58:23
Start time: 22-dec-2023 13:58:26
End time: 22-dec-2023 13:58:42
Elapsed time: 16 secs
Priority: 10
FD Files Written: 7,280
SD Files Written: 7,280
FD Bytes Written: 78,960 (78.96 KB)
SD Bytes Written: 1,265,014 (1.265 MB)
Rate: 4.9 KB/s
Software Compression: None
Comm Line Compression: None
Snapshot/VSS: no
Encryption: no
Accurate: no
Volume name(s): TestVolume001
Volume Session Id: 2
Volume Session Time: 1703249766
Last Volume Bytes: 8,453,022 (8.453 MB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
[...]
22-dec. 13:58 host-dir JobId 10: Bacula host-dir 18.0.0 (20Nov23):
Build OS: x86_64-pc-linux-gnu archlinux
JobId: 10
Job: BigFsTest.2023-12-22_13.58.32_09
Backup Level: Incremental, since=1970-01-01 01:00:01
Client: "host-fd" 18.0.0 (20Nov23) x86_64-pc-linux-gnu,archlinux,
FileSet: "big-fs" 2023-12-22 13:48:22
Pool: "Default" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "File" (From Job resource)
Scheduled time: 22-dec-2023 13:58:32
Start time: 22-dec-2023 13:58:35
End time: 22-dec-2023 13:58:42
Elapsed time: 7 secs
Priority: 10
FD Files Written: 117
SD Files Written: 117
FD Bytes Written: 1,057 (1.057 KB)
SD Bytes Written: 20,091 (20.09 KB)
Rate: 0.2 KB/s
Software Compression: None
Comm Line Compression: None
Snapshot/VSS: no
Encryption: no
Accurate: no
Volume name(s): TestVolume001
Volume Session Id: 4
Volume Session Time: 1703249766
Last Volume Bytes: 8,453,022 (8.453 MB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
[...]
Go back to: Operations.