Note

You can download this article as a PDF

NetApp HFC (Incremental Accelerator)

Overview
Scope
Presentation
Using The Incremental Accelerator for NetApp
Installation
Configuration
Estimate Information
Accurate Mode
Backup Information
Restore Scenarios
Limitations

Overview 

This user’s guide presents various techniques and strategies to backup NetApp NAS having a huge number of files with Bacula Enterprise.

Scope 

This paper will present solutions for Bacula Enterprise 6.0.6 and later, which are not applicable to prior versions.

Presentation 

The plugin Incremental Accelerator for NetApp is designed to simplify and the optimize backup and restore performance of your NetApp NAS hosting a huge number of files.

When using the plugin, for Incremental backup Bacula Enterprise will query the NetApp for a previous backup snapshot then quickly determine a list of all files modified since the last backup instead of having to walk through the entire filesystem. Once Bacula has the backup list, it will use a standard network share (such as NFS or CIFS) to access files.

In order to compute the file list modified since the last backup, Bacula must store a few Snapshots on the NetApp device. The Incremental Accelerator Plugin will manage the snapshot list to minimize resource usage.

This plugin is available for Red Hat Enterprise Linux 5 and 6 (32 / 64 bit) and SUSE Linux Enterprise Server 11 and 12 (64 bit) and supports NetApp 7-mode 7.3.6, 8.0, 8.1, 8.2 and 8.3, as well as all versions in c-mode.

Most recent product tests have been performed with SUSE Linux Enterprise Server 12 SP3 and NetApp Release 9.1.

Using The Incremental Accelerator for NetApp 

Installation 

The Incremental Accelerator for NetApp Plugin is available as a Bacula Enterprise package for all supported platforms. The package is composed of:

/opt/bacula/plugins/netapp-hfc-fd.so Bacula Enterprise File Daemon plugin
/opt/bacula/bin/snapmgr Bacula Enterprise NetApp Snapshot Manager

You must install this plugin on a Client machine that has network access to the NetApp filer. Bacula Systems advises you to use the Client that resides on your Storage Daemon so that the File daemon to Storage daemon data transfers are made internally rather than across the network.

Download the NetApp SDK installation files from the NetApp Support site for All Platforms: http://mysupport.netapp.com/NOW/cgi-bin/software/?product=NetApp+Manageability+SDK&platform=All+Platforms. Note: A form is required to download the software. Please complete and state that you want to use the SDK for Data ONTAP with Perl, and in the solution field enter “NetApp SDK”

% unzip netapp-manageability-sdk-5.4.zip netapp-manageability-sdk-5.4/lib/perl/NetApp/*
Archive: netapp-manageability-sdk-5.4.zip
creating: netapp-manageability-sdk-5.4/lib/perl/NetApp/
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/DfmErrno.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/NaElement.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/NaErrno.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/NaServer.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/ONTAPILogParser.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/ONTAPITestContainer.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/SdkEnv.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/Test.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/OCUMAPI.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/OCUMClassicAPI.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/Ontap7ModeAPI.pm
inflating: netapp-manageability-sdk-5.4/lib/perl/NetApp/OntapClusterAPI.pm

% mv netapp-manageability-sdk-5.4/lib/perl/NetApp /opt/bacula/bin/

In order to save Windows extended file attributes such as NTFS ACLs, you will need to install the Incremental Accelerator for NetApp Plugin on a Windows server, this functionality will be available in a future version.

Configuration 

As with all Bacula plugins, you must to specify the Plugin Directory directive in the FileDaemon resource of the bacula-fd.conf file.

FileDaemon {
  Name = test-fd
  ...
  Plugin Directory = /opt/bacula/plugins
}

Your Backup Job should be defined as:

Job {
 Name = "NetApp-HFC"
 Client = storage-fd
 FileSet = FS_netapp
 ...
}

FileSet {
 Name = FS_netapp
 Include {
   Options {
     Signature = MD5
   }
   Plugin = "netapp-hfc: host=nas1 password=pw exclude=vol0"
 }
}

FileSet {
 Name = FS_netapp_cmode
 Include {
   Options {
     Signature = MD5
   }
   Plugin = "netapp-hfc: host=nas1 password=pw exclude=vol0 vserver=vserver1 https"
 }
}

In this example, all NetApp volumes have to be mounted in the /nas1 directory (the mount_base plugin option permits to configure the base mount point). The second FileSet FS_netapp_cmode is valid for a “Cluster Mode”.

% ls /nas1
vol1   vol2   vol3   vol4

The Incremental Accelerator for NetApp plugin accepts the parameters listed in table ‘Incremental Accelerator Plugin Options’.

Incremental Accelerator Plugin Options
Option	Required*	Default	Info	Example
host	Yes		NAS Hostname	`host=19 2.168.0.1`
user	Yes	root	Username	`user=root`
password	Yes		Password	`password =password`
volume	No		Volume to backup	`vo lume=vol0`
include	No	*	Volumes to backup	`incl ude=vol1*`
exclude	No		Volumes to exclude from backup	`exc lude=vol0`
mount_base	No	/host	Where to find the volume mounted using NFS / CIFS	`mount_ base=/mnt`
key	No	Job Name	Advanced snapshot pruning control	`key=MyKey`
snap_dir	No	.snapshot	Name of the directory where snapshots are visible	`snap_ dir=.snap`
vserver	No		Vserver name. This option is required for C-Mode support	`vserver =vserver1`
https	No	No	Use HTTPS with NetApp	`https`

Note: Because the NetApp Incremental Accelerator Plugin uses a dynamic strip FileSet option, the FileSet Include section should include ONLY the NetApp plugin; mixing other plugins or normal files with the HFC Plugin is not supported.

Note also that the NetApp filer needs to be configured to allow management access on the data access network interface, and a user with http access and the “vsadmin” role assigned should be used for this plugin. In c-mode, this may require creation of a user account on the SVM being backed up.

Snapshot Pruning

The NetApp SnapDiff API is based on Snapshots to generate the list of the changes. Once a snapshot is no longer required, the Bacula NetApp HFC Plugin will prune the snapshot.

The snapshot registry that is maintained by the plugin is stored in the Bacula working directory. The registry uses the Job name as a key to compute the snapshot chain. If you are doing multiple backups with the same Job or you have the same volume name in different context, we advise to set the extra key parameter in the Plugin command line to uniquely identify your jobs.

FileSet {
 Name = FS_netapp_cmode
 Include {
   Options {
     Signature = MD5
   }
   Plugin = "netapp-hfc: host=n1 password=pw volume=vol0 vserver=v1 key=prod https"
 }
}

External Password Storage

To avoid storing passwords in the Bacula Director configuration file, you can create a special configuration file located where the Plugin is installed and store the password in it.

% cat /opt/bacula/etc/snapmgr.conf
nas1:root = mypassword
netapp2:admin = myotherpassword

The first line will define the password for the root account on the “nas1” nas box. The Bacula NetApp HFC Plugin will use this entry with the following plugin command:

plugin = "netapp-hfc: host=nas1 user=root exclude=vol0"

You can have multiple lines in the snapmgr.conf.

NFS Configuration

The NetApp HFC Plugin will detect the list of all volumes from the NetApp device, then the Plugin will assume that each volume is mounted and available under mount_base directory. For example, if you have the following volumes defined on your NetApp device (Fig NetApp Volume List).

You will need to mount each of them under your mount_base directory. For example, in /mnt:

% ls -l /mnt
drwxr-xr-x 2 root root 4096 May 23 14:52 vol0
drwxr-xr-x 2 root root 4096 May 23 14:52 vol1
drwxr-xr-x 2 root root 4096 May 23 14:52 vol2
drwxr-xr-x 2 root root 4096 May 22 10:07 vol3

We advise you to create a directory with your NetApp host name and mount volumes under it. For example, instead of using /mnt as the previous example was showing, using /netapp-box will simplify restore operation and file identification during restore. (Fig NetApp and Volume Structure).

% ls -l /netapp-box
drwxr-xr-x 2 root root 4096 May 23 14:52 vol0
drwxr-xr-x 2 root root 4096 May 23 14:52 vol1
...

The following article will give you information about how to find optimal settings for NFS mounts http://nfs.sourceforge.net/nfs-howto/ar01s05.html

For example, you may want to mount NFS shares with the following options:

% mount -t nfs -o soft,intr nas1:/vol/vol3 /nas1/vol3

Snapshots should be visible in the .snapshots directory.

Snapshot Management

The Incremental Accelerator for NetApp Plugin uses Snapshots to determine which files changed since the last backup. The Plugin will keep a number of Snapshots on your NetApp filer. Depending on your strategy, the Plugin will keep 3 to 4 Snapshots per volume.

Volume Management

To list all volumes available on the NetApp system, the following command might be used:

 # /opt/bacula/bin/snapmgr -V -f host -u user -p pass

or

 # /opt/bacula/bin/snapmgr -V -f host -u user -p pass -e vserver1

or

 # /opt/bacula/bin/snapmgr -s -V -f host.with.https -u user -p pass -e vserver1

Where, host is the NetApp hostname or address, user is the username used to connect the NetApp and pass is the required password.

With NetApp c-mode, the vserver option is required.

In version prior to 12.4.1, snapmgr error messages can be found in working/snapmgr.err or /tmp/snapmgr.err. Starting with 12.4.1, error messages are automatically included in the Bacula Job log.

Estimate Information 

The information returned by the estimate command will give you the list of all files that have changed. However, be careful when using the estimate command with the level Full (default). It will result in a heavy load of your NetApp filer if it contains many files.

Accurate Mode 

By default, the Incremental Accelerator for NetApp Plugin backup result will be similar to what a traditional backup traversing the whole filesystem would do. Some operations (such as deleting or moving) on a directory (see figure Directory Structure When Restoring (Without Accurate Support)) will lead to non accurate directory structure when restoring.

Directory Structure When Restoring (Without Accurate Support)

This issue can be solved using Bacula’s Accurate feature in your Job definition. However, be aware that the Accurate feature will consume additional memory on your File Daemon machine and CPU cycles on your catalog database system, and when dealing with million of files, you need to properly configure your File Daemon hardware. Using Bacula Accurate mode is transparent for the NetApp system.

Bacula Enterprise allows you to turn Accurate mode on or off in the Job Schedule. Doing that, you are able to, for example, turn the Accurate mode on only for weekly differential jobs. This can provide a good compromise by having a completely accurate weekly backup without the additional overhead each day.

Backup Information 

During a backup, some essential information is stored in the netapp-hfc.dat file located in the working directory that is required for sequences of Jobs, so please take care not to remove the netapp-hfc.dat files.

Restore Scenarios 

During restoration, files will be presented in a simple virtual filesystem structure.

/<netapp host name>/<volume>/<files and directories>/...

For example (Fig NetApp and Volume Structure), the file /etc/passwd in the volume vol0 of the NetApp device netapp-box will be stored as:

/netapp-box/vol0/etc/passwd

Limitations 

NetApp Data ONTAP version 7.3.5 and lower did not properly support Unicode characters in filenames outside of 7-bit ASCII. All NetApp versions after 7.3.5 support the full range of Unicode characters.
The Incremental Accelerator plugin only works with NetApp appliances.

The restart command has limitations with plugins, as it initiates the Job from scratch rather than continuing it. Bacula determines whether a Job is restarted or continued, but using the restart command will result in a new Job.