cancel
Showing results for 
Search instead for 
Did you mean: 

Exclude lists and Timing Out

The_Director
Level 5
Certified
I have most of my policies set to backup all local drives and then set exclude lists to exclude the /var directory and include list to include certain directories in /var. The problem is that netbackup times out constantly because its going through the entire /var directory looking for includes and excludes.

Is there a way that i can create an exclude list that will not search through the large mail directories in /var?
3 REPLIES 3

DavidParker
Level 6
Mark,
Have you increased the client timeout value yet?
Say, to something like 900 seconds?

The_Director
Level 5
Certified
yes, we increased it to 600 first then 900.

Right now we just created a separate policy for this client but i would rather not have to work about another policy.

Rakesh_Khandelw
Level 6
Increase your time out values for CLIENT_CONNECT_TIMEOUT and CLIENT_READ_TIMEOUT to say 3600 and run your backup.

Also, you may want to look at tech note describing why it goes through the entire directory sructure. I thought it will be helpful to understand the behaviour.

Details:
Overview:

It is normal behavior for the addition of an /usr/openv/netbackup/include_list file to cause the backup of empty directories for all entries in the /usr/openv/netbackup/exclude_list. While the directory structure will be backed up, the actual files will be properly excluded from the backup.

Troubleshooting:

Consider the following scenario
A /usr/openv/netbackup/exclude_list contains:
/development
/projects
With no include_list present, these two directories and all subdirectories and files will be excluded from the backup.

Then add a /usr/openv/netbackup/include_list that contains.
/projects/data1
The addition of an include_list will cause the entire directory structure for all directories in the exclude_list to be backed up. This is normal and only files in /projects/data1 will actually be included in the backup.

Master Server Log Files: n/a

Media Server Log Files: n/a

Client Log Files: n/a

Resolution:

This is normal behavior for NetBackup. If his behavior is undesired, a workaround is to manage the directories to be excluded solely from the /usr/openv/netbackup/exclude_list. This will mean adding entries for every subdirectory you want to exclude. This will ensure there is no include_list file to check for and the empty directory entries will not be added to the backup image.

NetBackup has no means to determine if an include_list entry will appear in an excluded filesystem. It traverses the entire excluded filesystem and compares the contents against the include_list looking for matches. The directory structure has to be added to the backup image in case a match is found. There is no post processing to remove these entries from the backup. As a result, the excluded directory names will appear in the backup image and when a restore is run.

The reason for this is if you have an include_list, each file that has been excluded may have to be re-included because it may be in the include_list, therefore the entry for the excluded directory is retained and backed up. There is no "post processing" of the file list after the include processing to delete directory entries which have no files to backup. It is more efficient to backup the directory entry than it would be to reprocess the complete list again, checking excludes vs includes. In a large environment with lots of files to keep a record of files included / excluded while processing could consume huge amounts of memory . The trade off of keeping the "entry" for excluded directories is much more efficient that the post processing would be. As the record is relatively small it really doesn't require that much media when weighed against the amount of processing it would take to create a final image list before starting the backup. Processing time being of "higher cost" in most instances than media. Also the thing to keep in mind is the processing is actually taking place on the client.