AIX client ignoring exclude_list

[Wile_E__Coyote]

My Netbackup master is 7.0.1.

Client is AIX 6.1 running Netbackup 6.5.5 client software. Unfortunately client upgrade requires approval from Santa Claus and the Easter Bunny due to very strict change controls, so an upgrade is not likely any time soon.

It's ignoring /usr/openv/netbackup/exclude_list completely, causing annoying "71" error codes to come up on my pager at 2:00AM.

The directory to ignore is listed exactly, perfectly, in the exclude_list. We've recreated it, tried six ways from Sunday to find a hidden character. No luck.

I can find references to where WINDOWS ignores the exclude_list, but not AIX.

[Stumpr2]

Is there another exclude_list.$POLICY.$SCHEDULE existing in /usr/openv/netbackup?

[Stumpr2]

If not then could you create one for the policy/schedule having stat 71?

[watsons]

What about the permission of the exclude_list file?

If everything checks out fine, turn on VERBOSE=5 on client bpbkar log, run a test backup and check the logs to see whether it reads the exclude_list file, and if not, maybe there is a clue.

[Marianne · Accepted solution]

Please provide contents of exclude_list as well as Backup Selection in policy.

Bear in mind that backup streams are set up BEFORE exclude_list is consulted.

If Backup Selection has for example /data in Backup Selection, but no such file, folder or filesystem exists on the client, it will not help to have /data in exclude_list.
A workaround is to disable Multiple Data Streams, but this will affect backup performance as only one backup stream will be generated. 
'ALL_LOCAL_DRIVES' may be a good choice combined with exclude_list?

[Andy_Welburn]

Bear in mind that backup streams are set up BEFORE exclude_list is consulted.
...
A workaround is to disable Multiple Data Streams

Quite probably this.

If you have multiple data streams enabled in the policy these particular jobs (streams) are created prior to any exclude lists being consulted. Now if the exclude list excludes *any* of these streams in toto then you'll end up with 71's as a stream is created & then everything within it is excluded.

[Wile_E__Coyote]

Yes, I have multiple data streams enabled. This is a very large ~4.5TB directory with millions of files that absolutely must be backed up to tape fully once a week. Customer doesn't care that it's impractical. It's what he wants and we have to oblige. With streams it takes 52 hours. Without streams it takes ~10 days. The problem is, the stream that's erroring out with a 71 is a single file that is there when the backup starts, but is deleted by an automated process during the backup. /cdr/data/RESCAN is the name of the offending file. "/cdr/data/RESCAN" is what's listed in exclude_list. "/cdr/data/*" is the backup selection. The customer will not provide a list of specific backup targets in /cdr/data, so I can't explicity list each subdirectory in the backup policy. Does anyone have an idea how I can stop the error code 71?

[Marianne]

You can try and 'play' with wildcards. 

Start by looking at bplist output from successful backup, e.g.:

bplist -t 0 -C <client-name> -s <start-date> -e <end-date> -R 3 /cdr/data/

Depending on output, you could break up the backup selection into something like this:

NEW_STREAM
/cdr/data/a*
/cdr/data/b*
NEW_STREAM
/cdr/data/c*
/cdr/data/d*
NEW_STREAM
/cdr/data/e*
/cdr/data/f*

etc...

[Stumpr2]

Telephone Keypad Strategy
I posted this in this forum long ago. here is the encore :)

I have a client that I setup 10 policies to backup the files. I used the telephone keypad as a model

Policy_0
D:\dept\0*

Policy_1
D:\dept\1*

Policy_2
NEW_STREAM
D:\dept\2*
NEW_STREAM
D:\dept\A*
NEW_STREAM
D:\dept\B*
NEW_STREAM
D:\dept\C*

Policy_3
NEW_STREAM
D:\dept\2*
NEW_STREAM
D:\dept\D*
NEW_STREAM
D:\dept\E*
NEW_STREAM
D:\dept\F*

and so on.....

I needed to break them up into separate policies so that I could split up the full backups throughout the week. Otherwise the 2.5TB of data would take too long to do it at one time

[Mark_Solutions]

I have a customer with similar systems and we do a lot of streams for their servers based on a* to z* and 0* to 9*, though we do limit the number of streams that run at any one time for the client (trial and error this one to see how much punishment it can take)

Another option that may really help you is to use FlashBackup

You need an Enterprise Client but generally it will give you the best possible performance as it doesn't care how many files there are, it just backs up the whole volume at block level (bear in mind that is the whole volume including empty file system space)

If you have a license or can afford to try it it may be worth while - your Symantec Partner may even let you try it with an Eval key before purchasing it

Hope this helps

[Wile_E__Coyote]

Hey guys sorry for the hiatus.

Still have the problem, but I haven't made any changes either so that's no surprise.

The wildcard games are interesting, but not practical because I have no idea what I'm backing up, and doesn't account for new directories being added and old ones deleted. Keeping it up to date would require constantly pestering the customer to make sure they kept us informed of changes. Otherwise, they won't, and things won't get backed up.

Flashbackup is about the most practical option but I've heard that it's not an option for AIX. We're licensed for it and use it heavily on large Windows backups.

Even if Flashbackup is an option in AIX, we would still have to get past the whole process of upgrading the client and installing the license, which again requires blessings from Santa Claus, the Easter Bunny, the Tooth Fairy and Ringo Starr due to Sarbanes-Oxley.

[Mark_Solutions]

I suppose it would need John Lennon or Ronald Regan to authorise converting it to virtual and using the vSphere API to back it up and a single lump?

[Wile_E__Coyote]

Nope, just Jimmy Hoffa LOL ;)

[Stumpr2]

That is what is nice about the telephone keypad wildcard method as it covers each letter of the alphabet and numbers 0-9. Whatever new folder that either starts with any letter or number will automatically be backed up. Also by using ten policies the size of the images will be managable.

[Wile_E__Coyote]

Welp, that didn't work.

I tried the whole playing with wildcards trick, and there are no other streams starting with R so it still 71's.

[Stumpr2]

can you

# touch /cdr/data/R_do_not_delete