09-12-2013 11:59 PM
Hi, I have jobs failing with status 196, but the windos has just opened.....
Environment 7.5.0.4 on master, media and client servers.
I have a policy with 35 2003/2008 servers in it with Backup Selections set to ALL_LOCAL_DRIVES (no New Stream directive set) with a schedule from 18:00 - 06:00, 2 weeks retention and 13 hours frequency.
All jobs in the policy fail 1 minute after the backup window openen with the same error;
12-9-2013 18:00:02 - Info nbjm(pid=5996) starting backup job (jobid=1238974) for client CLIENTNAME, policy POLICYNAME, schedule Daily
12-9-2013 18:00:02 - Info nbjm(pid=5996) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=1238974, request id:{D56583B4-94E4-44C9-929B-46321B97AB63})
12-9-2013 18:00:02 - requesting resource RESOURCE
12-9-2013 18:00:02 - requesting resource MASTERSERVERNAME.NBU_CLIENT.MAXJOBS.CLIENTNAME12-9-2013 18:00:02 - requesting resource MASTERSERVERNAME.NBU_POLICY.MAXJOBS.POLICYNAME
12-9-2013 18:00:13 - awaiting resource RESOURCE - No drives are available
12-9-2013 18:00:59 - awaiting resource RESOURCE - Maximum job count has been reached for the storage unit
client backup was not attempted because backup window closed(196)
Not sure how this can happen, please advise...
Solved! Go to Solution.
09-17-2013 01:20 AM
Jobs are running properly after the reboot of all master and media servers, which also forced a re-read am i correct?
09-13-2013 01:51 AM
Please show us the output of the pollicy
bppllist <policyname> -U
bpstulist <stuname> -L
09-13-2013 02:26 AM
bppllist output;
Policy Name: POLICYNAME
Policy Type: MS-Windows
Active: yes
Effective date: 07/06/2006 14:48:41
Backup network drvs: no
Collect TIR info: no
Mult. Data Streams: yes
Client Encrypt: no
Checkpoint: yes
Interval: 15
Policy Priority: 0
Max Jobs/Policy: Unlimited
Disaster Recovery: 0
Collect BMR info: no
Residence: TapeLD
Volume Pool: NetBackup
Server Group: *ANY*
Keyword: (none specified)
Data Classification: -
Residence is Storage Lifecycle Policy: no
Application Discovery: no
Discovery Lifetime: 0 seconds
ASC Application and attributes: (none defined)
Granular Restore Info: no
Ignore Client Direct: no
Enable Metadata Indexing: no
Index server name: NULL
Use Accelerator: no
HW/OS/Client: Windows-x86 Windows2003 hostname
Windows-x86 Windows2003 hostname
Windows-x86 Windows2003 hostname (all hosts listed here but deleted by me to anonimize)
Include: ALL_LOCAL_DRIVES
Schedule: Yearly
Type: Full Backup
Maximum MPX: 8
Synthetic: 0
Checksum Change Detection: 0
PFI Recovery: 0
Retention Level: 10 (infinity)
Number Copies: 1
Fail on Error: 0
Residence: (specific storage unit not required)
Volume Pool: (same as policy volume pool)
Server Group: (same as specified for policy)
Calendar sched: Enabled
SPECIFIC DATE 0 - 01/05/2008
SPECIFIC DATE 1 - 01/12/2008
SPECIFIC DATE 2 - 01/03/2009
SPECIFIC DATE 3 - 01/01/2010
SPECIFIC DATE 4 - 01/01/2011
SPECIFIC DATE 5 - 01/07/2012
SPECIFIC DATE 6 - 01/05/2013
SPECIFIC DATE 7 - 01/12/2013
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
zondag 01:00:00 --> maandag 01:00:00
maandag 01:00:00 --> dinsdag 01:00:00
dinsdag 01:00:00 --> woensdag 01:00:00
woensdag 01:00:00 --> donderdag 01:00:00
donderdag 01:00:00 --> vrijdag 01:00:00
vrijdag 01:00:00 --> zaterdag 01:00:00
zaterdag 01:00:00 --> zondag 01:00:00
Schedule: Monthly
Type: Full Backup
Maximum MPX: 8
Synthetic: 0
Checksum Change Detection: 0
PFI Recovery: 0
Retention Level: 8 (1 year)
Number Copies: 1
Fail on Error: 0
Residence: (specific storage unit not required)
Volume Pool: (same as policy volume pool)
Server Group: (same as specified for policy)
Calendar sched: Enabled
SPECIFIC DATE 0 - 11/08/2008
Saturday, Week 1
EXCLUDE DATE 0 - 01/05/2013
EXCLUDE DATE 1 - 01/06/2013
EXCLUDE DATE 2 - 01/07/2013
EXCLUDE DATE 3 - 01/12/2013
EXCLUDE DATE 4 - 01/13/2013
EXCLUDE DATE 5 - 01/14/2013
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
zaterdag 17:00:00 --> maandag 05:00:00
Schedule: Weekly
Type: Full Backup
Maximum MPX: 8
Synthetic: 0
Checksum Change Detection: 0
PFI Recovery: 0
Retention Level: 4 (2 months)
Number Copies: 1
Fail on Error: 0
Residence: (specific storage unit not required)
Volume Pool: (same as policy volume pool)
Server Group: (same as specified for policy)
Calendar sched: Enabled
Saturday, Week 2
Saturday, Week 3
Saturday, Week 4
Saturday, Week 5
EXCLUDE DATE 0 - 01/05/2013
EXCLUDE DATE 1 - 01/06/2013
EXCLUDE DATE 2 - 01/07/2013
EXCLUDE DATE 3 - 01/12/2013
EXCLUDE DATE 4 - 01/13/2013
EXCLUDE DATE 5 - 01/14/2013
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
zaterdag 17:00:00 --> maandag 06:00:00
Schedule: Daily
Type: Differential Incremental Backup
Frequency: every 13 hours
Maximum MPX: 8
Synthetic: 0
Checksum Change Detection: 0
PFI Recovery: 0
Retention Level: 1 (2 weeks)
Number Copies: 1
Fail on Error: 0
Residence: (specific storage unit not required)
Volume Pool: (same as policy volume pool)
Server Group: (same as specified for policy)
EXCLUDE DATE 0 - 12/31/2009
EXCLUDE DATE 1 - 01/01/2010
EXCLUDE DATE 2 - 01/02/2010
EXCLUDE DATE 3 - 01/03/2010
EXCLUDE DATE 4 - 01/04/2010
EXCLUDE DATE 5 - 01/05/2010
EXCLUDE DATE 6 - 01/06/2010
EXCLUDE DATE 7 - 01/07/2010
EXCLUDE DATE 8 - 01/12/2013
EXCLUDE DATE 9 - 01/13/2013
EXCLUDE DATE 10 - 01/14/2013
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
maandag 18:00:00 --> dinsdag 06:00:00
dinsdag 18:00:00 --> woensdag 06:00:00
woensdag 18:00:00 --> donderdag 06:00:00
donderdag 18:00:00 --> vrijdag 06:00:00
bpstulist -group GROUPNAME output;
GROUPNAME 4 MEDIESERVERNAME1-hcart-robot-tld-0 MEDIASERVER2-hcart-robot-tld-0
09-13-2013 02:59 AM
I have seen this where you get overlap in the window and frequency
Your window opens at 1800 and closes at 0600 with a frequency of 13 hours
So 0600 to 1800 is only 12 hours so gives the potential over overlap and what is known as policy creep
What you need to know is how long it takes beofre all jobs are usuallu running and adjust things to suit
So try and get something like 1800 to 0400 window with a frequency of 11 hours so that you can never get any overlapping policy creep
I know it shouldnt really do this but it can
What does concern me is that you also have a second daily schedule of 0100 to 0100 - that may well be clashing with the 1800 to 0600 one as well - you really should only have one daily schedule
Other than that i have seen it when schedules have become corrupt and need re-creating.
Hope this helps
09-13-2013 03:00 AM
First i am seeing some configuration issue.. at this state i am not sure 196 is because of this, but worth fixing this before movng forward.
as per your Daily schedule you are tyring to take Incr backup on every 13 hours.but the backup window that you open is not allowing to do that..
window is Monday 18:00:00 - Tuseday 06:00:00
lets say you have backup compleated on 19:00:00, as per the freqeucy schedule of 13 hours next job needs to trigger by 08:00:00, but there is no Open window at 08:00;00 to trigger the backups.
so please chage the backup window to honour the 13 hours frequency first..
may be something like 18:00:00 to 20:00:00 & 06:00:00 to 10:00:00 for eachday.. something like that...
09-13-2013 03:03 AM
Oops.. Mark is faster....
09-13-2013 04:15 AM
Guys, I beg to differ - IMHO, the frequency of every 13 hours is perfectly fine to prevent Frequency schedule creeping with a 12-hour backup window. (We've been recommending this for years - see NetBackup Frequency Based Scheduling )
I cannot see any explanation for the status 196 within 1 minute of backup starting.
Has this (and other similar policies) been working file previouly?
There is a possibility that some sort of corruption may have crept in.
It may be worth your while to recreate this policy (not copy) before the next Full backup is due (tomorrow).
09-13-2013 04:32 AM
OK - ignore my message about two daily ones - re-read the output and can see there is only one
I do always go for a frequency one hour more than the backup window but have seen recently a customer who had a window of 12 hours or more and this setting caused the jobs to fail with a 196
Reducing the window and the frequency resolved this
Maybe it is a bug but they are on 7.1.0.4 and you are on 7.5 but worth trying it anyway?
Remember to run nbpemreq -updatepolicies once you make any changes
As i said earlier and Marianne confirms it can also be schedule or policy corruption so if the window change does not work try re-cerating the schedule or policy - but remember to do this just before a full backup run to avoid your diff jobs running as fulls and run the nbpemreq command once you have done it.
09-13-2013 04:40 AM
The policy has been working fine for a longer period, so i will not change it for now and refresh my complete environment by rebooting all the servers to begin with.
09-13-2013 04:45 AM
If you do not want to change it at least run nbpemreq -updatepolicies just to force through a re-read
09-17-2013 01:20 AM
Jobs are running properly after the reboot of all master and media servers, which also forced a re-read am i correct?
09-17-2013 02:05 AM
A reboot would cause the timers to be re-set for the backups which is pretty much the same as a re-read of the policies
The other advantage of a reboot is that it clears down any orphaned processes that could have contributed to the error (hung bpbrm processes on the Media Servers etc.)
Glad you are now sorted
09-17-2013 02:09 AM
Thnx for the help, this was the first time i used the forum to solve an issue in stead of creating a call with Symantec.
What shall i mark for solution?
09-17-2013 02:13 AM
What we usually ask is that if peoples advice led you to sort out an issue that you choose the comment that helped you the most and mark that one
Sometimes that may be your own but it has to be your choice as to which you feel helped the most
Users do get points for their comments and solutions so it is nice to award those points to a user that helped out
Your choice