11-26-2013 07:32 AM
Good morning all,
i've just realized that some tasks schedule, within policies, got missed randomly.
First cases were with CATALOG_BACKUP, some days no catalog without any errors or issues.
Today i've found a SAP FULL DB that never get started within the BU window, again without any real issues around: master up, media up, agent up, and other tasks ongoing normaly.
Where do i start to troubleshoot something like this?
Can i set an alert in OpsCenter to get some alerts, for example?
Thank you.
Kind Regards,
Michele
Solved! Go to Solution.
11-26-2013 09:12 AM
alternatly you can try changing the scheule from Freqeucy to Calander based and select all days in recurring days without changing the backup start times.
11-26-2013 07:46 AM
Could we see the output for the catalog and SAP policy please - i am guessing it is related to your window and frequency causing it
bppllist policyname -U
11-26-2013 07:47 AM
I would start from Verifying the policy configuration.
please post the output of below command
bppllist <policyname> -L
11-26-2013 07:55 AM
Hi Gents,
thank you for the support.
Here it is some more info:
------------------------------------------------------------
Policy Name: MNG_AMS20_SAP_DBPW
Policy Type: SAP
Active: yes
Effective date: 08/02/2011 17:47:45
Mult. Data Streams: no
Client Encrypt: no
Checkpoint: no
Policy Priority: 100
Max Jobs/Policy: Unlimited
Disaster Recovery: 0
Collect BMR info: no
Residence: MnG
Volume Pool: DataStore
Server Group: *ANY*
Keyword: (none specified)
Data Classification: -
Residence is Storage Lifecycle Policy: no
Application Discovery: no
Discovery Lifetime: 0 seconds
ASC Application and attributes: (none defined)
Granular Restore Info: no
Ignore Client Direct: no
Enable Metadata Indexing: no
Index server name: NULL
Use Accelerator: no
HW/OS/Client: Windows-x64 Windows2003 ph300202.ams20.vzbi.caas
Windows-x64 Windows2008 ph967702.ams20.vzbi.caas
Include: C:\backup-scripts\backup-online_NetBackup.bat
Schedule: DAILY_FULL_SAP_DB
Type: Automatic Full Backup
Frequency: every 1 day
Synthetic: 0
Checksum Change Detection: 0
PFI Recovery: 0
Retention Level: 1 (2 weeks)
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
Sunday 09:20:00 --> Sunday 17:20:00
Monday 08:20:00 --> Monday 15:20:00
Tuesday 02:30:00 --> Tuesday 06:30:00
Wednesday 02:30:00 --> Wednesday 06:30:00
Thursday 02:30:00 --> Thursday 06:30:00
Friday 02:30:00 --> Friday 06:30:00
Saturday 02:30:00 --> Saturday 09:30:00
Schedule: SAP
Type: Application Backup
Maximum MPX: 1
Synthetic: 0
Checksum Change Detection: 0
PFI Recovery: 0
Retention Level: 1 (2 weeks)
Number Copies: 1
Fail on Error: 0
Residence: (specific storage unit not required)
Volume Pool: (same as policy volume pool)
Server Group: (same as specified for policy)
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
Sunday 00:00:00 --> Sunday 24:00:00
Monday 00:00:00 --> Monday 24:00:00
Tuesday 00:00:00 --> Tuesday 24:00:00
Wednesday 00:00:00 --> Wednesday 24:00:00
Thursday 00:00:00 --> Thursday 24:00:00
Friday 00:00:00 --> Friday 24:00:00
Saturday 00:00:00 --> Saturday 24:00:00
Policy Name: MNG_AMS20_SAP_DBPW
Options: 0x0
template: FALSE
audit_reason: ?
Names: (none)
Policy Type: SAP (17)
Active: yes
Effective date: 08/02/2011 17:47:45
Mult. Data Stream: no
Perform Snapshot Backup: no
Snapshot Method: (none)
Snapshot Method Arguments: (none)
Perform Offhost Backup: no
Backup Copy: 0
Use Data Mover: no
Data Mover Type: 2
Use Alternate Client: no
Alternate Client Name: (none)
Use Virtual Machine: 0
Hyper-V Server Name: (none)
Enable Instant Recovery: no
Policy Priority: 100
Max Jobs/Policy: Unlimited
Disaster Recovery: 0
Collect BMR Info: no
Keyword: (none specified)
Data Classification: -
Residence is Storage Lifecycle Policy: no
Client Encrypt: no
Checkpoint: no
Residence: MnG
Volume Pool: DataStore
Server Group: *ANY*
Granular Restore Info: no
Exchange Source attributes: no
Exchange 2010 Preferred Server: (none defined)
Application Discovery: no
Discovery Lifetime: 0 seconds
ASC Application and attributes: (none defined)
Generation: 112
Ignore Client Direct: no
Enable Metadata Indexing: no
Index server name: NULL
Use Accelerator: no
Client/HW/OS/Pri/DMI: ph300202.ams20.vzbi.caas Windows-x64 Windows2003 0 0 0 0 ?
Client/HW/OS/Pri/DMI: ph967702.ams20.vzbi.caas Windows-x64 Windows2008 0 1 0 0 ?
Include: C:\backup-scripts\backup-online_NetBackup.bat
Schedule: DAILY_FULL_SAP_DB
Type: FULL SSAP (0)
Frequency: 1 day(s) (86400 seconds)
Maximum MPX: 1
Synthetic: 0
Checksum Change Detection: 0
PFI Recovery: 0
Retention Level: 1 (2 weeks)
u-wind/o/d: 0 0
Incr Type: DELTA (0)
Alt Read Host: (none defined)
Max Frag Size: 0 MB
Number Copies: 1
Fail on Error: 0
Residence: (specific storage unit not required)
Volume Pool: (same as policy volume pool)
Server Group: (same as specified for policy)
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
Day Open Close W-Open W-Close
Sunday 009:20:00 017:20:00 009:20:00 017:20:00
Monday 008:20:00 015:20:00 032:20:00 039:20:00
Tuesday 002:30:00 006:30:00 050:30:00 054:30:00
Wednesday 002:30:00 006:30:00 074:30:00 078:30:00
Thursday 002:30:00 006:30:00 098:30:00 102:30:00
Friday 002:30:00 006:30:00 122:30:00 126:30:00
Saturday 002:30:00 009:30:00 146:30:00 153:30:00
Schedule: SAP
Type: UBAK SAP (2)
Frequency: 7 day(s) (604800 seconds)
Maximum MPX: 1
Synthetic: 0
Checksum Change Detection: 0
PFI Recovery: 0
Retention Level: 1 (2 weeks)
u-wind/o/d: 0 0
Incr Type: DELTA (0)
Alt Read Host: (none defined)
Max Frag Size: 0 MB
Number Copies: 1
Fail on Error: 0
Residence: (specific storage unit not required)
Volume Pool: (same as policy volume pool)
Server Group: (same as specified for policy)
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
Day Open Close W-Open W-Close
Sunday 000:00:00 024:00:00 000:00:00 024:00:00
Monday 000:00:00 024:00:00 024:00:00 048:00:00
Tuesday 000:00:00 024:00:00 048:00:00 072:00:00
Wednesday 000:00:00 024:00:00 072:00:00 096:00:00
Thursday 000:00:00 024:00:00 096:00:00 120:00:00
Friday 000:00:00 024:00:00 120:00:00 144:00:00
Saturday 000:00:00 024:00:00 144:00:00 168:00:00
---------------------------------------------------
Policy Name: CATALOG_BACKUP
Policy Type: NBU-Catalog
Active: yes
Effective date: 07/21/2011 20:42:26
Mult. Data Streams: no
Client Encrypt: no
Checkpoint: no
Policy Priority: 100
Max Jobs/Policy: 1
Disaster Recovery: 0
Collect BMR info: no
Residence: NBU-CAT
Volume Pool: CatalogBackup
Server Group: *ANY*
Keyword: (none specified)
Data Classification: -
Residence is Storage Lifecycle Policy: no
Application Discovery: no
Discovery Lifetime: 0 seconds
ASC Application and attributes: (none defined)
Granular Restore Info: no
Ignore Client Direct: no
Enable Metadata Indexing: no
Index server name: NULL
Use Accelerator: no
HW/OS/Client: Linux Linux ph469702.ams20.vzbi.caas
Include: CATALOG_DRIVEN_BACKUP
Schedule: CATALOG_BACKUP
Type: Full Backup
Frequency: every 1 day
Maximum MPX: 1
Synthetic: 0
Checksum Change Detection: 0
PFI Recovery: 0
Retention Level: 1 (2 weeks)
Number Copies: 1
Fail on Error: 0
Residence: NBU-CAT
Volume Pool: (same as policy volume pool)
Server Group: (same as specified for policy)
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
Sunday 12:30:00 --> Sunday 13:10:00
Monday 13:50:00 --> Monday 14:30:00
Tuesday 12:40:00 --> Tuesday 13:20:00
Wednesday 12:40:00 --> Wednesday 13:20:00
Thursday 12:40:00 --> Thursday 13:20:00
Friday 12:40:00 --> Friday 13:20:00
Saturday 12:40:00 --> Saturday 13:20:00
Catalog Disaster Recovery Configuration:
Email Address: (none specified)
Disk Path: /local/NBUDR
User Name: (none specified)
Pass Word: (none specified)
Critical policy: (none specified)
Policy Name: CATALOG_BACKUP
Options: 0x0
template: FALSE
audit_reason: ?
Names: (none)
Policy Type: NBU-Catalog (35)
Active: yes
Effective date: 07/21/2011 20:42:26
Mult. Data Stream: no
Perform Snapshot Backup: no
Snapshot Method: (none)
Snapshot Method Arguments: (none)
Perform Offhost Backup: no
Backup Copy: 0
Use Data Mover: no
Data Mover Type: 2
Use Alternate Client: no
Alternate Client Name: (none)
Use Virtual Machine: 0
Hyper-V Server Name: (none)
Enable Instant Recovery: no
Policy Priority: 100
Max Jobs/Policy: 1
Disaster Recovery: 0
Collect BMR Info: no
Keyword: (none specified)
Data Classification: -
Residence is Storage Lifecycle Policy: no
Client Encrypt: no
Checkpoint: no
Residence: NBU-CAT
Volume Pool: CatalogBackup
Server Group: *ANY*
Granular Restore Info: no
Exchange Source attributes: no
Exchange 2010 Preferred Server: (none defined)
Application Discovery: no
Discovery Lifetime: 0 seconds
ASC Application and attributes: (none defined)
Generation: 25
Ignore Client Direct: no
Enable Metadata Indexing: no
Index server name: NULL
Use Accelerator: no
Client/HW/OS/Pri/DMI: ph469702.ams20.vzbi.caas Linux Linux 0 0 0 0 ?
Include: CATALOG_DRIVEN_BACKUP
Schedule: CATALOG_BACKUP
Type: FULL (0)
Frequency: 1 day(s) (86400 seconds)
Maximum MPX: 1
Synthetic: 0
Checksum Change Detection: 0
PFI Recovery: 0
Retention Level: 1 (2 weeks)
u-wind/o/d: 0 0
Incr Type: DELTA (0)
Alt Read Host: (none defined)
Max Frag Size: 0 MB
Number Copies: 1
Fail on Error: 0
Residence: NBU-CAT
Volume Pool: (same as policy volume pool)
Server Group: (same as specified for policy)
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
Day Open Close W-Open W-Close
Sunday 012:30:00 013:10:00 012:30:00 013:10:00
Monday 013:50:00 014:30:00 037:50:00 038:30:00
Tuesday 012:40:00 013:20:00 060:40:00 061:20:00
Wednesday 012:40:00 013:20:00 084:40:00 085:20:00
Thursday 012:40:00 013:20:00 108:40:00 109:20:00
Friday 012:40:00 013:20:00 132:40:00 133:20:00
Saturday 012:40:00 013:20:00 156:40:00 157:20:00
Catalog Disaster Recovery Configuration:
Email Address: (none specified)
Disk Path: /local/NBUDR
User Name: (none specified)
Pass Word: (none specified)
Critical policy: (none specified)
11-26-2013 08:03 AM
Policy Name: MNG_AMS20_SAP_DBPW
Monday 08:20:00 --> Monday 15:20:00
Tuesday 02:30:00 --> Tuesday 06:30:00
may be this would be the issue...
once after Monday backup tirggred and compleated at 8:20, then next due would be at Tuesday 8:20 but unfortunatly the backup window is closed by 6.30, which is not giving the 1 day frequency.. so you probalby miss the backup on Tuesday.
same case i am seeing for policy CATALOG_BACKUP
Monday 13:50:00 --> Monday 14:30:00
Tuesday 12:40:00 --> Tuesday 13:20:00
11-26-2013 08:28 AM
As i suspected - your frequency is too much causing overlap
When using frequency based scheduling remember that the window you set is just a start window and as long as the job gets running during that period it will continue (not if it is just queued though)
It is best to have something like:
Window 08:20 to 12:20 (which is a 4 hour duration) and then a frequency of 5 hours
In this way the frequency is just higher than the window duration so it will only run once during the backup window - but it will always be ready to run immediately when the next window opens
When running jobs daily a frequency of 1 day is too much, but any less means you window needs to be less too
Hope all this makes sense and helps
11-26-2013 08:57 AM
Hi Nagalla,
infact i see today (tue) it was missed.
For catalog pretty the same.
The reason about these BU windows depends heavily on the fact i need to avoid overlap for other jobs.
What could be done to have exactly that time range ?
Do i need to change the starting date to have the bu window cover 24hrs then?
Thank you.
Regards,
Michele
11-26-2013 09:08 AM
The question is do you really need a 24 hour window?
It is just a start time not a run time
If you do not have many resources (disk or tape) then you need to stagger your policies to suit your environment
Without an indepth look at your environment it is hard to say exactly what you need
Do you often have jobs queued - if so for how long and why?
Do you use disk or tape to backup to - how many streams can you run at a time and how many jobs do you run per day
The above are the things to look at when deciding what needs to be done with your system but a 24 hour duration window is very rarely needed
Hope this helps and gives food for thought
11-26-2013 09:10 AM
try to keep the same start window for all days.. that would avoide these issue...
or else,make sure that the next start windows is falling in 24 hours range...
in above case if you have a backup window till 8:30 in tuesday , you would not have miss the backup...
but better to have the same backup windows for all days.
11-26-2013 09:12 AM
alternatly you can try changing the scheule from Freqeucy to Calander based and select all days in recurring days without changing the backup start times.
11-26-2013 10:44 AM
In place of frequency based schedule go for Calender based schedule.
11-27-2013 12:57 AM
Thanks to all.
I've got the issue, and now moved from frequency based to calendar.
I already know problems about overlap for different tasks, specifically about SAP ones.
So the necessity to have that kind of BU window, that do not cover 24hrs, it is based on experience on that system.
So, thank you for the tips.
11-27-2013 01:04 AM
As the others pointed out, this is your problem:
Frequency: every 1 day
..
Daily Windows:
Sunday 09:20:00 --> Sunday 17:20:00
Monday 08:20:00 --> Monday 15:20:00
Tuesday 02:30:00 --> Tuesday 06:30:00
Wednesday 02:30:00 --> Wednesday 06:30:00
Thursday 02:30:00 --> Thursday 06:30:00
Friday 02:30:00 --> Friday 06:30:00
Saturday 02:30:00 --> Saturday 09:30:00
Frequency of 'every 1 day' means 'once every 24 hours'.
So, when the backup window opens, NBU checks:
'when was the last successful backup?' (the backup start time is checked)
'has enough time (24 hours) elapsed since the last backup?'
If the answer is NO to above question, the backup will not be submitted.
You can see by looking at your backup window that the answer will be NO on some days.
Change the frequency to 'every 12 hours'.
Backups cannot start every 12 hours because of the backup window. It will merely ensure that 'enough time has elapsed' when backup window opens.
Please see David Chapa's article on Frequency Based Scheduling:
https://www-secure.symantec.com/connect/articles/netbackup-frequency-based-scheduling-1