Detecting run-away DIFF backup jobs

[X2]

Once in a while I notice that a backup job is running a DIFF and it backs up almost all the data as if it was a FULL backup. Sometimes, this is not due to some change in the NetBackup side (e.g policy changes, etc) and could go unseen. E.g. I noticed that a NDMP backup of some video recording data was running like that since a couple of days after the FULL. The full backup is monthly as video recording data does not change and only new data is added during the week. Last week I saw the long running job and after investigation, I noticed that the job is backing up more than 10 TB per day! So I ran a FULL and it seems to be behaving better now. Also, I'm fixing another job with 31 72TB of data!

Normal backups running like this is not an issue but when big ones like this (> 5 TB) become "run-away", I would like to know. How do others work around such issue? Is there a way in OpsCenter to generate an alert? Or does someone have some home-brew script to monitor such jobs?

Any help would be appreciated.

[Alexis_Jeldrez]

I would make sure folders aren't being renamed or moved and try using Accelerator if possible. If NetBackup Accelerator is not compatible with your configuration or policy, then try Synthetics. I guess Full retentions are longer than a month.

[X2]

For the smaller job (totaling 17TB of data for now), the root of the tree contains several folders for video recordings named using event. Every time there is a new recording fo the event series, the content is put in a date-named folder under the event folder. Older folders are not touched.

The FULL  has 1 year retention while DIFFs have 3 months.

About Synthetic backups - it is NDMP type of backup and Synthetic backups support only Standard or MS-Windows. Accelerator - these backups are going straight to tape due to size and video data (low deduplication rates on DataDomain)

[sdo]

The backup style you are using is traditional and very very stable.  I quite strongly suspect that you have not unearthed a new bug, instead I would suggest that the only possible root cause is that the files are touched (opened?) in some way, and one or more of the attributes of the files is/are changing - maybe A/V opens the video files for read to check the header, and closes the file, and this updates the last access time, and NetBackup sees that as a change - and so takes a backup.   Or maybe "indexing" opens the video file to collect a summary of content, or maybe a script is running somewhere doing something.

I really think that a great many many more poeple would be experiencing this issue with their backups if there was a new problem/bug the older most stable elements of NetBackup.

Next steps would be:  1) run a daily/weekly script to collect/list entire folder tree in such a way that the folder walk does not cause any updates 2) compare day-to-day or week-to-week and look for attribute changes.