"catalog backup change successful" e-mails

[danpritts]

Hi, 

NBU 7.5.0.6, RHEL6.  

we've recently started to recieve nightly emails with the subject "netbackup catalog change successful."  We get a LOT of them.  Every morning we get one for each of the last week or two's worth of catalog backups; at a glance it is identical to the "netbackup catalog backup successful" report we get after each catalog backup.

Jan 5 at 5:41, netbackup sent me 18 individual emails.  Jan 6 5:41, 10 more.  Jan 7 5:41, 5.   Jan 8, 10.  jan 9, 8.  etc.     This is not a mail system problem; postfix logs show netbackup is sending them each morning.

I haven't analyzed specifically which days we are sent which logs, 

My catalog backup policy specifies that 3 copies are made of each full catalog backup:  2 to tape, and 1 to disk.  2 copies made of each incremental, 1 tape and 1 disk.  Presumably importantly, the disk images expire after 2 weeks (tapes 3 months/1 year).  

I see discussion of the "new feature" here: http://www.symantec.com/connect/forums/netbackup-catalog-backup-change-successful-started-showing  

Unfortunately there's no solution listed there.  

Help, i'm drowning in spam.  Worse, my boss is complaining.  

thanks!

[RonCaplinger]

Those emails you are receiving are valid and are important, in the event of a disaster.  Every time the NetBackup catalog backup runs, you should receive one of these. 

In addition, every time the backup images required to recover the catalog change from one copy number to another, you will receive an updated email that contains the new DR info.

For instance:

1. when you run a full catalog backup to disk (copy 1), you will get an email
2. when that image is duplicated to another storage unit (copy 2), you should not get an email
3. when that first image (copy 1) expires and the duplicated version (copy 2) becomes the primary, you will get another email

I would suggest setting up an email account outside your company to send a copy of this email.  Make sure that you document where this email can be located during a disaster, and that more than one person has access.  I would also suggest keeping a copy of the DR email locally, just set up a folder in your email system and a rule to move the messages to that folder.

[danpritts]

Hi Ron -

I'm perfectly willing to get one of these emails every night.  I understand the importance of the DR file.

I get an e-mail from the nightly catalog backup.  The subject of the nightly backup is "Netbackup catalog backup successful."  That comes in just after the catalog backup job runs at midnight.  As you suggest, we have documented and offsited this file.  

I *also* get a set of emails every night that includes the DR files from several days back.  I looked more closely based on your item (3), and it does appear that each time i get a new copy of this DR email, a disk image has expired.

Let's take my Dec 21 catalog backup.  I got a DR file message on Dec 21, yay.

Starting on Jan 5, I received 14 more messages explaining that there was a change to that particular catalog backup.  In fact, each day, I received two messages in quick succession telling me that first one, and then another, catalog backup image file had been expired.  

E.g., first message:

Date: Mon,  5 Jan 2015 05:41:36 -0500 (EST)

A disk image has expired.  Great to know.

Next message, 2 seconds later:

Date: Mon,  5 Jan 2015 05:41:38 -0500 (EST)

Another disk image has expired.  Awesome.  

And so on, 2 messages per day for 6 more days.  

this is only for the Dec 21 catalog backup; same sort of thing for each and every day's catalog backup. 

But back to your point (3).  None of the files that are expiring are ever the primary copy of a particular image.  The * in the listing is always in front of a tape.  The tapes never expire.  (At least not in this time frame)

Regardless, any idea that I would need all 18 of the messages I received this morning for DR purpose is insane.  In a disaster the last thing I'm gonna want to do is figure out which of ten thousand messages I want.  

[danpritts]

(I read my response, and I think it was kind of snippy at you Ron, my apologies)

[Andrew_Madsen]

Actually in a disaster the email you want is the LAST one you received because it tells you where everything is. You can dispense with emails entirely if you are saving the DR information to a NFS share (or CIFS if you are using a Windows master). You do need to be able to recover these files in the event of a disaster though.

[danpritts]

I'd think that what I really want is the one associated with the most recent catalog backup, not the last one i received.  Consider my scenario, summarized from above:

a bit after midnight i get the dr file from the catalog backup that ran at midnight -> this is the one i want, right?

at 5:41 am I get a bunch of updated versions of DR files from the last few weeks' catalog backups, which have had some of their images expire.  

[danpritts]

oh yeah, we do save them to an NFS share.  But, like Ron suggests, we also send them via email, with a copy going to a google group we've set up for the purpose.

[jim_dalton]

And is the nfs share going to be part of your disaster? Jim

[danpritts]

Uh, probably, yes.  This is why, as I mentioned, we send the reports offsite via email.

[Andrew_Madsen]

You want the email on the backup that ran at midnight. You also want the ones that tells you the catalog backup expires because there is no guarantee that the last backup got off site successfully. If you got JUST the last successful notification and no other notification of change to storage unit or vault then you would be scrambling looking for where your catalog is. This also applies if the last two backups were garble on tape.

[Andrew_Madsen]

Sorry. One more note. While the NAS will most likely be in any data center disaster the most common issue is a server disaster. No smoking hole in the ground where the data center used to sit just a smoking server in the rack where your master server use to sit.

In the latter case you have your DR files sitting right there and you can scan through them for the needed information. In the former case you want some way of correlating what you have with what you re supposed to have which is why Symantec alerts when some step or process affects the location or existence of your DR files.