Manually deleting images, catalog has no record of

[Phibar]

I have about 4.5 TB on a basic disk storage unit that isnt valid I just discovered. The partition is 7TB used for Server_Diff storage.  This storage unit is for Diffs only and actively being written too without issue.  I notice these images are not listed in the catalog for the clients.  I did a search and didnt see anything identical to this issue in the forms.  If I delete these images manually is this going to cause any issues.  My thought is no based on my understanding, but I would love a more knowledgeable confirmation.  Thanks.

EDIT:  The images here shouldnt be older than 2 weeks based on retention.  Some are 2-3 months old.

[sdo]

Please describe exactly all of the steps/methods that you undertook, and also detail your rationale, that have led to you reaching this conclusion.

(FYI - many have walked this path - and lived to regret it...)

I, for one, am glad that you are seeking re-assurances before deleting anything.

Are you absolutely, totally, 100%, sure that you checked all copy numbers?

[Phibar]

Sure thanks for the response.  I'll explain the best I can.

I'm in the process of moving data to a new location and readdressing this space that is actually on the media server as Diff Storage.  All diff policies should only have 2 week retention.  I happened to look at this folder that is for my Servers Diff job.  I noticed the Oldest images in aparticular folder was from 1/23.  This is much older than they should be of course.  I began looking through the catalog to see if it has a listing for that client with matching dates.  I originally checked only primary on each client.  Each client has the yearly and monthly jobs, and then the <2 week DIffs listed.  the yearly and monthly are strictly notated as tape media of course.  I went by client and deleted any images on a couple of clients until I got this gut feeling I should make sure this is OK.  I went through no formal Netbackup trainign before this was pushed onto me so I'm still learning as I go.  I started looking through more copies other than primary on the clients I did remove after I read your response.  I havent found anything of note that would match the dates.  

Here is a date example.

Images are all from 1/23-1/31

Catalog lists only 1/4 for yearly job to tape. Then nothing until February. 

[sdo]

1) What version of NetBackup?

2) And what version of OS?

3) And are you sure that the storage unit is Basic Disk, and not Advanced Disk?

4) Is the storage unit a 'DSU' or a 'DSSU'?

5) Are you sure that there are no other backup policies writing to this storage unit?

6) Does the storage unit point to a UNC path, or does it point directly to a locally attached volume?

7) No matter whether it is a UNC path,or a locally attached volume, then has the target folder path been 'shared' to any other media server(s)?

.

I'll add that it is highly unusual for NetBackup to orphan backup images, as 'backup image management' has traditionally been one of the most robust and stable areas of the product.  Yes, backup images can still be oprpahaned, but this is usually due to administrative error - or wider scale infrastrucure issues - and if either case is proven to be true, then I would suggest that in either or both cases one should really address the other issues before deleting images directly at the storage unit 'storage path' layer.  In short, this is most unusual, and gives me deep cause for concern, and I certainly would not recommend taking this path if you are not at all familar with NetBackup.

I don't mean to push this too hard, but if I were you I would show this posting to your manager, and suggest that perhaps you shouldn't be doing this on your own.  Two heads are better than one, as the saying goes - that way you both get to bounce ideas off of each other - and challenge each other's thinking, and make yourselves explore the both the possible root causes, and the planned actions.

It seems to me to be deeply unfair to push such a potentially dangerous house keeping task on to an unqualified administrator.  In my opinion, if they want you to personally take responsibility for what are actually quite grave risks associated with tasks such as this, then they should at least reward you a course BEFORE you tackle the problem.  At least then you will have been trained (for free), if you suddenly find yourself in the position of having to find a new employer.

What is your recovery plan - if you find that you have deleted something that you should not have?

[Phibar]

1) 7.6

2) MS 2012

3) BASIC

4)No staging

5) There are multiple policies but are the same policy type/retention.

6)UNC

7)Not Shared

I appreciate the insight, I really shouldn't get in to that on a forum but it's an issue regarding limited "bandwidth".  

 I think I may have found the issue by examining the image cleanup job more thoroughly.  I have a generic error referencing a storage unit that is on the same partition.  I don't have a timeframe of when this happened but I believe after January I  and possibly 2/6 setup a SUG and included this SU.  However I believe I actually changed the name to make it more decernable. For Example: j:\Servers_Diff turned into Servers_Diff_J but the storage unit name was not changed just the UNC path.  So now I have 3/20>4/2 in the SU but also images from 2/6>1/23.  I dont have any reference of those older images in the catalog so I think what happened is they are orphaned because I incorrectly performed the SU change.  Atleast these are just Differential jobs that were long superseeded by my yearly retention jobs to tape.

If this makes sense, I should be able to delete them without "consequence"?

Our DR is to rely on the CAT and our most recent to TAPE jobs.  There's no duplication to another site or anything like that.  It's rather primitive to me but that's what we deal with.

[sdo · Accepted solution]

Ok - you appear to have a good handle on root cause, and a sensible evluation of both risk and the value of the old data.  So, removing it manually would seem to be the way to go.

Final point, personally, I'd get somone else on site to agree (in writing / email) with your conclusion and decision - before deleting anything - even if for no other reason that a final double check of date ranges, and image CTIMEs.

And take a safe listing of what was deleted, i.e. take directory listings before and after... maybe along these lines...

...firstly, spend a few moments identifying a command which will list file names, in one column, sorted by name...

...then use this command to...

1) Take a safe listing before deleteing anything, and never amend this file, which we will call 'list A'.

2) Copy list A to list B.

3) Edit list B and trim down to what you think you will have left AFTER deleting.

4) Copy list A to list C.

5) Edit list C and trim down to what you think you will be DELETING..

6) Do the delete.

7) Take another full list - to list D.

...with these next three steps you'll need a command:    for Unix use: diff      for Windows use: fc

8) Compare list D to list B - it should be the same.

9) Take a difference list D to list A - to create list E

10) Compare list E to list C - it should be the same.

HTH.

[Phibar]

I'l go ahead and start to delete them after I record what is in there.  Thank you again for the great support.

[sdo]

Not sure I did anything, you seem to know what you're doing - and simply wanted someone to bounce ideas off I guess - it's a good a thing to stop and take a moment (hour or two or more) when faced with having to consider doing manual deletes.  You did the right thing.