How to consolidate archived backup tapes

[DSAI]

Hi All,

We have alot of archived tapes going back a couple of years that contain alot of duplicate data. Approx 500Tb on at least one hundred LTO4 tapes. I can't say for sure but I think its at least 50% duplicate files. The previous backup administrator would perform weekly full backups of our fileservers with no tape rotation (sigh).

I wanted to know if Backup Exec had options to consolidate the data.

I didn't want to use deduplication because that requires more disk space than we have available. I was also concerned that the resulting de-dup'ed data would be stored in a Symantec proprietary format.

At the moment I think what I have to do is

1. load the tapes

2. catalog the tapes

3. Identify what are the duplicate files and delete them. (not sure how to do this step)

4. consolidate the data.

I was wondering what would be the easiest way to do this? We have Backup Exec 2012 with a dual LTO6 tape library and a spare LTO4 tape library

any suggestions would be appreciated.

[Kunal_Mudliyar1]

Identify what are the duplicate files and delete them. (not sure how to do this step)

= This cant be done as you cannot delete individual files of a tape, you have to erase the tape and it will delete all the data inside it.

consolidate the data.

= In BE you cannot consolidate data from 2 tapes to 1 , you have to restore the data from both the tapes and write it to a single tape

I guess the best way  to do this is connect the LTO 4  library to the BackupExec

Catalog the tapes

Find the duplicate data and erase that particular tape.

[DSAI]

Hi,

the tapes have a mix of unique and duplicate data..so i wouldn't be able to delete the entire tape.

It sounds like I will have to restore the data from the tapes....merge it using a third-party tool and then back it up again.

does this sound right?

[Kunal_Mudliyar1 · Accepted solution]

That sound right.

[DSAI]

ok...This is going to be an extremely long project. :(

thanks for the information.

[pkh]

When you talk about files, are you referring to a file like a.txt or a file created by a backup job?  What is this unique data that you are talking about?  Is it a backup that has gone straight to tape?  Depending on your answers, you may have not gotten the correct information regarding your situation.  You probably would not need any 3rd party software.

[DSAI]

Hi pkh,

When I refer to files, I mean the actual data like the .txt or jpg files.

What I mean by unique data is that we would run a full backup of a fileserver to tape every week with no rotation and so some of the files that had not been touched (modified) would be duplicates whilst others that had been modified would be 'unique'.

Yes, the backups did go straight to tape.

regards,

[pkh]

Suppose you have 

Backup 1 - a.txt and b.txt

Backup 2 - a.text (modified) and b.txt (unmodified)

I think what you are proposing to do is to eliminate b.txt from one of your backups when you consolidate.  This is equivalent to shredding documents which may have legal implications. Have you checked with your lawyers as to the legality of what you are trying to do?