List/ compare files NOT backed up (omitted / skipped / ignored / left out) due to filter

[dl2rcf]

Dear Backup Exec users and technical support,

In my department we use BE2010 R3 on Windows Server 2008 R2 with a big tape robot. Thanks to modern times, admins come and go; I think they call it "temporary employment" and it's hip :)

This leads to possible and actual filter policy change of the backup jobs to tape(We use positive filters like "archive *.avi, *.txt" ...; I know it is a social problem, but I am trying to find a technical solution for this) and therefore I don't know what folders with yet archived files on the server I can securely delete when the server's drive is full. Besides it can occur that users store files on the server and don't communicate to us, that file extension xyz shall be included in the backup jobs from now on...but do that "much later"

- Is using "positive" filters state-of-the-art when archiving in an environment like the one described above?

- Is there a way to get a list of all files NOT archived (and therefore not present in BE's catalog) but present on the server?

Here are some thoughts about possible solutions:

- For example I am dreaming of a way to "mount" BE's catalog to virtual windows drive(s)/ directories and then do an (of course a non-file-content) comparison with a standard software such as "Total Commander".

- 3rd party tool that does this comparison and has a GUI like the windows file explorer (but with the difference that You always see all the files of all subfolders of the directory selected by the user) but where you can browse "only NOT yet archived on tape" files in a tree view

- Or of course list all files NOT backed up of a folder (and its subfolders) in a report: show files not present in tape backup/ catalog but on the server's directory.

Any ideas about that? Thanks in advance for posting!

Ruben

[Jaydeep_S]

I think, you might want to see the Backup Exec Archiving option. This would automate and ease the daily job complexities in the situation mentioned above. Here is some documentation about this -

How the Archiving Option works

Best practices for the Archiving Option

About creating an Archiving Option archive job

Besides this, there are full scale archiving products available like Enterprise vault. 

[dl2rcf]

Hi there,

thank You for the quick reply

Depending on an option that you specify, deletion occurs immediately after the archive job completes or after the vault store is backed up

Ensure that an item is included in only one archive job. Unlike backup jobs, archive jobs cannot share the same items.

I assume that the "archive option" is not an option for our environment, since it seems to require (?) to delete the files A.S.A.P. ?

We have a 20 TB server RAID and want to only delete the oldest items, so that users can read access those files fast on their own without the need of (us manually) restoring the items with which a user works.

Mhhm - but on the other hand You are right... Instead of automatic incremental-only backup jobs why not replace them completely by manual archieve job (that delete the files/ directories afterwards automatically (did I understand that right?) ).

[thinking loud] This would solve various problems:

- You can be sure that You got Your stuff on tape: When it is logged in our "homegrown" process logfile as processed and is NOT located on the server (any more), it surely is on tape.

- You woudn't have to mess around much with BE2010's wacky implementation of incremental backups (that by the way bloat the catalog and is a not recommended to be used exclusively without a full backup from time to time)

mhhmm...but can You please confirm me:

Is there something like a "template", so that You can do archives "semi-automatically" e.g. do the same filter and so on that you did ONCE for directory xyz now for directory abc?

Many thanks in advance,

Ruben

[Jaydeep_S]

Yes there are filter based on which the archive job runs. I would say, you could setup a test Be server and enable archiving option in trial and explore the various options that you get. The only reason I am not sugesting you to do this on a seperate server as the Archiving option would make a lot of changes to the setup, like installing its own SQL database, etc.

Here is a white paper

http://www.symantec.com/docs/TECH129691

[dl2rcf]

I read the white paper with much interest...but I asked myself if that isn't a BE2010 option but a completely different software (Web based, SQL)...

Speaking of SQL, what format is the "standard" BE2010 catalog and how to access it via an API/ C#?

I still have the problem (even with Your mentioned archive option) that I need to know, if I can delete a whole folder. For that I need some catalog/ database information "Are there files in this folder that haven't been put to tape yet (e.g. because some user put them recently to the folder before backup/archive happened" and list those files. E.g. only deleting folders with a own software that browses the folders and warns "In the selected folders are the files: xyz.avi, abc.doc, ... that haven't been put to tape yet. Are You sure, You wan't to delete them anyway? (YES) (Abort)

A nice report about this would help me, too, but I can't find a suitable one...

[Jaydeep_S]

Here is a utility that allow you to dump the catalogs to a text file.

http://www.symantec.com/docs/TECH136392

[dl2rcf]

:)

I don't want to sound like a nagger, but I think this is my favorite utility of last year :)

https://www-secure.symantec.com/connect/forums/detailed-information-about-catalog-cattools-not-worki...

Isn't there something more high-level like, where you can browse the entire catalogs like "search catalogs" on an API/ command line level. BEMSDK or BEMCMD

bemsdk_managed.dll sounds promising for doing stuff in .NET (?), but there I can't find any documentation

And for BEMCMD there are like 3 examples of instructions in the 1000+page BE 2010 manual...

Where is the documentation for those tools?

Meanwhile I will try getting Cattools to run...

[Jaydeep_S]

Symantec Backup Exec (TM) 2010 Command Line Applet (BEMCMD) Guide - (English)

http://www.symantec.com/docs/TECH126925     

[dl2rcf]

BEMCMD with the option -o38 (search catalog) looks very nice at first glance indeed. I am using 13.0.5204

However, results seem to differ from the "search catalog" option in BE GUI's restore menu's "search catalog" and from reality:

I have drives M, N, O, P as drives on which are running backup jobs

bemcmd -o38 -vn:M: -sn:MYSERVER -s -pa:MYDIR -fn:*.raw

I get 384 results, that seem pretty nice. Though all file sizes shown are wrong (~400 Bytes each instead of some GBs)

But...files from incremental backup job will not be shown here ?!?!

then I did:

bemcmd -o38 -vn:O: -sn:MYSERVER -s -pa:MYOTHERDIR -fn:*.raw

I get in return 0 items found ?! Even no files from full backups

It is not a case sensitivity issue...and "search catalog" within restore menu does list many files! (and in reality there are many raw files on tape, since I restored some yet and they work just fine)

Please help me, since this functionality comes close to what I need (even though I am still looking for a "report" that list not backupped files...)

Do I perhaps need to refresh the catalog somehow?

[dl2rcf]

Is BEMCMD 13.0.5204 still the most recent version?

I read the Service Packs 1-3 description if BEMCMD was target for any of those updates, but negative...

I still need a way to access the catalog via BEMCMD, but as described above, results differ from BE2010's GUI "search catalog" 's results to BEMCMD -o38 's "search catalog" 's results.

I also tried "rebuild catalog", but no change

Any idea (refresh, rebuilt, new version of BEMCMD for standalone download)?

Thanks a lot in advance!