Classification generating GBs of temp data on the file server

[TTree2007]

Hi,

I'm running a test of the classification on our internal Windows file server (DI 6.0).  This was working fine with a small folder of information but when running against a share with 240+GB it appears to be creating lots of temp files on the FILE server (\Datainsight\Data\Content...).  Although I can see and have set a 'Classification Safeguard' setting to ensure you don't run out of disk space I assumed this was on the DI server (with Classification role) rather than the actual file server itself-? 

I'm reading the Admin guide at the moment but any advice welcome or pointers to documentation on how I can restrict the temp files (size or free space setting) on the file server would be much appreciated.

Thanks 

[Rod_p1 · Accepted solution]

TTree, in the shipping binaries there are documentation files. Specifically there is a Veritas_Data_Insight_Classification_Reference_Guide. You are correct the safeguard settings monitored by the watch dog service are for the Classification Server node.

Scoping of the classification targets (File, folder or whole share, perhaps a watch list) will classify the scope against your enabled policies by contents. Classification is enabled as long as one policy is set to enabled. As the ClassifyFetchJob runs against the scope we create an input database and mark the location of the numerical hierarchy where the file itself is being kept in the \DataInsight\data\classification\content folder.

Based on the number of threads configured under Total threads (default 1 thread for winnas devices which your question implies) the content will be transferred for assessment using the FileTransferJob_content. This will backlog if the classification server safeguard kicks in, classification server is slow, underpowered (see minimum specs in installation guide table 2-3) or the network cannot keep the transfer going  (the request occurs each 10 second interval).

If monitoring the classification, safeguard or buildup is causing you concern you could open a support case with Veritas and have a technician review the VIC classification logs and status to ensure it is progressing and review the resources to see that classification should be able to proceed unencumbered or there are needs that have yet to be met.

Note: References in the manual for classification will lead you to the jobs descriptions and log titles / purposes so it is redundant to include it in this message.

Rod

[TTree2007]

Hi Rod, Thanks for the quick response.

First, yes, the DI server is a test one so doesn't have anything like the recommended processors/memory, but I'm happy with that side.

It's good to understand the process so thanks for explaining that.  I had it in my head that it just queued the file on the classification server but I get that there must be 'safeguards' at various stages of the process. 

I noted while reviewing the documentation yesterday the location for the Safeguard settings for Windows File Server Agents under the 'Scanning and Events Monitoring' in Global Settings.  It was set to 2GB (default) so I've changed to a percentage (so not to trigger our file server alerts!).  When it's running it seems to be performing remarkably well seeing as our DI server only has 4 vprocessors and 8GB RAM. I appreciate it's only a 'test' and not under real world loads!

Cheers

[Rod_p1]

Glad to help TTree

We have seen underpowered machines bottleneck and cause some congestion. Since it is a test I think you will be looking at policies and search capabilities. You should do just fine after the classification is completed as the rest is just querying tags databases in the indexes and running reports.

Rod

[TTree2007]

Hi again Rod,

I can see the guide when I browse the 'help' on the DI server but could you advise where I find the Classificaiton guide as a pdf... or send a link to the guide please?

The guide doesn't appear on the support pages or SORT...

https://www.veritas.com/content/support/en_US/DocumentBrowsing.html?product=Data%20Insight

https://sort.veritas.com/documents/doc_details/SDI/6.0/Windows/ProductGuides/

Thanks 

[Rod_p1]

TTree we do not publicly release a link to that document but the programmer's guide and Classification guide are in the root folder of the extracted binaries you downloaded for installation (latest version is Data Insight 6.1) or on the media kit if you had it shipped to you.

When you installed  your Management Server from Veritas_DataInsight_windows_6_0_0_9500_x64.exe were they not in the same folder?

 Rod

[TTree2007]

Ignore this Rod, I've found it, it's part of the actual binaries!  Would make sense to put it up on SORT too though