Indexer/err folders file retention

[drahrig]

In the last few weeks the indexer/err folders have been receiving and retaining files. There files contain .err, .sqlite and .isqlite files. What is best practice for getting these files out of the folders. In a couple of instances the backlog is nearing 20,000.

[Rod_p1 · Accepted solution]

Dave - there can be many reasons for failing to index the files prepared for the indexer from the collector. More typically we are interested in the names of the files as they can list the intent of the files like audit, scan, tag, etc.

We should investigate why your files are failing. Are they out of sequence? We do not index older scan data and they can be deleted; as new scans are performed the data will be updated. However Audit data is one time only data and cannot be recreated making it not a candidate for removal by deletion and the risk is missing audit events from I/O operations that cannot be duplicated to be regenerated.

Reasons run from indices corruption to disk full conditions and you will likely want to have a Symantec expert assist you in the decision to mitigate risk to the data collected.

You could post a listing of the files you have to the forum for a cursory review to see what types they are. I would prefer an attachment versus 20K files in a comment box personally. We could at the least let you know the risks associated with the data and its removal.

A better suggestion would be to initiate a support case with Symantec and discuss your situation interactively with a Support professional who can guide you on the different file types, review contents and troubleshoot the reasoning for the failure to index.

Since it is on the indexer and not on the collector we know that you are receiving files from your devices and they are properly parsed and prepared for indexing and transferred which is eliminating many of the primary causes for failure.

On your own you can check the commd logs {Default - C:\Program Files\Symantec\DataInsight\log\commd0.0.log} from the collector and indexer for the transfer and receipt of the files. Then the process of batching [IndexWriter$IndexerThread.indexBatch] and running the idxwriter.exe binary [IndexWriter$IndexerThread.indexFiles] to process them. You can also check the idxcheck#.log file on the indexer worker node to see if there is an obvious issue with a index in the file.
 

Note:

the index number and the MSU number from the log file name should coincide.

INDEX: 12 (C:/DataInsight/data/indexer/default/12/12)  / audit_cifs_12_1419350732097527_2.sqlite

C:\Program Files\Symantec\DataInsight\log\err\commd0.0_err.log should contain failures from the indexing such as:

WARNING: #{33} [IndexWriter$IndexerThread.indexFiles] 3 files could not be consumed in batch mode
WARNING: #{33} [IndexWriter$IndexerThread.indexFile] **** Error indexing audit_cifs_12_1419350732097527_2.sqlite. Moving to $data/indexer/err ****
WARNING: #{33} [IndexWriter$IndexerThread.indexFile] **** Error indexing audit_cifs_12_1419351332545376_2.sqlite. Moving to $data/indexer/err ****
WARNING: #{33} [IndexWriter$IndexerThread.indexFile] **** Error indexing audit_cifs_12_1419365549139766_3.sqlite. Moving to $data/indexer/err ****
 

You can check the contents of  C:\Program Files\Symantec\DataInsight\log\err\indexcli0.0_err.log

All files are in the default location please change the driver letter or location accordingly if you have installed under an alternative location.

Hopefully that will get you started in the review necessary to message the risk to the data which is failing to be indexed.

Rod