05-04-2020 03:09 AM
Hi there,
I've been seeing an odd status within the DI System Overview stating that:
"1 nodes have files piling up in indexer err folder"
Looking into the error in detail, it looks like I'm actually getting a:
"Number of files in directory indexer/err is consistently above threshold" - which is consistent with the number of SQL-Lite error files I'm seeing in there when I look manually.
Any thoughts for where I should start looking to nail down where the problem is please? I was wondering on drive space, but the indexer is currently running at only 5% (16GB), and it's Classification where the majority of space is being held (93%).
Any advice to help me drill-down to identify the issues would be much appreciated. Thanks.
Solved! Go to Solution.
05-04-2020 06:03 AM
Mike, you did not state which files are going to the error folder. If your threshold is still at the default setting it takes a relatively small number to cause the alert.
If they are scan_ files they could be out of time sequence and pretty much useless if so as they will be generated again upon the next scan event. The type is denoted by the extension .isqlite are incremental scans and .sqlite are full scans.
They have failed indexing and there are two places to check on why they failed. The commd0.0.log or an iteration of it records the indexing (check for idxwriter) and the index-<MSU_ID>.log is the particular indexing of those files. you can start your investigation there.
Pix
BTW the MSU_ID is the number in the file name the long 15........ number is time in epoch the _###_ is the ID. The logs are located at C:\Program Files\DataInsight\log\ and C:\Program Files\DataInsight\log\indexer (By default)
05-04-2020 07:43 AM
Mike the alert setting can be made per indexer in the advanced settings tab on the server view in the console.
Choose the server from the inventory list, servers.
05-04-2020 06:03 AM
Mike, you did not state which files are going to the error folder. If your threshold is still at the default setting it takes a relatively small number to cause the alert.
If they are scan_ files they could be out of time sequence and pretty much useless if so as they will be generated again upon the next scan event. The type is denoted by the extension .isqlite are incremental scans and .sqlite are full scans.
They have failed indexing and there are two places to check on why they failed. The commd0.0.log or an iteration of it records the indexing (check for idxwriter) and the index-<MSU_ID>.log is the particular indexing of those files. you can start your investigation there.
Pix
BTW the MSU_ID is the number in the file name the long 15........ number is time in epoch the _###_ is the ID. The logs are located at C:\Program Files\DataInsight\log\ and C:\Program Files\DataInsight\log\indexer (By default)
05-04-2020 07:26 AM - edited 05-04-2020 07:27 AM
Thanks for the reply. I'm seeing a combination of audit and scan SQLITE files, but I'm suspecting this may actually be more down to an overly sensitive threshold for the indexer errors - there's not that many in there!
Could you let me know how I can change the threshold please, and if there's any guidance for what a realistic threshold would be?
I had some issues with some errant drive shares it was trying to scan, and I've been doing some tidying up and manual scanning to get new complete scans completed, which has worked well and I can begin to re-classify some key areas again. If I can get the threshold amended, I think that should look a lot healthier.
Many thanks.
05-04-2020 07:43 AM
Mike the alert setting can be made per indexer in the advanced settings tab on the server view in the console.
Choose the server from the inventory list, servers.