reports with large output failing

We have a need to generate data that is creating large reports, and failing.

 

Ideally, we wish to get "Access Details for Paths" to generate a CSV for a large base of folders/files, across a large period of time.   Even carving the time down to single days, results in timeouts.   Carving it further into sections of folders and by hour, is a tedius process and taking too long.

 

We have the maximum value set for automatic truncating of results set, but may need to exceed this.  Regardless, its not truncating its simply failing to complete (example last report ran for about 100 hours before failing).

We are running version 5 with RP1

Any suggestions, ideas?  Thanks!

2 Replies
Highlighted

Re: reports with large output failing

KDrone:

Reports run on the indexers and then are transferred back to the Management Server (MS) for the DataInsight (DI) environment. During these processes there are memory requirements and connectivity between the nodes must be maintained. Assuming all is well in that regard the reports are transferred for each indexer in your data scope back to the MS and merged together for the final product.

These steps can be monitored or viewed in a pop-up window accessible from the report execution window where it was created and run. This is the first location to validate there was an error and where it occurred.

Each process creates log files that may be used to confirm the progression of the step and the outcome. You have mentioned a version (5.0.1.1001 / 5.0RP1) and ,assuming a default installation, C:\DataInsight\Data\Console\Reports\Reportruns contains a numerical list by the ReportID number of the data retrieved on the MS or data being prepared (it is deleted post transfer) on an indexer. To get the ID number of the report

 

RID_0.jpg

Logs are located under C:\Program Files\Symantec\DataInsight\log\reports in the same numerical format.
A review of the log data for error messages can point you to the problem be it  with too little memory assigned or other issues.

Please note there is a known issue with multiple threads configured for a report on a indexer and running entitlement reports in parallel for performance enhancement that is resolved under a recent patch.

If you can provide more details from those logs and assist me in narrowing down the problem determination I can offer some further debugging in the forum. If you would prefer the privacy and interactive nature of a Support case then Please call into Veritas Support Services and one of our trained technical experts will assist you in walking through the issue.


Rod

Highlighted

Re: reports with large output failing

Is there a known issue with DI 5.0.1 with DQL and outputting reports to 1 file?

My reports will not compile into a songle file.. I get mutiple files.

Here is my DQL statement.

From user
get name, principal_name, login, Home_directory, Department
If Home_Directory not in ('nowhere')
format user AS CSV;

Thanks for the help!