cancel
Showing results for 
Search instead for 
Did you mean: 

DQL report problem

drahrig
Level 4
Partner

I am attempting to create a Sensitive files with violated policies report from the template. I have uploaded a .csv with the policies named, however when the report run and I look at the output, no policy is listed. I have included the script, any suggestions?

 

FROM   path

GET      device.name,

             msu.name,

             absname,

             activity_count,

             dlp_policies

 

IF           issensitive=1

 

AND       path.msu.name = "xxx"

              path.msu.name = "yyy"

              path.msu.name = "zzz"

 

FORMAT    dlp_policies AS CSV;

1 ACCEPTED SOLUTION

Accepted Solutions

drahrig
Level 4
Partner

The issue was resolved when we increased the memory allocated to the sensitivefilesjob on the management server.

View solution in original post

10 REPLIES 10

Phil_Rosier
Level 4

 

Hi Drahrig

I think the list of MSU's should be tied together as an OR function, such that the report will create a record if there is a sensive file exists in either msu "xxx", "yyy" or "zzz". Please see below.

FROM  path

GET    device.name,
           msu.name,
           absname,
           activity_count,
           dlp_policies

IF        issensitive=1 
           AND (msu.name = "xxx"
           OR msu.name = "yyy"
           OR msu.name = "zzz")

 

FORMAT dlp_policies AS CSV;

 

Alternatively you could specify the list of msu's as a list.

FROM  path

GET    device.name,
           msu.name,
           absname,
           activity_count,
           dlp_policies

IF        issensitive=1 
           AND msu.name IN ( "xxx", "yyy", "zzz")

FORMAT dlp_policies AS CSV;

 

Or if it's a long list, you can import it as a csv file

FROM  path

GET    device.name,
           msu.name,
           absname,
           activity_count,
           dlp_policies

IF        issensitive=1 
           AND msu.name IN FILE ( "msu_list.csv")

FORMAT dlp_policies AS CSV;

 

Kind regards

Phil

 

drahrig
Level 4
Partner

When I run the report the msu's show up, however, the dlp filed is populated with "" instead of the policies.

Rishi_Thaper
Level 4
Employee

Hi Drahig,

The following query comes back with policy list for me. Can you check if  policy names correctly appear in Data Insight dashboard / context map? Also, what version of DI are you using?

 

FROM  path

GET    device.name,
           msu.name,
           absname,
           activity_count,
           dlp_policies

IF        issensitive=1 
           AND msu.name IN ( "xxx", "yyy", "zzz")

FORMAT dlp_policies AS CSV;

 

Regards,

Rishi

 

drahrig
Level 4
Partner

I have used the list from the DI dashboard as well as a list provided by the DLP admin. Both return the same results: dlp_policies """","""","""". The policies are not being populated into the column. 

drahrig
Level 4
Partner

Sorry, we are using DI 4.5.

Rod_p1
Level 6
Employee Accredited Certified

Dave do you have policies showing up in the dashboard report for the filers you are querying?

Since you are using this list it is assumed yes.

Assuming we have proper policies we would expect them to be populated.

Are there any errors in the reportrun log for the report?


Rod

 

 

Rod_p1
Level 6
Employee Accredited Certified

Some new information after an internal discussion today over your issue Dave.

Seems in a properly configured DLP / DI integration we create new tables that can be polled when creating a 'PATHS' view for DQL.

You can check the values are present in the C:\DataInsight\data\conf\policy.db.### database.

The quick procedure to query (Never change anything in the databases!) the database would be to determine the tables and see what is present.

Example:
Do not use limits if you wish to see them all and output to CSV if you wish to have a list.

DQLquery_0.jpg

You can ignore the double-byte characters in my example I do not have the correct character set loaded.

We should expect that you have the table populated for the Policy names to show up.

My data is limited in the lab to provide a sample.

Report Output Sample
path_rowid device_name msu_name absname activity_count dlp_policies
1 XXX.XXX.89.2 scaleshare10 \\XXX.XXX.89.2\scaleshare10\dfs_final1\data_insight-DataTest 1 "SNA"  
2 XXX.XXX.89.2 scaleshare10 \\XXX.XXX.89.2\scaleshare10\dfs_final1\diiii\sna1.csv 1 "HR"  
3 XXX.XXX.89.2 scaleshare10 \\XXX.XXX.89.2\scaleshare10\dfs_mismatch\dir1\f1 1 "dashboard"
14 XXX.XXX.89.2 scaleshare10 \\XXX.XXX.89.2\scaleshare10\backup\Rod_test_dfs\dir1\file1 1 "Sensitive Policy"
28 XXX.XXX.89.3 anushare4 \\XXX.XXX.89.3\anushare4\texas.txt 1 "Mgmt Policy"
97 XXX.XXX.89.3 ShareExportTest \\XXX.XXX.89.3\ShareExportTest\two_paths_issue\report_output.zip 2 "Mgmt Policy"

 

 

 

 

 

 

Rod

 

Rod_p1
Level 6
Employee Accredited Certified

Some new information after an internal discussion today over your issue Dave.

Seems in a properly configured DLP / DI integration we create new tables that can be polled when creating a 'PATHS' view for DQL.

You can check the values are present in the C:\DataInsight\data\conf\policy.db.### database.

The quick procedure to query (Never change anything in the databases!) the database would be to determine the tables and see what is present.

Example:
Do not use limits if you wish to see them all and output to CSV if you wish to have a list.

DQLquery_0.jpg

You can ignore the double-byte characters in my example I do not have the correct character set loaded.

We should expect that you have the table populated for the Policy names to show up.

My data is limited in the lab to provide a sample.

Report Output Sample
path_rowid device_name msu_name absname activity_count dlp_policies
1 XXX.XXX.89.2 scaleshare10 \\XXX.XXX.89.2\scaleshare10\dfs_final1\data_insight-DataTest 1 "SNA"  
2 XXX.XXX.89.2 scaleshare10 \\XXX.XXX.89.2\scaleshare10\dfs_final1\diiii\sna1.csv 1 "HR"  
3 XXX.XXX.89.2 scaleshare10 \\XXX.XXX.89.2\scaleshare10\dfs_mismatch\dir1\f1 1 "dashboard"
14 XXX.XXX.89.2 scaleshare10 \\XXX.XXX.89.2\scaleshare10\backup\Rod_test_dfs\dir1\file1 1 "Sensitive Policy"
28 XXX.XXX.89.3 anushare4 \\XXX.XXX.89.3\anushare4\texas.txt 1 "Mgmt Policy"
97 XXX.XXX.89.3 ShareExportTest \\XXX.XXX.89.3\ShareExportTest\two_paths_issue\report_output.zip 2 "Mgmt Policy"

 

 

 

 

 

 

Rod

 

drahrig
Level 4
Partner

The issue was resolved when we increased the memory allocated to the sensitivefilesjob on the management server.

Rod_p1
Level 6
Employee Accredited Certified

Assumptions:
You have greater RAM available than the setting you use and it is extra in the machine.
Default setting in a 64bit machine is 4GB of RAM (4096)

Command should be performed on the node assigned as Management Server (MS) hosting the console.

Note: the MS node should always be number 1
 

Error is:

V-378-1312-807: Error importing Sensitive file paths from Data Loss Prevention Server (Details: OutOfMemoryError detected. SOLUTION: Set higher memory limit for DLP scan by issuing following command on the Management Server [default (mb):512]: configdb -o -T node -k 1 -J sensitivefilejob.max.memory -j <new_limit_mb>)

Remedy:

(Perform these steps in a DOS window opened as Administrator with permissions to run the binaries)

Check for the inclusion of any prior changes in the Object Attributes:

C:\Program Files\Symantec\DataInsight\bin>configdb -p -T objattr -k 1 |findstr sensitive

Modify the setting to  a larger number to test the download (Example 12GB):

C:\Program Files\Symantec\DataInsight\bin>configdb -o -T node -k 1 -J sensitivefilejob.max.memory -j 12288

Check to verify the change:

C:\Program Files\Symantec\DataInsight\bin>configdb -p -T objattr -k 1 |findstr sensitive

node:1:sensitivefilejob.max.memory=12288

Run the job to initiate the download:

Note: the job name is case sensitive.

C:\Program Files\Symantec\DataInsight\bin>configcli list_jobs|findstr Dlp
| DlpSensitiveFilesJob|10/20/15 12:00 AM|10/21/15 12:00 AM|      0 00 00 * * ? *|

C:\Program Files\Symantec\DataInsight\bin>configcli execute_job DlpSensitiveFilesJob
Job started

The error should be removed and the download successful. Continue testing with a larger value until there is no longer an error and the download is successful if the first attempt still results in a dataset larger than the allocated memory.